1 | <html> |
---|
2 | <head> |
---|
3 | <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
---|
4 | <title> |
---|
5 | Overview and feature list for the SimpleTest PHP unit tester and web tester |
---|
6 | </title> |
---|
7 | <link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> |
---|
8 | </head> |
---|
9 | <body> |
---|
10 | <div class="menu_back"><div class="menu"> |
---|
11 | <a href="index.html">SimpleTest</a> |
---|
12 | | |
---|
13 | <span class="chosen">Overview</span> |
---|
14 | | |
---|
15 | <a href="unit_test_documentation.html">Unit tester</a> |
---|
16 | | |
---|
17 | <a href="group_test_documentation.html">Group tests</a> |
---|
18 | | |
---|
19 | <a href="mock_objects_documentation.html">Mock objects</a> |
---|
20 | | |
---|
21 | <a href="partial_mocks_documentation.html">Partial mocks</a> |
---|
22 | | |
---|
23 | <a href="reporter_documentation.html">Reporting</a> |
---|
24 | | |
---|
25 | <a href="expectation_documentation.html">Expectations</a> |
---|
26 | | |
---|
27 | <a href="web_tester_documentation.html">Web tester</a> |
---|
28 | | |
---|
29 | <a href="form_testing_documentation.html">Testing forms</a> |
---|
30 | | |
---|
31 | <a href="authentication_documentation.html">Authentication</a> |
---|
32 | | |
---|
33 | <a href="browser_documentation.html">Scriptable browser</a> |
---|
34 | </div></div> |
---|
35 | <h1>Overview of SimpleTest</h1> |
---|
36 | This page... |
---|
37 | <ul> |
---|
38 | <li> |
---|
39 | <a href="#summary">Quick summary</a> |
---|
40 | of the SimpleTest tool for PHP. |
---|
41 | </li> |
---|
42 | <li> |
---|
43 | <a href="#features">List of features</a>, |
---|
44 | both current ones and those planned. |
---|
45 | </li> |
---|
46 | <li> |
---|
47 | There are plenty of <a href="#resources">unit testing resources</a> |
---|
48 | on the web. |
---|
49 | </li> |
---|
50 | </ul> |
---|
51 | <div class="content"> |
---|
52 | <p><a class="target" name="summary"><h2>What is SimpleTest?</h2></a></p> |
---|
53 | <p> |
---|
54 | The heart of SimpleTest is a testing framework built around |
---|
55 | test case classes. |
---|
56 | These are written as extensions of base test case classes, |
---|
57 | each extended with methods that actually contain test code. |
---|
58 | Top level test scripts then invoke the <span class="new_code">run()</span> |
---|
59 | methods on every one of these test cases in order. |
---|
60 | Each test method is written to invoke various assertions that |
---|
61 | the developer expects to be true such as |
---|
62 | <span class="new_code">assertEqual()</span>. |
---|
63 | If the expectation is correct, then a successful result is dispatched to the |
---|
64 | observing test reporter, but any failure triggers an alert |
---|
65 | and a description of the mismatch. |
---|
66 | </p> |
---|
67 | <p> |
---|
68 | A <a href="unit_test_documentation.html">test case</a> looks like this... |
---|
69 | <pre> |
---|
70 | <?php |
---|
71 | require_once('simpletest/autorun.php'); |
---|
72 | |
---|
73 | class <strong>MyTestCase</strong> extends UnitTestCase { |
---|
74 | <strong> |
---|
75 | function testCreatedLogFile() { |
---|
76 | $log = &new Log('my.log'); |
---|
77 | $log->message('Hello'); |
---|
78 | $this->assertTrue(file_exists('my.log')); |
---|
79 | }</strong> |
---|
80 | } |
---|
81 | ?> |
---|
82 | </pre> |
---|
83 | </p> |
---|
84 | <p> |
---|
85 | These tools are designed for the developer. |
---|
86 | Tests are written in the PHP language itself more or less |
---|
87 | as the application itself is built. |
---|
88 | The advantage of using PHP itself as the testing language is that |
---|
89 | there are no new languages to learn, testing can start straight away, |
---|
90 | and the developer can test any part of the code. |
---|
91 | Basically, all parts that can be accessed by the application code can also be |
---|
92 | accessed by the test code, if they are in the same programming language. |
---|
93 | </p> |
---|
94 | <p> |
---|
95 | The simplest type of test case is the |
---|
96 | <a href="unit_tester_documentation.html">UnitTestCase</a>. |
---|
97 | This class of test case includes standard tests for equality, |
---|
98 | references and pattern matching. |
---|
99 | All these test the typical expectations of what you would |
---|
100 | expect the result of a function or method to be. |
---|
101 | This is by far the most common type of test in the daily |
---|
102 | routine of development, making up about 95% of test cases. |
---|
103 | </p> |
---|
104 | <p> |
---|
105 | The top level task of a web application though is not to |
---|
106 | produce correct output from its methods and objects, but |
---|
107 | to generate web pages. |
---|
108 | The <a href="web_tester_documentation.html">WebTestCase</a> class tests web |
---|
109 | pages. |
---|
110 | It simulates a web browser requesting a page, complete with |
---|
111 | cookies, proxies, secure connections, authentication, forms, frames and most |
---|
112 | navigation elements. |
---|
113 | With this type of test case, the developer can assert that |
---|
114 | information is present in the page and that forms and |
---|
115 | sessions are handled correctly. |
---|
116 | </p> |
---|
117 | <p> |
---|
118 | A <a href="web_tester_documentation.html">WebTestCase</a> looks like this... |
---|
119 | <pre> |
---|
120 | <?php |
---|
121 | require_once('simpletest/autorun.php'); |
---|
122 | require_once('simpletest/web_tester.php'); |
---|
123 | |
---|
124 | class <strong>MySiteTest</strong> extends WebTestCase { |
---|
125 | <strong> |
---|
126 | function testHomePage() { |
---|
127 | $this->get('http://www.my-site.com/index.php'); |
---|
128 | $this->assertTitle('My Home Page'); |
---|
129 | $this->clickLink('Contact'); |
---|
130 | $this->assertTitle('Contact me'); |
---|
131 | $this->assertPattern('/Email me at/'); |
---|
132 | }</strong> |
---|
133 | } |
---|
134 | ?> |
---|
135 | </pre> |
---|
136 | </p> |
---|
137 | |
---|
138 | <p><a class="target" name="features"><h2>Feature list</h2></a></p> |
---|
139 | <p> |
---|
140 | The following is a very rough outline of past and future features |
---|
141 | and their expected point of release. |
---|
142 | I am afraid it is liable to change without warning, as meeting the |
---|
143 | milestones rather depends on time available. |
---|
144 | Green stuff has been coded, but not necessarily released yet. |
---|
145 | If you have a pressing need for a green but unreleased feature |
---|
146 | then you should check-out the code from Sourceforge SVN directly. |
---|
147 | <table> |
---|
148 | <thead> |
---|
149 | <tr> |
---|
150 | <th>Feature</th> |
---|
151 | <th>Description</th> |
---|
152 | <th>Release</th> |
---|
153 | </tr> |
---|
154 | </thead> |
---|
155 | <tbody> |
---|
156 | <tr> |
---|
157 | <td>Unit test case</td> |
---|
158 | <td>Core test case class and assertions</td> |
---|
159 | <td style="color: green;">1.0</td> |
---|
160 | </tr> |
---|
161 | <tr> |
---|
162 | <td>Html display</td> |
---|
163 | <td>Simplest possible display</td> |
---|
164 | <td style="color: green;">1.0</td> |
---|
165 | </tr> |
---|
166 | <tr> |
---|
167 | <td>Autoloading of test cases</td> |
---|
168 | <td> |
---|
169 | Reading a file with test cases and loading them into a |
---|
170 | group test automatically |
---|
171 | </td> |
---|
172 | <td style="color: green;">1.0</td> |
---|
173 | </tr> |
---|
174 | <tr> |
---|
175 | <td>Mock objects</td> |
---|
176 | <td> |
---|
177 | Objects capable of simulating other objects removing |
---|
178 | test dependencies |
---|
179 | </td> |
---|
180 | <td style="color: green;">1.0</td> |
---|
181 | </tr> |
---|
182 | <tr> |
---|
183 | <td>Web test case</td> |
---|
184 | <td>Allows link following and title tag matching</td> |
---|
185 | <td style="color: green;">1.0</td> |
---|
186 | </tr> |
---|
187 | <tr> |
---|
188 | <td>Partial mocks</td> |
---|
189 | <td> |
---|
190 | Mocking parts of a class for testing less than a class |
---|
191 | or for complex simulations |
---|
192 | </td> |
---|
193 | <td style="color: green;">1.0</td> |
---|
194 | </tr> |
---|
195 | <tr> |
---|
196 | <td>Web cookie handling</td> |
---|
197 | <td>Correct handling of cookies when fetching pages</td> |
---|
198 | <td style="color: green;">1.0</td> |
---|
199 | </tr> |
---|
200 | <tr> |
---|
201 | <td>Following redirects</td> |
---|
202 | <td>Page fetching automatically follows 300 redirects</td> |
---|
203 | <td style="color: green;">1.0</td> |
---|
204 | </tr> |
---|
205 | <tr> |
---|
206 | <td>Form parsing</td> |
---|
207 | <td>Ability to submit simple forms and read default form values</td> |
---|
208 | <td style="color: green;">1.0</td> |
---|
209 | </tr> |
---|
210 | <tr> |
---|
211 | <td>Command line interface</td> |
---|
212 | <td>Test display without the need of a web browser</td> |
---|
213 | <td style="color: green;">1.0</td> |
---|
214 | </tr> |
---|
215 | <tr> |
---|
216 | <td>Exposure of expectation classes</td> |
---|
217 | <td>Can create precise tests with mocks as well as test cases</td> |
---|
218 | <td style="color: green;">1.0</td> |
---|
219 | </tr> |
---|
220 | <tr> |
---|
221 | <td>XML output and parsing</td> |
---|
222 | <td> |
---|
223 | Allows multi host testing and the integration of acceptance |
---|
224 | testing extensions |
---|
225 | </td> |
---|
226 | <td style="color: green;">1.0</td> |
---|
227 | </tr> |
---|
228 | <tr> |
---|
229 | <td>Browser component</td> |
---|
230 | <td> |
---|
231 | Exposure of lower level web browser interface for more |
---|
232 | detailed test cases |
---|
233 | </td> |
---|
234 | <td style="color: green;">1.0</td> |
---|
235 | </tr> |
---|
236 | <tr> |
---|
237 | <td>HTTP authentication</td> |
---|
238 | <td> |
---|
239 | Fetching protected web pages with basic authentication |
---|
240 | only |
---|
241 | </td> |
---|
242 | <td style="color: green;">1.0</td> |
---|
243 | </tr> |
---|
244 | <tr> |
---|
245 | <td>SSL support</td> |
---|
246 | <td>Can connect to https: pages</td> |
---|
247 | <td style="color: green;">1.0</td> |
---|
248 | </tr> |
---|
249 | <tr> |
---|
250 | <td>Proxy support</td> |
---|
251 | <td>Can connect via. common proxies</td> |
---|
252 | <td style="color: green;">1.0</td> |
---|
253 | </tr> |
---|
254 | <tr> |
---|
255 | <td>Frames support</td> |
---|
256 | <td>Handling of frames in web test cases</td> |
---|
257 | <td style="color: green;">1.0</td> |
---|
258 | </tr> |
---|
259 | <tr> |
---|
260 | <td>File upload testing</td> |
---|
261 | <td>Can simulate the input type file tag</td> |
---|
262 | <td style="color: green;">1.0.1</td> |
---|
263 | </tr> |
---|
264 | <tr> |
---|
265 | <td>Mocking interfaces</td> |
---|
266 | <td> |
---|
267 | Can generate mock objects to interfaces as well as classes |
---|
268 | and class interfaces are carried for type hints |
---|
269 | </td> |
---|
270 | <td style="color: green;">1.0.1</td> |
---|
271 | </tr> |
---|
272 | <tr> |
---|
273 | <td>Testing exceptions</td> |
---|
274 | <td>Similar to testing PHP errors</td> |
---|
275 | <td style="color: green;">1.0.1</td> |
---|
276 | </tr> |
---|
277 | <tr> |
---|
278 | <td>HTML label support</td> |
---|
279 | <td>Can access all controls using the visual label</td> |
---|
280 | <td style="color: green;">1.0.1</td> |
---|
281 | </tr> |
---|
282 | <tr> |
---|
283 | <td>Base tag support</td> |
---|
284 | <td>Respects page base tag when clicking</td> |
---|
285 | <td style="color: green;">1.0.1</td> |
---|
286 | </tr> |
---|
287 | <tr> |
---|
288 | <td>PHP 5 E_STRICT compliant</td> |
---|
289 | <td>PHP 5 only version that works with the E_STRICT error level</td> |
---|
290 | <td style="color: red;">1.1</td> |
---|
291 | </tr> |
---|
292 | <tr> |
---|
293 | <td>BDD style fixtures</td> |
---|
294 | <td>Can import fixtures using a mixin like given() method</td> |
---|
295 | <td style="color: red;">1.5</td> |
---|
296 | </tr> |
---|
297 | <tr> |
---|
298 | <td>Reporting machinery enhancements</td> |
---|
299 | <td>Improved message passing for better cooperation with IDEs</td> |
---|
300 | <td style="color: red;">1.5</td> |
---|
301 | </tr> |
---|
302 | <tr> |
---|
303 | <td>Fluent mock interface</td> |
---|
304 | <td>More flexible and concise mock objects</td> |
---|
305 | <td style="color: red;">1.6</td> |
---|
306 | </tr> |
---|
307 | <tr> |
---|
308 | <td>Localisation</td> |
---|
309 | <td>Messages abstracted and code generated</td> |
---|
310 | <td style="color: red;">1.6</td> |
---|
311 | </tr> |
---|
312 | <tr> |
---|
313 | <td>CSS selectors</td> |
---|
314 | <td>HTML content can be examined using CSS selectors</td> |
---|
315 | <td style="color: red;">1.7</td> |
---|
316 | </tr> |
---|
317 | <tr> |
---|
318 | <td>HTML table assertions</td> |
---|
319 | <td>Can match HTML or table elements to expectations</td> |
---|
320 | <td style="color: red;">1.7</td> |
---|
321 | </tr> |
---|
322 | <tr> |
---|
323 | <td>Unified acceptance testing model</td> |
---|
324 | <td>Content searchable through selectors combined with expectations</td> |
---|
325 | <td style="color: red;">1.7</td> |
---|
326 | </tr> |
---|
327 | <tr> |
---|
328 | <td>DatabaseTestCase</td> |
---|
329 | <td>SQL selectors and DB drivers</td> |
---|
330 | <td style="color: red;">1.7</td> |
---|
331 | </tr> |
---|
332 | <tr> |
---|
333 | <td>IFrame support</td> |
---|
334 | <td>Reads IFrame content that can be refreshed</td> |
---|
335 | <td style="color: red;">1.8</td> |
---|
336 | </tr> |
---|
337 | <tr> |
---|
338 | <td>Alternate HTML parsers</td> |
---|
339 | <td>Can detect compiled parsers for performance improvements</td> |
---|
340 | <td style="color: red;">1.8</td> |
---|
341 | </tr> |
---|
342 | <tr> |
---|
343 | <td>Integrated Selenium support</td> |
---|
344 | <td>Easy to use built in Selenium driver and tutorial</td> |
---|
345 | <td style="color: red;">1.9</td> |
---|
346 | </tr> |
---|
347 | <tr> |
---|
348 | <td>Code coverage</td> |
---|
349 | <td>Reports using the bundled tool when using XDebug</td> |
---|
350 | <td style="color: red;">1.9</td> |
---|
351 | </tr> |
---|
352 | <tr> |
---|
353 | <td>Deprecation of old methods</td> |
---|
354 | <td>Simpler interface for SimpleTest2</td> |
---|
355 | <td style="color: red;">2.0</td> |
---|
356 | </tr> |
---|
357 | <tr> |
---|
358 | <td>Javascript suport</td> |
---|
359 | <td>Use of PECL module to add Javascript to the native browser</td> |
---|
360 | <td style="color: red;">3.0</td> |
---|
361 | </tr> |
---|
362 | </tbody> |
---|
363 | </table> |
---|
364 | PHP5 migraton will start straight after the version 1.0.1 series, |
---|
365 | whereupon only PHP 5.1+ will be supported. |
---|
366 | SimpleTest is currently compatible with PHP 5, but will not |
---|
367 | make use of all of the new features until version 1.1. |
---|
368 | </p> |
---|
369 | |
---|
370 | <p><a class="target" name="resources"><h2>Web resources for testing</h2></a></p> |
---|
371 | <p> |
---|
372 | Process is at least as important as tools. |
---|
373 | The type of process that makes the heaviest use of a developer's |
---|
374 | testing tool is of course |
---|
375 | <a href="http://www.extremeprogramming.org/">Extreme Programming</a>. |
---|
376 | This is one of the |
---|
377 | <a href="http://www.agilealliance.com/articles/index">Agile Methodologies</a> |
---|
378 | which combine various practices to "flatten the cost curve" of software development. |
---|
379 | More extreme still is <a href="http://www.testdriven.com/modules/news/">Test Driven Development</a>, |
---|
380 | where you very strictly adhere to the rule of no coding until you have a test. |
---|
381 | If you're more of a planner, or believe that experience trumps evolution, |
---|
382 | you may prefer the |
---|
383 | <a href="http://www.therationaledge.com/content/dec_01/f_spiritOfTheRUP_pk.html">RUP</a> approach. |
---|
384 | I haven't tried it, but even I can see that you will need test tools (see figure 9). |
---|
385 | </p> |
---|
386 | <p> |
---|
387 | Most unit testers clone <a href="http://www.junit.org/">JUnit</a> to some degree, |
---|
388 | as far as the interface at least. There is a wealth of information on the |
---|
389 | JUnit site including the |
---|
390 | <a href="http://junit.sourceforge.net/doc/faq/faq.htm">FAQ</a> |
---|
391 | which contains plenty of general advice on testing. |
---|
392 | Once you get bitten by the bug you will certainly appreciate the phrase |
---|
393 | <a href="http://junit.sourceforge.net/doc/testinfected/testing.htm">test infected</a> |
---|
394 | coined by Eric Gamma. |
---|
395 | If you are still reviewing which unit tester to use you can find pretty complete |
---|
396 | lists from |
---|
397 | <a href="http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks">Wikipedia</a>, |
---|
398 | <a href="http://www.testingfaqs.org/t-unit.html">Software testing FAQ</a>, |
---|
399 | and <a href="http://www.opensourcetesting.org/functional.php">Open source testing</a>. |
---|
400 | </p> |
---|
401 | <p> |
---|
402 | There is still very little material on using mock objects, which is a shame |
---|
403 | as unit testing without them is a lot more work. |
---|
404 | The <a href="http://www.sidewize.com/company/mockobjects.pdf">original mock objects paper</a> |
---|
405 | is very Java focused, but still worth a read. |
---|
406 | The most authoritive sources are probably |
---|
407 | <a href="http://mockobjects.com">the original mock objects site</a> and |
---|
408 | <a href="http://jmock.org/">JMock</a>. |
---|
409 | Java centric, but tucked away in PDFs they contain some deep knowledge on using mocks from the |
---|
410 | extended experience of the concept inventors. |
---|
411 | As a new technology there are plenty of discussions and debate on how to use mocks, |
---|
412 | often on Wikis such as |
---|
413 | <a href="http://xpdeveloper.com/cgi-bin/oldwiki.cgi?MockObjects">Extreme Tuesday</a> |
---|
414 | or <a href="http://www.mockobjects.com/MocksObjectsPaper.html">www.mockobjects.com</a> |
---|
415 | or <a href="http://c2.com/cgi/wiki?MockObject">the original C2 Wiki</a>. |
---|
416 | Injecting mocks into a class is the main area of debate for which this |
---|
417 | <a href="http://www-106.ibm.com/developerworks/java/library/j-mocktest.html">paper on IBM</a> |
---|
418 | makes a good starting point. |
---|
419 | </p> |
---|
420 | <p> |
---|
421 | There are plenty of web testing tools, but the scriptable ones |
---|
422 | are mostly are written in Java and |
---|
423 | tutorials and advice are rather thin on the ground. |
---|
424 | The only hope is to look at the documentation for |
---|
425 | <a href="http://httpunit.sourceforge.net/">HTTPUnit</a>, |
---|
426 | <a href="http://htmlunit.sourceforge.net/">HTMLUnit</a> |
---|
427 | or <a href="http://jwebunit.sourceforge.net/">JWebUnit</a> and hope for clues. |
---|
428 | There are some XML driven test frameworks, but again most |
---|
429 | require Java to run. |
---|
430 | </p> |
---|
431 | <p> |
---|
432 | Most significant is a new generation of tools that run directly in the web browser |
---|
433 | are now available. |
---|
434 | These include |
---|
435 | <a href="http://www.openqa.org/selenium/">Selenium</a> and |
---|
436 | <a href="http://wtr.rubyforge.org/">Watir</a>. |
---|
437 | They are non-trivial to set up and slow to run, but can essentially test anything. |
---|
438 | As SimpleTest does not support JavaScript you would probably |
---|
439 | have to look at these tools anyway if you have highly dynamic |
---|
440 | pages. |
---|
441 | </p> |
---|
442 | |
---|
443 | </div> |
---|
444 | References and related information... |
---|
445 | <ul> |
---|
446 | <li> |
---|
447 | <a href="unit_test_documentation.html">Documentation for SimpleTest</a>. |
---|
448 | </li> |
---|
449 | <li> |
---|
450 | <a href="http://www.lastcraft.com/first_test_tutorial.php">How to write PHP test cases</a> |
---|
451 | is a fairly advanced tutorial. |
---|
452 | </li> |
---|
453 | <li> |
---|
454 | <a href="http://simpletest.org/api/">SimpleTest API</a> from phpdoc. |
---|
455 | </li> |
---|
456 | </ul> |
---|
457 | <div class="menu_back"><div class="menu"> |
---|
458 | <a href="index.html">SimpleTest</a> |
---|
459 | | |
---|
460 | <span class="chosen">Overview</span> |
---|
461 | | |
---|
462 | <a href="unit_test_documentation.html">Unit tester</a> |
---|
463 | | |
---|
464 | <a href="group_test_documentation.html">Group tests</a> |
---|
465 | | |
---|
466 | <a href="mock_objects_documentation.html">Mock objects</a> |
---|
467 | | |
---|
468 | <a href="partial_mocks_documentation.html">Partial mocks</a> |
---|
469 | | |
---|
470 | <a href="reporter_documentation.html">Reporting</a> |
---|
471 | | |
---|
472 | <a href="expectation_documentation.html">Expectations</a> |
---|
473 | | |
---|
474 | <a href="web_tester_documentation.html">Web tester</a> |
---|
475 | | |
---|
476 | <a href="form_testing_documentation.html">Testing forms</a> |
---|
477 | | |
---|
478 | <a href="authentication_documentation.html">Authentication</a> |
---|
479 | | |
---|
480 | <a href="browser_documentation.html">Scriptable browser</a> |
---|
481 | </div></div> |
---|
482 | <div class="copyright"> |
---|
483 | Copyright<br>Marcus Baker 2006 |
---|
484 | </div> |
---|
485 | </body> |
---|
486 | </html> |
---|