Context Navigation

source: trunk/server/www/vendors/simpletest/docs/en/overview.html @ 6

Last change on this file since 6 was 6, checked in by sander, 12 years ago
Added SimpleTest? test framework
File size: 22.5 KB

Line
1	<html>
2	<head>
3	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
4	<title>
5	Overview and feature list for the SimpleTest PHP unit tester and web tester
6	</title>
7	<link rel="stylesheet" type="text/css" href="docs.css" title="Styles">
8	</head>
9	<body>
10	<div class="menu_back"><div class="menu">
11	<a href="index.html">SimpleTest</a>
12	\|
13	<span class="chosen">Overview</span>
14	\|
15	<a href="unit_test_documentation.html">Unit tester</a>
16	\|
17	<a href="group_test_documentation.html">Group tests</a>
18	\|
19	<a href="mock_objects_documentation.html">Mock objects</a>
20	\|
21	<a href="partial_mocks_documentation.html">Partial mocks</a>
22	\|
23	<a href="reporter_documentation.html">Reporting</a>
24	\|
25	<a href="expectation_documentation.html">Expectations</a>
26	\|
27	<a href="web_tester_documentation.html">Web tester</a>
28	\|
29	<a href="form_testing_documentation.html">Testing forms</a>
30	\|
31	<a href="authentication_documentation.html">Authentication</a>
32	\|
33	<a href="browser_documentation.html">Scriptable browser</a>
34	</div></div>
35	<h1>Overview of SimpleTest</h1>
36	This page...
37	<ul>
38	<li>
39	<a href="#summary">Quick summary</a>
40	of the SimpleTest tool for PHP.
41	</li>
42	<li>
43	<a href="#features">List of features</a>,
44	both current ones and those planned.
45	</li>
46	<li>
47	There are plenty of <a href="#resources">unit testing resources</a>
48	on the web.
49	</li>
50	</ul>
51	<div class="content">
52	<p><a class="target" name="summary"><h2>What is SimpleTest?</h2></a></p>
53	<p>
54	The heart of SimpleTest is a testing framework built around
55	test case classes.
56	These are written as extensions of base test case classes,
57	each extended with methods that actually contain test code.
58	Top level test scripts then invoke the <span class="new_code">run()</span>
59	methods on every one of these test cases in order.
60	Each test method is written to invoke various assertions that
61	the developer expects to be true such as
62	<span class="new_code">assertEqual()</span>.
63	If the expectation is correct, then a successful result is dispatched to the
64	observing test reporter, but any failure triggers an alert
65	and a description of the mismatch.
66	</p>
67	<p>
68	A <a href="unit_test_documentation.html">test case</a> looks like this...
69	<pre>
70	<?php
71	require_once('simpletest/autorun.php');
72
73	class <strong>MyTestCase</strong> extends UnitTestCase {
74	<strong>
75	function testCreatedLogFile() {
76	$log = &new Log('my.log');
77	$log->message('Hello');
78	$this->assertTrue(file_exists('my.log'));
79	}</strong>
80	}
81	?>
82	</pre>
83	</p>
84	<p>
85	These tools are designed for the developer.
86	Tests are written in the PHP language itself more or less
87	as the application itself is built.
88	The advantage of using PHP itself as the testing language is that
89	there are no new languages to learn, testing can start straight away,
90	and the developer can test any part of the code.
91	Basically, all parts that can be accessed by the application code can also be
92	accessed by the test code, if they are in the same programming language.
93	</p>
94	<p>
95	The simplest type of test case is the
96	<a href="unit_tester_documentation.html">UnitTestCase</a>.
97	This class of test case includes standard tests for equality,
98	references and pattern matching.
99	All these test the typical expectations of what you would
100	expect the result of a function or method to be.
101	This is by far the most common type of test in the daily
102	routine of development, making up about 95% of test cases.
103	</p>
104	<p>
105	The top level task of a web application though is not to
106	produce correct output from its methods and objects, but
107	to generate web pages.
108	The <a href="web_tester_documentation.html">WebTestCase</a> class tests web
109	pages.
110	It simulates a web browser requesting a page, complete with
111	cookies, proxies, secure connections, authentication, forms, frames and most
112	navigation elements.
113	With this type of test case, the developer can assert that
114	information is present in the page and that forms and
115	sessions are handled correctly.
116	</p>
117	<p>
118	A <a href="web_tester_documentation.html">WebTestCase</a> looks like this...
119	<pre>
120	<?php
121	require_once('simpletest/autorun.php');
122	require_once('simpletest/web_tester.php');
123
124	class <strong>MySiteTest</strong> extends WebTestCase {
125	<strong>
126	function testHomePage() {
127	$this->get('http://www.my-site.com/index.php');
128	$this->assertTitle('My Home Page');
129	$this->clickLink('Contact');
130	$this->assertTitle('Contact me');
131	$this->assertPattern('/Email me at/');
132	}</strong>
133	}
134	?>
135	</pre>
136	</p>
137
138	<p><a class="target" name="features"><h2>Feature list</h2></a></p>
139	<p>
140	The following is a very rough outline of past and future features
141	and their expected point of release.
142	I am afraid it is liable to change without warning, as meeting the
143	milestones rather depends on time available.
144	Green stuff has been coded, but not necessarily released yet.
145	If you have a pressing need for a green but unreleased feature
146	then you should check-out the code from Sourceforge SVN directly.
147	<table>
148	<thead>
149	<tr>
150	<th>Feature</th>
151	<th>Description</th>
152	<th>Release</th>
153	</tr>
154	</thead>
155	<tbody>
156	<tr>
157	<td>Unit test case</td>
158	<td>Core test case class and assertions</td>
159	<td style="color: green;">1.0</td>
160	</tr>
161	<tr>
162	<td>Html display</td>
163	<td>Simplest possible display</td>
164	<td style="color: green;">1.0</td>
165	</tr>
166	<tr>
167	<td>Autoloading of test cases</td>
168	<td>
169	Reading a file with test cases and loading them into a
170	group test automatically
171	</td>
172	<td style="color: green;">1.0</td>
173	</tr>
174	<tr>
175	<td>Mock objects</td>
176	<td>
177	Objects capable of simulating other objects removing
178	test dependencies
179	</td>
180	<td style="color: green;">1.0</td>
181	</tr>
182	<tr>
183	<td>Web test case</td>
184	<td>Allows link following and title tag matching</td>
185	<td style="color: green;">1.0</td>
186	</tr>
187	<tr>
188	<td>Partial mocks</td>
189	<td>
190	Mocking parts of a class for testing less than a class
191	or for complex simulations
192	</td>
193	<td style="color: green;">1.0</td>
194	</tr>
195	<tr>
196	<td>Web cookie handling</td>
197	<td>Correct handling of cookies when fetching pages</td>
198	<td style="color: green;">1.0</td>
199	</tr>
200	<tr>
201	<td>Following redirects</td>
202	<td>Page fetching automatically follows 300 redirects</td>
203	<td style="color: green;">1.0</td>
204	</tr>
205	<tr>
206	<td>Form parsing</td>
207	<td>Ability to submit simple forms and read default form values</td>
208	<td style="color: green;">1.0</td>
209	</tr>
210	<tr>
211	<td>Command line interface</td>
212	<td>Test display without the need of a web browser</td>
213	<td style="color: green;">1.0</td>
214	</tr>
215	<tr>
216	<td>Exposure of expectation classes</td>
217	<td>Can create precise tests with mocks as well as test cases</td>
218	<td style="color: green;">1.0</td>
219	</tr>
220	<tr>
221	<td>XML output and parsing</td>
222	<td>
223	Allows multi host testing and the integration of acceptance
224	testing extensions
225	</td>
226	<td style="color: green;">1.0</td>
227	</tr>
228	<tr>
229	<td>Browser component</td>
230	<td>
231	Exposure of lower level web browser interface for more
232	detailed test cases
233	</td>
234	<td style="color: green;">1.0</td>
235	</tr>
236	<tr>
237	<td>HTTP authentication</td>
238	<td>
239	Fetching protected web pages with basic authentication
240	only
241	</td>
242	<td style="color: green;">1.0</td>
243	</tr>
244	<tr>
245	<td>SSL support</td>
246	<td>Can connect to https: pages</td>
247	<td style="color: green;">1.0</td>
248	</tr>
249	<tr>
250	<td>Proxy support</td>
251	<td>Can connect via. common proxies</td>
252	<td style="color: green;">1.0</td>
253	</tr>
254	<tr>
255	<td>Frames support</td>
256	<td>Handling of frames in web test cases</td>
257	<td style="color: green;">1.0</td>
258	</tr>
259	<tr>
260	<td>File upload testing</td>
261	<td>Can simulate the input type file tag</td>
262	<td style="color: green;">1.0.1</td>
263	</tr>
264	<tr>
265	<td>Mocking interfaces</td>
266	<td>
267	Can generate mock objects to interfaces as well as classes
268	and class interfaces are carried for type hints
269	</td>
270	<td style="color: green;">1.0.1</td>
271	</tr>
272	<tr>
273	<td>Testing exceptions</td>
274	<td>Similar to testing PHP errors</td>
275	<td style="color: green;">1.0.1</td>
276	</tr>
277	<tr>
278	<td>HTML label support</td>
279	<td>Can access all controls using the visual label</td>
280	<td style="color: green;">1.0.1</td>
281	</tr>
282	<tr>
283	<td>Base tag support</td>
284	<td>Respects page base tag when clicking</td>
285	<td style="color: green;">1.0.1</td>
286	</tr>
287	<tr>
288	<td>PHP 5 E_STRICT compliant</td>
289	<td>PHP 5 only version that works with the E_STRICT error level</td>
290	<td style="color: red;">1.1</td>
291	</tr>
292	<tr>
293	<td>BDD style fixtures</td>
294	<td>Can import fixtures using a mixin like given() method</td>
295	<td style="color: red;">1.5</td>
296	</tr>
297	<tr>
298	<td>Reporting machinery enhancements</td>
299	<td>Improved message passing for better cooperation with IDEs</td>
300	<td style="color: red;">1.5</td>
301	</tr>
302	<tr>
303	<td>Fluent mock interface</td>
304	<td>More flexible and concise mock objects</td>
305	<td style="color: red;">1.6</td>
306	</tr>
307	<tr>
308	<td>Localisation</td>
309	<td>Messages abstracted and code generated</td>
310	<td style="color: red;">1.6</td>
311	</tr>
312	<tr>
313	<td>CSS selectors</td>
314	<td>HTML content can be examined using CSS selectors</td>
315	<td style="color: red;">1.7</td>
316	</tr>
317	<tr>
318	<td>HTML table assertions</td>
319	<td>Can match HTML or table elements to expectations</td>
320	<td style="color: red;">1.7</td>
321	</tr>
322	<tr>
323	<td>Unified acceptance testing model</td>
324	<td>Content searchable through selectors combined with expectations</td>
325	<td style="color: red;">1.7</td>
326	</tr>
327	<tr>
328	<td>DatabaseTestCase</td>
329	<td>SQL selectors and DB drivers</td>
330	<td style="color: red;">1.7</td>
331	</tr>
332	<tr>
333	<td>IFrame support</td>
334	<td>Reads IFrame content that can be refreshed</td>
335	<td style="color: red;">1.8</td>
336	</tr>
337	<tr>
338	<td>Alternate HTML parsers</td>
339	<td>Can detect compiled parsers for performance improvements</td>
340	<td style="color: red;">1.8</td>
341	</tr>
342	<tr>
343	<td>Integrated Selenium support</td>
344	<td>Easy to use built in Selenium driver and tutorial</td>
345	<td style="color: red;">1.9</td>
346	</tr>
347	<tr>
348	<td>Code coverage</td>
349	<td>Reports using the bundled tool when using XDebug</td>
350	<td style="color: red;">1.9</td>
351	</tr>
352	<tr>
353	<td>Deprecation of old methods</td>
354	<td>Simpler interface for SimpleTest2</td>
355	<td style="color: red;">2.0</td>
356	</tr>
357	<tr>
358	<td>Javascript suport</td>
359	<td>Use of PECL module to add Javascript to the native browser</td>
360	<td style="color: red;">3.0</td>
361	</tr>
362	</tbody>
363	</table>
364	PHP5 migraton will start straight after the version 1.0.1 series,
365	whereupon only PHP 5.1+ will be supported.
366	SimpleTest is currently compatible with PHP 5, but will not
367	make use of all of the new features until version 1.1.
368	</p>
369
370	<p><a class="target" name="resources"><h2>Web resources for testing</h2></a></p>
371	<p>
372	Process is at least as important as tools.
373	The type of process that makes the heaviest use of a developer's
374	testing tool is of course
375	<a href="http://www.extremeprogramming.org/">Extreme Programming</a>.
376	This is one of the
377	<a href="http://www.agilealliance.com/articles/index">Agile Methodologies</a>
378	which combine various practices to "flatten the cost curve" of software development.
379	More extreme still is <a href="http://www.testdriven.com/modules/news/">Test Driven Development</a>,
380	where you very strictly adhere to the rule of no coding until you have a test.
381	If you're more of a planner, or believe that experience trumps evolution,
382	you may prefer the
383	<a href="http://www.therationaledge.com/content/dec_01/f_spiritOfTheRUP_pk.html">RUP</a> approach.
384	I haven't tried it, but even I can see that you will need test tools (see figure 9).
385	</p>
386	<p>
387	Most unit testers clone <a href="http://www.junit.org/">JUnit</a> to some degree,
388	as far as the interface at least. There is a wealth of information on the
389	JUnit site including the
390	<a href="http://junit.sourceforge.net/doc/faq/faq.htm">FAQ</a>
391	which contains plenty of general advice on testing.
392	Once you get bitten by the bug you will certainly appreciate the phrase
393	<a href="http://junit.sourceforge.net/doc/testinfected/testing.htm">test infected</a>
394	coined by Eric Gamma.
395	If you are still reviewing which unit tester to use you can find pretty complete
396	lists from
397	<a href="http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks">Wikipedia</a>,
398	<a href="http://www.testingfaqs.org/t-unit.html">Software testing FAQ</a>,
399	and <a href="http://www.opensourcetesting.org/functional.php">Open source testing</a>.
400	</p>
401	<p>
402	There is still very little material on using mock objects, which is a shame
403	as unit testing without them is a lot more work.
404	The <a href="http://www.sidewize.com/company/mockobjects.pdf">original mock objects paper</a>
405	is very Java focused, but still worth a read.
406	The most authoritive sources are probably
407	<a href="http://mockobjects.com">the original mock objects site</a> and
408	<a href="http://jmock.org/">JMock</a>.
409	Java centric, but tucked away in PDFs they contain some deep knowledge on using mocks from the
410	extended experience of the concept inventors.
411	As a new technology there are plenty of discussions and debate on how to use mocks,
412	often on Wikis such as
413	<a href="http://xpdeveloper.com/cgi-bin/oldwiki.cgi?MockObjects">Extreme Tuesday</a>
414	or <a href="http://www.mockobjects.com/MocksObjectsPaper.html">www.mockobjects.com</a>
415	or <a href="http://c2.com/cgi/wiki?MockObject">the original C2 Wiki</a>.
416	Injecting mocks into a class is the main area of debate for which this
417	<a href="http://www-106.ibm.com/developerworks/java/library/j-mocktest.html">paper on IBM</a>
418	makes a good starting point.
419	</p>
420	<p>
421	There are plenty of web testing tools, but the scriptable ones
422	are mostly are written in Java and
423	tutorials and advice are rather thin on the ground.
424	The only hope is to look at the documentation for
425	<a href="http://httpunit.sourceforge.net/">HTTPUnit</a>,
426	<a href="http://htmlunit.sourceforge.net/">HTMLUnit</a>
427	or <a href="http://jwebunit.sourceforge.net/">JWebUnit</a> and hope for clues.
428	There are some XML driven test frameworks, but again most
429	require Java to run.
430	</p>
431	<p>
432	Most significant is a new generation of tools that run directly in the web browser
433	are now available.
434	These include
435	<a href="http://www.openqa.org/selenium/">Selenium</a> and
436	<a href="http://wtr.rubyforge.org/">Watir</a>.
437	They are non-trivial to set up and slow to run, but can essentially test anything.
438	As SimpleTest does not support JavaScript you would probably
439	have to look at these tools anyway if you have highly dynamic
440	pages.
441	</p>
442
443	</div>
444	References and related information...
445	<ul>
446	<li>
447	<a href="unit_test_documentation.html">Documentation for SimpleTest</a>.
448	</li>
449	<li>
450	<a href="http://www.lastcraft.com/first_test_tutorial.php">How to write PHP test cases</a>
451	is a fairly advanced tutorial.
452	</li>
453	<li>
454	<a href="http://simpletest.org/api/">SimpleTest API</a> from phpdoc.
455	</li>
456	</ul>
457	<div class="menu_back"><div class="menu">
458	<a href="index.html">SimpleTest</a>
459	\|
460	<span class="chosen">Overview</span>
461	\|
462	<a href="unit_test_documentation.html">Unit tester</a>
463	\|
464	<a href="group_test_documentation.html">Group tests</a>
465	\|
466	<a href="mock_objects_documentation.html">Mock objects</a>
467	\|
468	<a href="partial_mocks_documentation.html">Partial mocks</a>
469	\|
470	<a href="reporter_documentation.html">Reporting</a>
471	\|
472	<a href="expectation_documentation.html">Expectations</a>
473	\|
474	<a href="web_tester_documentation.html">Web tester</a>
475	\|
476	<a href="form_testing_documentation.html">Testing forms</a>
477	\|
478	<a href="authentication_documentation.html">Authentication</a>
479	\|
480	<a href="browser_documentation.html">Scriptable browser</a>
481	</div></div>
482	<div class="copyright">
483	Copyright<br>Marcus Baker 2006
484	</div>
485	</body>
486	</html>

Note: See TracBrowser for help on using the repository browser.

Download in other formats: