Automated Testing, and its importance to Drupal Development

Drupal - Automated Testing

While Drupal 6 is still getting ready for production use, work on Drupal 7 is already under way. One of the major patches that made it in so far was the inclusion of SimpleTest into core. Unlike most patches, it doesn't affect the end-user, but it affects developers and the development process.

Automated testing is already a recognized best practice in software engineering, allowing to develop better, more flexible and more secure code, but since Drupal didn't have a well-established testing framework, we (as drupal developers) have been lucky enough not to have to learn whatever that automated testing is, until now.

What it is

Automated tests are a way to simulate the execution of a certain piece of code in controlled settings, and check that the resulting output matches the desired output.

In drupal, automated tests can be separated in two groups: functional tests and unit tests. Functional tests simulate the actions of a user going through specific steps, such as logging in, creating a node, viewing that node, editing that node, viewing that node again and deleting the node. At every step we check the content of the returned HTML page for content that confirms the expected behavior, such as the presence of status messages, form fields or a specific text. A good example of functional tests is in user.test.

Unit tests on the other hand test the code at a lower lever, for example testing the return value of a function when given specific parameters. Some functions will depend on external data not controlled by parameters, such as the content of the database. Adding the ability to control such external data could be possible but also very complex, and it was decided that for now the level of control that we have by adjusting the external data to our needs is sufficient. A good example of unit testing is in search.test.

Running tests

First, get the latest version of Drupal 7 from CVS (HEAD). SimpleTest is now in core so you don't need to install any additional modules or libraries as it used to be the case in previous versions. You just need to enable the SimpleTest module and go to admin/build/testing and choose which tests you want to run.

For every test, SimpleTest will install drupal with a new random database prefix so that your current site does not get altered. All tests are run on that sandboxed drupal installation, which gets deleted at the end of the tests (if it doesn't, the SimpleTest module lets us clean up obsolete files and database tables with a simple click). After a test is run, we get a list of all the conditions that were checked for (also called Assertions) and which ones passed, which ones failed, and a list of exceptions that were thrown during the test execution.

Writing tests

SimpleTest looks for .test files matching modules (like user.test for user.module) and .test files matching files in the includes folder (like boostrap.inc.test for bootstrap.inc). Test files must contain a class which extends the DrupalWebTestCase class, and which implements a getInfo() method.

For each test class, SimpleTest automatically detects the methods starting with 'test' and considers them as a test. Before any test method is run, the setUp() method is called (which is practical to make sure that your modules are enabled, etc), and the tearDown() method is called afterwards (but it is rarely needed since we're operating in a sandboxed environment which is going to be destroyed anyway).

How to learn it

Although the SimpleTest framework is in drupal core, there's still a lot of work to be done to have a good test coverage of all the code. Some existing tests need a good clean-up, and even though most modules have tests, very few of the core include files do. Pick a simple function, and write a test for it using existing tests as an example. Plus, you're going to get a free code review by the community!