My Approach to Test Driven Development Part 3 - Testing Against a Database

Confidently unit testing code that relies on a database is troublesome. There are many different approaches and many opinions on the matter. I've previously written about using transactions to safely rollback the database after every test. This worked well for me at the time, but there are pitfalls in using that approach. Recently, I've done some more research on the subject and even attempted, without success, to integrate DBUnit into my database reliant unit tests. After some trial and error, I landed on the following approach that is a culmination of a few ideas I'd previously heard of, but never tried. The goal of this approach is to:

  1. Run tests against a known data set.
  2. Reset the entire database prior to each test.

The example code that follows requires a test database, test database user and a test datasource targeting the test database.

In my previous post, I described my local setup for the fictitious site www.langerhans.local. Let's assume that the database for this site is named langerhans and it is a MySQL db. For testing purposes, I would create a test specific version called langerhans_test which can be either a point-in-time backup, or an initial creation script with static data tables. To reset my database to this state repeatedly, I use an sql script. The following script is a simple example of a database creation script with a few tables and default data.

DROP schema langerhans_test;

CREATE schema langerhans_test;

USE langerhans_test;

DROP TABLE IF EXISTS tbl_user;
CREATE TABLE tbl_user (
UserId INT UNSIGNED NOT NULL AUTO_INCREMENT,
FirstName varchar(50) NOT NULL,
LastName varchar(50) NOT NULL,
Email varchar(100) NOT NULL,
Username varchar(16) NOT NULL,
Password char(32) NOT NULL,
PRIMARY KEY (UserId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

DROP TABLE IF EXISTS tbl_role;
CREATE TABLE tbl_role (
RoleId TINYINT UNSIGNED NOT NULL,
Name varchar(50) NOT NULL,
Description varchar(100) NOT NULL,
PRIMARY KEY (RoleId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO tbl_role (RoleId,Name,Description) VALUES (1,'Master','Full priviledge account.');
INSERT INTO tbl_role (RoleId,Name,Description) VALUES (2,'Admin','Admin account with grant priviledge.');
INSERT INTO tbl_role (RoleId,Name,Description) VALUES (3,'User','Normal user account.');
INSERT INTO tbl_role (RoleId,Name,Description) VALUES (4,'Guest','Anonymous account.');

DROP TABLE IF EXISTS tbl_user_role;
CREATE TABLE tbl_user_role (
UserId INT UNSIGNED NOT NULL,
RoleId TINYINT UNSIGNED NOT NULL,
PRIMARY KEY (UserId,RoleId),
KEY ix_UserId (UserId),
KEY ix_RoleId (RoleId),
CONSTRAINT user_role_has_user FOREIGN KEY (UserId) REFERENCES tbl_user (UserId),
CONSTRAINT user_role_has_role FOREIGN KEY (RoleId) REFERENCES tbl_role (RoleId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

DROP TABLE IF EXISTS tbl_permission;
CREATE TABLE tbl_permission (
PermissionId TINYINT UNSIGNED NOT NULL,
Name varchar(50) NOT NULL,
Description varchar(100) NOT NULL,
PRIMARY KEY (PermissionId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO tbl_permission (PermissionId,Name,Description) VALUES (1,'CREATE','create permission');
INSERT INTO tbl_permission (PermissionId,Name,Description) VALUES (2,'VIEW','view permission');
INSERT INTO tbl_permission (PermissionId,Name,Description) VALUES (3,'EDIT','edit permission');
INSERT INTO tbl_permission (PermissionId,Name,Description) VALUES (4,'DELETE','delete permission');
INSERT INTO tbl_permission (PermissionId,Name,Description) VALUES (5,'GRANT','grant permission');


DROP TABLE IF EXISTS tbl_role_permission;
CREATE TABLE tbl_role_permission (
RoleId TINYINT UNSIGNED NOT NULL,
PermissionId TINYINT UNSIGNED NOT NULL,
PRIMARY KEY (RoleId,PermissionId),
KEY ix_RoleId (RoleId),
KEY ix_PermissionId (PermissionId),
CONSTRAINT role_permission_has_role FOREIGN KEY (RoleId) REFERENCES tbl_role (RoleId),
CONSTRAINT role_permission_has_permission FOREIGN KEY (PermissionId) REFERENCES tbl_permission (PermissionId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (1,1);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (1,2);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (1,3);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (1,4);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (1,5);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (2,2);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (2,3);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (2,4);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (2,5);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (3,2);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (3,3);
INSERT INTO tbl_role_permission (RoleId,PermissionId) VALUES (4,2);

To execute this script, I pass the path to the sql file and datasource to a utility component.

<cfcomponent displayname="model.util.DBScript" output="false">

<cffunction name="init" access="public" returntype="any" output="false">
<cfargument name="datasource" type="string" required="true">
<cfargument name="source" type="string" required="true">
<cfset var script = "">
<cffile action="read" file="#expandPath(arguments.source)#" charset="utf-8" variable="script">
<cfset variables.datasource = arguments.datasource>
<cfset variables.source = source>
<cfset variables.script = script>
<cfreturn this>
</cffunction>

<cffunction name="execute" access="public" output="false" returntype="boolean">
<cfset var reset = "">
<cfset var success = false>
<cftry>
<cfquery name="reset" datasource="#variables.datasource#">
#preserveSingleQuotes(variables.script)#
</cfquery>
<cfset success = true>
<cfcatch type="database">
<cfthrow message="script [#variables.source#] could not be executed.">
</cfcatch>
</cftry>
<cfreturn success>
</cffunction>

</cfcomponent>

I would use the following Coldspring configuration for testing a database gateway.

<bean id="Gateway" class="path.to.Gateway">
<constructor-arg name="datasource">
<value>${datasource}</value>
</constructor-arg>
</bean>
<bean id="DBScript" class="model.util.DBScript">
<constructor-arg name="datasource"><value>${datasource}</value></constructor-arg>
<constructor-arg name="source"><value>${resetScript}</value></constructor-arg>
</bean>

And finally, here is my Gateway test case.

<cfcomponent displayname="GatewayTest" extends="tests.BaseTestCase">

<cffunction name="setUp" access="public" returntype="void">
<cfscript>
var beanDefFileLocation = expandPath("/" & Replace(Getmetadata(this).displayname,'.','/','all') & ".xml");
var params = structnew();
params.datasource = 'langerhans_test';
params.resetScript= '/sql/reset.sql';
setBeanFactory(beanDefFileLocation,params);
getBeanfactory().getBean('DBScript').execute();
</cfscript>
</cffunction>

<cffunction name="getRoleListReturnsQueryWithFourRecords" access="public" returntype="void">
var roleList = getBeanFactory().getBean("Gateway").getRoleList();
assertTrue(isQuery(roleList),"roleList is not a query");
assertTrue(roleList.recordcount eq 4);
</cffunction>

</cfcomponent>

Executing the DBScript within the setup() method ensures that the database is reset to a known state prior to each test. This works well for a smallish database, but it might be unwieldy for larger projects or databases.

Comments
Nathan Mische's Gravatar I'm currently using the transaction technique, but I've definitely thought about this approach. It seems it may be easier in certain circumstances because you can you have a given set of test records in place. This would be useful in dealing with referential integrity issues with DAOs as well as reads with Gateways.

How is the speed? I currently insert all required records for each test, and then roll them back when the test completes, which makes for some slow tests.
# Posted By Nathan Mische | 9/9/08 9:10 AM
Paul Marcotte's Gravatar Hi Nathan,

I've seen mixed results with a MySQL db. On Mac, very fast. On Windows, dog slow, so you mileage may vary.

This brute force approach is clearly not applicable to all application development.
# Posted By Paul Marcotte | 9/9/08 6:41 PM
Jaime Metcher's Gravatar Paul,

I agree that dbUnit isn't the answer. I've found it useful for testing ETL scripts, but for general app testing it's too time consuming to set up and too fragile.

My preferred approach is to use my domain model API to create all my test data at test setup time, then remove it all again at teardown. This has a bunch of benefits:

1. the basic API of my model layer gets a huge workout
2. I have to actually have a complete model API early in the project. There's nothing worse than finding out three months into production that nobody wrote the delete() methods because they didn't want to wrap their head around the cascading issues (you may laugh...)
3. My tests are not sensitive to the initial state of the database. Most of my projects are updates and extensions to our core product. As they will be deployed into an existing, noisy environment, I like to test them against copies of the production databases.

So rather than write tests that say "table x should now have four records", I say "table x should now have four more records than it had before".

As for the model tests, I basically bootstrap them. Write the add() method first, do the teardown in raw SQL, test that. Then write the delete() method, test that. Then I can start writing tests using add() in setup and delete() in teardown, and I'm basically good to go.

Jaime
# Posted By Jaime Metcher | 9/10/08 5:42 PM
Paul Marcotte's Gravatar Jaime,

Thanks for outlining your process. I tried your approach and like it very much! I was a little leary at first since building the required data set involves more of my api than the unit under test. The good part is that I improved my api overall in the process of building the test fixtures.
# Posted By Paul Marcotte | 9/18/08 6:34 PM
Jaime Metcher's Gravatar "I tried your approach and like it very much!"

Cool, glad to help.

"I was a little leary at first since building the required data set involves more of my api than the unit under test"

I know what you mean - is it really unit testing if it has so many dependencies? The answer is - of course it is! Unit testing doesn't mean no dependencies, you just draw a line and put the stable (tested) parts of your stack on one side and the unstable (under development) parts on the other side. So once you've got a tested model API, it's just as legitimate a part of a unit test as is CFCUnit or the JDBC driver.

"The good part is that I improved my api overall in the process of building the test fixtures."

Ah, the awesome synergy of TDD!
# Posted By Jaime Metcher | 9/18/08 7:08 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.9. Contact Blog Owner