Top to Bottom Testing - Francis Hwang
Testing is harder in the real world
- Complexity
- External components
- Side-effects
- Speed
The quality elbow
quality on vertical, cost on horizontal axis
the basic idea is that at first cost pays off quickly with quality but overtime its payback results in lower and lower payback of increased quality.
E-Mail test Example
The message should know what the mailer is.
Sometimes a global variable is a global variable. The first thing you learn is that global variables are bad. The Mailer object is global! You will not change mailers in the middle of one invocation of the code.
He then deals with this issue in a IOC manner. In his case he creates mailer as a ContextualService.
references:
Senior Example
He is saying that you have to choose between:
1. Write a SQL parser
2. Write a thing like Criteria that generates the SQL for you
But it with option two you can run the tests much faster.
reference: senior example
Tradeoffs of complex mock classes
- Upsides
- Speed
- No side-effects and no cleanup
- Downsides
- Indirection
- Possible bugs in mock classes
- Time spent to build mock
You could mock anything
- Filesystem (mockfs)
- command line user entry
- network services
reference: mockfs example
Dynamicicity is your friend
Further improvement
- Test-centric libraries
- Domain-specific test languages – this can shorten the test setup code
Comments
DHH: They run against a real database. They use transactions to clean up the database after each test. He runs 410 tests in 60 seconds. Francis’ runs 500 tests in 3 minutes. But maybe the test suites are not equal.
Austin: Hey you could just use IOC
DHH: We looked at using Rails on Needle. Modify the load path of the mock classes before the regular library. Or do it after the load path of the library and just redefine the methods that you need change like deliver.
Azlak: You have to write mocks themselves.
Dude: Why not force a path to be prepended on the all file operations so that the mock.
Francis: Sometimes you need to simulate that a file system is full.
Dude: A comment is that with Francis’ approach, it is totally portable. You don’t really need a database or file system of a certain structure.
Dude: You still need some real tests against real things (DB, Mailer)
Patrick May: Whose responsibility is it to write mocks? Shouldn’t the library writer (say MySQL library) write the mocks for it.