Author Archives: jackkavanagh

Link

Test Driven Development

Refactoring

+ R1: A good test suite increases confidence that code changes won't cause 
+ unexpected side effects without being noticed.

+ R2: Tests reduce the cost of structural changes in later stages of a project 
+ because fewer bugs sneak into a system unnoticed.

- R3: Brittle tests can increase the cost of refactoring if they focus on 
- superficial details rather than core functionality.

- R4: Each refactoring may involve removing old tests and adding 
- new ones, increasing total effort per change.

Agility

+ A1: Well tested code tends to be well factored, which makes introducing 
+ new features easier.

+ A2: Tests may help developers arrive at better designs and interfaces 
+ between systems up front, reducing the amount of rework that is needed later.

- A3: In the early stages of projects, the cost of writing tests reduces the 
- speed of iterating through different ideas, so the cost of running into
- a dead end is higher.

- A4: Figuring out which tests to start with (and at what level of abstraction
- to start at) can be challenging in projects that are not well defined.

- A5: Tests force domain modeling decisions early on, making it harder to make 
- major decision changes later, and harder to defer design decisions. 

- A6: Slow test runs can reduce the benefits of an automated test suite by 
- slowing down the feedback loop.

Learnability

+ L1: A good test suite shows system use cases at various levels of abstraction, 
+ allowing developers to gain some insight into how different subsystems 
+ are meant to be used.

+ L2: Tests force interfaces to be thought about early on, which tends to 
+ increase their ease-of-use and quality.

+ L3: Tests help us resolve inconsistencies between documentation and 
+ implementation by verifying specific expectations.

- L4: Learning testing tools and techniques involves a significant 
- up-front commitment.

- L5: Test suites are often used as project documentation 
- (which can be useful), but are a poor substitute for high-level, 
- topical documentation and example programs because they can be 
- hard to interpret in isolation.

- L6: Poor quality test suites can be more confusing than 
- enlightening if they test via paths that would be awkward 
- in real use, use contrived examples, or involve a lot of 
- complex setup that is only loosely relevant to the task at hand.

- L7: The seemingly endless variety of testing tools and 
- methodologies make for a steep learning curve even for 
- experienced developers.

Correctness

+ C1: Writing tests requires a fairly clear understanding of requirements up 
+ front, which may encourage clarifying specifications and resolving their 
+ ambiguities early on.

+ C2: Regression tests help ensure that incorrect behavior does not 
+ recur once a fix is introduced.

- C3: Tests can represent incorrect assumptions or have defects in 
- them, just like implementation code can. This can lead to a false 
- sense of security for those who depend on passing test suites as a 
- sign of correctness.

Maintenance

+ M1: A good test suite makes changing or updating dependencies safer, 
+ because a test suite will help catch dependency-related problems.

+ M2: Well-tested projects make it easier for developers who are 
+ new to a codebase to safely make fixes and changes without 
+ understanding the nuances of the entire project, allowing 
+ maintenance effort to be distributed.

- M3: Every line of code in your tests also needs to be maintained, 
- and tests can often have much more code than implementation code does.

- M4: Testing tools introduce new dependencies and potential 
- maintenance costs into projects.

Design

+ D1: TDD can act as "training wheels" for software design, reducing the 
+ barrier to entry for beginner/intermediate programmers to concepts 
+ like SOLID, Law of Demeter, etc. The benefits of these techniques 
+ directly improve testability, which makes the effort in applying 
+ them feel worthwhile.

+ D2: TDD discourages taking shortcuts in API design, because brittle 
+ interfaces are harder to test than well-factored ones.

+ D3: TDD encourages clear, well defined domain models, as well as 
+ "discovered objects" (i.e. reusable bits of functionality that 
+ weren't directly visible from the project's specification).

- D4: TDD discourages unstructured play early on in a project, 
- which may reduce the number of different ideas considered 
- before a particular design is settled on.

- D5: Many insights about software design that can be learned via 
- TDD can be also be practiced in less formal ways, with similar 
- benefits and fewer costs.

- D6: TDD can lead to overengineering in some situations: the tendency 
- to focus on domain modeling at all levels of abstraction comes at 
- the cost of potentially spending too much energy improving 
- unimportant subsystems.

Quantitative Metrics

+ Q1: Measurements such as code coverage, test run times, and 
+ continuous integration reports give us useful ways of analyzing 
+ some important traits of our projects.

- Q2: It is easy to over-emphasize or misinterpret what quantitative metrics
- actually say about our projects.

Philosophy

+ Good software testing practices (in particular TDD), place strong
+ emphasis on software quality and maintainability.

+ Good testing practices lead to increased trust among developers 
+ in their own work as well as the work of others, which reduces 
+ defensive programming.

- Dogmatism about how to test and what tools to use can be 
- distracting and also lead to unproductive fears and insecurity 
- about whether one is "doing it right".

- Overemphasizing software testing as a golden hammer can make it
- easy to forget that product quality is not the same as code quality, 
- and that the former is more important than the latter. (Of course,
- it is worth keeping in mind that code quality at least implicitly
- affects product quality)