Tests and Testability and Debugging and Maintainability (2017-05-18)

Tests. Tests are good. They enable us to make sure that code works correctly – or, to be more precise, that it works according to the restatement of the requirement contained in the test.

Having to write tests is worthwhile all by itself. There is nothing like having to write tests to make you write code that’s inherently testable. And what’s most testable? Functions with well defined inputs and outputs that neither affect nor depend upon any external state. Pure functions, for example.

But, as the huckster said, there’s more

Any time you can limit the ‘universe’ that any piece of code can see, you also limit what you have to look at when things go wrong. And that can ease both debugging and future changes. Locality matters. Nearby is always better than far away. Unfortunately, it appears the predominant methodology of building web applications only serves to encourage spreading the semantics of code all over the codebase, segregating it by how it is used structurally by the program rather than by what it models. As a result adding functionality can become a chore. Rather than being able to just add all the behavior of a modeled type in one place, it must be added in several; the persistent data aspect is added in one place, the behavior of the modeled data in another, verification (i.e. “Can a coherent instance be built from these components?”) in another, and how it is to be displayed to or received from external sources in one or several more.

Now I don’t know about you, but in my case the more assets I need to touch, the more I can break. (I’m not even sure we can say it’s limited to being a linear relationship.)

So I tend not to be thrilled about having to make changes or additions all across a codebase.

Let’s take a step back: When a test fails, what does that mean? Obviously it means that some expectation of the code is not met; it also means “we have work to do!” But what isthat work? Where is it? What has gone wrong? Too often, because of the diffuse way we tend to structure things, debugging the problem requires taking on a rather significant cognitive load; you have to know too much and dig too deep to debug. Painful? Sure. Inefficient? Definitely.

Brief interlude:

Why do we like to use garbage-collected languages?

Because having to allocate and deallocate memory is hard and annoying? Well, sure. But what’s the real reason?

Because debugging things like memory leaks or, especially segfaults, SUCKS!!!

And why does it suck? Because the segfault itself is typically separated from its cause both in code and in time. As a result you have to work backwards through a richly constructed state to find the problem.

Sound familiar? You’re working on a codebase that’s new to you, adding a feature. You write some code, tests are in place and it all seems good. Suddenly (insert ominous music here) there’s a problem: Test #74, confirmFacilityDataIsUpdated fails with the message “0 is not greater than 0”. And, as a result, your change is not going out. At least not yet. Naturally, the first thing you do is look at the test itself. (Actually the second; the invective, whether private or shared, likely comes first.) Looking at the test’s code you see state being built up either in the form of mocks or, perhaps, by dragging data out of some database or another (and just assuming it’s there). Then some code is called to somehow mutate that state. Then, finally, some more code (the assertion itself) checking to see if the state has been mutated in some particular way. Wow.

Armed with all this knowledge, what’s the first thing you do? RERUN THE TESTS, THAT’S WHAT YOU DO! (And maybe, just maybe, it goes green, you issue the PR for your own – nicely tested – new feature and all is well.)

More likely, though, you start digging through the codebase like an ever increasing wavefront, eventually find a Heisenbug, think just a little bit less of your colleagues and move on. Though, yes, it’s part of what we do, it tends to be more painful than it needs to be. And pain, as a developer, reduces your bandwidth and makes you less effective. And often, we tend to test at a feature level – where a lot of state has already been munged – because imperative code has a nasty habit of swallowing up state changes, not producing intermediate, more fundamental results that can be tested more directly. And, perhaps, that’s the whole point,

Tests, at their best, provide several benefits; the first two are obvious, telling you that your code is:

  • ‘Correct’ (i.e. conforming to the specification expressed by the test itself)
  • ‘Still correct’ (i.e. nothing has broken that contract since the last time the code was run)

But don’t stop there. You can write the tests first, at which point they become very much akin to being a formal specification. And the tests will be simple. And the code you write against them will be more value oriented (when you write a test first, you’re going to write it as simply as you can; it’ always preferable to look at a value as opposed to collecting a bunch of state).

This post has gotten a bit long – but one more thing before we get out of here: In a TDD world, a failing test is a signal that code needs to be written or changed. And it’s the same if you’re debugging or enhancing existing code. So why not use the test suite as a place to hole information about that’s expected of the code? (Which is really what it does anyway.) Yes, zero may not be greater than zero – but knowing why that’s meaningful would sure be helpful.

[Addendum: On a podcast, I recently ran across the the idea of BDD (Behavior-Driven-Development) vs. TDD (Test-Driven-Development), corresponding, roughly, to unit tests and somewhere between ‘feature’ and integration tests, where some kind of end-to-end behavior is tested. There’s clearly some value here. Some of the effort devoted to the most trivial of unit tests could be utilized in testing the larger behaviors; any failure of behavior tests without corresponding failure of underlying unit tests would pretty well confine the error to the higher level behavior-oriented logic.]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s