13

against testing

 3 years ago
source link: https://flak.tedunangst.com/post/against-testing
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

I really dislike writing tests. There's some amount of discomfort I'd be willing to sustain if I felt it they were beneficial, but I also find they're rarely worth the bother.

In order to be effective, a test needs to exist for some condition not handled by the code. However, typically when the same mind writes the code and the tests, the coverage overlaps. Errors arise from unexpected conditions, but due to their unexpected nature, these are the conditions which also go untested.

Even tests which deliberately aim for edge conditions have a tendency to miss. An off by one test will fail to properly detect an off by one in the code. Testing x < limit and y < limit will have 100% coverage without testing x + y < limit.

Tests are very brittle, breaking due to entirely innocuous changes in the code because they inadvertently embed very specific implementation dependencies. Test for a race condition? Actually a test of the scheduler's round robin algorithm.

Every test breakage then requires an investigation to determine what's really being tested, in addition to and besides the claimed history of the test. This is equal in effort to determining what function a block of code actually performs, except the understanding gained is not useful for future development.

And then there's the discussion whether the implementation dependent test isn't in fact helpful because what if external users of the code have also come to depend on this behavior, despite its undocumented nature. If it's important enough to test, it should be documented. But then that idea gets pushed back, because we need to keep it undocumented in case we want to change. Except that's exactly what we're trying to do if it weren't for the interfering test.

Tests are also dependent, in a good way, on the testing platform, but this requires they be run on the affected platform as well. A test that checks a function is endian neutral will fail to catch mistakes if it's only ever run on little endian systems.

Other tests grow obsolete because the target platform is retired, but not the test. There's even less pressure to remove stale tests than useless code. Watch as the workaround for Windows XP is removed from the code, but not the test checking it still works.

Certainly, one approach is to just write good tests that don't have these problems. If you like writing tests and are very good at it, you may continue to do so.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK