Do you know your your code coverage percent?
posted under category: Browsers on March 19, 2008 by Nathan
Here's one that hits me excessively hard. What percentage of my code is covered with unit tests?
The reason I ask is because I know my code coverage percentage is abysmally low, easily in the single digits. It's hard to talk about because I'm a huge offender. However, even after looking in the mirror and viewing the plank in my eye, I know that this is typical in the world of ColdFusion developers. Most of you have never written a formal, automated test case. Ever. I know you. You don't. Admit it.
If you do it, kudos. Get yourself a cookie.
Now, code coverage. It's the measurement of how much of your application is covered by automated tests. I would argue that it is probably impossible to have 100% code coverage with things like environmental variables such as sessions, server operating systems, client browser dot versions and the types of input parameters your application can be given.
Let's take this simple function:function add(p1, p2) { return p1+p2; }
How many test cases do you need to write to have this code fully covered? Unfortunately, probably a lot. Starting with 0+0, 1+1, 0+1, 1+0, sure, that's easy. What about approaching the upper limits of a java integer? negative numbers? doubles and floats? Will it unbox automatically? What about those pentium and java bugs from years back that would return floating points on simple arithmetic? Maybe that's absurd, but at least we're not using it...
"IN HAZARDOUS ENVIRONMENTS REQUIRING FAIL-SAFE CONTROLS, INCLUDING [...] THE DESIGN, CONSTRUCTION, MAINTENANCE OR OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL, AND LIFE SUPPORT OR WEAPONS SYSTEMS"(what not to do, from the Coldfusion End User License Agreement 6.c)
So I think you can get 95% in just a few tests, zeros, small, medium and large sized numbers, as well as non-numbers and assert errors when it should throw errors.
(jump break, almost done)
How do you really define completely covered code? Is 95% good enough? Probably, as long as your unit tests are living and breathing with the application. I think one of my favorite quotes in this arena is "Make mistakes once." That is, once you find a bug in your software, add a condition to your test cases to recreate it, so that it never happens again.
While I know there are people even in the CF world that actually keep a good code coverage, and maybe a few that even track a code coverage percentage as if 100% were a goal. Where do you stand?