Code coverage goals for the API
What number would you give someone who wants a specific target number for API code coverage?
UPDATE. To clarify, operator / line code coverage. I understand that specific numbers don't make a lot of sense, but this is about the situation where you tell people that specific numbers don't make much sense and they still insist on getting a number from you no matter what. I wrote the API / SDK on purpose because some people might find lower code coverage more acceptable for application / GUI level software as opposed to libraries where there are more interfaces.
source to share
In fact, specific numbers don't make much sense.
If there are 14 equivalent cases, is your testing more biased if you don't test all 14? What if you have a bounds check that can't fail (I love to throw descriptive exceptions in these cases, which are then emailed to our Dev staff)?
Better to just make sure all logical paths are closed.
For a more detailed answer see this (very similar) question .
source to share
Take a look at CRAP - it combines coverage with complexity analysis. There is a Crap4J implementation if you are Java. We put together the Crap4Net that we wrote about here:
http://www.atalasoft.com/cs/blogs/insertqualityhere/archive/2008/03/28/crap4j-port-to-net.aspx
The idea is that you get good numbers if you have small simple methods or more complex ones have good coverage.
source to share
Just forget about code coverage. It's just a number and shouldn't be the focus when testing your code. Scripting should be the focus, followed by high quality APIs. I know this might sound like rhetorical bullshit, but you have to change your mindset from code coverage to scripting: Are you testing a lot of scripts that your API intends to handle?
Code coverage will be useful for detecting that you are missing some scripts, and if you write a lot of good scripts, you will have almost 100% coverage, but again, this is just a number and shouldn't be your focus.
respectfully
source to share
You can get some code coverage tools to give you "feature coverage", such as whether a function was executed. I would insist that every feature in the API has been covered.
Coverage of strings within an API implementation is a different matter. It is good practice to shoot for 70-80% coverage based on strings or statements and overall size.
source to share
If you're in PHP, 80% seems to be a good recommendation: http://jordionsoftware.blogspot.com/2009/09/code-coverage-targets.html
source to share