Acceptance testing and code coverage
When doing acceptance testing, I'd rather look at the functional coverage, that is, how many features (or user stories or use cases) of a given application have been tested. And for me, for every feature (or user story or use case), all scenarios or possible paths must be tested. In other words, functions should be the starting point for writing a test plan, and the goal should be to cover functions, not code. This is a small difference, but important: the code coverage acceptance test dimension is developer-oriented, the functional coverage dimension is end-user oriented.
In terms of measurement, Ron Jeffries, one of the founding fathers of Agile, has written an excellent summary of what he considers to be truly valuable metrics, RTF or Proven Feature Launch. You can view the article here . I am just quoting a small part below:
What is the essence of the project?
I'm just guessing, but I think the point of most software development projects is software that works and has as many features as possible for a dollar investment. I call this the concept of "Health Check" [Capabilities], and in fact it can be measured to some extent.
Imagine the following definition of RTF:
- Desired software is broken down into named functions (requirements, stories) that are part of what it means to deliver the desired system.
- For each named function, there is one or more automatic acceptance tests that, when they run, will indicate that the function has been implemented.
- The RTF metric shows, at each moment of the project, how many functions pass all their acceptance tests.
How many user-defined functions are known to work through independent testing? Now there is a metric that I could live with.
To communicate progress, I especially like cumulative flowcharts. They clearly show when something goes wrong, for example (here's the burnout style):
(source: xprogramming.com )
For a real life example, take a look at this post or this one (stylesheet).
source to share