Is it possible to check if the code coverage has increased after the test
I am analyzing the coverage of a large project using coverage support in gcc with gcov. I also run random tests to try and increase coverage. Ideally, when I create a test, I would like to save it if it increases the reach.
What would be a reliable way to do this?
For example, let's say we have something like
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[]) {
int x, y;
x = atoi(argv[1]);
y = atoi(argv[2]);
if (x < 0 || y < 0) {
x = -x;
y = -y;
}
}
What am I compiling with
gcc -fprofile-arcs -ftest-coverage -o test test.c
Is there a way to determine if the reach has increased and keep the corresponding entries? For example, using something like
while [ 1 ];
do
./gen_test.sh > input.txt
./test < input.txt
if [ COVERAGE_INCREASED ] then save input.txt fi
done;
It takes a long time to run gcov to generate a report, so I'd rather know if the coverage has increased as soon as the program exits.
I know I can look at the .gcda files and see if they have changed. This is unfortunately not a solid solution as I am running random tests in parallel, and I would prefer if the "increase in scope" test is atomic - the project is large and the number of gcda / source files is substantial.
I guess if something like this is possible, I will need to add some gcc code to my main one, but I have no idea where to look.
Attempt 1 : I compiled gcc from source (4.6.3) and looked at the gcov code. When the program under the coverage test finishes, it runs some merge functions to update the coverage data already in the files. I updated one of these merge functions (libgcov.c) to just print something if the coverage was updated positively, which seems to serve my purposes.
/* The profile merging function that just adds the counters. It is given
an array COUNTERS of N_COUNTERS old counters and it reads the same number
of counters from the gcov file. */
void
__gcov_merge_add (gcov_type *counters, unsigned n_counters)
{
gcov_type prev;
for (; n_counters; counters++, n_counters--) {
prev = gcov_read_counter ();
if (prev == 0 && *counters > 0) {
fprintf(stdout, "Coverage increase by add");
}
*counters += prev;
}
}
It seems to be doing the right thing to cover the line, but it lacks branch coverage. Also, it doesn't capture the initial launch, but that's less of a problem.
By following the above example, I get
./test 0 0
./test -1 0
Coverage increase by add
./test 0 -1
source to share
I solved it by overloading the gcc add method in the code. For good practice to obtain gcov_type
, you can install the package gcc-4.6-plugin-dev
.
Now the above program can be adapted as
#include <stdio.h>
#include <stdlib.h>
#include <gcc-plugin.h>
#include <basic-block.h>
gcov_type __gcov_read_counter (void);
void
__gcov_merge_add (gcov_type *counters, unsigned n_counters)
{
gcov_type prev;
for (; n_counters; counters++, n_counters--) {
prev = __gcov_read_counter ();
if (prev == 0 && *counters > 0) {
fprintf(stdout, "Coverage increased\n");
}
*counters += prev;
}
}
int main(int argc, char* argv[]) {
int x, y;
x = atoi(argv[1]);
y = atoi(argv[2]);
if (x < 0 || y < 0) {
x = -x;
y = -y;
}
}
To compile, you need to add a few additional
gcc -I/usr/lib/gcc/x86_64-linux-gnu/4.6/plugin/include \
-fprofile-arcs -ftest-coverage \
-o test test.c
This binary will print "Gain Increase" prints for each increase in Gamut.
source to share