Testing the complete program by comparing the output file with the referenced file: what did it call and is there a package for it?
I have a completely non-interactive python program that takes some command line parameters and input files and creates output files. It can be tested quite easily by choosing simple cases and writing the input and expected output files manually, then running the program on the input files and comparing the output files with the expected ones.
1) What is the name for this type of testing?
2) Is there a python package for this type of testing? It's not hard to set it up manually in its most basic form, and I've done it already. But then I came across cases like output files containing date and other information that can legitimately change between runs - I thought of writing something that would allow me to specify which sections of the referenced files should be different, and yet pass the test and realized that I could get into the "rethinking the wheel". (I rewrote a good portion of unittest functions before I caught myself the last time this happened ...)
source to share
I am assuming that you mean system testing .
No package will know which parts can legally change. My suggestion is to mock the sections of code that lead to changes so you can be sure the output is always the same - you can use tools like Mock . Comparing the two files is pretty easy, just dump each line and compare the lines.
source to share