Does property testing allow you to duplicate code?
I'm trying to replace some old unit tests with Property Based Test (PBT), specificley with scala
and scalatest - scalacheck
, but I think the problem is more general. Simplified situation, if I have a method that I want to test:
def upcaseReverse(s:String) = s.toUpperCase.reverse
Normally I would write block tests like:
assertEquals("GNIRTS", upcaseReverse("string"))
assertEquals("", upcaseReverse(""))
// ... corner cases I could think of
So, for each test, I write the output I expect, no problem. Now with PBT it will look like this:
property("strings are reversed and upper-cased") {
forAll { (s: String) =>
assert ( upcaseReverse(s) == ???) //this is the problem right here!
}
}
As I try to write a test that will be true for all inputs String
, I find that I need to re-write the method logic in the tests. In this case, the test will look like this:
assert ( upcaseReverse(s) == s.toUpperCase.reverse)
That is, I had to write an implementation in the test to make sure the result was correct. Is there a way out of this? I am misunderstanding PBT and should I be testing other properties like:
- "lines must be the same length as the original"
- "strings must contain all characters in the original"
- "strings must not contain lowercase characters" ...
This is also plausible, but sounds like a lot of contrived and less clear. Can anyone with extensive PBT experience shed some light here?
EDIT : Following @Eric's sources I got this post and here's an example of what I mean (when applying categories again): to test a method times
in ( F#
):
type Dollar(amount:int) =
member val Amount = amount
member this.Add add =
Dollar (amount + add)
member this.Times multiplier =
Dollar (amount * multiplier)
static member Create amount =
Dollar amount
the author ends up writing a test that looks like this:
let ``create then times should be same as times then create`` start multiplier =
let d0 = Dollar.Create start
let d1 = d0.Times(multiplier)
let d2 = Dollar.Create (start * multiplier) // This ones duplicates the code of Times!
d1 = d2
So, to check that the method is duplicated in the test. In this case, something as trivial as multiplication, but I think it extrapolates to more complex cases.
source to share
This presentation gives some hints about the types of properties you can write for your code without duplicating it.
In general, it's helpful to think about what happens when you compose a method you want to test with other methods in that class:
-
size
-
++
-
reverse
-
toUpperCase
-
contains
For example:
-
upcaseReverse(y) ++ upcaseReverse(x) == upcaseReverse(x ++ y)
Then think about what will break if the implementation breaks. If the property fails if:
- the size is not preserved?
- weren't all the characters in uppercase?
- the line was not changed correctly?
1. is actually meant to be 3. and I think the above property would have broken at 3. However, it would not have broken at 2 (if there were no top level, for example). Can we improve it? What about:
-
upcaseReverse(y) ++ x.reverse.toUpper == upcaseReverse(x ++ y)
I think this is ok, but don't trust me and don't run tests!
Anyway, I hope you get the idea:
- with other methods
- see if there are any equalities (like "round-tripping" or "idempotency" or "model checking" in the presentation)
- check to see if your property breaks with wrong code.
Note that 1. and 2. are implemented by a library named QuickSpec and 3. mutation testing .
Adding
About your edit: The operation Times
is just a wrapper around *
, so nothing to test. However, in a more complex case, you can check that the operation:
- has element
unit
- is associative
- is commutative
- is distributive with the addition
If any of these properties fail, it will be a big surprise. If you code these properties as general properties for any binary relation T x T -> T
, you should be able to use them very easily in all contexts (see Scalaz Monoid Laws).
Going back to your example upperCaseReverse
, I would write two separate properties:
"upperCaseReverse must uppercase the string" >> forAll { s: String =>
upperCaseReverse(s).forall(_.isUpper)
}
"upperCaseReverse reverses the string regardless of case" >> forAll { s: String =>
upperCaseReverse(s).toLowerCase === s.reverse.toLowerCase
}
This does not duplicate code and contains two different things that can break if your code is wrong.
In conclusion, I had the same question as before and I got very frustrated, but after a while I found more and more cases where I did not duplicate my code in properties, especially when I start thinking about
- combining the function under test with other functions (
.isUpper
in the first property) - comparison of the function under test with a simpler "model" of computation ("the opposite regardless of the case" in the second property)
source to share