Expect_equal () passes with the arguments in the same order, fails if the arguments are swapped
When trying to use testthat::expect_equal()
with two numbers and a tolerance argument, it passes when the arguments are in a specific order, but fails if the two digits change the position of the argument. I noticed that this function is referencing all.equal()
from the base package, and this function also has a throughput / crash difference when changing arguments.
I expect the same answer regardless of the order of the first two arguments for both functions. Please let me know if this is not the correct expectation.
library(testthat)
# expect_equal does not throw error with one pair of numbers to compare
expect_equal(5, 1, tolerance=1)
# But does when the two numbers are reversed in their arguments
tryCatch(expect_equal(1, 5, tolerance=1), expectation_failure=conditionMessage)
#> [1] "1 not equal to 5.\n1/1 mismatches\n[1] 1 - 5 == -4\n"
# Since this seems to reference `all.equal()` I tried there too, and see an issue:
all.equal(1, 5, tolerance=1)
#> [1] "Mean absolute difference: 4"
all.equal(5, 1, tolerance=1)
#> [1] TRUE
# My session info:
sessionInfo()
#> R version 3.3.3 (2017-03-06)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 7 x64 (build 7601) Service Pack 1
#>
#> locale:
#> [1] LC_COLLATE=English_United States.1252
#> [2] LC_CTYPE=English_United States.1252
#> [3] LC_MONETARY=English_United States.1252
#> [4] LC_NUMERIC=C
#> [5] LC_TIME=English_United States.1252
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] testthat_1.0.2
#>
#> loaded via a namespace (and not attached):
#> [1] backports_1.0.5 R6_2.2.1 magrittr_1.5 rprojroot_1.2
#> [5] tools_3.3.3 htmltools_0.3.6 yaml_2.1.14 crayon_1.3.2
#> [9] Rcpp_0.12.10 stringi_1.1.5 rmarkdown_1.5 knitr_1.15.1
#> [13] stringr_1.2.0 digest_0.6.12 evaluate_0.10
source to share
TL; DR
Your carry is greater than
mean(abs(target-current)) / abs(target)
mean(5 - 1) / 5
4 / 5
0.8
So the function returns after comparison 0.8 < 1
because you allowed it to pass this admission check
Full answer
The function all.equal()
has arguments target
andcurrent
If you dig into the code when target
is equal to 5 and current
equal to 1, it calculates the average absolute difference
mean(abs(target - current))
# 4
Then a comparison is made between the target and the tolerance, which in this case is
5 > 1
## TRUE
Since it is TRUE, it calculates the relative difference
4 / 5
## 0.8
Here 0.8 is not more than the tolerance, so it is returned from the function with TRUE
, ie
all.equal(5, 1, tolerance = 1)
# [1] TRUE
So, your value is tolerance
used when comparing the relative difference. Therefore, the valid default is small values1.5e-8
Here I took the relevant code from all.equal.numeric
, removed only the parts of interest, so you can see it a little clear
allEqual <- function (target, current, tolerance = sqrt(.Machine$double.eps))
{
msg <- NULL
target <- as.vector(target)
current <- as.vector(current)
out <- is.na(target)
out <- out | target == current
if (all(out))
return(if (is.null(msg)) TRUE else msg)
target <- target[!out]
current <- current[!out]
xy <- mean(abs(target - current))
## THIS BIT HERE vv
what <- {
xn <- mean(abs(target))
# print(paste0("xn: ", xn))
if (is.finite(xn) && xn > tolerance) {
#print("xn (target) is GREATER than the tolerance")
xy <- xy/xn
#print(paste0("relative difference: ", xy))
"relative"
}
else{
# print("xn (target) is SMALLER than the tolerance")
"absolute"
}
}
## THIS BIT HERE ^^
if (is.na(xy) || xy > tolerance)
#print("xy is GREATER than the tolerance")
msg <- c(msg, paste("Mean", what, "difference:", format(xy)))
if (is.null(msg))
TRUE
else msg
}
allEqual(5, 1, tolerance = 0.79)
# [1] "Mean relative difference: 0.8"
allEqual(5, 1, tolerance = 0.81)
# [1] TRUE
allEqual(5, 1, tolerance = 1)
# [1] TRUE
allEqual(1, 5, tolerance = 1)
# [1] "Mean absolute difference: 4"
source to share