Property-based testing doesn’t just find bugs; it finds classes of bugs by testing your code with systematically generated, often edge-case inputs.
Let’s say you’re building a function to sort a vector of integers:
pub fn sort_vec(vec: &mut Vec<i32>) {
vec.sort();
}
You’ve written a few manual tests:
#[test]
fn test_sort_empty() {
let mut v = vec![];
sort_vec(&mut v);
assert_eq!(v, vec![]);
}
#[test]
fn test_sort_single() {
let mut v = vec![5];
sort_vec(&mut v);
assert_eq!(v, vec![5]);
}
#[test]
fn test_sort_sorted() {
let mut v = vec![1, 2, 3];
sort_vec(&mut v);
assert_eq!(v, vec![1, 2, 3]);
}
#[test]
fn test_sort_reverse() {
let mut v = vec![3, 2, 1];
sort_vec(&mut v);
assert_eq!(v, vec![1, 2, 3]);
}
These are good, but they only cover specific, hand-picked scenarios. Property-based testing lets you define properties that should hold true for all valid inputs. For sort_vec, a key property is that the sorted vector should be identical to the original vector, but with its elements arranged in non-decreasing order.
We can use the quickcheck crate for this. First, add it to your Cargo.toml:
[dependencies]
quickcheck = "1.0.3"
Now, let’s write our property test. quickcheck works by defining a Testable trait, often implemented via a QuickCheck struct. You provide a function that takes generated arguments and returns true if the property holds, false otherwise.
use quickcheck::{quickcheck, TestResult};
// Assume sort_vec is in the same module or imported
// pub fn sort_vec(vec: &mut Vec<i32>) {
// vec.sort();
// }
fn is_sorted(vec: &[i32]) -> bool {
vec.windows(2).all(|w| w[0] <= w[1])
}
#[quickcheck]
fn prop_sort_preserves_elements(mut vec: Vec<i32>) -> TestResult {
let mut sorted_vec = vec.clone();
sort_vec(&mut sorted_vec);
// Property 1: The sorted vector must be sorted.
if !is_sorted(&sorted_vec) {
return TestResult::error("Vector was not sorted after calling sort_vec".to_string());
}
// Property 2: The sorted vector must contain the same elements as the original.
// To check this, we sort both and compare. If the original `vec` was already sorted,
// then `sorted_vec` should be identical to `vec`.
vec.sort(); // Sort the original `vec` for comparison
if sorted_vec != vec {
return TestResult::error("Sorted vector does not contain the same elements as the original".to_string());
}
TestResult::passed()
}
When you run cargo test, quickcheck will generate thousands of Vec<i32> instances and feed them to prop_sort_preserves_elements. If any of these generated vectors cause the function to return TestResult::error, quickcheck will report it and try to "shrink" the failing input to the smallest possible failing case.
For instance, quickcheck might generate vec![1, 0, 2]. sort_vec would turn it into [0, 1, 2]. is_sorted would return true. Then, it would sort the original vec (which becomes [0, 1, 2]) and compare it to sorted_vec ([0, 1, 2]). They match, so TestResult::passed().
If your sort_vec had a bug, say it incorrectly handled duplicates:
// Buggy sort_vec for demonstration
pub fn sort_vec_buggy(vec: &mut Vec<i32>) {
if vec.len() > 1 {
// Incorrectly swaps if elements are equal
if vec[0] == vec[1] {
vec.swap(0, 1);
}
vec.sort(); // Still calls the correct sort later, but the initial swap is wrong
}
}
A generated input like vec![1, 1, 0] might cause sort_vec_buggy to swap the 1s, resulting in [1, 1, 0], and then sort it to [0, 1, 1]. The is_sorted check would pass. However, the original vec would be [1, 1, 0]. After sorting the original vec for comparison, it becomes [0, 1, 1]. So, sorted_vec == vec would pass.
Wait, my buggy example didn’t fail. Let’s try a different bug:
// Another buggy sort_vec
pub fn sort_vec_buggy_2(vec: &mut Vec<i32>) {
if vec.len() > 1 {
// Incorrectly removes the second element if it's smaller than the first
if vec[0] > vec[1] {
vec.remove(1);
}
vec.sort(); // This sort might not be called on the modified vec
}
}
If quickcheck generates vec![2, 1, 3]:
sort_vec_buggy_2seesvec[0] > vec[1](2 > 1).- It removes
vec[1](the1), sovecbecomes[2, 3]. - It then calls
vec.sort(), resulting in[2, 3]. sorted_vecis[2, 3].is_sorted(&sorted_vec)istrue.- The original
vecwas[2, 1, 3]. Sorting it for comparison gives[1, 2, 3]. sorted_vec([2, 3]) is NOT equal to the sorted originalvec([1, 2, 3]).TestResult::erroris returned.
quickcheck would then report:
thread 'prop_sort_preserves_elements' panicked at 'Sorted vector does not contain the same elements as the original', src/lib.rs:27:14
And it would try to shrink vec![2, 1, 3] to the smallest failing case, which might be vec![1, 0].
The true power of property-based testing lies in its ability to explore the input space exhaustively. It forces you to think about the invariant properties your code must maintain, rather than just specific output values for specific inputs. This leads to more robust and correct code.
The "shrinking" mechanism is particularly clever. When a property fails, quickcheck doesn’t just give you a random large input. It systematically tries to simplify that input while still causing the failure. This process is crucial for pinpointing the exact edge case that breaks your logic, often reducing a complex failing input like vec![1, 5, 3, 8, 2, 9, 0, 4, 7, 6] down to something like vec![1, 0]. This makes debugging significantly easier.
A common pitfall is writing properties that are too trivial or too closely tied to the implementation details of the function being tested. The best properties are abstract, focusing on the observable behavior or guarantees of the function, independent of how it achieves that behavior. For example, testing that sort_vec results in a vector where v[i] <= v[i+1] is a good property. Testing that sort_vec uses a quicksort algorithm internally is not a good property test, because it ties the test to the implementation.
The next step after mastering basic property-based testing is exploring custom types and combinators for generating more complex data structures.