Consider the behavior of the data and procedure caches when you measure the performance of a system. When a test begins, the cache can be in any one of the following states:
Empty
Fully randomized
Partially randomized
Deterministic
An empty or fully randomized cache yields repeatable test results because the cache is in the same state from one test run to another.
A partially randomized or deterministic cache contains pages left by transactions that were just executed. Such pages could be the result of a previous test run. In these cases, if the next test steps request those pages, then no disk I/O will be needed.
Such a situation can bias the results away from a purely random test and lead to inaccurate performance estimates.
The best testing strategy is to start with an empty cache or to make sure that all test steps access random parts of the database. For more precise testing, execute a mix of queries that is consistent with the planned mix of user queries on your system.
