The drstatecmp DynamoRIO Machine State Comparison Extension provides mechanisms to enable systematic and exhaustive machine state comparisons across instrumentation sequences. These comparisons will detect instrumentation-induced corruptions of the application's machine state that would typically lead to obscure bugs.

Setup

To use drstatecmp with your client simply include this line in your client's CMakeLists.txt file:

use_DynamoRIO_extension(clientname drstatecmp)

That will automatically set up the include path and library dependence.

The initialization routine drstatecmp_init() enables the insertion of the machine state checks. When drstatecmp is linked but the client does not invoke drstatecmp_init(), there are no inserted checks and no overhead to the client. Thus, a client can wrap the initialization of drstatecmp with a flag or an ifdef and invoke it only on debug builds. Each call to drstatecmp_init() should be paired with a call to drstatecmp_exit() on client exit.

Checks

There are two types of checks depending on whether a basic block has any side-effects. Only checks for side-effect-free basic blocks are currently supported.

Side-effect-free Basic Blocks

Side-effect-free basic blocks are duplicated and executed twice, once with the full instrumentation sequence and once with no instrumentation except for a state comparison at the end. Essentially, we check whether the machine state at the end of the basic block execution is the same with and without instrumentation code. This check should catch any state clobbering by instrumentation.

Beyond instrumentation bugs, drstatecmp can also catch bugs introduced in the application-to-application phase. To that end, instead of using for re-execution the post-app2app-phase version of each side-effect-free basic block (i.e., instrumentation-free copy but with potentially modified application instructions), it uses the pre-app2app-phase version (contains only the original app instructions), unless any of original app instructions requires emulation (i.e., not executable).

The duplication of side-effect-free basic blocks and the checks insertion occurs at drmgr's post-instrumentation stage. An instrumentation-free copy of each side-effect-free basic block is appended immediately after the instrumented version of the basic block. Before the execution of the instrumented basic block, its machine state is saved. This state will be used for state restoration before the re-execution of the basic block. At the end of the instrumented basic block, code is inserted to save the current state for later comparison and then restore the machine state to the state before executing the instrumented basic block. At the end of the instrumentation-free copy of the basic block, a clean call is inserted that compares the current machine state of the instrumentation-free basic block with the machine state that was saved at the end of the instrumented basic block. A user-supplied callback is invoked in case of any state mismatch. If the user provides no such callback, DR_ASSERT_MSG is triggered on mismatches.

Basic Blocks with Side Effects

Checking basic blocks with side effects is not yet implemented but this section serves as a high-level design overview. Instructions with side effects include instructions that write to memory, interrupts, and system calls.

Basic blocks with side-effects cannot be executed twice (at least not without a lot of extra complexity to monitor and restore memory state). In this case, the checks are inserted throughout the basic block whenever needed taking into account the app instructions, existing instrumentation, and the lazy restoration by drreg. As a result, incomplete-decoder bugs and lazy-condition-drreg bugs will not be detected.

Some of the side effects could be handled in a way that enables re-execution of more basic blocks. For example, system calls could be executed once in the original basic block and then in the basic block copy the system call is just emulated by using the return value seen in the first execution. Other side effects such as storing to memory can be executed only once in cases where the stored value is not loaded within the basic block. Memory could also be handled by saving and restoring the memory state in addition to the machine state. Naturally, monitoring of the written memory addresses is needed to limit the memory state size.

Machine States Saved

During execution, thread-local storage is used to keep two machine states for each side-effect-free basic block. One is the state of the instrumented basic block before its execution (used for state restoration) and the other is the state of the instrumented basic block after its execution (used for state comparison).

Comparison with DrBBDup

The main functionality and most of the complexity of DrStateCmp is the machine state comparison and the instrumentation added to enable it. The basic block duplication is relatively simple and a small part of DrStateCmp's code, and even though there is a bit of redundancy for this part, the use case in DrStateCmp is different enough from what DrBBDup provides so that DrBBDup cannot be used as it is (or with minimum changes). A main difference is that DrBBDup dispatches control according to runtime conditions and only one copy/case is executed every time. Instead, for duplicated basic blocks DrStateCmp executes both the instrumented basic block and its non-instrumented copy.