Trace Format

drmemtrace traces are records of all user mode retired instructions and memory accesses during the traced window.

A trace is presented to analysis tools as a stream of records. Each record entry is of type memref_t and represents one instruction or data reference or a metadata operation such as a thread exit or marker. There are built-in scheduling markers providing the timestamp and cpu identifier on each thread transition. Other built-in markers indicate disruptions in user mode control flow such as signal handler entry and exit.

Each entry contains the common fields type, pid, and tid. The type field is used to identify the kind of each entry via a value of type trace_marker_type_t. The pid and tid identify the process and software thread owning the entry. By default, all traced software threads are interleaved together, but with offline traces (see Offline Traces and Analysis) each thread's trace can easily be analyzed separately as they are stored in separate files.

Instruction Records

Executed instructions are stored in _memref_instr_t. The program counter and length of the encoded instruction are provided. The length can be used to compute the address of the subsequent instruction.

The raw encoding of the instruction is provided. This can be decoded using the drdecode decoder or any other decoder. An additional field encoding_is_new is provided to indicate when any cached decoding information should be invalidated due to possibly changed application code. (For online traces, encodings are not provided unless the option -instr_encodings is passed, as encodings add overhead and are not needed for many tools.)

Older legacy traces may not contain instruction encodings. For those traces, encodings for static code can be obtained by disassembling the application and library binaries. The provided interfaces module_mapper_t::get_loaded_modules() and module_mapper_t::find_mapped_trace_address() facilitate loading in copies of the binaries and reading the raw bytes for each instruction in order to obtain the opcode and full operand information. See also Core Simulation Support.

Branch targets are also not explicitly recorded (a design decision). The executed target as well as branch direction can be obtained by examining the address of the instruction immediately following a branch. If the program flow is changed by the kernel such as by signal delivery, the branch target is explicitly recorded in the trace in a metadata marker entry of type TRACE_MARKER_TYPE_KERNEL_EVENT.

Memory Access Records

Memory accesses (data loads and stores) are stored in _memref_data_t. The program counter of the instruction performing the memory access, the virtual address (convertable to physical: see Physical Addresses), and the size are provided.

Other Records

Besides instruction and memory records, other trace entry types include _memref_marker_t, _memref_flush_t, _memref_thread_exit_t, etc. These records provide specific inforamtion about events that can alter the program flow or the system's states.

Trace markers are particularly important to allow reconstruction of the program execution. Marker records in _memref_marker_t provide metadata identifying some event that occurred at this point in the trace. Each marker record contains two additional fields:

  • marker_type - identifies the type of marker
  • marker_value - carries the value of the marker

Some of the more important markers are:

  • TRACE_MARKER_TYPE_KERNEL_EVENT - This identifies kernel-initiated control transfers such as signal delivery. The next instruction record is the start of the handler for a kernel-initiated event. The value of this type of marker contains the program counter at the kernel interruption point. If the interruption point is just after a branch, this value is the target of that branch.
  • TRACE_MARKER_TYPE_KERNEL_XFER - This identifies a system call that changes control flow, such as a signal return.
  • TRACE_MARKER_TYPE_TIMESTAMP - The marker value provides a timestamp for this point of the trace (in units of microseconds since Jan 1, 1601 UTC). This value can be used to synchronize records from different threads. It is used in the sequential analysis of a multi-threaded trace.
  • TRACE_MARKER_TYPE_CPU_ID - The marker value contains the CPU identifier on which the subsequent records were collected. It is useful to help track thread migrations occurring during execution. This marker is written to the header of each trace buffer when the buffer is flushed. Note that if the thread migrates to a different CPU due to preemption by the kernel before a buffer is full, we do not output a separate TRACE_MARKER_TYPE_CPU_ID marker to capture the previous CPU identifier. However, we expect such cases to be rare.
  • TRACE_MARKER_TYPE_FUNC_ID, TRACE_MARKER_TYPE_FUNC_RETADDR, TRACE_MARKER_TYPE_FUNC_ARG, TRACE_MARKER_TYPE_FUNC_RETVAL - These markers are used to capture information about function calls. Which functions to capture must be explicitly selected at tracing time. Typical candiates are heap allocation and freeing functions. See Tracing Function Calls.
  • TRACE_MARKER_TYPE_WINDOW_ID - The marker value contains the ordinal of the trace burst window when the subsequent entries until the next TRACE_MARKER_TYPE_WINDOW_ID or end-of-trace were collected (see Tracing a Subset of Execution).

The full set of markers is listed under the enum trace_marker_type_t.