|
DynamoRIO
|
#include <analysis_tool.h>
Data Structures | |
| struct | interval_state_snapshot_t |
Public Member Functions | |
| analysis_tool_tmpl_t () | |
| virtual std::string | initialize () |
| virtual std::string | initialize_stream (memtrace_stream_t *serial_stream) |
| virtual std::string | initialize_shard_type (shard_type_t shard_type) |
| virtual bool | operator! () |
| virtual std::string | get_error_string () |
| virtual bool | process_memref (const RecordType &entry)=0 |
| virtual bool | print_results ()=0 |
| virtual interval_state_snapshot_t * | generate_interval_snapshot (uint64_t interval_id) |
| virtual interval_state_snapshot_t * | combine_interval_snapshots (const std::vector< const interval_state_snapshot_t * > latest_shard_snapshots, uint64_t interval_end_timestamp) |
| virtual bool | print_interval_results (const std::vector< interval_state_snapshot_t * > &interval_snapshots) |
| virtual bool | release_interval_snapshot (interval_state_snapshot_t *interval_snapshot) |
| virtual bool | parallel_shard_supported () |
| virtual void * | parallel_worker_init (int worker_index) |
| virtual std::string | parallel_worker_exit (void *worker_data) |
| virtual void * | parallel_shard_init (int shard_index, void *worker_data) |
| virtual void * | parallel_shard_init_stream (int shard_index, void *worker_data, memtrace_stream_t *shard_stream) |
| virtual bool | parallel_shard_exit (void *shard_data) |
| virtual bool | parallel_shard_memref (void *shard_data, const RecordType &entry) |
| virtual std::string | parallel_shard_error (void *shard_data) |
| virtual interval_state_snapshot_t * | generate_shard_interval_snapshot (void *shard_data, uint64_t interval_id) |
| analysis_tool_tmpl_t () | |
| virtual std::string | initialize () |
| virtual std::string | initialize_stream (memtrace_stream_t *serial_stream) |
| virtual std::string | initialize_shard_type (shard_type_t shard_type) |
| virtual bool | operator! () |
| virtual std::string | get_error_string () |
| virtual bool | process_memref (const RecordType &entry)=0 |
| virtual bool | print_results ()=0 |
| virtual interval_state_snapshot_t * | generate_interval_snapshot (uint64_t interval_id) |
| virtual interval_state_snapshot_t * | combine_interval_snapshots (const std::vector< const interval_state_snapshot_t * > latest_shard_snapshots, uint64_t interval_end_timestamp) |
| virtual bool | print_interval_results (const std::vector< interval_state_snapshot_t * > &interval_snapshots) |
| virtual bool | release_interval_snapshot (interval_state_snapshot_t *interval_snapshot) |
| virtual bool | parallel_shard_supported () |
| virtual void * | parallel_worker_init (int worker_index) |
| virtual std::string | parallel_worker_exit (void *worker_data) |
| virtual void * | parallel_shard_init (int shard_index, void *worker_data) |
| virtual void * | parallel_shard_init_stream (int shard_index, void *worker_data, memtrace_stream_t *shard_stream) |
| virtual bool | parallel_shard_exit (void *shard_data) |
| virtual bool | parallel_shard_memref (void *shard_data, const RecordType &entry) |
| virtual std::string | parallel_shard_error (void *shard_data) |
| virtual interval_state_snapshot_t * | generate_shard_interval_snapshot (void *shard_data, uint64_t interval_id) |
Detailed Description
template<typename RecordType>
class dynamorio::drmemtrace::analysis_tool_tmpl_t< RecordType >
The base class for a tool that analyzes a trace. A new tool should subclass this class.
Concurrent processing of traces is supported by logically splitting a trace into "shards" which are each processed sequentially. The default shard is a traced application thread, but the tool interface can support other divisions. For tools that support concurrent processing of shards and do not need to see a single thread-interleaved merged trace, the interface functions with the parallel_ prefix should be implemented, and parallel_shard_supported() should return true. parallel_shard_init_stream() will be invoked for each shard prior to invoking parallel_shard_memref() for any entry in that shard; the data structure returned from parallel_shard_init_stream() will be passed to parallel_shard_memref() for each trace entry for that shard. The concurrency model used guarantees that all entries from any one shard are processed by the same single worker thread, so no synchronization is needed inside the parallel_ functions. A single worker thread invokes print_results() as well.
For serial operation, process_memref(), operates on a trace entry in a single, sorted, interleaved stream of trace entries. In the default mode of operation, the analyzer_t class iterates over the trace and calls the process_memref() function of each tool. An alternative mode is supported which exposes the iterator and allows a separate control infrastructure to be built. This alternative mode does not support parallel operation at this time.
Both parallel and serial operation can be supported by a tool, typically by having process_memref() create data on a newly seen traced thread and invoking parallel_shard_memref() to do its work.
For both parallel and serial operation, the function print_results() should be overridden. It is called just once after processing all trace data and it should present the results of the analysis. For parallel operation, any desired aggregation across the whole trace should occur here as well, while shard-specific results can be presented in parallel_shard_exit().
RecordType is the type of entry that is analyzed by the tool. Currently, we support memref_t and trace_entry_t. analysis_tool_tmpl_t<memref_t> is the primary type of analysis tool and is used for most purposes. analysis_tool_tmpl_t≤::trace_entry_t> is used in special cases where an offline trace needs to be observed exactly as stored on disk, without hiding any internal entries.
Analysis tools can subclass either of the specialized analysis_tool_tmpl_t<memref_t> or analysis_tool_tmpl_t<trace_entry_t>, or analysis_tool_tmpl_t itself, as required.
Constructor & Destructor Documentation
◆ analysis_tool_tmpl_t() [1/2]
|
inline |
Errors encountered during the constructor will set the success flag, which should be queried via operator!. On an error, get_error_string() provides a descriptive error message.
◆ analysis_tool_tmpl_t() [2/2]
|
inline |
Errors encountered during the constructor will set the success flag, which should be queried via operator!. On an error, get_error_string() provides a descriptive error message.
Member Function Documentation
◆ combine_interval_snapshots() [1/2]
|
inlinevirtual |
Invoked by the framework to combine the shard-local interval_state_snapshot_t objects pointed at by latest_shard_snapshots, to create the combined interval_state_snapshot_t for the whole-trace interval that ends at interval_end_timestamp. This is useful in the parallel mode of the analyzer, where each trace shard is processed in parallel. In this mode, the framework creates the interval_state_snapshot_t for each whole-trace interval, one by one. For each interval, it invokes combine_interval_snapshots() with the latest seen interval snapshots from each shard (in latest_shard_snapshots) as of the interval ending at interval_end_timestamp. latest_shard_snapshots will contain at least one non-null element, and at least one of them will have its interval_state_snapshot_t::interval_end_timestamp set to interval_end_timestamp. Tools must not return a nullptr. Tools must not modify the provided snapshots.
interval_end_timestamp helps the tool to determine which of the provided latest_shard_snapshots are relevant to it.
- for cumulative metrics (e.g. total instructions till an interval), the tool may want to combine latest interval snapshots from all shards to get the accurate cumulative count.
- for delta metrics (e.g. incremental instructions in an interval), the tool may want to combine interval snapshots from only the shards that were observed in the current interval (with
latest_shard_snapshots[i]->interval_end_timestamp ==interval_end_timestamp) - or if the tool mixes cumulative and delta metrics: some field-specific logic that combines the above two strategies.
◆ combine_interval_snapshots() [2/2]
|
inlinevirtual |
Invoked by the framework to combine the shard-local interval_state_snapshot_t objects pointed at by latest_shard_snapshots, to create the combined interval_state_snapshot_t for the whole-trace interval that ends at interval_end_timestamp. This is useful in the parallel mode of the analyzer, where each trace shard is processed in parallel. In this mode, the framework creates the interval_state_snapshot_t for each whole-trace interval, one by one. For each interval, it invokes combine_interval_snapshots() with the latest seen interval snapshots from each shard (in latest_shard_snapshots) as of the interval ending at interval_end_timestamp. latest_shard_snapshots will contain at least one non-null element, and at least one of them will have its interval_state_snapshot_t::interval_end_timestamp set to interval_end_timestamp. Tools must not return a nullptr. Tools must not modify the provided snapshots.
interval_end_timestamp helps the tool to determine which of the provided latest_shard_snapshots are relevant to it.
- for cumulative metrics (e.g. total instructions till an interval), the tool may want to combine latest interval snapshots from all shards to get the accurate cumulative count.
- for delta metrics (e.g. incremental instructions in an interval), the tool may want to combine interval snapshots from only the shards that were observed in the current interval (with
latest_shard_snapshots[i]->interval_end_timestamp ==interval_end_timestamp) - or if the tool mixes cumulative and delta metrics: some field-specific logic that combines the above two strategies.
◆ generate_interval_snapshot() [1/2]
|
inlinevirtual |
Notifies the analysis tool that the given trace interval_id has ended so that it can generate a snapshot of its internal state in a struct derived from interval_state_snapshot_t, and return a pointer to it. The returned pointer will be provided to the tool in later combine_interval_snapshots() and print_interval_result() calls.
interval_id is a positive ordinal of the trace interval that just ended. Trace intervals have a length equal to the -interval_microseconds specified to the framework. Trace intervals are measured using the value of the TRACE_MARKER_TYPE_TIMESTAMP markers. The provided interval_id values will be monotonically increasing but may not be continuous, i.e. the tool may not see some interval_id if the trace did not have any activity in that interval.
The returned interval_state_snapshot_t* will be passed to the combine_interval_snapshots() API which is invoked by the framework to merge multiple interval_state_snapshot_t from different shards in the parallel mode of the analyzer.
Finally, the print_interval_result() API is invoked with a list of interval_state_snapshot_t* representing interval snapshots for the whole trace.
The tool must not de-allocate the state snapshot until release_interval_snapshot() is invoked by the framework.
An example use case of this API is to create a time series of some output metric over the whole trace.
◆ generate_interval_snapshot() [2/2]
|
inlinevirtual |
Notifies the analysis tool that the given trace interval_id has ended so that it can generate a snapshot of its internal state in a struct derived from interval_state_snapshot_t, and return a pointer to it. The returned pointer will be provided to the tool in later combine_interval_snapshots() and print_interval_result() calls.
interval_id is a positive ordinal of the trace interval that just ended. Trace intervals have a length equal to the -interval_microseconds specified to the framework. Trace intervals are measured using the value of the TRACE_MARKER_TYPE_TIMESTAMP markers. The provided interval_id values will be monotonically increasing but may not be continuous, i.e. the tool may not see some interval_id if the trace did not have any activity in that interval.
The returned interval_state_snapshot_t* will be passed to the combine_interval_snapshots() API which is invoked by the framework to merge multiple interval_state_snapshot_t from different shards in the parallel mode of the analyzer.
Finally, the print_interval_result() API is invoked with a list of interval_state_snapshot_t* representing interval snapshots for the whole trace.
The tool must not de-allocate the state snapshot until release_interval_snapshot() is invoked by the framework.
An example use case of this API is to create a time series of some output metric over the whole trace.
◆ generate_shard_interval_snapshot() [1/2]
|
inlinevirtual |
Notifies the analysis tool that the given trace interval_id in the shard represented by the given shard_data has ended, so that it can generate a snapshot of its internal state in a struct derived from interval_state_snapshot_t, and return a pointer to it. The returned pointer will be provided to the tool in later combine_interval_snapshots() and print_interval_result() calls.
Note that the provided interval_id is local to the shard that is represented by the given shard_data, and not the whole-trace interval. The framework will automatically create an interval_state_snapshot_t for each whole-trace interval by using combine_interval_snapshots() to combine one or more shard-local interval_state_snapshot_t corresponding to that whole-trace interval.
interval_id is a positive ordinal of the trace interval that just ended. Trace intervals have a length equal to the -interval_microseconds specified to the framework. Trace intervals are measured using the value of the TRACE_MARKER_TYPE_TIMESTAMP markers. The provided interval_id values will be monotonically increasing but may not be continuous, i.e. the tool may not see some interval_id if the trace shard did not have any activity in that interval.
The returned interval_state_snapshot_t* will be passed to the combine_interval_snapshot() API which is invoked by the framework to merge multiple interval_state_snapshot_t from different shards in the parallel mode of the analyzer.
Finally, the print_interval_result() API is invoked with a list of interval_state_snapshot_t* representing interval snapshots for the whole trace. In the parallel mode of the analyzer, this list is computed by combining the shard-local interval_state_snapshot_t using the tool's combine_interval_snapshot() API.
The tool must not de-allocate the state snapshot until release_interval_snapshot() is invoked by the framework.
An example use case of this API is to create a time series of some output metric over the whole trace.
◆ generate_shard_interval_snapshot() [2/2]
|
inlinevirtual |
Notifies the analysis tool that the given trace interval_id in the shard represented by the given shard_data has ended, so that it can generate a snapshot of its internal state in a struct derived from interval_state_snapshot_t, and return a pointer to it. The returned pointer will be provided to the tool in later combine_interval_snapshots() and print_interval_result() calls.
Note that the provided interval_id is local to the shard that is represented by the given shard_data, and not the whole-trace interval. The framework will automatically create an interval_state_snapshot_t for each whole-trace interval by using combine_interval_snapshots() to combine one or more shard-local interval_state_snapshot_t corresponding to that whole-trace interval.
interval_id is a positive ordinal of the trace interval that just ended. Trace intervals have a length equal to the -interval_microseconds specified to the framework. Trace intervals are measured using the value of the TRACE_MARKER_TYPE_TIMESTAMP markers. The provided interval_id values will be monotonically increasing but may not be continuous, i.e. the tool may not see some interval_id if the trace shard did not have any activity in that interval.
The returned interval_state_snapshot_t* will be passed to the combine_interval_snapshot() API which is invoked by the framework to merge multiple interval_state_snapshot_t from different shards in the parallel mode of the analyzer.
Finally, the print_interval_result() API is invoked with a list of interval_state_snapshot_t* representing interval snapshots for the whole trace. In the parallel mode of the analyzer, this list is computed by combining the shard-local interval_state_snapshot_t using the tool's combine_interval_snapshot() API.
The tool must not de-allocate the state snapshot until release_interval_snapshot() is invoked by the framework.
An example use case of this API is to create a time series of some output metric over the whole trace.
◆ get_error_string() [1/2]
|
inlinevirtual |
Returns a description of the last error.
◆ get_error_string() [2/2]
|
inlinevirtual |
Returns a description of the last error.
◆ initialize() [1/2]
|
inlinevirtual |
Destructor.
- Deprecated:
- The initialize_stream() function is called by the analyzer; this function is only called if the default implementation of initialize_stream() is left in place and it calls this version. On an error, this returns an error string. On success, it returns "".
◆ initialize() [2/2]
|
inlinevirtual |
Destructor.
- Deprecated:
- The initialize_stream() function is called by the analyzer; this function is only called if the default implementation of initialize_stream() is left in place and it calls this version. On an error, this returns an error string. On success, it returns "".
◆ initialize_shard_type() [1/2]
|
inlinevirtual |
Identifies the shard type for this analysis. The return value indicates whether the tool supports this shard type, with failure (a non-empty string) being treated as a fatal error for the analysis.
◆ initialize_shard_type() [2/2]
|
inlinevirtual |
Identifies the shard type for this analysis. The return value indicates whether the tool supports this shard type, with failure (a non-empty string) being treated as a fatal error for the analysis.
◆ initialize_stream() [1/2]
|
inlinevirtual |
Tools are encouraged to perform any initialization that might fail here rather than in the constructor. The serial_stream interface allows tools to query details of the underlying trace during serial operation; it is nullptr for parallel operation (a per-shard version is passed to parallel_shard_init_stream()). On an error, this returns an error string. On success, it returns "".
◆ initialize_stream() [2/2]
|
inlinevirtual |
Tools are encouraged to perform any initialization that might fail here rather than in the constructor. The serial_stream interface allows tools to query details of the underlying trace during serial operation; it is nullptr for parallel operation (a per-shard version is passed to parallel_shard_init_stream()). On an error, this returns an error string. On success, it returns "".
◆ operator!() [1/2]
|
inlinevirtual |
Returns whether the tool was created successfully.
◆ operator!() [2/2]
|
inlinevirtual |
Returns whether the tool was created successfully.
◆ parallel_shard_error() [1/2]
|
inlinevirtual |
Returns a description of the last error for this shard.
◆ parallel_shard_error() [2/2]
|
inlinevirtual |
Returns a description of the last error for this shard.
◆ parallel_shard_exit() [1/2]
|
inlinevirtual |
Invoked once when all trace entries for a shard have been processed. shard_data is the value returned by parallel_shard_init_stream() for this shard. This allows a tool to clean up its thread data, or to report thread analysis results. Most tools, however, prefer to aggregate data or at least sort data, and perform nothing here, doing all cleanup in print_results() by storing the thread data into a table. Return whether exiting was successful. On failure, parallel_shard_error() returns a descriptive message.
◆ parallel_shard_exit() [2/2]
|
inlinevirtual |
Invoked once when all trace entries for a shard have been processed. shard_data is the value returned by parallel_shard_init_stream() for this shard. This allows a tool to clean up its thread data, or to report thread analysis results. Most tools, however, prefer to aggregate data or at least sort data, and perform nothing here, doing all cleanup in print_results() by storing the thread data into a table. Return whether exiting was successful. On failure, parallel_shard_error() returns a descriptive message.
◆ parallel_shard_init() [1/2]
|
inlinevirtual |
- Deprecated:
- The parallel_shard_init_stream() is what is called by the analyzer; this function is only called if the default implementation of parallel_shard_init_stream() is left in place and it calls this version.
◆ parallel_shard_init() [2/2]
|
inlinevirtual |
- Deprecated:
- The parallel_shard_init_stream() is what is called by the analyzer; this function is only called if the default implementation of parallel_shard_init_stream() is left in place and it calls this version.
◆ parallel_shard_init_stream() [1/2]
|
inlinevirtual |
Invoked once for each trace shard prior to calling parallel_shard_memref() for that shard, this allows a tool to create data local to a shard. The shard_index is a unique identifier allowing shard data to be stored into a global table if desired (typically for aggregation use in print_results()). The worker_data is the return value of parallel_worker_init() for the worker thread who will exclusively operate on this shard. The shard_stream allows tools to query details of the underlying trace shard during parallel operation; it is valid only until parallel_shard_exit() is called. The return value here will be passed to each invocation of parallel_shard_memref() for that same shard.
◆ parallel_shard_init_stream() [2/2]
|
inlinevirtual |
Invoked once for each trace shard prior to calling parallel_shard_memref() for that shard, this allows a tool to create data local to a shard. The shard_index is a unique identifier allowing shard data to be stored into a global table if desired (typically for aggregation use in print_results()). The worker_data is the return value of parallel_worker_init() for the worker thread who will exclusively operate on this shard. The shard_stream allows tools to query details of the underlying trace shard during parallel operation; it is valid only until parallel_shard_exit() is called. The return value here will be passed to each invocation of parallel_shard_memref() for that same shard.
◆ parallel_shard_memref() [1/2]
|
inlinevirtual |
The heart of an analysis tool, this routine operates on a single trace entry and takes whatever actions the tool needs to perform its analysis. The shard_data parameter is the value returned by parallel_shard_init_stream() for this shard. Since each shard is operated upon in its entirety by the same worker thread, no synchronization is needed. The return value indicates whether this function was successful. On failure, parallel_shard_error() returns a descriptive message.
◆ parallel_shard_memref() [2/2]
|
inlinevirtual |
The heart of an analysis tool, this routine operates on a single trace entry and takes whatever actions the tool needs to perform its analysis. The shard_data parameter is the value returned by parallel_shard_init_stream() for this shard. Since each shard is operated upon in its entirety by the same worker thread, no synchronization is needed. The return value indicates whether this function was successful. On failure, parallel_shard_error() returns a descriptive message.
◆ parallel_shard_supported() [1/2]
|
inlinevirtual |
Returns whether this tool supports analyzing trace shards concurrently, or whether it needs to see a single thread-interleaved stream of traced events. This may be called prior to initialize().
◆ parallel_shard_supported() [2/2]
|
inlinevirtual |
Returns whether this tool supports analyzing trace shards concurrently, or whether it needs to see a single thread-interleaved stream of traced events. This may be called prior to initialize().
◆ parallel_worker_exit() [1/2]
|
inlinevirtual |
Invoked once when a worker thread is finished processing all shard data. The worker_data is the return value of parallel_worker_init(). A non-empty string indicates an error while an empty string indicates success.
◆ parallel_worker_exit() [2/2]
|
inlinevirtual |
Invoked once when a worker thread is finished processing all shard data. The worker_data is the return value of parallel_worker_init(). A non-empty string indicates an error while an empty string indicates success.
◆ parallel_worker_init() [1/2]
|
inlinevirtual |
Invoked once for each worker thread prior to calling any shard routine from that thread. This allows a tool to create data local to a worker, such as a cache of data global to the trace that crosses shards. This data does not need any synchronization as it will only be accessed by this worker thread. The worker_index is a unique identifier for this worker. The return value here will be passed to the invocation of parallel_shard_init_stream() for each shard upon which this worker operates.
◆ parallel_worker_init() [2/2]
|
inlinevirtual |
Invoked once for each worker thread prior to calling any shard routine from that thread. This allows a tool to create data local to a worker, such as a cache of data global to the trace that crosses shards. This data does not need any synchronization as it will only be accessed by this worker thread. The worker_index is a unique identifier for this worker. The return value here will be passed to the invocation of parallel_shard_init_stream() for each shard upon which this worker operates.
◆ print_interval_results() [1/2]
|
inlinevirtual |
Prints the interval results for the given series of interval state snapshots in interval_snapshots.
This is currently invoked with the list of whole-trace interval snapshots (for the parallel mode, these are the snapshots created by merging the shard-local snapshots).
The framework should be able to invoke this multiple times, possibly with a different list of interval snapshots. So it should avoid free-ing memory or changing global state. This is to keep open the possibility of the framework printing interval results for each shard separately in future.
◆ print_interval_results() [2/2]
|
inlinevirtual |
Prints the interval results for the given series of interval state snapshots in interval_snapshots.
This is currently invoked with the list of whole-trace interval snapshots (for the parallel mode, these are the snapshots created by merging the shard-local snapshots).
The framework should be able to invoke this multiple times, possibly with a different list of interval snapshots. So it should avoid free-ing memory or changing global state. This is to keep open the possibility of the framework printing interval results for each shard separately in future.
◆ print_results() [1/2]
|
pure virtual |
This routine reports the results of the trace analysis. It should leave the i/o state in a default format (std::dec) to support multiple tools. The return value indicates whether it was successful. On failure, get_error_string() returns a descriptive message.
◆ print_results() [2/2]
|
pure virtual |
This routine reports the results of the trace analysis. It should leave the i/o state in a default format (std::dec) to support multiple tools. The return value indicates whether it was successful. On failure, get_error_string() returns a descriptive message.
◆ process_memref() [1/2]
|
pure virtual |
The heart of an analysis tool, this routine operates on a single trace entry and takes whatever actions the tool needs to perform its analysis. If it prints, it should leave the i/o state in a default format (std::dec) to support multiple tools. The return value indicates whether it was successful. On failure, get_error_string() returns a descriptive message.
◆ process_memref() [2/2]
|
pure virtual |
The heart of an analysis tool, this routine operates on a single trace entry and takes whatever actions the tool needs to perform its analysis. If it prints, it should leave the i/o state in a default format (std::dec) to support multiple tools. The return value indicates whether it was successful. On failure, get_error_string() returns a descriptive message.
◆ release_interval_snapshot() [1/2]
|
inlinevirtual |
Notifies the tool that the interval_state_snapshot_t object pointed to by interval_snapshot is no longer needed by the framework. The tool may de-allocate it right away or later, as it needs. Returns whether it was successful.
◆ release_interval_snapshot() [2/2]
|
inlinevirtual |
Notifies the tool that the interval_state_snapshot_t object pointed to by interval_snapshot is no longer needed by the framework. The tool may de-allocate it right away or later, as it needs. Returns whether it was successful.
The documentation for this class was generated from the following file:
- /home/runner/work/dynamorio/dynamorio/build_release-64/clients/include/drmemtrace/analysis_tool.h
DynamoRIO version 10.0.19627 --- Fri Sep 29 2023 03:13:26