Senior Engineer Rickey C. Weisner has written a tutorial on DTrace to illustrate how the diagnostic tool can be used to help analyze real-world applications. With an emphasis on practical use, Weisner employs DTrace to analyze several applications and uses mpstat and prstat information to show which questions one should ask and how one can use DTrace to answer these questions.
In Weisner's introduction to DTrace, he reviews how to disassemble a kernel function and initiate a probe. He then moves on to examples, initially explaining where readers can get more information on the ones in Solaris OS from the DTrace user guide found in /usr/demo/dtrace.
Also referenced is Fishworks Engineer Brendan Gregg's toolkit, which contains numerous d-scripts as well as perl scripts that preprocess and postprocess DTrace. Highlighted is the example directory, containing output from each of the available programs.
Next, Weisner describes the mpstat program, which is one of the programs that frequently illustrates which questions to ask. The mpstat program presents a high-level view of kernel activity. He proceeds to prstat - another tool that provides useful first-level analysis metrics, providing similar information as mpstat but at a thread level.
Specific examples used in the tutorial cover memory management, condition variables, memset, FIFO and user error. Weisner also offers that dtrace -l lists all available probes, which can be useful in various situations.
"DTrace does not replace any of the existing performance measuring tools that a practitioner typically employs to characterize performance. Where once the characterization led to experiments, conjectures, static instrumentation of code, and so on, DTrace nows allows the analyst to directly measure in live production environments, and thus to avoid experiments, static instrumentation, and hidden assumptions," Weisner writes. "With DTrace we can get to the root cause quickly, directly, and without ambiguity."
More Information
Tutorial: DTrace by Example - Weisner's tutorial
DTrace Documentation
DTrace Toolkit - by Brendan Gregg
[...read more...]