There's *no* pprof documentation?

<p>Hey All, has anyone got a better lead on this than I do? It seems that the documentation for the pprof tool doesn&#39;t really ... exist. I can&#39;t find info on what its output is supposed to mean, how it works. I can find a lot of blog posts that are little more than glorified &#34;run this command, see, now we&#39;re profiling!&#34;... What gives?</p> <p><strong>edit</strong>: WOW thank you <a href="/r/golang">/r/golang</a>! I really can&#39;t believe how you all came in to help with this, I&#39;m overjoyed. There&#39;s so much detail, so many resources. I feel as though the contents of this thread should be part of some doc somewhere.</p> <hr/>**评论：**<br/><br/>dgryski: <pre><p>The Go <code>pprof</code> command is basically just Google <code>pprof</code> with a few small patches to make it by nicer with Go by default. Documentation is available at <a href="https://github.com/google/pprof/tree/master/doc">https://github.com/google/pprof/tree/master/doc</a> and by typing <code>help</code> at the <code>pprof</code> prompt. You can also see more options with <code>go tool pprof -help</code>.</p></pre>jmoiron: <pre><p>The blog posts I&#39;ve seen cover it in fairly more detail than you&#39;re describing, though I&#39;m not sure I&#39;d call them documentation:</p> <ul> <li><a href="https://blog.golang.org/profiling-go-programs">Profiling Go Programs</a> from the Go Blog</li> <li><a href="https://jvns.ca/blog/2017/09/24/profiling-go-with-pprof/">Profiling Go programs with pprof</a> from Julia Evans</li> <li><a href="https://www.integralist.co.uk/posts/profiling-go/">Profiling Go</a> by Mark McDonnell (also includes expvar/memstats documentation, though actual <a href="https://golang.org/pkg/runtime/#MemStats">source documentation</a> for memstats is excellent and there are <a href="https://golang.org/src/runtime/mstats.go">more goodies</a> in mstats.go) </li> <li><a href="https://artem.krylysov.com/blog/2017/03/13/profiling-and-optimizing-go-web-applications/">Profiling and Optimizing Go Web Applications</a> by Artem Krylysov</li> </ul> <p>Between these a lot of details and use cases are covered. Do you have specific questions not covered by these?</p></pre>trchttrhydrn: <pre><p>Yep, I&#39;d churned through all of those already. And yet I find no description of <em>how to read the output</em>. I ctrl+F&#39;d for &#34;cum&#34; in every single one of those pages and ... no money shot. What does &#34;cum&#34; mean? Where are the docs? </p> <p><a href="https://www.youtube.com/watch?v=gDDguYksysk" rel="nofollow">me rn</a></p> <p>Thanks for taking the time to grab all these links, still...</p> <p>PS - I&#39;m vaguely aware it means &#34;cumulative&#34;</p></pre>jmoiron: <pre><p>There are various profilers and they work via different mechanisms. There is also the tracing framework, which imparts a heavier runtime hit, produces vastly more output, has much more to it and is similarly fairly undocumented.</p> <p>The most commonly used profiles are the CPU and Heap profile, though. I&#39;ve generally obtained these by using <code>net/http/pprof</code>, because I work on mainly daemon processes rather than scripts or batch processes.</p> <p>The CPU profile works by taking a snapshot of the running goroutine stacks periodically and then using that to estimate how much time has been spent in each function. Since stacks have a hierarchy to them (the first function calls the next, calls the next, and so on) you can use this to estimate not only how much time was spent in a particular function (how many times was it at the end of the stack) as well as down what paths it had been called. The first measure is the time spent in a function that is executing, but often you want to now how much cumulative time was spent in particular functions that call many other functions, which is where the cumulative time comes in.</p> <p>The heap profiler is different, and there are a few different modes as well. While the CPU profile samples over a period of time (30s by default), the heap profiler can tell you about various memory allocation statistics. Typically, you want to either know &#34;what allocated the memory currently on the heap?&#34;, which is the default when using http/pprof, or &#34;where has memory been allocated thusfar in the lifetime of this process?&#34;, which can help you identify places where many allocations are made even if they do not live for a long time.</p> <p>The fancy output organizes this information into a hierarchical graph so you can more visually and easily figure out where the expensive paths in your program are. For instance, if you have a function that is being called from multiple places, and you are spending a lot of time in there, you can often see fairly easily on the visual output which callsite is contributing the most to that time spent. The same is true for memory allocations.</p> <p>pprof has a lot of other nice things, like being able to show to-the-line where time is being spent (or mem is being allocated) with <code>list</code> and even show to-the-asm-instruction where time is being sent with <code>disasm</code>. If you have something you want to profile, I suggest you poke around a bit with the online <code>pprof</code> tool and then produce graph output with <code>pdf &gt; out.pdf</code> to see its output on code you are familiar with.</p> <p>This is all covered fairly comprehensively by the official Go blog post:</p> <blockquote> <p>When CPU profiling is enabled, the Go program stops about 100 times per second and records a sample consisting of the program counters on the currently executing goroutine&#39;s stack. [..] In the <code>go tool pprof</code> output, there is a row for each function that appeared in a sample. The first two columns show the number of samples in which the function was running (as opposed to waiting for a called function to return), as a raw count and as a percentage of total samples. [..] The top10 output is sorted by this sample count. The third column shows the running total during the listing [..]. The fourth and fifth columns show the number of samples in which the function appeared (either running or waiting for a called function to return). The main.FindLoops function was running in 10.6% of the samples, but it was on the call stack (it or functions it called were running) in 84.1% of the samples.</p> <p>To sort by the fourth and fifth columns, use the -cum (for cumulative) flag:</p> </blockquote> <p>Given this, you should be more than vaguely aware that <code>-cum</code> stands for cumulative.</p> <p>I suggest reading the official Go blog and the package documentation carefully, sometimes multiple times if you&#39;re feeling like you are missing something. It has a purposefully terse and information dense style; when learning I often feel lost for ages until I finally figured something out at great effort, only to find that the documentation already covered my issue adequately but I kept on skimming over a word or sentence that contained the vital insight I was seeking.</p></pre>trchttrhydrn: <pre><p>THANK YOU, yes, that&#39;s exactly what happened :S</p></pre>LK4D4: <pre><p><a href="http://goog-perftools.sourceforge.net/doc/heap_profiler.html">http://goog-perftools.sourceforge.net/doc/heap_profiler.html</a> Here is docs for C++ pprof, but I believe columns and flags are the same.</p></pre>andrestc: <pre><p>The section on &#34;Dynamic Analysis&#34; has some good content: <a href="https://github.com/campoy/go-tooling-workshop" rel="nofollow">https://github.com/campoy/go-tooling-workshop</a></p></pre>rbetts: <pre><p>See also: <a href="https://golang.org/doc/diagnostics.html" rel="nofollow">https://golang.org/doc/diagnostics.html</a></p></pre>