这两天用golang在写一个监控的agent，发现长时间运行后会有内存泄露的情况，着实让人郁闷半天… 要解决golang泄露的问题，要理解goalng gc那是事情，再就是利用pprof监视golang的运行环境。
该文章写的有些乱，欢迎来喷 ! 另外文章后续不断更新中，请到原文地址查看更新。http://xiaorui.cc/?p=3000
这Golang GC垃圾回收我就先不多说了，等我自己深入了解了，会专门找个时间聊这事.
在golang中用来做监控分析的库包，一般用都是pprof库包… pprof可以在两个地方引入：
|
1 2 3 4 |
net/http/pprof runtime/pprof |
其实net/http/pprof中只是使用runtime/pprof包来进行封装了一下，并在http端口上暴露出来。 runtime/pprof可以用来产生dump文件，再使用go tool pprof来分析这运行日志.
使用net/http/pprof可以做到直接看到当前web服务的状态，包括CPU占用情况和内存使用情况等。
这次重点说些pprof web显示的模式，我自己主要是关注heap,profile两个数据源。
关于golang运行环境heap的信息、内存mem等
|
1 2 |
http://localhost:7777/debug/pprof/heap |
关于profile、CPU计算百分比等
|
1 2 |
http://localhost:7777/debug/pprof/profile |
上面简单介绍了pprof的监控接口，但怎么查看这些数据？ 有这么几个查看方式.
交互模式, 可以通过用help查看pprof各种命令。 每次运行后拿到的数据是固定的，不会动态更新该数据。
|
1 2 3 |
go tool pprof http://localhost:7777/debug/pprof/heap go tool pprof http://localhost:7777/debug/pprof/profile |
如果你不想使用交互模式，当然这每次都是新数据：
|
1 2 |
go tool pprof --text http://localhost:7777/debug/pprof/heap |
网页查看模式:
http://localhost:7777/debug/pprof/
下面是我随便写的一段伪业务代码. 大家可以长时间运行这段代码，会发现内存在缓慢的增长中.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
#xiaorui.cc package main import ( "flag" "fmt" "io/ioutil" "log" "net/http" _ "net/http/pprof" "sync" "time" ) func counter() { list := []int{1} c := 1 for i := 0; i < 10000000; i++ { httpGet() c = i + 1 + 2 + 3 + 4 - 5 list = append(list, c) } fmt.Println(c) fmt.Println(list[0]) } func work(wg *sync.WaitGroup) { for { counter() time.Sleep(1 * time.Second) } wg.Done() } func httpGet() int { queue := []string{"start..."} resp, err := http.Get("http://www.163.com") if err != nil { // handle error } //defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { // handle error } queue = append(queue, string(body)) return len(queue) } func main() { flag.Parse() //这里实现了远程获取pprof数据的接口 go func() { log.Println(http.ListenAndServe("localhost:7777", nil)) }() var wg sync.WaitGroup wg.Add(10) for i := 0; i < 100; i++ { go work(&wg) } wg.Wait() time.Sleep(3 * time.Second) } |
下面是pprof统计的heaq信息
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
#xiaorui.cc $ go tool pprof --text http://localhost:7777/debug/pprof/heap Fetching profile from http://localhost:7777/debug/pprof/heap Saved profile in /Users/ruifengyun/pprof/pprof.localhost:7777.inuse_objects.inuse_space.024.pb.gz 60.81MB of 60.81MB total ( 100%) Dropped 62 nodes (cum <= 0.30MB) flat flat% sum% cum cum% 56.23MB 92.46% 92.46% 56.23MB 92.46% bytes.makeSlice 3.08MB 5.07% 97.53% 3.08MB 5.07% compress/flate.NewReader 0.50MB 0.83% 98.36% 0.50MB 0.83% net/http.(*Transport).dialConn 0.50MB 0.82% 99.18% 0.50MB 0.82% net/http.readTransfer 0.50MB 0.82% 100% 0.50MB 0.82% net/http.NewRequest 0 0% 100% 59.31MB 97.53% bytes.(*Buffer).ReadFrom 0 0% 100% 3.08MB 5.07% compress/gzip.(*Reader).readHeader 0 0% 100% 3.08MB 5.07% compress/gzip.NewReader 0 0% 100% 59.31MB 97.53% io/ioutil.ReadAll 0 0% 100% 59.31MB 97.53% io/ioutil.readAll 0 0% 100% 59.81MB 98.35% main.counter 0 0% 100% 59.81MB 98.35% main.httpGet 0 0% 100% 59.81MB 98.35% main.work 0 0% 100% 0.50MB 0.82% net/http.(*Client).Get 0 0% 100% 0.50MB 0.83% net/http.(*Transport).getConn.func4 0 0% 100% 3.08MB 5.07% net/http.(*bodyEOFSignal).Read 0 0% 100% 3.08MB 5.07% net/http.(*gzipReader).Read 0 0% 100% 0.50MB 0.82% net/http.(*persistConn).readLoop 0 0% 100% 0.50MB 0.82% net/http.(*persistConn).readResponse 0 0% 100% 0.50MB 0.82% net/http.Get 0 0% 100% 0.50MB 0.82% net/http.ReadResponse 0 0% 100% 60.81MB 100% runtime.goexit |
另外需要说的一点，pprof可以生成一个svg的矢量图，可以通过这svg图确认代码的流程及调用情况. svg是使用graphviz生成的，mac下直接brew install graphviz就能安装，centos下，yum -y install graphviz .
下面是pprof help的使用文档.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
#xiaorui.cc $ go tool pprof http://localhost:6060/debug/pprof/heap Fetching profile from http://localhost:6060/debug/pprof/heap Saved profile in /Users/ruifengyun/pprof/pprof.localhost:6060.inuse_objects.inuse_space.032.pb.gz Entering interactive mode (type "help" for commands) (pprof) help Commands: cmd [n] [--cum] [focus_regex]* [-ignore_regex]* Produce a text report with the top n entries. Include samples matching focus_regex, and exclude ignore_regex. Add --cum to sort using cumulative data. Available commands: callgrind Outputs a graph in callgrind format disasm Output annotated assembly for functions matching regexp or address dot Outputs a graph in DOT format eog Visualize graph through eog evince Visualize graph through evince gif Outputs a graph image in GIF format gv Visualize graph through gv list Output annotated source for functions matching regexp pdf Outputs a graph in PDF format peek Output callers/callees of functions matching regexp png Outputs a graph image in PNG format proto Outputs the profile in compressed protobuf format ps Outputs a graph in PS format raw Outputs a text representation of the raw profile svg Outputs a graph in SVG format tags Outputs all tags in the profile text Outputs top entries in text form top Outputs top entries in text form tree Outputs a text rendering of call graph web Visualize graph through web browser weblist Output annotated source in HTML for functions matching regexp or address peek func_regex Display callers and callees of functions matching func_regex. dot [n] [focus_regex]* [-ignore_regex]* [>file] Produce an annotated callgraph with the top n entries. Include samples matching focus_regex, and exclude ignore_regex. For other outputs, replace dot with: - Graphic formats: dot, svg, pdf, ps, gif, png (use > to name output file) - Graph viewer: gv, web, evince, eog callgrind [n] [focus_regex]* [-ignore_regex]* [>file] Produce a file in callgrind-compatible format. Include samples matching focus_regex, and exclude ignore_regex. weblist func_regex [-ignore_regex]* Show annotated source with interspersed assembly in a web browser. list func_regex [-ignore_regex]* Print source for routines matching func_regex, and exclude ignore_regex. disasm func_regex [-ignore_regex]* Disassemble routines matching func_regex, and exclude ignore_regex. tags tag_regex [-ignore_regex]* List tags with key:value matching tag_regex and exclude ignore_regex. quit/exit/^D Exit pprof. option=value The following options can be set individually: cum/flat: Sort entries based on cumulative or flat data call_tree: Build context-sensitive call trees nodecount: Max number of entries to display nodefraction: Min frequency ratio of nodes to display edgefraction: Min frequency ratio of edges to display focus/ignore: Regexp to include/exclude samples by name/file tagfocus/tagignore: Regexp or value range to filter samples by tag eg "1mb", "1mb:2mb", ":64kb" functions: Level of aggregation for sample data files: lines: addresses: unit: Measurement unit to use on reports Sample value selection by index: sample_index: Index of sample value to display mean: Average sample value over first value Sample value selection by name: alloc_space for heap profiles alloc_objects inuse_space inuse_objects total_delay for contention profiles mean_delay contentions : Clear focus/ignore/hide/tagfocus/tagignore (pprof) |
根据pprof的统计信息我们可以找到CPU过度计算及内存泄露的大概的点。 现在越来越觉得Golang gc有些让人摸不清头脑. 看来有必要深入学习Golang gc垃圾回收原理.
有疑问加站长微信联系（非本文作者）
