<p>Give them more memory!</p>
<pre><code>$ time go build -a std
real 0m10.248s
user 0m25.388s
sys 0m2.832s
$ export GOGC=2000
$ time go build -a std
real 0m7.249s
user 0m14.912s
sys 0m2.644s
</code></pre>
<p>And BTW compiling the whole standard library in 10 seconds is quite impressive.</p>
<hr/>**评论：**<br/><br/>inhies: <pre><p>Is this a permanent effect for the built application, or does it just affect the go tool while building?</p></pre>abcded1234234: <pre><p>GOGC environment variable is read at start up by any Go application (such as Go compiler). This value doesn't become permanent for the built application.</p></pre>Kyrra: <pre><p>And as another post points out, debug.setGCPercent can change this at runtime.</p></pre>dirkharrington: <pre><p>GOGC does not "give more memory"</p></pre>abcded1234234: <pre><p>Could you please elaborate?</p>
<p>From <a href="https://golang.org/pkg/runtime/">https://golang.org/pkg/runtime/</a></p>
<blockquote>
<p>The GOGC variable sets the initial garbage collection target percentage. A collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage.</p>
</blockquote>
<p>Effectevely by giving an application more memory you make GC run less frequently thus freeing some CPU time for the application.</p></pre>the_birds_and_bees: <pre><p>I guess it's a pedantic point, but you're not really giving the application more memory. You're just saying that you can wait longer between triggering GC sweeps.</p>
<p>This means your program will consume more memory (your creating just as much garbage, but collecting it less often) with the benefit that less CPU is wasted on GC.</p>
<p>It's a trade off that works well in your case where you evidently have plenty of memory to play with, but in a more memory constrained environment it's not necessarily such a good idea.</p>
<p>ed: pedantic, not semantic</p></pre>dirkharrington: <pre><p>It's not semantic although I'd admit pedantic. When stated as-is it indicates the OP doesn't fully understand why result was achieved. Giving you a +1 though as you ran the point to ground.</p></pre>abcded1234234: <pre><blockquote>
<p>It's a trade off that works well in your case where you evidently have plenty of memory to play with, but in a more memory constrained environment it's not necessarily such a good idea.</p>
</blockquote>
<p>There is no such thing as a free lunch. There is always a trade off.</p></pre>izuriel: <pre><p>I know it's been disputed but nothing in there says it gives more memory. It's a percentage of the ratio of memory used to determine if a collection happens. </p>
<p>Either way the same memory is used, but more or less will be held simultaneously depending on the setting. </p></pre>abcded1234234: <pre><p>Try understanding the effect of increasing GOGC. If application live heap size is X ("live data remaining after the previous collection") then the next GC will happen when "when the ratio of freshly allocated data to" X reaches GOGC. The larger GOGC the more memory your application will use.</p></pre>izuriel: <pre><p>I understand it. I don't think you do. It says in its description that it's the ratio used to determine when the collection of garbage happens. </p>
<p>That's it. That's what the value is for. I would hazard there a point where this would actually <em>decrease</em> the amount of memory used by the application because the memory would be tied up waiting on a sweep to start. </p>
<p>The wording is very clear. </p></pre>abcded1234234: <pre><blockquote>
<p>I would hazard there a point where this would actually decrease the amount of memory used by the application because the memory would be tied up waiting on a sweep to start.</p>
</blockquote>
<p>I must admit that I do not understand this statement. Can you please explain?</p>
<p>Certainly increasing GOGC will result in an application requesting more memory from OS.</p></pre>izuriel: <pre><p>From your post from source:</p>
<blockquote>
<p>The GOGC variable sets the initial garbage collection target percentage. A collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage.</p>
</blockquote>
<p>First things first let's assume our program over the course of its run consumes a total of X bytes. This is the amount of memory used in total, not the amount it uses at any given time. This is just space allocated on the heap and stack as it does its work. </p>
<p>If we took a slice before a garbage collection sweep we might see Y bytes in use and immediately after the seep it might be actively using Z bytes (where Z < Y) due to memory being freed. No memory was "created," instead memory that had no more references in love code was released and potentially reordered (I honestly don't know the specifics of the Go GC). </p>
<p>Now let's break this documentation down. </p>
<blockquote>
<p>The GOGC variable sets the initial garbage collection target percentage.</p>
</blockquote>
<p>What we can gather is the value of GOGC is a percentage. Usually used in cases where 0-100 is significant. I'm not sure what "initial" is meant here, could mean it's not the permanent value but just the first time they need a value this is where they get it.</p>
<p>But right off the bat no mention of memory, I would hazard if this had a direct impact on memory given to the program we would have already been made aware of it. Lets move on anyways. </p>
<blockquote>
<p>A collection is triggered when the ratio [...] reaches this percentage.</p>
</blockquote>
<p>(We'll look at the omission next)</p>
<p>So now we see the significance. Whatever this percentage is of its used to trigger a collection. With a name like GOGC and the "collection" I'm going to immediately assume we're talking a GC sweep which <em>frees</em> memory no longer in use by the program. It does more, but for this we don't care about that. </p>
<p>This would be that collection that will clear up I used chunks of memory for us (be thankful we don't have to free or delete memory!). So the question now is what is it a ratio of?</p>
<blockquote>
<p>... freshly allocated data to live data remaining after the previous collection ... </p>
</blockquote>
<p>So when data that has been allocated (memory taken by this program) divided by the amount of live (I assume they mean available) memory that was last after the last collection. So on you case when the amount of data allocated is 20 times the amount of remaining available data to consume then another garbage collection is started.</p>
<p>But still no mention of giving memory, just discussing at what threshold the GC of used/available memory the GC starts a sweep. </p>
<p>Now. Our program hasn't changed and is not altered. So over its lifetime it <em>still</em> consumes only X bytes. So even if we did give it more memory it would be wasted unless we constantly kept heap and stack space full with active memory, most likely we won't. </p>
<hr/>
<p>Now. What does that have to do with my statement?</p>
<blockquote>
<p>I would hazard there a point where this would actually decrease the amount of memory used by the application because the memory would be tied up waiting on a sweep to start.</p>
</blockquote>
<p>First let's note that said "I would hazard" so I'm guessing. This could be incorrect but I think its true. </p>
<p>Now, since we've previously established thenGOGC value is a ratio of active/free memory the assumption was made that at some point the ratio could be set such that there is a build up of unused, allocated memory that we can't use until the next sweep of the GC and we need to allocate some number of bytes greater than we have available.</p>
<p>Now since I've made this comment I've taken on this leading tons crash rather than a "pause." Still an assumption either way.</p>
<hr/>
<p>So my theory is you mounded stood GOGC and based on that you played with it and it coincidentally seemed to do what you expected it to do. When in reality the speed up is most likely associated to reducing the number of times the GC pauses and runs during the runtime of the program. That's my theory. </p></pre>abcded1234234: <pre><p>Interesting. So based on your theory how GC in GO might work you concluded that my understanding is wrong? ;)</p>
<p>Updated:</p>
<blockquote>
<p>So when data that has been allocated (memory taken by this program) divided by the amount of live (I assume they mean available) memory that was last after the last collection. </p>
</blockquote>
<p>The documentation says "freshly allocated memory". This means memory allocated since the last GC. "Live data" means all the data that is reachable from the root set (stack and global variables) via pointers, i.e. all data that shouldn't be garbage collected.</p>
<blockquote>
<p>So even if we did give it more memory it would be wasted </p>
</blockquote>
<p>It would indeed be wasted, but the application might run faster. On other hand if applications running on a server with 256GB of RAM use only 10GB then all available RAM is wasted as well.</p>
<p>The speed up most likely comes from the fact that the current GC algorithms can cleanup 100MB of memory faster than 10 times 10MB. Larger GOGC value might lead to larger stop the world pauses. Even with the larger pauses the latency of your application might still be better (or maybe not). This is application dependent and should be measured to make an informed decision.</p></pre>izuriel: <pre><p>No. I concluded you were wrong based on the definition you used to defend your stance and my knowledge of how memory is allocated to a process, used by a program and why a GC is beneficial.</p></pre>izuriel: <pre><p>I see you edit your posts without noting they're edited. That's a lot of information to silently edit in. </p>
<p>You're confirming my suspicions. </p>
<blockquote>
<p>This means memory allocated since the last GC. "Live data" means all the data that is reachable from the root set (stack and global variables) via pointers, i.e. all data that shouldn't be garbage collected.</p>
</blockquote>
<p>This is exactly what I meant. You still reference memory. The GOGC documentation says nothing about changing the amount of memory. Longer periods between GC pauses (which I would associate to larger values of GOGC) would result in <em>more memory</em> having to be processed by the GC but fewer GC pauses. For smaller values you see more GC pauses but smaller amounts of memory to process. Either way the total amount of memory available to the programs is the same, and the total amount of memory used by the program is the same so changing this value <em>has no effect on increasing the memory available to your program</em>. </p></pre>earthboundkid: <pre><p>Nothing changes the amount of memory potentially available to modern applications. The amount of memory available is the size of virtual memory. The application has no control over it. It's set by the OS and physical hardware. </p>
<p>Pedantically speaking, "increase the size of memory" is wrong, but anyone interpreting it should understand the meaning, which is to consume more memory at one time due to less aggressive GC cleanup. </p></pre>abcded1234234: <pre><p>Apologies for editing the post without noting. I've added "Updated:" comment now.</p>
<p>Don't get me wrong, I am not trying to prove that you are wrong. But from your posts we either completely misunderstand each other or you seem to be mistaken. I could be completely wrong too but so far I haven't seen any convincing arguments ;)</p>
<p>The amount of memory available to a program is limited by OS (e.g. by setting appropriate ulimits) and hardware.</p>
<p>However the amount of memory that is allocated by a program has direct relationship to the GOGC value. In fact it is very easy to check. If you run the following program <a href="http://play.golang.org/p/snRk6E_7qt" rel="nofollow">http://play.golang.org/p/snRk6E_7qt</a> with GODEBUG=gctrace=1 and different GOGC values you will see that with GOGC=100 the maximum heap size is 4mb, while with GOGC=2000 the maximum heap size is 80Mb. Resident size of the program (from top output) also different with different GOGC values.</p></pre>mcouturier: <pre><blockquote>
<p>The GOGC variable sets the initial garbage collection target percentage. A collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage. The default is GOGC=100. Setting GOGC=off disables the garbage collector entirely. The runtime/debug package's SetGCPercent function allows changing this percentage at run time. Seehttps://golang.org/pkg/runtime/debug/#SetGCPercent.
<a href="https://golang.org/pkg/runtime/" rel="nofollow">https://golang.org/pkg/runtime/</a></p>
</blockquote></pre>rv77ax: <pre><p>Turn out using <code>GOGC=off</code> make building Go from Git failed, with this error messages, it said need GC value greater than 2.</p>
<pre><code>ok regexp/syntax 0.451s
--- FAIL: TestPeriodicGC (0.10s)
gc_test.go:183: no periodic GC: got 0 GCs, want >= 2
--- FAIL: TestMemStats (0.00s)
malloc_test.go:37: Insanely high value (overflow?): {Alloc:14420016 TotalAlloc:57296800 Sys:33089784 Lookups:439 Mallocs:157119 Frees:104401 HeapAlloc:14420016 HeapSys:27688960 HeapIdle:12607488 HeapInuse:15081472 HeapRe
leased:0 HeapObjects:52718 StackInuse:622592 StackSys:622592 MSpanInuse:137040 MSpanSys:393216 MCacheInuse:9600 MCacheSys:16384 BuckHashSys:1447293 GCSys:923648 OtherSys:1997691 NextGC:184467440737335470 LastGC:14562535127674427
57 PauseTotalNs:58208592 PauseNs:[1104847 3177868 593614 592411 573854 730914 822863 769439 796090 841594 8232448 689357 565307 387802 1148173 1160727 1010786 327283 313168 347243 1110837 441733 1226354 5307645 1510407 776550 76
5348 742427 749354 747003 765102 793537 780891 775395 733615 812284 585673 812277 596963 851949 592407 849548 575580 814108 695078 741219 728967 6812728 605094 722731 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] PauseEnd:[1456253506787450387 1456253506791701223 1456253506793362708 1456253506795024950 14562535067966
67798 1456253506798469191 1456253506800372320 1456253506802223203 1456253506804095050 1456253506806012010 1456253512609995599 1456253512610825509 1456253512611398324 1456253512612377218 1456253512614154138 1456253512616015902 14
56253512617782588 1456253512620448328 1456253512622536449 1456253512624971347 1456253512627270044 1456253512628868815 1456253512631232497 1456253512636849658 1456253512638715727 1456253512639502433 1456253512640273873 1456253512
641022720 1456253512641777552 1456253512642530534 1456253512643301530 1456253512644102404 1456253512644890885 1456253512645672927 1456253512646494431 1456253512647328254 1456253512647935021 1456253512648765608 145625351264938471
3 1456253512650258605 1456253512650874213 1456253512651741600 1456253512652336996 1456253512653175123 1456253512653964934 1456253512654867818 1456253512759177177 1456253512766005773 1456253512766634772 1456253512767442757 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] NumGC:50 GCCPUFraction:0.009846998246626222 Enabl
eGC:true DebugGC:false BySize:[{Size:0 Mallocs:0 Frees:0} {Size:8 Mallocs:930 Frees:876} {Size:16 Mallocs:2503 Frees:2194} {Size:32 Mallocs:23194 Frees:22961} {Size:48 Mallocs:954 Frees:859} {Size:64 Mallocs:1532 Frees:1003} {Si
ze:80 Mallocs:10103 Frees:94} {Size:96 Mallocs:41267 Frees:41228} {Size:112 Mallocs:3231 Frees:3220} {Size:128 Mallocs:290 Frees:268} {Size:144 Mallocs:5562 Frees:5538} {Size:160 Mallocs:34 Frees:28} {Size:176 Mallocs:40973 Free
s:13} {Size:192 Mallocs:70 Frees:59} {Size:208 Mallocs:64 Frees:54} {Size:224 Mallocs:57 Frees:54} {Size:240 Mallocs:17 Frees:17} {Size:256 Mallocs:163 Frees:142} {Size:288 Mallocs:117 Frees:113} {Size:320 Mallocs:61 Frees:57} {
Size:352 Mallocs:5 Frees:5} {Size:384 Mallocs:166 Frees:0} {Size:416 Mallocs:10 Frees:6} {Size:448 Mallocs:0 Frees:0} {Size:480 Mallocs:2 Frees:2} {Size:512 Mallocs:141 Frees:124} {Size:576 Mallocs:103 Frees:101} {Size:640 Mallo
cs:13 Frees:9} {Size:704 Mallocs:6 Frees:6} {Size:768 Mallocs:41 Frees:41} {Size:896 Mallocs:10 Frees:9} {Size:1024 Mallocs:20070 Frees:20043} {Size:1152 Mallocs:112 Frees:101} {Size:1280 Mallocs:6 Frees:2} {Size:1408 Mallocs:10
Frees:9} {Size:1536 Mallocs:78 Frees:76} {Size:1664 Mallocs:7 Frees:4} {Size:2048 Mallocs:4053 Frees:4042} {Size:2304 Mallocs:101 Frees:101} {Size:2560 Mallocs:4 Frees:0} {Size:2816 Mallocs:0 Frees:0} {Size:3072 Mallocs:2 Frees
:1} {Size:3328 Mallocs:44 Frees:42} {Size:4096 Mallocs:14 Frees:6} {Size:4608 Mallocs:101 Frees:101} {Size:5376 Mallocs:48 Frees:0} {Size:6144 Mallocs:10 Frees:8} {Size:6400 Mallocs:0 Frees:0} {Size:6656 Mallocs:1 Frees:0} {Size
:6912 Mallocs:0 Frees:0} {Size:8192 Mallocs:8 Frees:3} {Size:8448 Mallocs:0 Frees:0} {Size:8704 Mallocs:0 Frees:0} {Size:9472 Mallocs:101 Frees:101} {Size:10496 Mallocs:6 Frees:0} {Size:12288 Mallocs:0 Frees:0} {Size:13568 Mallo
cs:3 Frees:0} {Size:14080 Mallocs:0 Frees:0} {Size:16384 Mallocs:1 Frees:1} {Size:16640 Mallocs:0 Frees:0} {Size:17664 Mallocs:24 Frees:0}]}
</code></pre>
<p>But building with <code>GOGC=2000</code> works.</p></pre>f2f: <pre><p>looks like you're running not just the build but all tests too, which fail at the tests specifically expecting periodic GC to work. instead just use make.bash:</p>
<pre><code>$ export GOGC=off
$ time ./make.bash 2> /dev/null
##### Building Go bootstrap tool.
cmd/dist
##### Building Go toolchain using /Users/.../go1.4.
##### Building go_bootstrap for host, darwin/amd64.
##### Building packages and commands for darwin/amd64.
---
Installed Go for darwin/amd64 in /Users/.../go
Installed commands in /Users/.../go/bin
real 0m48.378s
user 2m0.226s
sys 0m13.555s
$ echo $?
0
</code></pre></pre>rv77ax: <pre><p>Oh, I see. Here is mine, without and with GOGC.</p>
<pre><code>ms 0 % time ./make.bash 2>/dev/null
##### Building Go bootstrap tool.
cmd/dist
##### Building Go toolchain using /home/ms/Programs/go.
##### Building go_bootstrap for host, linux/amd64.
##### Building packages and commands for linux/amd64.
---
Installed Go for linux/amd64 in /home/ms/git/go
Installed commands in /home/ms/git/go/bin
./make.bash 2> /dev/null 117,86s user 5,81s system 469% cpu 26,363 total
~/git/go/src
ms 0 % export GOGC=off
~/git/go/src
ms 0 % time ./make.bash 2>/dev/null
##### Building Go bootstrap tool.
cmd/dist
##### Building Go toolchain using /home/ms/Programs/go.
##### Building go_bootstrap for host, linux/amd64.
##### Building packages and commands for linux/amd64.
---
Installed Go for linux/amd64 in /home/ms/git/go
Installed commands in /home/ms/git/go/bin
./make.bash 2> /dev/null 64,48s user 6,34s system 360% cpu 19,671 total
</code></pre></pre>eikenberry: <pre><p>Why 2000? Did you test various settings or just pick one and share the good results?</p></pre>abcded1234234: <pre><p>It could be anything bigger than 100 (the default value). Or it can be less than 100 if you want your application to use less memory and more CPU. The optimal value is application specific and can even be different between different Go versions.</p></pre>dlsniper: <pre><p>I think that it would be better if a proper explanation of how GOGC works and what changing it entails would be posted. This way, it could be concluded that for a specific use-case on a specific machine, value X is the one that Improves <insert here>. </p></pre>earthboundkid: <pre><p>Interesting. In a talk about rewriting the compiler in Go, Rob Pike mentions that one reason the old C-based compiler is/was faster is that it never frees allocated memory, since it knows that it's only going to be alive for a few seconds. This puts the new compiler on even footing. </p></pre>ggggopher: <pre><p>very good tip for the fast compiling</p></pre>dlsniper: <pre><p>Have you tried GOGC=off?</p></pre>abcded1234234: <pre><p>It becomes marginally faster. However turning GC off might be ok for a command line utility that quickly does its job and quits (like a compiler). Turning it off for a web application is not desireable. </p></pre>dlsniper: <pre><p>Which is exactly what the title says, go compiler :)</p></pre>abcded1234234: <pre><p>The advice applies to any application written in Go though. And it is so easy to try and see if a larger GOGC value makes any difference for your application. Compare this with 1000 JVM GC options ;)</p></pre>Smartsheet_SysEng: <pre><p>I <3 G1GC.</p></pre>boarhog: <pre><p>What is G1GC?</p></pre>ChasingLogic: <pre><p>The garbage collection mechanism primarily used by the JVM</p></pre>
