Crash after launching ~40k goroutines

<p>I&#39;m trying to experiment with goroutines and launched ~40k goroutines when a POST request reaches my net/http server. This crashes the program without a stacktrace and lists the stack trace of the 40k goroutines.</p> <p>This, as you can imagine, is gigantic console output - is there a way to quieten it? How do I start debugging what actually went wrong - the 40k goroutine outputs don&#39;t seem to be related to the error.</p> <p>Reducing the number to 10k seems to make it work - but that leads me to think if I&#39;m overrunning memory. Could this be the case? All 40k goroutines are writing to the same channel. How do I make it a buffered channel?</p> <pre><code>The log looks like this: Max threads: 32 Listening for requests on port 8002 ... runtime/cgo: pthread_create failed: Resource temporarily unavailable SIGABRT: abort PC=0x3ba0432925 goroutine 0 [idle]: goroutine 1 [IO wait]: net.(*pollDesc).Wait(0xc208010140, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc208010140, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).accept(0xc2080100e0, 0x0, 0x7ffff7fa9b70, 0xc208090460) /usr/local/go/src/net/fd_unix.go:419 +0x40b net.(*TCPListener).AcceptTCP(0xc20807a020, 0x46a3de, 0x0, 0x0) /usr/local/go/src/net/tcpsock_posix.go:234 +0x4e net/http.tcpKeepAliveListener.Accept(0xc20807a020, 0x0, 0x0, 0x0, 0x0) </code></pre> <p>(followed by 200k lines of log)</p> <hr/>**评论：**<br/><br/>klauspost: <pre><p>You may be running out of file descriptors. You don&#39;t say which OS you are running, but that would be a place to start.</p></pre>hayzeus: <pre><p>Yeah -- I&#39;d tend to agree. </p></pre>davecheney: <pre><p>Can you put the entire message in a pastebin and post a link to it here, it should be pretty easy to figure out what went wrong. </p></pre>bourbondog: <pre><p>I put it in the original question.</p></pre>legec: <pre><p>If I read the first line correctly, it mentions &#34;cgo&#34; and pthread_create.</p> <p>Does your code call a linked C lirary, which spawns <em>threads</em> (not goroutines) ?</p></pre>bonekeeper: <pre><pre><code> EAGAIN A system-imposed limit on the number of threads was encountered. There are a number of limits that may trigger this error: the RLIMIT_NPROC soft resource limit (set via setrlimit(2)), which limits the number of processes and threads for a real user ID, was reached; the kernel&#39;s system- wide limit on the number of processes and threads, /proc/sys/kernel/threads-max, was reached (see proc(5)); or the maximum number of PIDs, /proc/sys/kernel/pid_max, was reached (see proc(5)). </code></pre> <p><a href="http://man7.org/linux/man-pages/man3/pthread_create.3.html" rel="nofollow">http://man7.org/linux/man-pages/man3/pthread_create.3.html</a></p></pre>010a: <pre><p>I&#39;m very curious why there&#39;s a reference to <code>runtime/cgo</code> in your stacktrace. Were you using <code>cgo</code> to call C code from your goroutines? What version of Go are you using? What OS?</p> <p>I&#39;m not totally familiar with how the go compiler creates goroutines, but it doesn&#39;t make any sense that it would be calling pthread_create for each goroutine, as that&#39;s a standard pthread and it very much would run out of file descriptors after a few thousand, if that. Goroutines are supposed to be much lighter. </p> <p>I&#39;ve tested a simple program (<a href="https://play.golang.org/p/LYvvZdyxe3" rel="nofollow">here</a>) up to 10M goroutines and while it takes ~4 minutes to see any output as the main thread is starting up all those goroutines we do eventually start seeing the expected output. Go 1.4.2 on OSX. </p></pre>bom_d_van: <pre><p>Usually, the problems causing the crash are printed at the beginning of the outputs.</p></pre>hipone: <pre><p>If it&#39;s an experiment, mind sharing code? Folks <a href="https://github.com/golang/go/issues/5233" rel="nofollow">here</a> wouldn&#39;t mind. </p></pre>fr4nk3n: <pre><p>I think you are correct about running out of memory, similar issue <a href="https://github.com/gocql/gocql/issues/330" rel="nofollow">here</a></p></pre>davecheney: <pre><p>This error is usually associated with either making a lot of outgoing network connections (requiring a dns lookup, which is handled by cgo by default on linux), or by not closing http request bodies.</p></pre>