Go HTTP server connection pooling - discussion

<p>Hi, I was wondering why the standard <em>net/http.Server</em> does not support pooled connection serving. It seems to be a good alternative for the current solution, in which every request is served by a seperate goroutine:</p> <pre><code>func (srv *Server) Serve(l net.Listener) error { defer l.Close() var tempDelay time.Duration // how long to sleep on accept failure for { rw, e := l.Accept() if e != nil { if ne, ok := e.(net.Error); ok &amp;&amp; ne.Temporary() { if tempDelay == 0 { tempDelay = 5 * time.Millisecond } else { tempDelay *= 2 } if max := 1 * time.Second; tempDelay &gt; max { tempDelay = max } srv.logf(&#34;http: Accept error: %v; retrying in %v&#34;, e, tempDelay) time.Sleep(tempDelay) continue } return e } tempDelay = 0 c, err := srv.newConn(rw) if err != nil { continue } c.setState(c.rwc, StateNew) // before Serve can return go c.serve() } } </code></pre> <p><em>c.serve()</em> can be executed by a pooled, created earlier worker. Do the goroutine creation time is so small that it can be skipped? Even though, having a huge amount of goroutines the application becomes unstable and scheduling is a pain.</p> <hr/>**评论：**<br/><br/>thepciet: <pre><blockquote> <p>Do the goroutine creation time is so small that it can be skipped?</p> </blockquote> <p>This is the goal, where goroutines are NOT threads but a finer lighter construct meant to simplify real world concurrent programming (like just starting a goroutine instead of managing a pool).</p> <p>The official golang blog covers a ton, as does the official documentation. e.g.: <a href="https://blog.golang.org/pipelines" rel="nofollow">https://blog.golang.org/pipelines</a></p></pre>sinatosk: <pre><p>Just implement the net.Listener interface to do pooling... I think that&#39;s what other people do... But if you want to respond to the client in http protocol then you just implement that in your handlers but that means accepting a connection which you want to pool... Either way I don&#39;t see the need of the net/http implementing that</p> <p>Not really an answer to why as you asked</p></pre>m173314: <pre><p>The need comes from a simple fact - the cost of goroutine creation.</p></pre>kron4eg: <pre><p>goroutine creation is very cheap, you&#39;re trying to solve unexacting problem.</p></pre>m173314: <pre><p>So you try to tell me that having spawned 300k+ goroutines seems legit? I mean spawned by HTTP server, not business logic workers.</p></pre>xena-warrior-prince: <pre><p>Yes.</p></pre>TheMerovius: <pre><p>I tested it out, on my system I can spawn about 2 million goroutines without any significant scheduling problems. Go routines are dead cheap, that&#39;s the whole point.</p> <p>Look at it this way: Effectively, the go runtime does the connection pooling you wish for: There is a finite number of workers (threads in this case) that handle units of work (goroutines in this case).</p></pre>kron4eg: <pre><p>yep, managing incoming connections pool is work for HAproxy/nginx and similar software.</p></pre>hayzeus: <pre><p>That is absolutely legit. I do an intro to go talk where I demo a program that spawns a couple of million on a dual core mbp. Takes around 7-8 microseconds per routine. Not zero overhead, but very little.</p></pre>SportingSnow21: <pre><p>If you&#39;re looking for savings on this fine-grain of a level, take a look at <a href="https://github.com/valyala/fasthttp" rel="nofollow">fasthttp</a> library. It takes care of the resource pooling to squeeze out more performance.</p></pre>valyala: <pre><p>While <a href="https://github.com/valyala/fasthttp/blob/master/workerpool.go" rel="nofollow">worker pool used in fasthttp</a> gives measurable performance gain comparing to creating a goroutine per request, it may be completely eliminated in the future if golang runtime is optimized properly, i.e. if it will keep per-cpu stack of stopped goroutines (metadata + stack) for subsequent reuse.</p> <p>I don&#39;t know how golang runtime manages stopped goroutines at the moment, but it looks like there is a room for further optimizations.</p></pre>