<p>I want to build something very similar to the Twitter Firehose with their streaming API.</p>
<p>I want to have one or more API clients connect in and to be able to send down a JSON (compressed/uncompressed) payload.</p>
<p>I have already acheived this using straightforward TCP4 connections but the Twitter way with HTTP seems a little more elegant. Also by replicating Twitter there are a ton of examples in lots of languages that people could use to connect in.</p>
<p>Question is...where do you start with this in Go? It seems a little backwards to keep a persistent HTTP connection open and for the server to send down to the client.</p>
<p>Any advice/guidance appreciated!!</p>
<hr/>**评论：**<br/><br/>mm_5687: <pre><blockquote>
<p>It seems a little backwards to keep a persistent HTTP connection open and for the server to send down to the client.</p>
</blockquote>
<p>That's exactly how it works though.</p>
<p>It's a chunked http connection, there's not much more to it.</p></pre>lesmond: <pre><p>Ah ok, so it's just a chunked connection. So you'd use the standard HTTP server and just set the headers and send those down to the client on connection?</p></pre>mm_5687: <pre><p>I don't even think you need to manually add the headers, Go handles that for you: <a href="https://golang.org/pkg/net/http/#Request.Write" rel="nofollow">https://golang.org/pkg/net/http/#Request.Write</a></p></pre>lesmond: <pre><p>So I have this example that I just put together but the connection closes but I essentially want to keep the connection open until a write happens (or a timeout).</p>
<pre><code>func startHTTP(pathPrefix string) {
log.Println("Starting HTTPS Server")
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// Wait here until a write happens to w
})
log.Print("HTTPS listening on :5556")
log.Fatal(http.ListenAndServeTLS(":5556", pathPrefix+".crt", pathPrefix+".key", nil))
}
</code></pre></pre>tv64738: <pre><p>Typed quickly straight into reddit but something like this:</p>
<pre><code>import "github.com/tv42/topic"
type MyApp struct {
msgs *topic.Topic
}
func NewApp() *MyApp {
app := &MyApp{
msgs: topic.New(),
}
// simulate activity; this silly example leaks a goroutine
go func() {
for {
time.Sleep(100*time.Millisecond)
app.msgs.Broadcast <- "hello, world"
}
}()
return app
}
func (a *MyApp) ServeHTTP(w http.ResponseWriter, req *http.Request) {
ctx := req.Context()
enc := json.NewEncoder(w)
ch := make(chan interface{}, 100)
a.msgs.Register(ch)
defer a.msgs.Unregister(ch)
for {
select {
case msg, ok := <-ch:
if !ok {
log.Printf("kicking slow client")
return
}
if err := enc.Encode(msg); err != nil {
log.Printf("cannot marshal json: %v", err)
return
}
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
case <-ctx.Done():
// client canceled request
return
}
}
}
</code></pre></pre>Redundancy_: <pre><p>Out of curiosity, why not go with websockets?</p></pre>lesmond: <pre><p>Good point but no need for all the extra overhead IMHO. </p></pre>Redundancy_: <pre><p>I thought that websockets were generally more efficient than long-polling connections, and have built in support in a number of languages.</p></pre>girishramnani1234: <pre><p>You also checkout <a href="http://www.nchan.io" rel="nofollow">nchan</a>. It supports more than persistent http connections.</p></pre>Redundancy_: <pre><p>Forbidden on http and bad cert on https?</p></pre>
