<p>I'm totally new to Go, but pretty enthusiastic about it, coming from Ruby. It's fast, stdlib is pretty small, and community seems really cool. I'm working on my first kind of fun project and it's more or less a simple Rest API, using net/http and postgres. Where it gets a little tricky is implementing RabbitMQ support. I want to have all the basic CRUD actions for dealing with a 'comment' object. Additionally, since creating comments is not always required immediately, I'd like to have a message consumer subscribing to an exchange that can create new comments in the background. The questions are:
Is it possible for me to have a little server AND a little consumer in the same app?
Is this insane?
Is this impossible?
Is this just bad design?</p>
<hr/>**评论：**<br/><br/>pierrrre: <pre><p>Why?
Why can't you create the comment immediately?</p>
<p>I've used this library: <a href="https://github.com/streadway/amqp" rel="nofollow">https://github.com/streadway/amqp</a> , but only on the "producer" side.
It's easy to use.</p>
<p>If you don't want to create the comment immediately, you can use a buffered chan.</p></pre>panama_canals: <pre><p>I know Go is fast for sure, but this has more to with the whole "microservices" architecture thing.</p>
<p>Sometimes an object does not need to be created immediately. Let's say I've got a Rails app; I click a link and it builds a PDF or something like that. This is a long running process, I don't want my users waiting for the PDF to finish, so it's suitable for a non-blocking background job. In my case I want to send a message via RabbmitMQ to this Go service. The services listens for a job and does it. From the Rails app, I hit /pdfs/index or whatever, and see a list of PDFs done, which is pulled from the REST aspect of the Go server. This is a pretty involved example, but totally possible in the real world.</p></pre>pierrrre: <pre><p>OK, that's a good reason.</p>
<p>Read <a href="https://github.com/rabbitmq/rabbitmq-tutorials/tree/master/go" rel="nofollow">https://github.com/rabbitmq/rabbitmq-tutorials/tree/master/go</a> & <a href="https://godoc.org/github.com/streadway/amqp" rel="nofollow">https://godoc.org/github.com/streadway/amqp</a></p></pre>jasonbx: <pre><blockquote>
<p>I don't want my users waiting for the PDF to finish, </p>
</blockquote>
<p>Why not call the pdf generating part in a goroutine and return the response back immediately to the user? In this case, it will be non blocking for the user. Although, if the load is too large that the system cannot scale, you will need to go for a queuing system.</p></pre>roveboat: <pre><blockquote>
<p>Although, if the load is too large that the system cannot scale, you will need to go for a queuing system.</p>
</blockquote>
<p>His usecase was that the PDF generation is triggered from a Rails app - so this wouldn't work. </p>
<p>Also, even if his API was on Go as well, this would quickly run into the scaling problem you point out - you'd basically be killing your API in case the PDF generation load on the server got too high.</p></pre>jasonbx: <pre><blockquote>
<p>His usecase was that the PDF generation is triggered from a Rails app - so this wouldn't work.</p>
</blockquote>
<p>Assuming there is no load issue, this will work perfectly with golang. Rails app calls a golang API server which can immediately return the response back to the app while creating the PDF in the background.</p></pre>google_you: <pre><p>For asynchronous tasks, you create a task resource clients can check.
For example,</p>
<pre><code>POST /comments/submissions HTTP/1.1
; response
HTTP/1.1 202 Accepted
Content-Location: http://fuck/comments/submissions/1
</code></pre>
<p>Clients can keep checking that resource to see the status:</p>
<pre><code>GET /comments/submissions/1 HTTP/1.1
; response
HTTP/1.1 200 Ok
Content-Type: text/html
<link href="http://fuck/comments/submissions/1" rel="self">
<p>Pending review bro.</p>
</code></pre>
<p>When asynchronous task is done, you'd redirect clients:</p>
<pre><code>GET /comments/submissions/1 HTTP/1.1
; response
HTTP/1.1 303 See Other
Location: http://fuck/comments/1
</code></pre>
<p>Clients can now <code>GET</code>, <code>PUT</code>, <code>DELETE</code> <code>/comments/1</code>....</p>
<p>For implementing this in Go, I would create a process for HTTP stuff. And another process that works on asynchronous task (consuming RabbitMQ messages). If worker is down, HTTP is still up. </p>
<p>It's usually better to create a simple program that does one thing well. Once you have many simple programs, you can have a supervisor program that manages lifetime of those programs. </p></pre>qu33ksilver: <pre><p>Oh this is nice, I didn't know about the 202 status code.</p></pre>smallfishxy: <pre><p>try <a href="https://github.com/smallfish/rabbitmq-http" rel="nofollow">https://github.com/smallfish/rabbitmq-http</a> ?</p></pre>panama_canals: <pre><p>Followup: so I'm still learning how go works, but I liked the answer about long polling, and I liked the answer about using a goroutine. I tried this and it totally works...</p>
<pre><code>func main() {
initializeDB() // get a connection to dat yung db
go initializeRabbitMQ() // listen for messages
goji.Get("/", Root)
goji.Get("/pdfs", GetPdf)
goji.Post("/pdfs", CreatePdf)
// more routes...
goji.Serve()
}
func initializeRabbitMQ() {
// dial rabbitmq, setup channel, setup exchange, setup queue, start long running consumer
}
</code></pre>
<p>Is this approach idiomatic to Go? Any really obvious issues? I guess I'd probably want to consume each message on my RabbitMQ channel/exchange/queue in a goroutine so it doesn't block. I dunno.</p></pre>argandas: <pre><p>I have never used RabbitMQ, but you should totally take a look at martini for REST APIs
<a href="https://github.com/go-martini/martini" rel="nofollow">https://github.com/go-martini/martini</a>
About having producer-consumer loops inside the app, I suppose is for testing purposes, try with in-built testing package for Go.
<a href="http://golang.org/pkg/testing/" rel="nofollow">http://golang.org/pkg/testing/</a></p></pre>ecmdome: <pre><p>Do not use Martini for REST API</p>
<p>Use the standard net/http library or possibly Gorilla MUX to handle your routes and maybe Negroni for middleware.</p></pre>Bromlife: <pre><p>I agree with your statement 50%. Do not use Martini.</p>
<p>But you don't need Gorilla or Negroni either. The best thing I ever did for my web application was writing my own muxer. Sure, I initially started using Gorilla, but I soon realised it didn't <em>fit</em> my needs perfectly. Looking at the Gorrilla Mux source code made me realise that I could easily cut out the middle man & doing so also cut out <em>a lot</em> of overhead. It's can't be overstated just how much overhead these libraries add.</p>
<p>They're great to get you started, and I would use them for simple "Hackathon" style hobby projects. But for real apps you're building - do it yourself.</p></pre>ecmdome: <pre><p>I actually very much agree with you.</p>
<p>I started with standard library and went to Gorilla for the muxer just because I wanted to save some time...</p>
<p>I actually like Negroni... But yeah you can essentially build it pretty simply or something like it...</p></pre>Bromlife: <pre><p>When using Negroni (or Alice) I quickly realised that "middleware" really only applies in a framework context. Seeing as you're building software from the ground up, with awesome first party libraries, in my opinion it doesn't fit in the same way. Why would you need middleware if you're building the routing engine?</p></pre>ecmdome: <pre><p>I don't see how mifdlewares only applies in a framework context.</p>
<p>I mean essentially I'm building a custom framework for my application.</p>
<p>I like the middleware pattern for auth and handling some session data. </p></pre>argandas: <pre><p>Any good reason not to use martini? It's.. really easy to use, I'm kind of noob on this. So far only used mastini.</p></pre>Mteigers: <pre><p>Martini due to the amount of reflection use and dependency injection leveraged is quite a bit slower than other frameworks. Though when I say quite a bit slower we're still measuring in the nanoseconds range. </p>
<p>We actually run a production REST API in Martini. At the time most of our developers only knew PHP (our company recently switched to Go) and Martini was the most familiar feeling to the team to allow us to deliver a product the fastest. Is it fast for us? Go/Martini has never been a concern for what it actually does. Will we rewrite it and ditch Martini? Probably. </p></pre>ecmdome: <pre><p>Its not idiomatic to how Go was designed. The creator of Martini even wrote a response to the criticism of it saying he built it before fully understanding Go.</p>
<p>I reccomend starting with standard library only and going to 3rd party when you see a need.</p>
<p>You will most likely use the Gorilla MUX rourter, but I recommend building your own first before jumping to that.</p>
<p>Look at Go as modular rather than one framework to rule them all.</p>
<p>Also ORM is a tricky one in Go. I use Gorm, which can be really nice but I sometimes wish I just abandoned an ORM and went with the standard library or maybe Gorp.</p>
<p>Go is really fun though so enjoy the learning experience! </p></pre>
