<p>I'm looking for people's experiences, relevant posts, and <em>ideally</em> sample apps that they found valuable. I ask because I'm looking to get a sense for the application structure, potential packages, gotchas, etc.</p>
<p>By complex, I mean a site that has auth, the basic set of web request middleware, database connectivity, template creation, complex business logic (more than just a JSON API with minimal business logic).</p>
<hr/>**评论：**<br/><br/>MoneyWorthington: <pre><p>I have. My current project at work is a Go web application that's being built by me, and one other person who's doing primarily front-end design work. It's my first experience building a Go web application from the ground up, and for libraries I went with <a href="http://www.gorillatoolkit.org/">Gorilla</a> for routing and session handling, <a href="http://jmoiron.github.io/sqlx/">sqlx</a> and hand-written SQL for database access, <a href="https://justinas.org/alice-painless-middleware-chaining-for-go/">alice</a> for middleware chaining, and <code>net/http</code> + <code>html/template</code> for everything else.</p>
<p>My structure grew very organically, and consists mainly of packages separated out by what seemed to be a logical grouping at the time (e.g. separate packages for form handling, security-related tasks, handlers, etc.).</p>
<p>We're nearing the end of the project, and my main takeaways are:</p>
<ol>
<li><p>I like the flexibility of sqlx with raw SQL, but the standard CRUD operations end up being quite redundant. For my next project, I'm likely going to use reflection to build CRUD queries on demand, but more complex queries will still be hand-written. The hand-written queries use an approach inspired by <a href="https://github.com/krisajenkins/yesql">yesql</a>, where they're written in SQL files, then loaded and parsed at application startup, and minus the aforementioned redundancies, it's worked out great.</p></li>
<li><p>The most annoying part of the whole project is proper form handling. I've developed a newfound respect for systems like Django that can take a database model and turn it into a usable CRUD web interface. I'm not sure if my next project will take the plunge and use an existing CMS (I just learned about QOR as well, and I'm definitely going to give it a look), or if I'm just going to try and build a custom package for generating form pages, but this is one area that I really need to work on.</p></li>
<li><p>Gorilla turned out to be a fantastic choice for routing and session handling. The session documentation had me scratching my head for a bit, but once I figured it out, it turned out to be an amazingly flexible solution, and once session values are working, authentication becomes a breeze, provided you're storing your passwords correctly. Routing also was very easy, though the structure of the URL's themselves require some thought and care. I currently just have a method called from main that registers most of the routes, but one feature of Gorilla's router that I'd like to take more advantage of is its subrouters, dividing up the site into sections and then delegating the work of actually registering the routes elsewhere.</p></li>
</ol>
<p>I could go on and on, but my main piece of advice (and one that I wish I'd taken earlier on in the project) is to divide the packages by area, rather than functionality. I have one package called <code>handler</code> that contains all my HTTP handler methods, and that ended up being a mistake. This realization felt a little counterintuitive to me, given that it <em>seems</em> like it would be good practice to group things by what they do ("these are the handlers", "these are the templates"), but ultimately it would make it much easier if <em>all</em> of the relevant pieces for any given page are grouped closely together.</p>
<p>I have more thoughts, but I've rambled on long enough. I'd be happy to provide additional thoughts if you have anything in particular you'd like to hear more about.</p></pre>elithrar_: <pre><blockquote>
<p>The hand-written queries use an approach inspired by yesql[4] , where they're written in SQL files, then loaded and parsed at application startup, and minus the aforementioned redundancies, it's worked out great.</p>
</blockquote>
<p>yesql is <em>lovely</em>. In the event you haven't seen it already, <a href="https://github.com/smotes/purse">smotes/purse</a> is a very similar package for Go. Minus multiple-queries per file, but works with <code>go generate</code> (optionally).</p>
<blockquote>
<p>The most annoying part of the whole project is proper form handling.</p>
</blockquote>
<p>Agree. Stuff like <code>gorilla/schema</code> or <code>goji/param</code> make the POST values to Go struct part easy.</p>
<p>The fun part is validation: in (e.g.) Django, field length and limits are tied to the model. In Go, you either overload struct tags with a custom [unchecked!] DSL and use reflect (been there; never again) or check field-by-field. I'm now using <a href="https://github.com/kat-co/vala">vala</a> within a <code>Validate</code> method on my structs to simplify the basic checks (too long, not nil, greater than) and am using custom validators where I need to accept a fixed-list of values (i.e. mirroring a drop-down list in the template). The key is to try and tie the "available fields" to the template to avoid disconnect later (i.e. update template but not validation method, which will result in non-obvious bugs).</p>
<p>For the OP: <a href="https://github.com/drone/drone/tree/master/server">https://github.com/drone/drone/tree/master/server</a> is a good example of a complex Go web application. I'd argue that structure/layout is fairly objective, and there's unlikely to ever be a consensus about it. Getting too hung up on that will just hamstring you when you could be getting other things done. Further, by disallowing circular imports, Go helps avoid some craziness here. </p></pre>daveddev: <pre><p>Thoughts on my own take on purse? <a href="https://github.com/codemodus/tote" rel="nofollow">https://github.com/codemodus/tote</a></p>
<p>I am considering adding sql validation, and possibly handling for templates.</p></pre>elithrar_: <pre><p>How would you tackle validation? The syntax between even the popular SQL engines (mySQL/Postgres/MSSQL) varies so wildly as to make that a huge task with a lot of corner cases (quotes, params, etc). Integration tests on the application side are also <em>still</em> going to be more robust.</p>
<p>Otherwise: it looks solid. The <code>pkg.DirName.FileName</code> approach is alright, although even with 30-40 odd queries you end up with a lot of files to deal with (even a well-tuned vim doesn't make this an exciting prospect).</p>
<p>I'm hoping (due to my own lack of time) that someone makes a 1:1 copy of <code>yesql</code> for Go, using the standard SQL comments to provide naming. Putting the queries into a <code>map[string]string</code> keeps it simple too (and can tie in with go generate if you want to generate the map AOT).</p></pre>MoneyWorthington: <pre><p><code>gorilla/schema</code> was a lifesaver for me, but it doesn't work for file uploads, and my app uses dynamic tables pretty extensively, which initally utilized some hacks to get them to comply with <code>schema</code>. Honestly, my biggest mistake here was trying to jQuery everything, rather than use a more declarative approach to the javascript. I'm currently in the processing of rewriting a particularly hairy piece of it in Backbone, because I don't have enough time in the project to learn React or anything like that, but I dream of a form-generation package that can take a struct and spit out a fully-functioning CRUD UI based on its fields.</p>
<p>Standard forms that don't require any dynamic client-side behavior are pretty easy, but unfortunately I underestimated how much more work a dynamic page would be.</p></pre>Dont_Reddit_Me: <pre><p>Are you me?</p>
<p>Also, QOR is an interesting piece to check out its architecture. But it uses ORM and other tools that are not so idiomatic and I rather avoid.</p></pre>no1youknowz: <pre><p>Have you seen this? <a href="http://getqor.com/">http://getqor.com/</a></p>
<p>Most of what you describe is there.</p></pre>captchacleat: <pre><p>I have not, this is great! Thanks for sharing</p></pre>tomcam: <pre><p>Looks super good. Love the tutorial.</p></pre>jwcrux: <pre><p>Sure have! I have been working on a fairly large web app here: <a href="http://github.com/jordan-wright/gophish">http://github.com/jordan-wright/gophish</a>. It has a full api, middleware, etc.</p>
<p>Let me know if you have any questions about design decisions, etc.</p></pre>CaptaincCodeman: <pre><p>Yes, Angular frontend to API server with OAuth server and client (see brief writeup here: <a href="https://www.reddit.com/r/golang/comments/3ewed0/eli5_rangelrealeosin_package_oauth2/">https://www.reddit.com/r/golang/comments/3ewed0/eli5_rangelrealeosin_package_oauth2/</a>).</p>
<p>It also includes an image processing system using Go, VIPS, GroupCache and AppEngine Managed VMs. The system auto-scales and auto-discovers peers for GroupCache which caches thumbnails + image tiles and also ensures they are only generated once across all nodes. VIPS is used via CGO for fast image handling.</p>
<p>I'm using labstack/echo for the actual web-app which is a very nice + fast micro-framework. I tried lots of them, ended up settling on that one (although goji + gin were also good).</p>
<p>My previous experience is C# / .NET + some Java, Python and now Go. I prefer Go and find it the most productive to use.</p>
<p>There is a lack of 'large reference apps' but part of that is to do with the language and platform - it does encourage simplicity. Still, I think a lot of examples are a bit too simple. It doesn't hurt to separate services with interfaces and modularize handler endpoints instead of having everything in one package.</p>
<p>I find it good to question whether it's worth adding a dependency on a package or whether the std library is enough. Adding too many 3rd party packages does tend to make things feel more complicated.</p></pre>jamra06: <pre><p>There are a whole bunch of frameworks out there with examples. Of those frameworks, there are varying levels of handholding you can expect. </p>
<p>If you're trying to assess whether or not Go is capable, you have to build it yourself. If you are trying to find out how to write code with Go, try these links:</p>
<ul>
<li><p><a href="http://www.alexedwards.net/blog/organising-database-access" rel="nofollow">How to access a Database</a></p></li>
<li><p><a href="http://shadynasty.biz/blog/2012/09/05/auth-and-sessions/" rel="nofollow">One person's opinions about templates</a></p></li>
</ul>
<p>If you want to write complex business logic, that's up to you. I'm not able to share the code I write for my workplace.</p></pre>space-llama: <pre><p>I'm part of a project at Spring (shopspring.com) which builds extensive distributed systems in Go. We have basic API backends that mobile apps hit, a few CMS' for vendors and other data sources, several dozen background workers, you name it. Overall we have anywhere from 100,000 - 400,000 LOC of Go code (hard to count with various packages). </p>
<p>Tbh it's pretty boring and straight forward. Just about everyone makes their own error struct (don't do it because of special nil-value behavior by the Go compiler), has a hierarchy of <code>common</code> packages that can be imported by any package (to avoid hard circular dependencies), and a <code>types</code> package to share common types across services. We also have a monolithic codebase which is great for maintaining strong contracts between your services and being able to change how objects work across all of your services at a single time.</p>
<p>Our Go code mostly serves Angular to the frontend, so we don't have to deal with templates and stuff like that in Go. Gorilla/mux and context-based handlers make it super easy to add new endpoints. For SQL we use Postgres and gorp. It's ok and sanitizes most things.</p>
<p>Essentially the goal is so that when you have an idea to add a feature or create an endpoint, it should be incredibly boring and straightforward to implement. Don't use magic, keep it super simple, even verbose. </p></pre>QThellimist: <pre><p>Could you expalin the "error struct" more detailed. </p></pre>space-llama: <pre><p>So everyone makes <a href="https://godoc.org/github.com/dropbox/godropbox/errors#DropboxError" rel="nofollow">a custom error interface which is returned from all their functions</a>, with a custom error struct underneath. Essentially it makes it possible to store more info in the error like severity, stacktraces, additional info. It's important to compare the interfaces and not the structs. Basically it comes down to how Go does comparisons between interfaces vs structs/types. <a href="http://www.goinggo.net/2014/10/error-handling-in-go-part-i.html" rel="nofollow">Here's a good article</a> that describes how these things work.</p></pre>excited_by_typos: <pre><p>I run a real-time charting service for Bitcoin markets, and the backend is 100% in Golang. It gets quite a bit of traffic. <a href="https://cryptowat.ch" rel="nofollow">https://cryptowat.ch</a></p>
<p>It consists of HTTP + Websocket servers and has a lot of special logic for getting data from different exchanges. I also have contracts with some clients who asked for customized views of the whole thing, so the template logic is pretty complex.</p>
<p>The biggest gotcha was that it took me some time to get accustomed to how to write good templates in Go. I was unable to find any good examples of <em>complex</em> Go template setups (just simple stuff). But I didn't have too much trouble coming up with my own patterns because, like the language itself, the template syntax is so simple!</p>
<p>Database-wise, it uses persistent Redis for most data storage and MongoDB for some small user-related collections. I find most popular services like those have really well written Go drivers so it has never been an issue trying to integrate services.</p>
<p>Sadly none of it is open source, but I want to open it up in parts once it stabilizes as a business. But with regards to Go, yeah, I am very glad I decided to use Go.</p></pre>ronny10: <pre><p>wow nice site! how are the stats for your server?</p></pre>excited_by_typos: <pre><p>Not sure what you mean by stats. It's all hosted on Linode right now, currently there's 5 different servers for the whole stack. They're the 2048 size that Linode offers. They don't come close to hitting capacity, Go's resource efficiency is great.</p></pre>icodl: <pre><p>which css did you use? It looks quite nice</p></pre>ViperOrel23: <pre><p>I'm having good times and luck working with beego as a web application platform.</p></pre>mnsota: <pre><p>I've built a number of large web and server-to-server apis with go. I've found the middleware approach demonstrated <a href="https://github.com/rcrowley/go-tigertonic" rel="nofollow">here</a> to be an example of good api design with go.</p></pre>FootbaII: <pre><p>Maybe this isn't complex enough for your requirements, but take a look at <a href="https://github.com/sourcegraph/thesrc" rel="nofollow">https://github.com/sourcegraph/thesrc</a></p></pre>mrkaspa: <pre><p>I built this one in revel/go <a href="https://gotadevida.co/" rel="nofollow">https://gotadevida.co/</a> BTW is text are in spanish for the moment</p></pre>remigijusj: <pre><p>I have done a 5k LOC web project this year. No fancy Javascript, just plain page generation with <code>html/template</code>. I have used <a href="https://github.com/gin-gonic/gin" rel="nofollow">https://github.com/gin-gonic/gin</a> as my main engine for routing and middleware, <code>gorilla/sessions</code> and <code>bcrypt</code>for authentication and Rails-style "flash" notifications. Also <code>go-sqlite3</code> with <code>database/sql</code> for data handling, this is a low-traffic website, and Sqlite does the job nicely. Apart of that, just standard library. </p>
<p>File structure is flat, with separate files for <code>const</code>-based config and SQL queries, also separate files for controllers and models corresponding to domain objects and their (mostly-CRUD) operations.</p>
<p>While working on the project, i have borrowed some solutions and structuring conventions from Rails, because i know it well and it was easiest to just transfer the knowledge. My main annoyances were largely with Go being lower-level language so it's not possible to write code as concisely as it can be done in Ruby/Rails. </p></pre>Spirit_of_Stallman: <pre><p>Yes. Gorilla (mux, sessions) + middle + <a href="https://github.com/eknkc/amber" rel="nofollow">amber</a> (pure <code>html/template</code> is a painful when you need build a really big and complex front-end) + sql driver (<a href="https://github.com/go-sql-driver/mysql" rel="nofollow">mysql</a> (with <a href="https://github.com/ziutek/mymysql" rel="nofollow">mymysql</a> have some troubles)), and some previous experience in Go for right project architecture.</p></pre>LimEJET: <pre><p>Yes!</p>
<p>Me and a few class mates made <a href="http://manaak.campus.ltu.se:35289/" rel="nofollow">this site</a> for a project at uni (uptime may be spotty, it's on a dev server), which is an online multiplayer chinese checkers game with a Go backend.</p>
<p>It uses the gorilla muxer, with a mongoDB document store (which I'd switch out in an instant with what I know now, probably for something like Bolt). We also use websockets for communication between clients and Mozilla's Persona thing to handle login (that way nothing bad happens if the database is compromised).</p>
<p>The styling (because people tend to ask about it) is a bootstrap skin called <a href="https://github.com/kristopolous/BOOTSTRA.386" rel="nofollow">BOOTSTRA.386</a>.</p></pre>blueblank: <pre><p>The service itself is in an alpha mess, unpresentable state, but uses packages I've built: <a href="https://github.com/thrisp/flotilla" rel="nofollow">flotilla</a>, <a href="https://github.com/thrisp/security" rel="nofollow">security</a>, <a href="https://github.com/thrisp/djinn" rel="nofollow">djinn</a>, <a href="https://github.com/thrisp/fork" rel="nofollow">fork</a>.</p></pre>michaelbironneau: <pre><p>My current project is a Go web application using <a href="https://revel.github.io/" rel="nofollow">revel</a> for routing/session/caching/configuration and <a href="https://github.com/mattbaird/elastigo" rel="nofollow">elastigo</a> as an Elasticsearch client. </p>
<p>My experience:</p>
<ul>
<li>I like revel's app structure and separation of routes into a separate file is great for giving to other developers (it's really easy to understand the app structure just from that file)</li>
<li>Compared to writing web apps in Python with Flask, this was more work and more boilerplate CRUD. Like other posters I think I'll be looking into some reflection techniques in the future.</li>
<li>The vast majority of the content is rendered client side but the templating in revel was more than sufficient for my needs</li>
<li>Middleware support in Revel via <code>InterceptFunc</code> makes authentication a breeze. It also makes logging metrics really easy, although I had to copy paste some of the <code>expvar</code> code into a custom handler to get it to register. We use Datadog and it supports custom expvar metrics, which is great. </li>
<li>Elastigo exposes a pretty intuitive DSL for queries and it's easy to pass a query object around, adding stuff to it as you go (eg. restricting a user to a given filter). It lacks support for some more complex queries - made a PR for this, as did another guy, not sure if either of them will get merged anytime soon. Also if you use this library make sure to call <code>Close()</code> on the <code>Conn</code> object when you're done otherwise you'll leak memory. </li>
<li>The testing features of Revel are also great to work with - you can run test suites from the /@tests endpoint in a nice UI. I'm sure this is not the only web framework to offer such a feature but it's the first one I've come accross.</li>
</ul>
<p><em>edited to add the bit on testing in Revel.</em></p></pre>Bjorkbat: <pre><p>Well, building. It's a platform for hosting special videos and photos that can be texture mapped on a sphere so that you basically have a 360 x 360 perspective, as if you're actually there. With a VR headset the feeling only becomes more immersive.</p>
<p>I have a pretty simple top-level directory structure that grows more complex the more you tunnel down. View logic in the views directory, models (including the database calls I need for them) in the models directory, a directory for app settings, and another directory for "utility" stuff. With the views directory, the directory structure is broken down such that with profiles, I have a dedicated profiles directory, with a file dedicated to POST requests, a file dedicated to GET requests, additional files if there are other requests, and also a file dedicated to grabbing and validating form inputs. It's worked pretty well thus far.</p>
<p>This is a tad embarrassing, but I must admit I really hate Gorilla's sessions implementation combined with the library used for MySQL as a session data store because the damn thing didn't exactly work the way I expected it to (by contrast, the JWT library I'm using works exactly how I expect it to). So, I wrote my own session library instead.</p>
<p>I'm not afraid to say my own session library is probably crude, but takes into account basic security (the session id's have 64 bytes of randomness, or a 88 char length base64 string, is that excessive or not enough?) and has satisfied my own self-imposed goal of being pretty simple to understand. The catch is that it's use case is pretty limited because I built it to solve an immediate problem. I'm working on expanding it to be more general purpose while still keeping the simplicity goal in mind.</p>
<p>Question: Has anyone found any templating engine that's more similar to Jinja / Django's default templating engine? I come from Django, so while I haven't had any issues with Go's built in templating engine, I definitely miss the Jinja way.</p>
<p>EDIT: Blast, the OWASP cheat sheet says I should be using at least 64 <em>bits</em> of entropy for session ids. So I'm probably overdoing it here by a long shot, but damn if my session ids are unguessable now.</p></pre>CaptaincCodeman: <pre><blockquote>
<blockquote>
<p>Question: Has anyone found any templating engine that's more similar to Jinja / Django's default templating engine?</p>
</blockquote>
</blockquote>
<p>Have a look at <a href="https://github.com/flosch/pongo2" rel="nofollow">https://github.com/flosch/pongo2</a></p></pre>Bjorkbat: <pre><p>Nice, thanks for the heads up.</p></pre>mikedelfino: <pre><p>I wonder what are Go advantages over PHP for the vast majority of websites / web-fronted systems most people would come up with. I just don't see it. There, I said it.</p></pre>eikenberry: <pre><p>Enjoyment.</p></pre>anxiousalpaca: <pre><p>You don't have to write in PHP</p></pre>dhdfdh: <pre><p>The exact answer I gave my boss a few weeks ago.</p></pre>icodl: <pre><p>performance. how do 120k requests / second @5000 concurrent sound?</p></pre>mikedelfino: <pre><p>I see. I'll check it out again when I have more than a handful of concurrent requests. Thanks for pointing it out.</p></pre>trycat: <pre><p>I'm not sure that I understand what concurrency means - I'm working on something now that returns suggestions as the user types, so if I get a lot of users it's going to be hitting the server pretty hard. Would Go handle it a lot better than raw PHP?</p></pre>knockknockwhosethere: <pre><p>Performance of any system is measured by how you have implemented it, selecting faster language doesn't always result in faster programs. Like wise in your case, if you're creating db client instance for each request ( which will establish new connection to db at every request ) then none of the languages are going to help, and if you circulate same client instance across each request then performance will improve on php it self. If you want to compare language performance <a href="http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=php&lang2=go" rel="nofollow">http://benchmarksgame.alioth.debian.org</a> is good place ( How much these benchmarks matter to your solution is debatable, because benchmarks don't give whole picture ).</p></pre>septaaa: <pre><p>Concurrency meaning doing multiple independent things simultaneously. Specifically, each request to the server would be handled on its own goroutine (lightweight thread). Also, Go is a statically compiled language while PHP is a dynamic scripting language, so Go executes faster. Both those things being said, yes Go would handle most anything a lot better than raw PHP.</p></pre>icodl: <pre><p>2 concurrent requests for instance are when 2 requests are made at the same time. </p>
<p>In PHP, with <? echo "hello world" ?> on my machine I would get around 200 or would be able to handle 200 concurrent requests before erroneous responses appeared (non-200). I would we able to make ~17k requests/second.</p>
<p>In go with net/http I would start with 200 then increase to 400 1000 2000 5000. Even at 5000 concurrent requests it would not return a non-200 response. The sweet spot was around 2000. Everything past that resulted in slower responses, however no non-200 responses.</p>
<p>Now if you add something like Symfony I couldn't go past ~50 concurrent requests in PHP. To achieve a higher number of handled requests I would have to install an in memory cache like Varnish.</p>
<p>Now in order to achieve the same number of concurrent requests with PHP as you would with Go you would have to add additional servers increasing costs.</p>
<p>Downside is Go in comparison to PHP is rather low level. In PHP you have many helper functions. For instance to fetch a remote page you can just call file_get_contents("<a href="http://domain.tld/resource.html%22" rel="nofollow">http://domain.tld/resource.html"</a>). In Go you need to write more lines to achieve the same. Or you could create a library of PHP to Go functions that would allow you to do the same in Go. Or there is no in_array(), you'd need to write it yourself. There are more downsides but there are also upsides.</p>
<p>Real life number of requests on my cluster is more along 2000 concurrent with Go and nginx in front and mongodb in the back. Still good compared to PHP and in general.</p>
<p>To answer your question, yes it would.</p></pre>
