<p>Hey guys,</p>
<p>I am using nodejs now, and utilizing multiple node websocket instances to scale with Redis. I really don't like that since memory is not shared and I have to utilize a ton of Pub/Sub signals across instances... It just doesn't seem intuitive to me. (maybe this is normal, maybe it's not I don't know)</p>
<p>Long story short.... I am brand new to Go, and coming from a JS background.. The syntax seems kinda different, but I have no problem spending time learning it.</p>
<p>I basically am wondering if I do make the switch, would I still need to spin up separate 'Go' instances to take advantage of multi-core servers? And would I still need to use Redis to share memory across the processes?</p>
<p>Edit: Btw, our game-server is very large so I would need to spend some time learning Go (as I said). But, if it's worth it.. I'm definitely ready :)</p>
<hr/>**评论：**<br/><br/>mtrn: <pre><blockquote>
<p>basically am wondering if I do make the switch, would I still need to spin up separate 'Go' instances to take advantage of multi-core servers? And would I still need to use Redis to share memory across the processes?</p>
</blockquote>
<p>No. Go supports concurrent execution through goroutines. You can think of them as very lightweight threads. They are multiplexed onto native threads. The Go runtime will schedule the goroutines, similar to an operating scheduling threads.</p>
<p>A Go <a href="https://www.youtube.com/watch?v=PAAkCSZUG1c">proverb</a> says: Share memory by communicating. You can share a value between concurrent parts of you program by sending and receiving on channels. These are anonymous conduits which in combination with the select statement form the <a href="https://www.golang-book.com/books/intro/10">basic concurrency primitives</a>.</p></pre>jringstad: <pre><blockquote>
<p>similar to an operating scheduling threads</p>
</blockquote>
<p>One very important difference to keep in mind is that they are not scheduled preemptively, meaning that long-running go-routines can "clog up" the schedulers and subsequently the latency of other, short-lived go-routines (e.g. request handlers et al) can skyrocket.</p>
<p>Workarounds can be to yield inside a long-running go-routine if possible (or to do other kind of work that will invoke the scheduler), to spin the long-running work out to a different process that is scheduled pre-emptively, etc. (Maybe LockOSThread() can also work, never tried/investigated it.)</p>
<p>But same issue in node.js, so OP should already be aware of that challenge.</p></pre>bkeroack: <pre><blockquote>
<p>One very important difference to keep in mind is that they are not scheduled preemptively</p>
</blockquote>
<p>While strictly true, it's not often an issue because the Go runtime can switch goroutines a) at every function call, and b) at several language keywords (select, etc) (not an exhaustive list of scheduling points, but you get the idea).</p>
<p>So generally even a collection of CPU-bound goroutines are scheduled reasonably fairly.</p></pre>jringstad: <pre><p>Yes, that's true, but it can occasionally happen when you have some sort of long-running [e.g. multi-dimensional] loop that is highly optimized (so no function calls anywhere.) Optimizing e.g. numerical code will mostly push you to move away from possible scheduling points, since all of those are associated with slow operations, so it's good to be aware of it, and -- if necessary -- take charge of the problem manually.</p></pre>bkeroack: <pre><p>Basically I agree. I've also seen cases with, eg, a HTTP server that spawns 10k CPU-intensive goroutines (admittedly a pathological case) and then has degraded request latency because the scheduler doesn't know how to distinguish between the small number of latency-sensitive connection goroutines and the large number of non-sensitive processing goroutines. </p>
<p>Of course you can avoid this with better application design.</p></pre>earthboundkid: <pre><p>Yeah, it never really makes sense to have more Go-routines than whatever the hardware bottleneck is. For CPU-limited things, there's no point in having number of cores + fudge factor workers. For web requests, it's hard to do more than a couple dozen simultaneous requests. File IO is limited by the number of open file descriptors the system allows. Generally speaking, when you spawn a goroutine, you should think about how many of them might end up being open at once, and put in a work queue or semaphore or something if there's a chance you'll end up with pathological number of them open. Remember that just spawning a go-routine costs 2KB of RAM: not a lot on a modern machine, but it will add up to 2GB if you spawn 1 million of them.</p></pre>bkeroack: <pre><p>Rewatch "Concurrency is not parallelism". Concurrency [number of goroutines] should reflect the problem domain and good code design, while parallelism can be independently and simultaneously controlled via GOMAXPROCS [OS threads] depending upon hardware, etc. They are two separate concerns that you can adjust independently on an M:N system like the Go runtime.</p></pre>RevMen: <pre><p>I moved from node.js to Go because of the need to do something similar. I had an elaborate system worked out in node for sending data between threads and it worked well, but it was so much effort to get it right.</p>
<p>I can accomplish the same things easily in Go using channels. No spinning up workers or interacting with Redis or RabbitMQ. You do it all in your Go code using Go's excellent tools for concurrency.</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>No spinning up workers or interacting with Redis or RabbitMQ. You do it all in your Go code using Go's excellent tools for concurrency.</p>
</blockquote>
<p>This seems like just what I want... For example, our gameserver is behind a load balancer that round robins towards 3 node instances. When a player joins a game and if the other player is on a different node instance guess what? The entire life of that game for all movement data, player skills, cooldowns, hp regeneration, everything all has to utilize Redis' Pub/Sub just to talk between those two node instances. The solution? I could disconnect the player that is about to join that game, and direct them to a different directory/upstream block on nginx that connects to that specific node instance (port).... It just seems so silly to do all this. I'm looking forward to Go now and appreciate your feedback. Seems like it's just what I need</p></pre>cheesechoker: <pre><p>Are you planning on replacing your 3 node instances with single Go-based instance of the game server though?</p>
<p>What if, for example, the game gets more popular and you need to add additional hardware? Or you decide to add another instance of the server for fault tolerance? Then you're back to needing synchronization.. A Go implementation might make this easier to manage, but it doesn't remove the problem entirely.</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>A Go implementation might make this easier to manage, but it doesn't remove the problem entirely.</p>
</blockquote>
<p>I was just thinking that single Go app for 1 physical server would be sufficient [and more appropriate]. (instead of having to manage multiple node instances per server) -- Then, use Redis for Pub/Sub to communicate between physical server's to scale later? (I'm not entirely sure here)</p></pre>hayzeus: <pre><p>He's right though -- you need to think about scaling. Sure, for now you can use channels for player communications, and go will use all cores by default anyway -- but if you need to scale, you'll still need something. Which means from a design standpoint, it might be better to do all of your communications through pub-sub anyway, even in-process (Go has excellent redis support). Then you could scale up transparently.</p></pre>balloonanimalfarm: <pre><p>Luckily Go provides really nice interfaces; so if you can find a clean abstraction you can worry about that later. If you no longer have redis pub/sub locally and everything is in the same memory space you'll be able to scale a lot longer before needing to move concurrently (if ever).</p>
<p>While you're looking into pub/sub frameworks, take a look at <a href="https://github.com/nats-io/gnatsd" rel="nofollow">gnatsd</a> as a pub/sub. The <a href="http://bravenewgeek.com/dissecting-message-queues/" rel="nofollow">throughput</a> is very impressive.</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>I moved from node.js to</p>
</blockquote>
<p>Hey, I forgot to ask. Since you came from nodejs as well, do you have any advice for a fellow JS dev to help with the migration process? Maybe things you wish you would of knew before switching, etc? The only thing I am worried about is the syntax differences. But, I think I will do fine once I start actually reading / learning.</p></pre>RevMen: <pre><p>So far I haven't run into any gotchas or common pitfalls. I spent some time watching videos to learn syntax and general good practice and I really didn't have much trouble writing good code.</p>
<p>It was pretty easy for me to understand the Go way of doing things, at least well enough to do the work I've done in the last couple of months. I think it's a lot like Python in that way. I haven't had to think myself out of any corners like the massive callback waterfalls I would have in node.</p>
<p>The thing I spent the most time trying to understand was interfaces, but that wasn't too bad, and it's actually something you can work without if you haven't got them figured out; it is better to use them, but you can put off understanding them until you've written enough Go code to see what the point of them is.</p>
<p>It also takes some time to understand the different concurrency tools. But if you're coming from doing concurrency in node assisted by Redis, this should be very easy for you to understand. There are just a handful of tools you need to understand, including mutexes, wait groups, channels, and go routines. All of them are simple to implement and pretty easy to understand. This was the biggest improvement, by far, coming from node.</p>
<p>When I was first learning the syntax, I found <a href="https://youtu.be/CF9S4QZuV30" rel="nofollow">this video</a> very helpful. He runs through pretty much all of the Go ways of doing things in less than an hour. I watched it through once, spent a couple of days trying things out by example, then watched it again and much of what was in the video stuck with me. I now have little trouble just writing code without looking stuff up (I started Go in February).</p>
<p><a href="https://gobyexample.com/" rel="nofollow">Go by Example</a> is also very helpful for a quick reference of how things should look.</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>When I was first learning the syntax, I found this video very helpful</p>
</blockquote>
<p>That's hilarious how you just mentioned him. <a href="https://i.gyazo.com/ac95414ffc5ff1c30f1e101d7a55cd1e.png" rel="nofollow">I'm watching</a> that right now, haha!</p>
<blockquote>
<p>It also takes some time to understand the different concurrency tools. But if you're coming from doing concurrency in node assisted by Redis, this should be very easy for you to understand. There are just a handful of tools you need to understand, including mutexes, wait groups, channels, and go routines. All of them are simple to implement and pretty easy to understand. This was the biggest improvement, by far, coming from node.</p>
</blockquote>
<p>Sounds great. I am actually starting to get used the syntax now. When writing a condition I start typing <code>(</code> and then realize.. STOP! </p>
<p>I am starting to get a hang of the syntax now, but, it will take weeks and maybe months to start subconsciously writing it. (As I do with JS already). But, I really like it thus far. The arrays / object are killing me atm though.</p></pre>RevMen: <pre><p>Embrace the struct and the slice. You'll rarely deal with arrays directly.</p>
<p>The thing about Go is you need to put your effort into configuration. It's worth the time to build out your structs completely and beautifully because it makes using them very easy.</p></pre>weberc2: <pre><blockquote>
<p>would I still need to spin up separate 'Go' instances to take advantage of multi-core servers?</p>
</blockquote>
<p>One Go program can fully utilize all cores without needing to spawn new processes. The Go runtime takes care of multiplexing goroutines (lightweight threads) to native operating system threads while preserving a shared memory environment.</p>
<p>I think you'll find parallelism concerns to be much easier in Go, though adapting to a statically-typed environment and Go's idioms in particular may take some patience.</p></pre>iends: <pre><p>I'm a Node.js developer in the day job. </p>
<p>Go is strictly better than JavaScript except for the package management story and the package ecosystem (go is quickly catching up). (Unless you're writing a one off script.)</p>
<p>You might find this talk interesting: <a href="https://www.twilio.com/signal/2015/videos/high-performance-messaging-with-golang-and-redis" rel="nofollow">https://www.twilio.com/signal/2015/videos/high-performance-messaging-with-golang-and-redis</a></p></pre>Khornettoh: <pre><p>It depend of what kind of data you want share between instances, but I don't think that you gonna need redis, go's channels rocks. And using go routine, you will take advantage of multi-core servers (-:</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>It depend of what kind of data you want share between instances</p>
</blockquote>
<p>Not sure if this is the correct term in Go, but, basically, I want to share something similar to a JavaScript Object. I just hate how we have to spin up multiple instances of our app to utilize multi-core servers and then ontop of that, we need to use Redis to communicate between them. I just feel like I could be using something better.. it's a weird feeling</p></pre>silvea12: <pre><p>I am switching from node to golang for the exact reason you are. My case is a chat system with 3-4k users on at once. Uses redis pub/sub and storage. Golang is a great choice :D</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>My case is a chat system with 3-4k users on at once. </p>
</blockquote>
<p>Damn, nice. Is that just one server with 1 Go app running? Again, sorry guys if my questions seem out of line with the "Go app" remarks, I'm just so used to the <code>nodejs's</code> "multiple instances" design. I keep thinking I will need to run multiple Go App's on 1 server, but that's not the case?</p></pre>varun06: <pre><p>No question is retarded. That's how we learn.</p></pre>silvea12: <pre><p>It is currently still node in production, I am still in the process of rewriting. It spins up 3 cluster children plus one master to recover and deal with some requests it makes to our internal API, along with allocating connections to the children.</p>
<p>I plan to have golang to just be one process, as golang will automatically spin up more threads if it needs to (as it has done in small scale tests)</p></pre>kaeshiwaza: <pre><p>You will see often that we use multiple Go Apps talking to each others, but it's not mandatory it's just a design decision about microservices.
But it's totally good to run only one App, it will be very efficient very easy to deploy and will use all the core of the system just by using goroutine. Even if some goroutine are waiting for cpu or io !</p></pre>turkish_gold: <pre><p>I'm always curious about chat apps (since I'm writing one myself in my free time). Is your chat system open source or paid? Do you have a link to it?</p></pre>Khornettoh: <pre><p>You can send "interface{}" through a channel if you want to send different kind of object with the same chan, and you can make channel of your custom type object too (to send only this type/object).
But there is something that I'm not sure. Like, you need to share a data between multiple instance process, or you need to send data from an instance to another ?
anyway, go routine + channel = win ;)</p></pre>BillOReillyYUPokeMe: <pre><blockquote>
<p>Like, you need to share a data between multiple instance process, or you need to send data from an instance to another ?</p>
</blockquote>
<p>Oops, my bad. I am just getting the responses now to this thread. I guess I don't need to run multiple instances, so I wouldn't really need to share anything. (Which is one huge reason why I want to move away from node to Go ), so that's a huge plus. I'm fine with Redis Pub/Subbing for communication between physical server's, but I feel like using that with multiple instances on node just seems so counter-intuitive. I am getting pretty hyped about Go right now though :)</p></pre>Khornettoh: <pre><p>Hehe, Go is really cool !</p></pre>hayzeus: <pre><p>Go is a good replacement for node. It's my main use case here at work</p></pre>Gacnt: <pre><p>I currently use both. I'm kinda biased towards Node just because I really love the language (JavaScript) and I just find it more fun to write and the ecosystem with npm it's really easy to find things to get stuff done quickly.</p>
<p>Especially with new features like async/await so you can write more flowing code rather than constant .then chains or callback hell. </p>
<p>Where I found Go surpassed Node was not really noticeable at all, I had rewritten an app I had originally done in Node (this was a party system for a game that has over 20k concurrent users a day) in Go. I was expecting to see an insane increase in speed just because the language was compiled and I had already read about how it was supposed to be fast. </p>
<p>But switching between the node / go app I couldn't really find any areas where go was superior. </p>
<p>Go has great concurrency management with its Channels and WaitGroups but then again Node has packages like async which can help accomplish basically the same tasks. </p>
<p>Like most people will say, go excels because of its channels and goroutines because they can spin up and can take almost less than a kb of space to run so you can have 100000 of these routines running all doing whatever. </p>
<p>Unless you have a strict task that you absolutely need to use a strict typed low level language like Go, I don't really see the need to do so. I've been using Node for about 4 years, and Golang for about 1.5 years. </p>
<p>I still have much to learn when it comes to go, but the main thing that really is keeping me from standing on the side of Node has to be a lack of something like npm, go get just feels so cheesy, it's just a buttered up curl download script is how it feels. Go offers no version support for your packages which makes me sad. </p>
<p>Also the biggest gotcha I suggest you watch out for when learning is to make sure you use exported fields in your structs when marshalling or unmarshalling json, or you'll be scratching your head for a while. </p>
<p>There is zero downside to wanting to try a new language so good luck and hopefully it works out for you. </p></pre>jax024: <pre><p>I'm a node developer, also looking to switch to go. My main likes for node is how well it "talks" with my JS front end with things like socket.io, can anyone comment how easy or friendly it is to build a react and web socket frontend with a Go server handling all my socket and REST functionality?</p></pre>eyecikjou567: <pre><p>Using <code>revel</code> you can fairly quickly bootstrap a REST Api, took me like a week or so to get some simple website setup.</p>
<p>I recommend using <code>gorm</code> to map go easily into a relational database.</p></pre>dlsniper: <pre><p>A week if you are new to the language. A few hours if you don't use any frameworks </p></pre>eyecikjou567: <pre><p>The week was also including the entire frontend, ie HTML, CSS and JS (read: some form of AngularJS abuse) to glue together a nice little website.</p></pre>imS0ul: <pre><p>Well I'd say the only challenge you'll face is the languages' strict nature, but you should be good.</p>
<p>JS is slow. If you need / want speed, learn Go.</p></pre>KenjiTakahashi: <pre><p>Technically, you don't spin up 'Go' instances at all, as Go is compiled to bytecode and then you run that (you don't need Go installed on your server at all).</p>
<p>That aside, no, you don't have to run multiple separate instances of your app to utilise multiple CPUs.</p></pre>CaffeineComa: <pre><p>Not bytecode, but a native executable. "bytecode" has a different meaning, e.g. Java or UCSD Pascal. </p></pre>KenjiTakahashi: <pre><p>Right! I must have wanted "machine code" out there.</p></pre>
