<p>I'm wondering if anyone has any ideas to solve my problem.</p>
<p>I have a Go app serving HTTP requests. I have this Go app running on several machines. I need some way to coordinate between them such that each request gets a unique, incrementing ID with no gaps. I'm trying to minimize the blocking that the requests need to do, and minimize the cross-chatter between servers.</p>
<p>Right now I'm thinking that there is a Ticker Server that generates sequential ranges of ID's and the Go app requests a batch when it's running low, generating all the ID's in that range and pushing them onto a channel. Each request in that Go app then pulls an ID from a channel.</p>
<p>Anyone have a better architecture?</p>
<hr/>**评论:**<br/><br/>ultra_brite: <pre><blockquote>
<p>incrementing ID with no gaps</p>
</blockquote>
<p>give up that requirement because you're just asking for trouble. Just use uuids just like any distributed system. If all your servers rely on a unique third party then your architecture isn't distributed. It is centralized.</p></pre>CaptaincCodeman: <pre><p>Seems like a bizarre and pointless requirement but you could just use a database with an identity column. Each request does an insert to retrieve the next number.</p>
<p>It's a terrible idea.</p></pre>niosop: <pre><p>How stringent are the various requirements?</p>
<p>Trying to do all 3 with solid guarantees is hard. Getting batches from a central tracker is fine, but each app will need to persist a record of the current batch and which have been used, otherwise you'll end up with gaps in the case of an application crash.</p>
<p>If the "no gaps" rule isn't that important, then you can do away with a central tracking server and just give each application it's own prefix or suffix to use. So Server-01 would issue 000101, 000201, 000301, etc. Server-02 would issue 000102, 000202, 000302, etc. Then they just need to keep track of where they are by writing it out to file every time they use a number.</p>
<p>If unique is more important than no gaps, then write out the last issued before handing it off to the requestor. You'll have a gap if the service crashes between writing the record and giving it to the requestor, or if the connection dies, or any other number of reasons. It's why exactly once delivery is difficult.</p>
<p>My recommendation is to relax the no gaps rule if possible, then you'll be able to satisfy the uniqueness constraint more easily.</p></pre>danredux: <pre><p>The no-gaps rule only needs to stand if there are no crashes or issues.</p></pre>qu33ksilver: <pre><p>It would be nice if you can give further background about this requirement. I am thinking, can we come up with a composite sort of ID like server_name+timestamp which allows you to know which server it came from and at the same time indicates order.</p>
<p>Or if you are just entering all of this data at a central DB, just use a identity column which will auto-generate increasing nos. and store them.</p></pre>sairamk: <pre><p>Your approach sounds reasonable.</p>
<p>If your app is consuming say 10k a second, distribute 600k in a separate go routing which prepares the ids and fetches them every minute or when required.</p>
<p>Central server could be a mutex to decide. If you don't want a single central server, you could trying using some consensus to elect a leader among your hosts which generates these ids and coordinates. every point of time everyone knows what the state is and can be elected leader. </p>
<p>In the second approach, you need to take care of ensuring the sub-system that cannot be reached would not allocate new batches of increments among themselves.</p></pre>programfog: <pre><p>I second the suggestion by ultra_brite about using uuids.</p>
<p>I found this slide deck useful when looking at a similar problem in the past</p>
<p><a href="https://www.slideshare.net/mobile/davegardnerisme/unique-id-generation-in-distributed-systems" rel="nofollow">https://www.slideshare.net/mobile/davegardnerisme/unique-id-generation-in-distributed-systems</a></p></pre>Matthias247: <pre><p>If there's no need on incrementing then GUIDs are a common solution for this problem. If you need this incrementing IDs then you will also need coordination between all those machines. E.g. store the last ID in a database, redis, etc. This will come with a serious performance hit and lots of reliability questions (how can you scale it, what do you do if the central server is not there, ...)</p></pre>gizzlon: <pre><p>Hm.. What do you mean by "no gaps" ? Does it mean that every request-id must be in-order across all the servers? So the first request to server A gets 1 and then the next request, to server B, gets id 2? If this is actually what you require I think I would just to one global counter in Redis. It's probably fast enough to touch on each request, simple to understand and implement and gives you the strongest guaranties. Only problem is that it will be a single point of failure..</p>
<p>Edit: Redis docs: <a href="http://redis.io/commands/incr" rel="nofollow">http://redis.io/commands/incr</a></p></pre>
![](https://static.golangjob.cn/static/img/banner.png)