Looking for a KV store (etc vs consul)

<p>I&#39;m need to chose between the two. I understand that consul offers service discovery in a more unique way however for the moment I only care about the KV Store feature.</p> <p>Does one offer more than the other?</p> <p>Edit: I&#39;m looking beyond a bet project and would prefer to understand the benefits of such databases in a production environment. I want to run this in a large cluster, multiple data centers, and have high data integrity.</p> <hr/>**评论：**<br/><br/>lobster_johnson: <pre><p>Consul is indeed better for service discovery. It&#39;s more specialized than Etcd.</p> <p>Etcd is great, but it&#39;s worth pointing out that while it implemented as a generic KV store, its intended use is similarly narrowly focused on distributed consensus and metadata: Locking, config, election, service discovery, coordination, etc. So far I&#39;ve not heard of anyone using Etcd as a backing store for content. Pretty sure it is not optimized for that. In particular, Etcd has some limitations on the sizes of values and of databases. They currently <a href="https://github.com/coreos/etcd/issues/2992" rel="nofollow">recommend</a> keeping the database below 25MB in size.</p> <p>You might want to look at Redis, MongoDB or CouchDB, all three of which are consistent and support optimistic locking of documents. They can be used as simple KV stores if you don&#39;t need all the bells and whistles.</p></pre>funny_falcon: <pre><p>That recomendation was for v2 storage. v3 storage is more memory efficient.</p></pre>inhies: <pre><p>Bolt?</p></pre>Smokey_Circles: <pre><p>We use consul at our office.</p> <p>Distributed mutex via consul KV store. Just ridiculous how well it works. </p></pre>sairamk: <pre><p>HashiCorp has a bunch of tools which solves most of the problems of managing large and multi data center clusters (<a href="https://www.hashicorp.com/terraform.html" rel="nofollow">https://www.hashicorp.com/terraform.html</a>)</p></pre>dlsniper: <pre><p>Read this first: <a href="https://www.consul.io/intro/vs/zookeeper.html" rel="nofollow">https://www.consul.io/intro/vs/zookeeper.html</a></p> <p>Then I suggest looking into a different technology for your datastore as neither of those are a good fit for what you need. </p> <p>I&#39;d choose Postgresql as it has a kv engine to but it can work with a json document just as well (even better than solutions like mongo for example).</p></pre>allhatenocattle: <pre><p>OP mentioned multiple data centers. If they need to be active-active I don&#39;t think Postgresql will fit that requirement.</p></pre>dlsniper: <pre><p>I&#39;ve missed that part. Then I guess Cassandra will be a better choice. Anyway neither Consul or Etcd make any sense whatsoever for the scenario proposed but it&#39;s funny to see how many people would abuse them. </p></pre>tscs37: <pre><blockquote> <p>how many people would abuse them.</p> </blockquote> <p>software is created to be abused until the bitter end. that is it&#39;s only purpose.</p> <p>sometimes it also serves practical stuff as a secondary side-effect.</p></pre>alecthomas: <pre><p><a href="https://github.com/docker/libkv/blob/master/README.md" rel="nofollow">libkv</a> has a comparison matrix which may be useful.</p> <p>Edit: and of course, if you&#39;re primarily using Go then you could just use libkv to try different backends without significant engineering work.</p></pre>fthedill: <pre><p>This is simply a library though. I&#39;m rather interested in actual lifecycle anecdotes. Is one easier to maintain in the long run? Does one often crash?</p></pre>allhatenocattle: <pre><p>To help people understand your requirements, what does &#39;large cluster&#39; mean to you? How many keys, how big are values, what are peak read and write rates?</p></pre>fthedill: <pre><p>Not huge values, but many many keys (millions)</p></pre>allhatenocattle: <pre><p>If you are talking about very large data sets (many millions or billions of keys) you might want to also look at Cassandra, which has a great story for large cluster sizes, active-active for multiple data centers, high data integrity. While it models data as row/column, it can certainly be used as k/v.</p></pre>bonekeeper: <pre><p>This just popped up today, might be useful for you if your data can fit in RAM: <a href="https://github.com/tidwall/summitdb" rel="nofollow">https://github.com/tidwall/summitdb</a></p></pre>