Best go database for . . . .

<p>I&#39;m looking for recommendations for a database. What I&#39;m going to store is time-based records. (Kind of like log lines).</p> <p>But, the records will contain multiple key-value pairs. I was thinking of using something like couchdb, but, I&#39;ve had issues in the past getting slices of large amounts of data from couch. (IIRC, it was doing the slicing after transmitting all of the data, instead of slicing at the server)</p> <p>Thoughts? In couch, I&#39;d convert the key=value pairs to json, and just store it. I&#39;m not sure how to represent it best in normal tables / sql.</p> <p>What I do need, at a minimum, is an ability to retrieve a date range of records.</p> <p>EDIT: </p> <p>Followup with more details: I&#39;m capturing the output from a Jucebox car charger from emotorwerks.com. It sends a UDP telemetry message every 10 seconds (8640 small messages w/ smaller responses per day) I&#39;m planning on storing up some data, and creating custom graphs / queries. So, I will be querying mostly by time slice, and deciding what to graph.</p> <p>The telemetry message is about 50 bytes, and the response about 15. </p> <hr/>**评论：**<br/><br/>lasizoillo: <pre><p>Take a look to <a href="http://influxdb.com/">influxdb</a> which is written in golang.</p></pre>FEiN: <pre><p>This sounds like a good fit for Cassandra, and a driver <a href="https://github.com/gocql/gocql">gocql</a>. I would also recommend checking out storing time series data with Cassandra</p></pre>singron: <pre><p>Range queries are really efficient in b+tree indexes. Any database that uses those will be pretty good (postgres, mysql, etc.).</p> <p>Edit: Postgres has json and jsonb datatypes, so if the keys are dynamic, you can use that. If the keys are static, just make them columns. You could also try having a key value pair table that belongs to a main record table, and use relational semantics to implement dynamic keys. The query plan starts to look more complicated in this case. You could try several different strategies and see what is best under your conditions.</p></pre>xargon7: <pre><p>The data is so small you could store it in memory and occasionally write out to a JSON file. You said you have about 65 bytes -- allowing some overhead, maybe we&#39;ll have 300 bytes total per sample. </p> <p>8640/day * ~300 bytes * 1 year = ~1GB /year</p> <p>That&#39;s not much, and you can query by anything you want.</p></pre>anacrolix: <pre><p>Sqlite3.</p></pre>dlsniper: <pre><p>PostgreSQL + jsonb as storage and gin / gin with json_path_spec (or what&#39;s it called) will get you to a working app and then you can easily decide how to best query and cache your data. Plus it&#39;s extra fun to run joins on top of json fields at basically the same cost as a normal storage (columns) mode. You also get all the benefits of existing tools built for PostgreSQL and you can focus on your app not on how to safely backup and restore the data (for example). </p></pre>Entropy: <pre><p>You&#39;re going to get 12 different replies from 10 different people because the use case is extremely common - nearly anything will satisfy. I could easily recommend something as simple as SQLite based on those requirements, because I don&#39;t know what you&#39;re trying to use this for, what you&#39;re connecting with, how much you&#39;re writing, what the read vs. write access pattern will be, how long the data needs to be retained, etc.</p></pre>Codemonky: <pre><p>Just added a followup with more details.</p></pre>Entropy: <pre><p>If you&#39;re just logging the data from one of them, then SQLite is more than enough. It would only take one or two tables.</p></pre>Codemonky: <pre><p>That&#39;s where things get a bit ugly. I&#39;m not sure how I would use sqlite. </p> <p>Obviously, I could do something like this table: * Timestamp * From Server? (boolean) * Payload (string)</p> <p>But, that means I have to parse out the data in the payload every time. The payload has N Key-Value pairs, that I&#39;d like to store individually in the database. That makes it a bit harder, because you&#39;ll have to have a table of the key value pairs, Joined with the main record. I couldn&#39;t figure out a way to make it clean. That&#39;s why I wanted to store json, but, not store json raw, but in a searchable format (ala couchdb) </p></pre>Entropy: <pre><p>You could denormalize to id, timestamp, from_server, doc_id, key, value. Indexes on (key, timestamp) and (timestamp). doc_id used to group k-v pairs from the same log event together.</p></pre>Codemonky: <pre><p>That&#39;s exactly what I was thinking, I just didn&#39;t use the right terminology :)</p> <p>So, I&#39;m eliminating these possibilities:</p> <ul> <li>postgress (sqlite should be fine -- and easier)</li> <li>cassandra -- couldn&#39;t get it working on OS X for my PoC</li> </ul> <p>I&#39;m going to try all of these, and see how they work:</p> <ul> <li>Sqlite3</li> <li>couchbase</li> <li>influxdb</li> </ul> <p>I&#39;ll post a github link when I&#39;ve got something useful.</p></pre>Entropy: <pre><p>Good luck! I&#39;ve messed with couchbase at work before. Performance is phenomenal, but the hardware requirements are astronomically higher than sqlite - they are almost diametrically opposed in what they provide. I wouldn&#39;t bother with it unless you need that much scalability (or maybe you just want to mess with it for kicks, which I totally understand :)</p> <p>You may also want to take a look at rethinkdb. Haven&#39;t used it yet myself, but its design and interface are interesting. Hardware reqs are also a lot lower than couchbase.</p></pre>