If I wanted to store lists of data in boltdb that could be fed to html/template, what would be the best performing format? I wanted to avoid JSON reflect overhead if possible.
评论:
THEHIPP0:
benbjohnson:I currently use the faster implementation of Protocol buffers, but you should check if
gob
or evenJSON
isn't fast enough for you.To clarify: I only use protobuf because instead of rendering templates I use gRPC and had to write the protobuf defintions anyway.
inhies:I use gogoprotobuf for encoding when I use Bolt in applications. There are options to use the
unsafe
package which lets you avoid some allocations.I use gogoprotobuf for this application and my decoding time is about 12µs per object (and that converts from the internal protobuf type to an application type). Those benchmarks are coming from a small $5/mo DigitalOcean box too so it's not a beefy server.
If you really need performance then you could also use a generated template language like ego.
elithrar_:JSON is great for debugging while building your application, gob will be faster though.
inhies:encoding/json is nearly always faster than gob, if you are using once-off encoders—which is likely when encoding templates. gob is sometimes faster when you're using it as intended: for streaming encoding.
elithrar_:I based my comment off of these benchmarks:
BenchmarkGobEncode 1000000 2191 ns/op
BenchmarkJsonEncode 500000 4738 ns/op
BenchmarkGobDecode 1000000 2019 ns/op
BenchmarkJsonDecode 200000 12993 ns/op
From https://npf.io/2014/07/intro-to-boltdb-painless-performant-persistence/
guesdo:encoding/json will be much faster than html/template, so I wouldn't discount it. template parsing is (relatively speaking) pretty slow due to escaping requirements.
Alternatively, you could use something like ffjson (https://github.com/pquerna/ffjson) to generate Marshal/Unmarshal methods on your structs and avoid reflection where possible, noting that reflection isn't always the performance killer it is perceived to be. As I touched on before, template rendering and DB access will be the bottlenecks.
seufert:https://golang.org/pkg/encoding/gob/
That might help. Although I would just use JSON or BSON or MsgPack.
I a Go only solution is sufficient then just use Go. It's the most convenient solution to use: No external description of your datastructure needed, best support for datatypes (afaik no time.Time on Protobuf for example) and the performance is much better than json. It's also quite conventient to add or remove fields.
