<p>I am working on an app that will be built using a bunch of microservices.</p>
<p>I have decided to use prometheus for telemetry and monitoring. There are tons of telemetry/metrics libraries available, some very popular and some not so popular.</p>
<p>I have ruled out gocraft/health, because it pushes to a PushGateway instead of exposing an HTTP endpoint for prometheus to scrape.</p>
<p>That leaves the <a href="https://github.com/prometheus/client_golang/tree/master/prometheus" rel="nofollow">official prometheus client</a> and <a href="https://github.com/go-kit/kit/tree/master/metrics" rel="nofollow">go-kit/health</a>.</p>
<p>go-kit/health might seem attractive in the sense that it tries to provide a uniform interface so that you can easily swap out backends (prometheus to statsd, etc). But, even then, you would still be using the prometheus adapter within go-kit and if you used the prometheus summary for a metric, that might not be easily translatable to statsd.</p>
<p>Is there any value in using an abstraction layer like go-kit/metrics, or am I better off just using the prometheus client?</p>
<hr/>**评论：**<br/><br/>callcifer: <pre><p>We are using the official Prometheus client with a high traffic (~1.2 billion reqs / day) application. It mostly works well, without noticeable overhead. The only significant problem is the <a href="https://github.com/prometheus/client_golang/blob/f6b16ed25641d9035a0847266c01325df0402035/prometheus/http.go#L78" rel="nofollow">HTTP instrumentation</a>. It causes <em>significantly</em> increased CPU load. We eventually replaced it with a custom, lightweight (minimal allocations) middleware which solved the issue.</p>
<p>I should have some before/after flamegraphs somewhere, I'll see if I can find them.</p></pre>Bromlife: <pre><p>Any chance of publishing the custom middleware?</p></pre>F21Global: <pre><p>Although I definitely won't be hitting that amount of traffic yet, I am really interested in seeing your graphs. Any chance you can share your middleware? :)</p></pre>callcifer: <pre><p>I'll see if I can find those graphs, this was a while ago. I'd <em>love</em> to share the code, but I'll have to clear it with higher ups :/</p></pre>netingle: <pre><blockquote>
<p>HTTP instrumentation</p>
</blockquote>
<p>We also developed our own prometheus http middleware, if its of interest: <a href="https://github.com/weaveworks/scope/blob/master/common/middleware/instrument.go" rel="nofollow">https://github.com/weaveworks/scope/blob/master/common/middleware/instrument.go</a></p>
<p>In particular we were trying to filter out the request duration of long lived websocket requests.</p></pre>peterbourgon: <pre><p>If you've decided to use Prometheus and that will never change, the advantages of using go-kit/metrics are not many.</p>
<p>—</p>
<p><a href="/u/bbrazil" rel="nofollow">/u/bbrazil</a></p>
<blockquote>
<p>it's trying to be the lowest common denominator so key features of the Prometheus client are lost such as full floating point support </p>
</blockquote>
<p>Prometheus is the design inspiration for metrics; in retrospect I think it was a mistake to try to change the data types and I'll probably move to floats in the near future.</p>
<blockquote>
<p>It also still requires you to construct all your Prometheus metrics, so you're not really making things simpler when it comes to changing later.</p>
</blockquote>
<p>No, it's definitely simpler: changing the implementation is changing one block of code in the func main, rather than changing all the callsites. That's the whole point.</p></pre>bbrazil: <pre><p>You're always better off using a Prometheus client directly.</p>
<p>Taking go-kit as an example, it's trying to be the lowest common denominator so key features of the Prometheus client are lost such as full floating point support and being strict about all time series in a metric having the same labels. It also still requires you to construct all your Prometheus metrics, so you're not really making things simpler when it comes to changing later.</p>
<blockquote>
<p>so that you can easily swap out backends (prometheus to statsd, etc)</p>
</blockquote>
<p>Prometheus clients support extracting data to write out to other systems. It's a little tricky in Go right now, but Java and Python provide good APIs for this and include the ability to write out to Graphite as an example.</p>
<blockquote>
<p>Is there any value in using an abstraction layer like go-kit/metrics, or am I better off just using the prometheus client?</p>
</blockquote>
<p>I personally consider any indirection around any instrumentation library to be a major anti-pattern, as I've yet to come across one that didn't "simplify" away key functionality and generally get in the way.</p></pre>peterbourgon: <pre><p>See my <a href="https://www.reddit.com/r/golang/comments/4o5yt0/is_there_any_value_in_using_something_like/d4a1ylg" rel="nofollow">reply here</a> but</p>
<blockquote>
<p>I personally consider any indirection around any instrumentation library to be a major anti-pattern, as I've yet to come across one that didn't "simplify" away key functionality and generally get in the way.</p>
</blockquote>
<p>I hope you'd be a bit more generous with go-kit/metrics. It's directly influenced by Prometheus' API and, apart from the floats issue, I think — at least, I've tried <em>very hard</em> to ensure that — no major functionality is lost in the translation.</p></pre>F21Global: <pre><p>This was what I was thinking. I didn't really see any benefits of having adapters for different metrics backends in this case.</p></pre>
