petrel - it's like websockets minus the web

<p>A long time ago, <del>in a galaxy far away</del> when this sub was much smaller, I posted here about a hands-off networking library I was working on named asock. In the year-and-a-half since then it&#39;s had a name change, grown in capability, gotten more reliable and efficient, and been used in production passing messages between hundreds of machines. And you can still use it to add a management port to your software -- the original use case.</p> <p>I just cut <a href="https://github.com/firepear/petrel">petrel version 0.26</a>, whose big feature is optional HMAC authentication for messages. With this added to the existing TLS support and available message length limits, I finally felt comfortable removing the &#34;Do Not Use This On Public Networks&#34; disclaimer which has been in the docs forever. You can now balance security and speed as you see fit.</p> <p>I&#39;m posting today because the library feels pretty clean and solid to me. A 1.0 release is likely on the horizon, which means freezing names and APIs, so some public feedback would be nice. If a library like this sounds useful, please check it out and let me know how you find it. Thanks!</p> <hr/>**评论：**<br/><br/>gabesullice: <pre><p>Super cool, what are the differences between this and something like zeromq or rabbitmq? I think they&#39;d probably share many of the same use cases.</p></pre>firepear: <pre><p>One big difference is that petrel isn&#39;t a message broker. There&#39;s no pub-sub or store-and-forward. In fact, there&#39;s no baked-in provision for clients to communicate with each other at all. You could build that on top of petrel, but at that point you probably should think about using rabbitmq/zeromq, since that&#39;s what they&#39;re designed for.</p> <p>The other big difference is that unlike rabbit/zeromq, there is no standalone server executable. petrel.Server is just a library component, which can be used by any program. (Same for petrel.Client, of course.) You can use petrel to bake network capabilities into anything.</p> <p>A single program can actually have multiple instances of petrel.Server; I&#39;ve done this with a network monitoring system, where the data aggregator has one petrel.Server that accepts data from the machines in the cluster (gathering stats) and another petrel.Server listens for requests from the query tool (displaying stats). If you wanted, you could have a program which had both petrel.Server and petrel.Client instances, and thus acted as a piece of middleware.</p></pre>justinisrael: <pre><p>Would this be similar then to nats.io and its ability to start a server embedded from the library in your code? If so, where does petrel diverge from the features of nats.io? </p></pre>firepear: <pre><p>I am unfamiliar with nats.io, but from a brief reading of their documentation, their server is standalone:</p> <blockquote> <p>The executable name for the NATS server is gnatsd, which stands for Go NATS Deamon.</p> </blockquote> <p>So it wouldn&#39;t seem, at a glance, that nats and petrel are similar in that regard. In general, nats looks much more like a contemporary of other zeromq and rabbitmq.</p> <p>I don&#39;t have the expertise (or the time to gain the expertise) to write something which promises to handle tens of thousands of transactions per second. Petrel started as a very small project, to scratch an itch. It proved surprisingly capable, so I expanded its scope a little bit, and tried to get the quality to a place I was happy with. I think it&#39;s useful, and I hope others may as well, but I&#39;ve never viewed it as competing with anything -- especially large projects from corporate teams :)</p></pre>justinisrael: <pre><blockquote> <p>I am unfamiliar with nats.io, but from a brief reading of their documentation, their server is standalone:</p> </blockquote> <p>It can run as a standalone daemon, but it is also a library. I am currently using it in a project where the server is embedded into the app to easily host client connections. </p> <blockquote> <blockquote> <p>The executable name for the NATS server is gnatsd, which stands for Go NATS Deamon.</p> </blockquote> </blockquote> <p>Yea that is what the cmd that is produced is called. </p> <blockquote> <p>So it wouldn&#39;t seem, at a glance, that nats and petrel are similar in that regard. In general, nats looks much more like a contemporary of other zeromq and rabbitmq.</p> <p>I don&#39;t have the expertise (or the time to gain the expertise) to write something which promises to handle tens of thousands of transactions per second. Petrel started as a very small project, to scratch an itch. It proved surprisingly capable, so I expanded its scope a little bit, and tried to get the quality to a place I was happy with. I think it&#39;s useful, and I hope others may as well, but I&#39;ve never viewed it as competing with anything -- especially large projects from corporate teams :)</p> </blockquote> <p>Totally a fair statement. My question was really just asking if you knew about how your newly announced solutions compares with others. I figured if you are announcing a new solution for people to use, that you might be able to give a high level comparison in that announcement to the existing options. Otherwise I would need to invest time to dive into every option that is announced just to see what it does compared to others. You being in the head space of this problem might have already known. </p> <p>No worries. Just had to ask if you had looked around and could give that high level explanation. </p></pre>BraveNewCurrency: <pre><p>It looks interesting, but you if you want people to investigate your library, you will need a bigger ReadMe. First and formost, tell people WHAT is is. &#34;websockets without the web&#34; is kind of meaningless.</p> <p>I assume you are trying to say &#34;it handles defined-length messages, instead of just bytes like TCP&#34;? What are the min/max sizes? What is the overhead? Does it provide any abstractions on top of that (i.e. routing, commands, marshaling, encoding, etc?)</p> <p>What are the typical use cases? What&#39;s wrong with using WebSockets? Or ProtoBufs/GRPC over TCP? What are the advantages/disadvantages. When would your library be a better/worse fit? (Same with 0mq)</p></pre>