Don't Fear 'reflect' (with benchmarks)

<p>There&#39;s a pattern my company uses quite a lot. Take a stream of data (lines of a log file, a kafka topic, whatever) and shard it out to an array of channels using some hashable consistent key, often user ID, to better utilize multi-core machines for doing some realistic processing work (inserting into a DB, or doing calculations, maybe).</p> <p>Out of curiosity I wanted to build a little library to do this with &#39;reflect&#39; with the assumption that it&#39;d get someone up and running with the algorithm, and then you could code-gen the strongly typed code later if it worked out for your use case.</p> <p>When in my benchmarks it wasn&#39;t noticeably slower, I decided to start ripping out all the other variables to get the smallest test harness that tested only that tight loop of channel operations and nothing else. The results, maybe 50-70% slower, but still in the nanosecond range, which is nowhere near the multiple-order-of-magnitude difference I predicted.</p> <p>I&#39;d love some opinions on if there&#39;s flaws in my testing methodologies. All of the benchmarks plus some more documentation at this gist: <a href="https://gist.github.com/crast/61779d00db7bfaa894c70d7693cee505">https://gist.github.com/crast/61779d00db7bfaa894c70d7693cee505</a></p> <hr/>**评论：**<br/><br/>dgryski: <pre><p>The real issue with reflect is not in these sorts of channel operations which are already expensive, but rather struct and map operations that change from a few processor instructions into an expensive function call.</p></pre>itsmontoya: <pre><p>For your every day use-case. Reflect won&#39;t cause much issue. When you get into performance critical low-level code, it&#39;s a big problem</p></pre>k_z-: <pre><p>Go polymorphism (interfaces) is a big problem for performance critical code even without reflect to begin with. The fact that they are implicit is convenient but retrospectively a huge performance trade off.</p></pre>Redundancy_: <pre><p>I&#39;m curious if someone has compared Go interfaces to abstract base classes / virtual function tables in terms of performance delta. If we&#39;re going to say that they&#39;re a huge performance trade-off, we should be comparing to similar polymorphic features in other languages, rather than the non-polymorphic case.</p> <p>They are significantly better in semantic form for allowing the interface to live with the code that requires it, rather than the code that implements it (See Uncle Bob&#39;s writing on the Interface Segregation Principle).</p></pre>epiris: <pre><p>Since this was just a thought exercise, ill share how I performed <a href="https://gist.github.com/cstockton/2c7e9346af7b7daad11bf9a754a3716e#file-chansendn-go-L7" rel="nofollow">unsafe channel sends</a> so I could test the potential value of a bulk chan send function before I wrote an <a href="https://gist.github.com/cstockton/5f2fea4ace0f099b4908a115409f2da3" rel="nofollow">SSA pass</a>. Maybe you could benchmark it too for fun if you wanted, iirc they are about the same speed.</p></pre>sacrehubert: <pre><blockquote> <p>unsafe channel sends</p> </blockquote> <p>Could you elaborate? What&#39;s the strategy here?</p></pre>epiris: <pre><p>Calling the runtime function that the compiler generates for a channel send, which takes an unsafe.Pointer instead of the empty interface.</p></pre>sacrehubert: <pre><p>Right, that much I got, but why is this more efficient? What overhead is avoided?</p></pre>epiris: <pre><p>It is more efficient because unsafe.Pointer is essentially free, while the Send method of reflect.Value is not. Anytime you see a reflect.Value there is an immediate cost of 1 allocation due to forcing all Value&#39;s to escape to the heap. For Send we require two allocations since it has a param as well the cost of ensuring type safety at runtime. The overhead avoided is the time to Send via reflect minus the time to send via unsafe. You would never do this, but it was relevant to the ops experiment so I shared.</p></pre>sacrehubert: <pre><p>Thanks for the explanation :)</p></pre>