io.Reader.Read() does it remove data from its own buffers after a Read call?

<p>I&#39;m trying to measure network download speed by creating multiple streams to download large files via go&#39;s Net library. I&#39;m recording the total speed by summing up speeds from individual streams. Since I don&#39;t have any use for the actual data in the streams and I don&#39;t want to flood the memory with large chunks of downloaded data, I wanted to know if there&#39;s a way to remove data from these internal buffers of io.Reader OR does the Read() method clear them itself after a Read call ?</p> <hr/>**评论：**<br/><br/>captncraig: <pre><p>note that io.Reader is an interface. There are many implementations which do different things. If it is a network request like you are using, then you can use the same buffer over and over again:</p> <pre><code>buf := make([]byte,128*1024) for{ conn.Read(buf) } </code></pre> <p>This loads straight from the os memory into your buffer. There is no additional allocation.</p></pre>dlsspy: <pre><p>But it&#39;s a copy. You could avoid the copy with a ReaderFrom</p></pre>captncraig: <pre><p>got an example?</p></pre>dlsspy: <pre><p>I&#39;m not sure off the top of my head how to do it in that direction. io.Copy automatically does it in the other direction from a file to a TCP connection. io.Copy to ioutil.Discard would be the first thing I&#39;d try for this type of problem, though.</p></pre>captncraig: <pre><p>even that allocates an 8k buffer (at least it is a pooled buffer): <a href="https://golang.org/src/io/ioutil/ioutil.go#L147" rel="nofollow">https://golang.org/src/io/ioutil/ioutil.go#L147</a></p></pre>dlsspy: <pre><p>Yes, Discard does. There was a bug where it used to not and bad things happened.</p></pre>xargon7: <pre><p>Yes, but memory bandwidth should <em>far</em> exceed network bandwidth (e.g. 4+GB/sec vs ~100MBps for a saturated Gbps network), so it shouldn&#39;t matter.</p> <p>You can avoid any heap allocation using an array:</p> <pre><code>var buf [128*1024]byte for { if _, err := conn.Read(buf[:]); err != nil { break } } </code></pre></pre>dlsspy: <pre><p>I prefer <code>io.Copy</code> for simplicity in most cases. On one hand, you&#39;re reinventing it when you make a loop, but on the other, you&#39;re removing its ability to optimize.</p> <p>Array or slice doesn&#39;t make a difference WRT heap. Your buf escapes to heap.</p> <pre><code>$ nl -ba x.go 1 package main 2 3 import ( 4 &#34;io&#34; 5 &#34;strings&#34; 6 ) 7 8 func WithArray(r io.Reader) error { 9 var buf [128 * 1024]byte 10 for { 11 if _, err := r.Read(buf[:]); err != nil { 12 return err 13 } 14 } 15 } 16 17 func main() { 18 WithArray(strings.NewReader(&#34;x&#34;)) 19 } $ go build -gcflags -m # misc/x ./x.go:18: inlining call to strings.NewReader ./x.go:9: moved to heap: buf ./x.go:11: buf escapes to heap ./x.go:8: leaking param: r ./x.go:18: strings.NewReader(&#34;x&#34;) escapes to heap ./x.go:18: &amp;strings.Reader literal escapes to heap </code></pre></pre>