Problems streaming output from a command. Any ideas would be appreciated.

<p>I have a c program that continuously runs and every 5-30 seconds spits out a bunch of formatted text. I want to parse the output and stream it to web clients. The server side events and client channels all work, but I&#39;m having issues with getting the command output. I tried using the exec package and a bufio scanner, but I cannot get it to work as expected. It does parse the output into lines, but it seems to do it randomly, not every 5-30 seconds like the program outputs. Sometimes it takes a couple minutes, then scans through all the buffered data at once. I got it working by setting a timeout on the command and restarting it every 30 seconds, not very ideal. My thinking is that the runtime isn&#39;t scheduling the scanner go routine so it just buffers it until its scheduled? Anyway to force it to run whenever it sees the newline token? The main problem is I would prefer to parse the c program output as soon as it is output, instead of waiting for the command to timeout, or the scheduler to do its thing.</p> <p>This is pretty much what I tried doing (minified):</p> <pre><code>send := make(chan Output) cmd := exec.Command(&#34;/usr/local/bin/mndp&#34;) out, err := cmd.StdoutPipe() scanner := bufio.NewScanner(out) if err := cmd.Start(); err != nil {return err} go func() { for scanner.Scan() { txt := scanner.Text() go parse(txt, send) //parses text into Output structs and sends to send chan } }() go func() { for out := range send { //range through client channel map and send output to clients } }() if err := cmd.Wait(); err != nil {return err} </code></pre> <p>Now its &#34;working&#34; with something similar to this:</p> <pre><code>send := make(chan Output) ticker := time.NewTicker(30 * time.Second) for _ = range ticker.C { con, can := context.WithTimeout(d.ctx, 28*time.Second) cmd := exec.CommandContext(con, &#34;/usr/local/bin/mndp&#34;) out, _ := cmd.Output() if len(out)&lt; 1 {continue} go parse(string(out), send) //parses text into Output structs and sends to send chan } go func() { for out := range send { //range through client channel map and send output to clients } }() </code></pre> <hr/>**评论：**<br/><br/>nsd433: <pre><p>It sure feels like there&#39;s a buffer somewhere. I&#39;d bet the output arrives in some nice 2^N size, probably 4kB.</p> <p>The first step I&#39;d do is to narrow down where the buffer(s) is(are): the C program or the Go side. Try using /usr/bin/strace and attach to the C program and see how it calls the write syscall. If you see it writing buffers out then the buffering is in the C program. If you see it writing a line at a time then that isn&#39;t the source. Keep in mind that it is common in C code to use a FILE* (what fopen returns) for IO, and a FILE* will buffer by default unless it&#39;s connected to a tty-type device. So your C program can act one way when writing to a terminal (which is tty) and another when writing to a pipe or redirected to a file.</p> <p>Once you find the buffer, remove or disable it, and lather, rinse, repeat.</p></pre>nsd433: <pre><p>Oh, and if you can&#39;t change the C program and the buffer is isatty() dependent then you&#39;re in for the additional pain of constructing a pseudo-tty and setting the slave side as the stdout of the C program, and driving it from master side. At least you (hopefully) won&#39;t have to emulate a terminal&#39;s escape sequences.</p></pre>madman2233: <pre><p>Looks like it is doing a bunch of write(1) calls. Looks a little weird though, most of them are of the form write(1, &#34;Standard output&#34;..., 131and the rest of the text output is outside quotes) = 131</p> <p>Not sure if that formatting is messed up from strace, or a bug. </p></pre>nsd433: <pre><p>strace elides strings longer than 32 bytes. Use the -s flag to override this by setting it to a much larger value.</p> <p>The 131, however, is the length of the write and the return value. 131 would be a strange amount to buffer. Were you running the C program in a terminal, or attaching to the C program started by your Go program?</p></pre>Justinsaccount: <pre><p>Not a go problem. Your C program is not flushing stdout. You need to call fflush in C, or run it using <code>stdbuf -oL</code>.</p></pre>