<p>I have a c program that continuously runs and every 5-30 seconds spits out a bunch of formatted text. I want to parse the output and stream it to web clients. The server side events and client channels all work, but I'm having issues with getting the command output. I tried using the exec package and a bufio scanner, but I cannot get it to work as expected. It does parse the output into lines, but it seems to do it randomly, not every 5-30 seconds like the program outputs. Sometimes it takes a couple minutes, then scans through all the buffered data at once. I got it working by setting a timeout on the command and restarting it every 30 seconds, not very ideal. My thinking is that the runtime isn't scheduling the scanner go routine so it just buffers it until its scheduled? Anyway to force it to run whenever it sees the newline token? The main problem is I would prefer to parse the c program output as soon as it is output, instead of waiting for the command to timeout, or the scheduler to do its thing.</p>
<p>This is pretty much what I tried doing (minified):</p>
<pre><code>send := make(chan Output)
cmd := exec.Command("/usr/local/bin/mndp")
out, err := cmd.StdoutPipe()
scanner := bufio.NewScanner(out)
if err := cmd.Start(); err != nil {return err}
go func() {
for scanner.Scan() {
txt := scanner.Text()
go parse(txt, send) //parses text into Output structs and sends to send chan
}
}()
go func() {
for out := range send {
//range through client channel map and send output to clients
}
}()
if err := cmd.Wait(); err != nil {return err}
</code></pre>
<p>Now its "working" with something similar to this:</p>
<pre><code>send := make(chan Output)
ticker := time.NewTicker(30 * time.Second)
for _ = range ticker.C {
con, can := context.WithTimeout(d.ctx, 28*time.Second)
cmd := exec.CommandContext(con, "/usr/local/bin/mndp")
out, _ := cmd.Output()
if len(out)< 1 {continue}
go parse(string(out), send) //parses text into Output structs and sends to send chan
}
go func() {
for out := range send {
//range through client channel map and send output to clients
}
}()
</code></pre>
<hr/>**评论：**<br/><br/>nsd433: <pre><p>It sure feels like there's a buffer somewhere. I'd bet the output arrives in some nice 2<sup>N</sup> size, probably 4kB.</p>
<p>The first step I'd do is to narrow down where the buffer(s) is(are): the C program or the Go side. Try using /usr/bin/strace and attach to the C program and see how it calls the write syscall. If you see it writing buffers out then the buffering is in the C program. If you see it writing a line at a time then that isn't the source. Keep in mind that it is common in C code to use a FILE* (what fopen returns) for IO, and a FILE* will buffer by default unless it's connected to a tty-type device. So your C program can act one way when writing to a terminal (which is tty) and another when writing to a pipe or redirected to a file.</p>
<p>Once you find the buffer, remove or disable it, and lather, rinse, repeat.</p></pre>nsd433: <pre><p>Oh, and if you can't change the C program and the buffer is isatty() dependent then you're in for the additional pain of constructing a pseudo-tty and setting the slave side as the stdout of the C program, and driving it from master side. At least you (hopefully) won't have to emulate a terminal's escape sequences.</p></pre>madman2233: <pre><p>Looks like it is doing a bunch of write(1) calls. Looks a little weird though, most of them are of the form write(1, "Standard output"..., 131and the rest of the text output is outside quotes) = 131</p>
<p>Not sure if that formatting is messed up from strace, or a bug. </p></pre>nsd433: <pre><p>strace elides strings longer than 32 bytes. Use the -s flag to override this by setting it to a much larger value.</p>
<p>The 131, however, is the length of the write and the return value. 131 would be a strange amount to buffer. Were you running the C program in a terminal, or attaching to the C program started by your Go program?</p></pre>Justinsaccount: <pre><p>Not a go problem. Your C program is not flushing stdout. You need to call fflush in C, or run it using <code>stdbuf -oL</code>.</p></pre>
