Lets assume i dont use any bufio or ioutil and i also dont hardCode the max buffer size and try to load things into a giant []byte
lets assume i am reading the connection byte by byte a single byte at a time.
How can i make sure that theres next byte available something like
func hasNext(net.Conn)bool
评论:
TheMerovius:
nefthias:In general, it is more idiomatic to simply block. Is there a reason you can't block?
carsncode:it will block any way what i am asking is how can i make sure theres more before blocking if theres no more then i would like to send the bytes to parser etc.
nefthias:It would be better to keep reading and blocking until you've caught some parseable block (depending on your data format - so maybe newlines or semicolons or whatever works in your situation), then send that to the parser. This solves the blocking issue and helps to decouple the parser from the network handling.
nefthias:In my case we are using binary protocol and sending the length in the 3rd byte so its no problem but i just wondered if theres any other way to do it without actually sending the length
carsncode:somthing like
as long as (hasNext) read end doSomethingWithFullData
```
TheMerovius:There has to be some indicator in the stream, generally: a preceding length value; a delimiter/end of data indicator; or closing the connection after transmission is complete. There is no way for a connection itself, devoid of protocol to know if more data is coming (other than the connection being closed). At best it can know if there's data in its buffer.
nefthias:There is a ioctl (SIOCINQ) you can use to see how much data is in the kernel inbound socket buffer; but note, that this will only tell you how much data the kernel has buffered you haven't read, it doesn't tell you whether or not the other side of the connection will send more data eventually - that part is fundamentally unknowable.
You know that the stream is done, when you get an EOF - until then, the best and idiomatic way to handle this is to blockingly read as much data as you get and hand it over piece-wise and do the framing on the application level.
You probably want your parser to take an
io.Reader
and just pass the connection directly. It'll be fine.
TheMerovius:oh yes that is what i am seeking for of course nobody knows whether the other party will send more messages or not but i would like to know how far i consumed of the message and how much is waiting to be consumed. I will see ioctl but cna you be more specific and is it cross platform ?
nefthias:I still do not understand why you can not just block. Go goes to great length to make this as fast and efficient as possible and will do all kinds of polling itself.
cna you be more specific
Not really, I don't know enough about network programming myself.
and is it cross platform ?
Definitely not.
pdffs:oh not being cross platform makes it really not reliable. this is a hypothetical question though in our system we use length as a parameter but anyway it could be really nice
sethammons:What do you mean by "if theres no more"? You absolutely need to delineate your message boundaries somehow.
You could push the bytes into a channel in a goroutine, then use
select
to determine if there's anything to be read from the channel at this precise moment, but that's not likely to give you what you think you want - what happens if there's a delay sending data over the socket, packet-loss or whatever? You'll end up sending a partial message to your parser. You can't rely timing to handle this.
nefthias:Like others have said, without protocol information, you simply can't know if more data is or is not coming. You need a blocking read. You can make a read timeout or similar. There is also Peek() which will let you know the next n bytes without advancing the reader, but you can only peek on what has already been sent. You can't know the future.
sethammons:where is the Peek located exactly ? I couldn't find it in net package.
Look in the bufio package
