Home > Archive > Smalltalk > September 2007 > How to write a "zero byte" to a stream on a tcp socket?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
How to write a "zero byte" to a stream on a tcp socket?
|
|
| pineapple.link@yahoo.com 2007-08-25, 7:18 pm |
| I have written a server which transfers date to a client on a tcp
socket (writing data using a "readAppendStream". The client
(unfortunately) uses some sort of flash "xmlsocket" which requires a
so-called "zero byte" to terminate the line. Note that this is not a
"zero character," i.e. $0, or '0'. It is a "zero byte." In various
web literature I've been able to scan, I've seen this written as
(char)0x00.
My question is how to write this zero byte onto the stream and
socket? I tried changing the mode of the socket to binary (after
writing characters in text mode) and sending a "nextNumber:1 put:0"
message to the stream. This actually worked when I was testing the
server on my local windows PC. It no longer worked when I moved the
server to the actual server machine (a unix system), and I was always
leery of switching modes like this in mid-stream anyway... didn't know
how safe that was.
I have verified that the server is actually sending data, because I
can telnet to the port and see the data. The client can connect to
the port, but never receives this data, because I'm not doing this
"zero byte" thing correctly. The client is correct, however, because
it can connect to my initial test machine and receive the data.
Any idea on how to correctly (and portably) write a "zero byte?"
THANKS!
| |
| Hans-Martin Mosner 2007-08-26, 7:18 pm |
| pineapple.link@yahoo.com wrote:
> My question is how to write this zero byte onto the stream and
> socket? I tried changing the mode of the socket to binary (after
> writing characters in text mode) and sending a "nextNumber:1 put:0"
> message to the stream. This actually worked when I was testing the
> server on my local windows PC. It no longer worked when I moved the
> server to the actual server machine (a unix system), and I was always
> leery of switching modes like this in mid-stream anyway... didn't know
> how safe that was.
That's generally correct, although you should just use "nextPut: 0".
Switching from text to binary is not a problem, you can just do it as needed.
Your issue is probably caused by not flushing the buffer after writing
the zero byte. If you write data to any stream, it is first stored in a
buffer that is only sent to the OS when you either close the file, try
to read from it, or use the "commit" message explicitly.
Cheers,
Hans-Martin
--
| |
| nowhereface@aol.com 2007-08-26, 7:18 pm |
| That's not the problem. I've been expressly using both flush AND
commit after writing the 0 byte.
If you are telling me I've been doing this correctly, then that's good
information. I suppose I don't know what to do next. I've tried
everything. Must be a problem with linux writing this zero-byte -- I
dunno.
| |
| Reinout Heeck 2007-08-27, 4:38 am |
| Hans-Martin Mosner wrote:
> pineapple.link@yahoo.com wrote:
>
>
> That's generally correct, although you should just use "nextPut: 0".
> Switching from text to binary is not a problem, you can just do it as needed.
Switching works on most streams but not all (gzipped streams have some
quirks in this respect).
If you are in character mode you could simply do
stream nextPut: (Character value: 0).
no switching of modes needed...
> Your issue is probably caused by not flushing the buffer after writing
> the zero byte. If you write data to any stream, it is first stored in a
> buffer that is only sent to the OS when you either close the file, try
> to read from it, or use the "commit" message explicitly.
>
R
-
| |
| nowhereface@aol.com 2007-08-27, 11:31 am |
| > If you are in character mode you could simply do
> stream nextPut: (Character value: 0).
>
> no switching of modes needed...
Thanks.
I continue to try everything I know to try. I try binary. I try
character. I try various encodings, whether ASCII or UTF_8 or
whatever. I try writing raw bytes using a ByteArray. No matter what
I try, it doesn't work. Oh, it transmits the data, and I can see the
data being sent just fine when I telnet to the port. But the client
running the flash xmlsocket never picks it up. It must have something
to do with the 0 byte, because again - the data is visibly being sent.
Since all of these changes (outlined above) work fine when I move the
server to a Windows PC (in other words, the client will at that point
pick up the data), it seems to be an issue either with Linux, or
VisualWorks running on Linux.
Does anyone have a clue what to do, or what the problem might be???
Has anyone ever had an issue with sending "zero bytes" or "null ASCII"
from Linux or VisualWorks on Linux?
THANKS.
| |
| Reinout Heeck 2007-08-27, 11:31 am |
| nowhereface@aol.com wrote:
>
> I continue to try everything I know to try. I try binary. I try
> character. I try various encodings, whether ASCII or UTF_8 or
> whatever. I try writing raw bytes using a ByteArray. No matter what
> I try, it doesn't work. Oh, it transmits the data, and I can see the
> data being sent just fine when I telnet to the port. But the client
> running the flash xmlsocket never picks it up. It must have something
> to do with the 0 byte, because again - the data is visibly being sent.
>
> Since all of these changes (outlined above) work fine when I move the
> server to a Windows PC (in other words, the client will at that point
> pick up the data), it seems to be an issue either with Linux, or
> VisualWorks running on Linux.
>
> Does anyone have a clue what to do, or what the problem might be???
> Has anyone ever had an issue with sending "zero bytes" or "null ASCII"
> from Linux or VisualWorks on Linux?
>
> THANKS.
>
Line end convention comes to mind.
Try sending a #lineEndCRLF to your socket's writeStream.
HTH,
Reinout
-------
| |
| nowhereface@aol.com 2007-08-27, 8:36 pm |
| > Try sending a #lineEndCRLF to your socket's writeStream.
Tried no line end conventions, nor using any line ends in my data at
all. I've tried lineEndCR. I've tried lineEndLF. I've tried
lineEndCRLF. Plus I tried adding CRs, LFs, etc. in my data stream.
No matter what I try, it always works on the windows test server (all
solutions), but never works when I send it over to the unix machine.
I also tried adding multiple "zero bytes" to terminate the data.
Nothing works. It behaves, in all ways, as if is sending the data,
but not the zero byte.
Any other ideas?
| |
|
| On Aug 27, 2:04 pm, nowheref...@aol.com wrote:
>
> Tried no line end conventions, nor using any line ends in my data at
> all. I've tried lineEndCR. I've tried lineEndLF. I've tried
> lineEndCRLF. Plus I tried adding CRs, LFs, etc. in my data stream.
> No matter what I try, it always works on the windows test server (all
> solutions), but never works when I send it over to the unix machine.
>
> I also tried adding multiple "zero bytes" to terminate the data.
> Nothing works. It behaves, in all ways, as if is sending the data,
> but not the zero byte.
>
> Any other ideas?
I would try running ethereal (http://www.ethereal.com/) on the unix
machine and see what the actual data packet looks like. I'd also
compare that to what is being sent by the Windows machine and see what
the difference is.
This will show if the problem is caused by incorrect line terminators,
the missing zero byte, or something else.
--Dan
| |
| Reinout Heeck 2007-08-28, 8:40 am |
|
+1, we'll need to look at what is going over the network.
If you don't feel like installing Ethereal (an X application) you can
try the command line tool tcpdump.
R
-
Dan wrote:
> On Aug 27, 2:04 pm, nowheref...@aol.com wrote:
>
> I would try running ethereal (http://www.ethereal.com/) on the unix
> machine and see what the actual data packet looks like. I'd also
> compare that to what is being sent by the Windows machine and see what
> the difference is.
>
> This will show if the problem is caused by incorrect line terminators,
> the missing zero byte, or something else.
>
> --Dan
>
| |
| nowhereface@aol.com 2007-08-28, 7:17 pm |
| Interesting.
Ran Ethereal and caught the data coming in. Ethereal shows a period
(.) where the "zero byte" should be located. When I export the data
and look at it in... uh... Notepad (cough)... it shows an invisible
character where the zero byte should be. On my telnet session (which
is how I got the data sent to my computer) the zero byte also shows up
as an invisible character.
If you tell me that Ethereal shows a zero byte as a period (no
decoding of data in Ethereal), then apparently a zero byte is being
sent. If so, the client doesn't pick it up, but strangely enough if I
run the same program from a Windows server (vs. Linux) the client will
pick it up. I certainly can't explain that.
If you tell me that Ethereal does not show a zero byte as a period, I
suppose that means a zero byte is not being sent, and something else
is being sent.
Thanks.
| |
| nowhereface@aol.com 2007-08-28, 10:12 pm |
| Looked at the file with a hexdump utility. The period (.) is indeed a
zero byte. So the zero byte is being written, yet the client doesn't
respond. Yet the client responds to the same exact output if it comes
from a Windows server on a local LAN. This is what was confusing
initially, and led me to blame either Linux, or VW on Linux.
CONCLUSION:
The flash "security" is differentiating between a LAN tcp connection,
and an internet tcp connection. The flash "security" is getting in my
way. So I'm now throwing any flash-based "solution" in the garbage
can.
| |
| Thomas Gagne 2007-09-10, 7:13 pm |
| pineapple.link@yahoo.com wrote:
> I have written a server which transfers date to a client on a tcp
> socket (writing data using a "readAppendStream". The client
> (unfortunately) uses some sort of flash "xmlsocket" which requires a
> so-called "zero byte" to terminate the line. Note that this is not a
> "zero character," i.e. $0, or '0'. It is a "zero byte." In various
> web literature I've been able to scan, I've seen this written as
> (char)0x00.
>
> My question is how to write this zero byte onto the stream and
> socket? I tried changing the mode of the socket to binary (after
> writing characters in text mode) and sending a "nextNumber:1 put:0"
> message to the stream. This actually worked when I was testing the
> server on my local windows PC. It no longer worked when I moved the
> server to the actual server machine (a unix system), and I was always
> leery of switching modes like this in mid-stream anyway... didn't know
> how safe that was.
>
> <snip>
>
>
In my STOMP client I used:
self stream cr;
nextPutAll: (aBody ifNil: [ '' ]);
nextPut: (Character value: 0);
commit.
--
Visit <http://blogs.instreamco.com/anything.php> to read
my rants on technology and the finance industry. Visit
<http://tggagne.blogspot.com/> for politics, society and culture.
| |
| Thomas Gagne 2007-09-10, 7:13 pm |
| nowhereface@aol.com wrote:
> Looked at the file with a hexdump utility. The period (.) is indeed a
> zero byte. So the zero byte is being written, yet the client doesn't
> respond. Yet the client responds to the same exact output if it comes
> from a Windows server on a local LAN. This is what was confusing
> initially, and led me to blame either Linux, or VW on Linux.
>
> CONCLUSION:
>
> The flash "security" is differentiating between a LAN tcp connection,
> and an internet tcp connection. The flash "security" is getting in my
> way. So I'm now throwing any flash-based "solution" in the garbage
> can.
>
>
I've done a bunch of network programming in both C and Smalltalk on
Windows and Unix systems (credentials). Seeing the zero bytes doesn't
necessarily mean anything if the person sending the message didn't
include the zero byte in the message's size (if such a value is located
elsewhere in the message).
You say the client responds when the /exact/ output comes from a Windows
server on the local LAN. Are both Windows and Unix using TCP? When the
server writes the message is it counting the 0-byte as part of its size?
An earlier poster was correct in that a close look at the output of
ethereal would be helpful. If you send me the capture from tcpdump for
the exact same messages, one from a Windows server and another from a
Unix server, I'll take a look at it and see what I might find, or at
least ask some better questions.
Remove the dashes from my email address and it should work fine.
--
Visit <http://blogs.instreamco.com/anything.php> to read
my rants on technology and the finance industry. Visit
<http://tggagne.blogspot.com/> for politics, society and culture.
| |
| pineapple.link@yahoo.com 2007-09-11, 8:10 am |
| <<You say the client responds when the /exact/ output comes from a
Windows
server on the local LAN. Are both Windows and Unix using TCP?>>
Yes. It's the exact same program, and exact same system (VisualWorks
7.3.x). The test/development platform is a Windows PC on a local
LAN. Client is on local LAN and can communicate no problem. The
production platform is a Linux PC out on the internet. Client cannot
communicate - doesn't pick up the zero byte, or has some other issue.
I can telnet to the production system's port and view the output no
problem - looks great. I can look at it with ethereal - looks great.
Client won't pick it up.
<< When the server writes the message is it counting the 0-byte as
part of its size? >>
I dunno. I use the transparent system provided by VW to send stuff
TCP. I don't care to dig into the guts of the system to determine
whether it is doing its job or not.
I greatly appreciate your offer of help. However, I have decided to
just trash the entire system. First the front-end server interfered
with my pushlets, creating problems. Rewrote using AJAX, but had
issues with reset/reload button working. Switched to Java - the Java
"security" was impossible to get around once I moved the server from
my local PC off to the internet. Rewrote again in Flash - there is
some issue which I believe is being caused by the Flash "security."
Went back to AJAX again, accepting the fact that the reload/reset
button would cause serious problems. The AJAX is unreliable over the
internet, and updates the screen sporadically, and sometimes not at
all. On and on (a nightmare). At this point I've had it up to "here"
with the horrible bother of doing web so-called "development" work,
something I wasn't looking forward to in the first place. This was a
project over a year in the making....
|
|
|
|
|