Home > Archive > Unix Programming > December 2005 > Flushing Sockets in C?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Flushing Sockets in C?
|
|
| Sparticus 2005-12-12, 10:10 pm |
| I have a problem. My C program writes data to a socket and the output
if buffered (and thus not sent immediately to client). What I need is
a way to flush the socket so that the data in the buffer gets sent
immediately.
I have spend that last 3 hours reading newsgroups and such and am on
the verge of giving up. I read that there is no way to flush a socket.
No way at all.
I also read that you can enable TCP_NODELAY which apparently should fix
my problem... but after adding the following code to my program it
didn't appear to help at all. Same problem. Not sure if my code if
correct.
if(setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, (char*) &on,
sizeof(int))<0)
{
// error occured
}
Can anyone help me? It seems really silly that after writing data to a
socket... many times the data just sits there.... and sits there... It
randomly seems to arrive to the client. This really isn't acceptable
for my program....
Can anyone give me any advice?
Thanx a bunch!!
Ryan
| |
| Gustavo Rios 2005-12-13, 4:13 am |
| I did not understand what you want! Probably i should read UNP (by
Stevens, R.) much more i have done so far.
Are you using stdio with a socket descriptor? If so, forget stdio when
playing with network programing. Another important aspect relates to
closing a socket descriptor before the output buffer has been
delivered, be carefull. Again, i think i should UNP much more i have
done so far.
| |
| David Schwartz 2005-12-13, 4:13 am |
|
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134440130.566134.244720@g47g2000cwa.googlegroups.com...
>I have a problem. My C program writes data to a socket and the output
> if buffered (and thus not sent immediately to client). What I need is
> a way to flush the socket so that the data in the buffer gets sent
> immediately.
TCP? UDP? stdio? What are we talking about here?
> I have spend that last 3 hours reading newsgroups and such and am on
> the verge of giving up. I read that there is no way to flush a socket.
> No way at all.
No. There is no reason to flush a socket.
> I also read that you can enable TCP_NODELAY which apparently should fix
> my problem... but after adding the following code to my program it
> didn't appear to help at all. Same problem. Not sure if my code if
> correct.
>
> if(setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, (char*) &on,
> sizeof(int))<0)
> {
> // error occured
> }
What is your problem exactly?
> Can anyone help me? It seems really silly that after writing data to a
> socket... many times the data just sits there.... and sits there... It
> randomly seems to arrive to the client. This really isn't acceptable
> for my program....
What does "randomly seems to arrive" mean? It arrives later? It
sometimes doesn't arrive? How much later?
Nobody has any idea what you're talking about.
DS
| |
| Maxim Yegorushkin 2005-12-13, 8:14 am |
|
Sparticus wrote:
> I have a problem. My C program writes data to a socket and the output
> if buffered (and thus not sent immediately to client). What I need is
> a way to flush the socket
There is none.
> so that the data in the buffer gets sent immediately.
In this case you should disable Nagle algorithm by using TCP_NODELAY
option.
> I also read that you can enable TCP_NODELAY which apparently should fix
> my problem... but after adding the following code to my program it
> didn't appear to help at all. Same problem. Not sure if my code if
> correct.
>
> if(setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, (char*) &on,
> sizeof(int))<0)
> {
> // error occured
> }
The proper way of doing it is:
int v = 1;
if(::setsockopt(s, SOL_TCP, TCP_NODELAY, &v, sizeof(v)))
// handle error
How do you figure out it does not help? Have you tried analyzing
tcpdump output?
> Can anyone help me? It seems really silly that after writing data to a
> socket... many times the data just sits there.... and sits there...
It happens when you write amounts smaller than MSS into a socket. In
this case data won't be sent until an ACK for the previous segment is
received - this is Nagle algorithm.
| |
| Alex Fraser 2005-12-13, 8:14 am |
| "Maxim Yegorushkin" <maxim.yegorushkin@gmail.com> wrote in message
news:1134471296.474632.287880@g44g2000cwa.googlegroups.com...
> Sparticus wrote:
>
> There is none.
>
>
> In this case you should disable Nagle algorithm by using TCP_NODELAY
> option.
Not good advice based on the information available. If the OP is using a
stdio buffer, then he should use fflush(); if the OP is making multiple
small send()/write() calls with related data, then he should buffer the data
within the application. If these don't apply or help, then it might be
better to change the protocol rather than use TCP_NODELAY.
Alex
| |
| Maxim Yegorushkin 2005-12-13, 8:14 am |
|
Alex Fraser wrote:
[]
>
> Not good advice based on the information available. If the OP is using a
> stdio buffer, then he should use fflush(); if the OP is making multiple
> small send()/write() calls with related data, then he should buffer the data
> within the application. If these don't apply or help, then it might be
> better to change the protocol rather than use TCP_NODELAY.
Agreed, I have never had a need to use TCP_NODELAY.
| |
| Konstantin Sorokin 2005-12-13, 8:14 am |
| Maxim Yegorushkin <maxim.yegorushkin@gmail.com> wrote:
> The proper way of doing it is:
>
> int v = 1;
> if(::setsockopt(s, SOL_TCP, TCP_NODELAY, &v, sizeof(v)))
> // handle error
>
It seems to me that we have to use IPPROTO_TCP instead of SOL_TCP
--
Konstantin Sorokin
| |
| Maxim Yegorushkin 2005-12-13, 7:04 pm |
|
Konstantin Sorokin wrote:
> Maxim Yegorushkin <maxim.yegorushkin@gmail.com> wrote:
>
>
> It seems to me that we have to use IPPROTO_TCP instead of SOL_TCP
man tcp(7)
....
SOCKET OPTIONS
To set or get a TCP socket option, call getsockopt(2) to read or
setsockopt(2) to write the option with the option level argument set to
SOL_TCP. In addition, most SOL_IP socket options are valid on TCP
sockets. For more information see ip(7).
....
| |
| Casper H.S. Dik 2005-12-13, 7:04 pm |
| "Maxim Yegorushkin" <maxim.yegorushkin@gmail.com> writes:
>Konstantin Sorokin wrote:
[color=darkred]
>man tcp(7)
>...
>SOCKET OPTIONS
>To set or get a TCP socket option, call getsockopt(2) to read or
>setsockopt(2) to write the option with the option level argument set to
>SOL_TCP. In addition, most SOL_IP socket options are valid on TCP
>sockets. For more information see ip(7).
Which manual is that? That is fairly unconventional.
(The level is generally the IPPROTO_FOO argument, not a separate
SOL_XXX define family)
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Maxim Yegorushkin 2005-12-13, 7:04 pm |
|
Casper H.S. Dik wrote:
> "Maxim Yegorushkin" <maxim.yegorushkin@gmail.com> writes:
>
>
>
>
>
> Which manual is that? That is fairly unconventional.
Linux manual. Online version
http://www.die.net/doc/linux/man/man7/tcp.7.html
> (The level is generally the IPPROTO_FOO argument, not a separate
> SOL_XXX define family)
Have not heard anything about this convention.
| |
| Casper H.S. Dik 2005-12-13, 7:04 pm |
| "Maxim Yegorushkin" <maxim.yegorushkin@gmail.com> writes:
>Linux manual. Online version
>http://www.die.net/doc/linux/man/man7/tcp.7.html
Sounds like a Linux invention.
[color=darkred]
>Have not heard anything about this convention.
It's from BSD where sockets originated.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Sparticus 2005-12-13, 7:04 pm |
| Fist of all... thanx for all the replies!
I guess I need to explain a bit more.... forgive me for I am no expert
in all this stuff...
Ok, I believe I am using TCP sockets. I included sys/socket.h and
below is how I create the socket :
void set_up_socket()
{
int on=1;
// Close the socket in case it was left open from last time
close(sockfd);
// Create the socket
if((sockfd = socket (AF_INET, SOCK_STREAM, 0)) <0)
{
perror("Could not create socket\n");
exit(-1);
}
// No clue what this does, but it's probably needed :)
if(setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, (char *)&on,
sizeof(&on))<0)
{
perror("Setsockopt failed\n");
exit(-1);
}
if(setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, (char*) &on,
sizeof(int))<0)
{
perror("TCP_NODELAY failed\n");
exit(-1);
}
// Clear the conects of addr
bzero(&addr, sizeof(addr));
// Set the properties of the socket
addr.sin_family = AF_INET;
addr.sin_port = htons (portno);
addr.sin_addr.s_addr = INADDR_ANY;
// Bind to the socket
if(bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) <0)
{
perror("Bind failed\n");
exit(-1);
}
// Start listening on the socket
if(listen(sockfd, 100) ,0)
{
perror("Unable to listen\n");
exit(-1);
}
}
I hope that cleared up some of the previous questions.... anyways...
from there I do the following :
send(client,"somestring",11,0); // which sends some text over to my
client program
So the real problem is that TCP buffers it's output to the socket....
so when I send a few bytes over the socket.... it doesn't get sent
right away. It sits in the buffer (someone mentioned until an ACK is
received)... or until the buffer is full.
What I need is a way to "send" data... and it actually sends right
away.... don't buffer anything....
I've read on google that what I need is "TCP_NODELAY" added.... as you
can see from my code above, I think I added it correctly... but the
output still seems to be buffered...
Can someone please point me in the right direction? It's been 2 days
now of reading on google and I'm getting no where...
Thanx for any help you can give me!!
Ryan
| |
| Alex Fraser 2005-12-13, 7:04 pm |
| "Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134490194.170374.206750@z14g2000cwz.googlegroups.com...
[snip]
> I hope that cleared up some of the previous questions.... anyways...
> from there I do the following :
>
> send(client,"somestring",11,0); // which sends some text over to my
> client program
>
> So the real problem is that TCP buffers it's output to the socket....
> so when I send a few bytes over the socket.... it doesn't get sent
> right away. It sits in the buffer (someone mentioned until an ACK is
> received)... or until the buffer is full.
Can you briefly describe the protocol?
Do you issue multiple small send() calls in rapid succession?
How long are the delays you see?
Alex
| |
| Sparticus 2005-12-13, 7:04 pm |
| It's basically a bunch of small send calls in rapid succession....
The client connects.... the server send a very small amount of info the
the client (ie. 15 bytes). Once the client receives the data, it
responds with more data.... the server recieves the data... and reponds
immediately with more data....
That continues about 6 times....
Sometimes the delays don't exists.... the server and the client talk
back and forth very very quick (under half a second total)... then
other times, the client sends the server data and the server waits
sometimes 15 seconds before responding (the data is waiting in the
buffer to be sent).... there are also other times where the output
buffer never releases the data (well, not after 40 seconds anyways).
Thanx for responding...
Ryan
| |
| David Schwartz 2005-12-13, 7:04 pm |
|
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134490194.170374.206750@z14g2000cwz.googlegroups.com...
> I hope that cleared up some of the previous questions.... anyways...
> from there I do the following :
>
> send(client,"somestring",11,0); // which sends some text over to my
> client program
>
> So the real problem is that TCP buffers it's output to the socket....
Why do you say that? What is the symptom?
> so when I send a few bytes over the socket.... it doesn't get sent
> right away. It sits in the buffer (someone mentioned until an ACK is
> received)... or until the buffer is full.
How are you determining this? Are you looking at the packets on the
network? Or are you looking in an application at the other end?
> What I need is a way to "send" data... and it actually sends right
> away.... don't buffer anything....
This is so frustrating. You keep telling us what you want and what you
think is causing your problem, but we have no clue what your actual problem
is.
> I've read on google that what I need is "TCP_NODELAY" added.... as you
> can see from my code above, I think I added it correctly... but the
> output still seems to be buffered...
Why do you say that? What does "seems to be buffered" mean?
> Can someone please point me in the right direction? It's been 2 days
> now of reading on google and I'm getting no where...
What goes wrong in your application?
DS
| |
| Alex Fraser 2005-12-13, 7:04 pm |
| "Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134498494.079776.238030@f14g2000cwb.googlegroups.com...
> It's basically a bunch of small send calls in rapid succession....
>
> The client connects.... the server send a very small amount of info the
> the client (ie. 15 bytes). Once the client receives the data, it
> responds with more data.... the server recieves the data... and reponds
> immediately with more data....
I should have been more explicit: when I said send() calls "in rapid
succession" I also meant without intervening recv() calls.
As that isn't the case - the client and server alternate sending and
receiving - TCP_NODELAY is more likely to do harm than good (so you should
not use it).
> Sometimes the delays don't exists.... the server and the client talk
> back and forth very very quick (under half a second total)... then
> other times, the client sends the server data and the server waits
> sometimes 15 seconds before responding (the data is waiting in the
> buffer to be sent).... there are also other times where the output
> buffer never releases the data (well, not after 40 seconds anyways).
It seems there are two basic possibilities: either you are mistaken - there
is no data buffered to be sent (perhaps due to a bug, perhaps delayed due to
a problem with something external like a database), or it isn't being sent
because of some kind of network problem.
From what you describe above, if there is a network problem, running tcpdump
on the client and server machines would probably show TCP segments being
resent (one direction or the other) during the time when you see these large
delays.
Tracing the syscalls made by the client and server might also offer clues.
Alex
| |
| David Schwartz 2005-12-13, 7:04 pm |
|
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134498494.079776.238030@f14g2000cwb.googlegroups.com...
> It's basically a bunch of small send calls in rapid succession....
Okay, don't do that.
> The client connects.... the server send a very small amount of info the
> the client (ie. 15 bytes). Once the client receives the data, it
> responds with more data.... the server recieves the data... and reponds
> immediately with more data....
>
> That continues about 6 times....
That's fine. That's is not "a bunch of small send calls in rapid
succession", that is "one send call followed by waiting for data".
> Sometimes the delays don't exists.... the server and the client talk
> back and forth very very quick (under half a second total)... then
> other times, the client sends the server data and the server waits
> sometimes 15 seconds before responding (the data is waiting in the
> buffer to be sent).... there are also other times where the output
> buffer never releases the data (well, not after 40 seconds anyways).
How did you establish that the data is waiting in the buffer to be sent?
TCP should never hold data for that long except in the face of major packet
loss.
DS
| |
| Sparticus 2005-12-13, 7:04 pm |
| Ok, I apologize for not being clear. I'm am fairly new to all this so
me not being clear is more lack of knowledge on my part.
When I had mentioned that it was a bunch of small calls in
succession... this is what I meant :
1.) Client connects to server
2.) Once connection is successful, client sends in their username and
password to server
3.) Server recieves data, checks database to see if username/password
is correct, if so let cleint know that they logged in successfully.
otherwise let client know login failed.
4.) Client receives message... assumign login was successful, they then
pass to server some request (ie. what is the date today)
5.) Server receives message, and replies
6.) Client receives message, then sends another request...
7.) etc...etc...
So it really isn't a bunch of quick "sends" in a row.... but each step
cannot continue until the previous step completes...
You had asked why I think it's buffering the data.... this is why I
think it is... I am probably missing something....
printf("one\n");
send(client,"somestring",11,0);
printf("two\n");
I added this to my server C code.... when run.... it prints to the
console both "one" and "two"..... but my client doesn't receive the
message.... I am fairly suree it's not the clients fault only because I
have "sends" thoughout my server code and it usually receives them
all.... but randomly a send never makes it to the other side....(by
randomly I mean if I run the program 10 times... 8 times outta 10 the
sent message will receive at the client.... the other 2 it never
arrives....
But I agree with you all.... I really should get a packet sniffer or
somethign and investigate more.... i'll write back in a bit when I have
more info.
Once again, I apologize for not being clear.... I am fairly new to all
this....
Thanx again,
Ryan
| |
| David Schwartz 2005-12-13, 7:04 pm |
|
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134506182.726535.231590@o13g2000cwo.googlegroups.com...
> You had asked why I think it's buffering the data.... this is why I
> think it is... I am probably missing something....
>
> printf("one\n");
> send(client,"somestring",11,0);
> printf("two\n");
>
> I added this to my server C code.... when run.... it prints to the
> console both "one" and "two"..... but my client doesn't receive the
> message....
I bet it did, you just didn't notice. I say that because you used the
magic word "message".
> I am fairly suree it's not the clients fault only because I
> have "sends" thoughout my server code and it usually receives them
> all.... but randomly a send never makes it to the other side....(by
> randomly I mean if I run the program 10 times... 8 times outta 10 the
> sent message will receive at the client.... the other 2 it never
> arrives....
How did you establish that it didn't arrive?
> Once again, I apologize for not being clear.... I am fairly new to all
> this....
I think this is the classic mistake of thinking that TCP sends and
receives messages. TCP sends and receives *bytes*. Output the number of
*bytes* sent and received on both sides and I'll bet they'll match up.
DS
| |
| Sparticus 2005-12-13, 7:04 pm |
| Well, I installed a packet sniffer on my machine. I am trying real
hard to understand what all the "stuff" is...
Anyways, I started the packet sniffer and then ran my program. I found
the line where data was sent from the client to the server. The very
next line is a TCP packet from the server (on the correct ports... ie,
the server and client are commumincating on port 1234) and the pack is
apparently an ACK packet. I assume that is the server's way of saying
I received the info you sent me....
However, after that, I receive nothing at all from the server on port
1234.... even though on the server console it says "one" and "two" (as
explained in my previous post).
So, the line of code that should send the data to the client had run...
but nothign was received....
Am I missing something... or is the data really getting buffered on the
server end?
| |
| David Schwartz 2005-12-13, 7:04 pm |
|
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote in message
news:1134516011.088661.213670@f14g2000cwb.googlegroups.com...
> Anyways, I started the packet sniffer and then ran my program. I found
> the line where data was sent from the client to the server. The very
> next line is a TCP packet from the server (on the correct ports... ie,
> the server and client are commumincating on port 1234) and the pack is
> apparently an ACK packet. I assume that is the server's way of saying
> I received the info you sent me....
>
> However, after that, I receive nothing at all from the server on port
> 1234.... even though on the server console it says "one" and "two" (as
> explained in my previous post).
That proves nothing.
> So, the line of code that should send the data to the client had run...
> but nothign was received....
How do you know nothing was received?
> Am I missing something... or is the data really getting buffered on the
> server end?
No, it was probably sent in that packet you saw. Can you 100% confirm
that the call to 'send' occured *after* that packet was sent? If not, what
does this prove?
DS
| |
| Barry Margolin 2005-12-13, 9:57 pm |
| In article <1134516011.088661.213670@f14g2000cwb.googlegroups.com>,
"Sparticus" <sparticusREMOVE@thesparticusarena.com> wrote:
> Well, I installed a packet sniffer on my machine. I am trying real
> hard to understand what all the "stuff" is...
Why did you have to install anything? Are there really versions of Unix
that don't come with tcpdump or snoop already installed? If you need a
nice GUI then I suppose you would have to install Ethereal, I don't
think it comes preinstalled on many systems yet.
>
> Anyways, I started the packet sniffer and then ran my program. I found
> the line where data was sent from the client to the server. The very
> next line is a TCP packet from the server (on the correct ports... ie,
> the server and client are commumincating on port 1234) and the pack is
> apparently an ACK packet. I assume that is the server's way of saying
> I received the info you sent me....
>
> However, after that, I receive nothing at all from the server on port
> 1234.... even though on the server console it says "one" and "two" (as
> explained in my previous post).
>
> So, the line of code that should send the data to the client had run...
> but nothign was received....
>
> Am I missing something... or is the data really getting buffered on the
> server end?
It's hard to tell what's going on from your description. Can you post
the section of the packet dump that shows this?
Is the client's receive window open?
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
| |
| Andrei Voropaev 2005-12-14, 4:00 am |
| On 2005-12-13, Sparticus <sparticusREMOVE@thesparticusarena.com> wrote:
>
> Ok, I believe I am using TCP sockets. I included sys/socket.h and
> below is how I create the socket :
>
> void set_up_socket()
> {
[...]
> // Start listening on the socket
> if(listen(sockfd, 100) ,0)
[...]
Do you actually call accept on this socket later? If you don't accept
the connection and don't read the accepted connection, then the client
may connect but no data will be exchanged.
--
Minds, like parachutes, function best when open
|
|
|
|
|