Unix Programming - TCP non-blocking multithreads preemption

Author

TCP non-blocking multithreads preemption

Ray

2007-09-28, 7:11 pm

Hi,

I am designing an application with two threads running with different
priority. Each thread uses the same TCP address and port to send
information out (i know i can easily change to have different port,
but i just wanted to know what would happen if they use the same
port).

My question is what data would the TCP send buffer contains if one
thread is running non-blocking send operation while another thread
with higher priority happens to preempt the previous thread to do
another non-blocking send operation.

For example,

Thread A has 10000 bytes to send. It is supposed to put 10000 bytes
to the send buffer, but the window size is smaller than that, say it
is
8000 bytes. So only 8000 bytes of 10000 bytes data is copied to the
send buffer and do the send operation for thread A.

While thread A is sending the 5000th bytes, thread B preempts it and
trys to send 4000 bytes of data. remember both thread A and thread B
are sharing the same connection with same address and port.

Would the send buffer look like 5000 bytes from A + 3000 bytes from
B,
then after thread A is done sending 5000 bytes and thread B is done
sending 3000 bytes, the send buffer looks like 1000 bytes from B +
3000 bytes from A + 2000 bytes from A? or it has different scheme?

Remember both send operations are running on non-blocking mode. What
if both send operations are running on blocking mode?

Thanks
Ray

Barry Margolin

2007-09-28, 10:10 pm

In article <1191009936.306068.53380@d55g2000hsg.googlegroups.com>,
Ray <alienatsf@gmail.com> wrote:

> Hi,
>
> I am designing an application with two threads running with different
> priority. Each thread uses the same TCP address and port to send
> information out (i know i can easily change to have different port,
> but i just wanted to know what would happen if they use the same
> port).
>
>
> My question is what data would the TCP send buffer contains if one
> thread is running non-blocking send operation while another thread
> with higher priority happens to preempt the previous thread to do
> another non-blocking send operation.
>
>
> For example,
>
> Thread A has 10000 bytes to send. It is supposed to put 10000 bytes
> to the send buffer, but the window size is smaller than that, say it
> is
> 8000 bytes. So only 8000 bytes of 10000 bytes data is copied to the
> send buffer and do the send operation for thread A.
>
>
> While thread A is sending the 5000th bytes, thread B preempts it and
> trys to send 4000 bytes of data. remember both thread A and thread B
> are sharing the same connection with same address and port.
>
>
> Would the send buffer look like 5000 bytes from A + 3000 bytes from
> B,
> then after thread A is done sending 5000 bytes and thread B is done
> sending 3000 bytes, the send buffer looks like 1000 bytes from B +
> 3000 bytes from A + 2000 bytes from A? or it has different scheme?

Since A has already sent 5,000 bytes, there are only 3,000 bytes in the
send buffer when B calls send(). The send buffer will then be 3,000
bytes from A followed by 4,000 bytes from B.

> Remember both send operations are running on non-blocking mode. What
> if both send operations are running on blocking mode?

In this case, I'm not sure that the specification says precisely what
happens. They may be interleaved as in the non-blocking case, or the
kernel might try to perform each call atomically.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Yakov

2007-09-30, 8:07 am

On Sep 28, 10:05 pm, Ray <aliena...@gmail.com> wrote:
> Hi,
>
> I am designing an application with two threads running with different
> priority. Each thread uses the same TCP address and port to send
> information out (i know i can easily change to have different port,
> but i just wanted to know what would happen if they use the same
> port).
>
> My question is what data would the TCP send buffer contains if one
> thread is running non-blocking send operation while another thread
> with higher priority happens to preempt the previous thread to do
> another non-blocking send operation.
>
> For example,
>
> Thread A has 10000 bytes to send. It is supposed to put 10000 bytes
> to the send buffer, but the window size is smaller than that, say it
> is
> 8000 bytes. So only 8000 bytes of 10000 bytes data is copied to the
> send buffer and do the send operation for thread A.
>
> While thread A is sending the 5000th bytes, thread B preempts it and
> trys to send 4000 bytes of data. remember both thread A and thread B
> are sharing the same connection with same address and port.
>
> Would the send buffer look like 5000 bytes from A + 3000 bytes from
> B,
> then after thread A is done sending 5000 bytes and thread B is done
> sending 3000 bytes, the send buffer looks like 1000 bytes from B +
> 3000 bytes from A + 2000 bytes from A? or it has different scheme?
>
> Remember both send operations are running on non-blocking mode. What
> if both send operations are running on blocking mode?

You might easily get intermixed stream *both* in blocking and in
non-blocking mode, if you forget that blocking send() can legally
return incomplete count, perfectly legally.

This all depends on the amount of data currently in kernel send
buffer.
Again, in blocking mode, send() will perfectly legally return
incomplete
count to you, especially under stress.

The correct solution is either
(1) to dedicate additional thread to writing properly
serialized data to the socket and keep the queue of data to this
thread, or
(2) serialize between threads using mutexes and/or cv or
(3) use separate sockets.

Yakov

Sponsored Links