poll and popen in 2 threads

Hi,

I have an application with two threads. In the first thread, I often do a
poll on an array of filedescriptors. In the second thread I sometimes have
a popen.

The application works really good...mostly :) But sometimes, when popen and
poll seem to run at the same time the application hangs up. The thread,
where the poll is currently active does not continue. Is it possible, that
poll and popen can not run at the same time? Should I lock it with mutexes?

The popen thread first does a popen and then, on success continues with a
fgets. What could be the problem? The popen or the fgets? Should I lock
only the popen, only the fgets part? Or both?

Greetings

Mike

Michael Gebhart wrote:
> Hi,
>
> I have an application with two threads. In the first thread, I often do a
> poll on an array of filedescriptors. In the second thread I sometimes have
> a popen.
>
> The application works really good...mostly :) But sometimes, when popen an
d
> poll seem to run at the same time the application hangs up. The thread,
> where the poll is currently active does not continue. Is it possible, that
> poll and popen can not run at the same time? Should I lock it with mutexes
?
>
> The popen thread first does a popen and then, on success continues with a
> fgets. What could be the problem? The popen or the fgets? Should I lock
> only the popen, only the fgets part? Or both?
>
> Greetings
>
> Mike

"application hangs up"  ???
Is it burning CPU or just blocked on input ?

NB "popen" is just a fork/exec. Are you running out of resources (fds,
processes) Are there child processes present, when hung ?

BTW, if you are using threads, you implicitly (by design) allow the
fgets to block. The other thread should not suffer from that.

HTH,
AvK

Hi, thanks for your answer.

> "application hangs up"  ???
> Is it burning CPU or just blocked on input ?

It's just blocked. Poll does not continue. I have also set a timeout for
poll, but the poll completely hangs.

> NB "popen" is just a fork/exec. Are you running out of resources (fds,
> processes) Are there child processes present, when hung ?

Yes, the programm I am starting with popen is active. But there is no reason
not to continue. The popen opens a shellscript and this only returns an
short string. The shellscript looks like this:

#!/bin/sh
echo "check"

So that should not hang up, so also fgets should not hang. If fget would be
blocked, the poll should continue although. I don't know, why popen is able
to block the poll. They do not depend on each other, so the other thread
should alway continue, if one thread is blocked.

> BTW, if you are using threads, you implicitly (by design) allow the
> fgets to block. The other thread should not suffer from that.

Yup, I know. But there is no reason, why fgets should be blocked. And
nevertheless: The poll should not be blocked by this.


Mike

Michael Gebhart wrote:

> It's just blocked. Poll does not continue. I have also set a timeout for
> poll, but the poll completely hangs.
>

> Yes, the programm I am starting with popen is active. But there is no reas
on
> not to continue. The popen opens a shellscript and this only returns an
> short string. The shellscript looks like this:
>
> #!/bin/sh
> echo "check"
>
> So that should not hang up, so also fgets should not hang. If fget would b
e
> blocked, the poll should continue although. I don't know, why popen is abl
e
> to block the poll. They do not depend on each other, so the other thread
> should alway continue, if one thread is blocked.

> Yup, I know. But there is no reason, why fgets should be blocked. And
> nevertheless: The poll should not be blocked by this.


fgets can block when there is nothing to read. eg on EOF.

If you say "the poll is blocked", do you mean it *should* return
instead, with som filedescriptors readable ?
Are you sure the fds are readable ? Why ?

>
> Mike

Try to (temporally) replace the popen("script.sh", "r" ), by an
fopen("file", "r" ) . Just to identify/isolate the usual suspects.

NB I don't know how fork/exec behaves in case of a multi-threaded
program. Probably the child process will have the same (number of)
threads, but the exec will finish them off anyway.

AvK

In article <41C558BA.5060405@localhost>, moi <avk@localhost> wrote:

> Michael Gebhart wrote:
> 
> 
> 
>
>
> fgets can block when there is nothing to read. eg on EOF.

fgets() should not block on EOF, it should return NULL immediately.
Poll() should also return immediately with an indication that the
descriptor is readable.

>
> If you say "the poll is blocked", do you mean it *should* return
> instead, with som filedescriptors readable ?
> Are you sure the fds are readable ? Why ?
> 
>
> Try to (temporally) replace the popen("script.sh", "r" ), by an
> fopen("file", "r" ) . Just to identify/isolate the usual suspects.
>
> NB I don't know how fork/exec behaves in case of a multi-threaded
> program. Probably the child process will have the same (number of)
> threads, but the exec will finish them off anyway.

Maybe it's implementation-dependent, but I think most systems just
duplicate the thread that called fork(), not the entire process.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Barry Margolin wrote:

 
>
>
> fgets() should not block on EOF, it should return NULL immediately.
> Poll() should also return immediately with an indication that the
> descriptor is readable.
>


Oops. I can't believe I actually wrote that ...

Thanks for correcting me :-]
AvK

Hi,
 

poll has to return, because it has a timeout. It returns nearly 10 times per
second, but now it suddenly stops when the other thread is seems to do the
popen action :)

I can nearly use popen 8000 times, then the program is locked. Sometimes it
also works 10000 times. I've tested it.
 

ok, I will try that.
 
>
> Maybe it's implementation-dependent, but I think most systems just
> duplicate the thread that called fork(), not the entire process.

It is very strange, because it works really good, but after about 10000
times of popen it locks. I also do a pclose after popen. There is only one
popen at one moment, but the poll is working at the same time. I will try
the program on another server. Maybe there is a problem with the linux
system.

I have also tried to replace the fgets with read. It works, but it also
locks after about 10000 times.

Greetings

Mike

"Michael Gebhart" <mail@miketech.net> wrote in message
news:cqaltg$gf0$1@news2.rz.uni-karlsruhe.de...

> It is very strange, because it works really good, but after about 10000
> times of popen it locks. I also do a pclose after popen. There is only one
> popen at one moment, but the poll is working at the same time. I will try
> the program on another server. Maybe there is a problem with the linux
> system.

Post the shortest complete, compilable program that demonstrates the
problem.

DS

Hi,

that's not easy *g* I've written a small program, what does a lot of popens
and it works. It must be something special in my big program. But now I've
found out, that popen is not the problem, that works. The program always
crashes at fgets. But I don't know why?

it works great a few times and then it locks at fgets. This is a snippet of
my program:

cout << "fgets follows" << endl;

while(ret_poll = fgets(buffer, sizeof(buffer), fp_in)) {
cout << "fgets liest " << buffer << " returned " << ret_poll << endl;
_ returns = returns + buffer;
}


First the fgets reads the data correctly. Then the file is read and the
while should stop. But after reading, while seems to continue with a new
fgets and then the application locks.

Greetings

Mike

Michael Gebhart wrote:
> Hi,
>
> that's not easy *g* I've written a small program, what does a lot of popen
s
> and it works. It must be something special in my big program. But now I've
> found out, that popen is not the problem, that works. The program always
> crashes at fgets. But I don't know why?
>
> it works great a few times and then it locks at fgets. This is a snippet o
f
> my program:
>
> cout << "fgets follows" << endl;
>
> while(ret_poll = fgets(buffer, sizeof(buffer), fp_in)) {
>   cout << "fgets liest " << buffer << " returned " << ret_poll << endl;
>   returns = returns + buffer;
> }
>
>
> First the fgets reads the data correctly. Then the file is read and the
> while should stop. But after reading, while seems to continue with a new
> fgets and then the application locks.
>
> Greetings
>
> Mike

First, you said it crashed. Now you say it locks. Be more precise,
please. Also: does this *snippet* fail even without other threads active
(within the same process) ?

Did the fgets() reach end of file ? How do you know ?

Gut feeling: [ I don't like the mixing of {threads, C++, networking,
signals} ]

I don't know c++ very well. Are these "cout <<" thingies *thread-safe* ?
Are your other constructors thread-safe ? is the 'new' operator
thread-safe ?
fgets _may_ call malloc() under the hood (typically at the first call
after the fopen) . Is your mixing of c/c++ allocations correct ?

HTH,
AvK