Java Help - SocketChannels, Selectors, Iterators and ConcurrentModificationExceptions,

Author

SocketChannels, Selectors, Iterators and ConcurrentModificationExceptions,

Frank van Schie

2006-05-19, 7:06 pm

[Note: Reposted my question from comp.lang.java. This group is more
appropriate, not to mention more active.]

Hi there,

I'm trying to make an instant messaging connection library that's rather
sparse with Threads, due to it being meant for a transport for XMPP.
Most libraries out there create a Thread for damn near everything that's
going on, so I'm writing my own.

I'm running into a bit of trouble with my Selector, which I periodically
poll from the lone Thread I have so far. If I have just one
SocketChannel registered, I have no problems, and the thing connects
fine and receives all the input it should. If I have two connections
open at the same time, then when I get my SelectionKey iterator and hit
'next()', it blows up with a java.util.ConcurrentModificationException.

While the iterator can throw the Exception if there is even the chance
of the iterator's underlying data being modified, I am

by this
not being a problem if I only have the single channel open. I have
checked and verified that I am not accidentally starting two Threads
(and even so, wouldn't the synchronization on the iterator be sufficient
to satify it?)

I'm not experienced at all with socket programming in general, and
java.nio.* in particular, so I wonder if anyone here could shed some light?

Some code from my manager Thread's run method:

public void run()
{
[...]
while(!stopped)
{
this.selector.selectNow();
Iterator<SelectionKey> selectionKeyIterator =
selector.selectedKeys().iterator();
synchronized(selectionKeyIterator)
{
SelectionKey selectionKey = null;
while(selectionKeyIterator.hasNext())
{
selectionKey = selectionKeyIterator.next(); // This goes BOOM.
selectionKeyIterator.remove();
/**
* Verify that the key is valid, get the socketchannel from the
* key, get my connection object, have it read the data, send it
* for processing by a 'listener' of sorts, etc...
}
}

Thread.sleep(50); // Complete with try/catch, natch.
}
}

--
Frank

Frank van Schie

2006-05-20, 7:04 pm

Matt Humphrey wrote:
> "Frank van Schie" <frankNOSPAM@email.it> wrote in message
> news:3ImdnVbOE6mHs_PZnZ2dnUVZ8qadnZ2d@ca
sema.nl...
>
> The synchronized statement is faulty in your code because each thread is
> synchronizing on a different object. The javadoc for Selector indicates
> that the key set is not thread safe. Are you sure only one thread is
> accessing the selector?A remove in one thread will likely cause the
> ConcurrentModificationException in the other.

I am certain I am starting only one Thread that handles the selector. I
have only the single Runnable object, and debug output in the run()
method shows up only once. Unless Threads can be forked, and that
without my explicit demand, I don't see how that could happen.

Besides that, my Runnable object has a private constructor, a static
reference to itself, and a getInstance() method that ensures only one
instance exists at any time. I made sure there was only one connection
manager, and the only other thread is the main one. It calls a method in
my manager to create an object which creates a SocketChannel and
registers it to the selector. I can see concurrency problems there, but
that's what I was hoping 'synchronized' would fix.

I have since replaced the synchronized statement by having it
synchronize on a different (class) attribute. No joy. Occasionally it
works fine, often it fails, but if it fails it's always right after
creating another SocketChannel and registering it with the selector, and
it always fails on the line where I get the 'next()' from the iterator.

> I othrewise don't see anything wrong with the iterator, unless the code
> section commented out also adds to or removes from the
> selector.selectedKeys().

Nope. It operates on the iterator. Now, I understand that not removing
things from the iterator causes problems with subsequent calls, so I'm
assuming that modifying the iterator causes the underlying
Set<SelectionKey> to be modified, which is the basis for the whole problem.

Thanks for the response. I'll be trying out a few more ideas and learn
about the beginnings of thread-safe programming at the same time.
--
Frank

Dale King

2006-05-21, 7:09 pm

Frank van Schie wrote:
> [Note: Reposted my question from comp.lang.java. This group is more
> appropriate, not to mention more active.]

And better yet this group actually exists. Comp.lang.java is just a
figment of your news server's imagination.

--
Dale King

Frank van Schie

2006-05-22, 7:12 pm

Dale King wrote:
> Frank van Schie wrote:
>
> And better yet this group actually exists. Comp.lang.java is just a
> figment of your news server's imagination.

Apparently a common and popular affliction, what with 40k posts on my
server (with about the same here, and 150-160k in c.l.j.programmer). But
I'm willing to take your word for it, gullible critter that I am.

And on-topic: The problem lay in that I, apparently, did have some code
in a function outside the main thread loop, which got called
asynchronously, and so interfered with the thread.

I fixed it by moving all code to do with adding, removing or altering a
selection key's interests to the main thread. I use a data structure to
store any changes to be made with all the information I need (like
SocketChannels to watch out for), then check that data structure to see
if it's empty. If it's not empty, I process it. I think this will be
cheap enough for my purposes.

Followup question that isn't entirely easy to Google for, and it would
be nice to have some idea: Assuming a fairly simply text-based protocol
like MSN (which is what I'm building an XMPP transport for), what would
be the more likely limiting scalability factor on a dual Xeon box with a
2 gigs of RAM running linux? CPU, memory, available sockets, Microsoft
not being nice? Unable to determine without more info?

--
Frank

Dale King

2006-05-22, 7:12 pm

Frank van Schie wrote:
> Dale King wrote:
>
> Apparently a common and popular affliction, what with 40k posts on my
> server (with about the same here, and 150-160k in c.l.j.programmer).

Sure, I know that there are many news servers with very active
imaginations. That doesn't change the fact that the comp.lang.java group
does not officially exist. It is a phantom created by news servers that
can't deal with reality and keep ignoring the RMGROUP messages that they
have been sent since around 1997. comp.lang.java was split into the
hierarchy we have today back in the 1996-1997 time frame and
comp.lang.java was removed.

> But
> I'm willing to take your word for it, gullible critter that I am.

You don't have to take my word for it. Here is a nice helpful page from
Jon Skeet (see number 3):

http://www.yoda.arachsys.com/java/newsgroups.html

Note here is the official list of newsgroups, which does not list
comp.lang.java:

ftp://ftp.isc.org/pub/usenet/CONFIG/newsgroups

This has been discussed here many times in the past. Despite what your
misconfigured news server is telling you comp.lang.java does not exist.

--
Dale King

Frank van Schie

2006-05-22, 7:12 pm

Dale King wrote:

[non-existant group has 40k messages on my server]

> Sure, I know that there are many news servers with very active
> imaginations. That doesn't change the fact that the comp.lang.java group
> does not officially exist. It is a phantom created by news servers that
> can't deal with reality and keep ignoring the RMGROUP messages that they
> have been sent since around 1997. comp.lang.java was split into the
> hierarchy we have today back in the 1996-1997 time frame and
> comp.lang.java was removed.
[snip]
> This has been discussed here many times in the past. Despite what your
> misconfigured news server is telling you comp.lang.java does not exist.

Dude. Unclench. I harbor no illusions about the validity of the groups
list on Usenet. Consider the c.l.j (there is no c.l.j) dropped from my
subscriptions. Thanks for the heads up.
--
Frank

Sponsored Links