Sorting and filtering blocks of text

System setup: jboss app server, Sun box running Solaris 8.
The jboss log has tons of entries that have the pattern, line with
word INFO followed by several lines with the word ERROR and then  the
whole thing starts over with a INFO line.

08:15:33,923 INFO  [STDOUT] Mon Mar 22 08:15:33 EST 2004 : Mon Mar 22
08:15:33 EST 2004 : skeva.engage.exception.InvalidSessio
nException: The session has either timed out or is no longer valid.
Please logon.
08:15:33,934 ERROR [STDERR]
skeva.engage.exception.InvalidSessionException: The session has either
timed out or is no longer v
alid.  Please logon.
08:15:33,936 ERROR [STDERR]     at
skeva.engage.controller.EngageServletUtil.getUserSession(EngageServletUtil.j
ava:681)
08:15:33,936 ERROR [STDERR]     at
skeva.engage.controller.EngageServletUtil.checkSessionStatus(EngageServletUt
il.java:250)
08:15:33,936 ERROR [STDERR]     at
org.apache.jsp.main_jsp._jspService(main_jsp.java:145)
08:15:33,936 ERROR [STDERR]     at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
08:15:33,937 ERROR [STDERR]     at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
08:15:33,937 ERROR [STDERR]     at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:2
10)
08:15:33,937 ERROR [STDERR]     at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
08:15:33,937 ERROR [STDERR]     at
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
08:15:33,938 ERROR [STDERR]     at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
08:15:33,938 ERROR [STDERR]     at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360)
08:15:33,938 ERROR [STDERR]     at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandl
er.java:294)
08:15:33,938 ERROR [STDERR]     at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:558)
08:15:33,939 ERROR [STDERR]     at
org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
08:15:33,939 ERROR [STDERR]     at
org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext
.java:507)
08:15:33,939 ERROR [STDERR]     at
org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
08:15:33,939 ERROR [STDERR]     at
org.mortbay.http.HttpServer.service(HttpServer.java:863)
08:15:33,940 ERROR [STDERR]     at
org.jboss.jetty.Jetty.service(Jetty.java:460)
08:15:33,940 ERROR [STDERR]     at
org.mortbay.http.HttpConnection.service(HttpConnection.java:775)
08:15:33,940 ERROR [STDERR]     at
org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:939)
08:15:33,940 ERROR [STDERR]     at
org.mortbay.http.HttpConnection.handle(HttpConnection.java:792)
08:15:33,941 ERROR [STDERR]     at
org.mortbay.http.SocketListener.handleConnection(SocketListener.java:201)
08:15:33,941 ERROR [STDERR]     at
org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
08:15:33,941 ERROR [STDERR]     at
org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:455)
08:15:33,975 INFO  [STDOUT] Mon Mar 22 08:15:33 EST 2004 : Mon Mar 22
08:15:33 EST 2004 : skeva.engage.exception.InvalidSessionException:
The session has either timed out or is no longer valid.  Please logon.

Currently these blocks of InvalidSessionException errors aren't that
helpful and make finding the more important errors harder to find.  So
if I search on /INFO/, /ERROR/ I get all the blocks but discarding the
block that follow a InvalidSessionException is not working for me.  I
either get all on none.

Any advise you can give would appreciated.

Thanks...Mike Bradley

"Mike Bradley" <mbradley@mgh.org> wrote in message news:e09b3e84.0403230514.5b720592@postin
g.google.com...
> System setup: jboss app server, Sun box running Solaris 8.
> The jboss log has tons of entries that have the pattern, line with
> word INFO followed by several lines with the word ERROR and then  the
> whole thing starts over with a INFO line.
>
> 08:15:33,923 INFO  [STDOUT] Mon Mar 22 08:15:33 EST 2004 : Mon Mar 22
> 08:15:33 EST 2004 : skeva.engage.exception.InvalidSessio
> nException: The session has either timed out or is no longer valid.
> Please logon.
> 08:15:33,934 ERROR [STDERR]
> skeva.engage.exception.InvalidSessionException: The session has either
> timed out or is no longer v
> alid.  Please logon.
> 08:15:33,936 ERROR [STDERR]     at
> skeva.engage.controller.EngageServletUtil.getUserSession(EngageServletUtil
.java:681)
> 08:15:33,936 ERROR [STDERR]     at
> skeva.engage.controller.EngageServletUtil.checkSessionStatus(EngageServlet
Util.java:250)
> 08:15:33,936 ERROR [STDERR]     at
> org.apache.jsp.main_jsp._jspService(main_jsp.java:145)
> 08:15:33,936 ERROR [STDERR]     at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
> 08:15:33,937 ERROR [STDERR]     at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
> 08:15:33,937 ERROR [STDERR]     at
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java
:210)
> 08:15:33,937 ERROR [STDERR]     at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
> 08:15:33,937 ERROR [STDERR]     at
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
> 08:15:33,938 ERROR [STDERR]     at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
> 08:15:33,938 ERROR [STDERR]     at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360)
> 08:15:33,938 ERROR [STDERR]     at
> org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHan
dler.java:294)
> 08:15:33,938 ERROR [STDERR]     at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:558)
> 08:15:33,939 ERROR [STDERR]     at
> org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
> 08:15:33,939 ERROR [STDERR]     at
> org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationConte
xt.java:507)
> 08:15:33,939 ERROR [STDERR]     at
> org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
> 08:15:33,939 ERROR [STDERR]     at
> org.mortbay.http.HttpServer.service(HttpServer.java:863)
> 08:15:33,940 ERROR [STDERR]     at
> org.jboss.jetty.Jetty.service(Jetty.java:460)
> 08:15:33,940 ERROR [STDERR]     at
> org.mortbay.http.HttpConnection.service(HttpConnection.java:775)
> 08:15:33,940 ERROR [STDERR]     at
> org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:939)
> 08:15:33,940 ERROR [STDERR]     at
> org.mortbay.http.HttpConnection.handle(HttpConnection.java:792)
> 08:15:33,941 ERROR [STDERR]     at
> org.mortbay.http.SocketListener.handleConnection(SocketListener.java:201)
> 08:15:33,941 ERROR [STDERR]     at
> org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
> 08:15:33,941 ERROR [STDERR]     at
> org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:455)
> 08:15:33,975 INFO  [STDOUT] Mon Mar 22 08:15:33 EST 2004 : Mon Mar 22
> 08:15:33 EST 2004 : skeva.engage.exception.InvalidSessionException:
> The session has either timed out or is no longer valid.  Please logon.
>
> Currently these blocks of InvalidSessionException errors aren't that
> helpful and make finding the more important errors harder to find.  So
> if I search on /INFO/, /ERROR/ I get all the blocks but discarding the
> block that follow a InvalidSessionException is not working for me.  I
> either get all on none.
>

I can't quite make out how much you want to discard.

Aiui, a block starts with an INFO line and continues
through any number of ERROR lines; there is no end-of-block
marker; the next INFO line starts the next block.
Is that right? And the lack of an end-of-block marker is
your main problem?

Being lazy, my inclination is to preprocess the log in order
to add an end-of-block marker, such as a blank line or
something more explicit, then pipe that into your existing
program.

awk '
/INFO/ { print "END-OF-BLOCK" }
{ print }
' logfile | awk '/INFO/,/END_OF_BLOCK/ { ... }'

--
John.

"John L" <jl@lammtarra.fslife.co.uk> wrote in message news:<c3pp59$kvc$1@news6.svr.pol.co.u
k>...
> "Mike Bradley" <mbradley@mgh.org> wrote in message news:e09b3e84.040323051
4.5b720592@posting.google.com... 
>
> I can't quite make out how much you want to discard.
>
> Aiui, a block starts with an INFO line and continues
> through any number of ERROR lines; there is no end-of-block
> marker; the next INFO line starts the next block.
> Is that right? And the lack of an end-of-block marker is
> your main problem?
>
> Being lazy, my inclination is to preprocess the log in order
> to add an end-of-block marker, such as a blank line or
> something more explicit, then pipe that into your existing
> program.
>
>  awk '
>     /INFO/ { print "END-OF-BLOCK" }
>     { print }
>     ' logfile | awk '/INFO/,/END_OF_BLOCK/ { ... }'

I was searching on /ERROR/,/INFO/ using the first INFO after an ERROR
as the end of block marker.  What I would like to happen is that any
block that contained InvalidSesson would be tossed out and the
remaining blocks could be written to another file or emailed.  So read
the file find

ERROR
text
more text
yet more text
INFO

then examine this block for InvalidSession if found in block discard
entire block from ERROR down to INFO. If not found write the block to
a file and go to the next block.

Thanks...Mike

"Mike Bradley" <mbradley@mgh.org> wrote in message news:e09b3e84.0403260908.52766267@postin
g.google.com...
> "John L" <jl@lammtarra.fslife.co.uk> wrote in message news:<c3pp59$kvc$1@n
ews6.svr.pol.co.uk>... 
>
> I was searching on /ERROR/,/INFO/ using the first INFO after an ERROR
> as the end of block marker.  What I would like to happen is that any
> block that contained InvalidSesson would be tossed out and the
> remaining blocks could be written to another file or emailed.  So read
> the file find
>
> ERROR
> text
> more text
> yet more text
> INFO
>
> then examine this block for InvalidSession if found in block discard
> entire block from ERROR down to INFO. If not found write the block to
> a file and go to the next block.
>

Forget the file format for a moment. If we can find a valid
record separator, then your program will be this:

awk '
BEGIN { RS = "RECORD_SEPARATOR" }
! /InvalidSession/ { print }
'

Now, aiui, you can use INFO as that record separator,
but then you will have to cheat to print INFO into
the file, but that is easy enough.

Or you can pre-process it, to add the separator
in front of each INFO. I'd do that, because it
makes it slightly easier to see what is going on
when you come to change it two years from now.

You can probably use a blank line as that record
separator, but you'd need to check the log file format.

--
John.

Mike Bradley wrote:

> "John L" <jl@lammtarra.fslife.co.uk> wrote in message news:<c3pp59$kvc$1@n
ews6.svr.pol.co.uk>...
> 

<snip>

> I was searching on /ERROR/,/INFO/ using the first INFO after an ERROR
> as the end of block marker.  What I would like to happen is that any
> block that contained InvalidSesson would be tossed out and the
> remaining blocks could be written to another file or emailed.  So read
> the file find
>
> ERROR
> text
> more text
> yet more text
> INFO
>
> then examine this block for InvalidSession if found in block discard
> entire block from ERROR down to INFO. If not found write the block to
> a file and go to the next block.

I tried reading the posts a couple of times but I'm still not sure if
you wanted to start with an INFO or end with one. Assuming your block
starts with INFO and ends with the last ERROR before another INFO or the
end of file, and that you want to ignore any block that contains the
test "InvalidSessionException", this is it (untested):

awk 'function prtBlock() {
for (i=0; i < blockSize; i++) { print block[i++] }
blockSize = 0
saveBlock = 1
}
$2 == "INFO" { prtBlock() }
/InvalidSessionException/ { saveBlock = 0; blockSize = 0 }
saveBlock == 1 { block[blockSize++] = $0 }
END { prtBlock() }'

Regards,

Ed.