Tcl - Regsub/pipes question

Author

Regsub/pipes question

Kevin Walzer

2006-09-24, 7:03 pm

I'm trying to use regsub to apply tags to certain strings in standard
output from an external process so I can display the data in a text
widget with syntax highlighting.

The following code fails to insert any output at all into the text
widget. I am not sure why. Can anyone enlighten me/suggest a different
way to get tags into standard output that's streamed from a pipe?

set in [open "|echo password | sudo -S tcpdump -i en1 -l -v " r]
while {![eof $in]} {
if {[gets $in line] >= 0} {
set newline [regsub -all {IP} $line {<tag>IP</tag}]
.t insert end $newline\n
.t see end
}

Thanks.
--
Kevin Walzer
Poetic Code
http://www.kevin-walzer.com

Bruce Hartweg

2006-09-25, 7:02 pm

Kevin Walzer wrote:
> I'm trying to use regsub to apply tags to certain strings in standard
> output from an external process so I can display the data in a text
> widget with syntax highlighting.
>
> The following code fails to insert any output at all into the text
> widget. I am not sure why. Can anyone enlighten me/suggest a different
> way to get tags into standard output that's streamed from a pipe?
>
>
> set in [open "|echo password | sudo -S tcpdump -i en1 -l -v " r]
> while {![eof $in]} {
> if {[gets $in line] >= 0} {
> set newline [regsub -all {IP} $line {<tag>IP</tag}]
^ -inline
> .t insert end $newline\n
> .t see end
> }
>
> Thanks.

add the -inline switch to have regsub return the modified string, else
it puts it in the variable provided (which you didn;t provide so it is thrown away)

bruce

Arjen Markus

2006-09-25, 7:02 pm

Kevin Walzer schreef:

> I'm trying to use regsub to apply tags to certain strings in standard
> output from an external process so I can display the data in a text
> widget with syntax highlighting.
>
> The following code fails to insert any output at all into the text
> widget. I am not sure why. Can anyone enlighten me/suggest a different
> way to get tags into standard output that's streamed from a pipe?
>
>
> set in [open "|echo password | sudo -S tcpdump -i en1 -l -v " r]
> while {![eof $in]} {
> if {[gets $in line] >= 0} {
> set newline [regsub -all {IP} $line {<tag>IP</tag}]
> .t insert end $newline\n
> .t see end
> }
>

I'd say:
- Put the handling in a fileevent handler for starters
- Put the text in like this:
.t insert end $newline tagname
- Configure the tag "tagname":
.t tag configure tagname -foreground green

(tags are simply one or more extra arguments when inserting text)

Regards,

Arjen

Helmut Giese

2006-09-25, 7:02 pm

On Mon, 25 Sep 2006 11:58:57 GMT, Bruce Hartweg
<bruce-news@hartweg.us> wrote:
> ^ -inline
>
> add the -inline switch to have regsub return the modified string, else
>it puts it in the variable provided (which you didn;t provide so it is thrown away)
I think there is some misunderstanding here. I regularly do
---
set s "hello world"
set s2 [regsub hello $s howdy]
puts $s2
---
which produces "howdy world".
Best regards
Helmut Giese

Kevin Walzer

2006-09-25, 7:02 pm

Bruce Hartweg wrote:
> Kevin Walzer wrote:
> ^ -inline
>
> add the -inline switch to have regsub return the modified string, else
> it puts it in the variable provided (which you didn;t provide so it is
> thrown away)
>
> bruce

I don't see an "-inline" switch for regsub in the man pages, and using
"inline" threw an error. I think "inline" is just for regexp only. Or is
this a new addition? (I'm using 8.4.13.)

--Kevin

--
Kevin Walzer
Poetic Code
http://www.kevin-walzer.com

Kevin Walzer

2006-09-25, 7:02 pm

Arjen Markus wrote:
> Kevin Walzer schreef:
>
>
> I'd say:
> - Put the handling in a fileevent handler for starters
> - Put the text in like this:
> .t insert end $newline tagname
> - Configure the tag "tagname":
> .t tag configure tagname -foreground green
>
> (tags are simply one or more extra arguments when inserting text)
>
> Regards,
>
> Arjen
>

Arjen,

Thanks. This is closer to what I'm looking for, but not quite. I
probably haven't phrased the question correctly.

Your example above tags all the text green. What I want to do is apply
tags to *specific strings*. So, for instance, I'd like to do something
like this:

..t tag configure redtag -foreground red
set newline [regsub -all {IP} $line [.t insert end {IP} redtag]]
..t insert end $newline\n

What this does is apply the "redtag" attributes to the phrase "IP"; then
the rest of the line is appended to "IP." The output looks something
like this:

IP22:27:28.473752 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42

where the IP is red. But the "IP" is in the wrong place. The original
stream looked like this:

22:27:28.473752 IP 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42

I'm sure my error is in how I'm constructing the Tcl command to apply
"redtag" to the specific string, and where that string is inserted in
the line. I can't quite figure that part out. The examples I've looked
at simply hard-code the markup and don't construct it dynamically, as
I'm trying to do above. Is this feasible to do with the standard Tk text
widget?

I know that the ctext package in tklib supports this kind of
thing--defining tags for specific strings--but I've had difficulty with
it, and I also don't want line numbers in the text widget (ctext is
ideal for code/text editing, but not so much for basic data display).

Thanks for any advice.

Kevin

--
Kevin Walzer
Poetic Code
http://www.kevin-walzer.com

Helmut Giese

2006-09-25, 7:02 pm

On Mon, 25 Sep 2006 10:18:39 -0400, Kevin Walzer <kw@kevin-walzer.com>
wrote:

>Your example above tags all the text green. What I want to do is apply
>tags to *specific strings*. So, for instance, I'd like to do something
>like this:
>
>.t tag configure redtag -foreground red
> set newline [regsub -all {IP} $line [.t insert end {IP} redtag]]
>.t insert end $newline\n
>
>What this does is apply the "redtag" attributes to the phrase "IP"; then
>the rest of the line is appended to "IP." The output looks something
>like this:
>
>IP22:27:28.473752 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42
>
>where the IP is red. But the "IP" is in the wrong place. The original
>stream looked like this:
>
>22:27:28.473752 IP 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42
Hi Kevin,
I think you will need to build a suite of commands which you can then
execute. The way the text widget works, there is no way (that I would
know of) to construct _one_ string which will achieve what you're
after.
Maybe this comes closer to what you want :
---
set s "some text IP some more text"
# build the command (this should be all on one line)
set cmd [regsub (.+)IP(.+) $s ".t insert end \"\\1\" ; .t insert end
IP redtag ; .t inset end \"\\2\""]
# execute it
eval $cmd
---
HTH
Helmut Giese

Bryan Oakley

2006-09-25, 7:02 pm

Kevin Walzer wrote:
> Your example above tags all the text green. What I want to do is apply
> tags to *specific strings*. So, for instance, I'd like to do something
> like this:
>
> .t tag configure redtag -foreground red
> set newline [regsub -all {IP} $line [.t insert end {IP} redtag]]
> .t insert end $newline\n

Tags are applied to the text, they aren't part _of_ the text. So, you
can't interject something into your string before you insert it into the
widget. You either need to apply the tags at the time you insert (eg: .t
insert end "this is red" redtag) or after the fact (eg: .t tag add
redtag $start $end).

I had a need to solve this problem a year or so ago, and I solved it by
writing a proc that iterates over a range of text, calling user code
each time it finds a match. This let me insert the text "raw" then apply
tags for portions of the text, and/or I can apply the tags once for a
whole widget or line by line.

I used it like this, if memory serves (I don't have the code handy):

set start [.t index insert]
.t insert $start $data
fortext .t $start end IP {
.t tag add redtag _matchstart _matchend
}

Would something like that solve your problem? I don't have the tested
code handy, but it is real easy to write. It goes something like this
(untested, off the top of my head):

proc fortext {path start end pattern body} {
$path mark set _start $start
$path mark set _end $end
while {[$path compare _start < _end]} {
set i [$path search -count count $pattern _start _end]
if {$i eq ""} break
$path mark set _matchstart $i
$path mark set _matchend [$path index "_matchstart + $count c"]
uplevel $body
$path mark set _start [$path index "_matchstart + 1 c"]
}
$path mark unset _start _end _matchstart _matchend
}

Arjen Markus

2006-09-25, 7:02 pm

Kevin Walzer schreef:

>
>
> I'm sure my error is in how I'm constructing the Tcl command to apply
> "redtag" to the specific string, and where that string is inserted in
> the line. I can't quite figure that part out. The examples I've looked
> at simply hard-code the markup and don't construct it dynamically, as
> I'm trying to do above. Is this feasible to do with the standard Tk text
> widget?
>

Helmut and Bryan have already explained some of the way the tags work.

Here is my attempt:

pack [text .t] -fill both
..t tag configure redtag -foreground red

set line "A line of text with i's in it"

set anchor 0

foreach ranges [regexp -all -inline -indices {in} $line] {
foreach {start stop} $ranges {break}
.t insert end [string range $line $anchor [expr {$start-1}]]
.t insert end [string range $line $start $stop] redtag
set anchor [expr {$stop+1}]
}
..t insert end [string range $line $anchor end]\n
..t see end

> I know that the ctext package in tklib supports this kind of
> thing--defining tags for specific strings--but I've had difficulty with
> it, and I also don't want line numbers in the text widget (ctext is
> ideal for code/text editing, but not so much for basic data display).
>

You can turn the line numbers off, but what specifically were your
difficulties with ctext?

Regards,

Arjen

Kaitzschu

2006-09-25, 7:02 pm

On Mon, 25 Sep 2006, Kevin Walzer wrote:

> What I want to do is apply tags to *specific strings*. So, for instance,
> I'd like to do something like this:
>
> .t tag configure redtag -foreground red
> set newline [regsub -all {IP} $line [.t insert end {IP} redtag]]
> .t insert end $newline\n
>
> What this does is apply the "redtag" attributes to the phrase "IP"; then
> the rest of the line is appended to "IP." The output looks something
> like this:
>
> IP22:27:28.473752 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42
>
> where the IP is red. But the "IP" is in the wrong place. The original
> stream looked like this:
>
> 22:27:28.473752 IP 10.0.1.253.49395 > 198.32.64.12.domain: UDP, length: 42

As Helmut Giese already suggested, this is by not a task for a one-liner.
But that doesn't mean this isn't doable, or that there is really a need
for "a suite of commands" (of course this is going to take a considerable
amount more code than a one-liner, but code is cheap). Since everyone
loves my poor examples with poor logic and poorer explanations, I'll write
one here. As a bonus I'll include some foolishly long lines and some weak
assumptions about your agenda.

First, we need the tags. Lists are good, so we'll use a list. As I doubt
that .t is really your widgets name we'll use a variable. And since you
are using 8.4.13 (that means you still have a long way ahead to reach the
heaven called 8.5) this example will contain some ugly [evil], so watch
out what is coming in, because that isn't necessarily what gets out.

code:
set tw {.t}
set pattags [list]
set tagc -1
foreach {pattern tagstuff} {
{\d+\.\d+\.\d+\.\d+} {-foreground red}
{UDP|TCP} {-background blue}
} {
eval [linsert $tagstuff 0 $tw tag configure tag[incr tagc]]
lappend pattags $pattern tag$tagc
}

Phew. And we are just starting here.

Here comes an assumption. So far your examples have been using [regsub] so
I believe you have some use for regular expressions. Naturally, I can go
wrong with this one, but hey, I haven't yet quite polished RYMADWYM module
to perfection.
That meaning, we are sooo going to bust up your CPU by using [regexp] and
not [string] commands. Yeehaw!

But what have we done so far? Nothing much, really. Constructed a list
that has a format of
pattern tagname pattern tagname pattern tagname
and created a bunch of tags for our [text] widget (which has been created
in the dark before running any of this code). Next we'll have to put these
funky tags into use. I'll let you figure out where I got the variable line
I'm using .)

[code]
set ii [$tw index insert]
$tw insert insert "$line\n"
foreach {patt tag} $pattags {
foreach ss [regexp -all -indices -inline -- $patt $line] {
foreach {si se} $ss {$tw tag add $tag "$ii + $si chars" "$ii + [incr se] chars"}
}
}
[code]

It just feels that there is something wrong in the Matrix when you see
three nested foreachs around one line of code. Probably there is, but we
aren't here to fix it, Neo has to do something to earn his share.

So what we did here? Again, not much. We just iterated the predefined list
of patterns and tagnames and iterated all start-stop-indices that [regexp]
could reap out of our line, and added the tag to somewhere near those same
spots in our [text].

But.. but.. that's it. Now we have done "sort of" context-coloring. This
is a very bad way to do context-coloring since this is line-based, and can
be run roughly once per line. But as I have assumed, you aren't coloring
up anything editable, so this should fit you just fine.

And I see Bryan Oakley beat my effort by an hour and a magnitude of class,
but I'll post this anyway, go uselessnet posting!

--
-Kaitzschu
s="TCL ";while true;do echo -en "\r$s";s=${s:1:${#s}}${s:0:1};sleep .1;done

Bruce Hartweg

2006-09-25, 7:02 pm

Kevin Walzer wrote:
> Bruce Hartweg wrote:
>
> I don't see an "-inline" switch for regsub in the man pages, and using
> "inline" threw an error. I think "inline" is just for regexp only. Or is
> this a new addition? (I'm using 8.4.13.)
>

Yes, I was misremembering horribly - omitting the variable does cause
it to return the new string, (and this is really pathetic because I
was co-author of TIP 76 - the only one I've ever done.)

Bruce

Kevin Walzer

2006-09-25, 7:02 pm

Arjen Markus wrote:

> You can turn the line numbers off, but what specifically were your
> difficulties with ctext?
>

I finally got ctext to work. Here's an abbreviated version of the script:

package require ctext

ctext .t -linemap 0

pack .t

::ctext::addHighlightClass .t foo blue [list IP >]

set in [open ~/Desktop/packet r]
while {![eof $in]} {
if {[gets $in line] >= 0} {
.t highlight 1.0 end
.t insert end $line\n
.t see end
}
}

I had read the man pages but didn't grok that the "-linemap" flag was
what turned line numbers on and off. And the ".t highlight 1.0 end"
command isn't documented in the man page--I had to cull that from the
sample code at the tklib site at SF. Without a line like that, none of
the syntax highlighting showed up.

Looks like ctext is the way to go. Now that I understand it better, I
see it's a *really* useful widget--perfect for what I need here. And
thanks to everyone who offered advice on the lower-level bits in the
text widget.

--
Kevin Walzer
Poetic Code
http://www.kevin-walzer.com

Cameron Laird

2006-09-25, 10:02 pm

In article <45181A04.7020401@kevin-walzer.com>,
Kevin Walzer <kw@kevin-walzer.com> wrote:

Cameron Laird

2006-09-26, 7:02 pm

In article <Pine.LNX.4.64.0609261140490.3689@lastu30.oulu.fi>,
Kaitzschu <kaitzschu@kaitzschu.cjb.net.nospam.plz.invalid> wrote:

Sponsored Links