Tcl Web Service with Authorization kills Tcl

I got it working at least using ActiveTcl8.4.17.0.283511 on my 64 bit
SuSE Linux 9.3 box by replacing the single line 731 in Utilities.tcl
proc ::WS::Utils::convertTypeToDict

if {$childName in $matchList} {

with

if {0 <= [lsearch -exact $matchList $childName]} {




Stefan Finzel schrieb:
> Just to create some more confusion:
>
> The call for offered services also uses basic authorization and never
> causes segmentation faults.
>
> I also tested tcl8.4.17 on a 64 bit SuSE Linux 9.3 and there are no
> segmentation faults.
>
> But both tcl8.4.17/18 on a 32 bit SuSE Linux 10.0 crash tclhttpd.
>
> My tclhttpd and applications have been running with authorization on
> several productive servers for more than 5 years without any problems on
> SunOS and Linux both with 32 and 64 bit so far.
>
> Maybe i am in trouble due to an operating system issue.
>

Stefan Finzel wrote:
> I got it working at least using ActiveTcl8.4.17.0.283511 on my 64 bit
> SuSE Linux 9.3 box by replacing the single line 731 in Utilities.tcl
> proc ::WS::Utils::convertTypeToDict
>
>   if {$childName in $matchList} {
>
> with
>
>   if {0 <= [lsearch -exact $matchList $childName]} {

Stefan,

Go immediately to tcl.sf.net and file a bug report with as much detail as
possible.

Also see if the following little program blows up on you:

set matchList [list a b c d]
set childNmae a
if {$childName in $matchList} {
puts {It is in!}
} else {
puts {It is not in!}
}

--
+--------------------------------+---------------------------------------+
| Gerald W. Lester                                                       |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+

* Stefan Finzel <Stefan.G.R.Finzel@T-Online.de>
| Looks like a NULL pointer in tdom.
--<snip-snip>--
| [b7f13410] mmap2(NULL, 1925189632, PROT_READ|PROT_WRITE,
| MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x44bcb000
| [00000000] --- SIGSEGV (Segmentation fault) @ 0 (0) ---

The first arg being NULL is ok in mmap().  The second argument
1925189632 however is probably nonsense, it gives the desired mapping
length and looks suspiciously like an uninit value to me.

Note that mmap() succeeds (returning a valid pointer), but if you
actually try to use that much memory (1.8GB), it might not be
possible...

Can you run it under valgrind?

R'

valgrind /opt/ActiveTcl-8.4.18/bin/tclsh8.4
/opt/tclhttpd3.5.1/bin/httpd.tcl -debug 1


==7267== Invalid write of size 4
==7267==    at 0x55DE43D: XML_SetParamEntityParsing (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7267==    by 0x580E971: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7267==  Address 0x474C9D4 is not stack'd, malloc'd or (recently) free'd
==7267==
==7267== Invalid write of size 1
==7267==    at 0x4024720: memmove (in
/usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==7267==    by 0x55DE76A: XML_GetBuffer (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7267==    by 0x55DE613: XML_Parse (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7267==    by 0x580EBD5: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7267==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==7267==
==7267== Process terminating with default action of signal 11 (SIGSEGV)
==7267==  Access not within mapped region at address 0x0
==7267==    at 0x4024720: memmove (in
/usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==7267==    by 0x55DE76A: XML_GetBuffer (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7267==    by 0x55DE613: XML_Parse (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7267==    by 0x580EBD5: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7267==
==7267== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 109 from 2)
==7267== malloc/free: in use at exit: 4,837,951 bytes in 51,907 blocks.
==7267== malloc/free: 353,882 allocs, 301,975 frees, 37,261,788 bytes
allocated.
==7267== For counts of detected errors, rerun with: -v
==7267== searching for pointers to 51,907 not-freed blocks.
==7267== checked 4,430,740 bytes.
==7267==
==7267== LEAK SUMMARY:
==7267==    definitely lost: 1,024 bytes in 1 blocks.
==7267==      possibly lost: 0 bytes in 0 blocks.
==7267==    still reachable: 4,836,927 bytes in 51,906 blocks.
==7267==         suppressed: 0 bytes in 0 blocks.
==7267== Rerun with --leak-check=full to see details of leaked memory.
Segmentation fault


Ralf Fassel schrieb:
> * Stefan Finzel <Stefan.G.R.Finzel@T-Online.de>
> | Looks like a NULL pointer in tdom.
> --<snip-snip>--
> | [b7f13410] mmap2(NULL, 1925189632, PROT_READ|PROT_WRITE,
> | MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x44bcb000
> | [00000000] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
>
> The first arg being NULL is ok in mmap().  The second argument
> 1925189632 however is probably nonsense, it gives the desired mapping
> length and looks suspiciously like an uninit value to me.
>
> Note that mmap() succeeds (returning a valid pointer), but if you
> actually try to use that much memory (1.8GB), it might not be
> possible...
>
> Can you run it under valgrind?
>
> R'

valgrind --leak-check=full /opt/ActiveTcl-8.4.18/bin/tclsh8.4
/opt/tclhttpd3.5.1/bin/httpd.tcl -debug 1

==7314== Invalid write of size 4
==7314==    at 0x55DE43D: XML_SetParamEntityParsing (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7314==    by 0x580E971: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7314==  Address 0x4411CFC is not stack'd, malloc'd or (recently) free'd
==7314==
==7314== Invalid write of size 1
==7314==    at 0x4024720: memmove (in
/usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==7314==    by 0x55DE76A: XML_GetBuffer (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7314==    by 0x55DE613: XML_Parse (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7314==    by 0x580EBD5: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7314==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==7314==
==7314== Process terminating with default action of signal 11 (SIGSEGV)
==7314==  Access not within mapped region at address 0x0
==7314==    at 0x4024720: memmove (in
/usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==7314==    by 0x55DE76A: XML_GetBuffer (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7314==    by 0x55DE613: XML_Parse (in
/opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
==7314==    by 0x580EBD5: domReadDocument (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7314==
==7314== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 109 from 2)
==7314== malloc/free: in use at exit: 4,837,951 bytes in 51,907 blocks.
==7314== malloc/free: 353,852 allocs, 301,945 frees, 37,260,974 bytes
allocated.
==7314== For counts of detected errors, rerun with: -v
==7314== searching for pointers to 51,907 not-freed blocks.
==7314== checked 4,430,740 bytes.
==7314==
==7314==
==7314== 1,024 bytes in 1 blocks are definitely lost in loss record 24 of 31
==7314==    at 0x40235B5: malloc (in
/usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==7314==    by 0x57FF853: parserCreate (in
/opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
==7314==
==7314== LEAK SUMMARY:
==7314==    definitely lost: 1,024 bytes in 1 blocks.
==7314==      possibly lost: 0 bytes in 0 blocks.
==7314==    still reachable: 4,836,927 bytes in 51,906 blocks.
==7314==         suppressed: 0 bytes in 0 blocks.
==7314== Reachable blocks (those to which a pointer was found) are not
shown.
==7314== To see them, rerun with: --leak-check=full --show-reachable=yes
Segmentation fault


Stefan Finzel schrieb:
> valgrind /opt/ActiveTcl-8.4.18/bin/tclsh8.4
> /opt/tclhttpd3.5.1/bin/httpd.tcl -debug 1
>
>
> ==7267== Invalid write of size 4
> ==7267==    at 0x55DE43D: XML_SetParamEntityParsing (in
> /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> ==7267==    by 0x580E971: domReadDocument (in
> /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
> ==7267==  Address 0x474C9D4 is not stack'd, malloc'd or (recently) free'd
> ==7267==
> ==7267== Invalid write of size 1
> ==7267==    at 0x4024720: memmove (in
> /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
> ==7267==    by 0x55DE76A: XML_GetBuffer (in
> /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> ==7267==    by 0x55DE613: XML_Parse (in
> /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> ==7267==    by 0x580EBD5: domReadDocument (in
> /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
> ==7267==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
> ==7267==
> ==7267== Process terminating with default action of signal 11 (SIGSEGV)
> ==7267==  Access not within mapped region at address 0x0
> ==7267==    at 0x4024720: memmove (in
> /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
> ==7267==    by 0x55DE76A: XML_GetBuffer (in
> /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> ==7267==    by 0x55DE613: XML_Parse (in
> /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> ==7267==    by 0x580EBD5: domReadDocument (in
> /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
> ==7267==
> ==7267== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 109 from 2)
> ==7267== malloc/free: in use at exit: 4,837,951 bytes in 51,907 blocks.
> ==7267== malloc/free: 353,882 allocs, 301,975 frees, 37,261,788 bytes
> allocated.
> ==7267== For counts of detected errors, rerun with: -v
> ==7267== searching for pointers to 51,907 not-freed blocks.
> ==7267== checked 4,430,740 bytes.
> ==7267==
> ==7267== LEAK SUMMARY:
> ==7267==    definitely lost: 1,024 bytes in 1 blocks.
> ==7267==      possibly lost: 0 bytes in 0 blocks.
> ==7267==    still reachable: 4,836,927 bytes in 51,906 blocks.
> ==7267==         suppressed: 0 bytes in 0 blocks.
> ==7267== Rerun with --leak-check=full to see details of leaked memory.
> Segmentation fault
>
>
> Ralf Fassel schrieb: 

* Stefan Finzel <Stefan.G.R.Finzel@T-Online.de>
| valgrind --leak-check=full /opt/ActiveTcl-8.4.18/bin/tclsh8.4
| /opt/tclhttpd3.5.1/bin/httpd.tcl -debug 1
--<snip-snip>--
| ==7314== Invalid write of size 1
| ==7314==    at 0x4024720: memmove (in
| /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
| ==7314==    by 0x55DE76A: XML_GetBuffer (in
| /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
| ==7314==    by 0x55DE613: XML_Parse (in
| /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
| ==7314==    by 0x580EBD5: domReadDocument (in
| /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
| ==7314==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
| ==7314==
| ==7314== Process terminating with default action of signal 11 (SIGSEGV)
| ==7314==  Access not within mapped region at address 0x0
| ==7314==    at 0x4024720: memmove (in
| /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
| ==7314==    by 0x55DE76A: XML_GetBuffer (in
| /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
| ==7314==    by 0x55DE613: XML_Parse (in
| /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
| ==7314==    by 0x580EBD5: domReadDocument (in
| /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)

Well, sounds like it is passing a 0-pointer as destination to
memmove()...

Next steps would be to get the code for libtdom and/or libTclexpat,
recompile with debug-info, repeat, find the cause for the 0-pointer,
and send in a patch ;-)

R'

> | ==7314== Process terminating with default action of signal 11 (SIGSEGV)
> | ==7314==  Access not within mapped region at address 0x0
> | ==7314==    at 0x4024720: memmove (in
> | /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
> | ==7314==    by 0x55DE76A: XML_GetBuffer (in
> | /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> | ==7314==    by 0x55DE613: XML_Parse (in
> | /opt/ActiveTcl-8.4.18/lib/Tclexpat2.6/libTclexpat2.6.so)
> | ==7314==    by 0x580EBD5: domReadDocument (in
> | /opt/ActiveTcl-8.4.18/lib/tdom0.8.2/libtdom0.8.2.so)
>
> Well, sounds like it is passing a 0-pointer as destination to
> memmove()...
>
> Next steps would be to get the code for libtdom and/or libTclexpat,
> recompile with debug-info, repeat, find the cause for the 0-pointer,
> and send in a patch ;-)

This sounds really weird..., tdom should not use libtclexpat, (unless
something changed in how AS builds their distro). tdom builds in expat
lib statically, and libtclexpat does too.., (tclexpat should be from
the tclxml package). There was some discussion about problems this
might cause between Jeff Hobbs and Rolf Ade IIRC, but not sure about
the details.

Try getting libtclexpat out of the picture...

Michael

On Mar 26, 6:42 am, "Gerald W. Lester" <Gerald.Les...@cox.net> wrote:
> tom.rmadilo wrote: 
>
> Tom,
>
> I plan to have it play standalone and with WUB when I get the time (I don'
t
> make use of that much of tclhttpd), but if someone wants to take a crack a
t
> it first, that would be great (drop me a short email about it since I have
> some ideas).
>
> I'd ***love*** if you would help to get it to working with AOLserver!

The first step in that direction is to remove any channel code,
otherwise you will probably have trouble reusing your code for
different I/O situations.

I was unable to get tWSDL/TWiST working with tclhttpd  because it does
something with the Tcl event loop, and the code gets stuck in fcopy.
The exact same code does work (doesn't get stuck) with Tcl sockets,
simple std* channels, Tcl Threads and with AOLserver. So the key is to
reformulate the basic application as a single threaded model, then
wrap it with whatever is necessary.

I also have a pure Tcl replacement for the AOLserver API (server
layer), at least enough to serve web services and tcl pages.

Anyway, I'll re-look at the WS::Server code once again.

tom.rmadilo wrote:
> On Mar 26, 6:42 am, "Gerald W. Lester" <Gerald.Les...@cox.net> wrote: 
>
> The first step in that direction is to remove any channel code,
> otherwise you will probably have trouble reusing your code for
> different I/O situations.
>
> I was unable to get tWSDL/TWiST working with tclhttpd  because it does
> something with the Tcl event loop, and the code gets stuck in fcopy.
> The exact same code does work (doesn't get stuck) with Tcl sockets,
> simple std* channels, Tcl Threads and with AOLserver. So the key is to
> reformulate the basic application as a single threaded model, then
> wrap it with whatever is necessary.
>
> I also have a pure Tcl replacement for the AOLserver API (server
> layer), at least enough to serve web services and tcl pages.
>
> Anyway, I'll re-look at the WS::Server code once again.

Tom,

I use ::Url_PrefixInstall to install the URLs that my service applies to.

I use ::Httpd_ReturnData to return the data.

I do not do anything with the event loop.

It should be straight forward to the package embeddable with its own little
"server".  I'm not experienced enough with Wub or AOLserver to replace those
two calls.

--
+--------------------------------+---------------------------------------+
| Gerald W. Lester                                                       |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+

On Mar 26, 1:51 pm, "Gerald W. Lester" <Gerald.Les...@cox.net> wrote:
> tom.rmadilo wrote: 
> 
> 
> 
> 
> 
> 
>
> Tom,
>
> I use ::Url_PrefixInstall to install the URLs that my service applies to.
>
> I use ::Httpd_ReturnData to return the data.
>
> I do not do anything with the event loop.
>
> It should be straight forward to the package embeddable with its own littl
e
> "server".  I'm not experienced enough with Wub or AOLserver to replace tho
se
> two calls.

Right, so for instance the below proc uses ::Httpd_ReturnData:

proc ::WS::Server::callOperation {service sock args} {...}


My thought is that the callOperation would change so that it doesn't
handle the sock, but probably the caller, or higher level wrapper, of
this proc would do that.