Numval

Hi,

I am having an accuracy problem with the numval function.  This is a
clip from my debugger:

10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 	008F9DF8
VALUE 	> 	+106896201177201599      	<
10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 	008A59C4
OCCURRENCE 	( 	   1 	)
VALUE 	> 	+106896201177201605
...
COMPUTE IGELEMVALNUM0 =
FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))

As you can see from the results of my compute statement, I am getting
a value that is off by 6. My thoughts are that it's due to the size of
the variable since my other tests have been successful with the numval
conversion.  Has there been any documentation that speaks to this
apparent bug with the numval function? Has anyone seen this before?
Any help would be greatly appreciated.

What compiler?  Have you contacted the vendor?

FYI - I remember that ONE vendor actually used "floating point" for their
original NumVal implementation.  (Unfortunately, the Intrinsic Function
amendment was NOT clear that NumVal should give an "exact" value).  I have N
OT
heard of this problem recently, but it could be your problem.

--
Bill Klein
wmklein <at> ix.netcom.com
"Ralph" <RPHILISTINE@UNITECHSYS.COM> wrote in message
news:47fa0497.0404220529.12a6de47@posting.google.com...
> Hi,
>
> I am having an accuracy problem with the numval function.  This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 008F9DF8
>       VALUE > +106896201177201599      <
> 10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 008A59C4
>  OCCURRENCE (    1 )
>       VALUE > +106896201177201605
> ...
>    COMPUTE IGELEMVALNUM0 =
>    FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion.  Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.

Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function.  This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 008F9DF8
>       VALUE > +106896201177201599      <
> 10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 008A59C4
>  OCCURRENCE (    1 )
>       VALUE > +106896201177201605
> ...
>    COMPUTE IGELEMVALNUM0 =
>    FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion.  Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.

"OCCURRENCE(  1)"  ???

Try this:
PERFORM VARYING I FROM 1 BY 1 UNTIL I > BIG-NUMBER
IF I NOT = FUNCTION NUMVAL(I)
DISPLAY 'EEK!"
END-IF
END-PERFORM

Or something similar.

"William M. Klein" <wmklein@nospam.netcom.com> wrote in message news:<HsUhc.7138$e4.1032@ne
wsread2.news.pas.earthlink.net>...
> What compiler?  Have you contacted the vendor?
>
> FYI - I remember that ONE vendor actually used "floating point" for their
> original NumVal implementation.  (Unfortunately, the Intrinsic Function
> amendment was NOT clear that NumVal should give an "exact" value).  I have
 NOT
> heard of this problem recently, but it could be your problem.
>
> --
> Bill Klein
>  wmklein <at> ix.netcom.com
> "Ralph" <RPHILISTINE@UNITECHSYS.COM> wrote in message
> news:47fa0497.0404220529.12a6de47@posting.google.com... 

Thanks for the reply.  I will ask my systems administrator about the
compiler version.  It seems to work as documented when the text string
is 17 bytes.  It's always that 18th byte that throws a wrench into
things.

Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function.  This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 	008F9DF8
>  	    VALUE 	> 	+106896201177201599      	<
> 10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 	008A59C4
>  OCCURRENCE 	( 	   1 	)
>  	    VALUE 	> 	+106896201177201605
> ...
>    COMPUTE IGELEMVALNUM0 =
>    FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion.  Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.

I ran your test using OS/390 V2R10 and Enterprise COBOL, using targets
which were Display, Binary, and Packed-Decimal, all PIC S9(18).  Here
are the results:
INPUT-FIELD    = 106896201177201605
AFTER COMPUTE:
NUMVAL-DISPLAY = 10689620117720159I.
NUMVAL-BINARY  = 0106896201177201600.
NUMVAL-PACKED  = 106896201177201599.

Looks broken to me!

With 17 input digits, and 18 digit targets, I got correct answers:
INPUT-FLD2     = 10689620117720160
AFTER COMPUTE:
NUMVAL-DISPLAY2= 01068962011772016:.
NUMVAL-BINARY2 = 0010689620117720160.
NUMVAL-PACKED2 = 010689620117720160.

"Hugh Candlin" <no@spam.com> wrote in message
news:UfYhc.11635$_o3.378612@bgtnsc05-news.ops.worldnet.att.net...
<snip>
> What do you get if you remove the COMP-3 ?
>
> What do you get if you change to PIC S9(19) COMP-3
> to eliminate the slack byte ?
>

FYI - for those of you not on a compiler that allows for greater than 18 dig
its,
there are some that DO allow more (as an extension to the '85 Standard).  Th
e
2002 Standard allows for up to 31 digits.



--
Bill Klein
wmklein <at> ix.netcom.com

Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function.  This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 	008F9DF8
>  	    VALUE 	> 	+106896201177201599      	<
> 10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 	008A59C4
>  OCCURRENCE 	( 	   1 	)
>  	    VALUE 	> 	+106896201177201605
> ...
>    COMPUTE IGELEMVALNUM0 =
>    FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion.  Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.

Could the sign along with the 18 digits be causing this?  I'm not quite
sure how that's addressed in the COBOL 85 standard.  If you don't need a
signed variable, try dropping that and see if you get accurate results.
If you do, and that's acceptable, you've fixed it.  If you don't, or
you do but that's not acceptable, I'd contact the vendor to get their
take on it.  Maybe you've uncovered a seam - I did that just today on my
compiler.  :)


--
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
~   /   \  /         ~        Live from Montgomery, AL!       ~
~  /     \/       o  ~                                        ~
~ /      /\   -   |  ~          LXi0007@Netscape.net          ~
~ _____ /  \      |  ~ http://www.knology.net/~mopsmom/daniel ~
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~         I do not read e-mail at the above address           ~
~    Please see website if you wish to contact me privately   ~
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~

William M. Klein wrote:

>Colin,
>   Can we say "PMR"????  (Let me know if you have any problems getting IBM 
to
>accept this as "APARABLE".)
>
>P.S.  As long as you are using Enterprise COBOL - did you try this with bot
h
>ARITH(EXTEND) and "COMPAT"?  It should NOT make any difference, but it migh
t
>anyway <G>
>
>
>
Bill,
I'll give that a shot tomorrow, along with any other arithmetic related
option changes I can see that might affect things.
Colin

LX-i <lxi0007@netscape.net> wrote in message news:<108grotec067231@corp.supernews.com>...[c
olor=darkred]
> Ralph wrote: 
>
> Could the sign along with the 18 digits be causing this?  I'm not quite
> sure how that's addressed in the COBOL 85 standard.  If you don't need a
> signed variable, try dropping that and see if you get accurate results.
>   If you do, and that's acceptable, you've fixed it.  If you don't, or
> you do but that's not acceptable, I'd contact the vendor to get their
> take on it.  Maybe you've uncovered a seam - I did that just today on my
> compiler.  :)
>
>
> --
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
> ~   /   \  /         ~        Live from Montgomery, AL!       ~
> ~  /     \/       o  ~                                        ~
> ~ /      /\   -   |  ~          LXi0007@Netscape.net          ~
> ~ _____ /  \      |  ~ http://www.knology.net/~mopsmom/daniel ~
> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
> ~         I do not read e-mail at the above address           ~
> ~    Please see website if you wish to contact me privately   ~
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~[/color]

I have tried it without the sign and without the comp-3.  I am still
waiting for a reply from my systems programmer as to the nature of the
compiler.  But changing compilers during the QA process is not exactly
something I can do.  Right now I am coding a work around for instances
where the field is greater than 17 bytes.  Thanks to all for your
replies they were very helpful.  At least I know I'm not going crazy.

Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function.  This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0                   PIC S9(18) COMP-3   ADDR 008F9DF8
>       VALUE > +106896201177201599      <
> 10 IIP-IGELEMVALTEXT-TEXT          PIC X(256)          ADDR 008A59C4
>  OCCURRENCE (    1 )
>       VALUE > +106896201177201605
> ...
>    COMPUTE IGELEMVALNUM0 =
>    FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion.  Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.

Hmm.

For what it's worth, the Fujitsu compiler for the PC returns the correct
value.