Home > Archive > Cobol > April 2004 > Numval
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| Ralph 2004-04-22, 10:30 am |
| Hi,
I am having an accuracy problem with the numval function. This is a
clip from my debugger:
10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
VALUE > +106896201177201599 <
10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
OCCURRENCE ( 1 )
VALUE > +106896201177201605
....
COMPUTE IGELEMVALNUM0 =
FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
As you can see from the results of my compute statement, I am getting
a value that is off by 6. My thoughts are that it's due to the size of
the variable since my other tests have been successful with the numval
conversion. Has there been any documentation that speaks to this
apparent bug with the numval function? Has anyone seen this before?
Any help would be greatly appreciated.
| |
| William M. Klein 2004-04-22, 3:30 pm |
| What compiler? Have you contacted the vendor?
FYI - I remember that ONE vendor actually used "floating point" for their
original NumVal implementation. (Unfortunately, the Intrinsic Function
amendment was NOT clear that NumVal should give an "exact" value). I have NOT
heard of this problem recently, but it could be your problem.
--
Bill Klein
wmklein <at> ix.netcom.com
"Ralph" <RPHILISTINE@UNITECHSYS.COM> wrote in message
news:47fa0497.0404220529.12a6de47@posting.google.com...
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
| |
| JerryMouse 2004-04-22, 6:30 pm |
| Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
"OCCURRENCE( 1)" ???
Try this:
PERFORM VARYING I FROM 1 BY 1 UNTIL I > BIG-NUMBER
IF I NOT = FUNCTION NUMVAL(I)
DISPLAY 'EEK!"
END-IF
END-PERFORM
Or something similar.
| |
|
| "William M. Klein" <wmklein@nospam.netcom.com> wrote in message news:<HsUhc.7138$e4.1032@newsread2.news.pas.earthlink.net>...[color=darkred]
> What compiler? Have you contacted the vendor?
>
> FYI - I remember that ONE vendor actually used "floating point" for their
> original NumVal implementation. (Unfortunately, the Intrinsic Function
> amendment was NOT clear that NumVal should give an "exact" value). I have NOT
> heard of this problem recently, but it could be your problem.
>
> --
> Bill Klein
> wmklein <at> ix.netcom.com
> "Ralph" <RPHILISTINE@UNITECHSYS.COM> wrote in message
> news:47fa0497.0404220529.12a6de47@posting.google.com...
Thanks for the reply. I will ask my systems administrator about the
compiler version. It seems to work as documented when the text string
is 17 bytes. It's always that 18th byte that throws a wrench into
things.
| |
| Colin Campbell 2004-04-22, 7:31 pm |
| Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
I ran your test using OS/390 V2R10 and Enterprise COBOL, using targets
which were Display, Binary, and Packed-Decimal, all PIC S9(18). Here
are the results:
INPUT-FIELD = 106896201177201605
AFTER COMPUTE:
NUMVAL-DISPLAY = 10689620117720159I.
NUMVAL-BINARY = 0106896201177201600.
NUMVAL-PACKED = 106896201177201599.
Looks broken to me!
With 17 input digits, and 18 digit targets, I got correct answers:
INPUT-FLD2 = 10689620117720160
AFTER COMPUTE:
NUMVAL-DISPLAY2= 01068962011772016:.
NUMVAL-BINARY2 = 0010689620117720160.
NUMVAL-PACKED2 = 010689620117720160.
| |
| William M. Klein 2004-04-22, 8:30 pm |
|
"Hugh Candlin" <no@spam.com> wrote in message
news:UfYhc.11635$_o3.378612@bgtnsc05-news.ops.worldnet.att.net...
<snip>
> What do you get if you remove the COMP-3 ?
>
> What do you get if you change to PIC S9(19) COMP-3
> to eliminate the slack byte ?
>
FYI - for those of you not on a compiler that allows for greater than 18 digits,
there are some that DO allow more (as an extension to the '85 Standard). The
2002 Standard allows for up to 31 digits.
--
Bill Klein
wmklein <at> ix.netcom.com
| |
|
| Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
Could the sign along with the 18 digits be causing this? I'm not quite
sure how that's addressed in the COBOL 85 standard. If you don't need a
signed variable, try dropping that and see if you get accurate results.
If you do, and that's acceptable, you've fixed it. If you don't, or
you do but that's not acceptable, I'd contact the vendor to get their
take on it. Maybe you've uncovered a seam - I did that just today on my
compiler. :)
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
~ / \ / ~ Live from Montgomery, AL! ~
~ / \/ o ~ ~
~ / /\ - | ~ LXi0007@Netscape.net ~
~ _____ / \ | ~ http://www.knology.net/~mopsmom/daniel ~
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~ I do not read e-mail at the above address ~
~ Please see website if you wish to contact me privately ~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
| |
| Colin Campbell 2004-04-23, 12:30 am |
| William M. Klein wrote:
>Colin,
> Can we say "PMR"???? (Let me know if you have any problems getting IBM to
>accept this as "APARABLE".)
>
>P.S. As long as you are using Enterprise COBOL - did you try this with both
>ARITH(EXTEND) and "COMPAT"? It should NOT make any difference, but it might
>anyway <G>
>
>
>
Bill,
I'll give that a shot tomorrow, along with any other arithmetic related
option changes I can see that might affect things.
Colin
| |
| Ralph 2004-04-23, 10:30 am |
| LX-i <lxi0007@netscape.net> wrote in message news:<108grotec067231@corp.supernews.com>...
> Ralph wrote:
>
> Could the sign along with the 18 digits be causing this? I'm not quite
> sure how that's addressed in the COBOL 85 standard. If you don't need a
> signed variable, try dropping that and see if you get accurate results.
> If you do, and that's acceptable, you've fixed it. If you don't, or
> you do but that's not acceptable, I'd contact the vendor to get their
> take on it. Maybe you've uncovered a seam - I did that just today on my
> compiler. :)
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
> ~ / \ / ~ Live from Montgomery, AL! ~
> ~ / \/ o ~ ~
> ~ / /\ - | ~ LXi0007@Netscape.net ~
> ~ _____ / \ | ~ http://www.knology.net/~mopsmom/daniel ~
> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
> ~ I do not read e-mail at the above address ~
> ~ Please see website if you wish to contact me privately ~
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~
I have tried it without the sign and without the comp-3. I am still
waiting for a reply from my systems programmer as to the nature of the
compiler. But changing compilers during the QA process is not exactly
something I can do. Right now I am coding a work around for instances
where the field is greater than 17 bytes. Thanks to all for your
replies they were very helpful. At least I know I'm not going crazy.
| |
| JerryMouse 2004-04-23, 10:30 am |
| Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
Hmm.
For what it's worth, the Fujitsu compiler for the PC returns the correct
value.
| |
| docdwarf@panix.com 2004-04-23, 10:30 am |
| In article <47fa0497.0404230536.5b6e4168@posting.google.com>,
Ralph <RPHILISTINE@UNITECHSYS.COM> wrote:
[snip]
>Thanks to all for your
>replies they were very helpful. At least I know I'm not going crazy.
Well... at least you know that in this instance there are others who
report seeing the same, or similar things.
DD
| |
|
| Ralph wrote:
> LX-i <lxi0007@netscape.net> wrote in message news:<108grotec067231@corp.supernews.com>...
(snip)
> I have tried it without the sign and without the comp-3. I am still
> waiting for a reply from my systems programmer as to the nature of the
> compiler. But changing compilers during the QA process is not exactly
> something I can do. Right now I am coding a work around for instances
> where the field is greater than 17 bytes. Thanks to all for your
> replies they were very helpful. At least I know I'm not going crazy.
Independent variables. ;)
| |
| Colin Campbell 2004-04-23, 3:30 pm |
| Ralph wrote:
> Hi,
>
> I am having an accuracy problem with the numval function. This is a
> clip from my debugger:
>
> 10 IGELEMVALNUM0 PIC S9(18) COMP-3 ADDR 008F9DF8
> VALUE > +106896201177201599 <
> 10 IIP-IGELEMVALTEXT-TEXT PIC X(256) ADDR 008A59C4
> OCCURRENCE ( 1 )
> VALUE > +106896201177201605
> ...
> COMPUTE IGELEMVALNUM0 =
> FUNCTION NUMVAL (IIP-IGELEMVALTEXT-TEXT(IGELEM-IDX))
>
> As you can see from the results of my compute statement, I am getting
> a value that is off by 6. My thoughts are that it's due to the size of
> the variable since my other tests have been successful with the numval
> conversion. Has there been any documentation that speaks to this
> apparent bug with the numval function? Has anyone seen this before?
> Any help would be greatly appreciated.
Ralph,
I ran again on Friday morning. I'm using the IBM compiler "Enterprise
COBOL for z/OS and OS/390 V3R2" on an OS/390 V2R10 system. For this
compile, I used the option ARITH(EXTEND), and this time, I got the
correct results for the 18-digit case:
INPUT-FIELD = 106896201177201605
AFTER COMPUTE:
NUMVAL-DISPLAY = 10689620117720160E.
NUMVAL-BINARY = 0106896201177201605.
NUMVAL-PACKED = 106896201177201605.
INPUT-FLD2 = 10689620117720160
AFTER COMPUTE:
NUMVAL-DISPLAY2= 01068962011772016:.
NUMVAL-BINARY2 = 0010689620117720160.
NUMVAL-PACKED2 = 010689620117720160.
By the way, if you are running on an IBM mainframe, you can answer the
question of "what compiler am I using" simply by looking at the first
line of your listing. The product name and product number appear there,
as well as sometimes an indicator of the maintenance level. You don't
need to ask a systems programmer for this information.
Example:
1PP 5655-G53 IBM Enterprise COBOL for z/OS and OS/390 3.2.0
Colin
| |
| William M. Klein 2004-04-25, 12:30 am |
| Colin,
Can we say "PMR"???? (Let me know if you have any problems getting IBM to
accept this as "APARABLE".)
P.S. As long as you are using Enterprise COBOL - did you try this with both
ARITH(EXTEND) and "COMPAT"? It should NOT make any difference, but it might
anyway <G>
--
Bill Klein
wmklein <at> ix.netcom.com
"Colin Campbell" <cmcampb@adelphia.net_remove_this> wrote in message
news:cDXhc.19$DB.4@dfw-service2.ext.ray.com...
> Ralph wrote:
>
> I ran your test using OS/390 V2R10 and Enterprise COBOL, using targets
> which were Display, Binary, and Packed-Decimal, all PIC S9(18). Here
> are the results:
> INPUT-FIELD = 106896201177201605
> AFTER COMPUTE:
> NUMVAL-DISPLAY = 10689620117720159I.
> NUMVAL-BINARY = 0106896201177201600.
> NUMVAL-PACKED = 106896201177201599.
>
> Looks broken to me!
>
> With 17 input digits, and 18 digit targets, I got correct answers:
> INPUT-FLD2 = 10689620117720160
> AFTER COMPUTE:
> NUMVAL-DISPLAY2= 01068962011772016:.
> NUMVAL-BINARY2 = 0010689620117720160.
> NUMVAL-PACKED2 = 010689620117720160.
>
|
|
|
|
|