For Programmers: Free Programming Magazines  


Home > Archive > C# > May 2005 > Re: BUG!! with Single and Math.Round









You are viewing an archived Text-only version of the thread. To view this thread in it's original format and/or if you want to reply to this thread please [click here]

 

Author Re: BUG!! with Single and Math.Round
Bruce Wood

2005-05-26, 3:57 am

The suffix F on a literal makes it a float, not a double. Floats have
23 bits of precision, which works out to 8,388,608 in decimal terms.

So, your float constant, 120294.719F is probably being rounded from the
outset. However, remember that doubles and floats aren't stored in
decimal... they're stored in binary. So, the value stored internally is
just an approximation of the decimal value you've entered.

When you then round it to two decimal places, the rounding is, again,
approximate. When you write it out, you're getting more decimal places
than you wanted because the stored value isn't exact. See this article
by John Skeet:

http://www.yoda.arachsys.com/csharp/floatingpoint.html

If you really want to print out exactly two decimal places, you have
two choices:

1. Use the format specifier in the WriteLine to indicate two decimal
places.

2. Copy the value into a decimal type, round that to two decimals, then
print it out. decimal values are stored in base 10: in decimal, so
rounding to two decimal places really does round to two decimal places.
See Jon's other article:

http://www.yoda.arachsys.com/csharp/decimal.html

Sponsored Links







Also available: Server administration forum archive | Web Design forum archive | Software forum archive | Hardware reviews archive

Copyright 2008 codecomments.com