Subject:
|
Re: IC Floating Point Bug
|
Newsgroups:
|
lugnet.robotics.handyboard
|
Date:
|
Tue, 28 Jul 1998 18:35:43 GMT
|
Original-From:
|
Rick Moll <rickmoll@mvp.#spamless#net>
|
Viewed:
|
1084 times
|
| |
|
|
Will Bain wrote:
> I get the same results with version 3.2. Curiously, if I remove the
> last digit in the a = 1.570796 statement (or replace it with a zero),
> then I get the correct answer. I guess it has something to do with
> specifying too many significant digits in a floating point literal.
> Very strange.
If you notice, the constant is PI/2. If you add more significant digits
you can even get "a/b" to become 0.5, that is twice the correct answer
rather than half of it!!!
The reason the bug concerns me so much is that since "a" and "b" print out
correctly, it does *not* appear to be a problem with compiling the
literal. That is, it appears that if that value happened to occur as a
result of some prior calculation, you would get the same wrong result.
Rick
|
|
Message is in Reply To:
 | | Re: IC Floating Point Bug
|
| (...) <snip> I get the same results with version 3.2. Curiously, if I remove the last digit in the a = 1.570796 statement (or replace it with a zero), then I get the correct answer. I guess it has something to do with specifying too many significant (...) (26 years ago, 28-Jul-98, to lugnet.robotics.handyboard)
|
5 Messages in This Thread:
  
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
