Subject:
|
RE: Incorrect result
|
Newsgroups:
|
lugnet.robotics.rcx.legos
|
Date:
|
Wed, 30 Jan 2002 19:30:03 GMT
|
Reply-To:
|
<rhempel@bmts#ihatespam#.com>
|
Viewed:
|
1880 times
|
| |
|
|
<snipped code>
> I have tried replacing the '>> 7' with '/ 128', substituting a->magnitude
> and a->direction with int am and ad (and equating), and removing the (long)
> cast while changing rx to int. No matter what I do, 256 * 128 / 128 = -256
> (???). Tell me there is a fix and what it is.
Robert, this is a classic problem with ranges of singed numbers in C
rx = (long)(a->magnitude * TA_costable[a->direction]) >> 7;
I don't have an RCX in frot of me, but I think you need to cast a->magnitude
to long before the multiplication, otherwise you're just casting the int result
of the multiplication to long, which gets you no benefit.
256*128 is 32768, which is -1 in signed ints.
Try the following:
rx = ((long)a->magnitude * TA_costable[a->direction]) >> 7;
This should help your situation...and you could try this out
interactively if you programmed in pbForth :-)
Regards, Ralph Hempel
--------------------------------------------------------------------
Check out pbFORTH for LEGO Mindstorms at:
<http://www.hempeldesigngroup.com/lego/pbForth>
Buy "Extreme Mindstorms: an Advanced Guide to Lego Mindstorms"
<http://www.amazon.com/exec/obidos/ASIN/1893115844/hempeldesigngrou>
--------------------------------------------------------------------
Reply to: rhempel at bmts dot com
--------------------------------------------------------------------
|
|
Message has 1 Reply:
 | | Re: Incorrect result
|
| (...) Yes, this is what happens when one is used to just making everything long and not considering overflows. Even worse was that I was doing the math on the computer's calculator, but not looking at the intermediate result, just typing in the (...) (22 years ago, 30-Jan-02, to lugnet.robotics.rcx.legos)
|
Message is in Reply To:
| | Incorrect result
|
| This problem, unlike the other, is definitely not a result of my eyes/brain being tired or something. I thoroughly displayed every value before and after evaluation and the result is ***WRONG***. Here is the code and the values: typedef struct { int (...) (22 years ago, 30-Jan-02, to lugnet.robotics.rcx.legos)
|
4 Messages in This Thread:
  
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
