| |
In lugnet.robotics.rcx.legos, Ralph Hempel writes:
> <snipped code>
>
> > I have tried replacing the '>> 7' with '/ 128', substituting a->magnitude
> > and a->direction with int am and ad (and equating), and removing the (long)
> > cast while changing rx to int. No matter what I do, 256 * 128 / 128 = -256
> > (???). Tell me there is a fix and what it is.
>
> Robert, this is a classic problem with ranges of singed numbers in C
>
> rx = (long)(a->magnitude * TA_costable[a->direction]) >> 7;
>
> I don't have an RCX in frot of me, but I think you need to cast a->magnitude
> to long before the multiplication, otherwise you're just casting the int result
> of the multiplication to long, which gets you no benefit.
>
> 256*128 is 32768, which is -1 in signed ints.
>
> Try the following:
>
> rx = ((long)a->magnitude * TA_costable[a->direction]) >> 7;
>
> This should help your situation...and you could try this out
> interactively if you programmed in pbForth :-)
>
> Regards, Ralph Hempel
Yes, this is what happens when one is used to just making everything long
and not considering overflows. Even worse was that I was doing the math on
the computer's calculator, but not looking at the intermediate result, just
typing in the numbers and symbols on the numeric keypad and looking at the
final result. Is it obvious that I've been away from *real* programming for
just a little too long (about a year), working with Java, which does just
about everything for you except program itself (even does that sometimes!)?
Maybe I need a vacation from my hobby ;(. And three less dogs - or should
that be velociraptors. They've been probing the fence (4"x4"x5' chicken wire
poled every 6' or less) for the past three days and finding ways to escape
despite fixing the lose fencing. Next is to secure the bottom of the entire
fence, install electric barriers, and put up machine gun nests! Ah, the
joys of working from home :).
Thanks! Ralph and Michael,
Robert Templeton
|
|
Message is in Reply To:
 | | RE: Incorrect result
|
| <snipped code> (...) Robert, this is a classic problem with ranges of singed numbers in C rx = (long)(a->magnitude * TA_costable[a->direction]) >> 7; I don't have an RCX in frot of me, but I think you need to cast a->magnitude to long before the (...) (23 years ago, 30-Jan-02, to lugnet.robotics.rcx.legos)
|
4 Messages in This Thread:
  
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
