Subject:
|
Re: NQC wishlist
|
Newsgroups:
|
lugnet.robotics.rcx.nqc
|
Date:
|
Thu, 6 Jan 2000 06:21:40 GMT
|
Viewed:
|
1867 times
|
| |
|
|
In article <dbaum-0501001842040001@207-229-150-21.d.enteract.com>,
dbaum@spambgoneenteract.com (Dave Baum) wrote:
> If I were to implement it, I probably wouldn't add a second pass. The
> real obstacle at present is that the literal text for a token cannot
> always be recovered after it leaves the lexer. For example, '01' and '1'
> both leave the lexer as an Integer token of value 1. Glomming tokens
> together with ## requires that you recover the original text and merge
> them. Adding this support (either in a new preprocessor or within the
> existing one) would probably be about the same amount of work and take up
> the same amount of memory.
I took a good look at the preprocessor code, and it shouldn't be too hard
to implement ## aside from the problem of recovering original token text.
Adding support for remembering the original token text is easy to do if
you're willing to be a memory hog and kill performance, and tricky to do
well.
As a first test, I'd add support for ## with the caveat that it will only
work reliably on glomming together two identifiers into a new identifier.
In the process of adding this support I'll be able to uncover any nasty
issues other than the original token text thing. If this proves
successul, then I'd probably release it as a beta while looking at solving
the token text problem. This brings up an interesting question...
How is everybody going to be using the ## directive? If its only to glom
two symbols into a new one, then I don't even have to worry about the
other tokens. In the spirit of doing 20% of the work to cover 80% of the
use, I'd like to get a feel for how important each of these cases are:
1) symbol ## symbol -> new symbol (e.g. foo ## bar)
2) symbol ## number -> new symbol (e.g. foo ## 12)
3) number ## number -> new number (e.g. 12 ## 34)
4) operator ## operator -> new operator (e.g. < ## = or + ## +)
5) symbol ## symbol -> keyword (e.g. br ## eak)
I'm ignoring strings since they are only valid in #include directives anyway.
Am I missing anything? Offhand, I think any other combination of tokens
leads to a tokenization error.
If I can ignore cases 4 and 5 then re-tokenization isn't necessary. If I
can ignore 3, then it gets even easier. Ignoring 2 means I don't even
have to solve the token text problem.
I know the 'purists' out there will insist I do full re-tokenization, but
time is short, so I'd like to focus on the most common cases.
Dave Baum
--
reply to: dbaum at enteract dot com
|
|
Message has 3 Replies:
 | | Re: NQC wishlist
|
| Dave, So far, I've lived without ##, and I'm quite sure I can live without 4) and 5). And can't 3) be replaced in many cases by arithmetic? In my brief stint programming "lego assembler" for the Scout, (Assembler! I'm embarrassed to admit how far (...) (25 years ago, 6-Jan-00, to lugnet.robotics.rcx.nqc)
| | | Re: NQC wishlist
|
| (...) That would still be handy. Just the other night I was readying some NQC for distribution and I had: #define FOO_SENSOR SENSOR_2 and what I wanted to do was (beyond the user configurable part): #define DISPLAY_FOO DISPLAY_ ## FOO_SENSOR so I (...) (25 years ago, 6-Jan-00, to lugnet.robotics.rcx.nqc)
| | | Re: NQC wishlist
|
| (...) I think 1 and 2 are the ones that would be really needed. 2 would be useful in writing macros to emulate arrays or "small" variables. But maybe there will be support for that in the compiler... cheers /Vlad (25 years ago, 6-Jan-00, to lugnet.robotics.rcx.nqc)
|
Message is in Reply To:
| | Re: NQC wishlist
|
| (...) If I were to implement it, I probably wouldn't add a second pass. The real obstacle at present is that the literal text for a token cannot always be recovered after it leaves the lexer. For example, '01' and '1' both leave the lexer as an (...) (25 years ago, 6-Jan-00, to lugnet.robotics.rcx.nqc)
|
17 Messages in This Thread:
 
  
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
