Subject:
|
Re: pictures at tracker
|
Newsgroups:
|
lugnet.cad
|
Date:
|
Thu, 16 May 2002 18:46:05 GMT
|
Viewed:
|
829 times
|
| |
|
|
Steve Bliss wrote:
>
> That sounds great! That's if it's ok with Jacob.
>
>
> > Here are some general thoughts....
> >
> > 1. You definitely need a dependency generator. This is pretty
> > easy to do with sed, awk, perl, tcl - whatever. This program
> > would spit out a list of heirarchical dependencies for each
> > thing that it processes.
>
>
> We've basically got one, although it might not be very efficient. Right
> now, it spits out a file of lines like:
>
> <datfile> <subfile> <library>
So something could be written that would convert this to Make format
or it could read the files themselves.
> > 2. The VPATH and inference rule features of GNU make are powerful
> > and easy to use constructs. I use them to handle building
> > complete libraries within subdirectories of the working
> > code. It should be easy to adapt to the parts tracker.
>
>
> Hmm. There are four directories in an ldraw library: parts, parts/s, p,
> and p/48. When a dat file references a subfile in the parts/s or p/48
> directories, it includes a leading s\ or 48\ on the subfile name. So
> 973.dat has a line like:
>
> 1 16 0 0 0 1 0 0 0 1 0 0 0 1 s\973s01.dat
>
> If this was written in a dependency as (as Kyle suggested for the file
> extensions)
> 973.png: 973.dat s\973s01.pic
> could VPATH be used to track down 973s01.dat in
> {symlib}/parts/s/973s01.dat? Otherwise, we can just create the
> dependency files with full paths.
Yes, because the s/ is in the datfile, and if it was kept in the
dependency file, Make would look for s/973s01.dat in all the directories
in the vpath, or VPATH. That will work.
On top of that make has variables like $@ and $> (I think - it's been
a while) that will expand to the location make found the file in.
>
> > Let me know exactly what you need and the tools (programs) you have
> > access to on the target box and I'll bang together a make script
> > for you.
>
>
> I *think* there would be only one production: producing a PNG file out
> of a dat, using ldglite. The exact commandline is depending on any
> changes Don makes to ldglite to support this effort.
>
> > If it's not clear, i LOVE make. I use it for all kinds of tasks,
> > including building pbForth from scratch and then building zip
> > archives and sending them to the server.
I love it also. I haven't needed it as much with Java, and it's
been way too long since I've used it with C :(. I'm available to
help, but it sounds like Ralph has it covered.
> Farther up the thread, but later in time, Ralph Hempel wrote:
>
> > So, I need a couple of things to get going on this:
> >
> > 1. A brief description of the box this will run on, including common
> > programs we have access to(perl, awk, sed) and a list of programs
> > we don't have access to.
>
>
> Uh, it's about 55cm x 25cm x ...
>
> Seriously, it's a linux box with most standard programs. I can't really
> tell you what we don't have access to -- I have no idea what you'd
> expect to have access to. Heck, I'm still coming to greps with what we
> do have. Jacob would be the best source for info about the server.
I'd be surprised if there was something a Linux box was missing ;)
>
> > 2. A brief description of the current part directory scheme and how
> > unofficial parts and official parts must coexist peacefully. If possible,
>
>
> For our purposes, unofficial parts are newer revisions of official
> parts. Basically, there are two trees, as I outlined in an earlier
> message. But, if we build a directory of symlinks, then we just have
> the standard LDraw library structure.
The symlinks should work.
I wonder if there is any need to keep both official and unofficial
picture libraries though?
.
.
.
>
> > I'm assuming that if I get the complete parts catalog and the updates and
> > expand them in an "official" tree, then get all of the unofficial parts
> > and expeand them in an "unofficail" tree, I could build a "release" tree
> > using nothing but symlinks.
>
>
> Yes, that's correct. Although I'm going to guess that we won't be
> totally rebuilding the symlibrary every 2 minutes (which is how often
> new submissions are processed into the tracker).
Well, if someone updates a primitive, like say 'stud.dat' this
*will* cause Make to consider pretty much every .png file to be
'out of date' which will cause it to remake the .png for every
.dat that depends on it. Actually, isn't this is the desired
behavior?
The trick is to make sure that this doesn't take CPU cycles away
from the WebServer processes. Something could be done with some
sort of locks so ensure that 2 Makes don't get run at once. Also
it may be possible to use 'nice' or something to have make (and
the commands it runs) run at a lower priority level.
Another alternative would be to run the make on a separate machine.
Or to not run it when the files are submitted, but instead at
certain intervals during the day.
I would think that a part submission should only cause make to
remake 1 .png. A subpart submission probably would trigger 5-30
.png's to be remade? primitves could cause many many .png's to
be remade.
Maybe changes from parts and sub-parts could be done at
submission, but changes from primitives are done every 4-6 hours?
Or a slightly different method would be, if 3001.dat is submitted,
then only run 'make 3001.png' instead of 'make.' Likewise if
stud.dat were submitted, you could only run 'make stud.png' right
away. Then at some point (maybe midnight) cron could run 'make'
which would update all the other files that depended on the files
that had been submitted that day.
-Kyle
--
_
-------------------------------ooO( )Ooo-------------------------------
Kyle J. McDonald (o o)
|||||
\\\//
(o o) kmcdonald@BigFoot.COM
-------------------------------ooO(_)Ooo-------------------------------
|
|
Message has 3 Replies:
 | | RE: pictures at tracker
|
| (...) Thanks. I knew that keeping make in the toolbox would come in handy! (...) Me too, but sometimes you need to make sure! (...) Yes. (...) Exactly. Inference rules can take care of this. Another option would be to run one of: make all - builds (...) (23 years ago, 16-May-02, to lugnet.cad)
| | | Re: pictures at tracker
|
| (...) Or the something that creates the file could write the Make format itself. (...) OK. The current dependency file already includes the location (up to p/ or parts/), but if that work could be pushed off on make, it might be worth it. (...) I'd (...) (23 years ago, 16-May-02, to lugnet.cad)
| | | Re: pictures at tracker
|
| (...) Nothing is ever missing from my Linux systems, but I may (more or less consiously :-) have decided that it should not be there. Jacob (23 years ago, 17-May-02, to lugnet.cad)
|
Message is in Reply To:
| | Re: pictures at tracker [DAT]
|
| Replying to a couple of things at once (and resisting the urge to XFUT lugnet.cad.dev.org.ldraw, which should have happened 30 messages ago...). (...) That sounds great! That's if it's ok with Jacob. (...) We've basically got one, although it might (...) (23 years ago, 16-May-02, to lugnet.cad)
|
77 Messages in This Thread:
 
 
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
