To LUGNET HomepageTo LUGNET News HomepageTo LUGNET Guide Homepage
 Help on Searching
 
Post new message to lugnet.generalOpen lugnet.general in your NNTP NewsreaderTo LUGNET News Traffic PageSign In (Members)
 General / 53957
53956  |  53958
Subject: 
Re: Brickshelf Backup Crawler for Mac
Newsgroups: 
lugnet.general, lugnet.cad.dev.mac
Date: 
Wed, 18 Jul 2007 19:08:45 GMT
Viewed: 
16884 times
  
&tIn lugnet.general, Travis Cobbs wrote:

   here is Peter’s sh script (all 844 bytes of it): • ; http://www.halibut.com/~tcobbs/ldraw/private/bsbackup.sh

Boy, that is a lot more compact than mine!

   Also, it seems to only back up images (not sure why that is).

It looks like it’s only considering links to /cgi-bin/gallery.cgi. However, links to non-image files from the Brickshelf folder pages point directly to /gallery/username/foo/bar.dat. (Ah, I see you have fixed this.)

   Make sure to put quotes around the URL on the command line, since brickshelf URLs usually include a question mark, which the shell will try to interpret.

That advice probably stands for my script’s -url option as well.

   As mentioned, you must have wget installed on your system (which is there by default on most Linux distros).

For anyone interested, I can confirm that wget 1.10.2 builds on my Mac with the following:

./configure
make
make install

As Travis noted, though, if you don’t have a /usr/local directory (as I do) you may want to configure a different installation location.

For what it’s worth, I just did an informal comparison between this script and mine. I timed the download of my account with each script. Keep in mind that network variations and in particular Brickshelf’s heavy traffic make this a relative comparison, not an absolute measure.

cd ~/bsbackup-test
time ./bsbackup.sh "http://www.brickshelf.com/cgi-bin/gallery.cgi?m=anoved"

This took 24:32 (using the initial version which did not retrieve my LDraw files).

time bscrawl.tcl -url  "http://www.brickshelf.com/cgi-bin/gallery.cgi?m=anoved" -dir ~/bscrawl-test  -pause 0.01

This took 31:20 to download all my files.

So, it looks like your script is a bit faster as well as a bit smaller.

Jim



Message has 1 Reply:
  Re: Brickshelf Backup Crawler for Mac
 
(...) Of course, I just tried bsbackup.sh again - the current version that gets all files - and it took 32:05. There's a little quicktime movie in there that probably accounts for most of the difference as the MPD files are quite small. So the (...) (17 years ago, 18-Jul-07, to lugnet.general, lugnet.cad.dev.mac, FTX)

Message is in Reply To:
  Re: Brickshelf Backup Crawler for Mac
 
(...) I didn't have any problem compiling and installing wget (although I did have to change the default install path by hand from /usr/local to /opt/local so that it would be in my pre-existing path). However, I suspect that most Mac users would be (...) (17 years ago, 18-Jul-07, to lugnet.general, lugnet.cad.dev.mac, FTX)

37 Messages in This Thread:




















Entire Thread on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact

This Message and its Replies on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact
    

Custom Search

©2005 LUGNET. All rights reserved. - hosted by steinbruch.info GbR