To LUGNET HomepageTo LUGNET News HomepageTo LUGNET Guide Homepage
 Help on Searching
 
Post new message to lugnet.announceOpen lugnet.announce in your NNTP NewsreaderTo LUGNET News Traffic PageSign In (Members)
 Announcements / 3573
3572  |  3574
Subject: 
Brickshelf Backup Crawler for Mac
Newsgroups: 
lugnet.general, lugnet.announce, lugnet.cad.dev.mac
Followup-To: 
lugnet.general
Date: 
Tue, 17 Jul 2007 05:28:10 GMT
Highlighted: 
(details)
Viewed: 
10237 times
  
As noted here, there is a BSBackup utility for Windows that helps you batch download the contents of Brickshelf galleries. (This is of interest at the moment due to Brickshelf’s imminent closure.) I’ve written a quick hack to accomplish the same thing on Mac OS X.

Download bscrawl.tcl (7k)

This is a command line script. (Hypothetically, it may also work with Linux or Windows if you have Tcl and tcllib.) I recognize this is not the ideal Macintosh interface, so I’ll write an AppleScript wrapper sometime soon that you can just click when you’re viewing a gallery you’d like to back up.

Assuming you’ve downloaded the script to your desktop, open the Terminal application and enter the following commands to move to your desktop folder and make the script executable:

cd Desktop
chmod +x bscrawl.tcl

Now you can retrieve your Brickshelf gallery with the following command:

./bscrawl.tcl -m username -dir MyBrickshelf

where username is your Brickshelf username and MyBrickshelf is a folder you have created on the desktop to contain your Brickshelf files (you can, of course, provide a different directory). The script works recursively, so that your Brickshelf folder hierarchy is reconstructed in the local directory.

If you only want to back up the contents of a particular folder (and its subfolders), use a command like this:

./bscrawl.tcl -f 233717 -dir ~/Documents/MyFavoriteFolder

Where 233717 is the folder id you see in the Brickshelf URL for that gallery.

You can also download a gallery (either a folder or a whole account) by specifying its URL:

./bscrawl.tcl -url http://www.brickshelf.com/cgi-bin/gallery.cgi?f=233717  -dir ~/Documents/MyFavoriteFolder

Use the -pause option to specify how many seconds to idle between downloads:

./bscrawl.tcl -f 233717 -dir ~/Documents/MyFavoriteFolder -pause 5

That summarizes the usage. A few notes:
  • Both image and data files are downloaded (.ldr, .txt, .mov, etc.)
  • Folder descriptions and keywords are not retrieved
  • Existing local files are skipped, so you can efficiently “update” a local backup by reissuing the initial command (it won’t notice changes to the same filename, though). Kind of useless since nothing new is going on Brickshelf now, although this could certainly be adapted for maj.com
  • Multi-page folders are handled
  • Basic progress messages are printed to the terminal
  • Very dependent on Brickshelf’s current HTML format
  • It was written in an evening, so there’s been negligible testing. I’m sure there are bugs. You accept that at your own risk. Some of the code isn’t very well thought out, either, but it works for me.
I hope some of you find this useful. Let me know how it works for you. As I mentioned, I’ll try to make it more user friendly in the next few days.

Jim



Message has 4 Replies:
  Re: Brickshelf Backup Crawler for Mac
 
(...) Here's a very rudimentary application that does the same thing for those who aren't comfortable using the Terminal: (URL) Download BrickshelfBackup.app.zip> (26k; probably requires Mac OS X 10.4, as does the bscrawl.tcl script) Double-click (...) (17 years ago, 17-Jul-07, to lugnet.general, FTX)
  Re: Brickshelf Backup Crawler for Mac
 
Thanks for that Jim. I got the script working for user backup, and Peter Bartfai (the author of the original script) also got one working for user backup (before seeing my email that I'd gotten it working). However, while I haven't looked at Peter's (...) (17 years ago, 17-Jul-07, to lugnet.general, lugnet.cad.dev.mac, FTX)
  Re: Brickshelf Backup Crawler for Mac
 
(...) Thank you Jim! This was great and saved me a lot of time :) -Anne (17 years ago, 17-Jul-07, to lugnet.general, FTX)
  Re: Brickshelf Backup Crawler for Mac
 
(...) Yesterday I realized that while my script handled multi-page folder galleries (URLs like f=1234), it did not handle multi-page user galleries (URLs like m=username). This has been fixed and "version 1.1" is now available at the same address (...) (17 years ago, 21-Jul-07, to lugnet.general, FTX)

Message is in Reply To:
  Re: Brickshelf back up (at least temporarily)
 
(...) No promises, but keep an eye on lugnet.cad.dev.mac tonight or tomorrow. I started something like this (but quicker and dirtier) Sunday morning before the https link went down. Jim (17 years ago, 16-Jul-07, to lugnet.general, FTX)

37 Messages in This Thread:




















Entire Thread on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact

This Message and its Replies on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact
    

Custom Search

©2005 LUGNET. All rights reserved. - hosted by steinbruch.info GbR