 | | | | | | |
| |
|
In lugnet.lego.direct, David Eaton wrote:
> In lugnet.lego.direct, Ross Crawford wrote:
> > I poll from Australia, roughly once a month.
>
> Hm. given what you're talking about, it doesn't seem terribly likely that they'd
> shut down your IP based on that alone-- unless your polls are REALLY hitting
> them hard. But I'm assuming you're doing something like:
>
> for each set in the Lugnet Database:
> - Skip the set if its release date is prior to (say) 1995.
> - Skip the set if you already have a record for it
> - Do a search on Lego for the set number. If a result comes back, save it, but
> don't actually download the PDF.
>
> But maybe you're actually testing each PDF to verify that the links are still
> valid?
It's more complicated than that, because the PDFs are stored by the 7 digit LEGO
number, not the set number. Basically I scan the header of all files in a
certain range (about 1 million), and if it is a PDF I check against my database,
downloading it to my hard drive if I don't already have it. I then have to
manually go through all new PDFs, to find out which set they are for.
> > I do not store the PDFs on Northstar, and I don't want to. There would I'm
> > sure be legal issues doing that, and with 9.5GB so far, I'd certainly have to
> > pay more for my hosting at Northstar.
>
> Oh, I wasn't meaning that you'd STORE them at NorthStar, but that as a fix
> (albeit a very bad one), you could set up a link that used the NS server to pull
> FROM Lego (assuming that NS's IP's aren't blocked), then immediately deliver the
> file to yourself (assuming that NS isn't blocking your IP). And as soon as the
> NS server is done transferring the file, away it goes. But yes, that's possibly
> also disallowed, since technically your server would be serving Lego's material
> which they've asked people not to do. (But so long as the link isn't *public*,
> you're ok)
Unfortunately, Northstar runs Windows Server, and setting up any kind of
complicated server-side stuff is probably beyond what I get for my hosting. And
besides, I really don't want to jump through hoops to do this - it's not that
important to me. If my polling IS the reason, and they don't want to negotiate
about it, I'd just like to know, and I will stop it (which I have done anyway
until it's resolved one way or the other).
> > If the polling is the reason they have blocked my IP (and a large range of
> > other IPs it would seem), I would just like to know. I would be happy to
> > discuss with them the possibility of updating my database in a "sanctioned"
> > way. But blocking a large range of IPs because of it seems to be like driving
> > a nail with a 10 ton truck.
>
> Agreed. It does seem rather... wrong of them to do. Of course, if they're
> blocking a wide range of IP's, it could also be unrelated to your polling in
> particular, which may be the case, depending on the order of events. I guess
> there's not much way to know unless Lego wants to chime in...
Indeed. Interestingly, I just got a reply from LEGO, saying they could not
duplicate the problem, I think probably because the first line of support
doesn't understand the problem. The dialogue continues...
ROSCO
| | | | | | | | | | | | | | | |
In lugnet.lego.direct, Ross Crawford wrote:
> It's more complicated than that, because the PDFs are stored by the 7 digit LEGO
> number, not the set number. Basically I scan the header of all files in a
> certain range (about 1 million), and if it is a PDF I check against my database,
> downloading it to my hard drive if I don't already have it.
Ahh. Yeah, I could see that as being potentially red-flagging, if they're
getting a million hits in a short period of time. I was definitely thinking
smaller scale than that.
> Indeed. Interestingly, I just got a reply from LEGO, saying they could not
> duplicate the problem, I think probably because the first line of support
> doesn't understand the problem. The dialogue continues...
Heh, the inevitable 1st level of tech support :) It will be interesting to hear
their explanation, assuming you get one.
DaveE
| | | | | | | | | | | | | | | | | | | | | |
| |
|
In lugnet.lego.direct, David Eaton wrote:
> In lugnet.lego.direct, Ross Crawford wrote:
> > It's more complicated than that, because the PDFs are stored by the 7 digit LEGO
> > number, not the set number. Basically I scan the header of all files in a
> > certain range (about 1 million), and if it is a PDF I check against my database,
> > downloading it to my hard drive if I don't already have it.
>
> Ahh. Yeah, I could see that as being potentially red-flagging, if they're
> getting a million hits in a short period of time. I was definitely thinking
> smaller scale than that.
Well it takes about 7 days to check them all. And it's only grabbing the header
unless it's a new file.
> > Indeed. Interestingly, I just got a reply from LEGO, saying they could not
> > duplicate the problem, I think probably because the first line of support
> > doesn't understand the problem. The dialogue continues...
>
> Heh, the inevitable 1st level of tech support :) It will be interesting to hear
> their explanation, assuming you get one.
Yeah, we'll see.
ROSCO
| | | | | | | | | | | | | | | | | | |
In lugnet.lego.direct, Ross Crawford wrote:
> It's more complicated than that, because the PDFs are stored by the 7 digit LEGO
> number, not the set number. Basically I scan the header of all files in a
> certain range (about 1 million), and if it is a PDF I check against my database,
> downloading it to my hard drive if I don't already have it.
Ahh. Yeah, I could see that as being potentially red-flagging, if they're
getting a million hits in a short period of time. I was definitely thinking
smaller scale than that.
> Indeed. Interestingly, I just got a reply from LEGO, saying they could not
> duplicate the problem, I think probably because the first line of support
> doesn't understand the problem. The dialogue continues...
Heh, the inevitable 1st level of tech support :) It will be interesting to hear
their explanation, assuming you get one.
DaveE
| | | | | | | | | | | | | | | | |
What email address are you using to contact them? I'll moan too bcause it's
not working for me and I'm not doing anything that might offend them.
Kerry
| | | | | | | | | | | | | | | | |
In lugnet.lego.direct, Kerry Raymond wrote:
> What email address are you using to contact them? I'll moan too bcause it's
> not working for me and I'm not doing anything that might offend them.
Great idea Kerry! I initially contacted them via the contact form on the
website. They have opened a problem ticket, so I'm communicating via that system
now, but if any other aussies are experiencing this problem, it can't hurt to
let them know.
http://www.lego.com/eng/service/contactus.asp
ROSCO
| | | | | | | | | | | | | | | | |
"Ross Crawford" <ross.{mylastname}@gmail.com> wrote in message
news:J5pK1q.o44@lugnet.com...
> In lugnet.lego.direct, Kerry Raymond wrote:
> > What email address are you using to contact them? I'll moan too bcause
> > it's
> > not working for me and I'm not doing anything that might offend them.
>
> Great idea Kerry! I initially contacted them via the contact form on the
> website. They have opened a problem ticket, so I'm communicating via that
> system
> now, but if any other aussies are experiencing this problem, it can't hurt
> to
> let them know.
>
> http://www.lego.com/eng/service/contactus.asp
>
> ROSCO
OK, I went, I complained.
Buth then suddenly one download worked for me. I did it again, it failed.
So, having now spent a number of minutes experimenting, it seems that
sometimes Download or Save Target As will work occasionally, but mostly they
won't. But try, try, again and you get lucky sometimes (less than 1 in 10).
Kerry
| | | | | | |
