Subject:
|
Re: Peeron.com crosses 1000 inventories posted!
|
Newsgroups:
|
lugnet.db.inv
|
Date:
|
Tue, 8 Jan 2002 17:16:41 GMT
|
Viewed:
|
935 times
|
| |
|
|
In lugnet.db.inv, Dan Boger writes:
> In lugnet.db.inv, Johannes Koehler wrote:
> > Unfortunately MOST German AFOLs can not use this database. And that's
> > really a shame because www.peeron.com is a very useful database.
>
> Unfortunatly, we had to block access to t-online.de after repeated
> spiders downloaded the whole site (ignoring robots.txt). We are limited
> in the bandwidth we can serve out a month, and didn't want to have the
> site go down to all, just because of some bad apples. When I get a
> chance, I'll try to put in a system where a user could log in, and get
> access anyway - that way, if a user misbehaves, I could block just that
> one user, instead of the whole domain.
Cool... will that be for all users across all domains or only for domains
you think are the source of bad spiders? (most users would not have to log
in but users from spiderrich domains would log in and get access, except for
those users that demonstrated they WERE spiders who would be locked out...
something like that?)
|
|
Message has 1 Reply:
Message is in Reply To:
 | | Re: Peeron.com crosses 1000 inventories posted!
|
| (...) Unfortunatly, we had to block access to t-online.de after repeated spiders downloaded the whole site (ignoring robots.txt). We are limited in the bandwidth we can serve out a month, and didn't want to have the site go down to all, just because (...) (23 years ago, 8-Jan-02, to lugnet.db.inv)
|
10 Messages in This Thread:
 
 
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
