Subject:
|
Re: News search function temporarily disabled
|
Newsgroups:
|
lugnet.admin.general
|
Date:
|
Sun, 17 Dec 2000 23:21:23 GMT
|
Viewed:
|
873 times
|
| |
|
|
Todd Lehman wrote:
> Certain bottlenecks are indeed standing out like sore thumbs. One big one is
> the dynamic generation of the /shop/ pages on guide.lugnet.com, and another
> is the dynamic generation of member-specific set lists. An even bigger one
> is (was) the dynamic generation of member listings (people) such as:
>
> http://www.lugnet.com/people/members/byfirstname.cgi
>
> That was taking several seconds of CPU and in some cases 200+ seconds of
> transmission time to people on slow modems. I can't do anything about the
> transmission time without reducing the page size, but for now I changed it
> to cache the content for faster page generation. Holding a large chunk of
> data in memory for 200 elapsed-time seconds while generating a page is bad,
> even if only a fraction of a CPU second is used.
How about breaking down the member pages into pages split by first
letter of first or last name, and by country/state? I'm not sure if your
"caching" means keeping the indexes by each sort but it would seem this
need not be true dynamic content. You don't get that many updates of
that content.
> Another bottleneck is that there seems to be a strong correlation between the
> CPU time to generate a page for a LEGO set and the number of people who've
> given comments or supplied personal data about the set. For example, this
> page:
>
> http://guide.lugnet.com/set/7130
>
> consumes 3 times as many CPU cycles as this page:
>
> http://guide.lugnet.com/set/1462
>
> or about 50 times as many CPU cycles above and beyond the regular overhead
> for a set page.
>
> A quick gnuplot graph with number of people on the x-axis and CPU time on
> the y-axis showed a nearly straight line correlation. That's good info for
> me to know. It ought to be is a gradual logarithmic-looking curve rather
> than a straight line.
>
> This isn't anyone's fault[1] and it doesn't mean that people need to stop
> or entering data -- it just means there's an identifiable inefficiency which
> I need to find a way to eliminate. A bottleneck like this won't get better
> with more RAM or a faster CPU -- it's a coding issue in need of algorithmic
> adjustments rather than quantitative system tuning.
How about making it optional if the personal data is included? Most
times I don't need it if my purpose for looking up the set is just to
see what the heck is that set. You can have a button on the bottom of
the page which says "show comments and personal data".
> The biggies for now are the set guide pages and, of course, bringing back
> the news search functionality. I think I can live with the inefficiencies
> of the set guide pages for a few more weeks, because their worst-case
> performance is still deterministic. The worst-case performance of the
> news-search function, however, was non-deterministic.
>
> I think I know of a way to make the search much more memory-efficient and
> later more time-efficient without radically ripping things apart. If a
> miracle occurs, the news search function might be back sometime this week.
Can you at all conveniently reduce the size of the news search if you
gave an option to "limit search to previous X posts" (where you pick one
or perhaps a handful of values for X, which would preferably be by time
span but could be by number [i.e. "limit search to the past week" would
be preferable to "limit search to the past 2000 posts"). Such search
constraints could certainly be useful and might constrain the search
enough to provide a serious performance improvement.
Frank
|
|
Message has 1 Reply:
 | | Re: News search function temporarily disabled
|
| (...) I think it's more of a problem of having to have the page in memory (all 300k of it) while a slow client downloads it. Nothing much to do about that except make the page smaller - perhaps move all the FONT FACE entries to an all encompassing (...) (24 years ago, 18-Dec-00, to lugnet.admin.general)
|
Message is in Reply To:
| | Re: News search function temporarily disabled
|
| (...) Certain bottlenecks are indeed standing out like sore thumbs. One big one is the dynamic generation of the /shop/ pages on guide.lugnet.com, and another is the dynamic generation of member-specific set lists. An even bigger one is (was) the (...) (24 years ago, 17-Dec-00, to lugnet.admin.general)
|
45 Messages in This Thread:
 
- Entire Thread on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
This Message and its Replies on One Page:
- Nested:
All | Brief | Compact | Dots
Linear:
All | Brief | Compact
|
|
|
|
