Geeking : 2482


Off-Topic / Geek / 2482	2481 \| 2483

Subject:	News search function temporarily disabled
Newsgroups:	lugnet.admin.general, lugnet.off-topic.geek, lugnet.announce
Followup-To:	lugnet.admin.general, lugnet.off-topic.geek
Date:	Mon, 11 Dec 2000 17:34:24 GMT
Highlighted:	! (details)
Viewed:	10 times

The system crashed again. Either it's running out of memory and that's causing a downward spiral or it's running out of CPU cycles and starving enough processes to build up and cause a meltdown. Either way some tuning needs to be done. This is going to be my top LUGNET priority until things stabilize -- they've been shaky for the past couple of weeks. I need to find the bottlenecks and eliminate them. As a first step, I've turned off the text-search function for news. I apologize for any inconvenience this may cause, but it will help me know how much of a bottleneck it was causing. Its algorithm for merging results is grossly inadequate for some of the queries it processes. As a second step, I'm going to install a monitoring log that records the elapsed time and CPU time of every dynamically generated webpage. From those results, the bottlenecks will stand out like a fistful of sore thumbs. (Fixing them is a different matter.) I'm taking lunch at work right now; I'll be able to install the monitoring script quickly and then I'll have a look at the stored results later tonight. I probably won't be able to re-activate the text-search function of news until after replacing it with something that has better worst-case performance. There's a chance I may have to disable other things, like the Mosaic Maker (if it's being used and contributing to the bottleneck -- I don't know yet whether it is). Sometime before the end of December, I plan also to upgrade the RAM on the system. In Q1 of 2001, we'll add another physical box as well. --Todd

Message has 4 Replies:

		Re: News search function temporarily disabled
<nod nod> Thanks for letting us know. I'm no geek in any means, but first things first, I would limit the search. Sometimes a single search brings up thousands of posts. I suggest if the search is too broad, say it and make the searcher narrow it (...) (24 years ago, 13-Dec-00, to lugnet.admin.general, lugnet.off-topic.geek)

		Re: News search function temporarily disabled
(...) Hi Todd, Did you ever consider a google search box? Like this one: www.sis.pitt.edu/~dist I don't know for sure, but I think that the search is done on google's server. Toki (24 years ago, 17-Dec-00, to lugnet.admin.general, lugnet.off-topic.geek)

		Re: News search function temporarily disabled
(...) Certain bottlenecks are indeed standing out like sore thumbs. One big one is the dynamic generation of the /shop/ pages on guide.lugnet.com, and another is the dynamic generation of member-specific set lists. An even bigger one is (was) the (...) (24 years ago, 17-Dec-00, to lugnet.admin.general)

		News search function reactivated (was: News search function temporarily disabled)
The LUGNET News search function is now re-enabled. I completely revamped the index data structures and list-merge algorithm and rewrote the core query engine in C. It's a much more solid implementation. Everyone's patience during the outage is much (...) (24 years ago, 2-Jan-01, to lugnet.admin.general, lugnet.off-topic.geek, lugnet.announce) !!

45 Messages in This Thread:

Entire Thread on One Page:: Nested: All | Brief | Compact | Dots
Linear: All | Brief | Compact

Custom Search