| |
I'm curious on why so many of the popular sets receive a "zero" rating.
It just doesn't make any sense. Unless it's a valid reason, these ratings
should be thrown out. Either these people are jealous or
complete morons.
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
I couldn't agree more, Ken! My addition to that question would be: why do
people rate a set they don't own? If you don't own it, then your rating
shouldn't count towards the member rating average because you have no idea
what you are truly reviewing.
</soapbox>
My guess would be that certain people feel that todays sets are lacking in
some way, so the just give a blanket rating of zero to anything that came
out after 199x.
Just my $.02
-Dave
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
One person has admitted to rating several sets a 0, but explained that it
was due to his misinterpreting the rating system-- he had thought it was the
highest score. IIRC he changed his votes. Others (there are reasons to
believe) are just done out of childishness...
DaveE
|
|
| |
In lugnet.general, David Eaton writes:
> Others (there are reasons to believe) are just done out of childishness...
As long as we're on the topic, I think far too many new sets get 90-100
ratings. I think that many people are just excited about new sets and seem
to mark them 90-100 to show approval of new offerings. I wonder how many of
these people go back and lower their score once a set has been out for a while.
Perhaps some people who mark everything as 0 are trying to "correct" the
skewed overall scores. I think that both of these actions (marking all new
sets high or low) are misguided. Really though, we all know that the rating
numbers are very subjective and not an accurate representation of everyone's
opinions.
|
|
| |
"Ken Godawa" <kengod@y-comm.com> wrote in message
news:H5pqzu.Mq1@lugnet.com...
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
Where are these ratings taking place?
Without seen the rating system mentioned I can say that there is a way to
get this rating better. Like IMDb (Internet Movie Database), there you can
see how many that have given a specific rating score. In IMDb there are
ratings from 0 to 10 where 10 is highest. As someone said in this thread,
some rate 0 just of childishness, and other probably rate max just of
excitment of a new film. So to REALY analyze the voting, you have to see
which rate has got most votes and where the other votes has gone.
To just take the average of the votes can give a wrong images of the rating.
Just a though.....
Regards
Øyvind Steinnes
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
This has been mentioned before. Generally, this kind of thing is an example
of ratings trashing. You like a particular set, but there are 20 sets in
front of it. You trash the ratings of the 20 sets by rating them 0 and
voila! Your favorite is now #1. There are ways to counter-act this, but
that involves someone doing some programming that he may not have the time
and energy for (or even see an overwhelming need for).
You can blame some of the 0 on me. I was curious to see what would happen
to the ratings by trying to even out all the zeroes - i.e. counter-trashing.
I meant to go back and remove all the zeros at some point, but I don't
recall doing so. You get a better idea of the relative rankings if I leave
them there, but I feel soiled by playing that same game, somehow.
A good set is a good set even if you aren't into that particular setting.
Bruce
|
|
| |
Hello!
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
As Ben said before I too think there are too many sets rated 100. I rated
just my absolute favourites in my collection a "100" (e.g. 6067, 6085) and
the absolute bad and superfluous sets a "0" (e.g. 6037).
Well, my most favourite sets are selected to be re-released by TLC. And this
is in my opinion one reason for some people to rate a set a "0": They want
to prevent these sets to become a "Legend", for TLC is looking after the
LUGNET ranking of the sets before they decide to make a set a "Legend" reissue.
On the other hand I guess many people rate a set (which they don't own) a
"100" just to raise a set's ranking to bring it into TLC's list of
candidates for re-releases.
Both of the procedures sound stupid.
Bye
Jojo
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
Can you give an example of one othe "popular sets" to which you refer? I'm
just curious to know which ones we're discussing.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
Perhaps it is a valid reason. I don't think very many folks around here are
morons. No moron could survive the LUGNET signup process. ;)
As well... what would your solution to this problem be? Would you like to
see the zero rating removed? Should every set, no matter how good or bad,
get a minimum 10 rating? Or, like in some judged sports, should we remove
the highest and the lowest rating in order to better average out the
remaining votes?
Regards,
Allan B.
|
|
| |
one example: Lego Star Destroyer 10030: http://guide.lugnet.com/set/10030
the zero was there before they even began to be shipped, and they still haven't
shipped to the states.. soooo...
> Perhaps it is a valid reason. I don't think very many folks around here are
> morons. No moron could survive the LUGNET signup process. ;)
perhaps not a moron.. perhaps a vindictive meanie.
perhaps give all sets a rank based on rankings, but not just taking into
account the average.. such as where most of the marks lay. (ex: all 100's and
90's, and one or two 10's or 0's.. obviously something is wrong) or perhaps
people should always have to give a comment as to why a set is at a certain
level.
-lenny
>
> As well... what would your solution to this problem be? Would you like to
> see the zero rating removed? Should every set, no matter how good or bad,
> get a minimum 10 rating? Or, like in some judged sports, should we remove
> the highest and the lowest rating in order to better average out the
> remaining votes?
>
> Regards,
> Allan B.
|
|
| |
"Allan Bedford" <ExpertBuilder-DELETE-TO-REPLY@apotome.com> wrote in message
news:H5qDpB.K2@lugnet.com...
> In lugnet.general, Ken Godawa writes:
> > I'm curious on why so many of the popular sets receive a "zero" rating.
>
> Can you give an example of one othe "popular sets" to which you refer? • I'm
> just curious to know which ones we're discussing.
Here are a few sets that I can not understand what valid reason(s) for a "0"
rating:
6399 Airport Shuttle - two "0" ratings
5571 Giant Truck / Black Cat - one "0" rating
3451 Sopwith Camel - one "0" rating
4558 Metroliner - one "0" rating
4547 Railroad Club Car - one "0" rating
4554 Metro Station - one "0" rating
6394 Metro Park & Service Tower - one "0" rating
6286 Skull's Eye Schooner - two "0" ratings
6285 Black Seas Barracuda - two "0" ratings
6339 Shuttle Launch Pad - one "0" rating
6067 Guarded Inn - two "0" ratings
6390 Main Street - one "0" rating
497 Galaxy Explorer - one "0" rating
375 Castle - one "0" rating
Ken
|
|
| |
In lugnet.general, Ken Godawa writes:
> "Allan Bedford" <ExpertBuilder-DELETE-TO-REPLY@apotome.com> wrote in message
> news:H5qDpB.K2@lugnet.com...
> > In lugnet.general, Ken Godawa writes:
> > > I'm curious on why so many of the popular sets receive a "zero" rating.
> >
> > Can you give an example of one othe "popular sets" to which you refer? • I'm
> > just curious to know which ones we're discussing.
>
>
> Here are a few sets that I can not understand what valid reason(s) for a "0"
> rating:
>
> 6399 Airport Shuttle - two "0" ratings
> 5571 Giant Truck / Black Cat - one "0" rating
> 3451 Sopwith Camel - one "0" rating
> 4558 Metroliner - one "0" rating
> 4547 Railroad Club Car - one "0" rating
> 4554 Metro Station - one "0" rating
> 6394 Metro Park & Service Tower - one "0" rating
> 6286 Skull's Eye Schooner - two "0" ratings
> 6285 Black Seas Barracuda - two "0" ratings
> 6339 Shuttle Launch Pad - one "0" rating
> 6067 Guarded Inn - two "0" ratings
> 6390 Main Street - one "0" rating
> 497 Galaxy Explorer - one "0" rating
> 375 Castle - one "0" rating
Interesting list. A wide cross-section of themes, from train to space and
from pirates to a new sculpture. Not one of them has received more than two
zeros. That doesn't seem quite enough to cause the set to be delisted as a
potential Legends reissue.
I think there are an equal number (or more) of sets that are over-rated. I
find that when I am about to rate a favorite set of mine, the temptation is
always to give it a 100. After all, if I like it, then it must be great,
right? Wrong. Few sets (like people) are perfect. So I often drop down to
a 90, 80 or 70. Still a worthy rating. By not giving it a 100, I'm
essentially trying to say, "yes, this set is great... but there are some
little things that could have been better." You can never get to perfect,
but it never hurts to try. :)
There could be any number of reasons someone rated a set with a zero:
1) Just plain doesn't like that theme and therefore sees the set as lacking
in design, although it may appeal to those who do like the theme.
2) An attempt to balance out some of the *over* rated sets. There are some
sets that crusty old-timers like me tend to sentimentalize. The Galaxy
Explorer might be one such example. Sure it's the coolest of the original
space sets, but is it really *that* good? Is bright blue and grey really a
good color scheme for space ships? Should space ships really be that
angular? There are a number of arguments to say, "yup, that set is
over-rated and I'm going to rate it low in an attempt to balance out the
overall rating."
3) Someone was disappointed that a particular set was released as a Legend,
or perhaps released at all. Maybe someone looked at the Sopwith Camel and
thought that it violated LEGO's own policy against releasing sets containing
20th century military weapons. Maybe the zero rating was their way of
saying, "LEGO, you messed up and I'm not happy at all that you released this
set."
4) Or, perhaps the set got a zero because the person felt that the LEGO
company really missed somehow. Maybe the set was badly designed from a
structural point of view. Maybe it has ugly colors. Maybe it didn't look
the least bit realistic.
I would say there are a number of non-moronic reasons that someone might
rate a set a zero. But in the overall scheme of things, I'm not sure it's
creating a problem. Two zeros for some of these sets really doesn't reduce
the overall rating that much when many of the other votes are 100.
I'm still curious though, what do you see as a solution? It's o.k. to
criticize (trust me, I'm an expert) but it is also appropriate in a case
like this to offer an alternative solution. Any thoughts on how to rectify
the "zero problem"?
All the best,
Allan B.
|
|
| |
In lugnet.general, Kevin Wilson writes:
> Dave Johann wrote in message ...
> > I couldn't agree more, Ken! My addition to that question would be: why >do
> > people rate a set they don't own? If you don't own it, then your rating
> > shouldn't count towards the member rating average because you have >no idea
> > what you are truly reviewing.
>
> People don't necessarily enter the sets they own on LUGNET - I don't - >so
> the two things, marking a set as owned and rating it, are really quite
> separate.
>
> Kevin
> -----------------------------------------------------------------------
> NEW Cottage kit, 577 pieces! >http://www.lionsgatemodels.com/cat-cott.htm
> -----------------------------------------------------------------------
> LEGO TOWN PLANNING information:
> http://www.lionsgatemodels.com/COntent/Townplan/townplan.htm
> BrickLink Lego parts store: >http://www.bricklink.com/store.asp?p=Kevinw1
> The Guild of Bricksmiths(TM): http://www.bricksmiths.com
> Personal Lego Web page:
> http://ourworld.compuserve.com/homepages/kwilson_tccs/lego.html
This brings up a whole new issue: what criteria is used in rating a set?
Personally, I wouldn't think of rating a set I don't own/haven't previously
owned simply because I don't know how much fun that set was to build, what
parts are in that set, overall play value, etc. This is what I thought the
ratings system was for rather than the 'trash the new sets because they
don't live up to the older standards' which seems to be what the ratings
system has devolved into.
Quite honestly, I find the comments (People who wrote notes about this set)
much more informative than the overall ratings. Helpful hints, hidden parts,
clearance prices and such can be found in this area.
-Dave
|
|
| |
"Allan Bedford" <ExpertBuilder-DELETE-TO-REPLY@apotome.com> wrote in message
news:H5qnsF.2ru@lugnet.com...
<snipe>
> I would say there are a number of non-moronic reasons that someone might
> rate a set a zero. But in the overall scheme of things, I'm not sure it's
> creating a problem. Two zeros for some of these sets really doesn't • reduce
> the overall rating that much when many of the other votes are 100.
>
> I'm still curious though, what do you see as a solution? It's o.k. to
> criticize (trust me, I'm an expert) but it is also appropriate in a case
> like this to offer an alternative solution. Any thoughts on how to • rectify
> the "zero problem"?
>
> All the best,
> Allan B.
Good post. I agree that some people over rate sets. But when the majority
of the ratings of a set are either 100 or 90 then I don't see how that same
set
can receive a 0. I would bet that those who rate a popular set a 0 don't
own that set. If they did, I would like to read why that person rated it a
0
in the comments section.
I would like to see a system where the top rating and bottom rating are
thrown out.
Similar to what is done in diving and ice skating scores. And depending on
how many
total ratings a set gets will determine how many of the top and bottom
ratings are thrown out.
Ken
|
|
| |
In lugnet.general, Allan Bedford writes:
> In lugnet.general, Ken Godawa writes:
> > I'm curious on why so many of the popular sets receive a "zero" rating.
>
> Can you give an example of one othe "popular sets" to which you refer? I'm
> just curious to know which ones we're discussing.
>
> > It just doesn't make any sense. Unless it's a valid reason, these ratings
> > should be thrown out. Either these people are jealous or
> > complete morons.
>
> Perhaps it is a valid reason. I don't think very many folks around here are
> morons. No moron could survive the LUGNET signup process. ;)
>
> As well... what would your solution to this problem be? Would you like to
> see the zero rating removed? Should every set, no matter how good or bad,
> get a minimum 10 rating?
I would say probably not... some sets really ARE duds.
> Or, like in some judged sports, should we remove
> the highest and the lowest rating in order to better average out the
> remaining votes?
That's not a bad idea. It works for Figure Skating (and we know how fair and
impartial THAT sport is!) Grin.... No seriously, it IS a good idea.
I would say this, though.... how abou this... (and it may be overengineering
the solution.) My supposition is that someone has went through and given a
lot of sets as 0. In fact wasn't there a case early in ratings where it was
known that someone went through and rated every single Pirate set with 0?
Maybe there should be a requirement that in order to be allowed to rate sets
you need to rate some sets good as well as some bad? Require that your
average rating across all sets you rated is at least 10 and no less than 90?
That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100. This will
encourage people to put some thought into what they like and dislike. You
get 5 0s (or 5 100s) and then you have to go find something you like (or
dislike, as the case may be) or else no more ratings are accepted until your
average is within range.
Seriously, if you think every single set LEGO ever made is worth only 0,
what are you doing here at LUGNET? I know Allan and I have different
opinions but we both can find sets we'd rate higher than 0, and sets we'd
rate lower than 100... I am sure of it.
Probably not really a good idea but maybe it will get people thinking?
(deciding it's not really a major problem is an OK outcome)
++Lar
|
|
| |
In lugnet.general, Dave Johann writes:
> This brings up a whole new issue: what criteria is used in rating a set?
> Personally, I wouldn't think of rating a set I don't own/haven't previously
> owned simply because I don't know how much fun that set was to build, what
> parts are in that set, overall play value, etc. This is what I thought the
> ratings system was for rather than the 'trash the new sets because they
> don't live up to the older standards' which seems to be what the ratings
> system has devolved into.
>
> Quite honestly, I find the comments (People who wrote notes about this set)
> much more informative than the overall ratings. Helpful hints, hidden parts,
> clearance prices and such can be found in this area.
>
> -Dave
My most important criterion is how much I love the set. One of my favorite
sets, 4161, the girls' pink oval suitcase, is an example. I love it because
it's a girl set and it has colors I love, and it has pink lampposts. And it
has sentimental value because it was one of the first sets I bought and
allowed myself to keep instead of giving it to a nephew because I was
embarrassed to be playing with LEGO. In the notes I write about the
lampposts, but I also mention the yellow corner slopes for the benefit of
others who may be interested. Still, I only rated it a 90 because I believe
that somewhere out there the Ultimate LEGO Set exists (and I don't mean set
#3800!).
Maggie C.
|
|
| |
Is there any way to make the rating public? The votes must be linked to the
voter somehow in the database. Could links be set up on the 100-0 scale to
let people see who voted 100, 90, ..., 0 for the set. Maybe people would
vote how they actually felt instead of trying to skew the results if they
were somehow held accountable for their vote. Just an idea...
James
|
|
| |
In lugnet.general, Larry Pieniazek writes:
> In lugnet.general, Allan Bedford writes:
> > In lugnet.general, Ken Godawa writes:
> > > I'm curious on why so many of the popular sets receive a "zero" rating.
> > As well... what would your solution to this problem be? Would you like to
> > see the zero rating removed? Should every set, no matter how good or bad,
> > get a minimum 10 rating?
>
> I would say probably not... some sets really ARE duds.
>
> > Or, like in some judged sports, should we remove
> > the highest and the lowest rating in order to better average out the
> > remaining votes?
>
> That's not a bad idea. It works for Figure Skating (and we know how fair and
> impartial THAT sport is!) Grin.... No seriously, it IS a good idea.
>
> I would say this, though.... how abou this... (and it may be overengineering
> the solution.) My supposition is that someone has went through and given a
> lot of sets as 0. In fact wasn't there a case early in ratings where it was
> known that someone went through and rated every single Pirate set with 0?
>
> Maybe there should be a requirement that in order to be allowed to rate sets
> you need to rate some sets good as well as some bad? Require that your
> average rating across all sets you rated is at least 10 and no less than 90?
> That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100. This will
> encourage people to put some thought into what they like and dislike. You
> get 5 0s (or 5 100s) and then you have to go find something you like (or
> dislike, as the case may be) or else no more ratings are accepted until your
> average is within range.
Just *slightly* overengineered. :)
To implement the drop high and low rating idea, it might look something like
this:
averageRating = ((sumOfAllRatings - lowestRating) - highestRating) /
(totalNumberOfVotes - 2)
I think. :)
> Seriously, if you think every single set LEGO ever made is worth only 0,
> what are you doing here at LUGNET? I know Allan and I have different
> opinions
Perhaps not as much now as we did at one time. I have really tried to focus
on the good things about LEGO, as I know I was guilty of not doing that in
the past. Generally, I've tried to stay out of discussions like this for
the last few months. But I saw in this thread the potential to help sort
out something that was clearly bothering another user.
<ANGEL MODE OFF> ;)
> but we both can find sets we'd rate higher than 0, and sets we'd
> rate lower than 100... I am sure of it.
Which is why I can think of perhaps only one or two sets I might ever rate
with a zero. And they are part of a series that rhymes with Black Throne. :)
But seriously, I do try to find at least *some* good in any set I'm about to
rate. I personally think more sets fall into the 40 - 60 range than some of
us might like to admit.
> Probably not really a good idea but maybe it will get people thinking?
> (deciding it's not really a major problem is an OK outcome)
Yes, I think the discussion of this issue is more important than effecting a
solution. From the evidence given, I don't think any sets are really
'hurting' from the zero ratings, but it obviously bothered Ken enough to
start this thread. I think getting everyone's opinion is just as critical
as fixing something that may not be that broken.
All the best,
Allan B.
|
|
| |
In lugnet.general, James Brink writes:
> Is there any way to make the rating public? The votes must be linked to the
> voter somehow in the database. Could links be set up on the 100-0 scale to
> let people see who voted 100, 90, ..., 0 for the set. Maybe people would
> vote how they actually felt instead of trying to skew the results if they
> were somehow held accountable for their vote. Just an idea...
A very good idea. I'm not sure how much screen real estate would be
required, but this would certainly keep people honest.
Regards,
Allan B.
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
Well, anyone actually using these ratings for anything important would throw
out outlying votes anyway. Unless they're complete morons 8?)
And if they're not using it for anything important, what difference does it
make?
I see no reason to change the way it currently works.
ROSCO
|
|
| |
In lugnet.general, Allan Bedford writes:
> In lugnet.general, James Brink writes:
> > Is there any way to make the rating public? The votes must be linked to the
> > voter somehow in the database. Could links be set up on the 100-0 scale to
> > let people see who voted 100, 90, ..., 0 for the set. Maybe people would
> > vote how they actually felt instead of trying to skew the results if they
> > were somehow held accountable for their vote. Just an idea...
>
> A very good idea. I'm not sure how much screen real estate would be
> required, but this would certainly keep people honest.
If it was done so that those interested could click on a rating, and that
would bring up a list of everyone that gave that rating, it need take up no
extra space at all...
I tend to check the database for stuff that I'm interested in buying (either
new or on e-bay), but I pay far more heed to the comments than I do to the
ratings, because I know who said what (and the comments tend to have
additional useful stuff as well).
Cheers
Richie Dulin
|
|
| |
"Larry Pieniazek" <lpieniazek@mercator.com> wrote in message
news:H5qp07.5wD@lugnet.com...
> Maybe there should be a requirement that in order to be allowed to rate sets
> you need to rate some sets good as well as some bad? Require that your
> average rating across all sets you rated is at least 10 and no less than 90?
> That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100.
If we have a 0-100 scale, then assuming some kind of normal distribution, we
would expect the average rating to be about 50 with a standard deviation of
(say) 20. Therefore, it becomes possible to moderate a member's ratings
according. Basically you take the set of actual ratings they have contributed,
and then calculate the actual mean and standard deviation of those ratings.
Then you adjust each actual rating N as follows:
X * N + Y
where X is the relative standard deviation and Y is the relative mean. The
effect of doing this is that the adjusted ratings should produce the desired
mean and standard deviation. Thus *overall* set ratings will have a mean of 50
and s/d of 20.
Individual set ratings can then be computed from these adjusted ratings
provided by each LUGnet member (just do a normal simple mean).
In simple terms what this means is that people who consistently over-rate sets
high will have their ratings scaled down, and vice versa. If people are being
stupid (e.g. allocating 100 to all their favourite sets and 0 to every thing
else), the standard deviation will be too large and cause the ratings to be
scaled back to about 70 and 30 respectively.
Before anyone says this is ridiculous, I would point out that here in
Queensland, we operate our tertiary admissions system on this basis. As we do
not have external exams taken by every final year student, each school sets its
own assessment and rates students accordingly. To prevent schools from
over-stating (or under-stating) the abilities of their students, the scores
provided by the schools are adjusted to achieve the mean and standard deviation
applicable to general intelligence tests applied to the same group of students
(these tests are mandatory and standardised across the state). So, while the
students's individual results come from school-based assessment, these are
adjusted to reflect the overall class performance to the state-based
assessment. The general effect of this is that subjects that tend to attract
only the brighter students (e.g. physics) have their results up-rated and that
subjects that tend to attract less-bright students (e.g. social maths) tend to
have their overall results down-rated. This system of university admission
assessment has been operational for about 30 years now, and apart from minor
tinkering, the basic principle of adjusting each class's
mean/standard-deviation has remained unchanged as a fair basis for tertiary
admission.
Because the process is entirely mechanical, it requires only programming and no
human intervention.
Note. This system is still based on the assumption that a high rating indicates
a set preferred by the user to a low rating. If people are reverse-rating sets
(through error, malice, or a desire to deliberately distort the results in some
way), then nothing can solve that problems, but the more extreme results will
probably be scaled back to something more reasonable and the consequent impact
on the overall rating of a set will be reduced.
So a person who submitted all 0-ratings would get scaled back to 50-ratings on
each set (i.e. no impact on the overall means). Similarly any person who
submitted only 100-ratings would get scaled back to 50-ratings as well. This
means that people need to equally willing to rate the sets they like and the
sets they dislike, as otherwise their ratings will be scaled up/down to produce
a more "average" mean. I've argued elsewhere on LUGnet that most recent
releases get rated (good/bad as appropriate), but only the older *better* sets
get rated (and hence rated favourably) as everyone forgets about the also-ran
sets of the past and doesn't bother to rate them. IMHO this is why older sets
tend to overrank newer sets
Kerry
|
|
| |
In lugnet.general, Kerry Raymond writes:
>
> In simple terms what this means is that people who consistently over-rate sets
> high will have their ratings scaled down, and vice versa. If people are being
> stupid (e.g. allocating 100 to all their favourite sets and 0 to every thing
> else), the standard deviation will be too large and cause the ratings to be
> scaled back to about 70 and 30 respectively.
>
> Before anyone says this is ridiculous, I would point out that here in
> Queensland, we operate our tertiary admissions system on this basis. As we do
> not have external exams taken by every final year student, each school sets
> its own assessment and rates students accordingly. To prevent schools from
> over-stating (or under-stating) the abilities of their students, the scores
> provided by the schools are adjusted to achieve the mean and standard
> deviation applicable to general intelligence tests applied to the same group
> of students (these tests are mandatory and standardised across the state). So,
> while the students's individual results come from school-based assessment,
> these are adjusted to reflect the overall class performance to the state-based
> assessment. The general effect of this is that subjects that tend to attract
> only the brighter students (e.g. physics) have their results up-rated and that
> subjects that tend to attract less-bright students (e.g. social maths) tend to
> have their overall results down-rated.
This is off-topic, but I really like the sound of your system. Here in
Canada, we don't have standardised testing (like the SATs in the US) or any
sort of grade balancing like this. Grade inflation is rampant.
> So a person who submitted all 0-ratings would get scaled back to 50-ratings on
> each set (i.e. no impact on the overall means). Similarly any person who
> submitted only 100-ratings would get scaled back to 50-ratings as well. This
> means that people need to equally willing to rate the sets they like and the
> sets they dislike, as otherwise their ratings will be scaled up/down to >produce a more "average" mean.
I only rate sets that I have built, and tend to buy only sets that I know
I'll like. So, most of my ratings are 70-100. If a system like this is
implemented, does this mean I'll have to poorly rate some sets that I don't
own if I want the sets I have rated highly to retain the full grade that I
think they deserve?
Jeff J
|
|
| |
To All,
This is a very interesting discussion, I rated just about every set I own,
with most of them having comments, and I think it is a great feature.
http://guide.lugnet.com/set/mlist.cgi?m=227
I am not sure why some people rate sets at 0, most others already offered up
explanations.
Here is a set I gave a zero to, a Bionicle named Nokama. I bought this set
because someone wanted to use to the little blue tool on a starship. After
building the guy, and seeing it, I simply detested it. I think the Bionicle
is a bad line, not in tune with what I consider TLC products to be.
I have received a number of e-mails about my ratings in regards to certain
sets, and have discussed it with the person. My favorite happened to be the
6514 Trail Ranger:
http://guide.lugnet.com/set/6514
Someone complained I rated a 30, since I thought it was good for parts, and
the main model was, IMHO, dumpy. I bumped it up to 40, but outside of parts,
I thought it was a bad set. I think we need to remember most people put
their preferences in the ranking, as opposed to any scientific ranking.
I think a revising of the rating might be a good way to go, perhaps you can
have a system where you rate different aspects of the set, say the quality /
quanitity of the parts, the quality of the model, etc. Then average those to
get a final ranking. It might be a good way to balance some of the numbers.
Scott Sanburn
Member, MichLUG MichLTC http://www.michlug.org
Webpages: http://www.scottesanburn.org
LEGO Pages: http://www.scottesanburn.org/legoindex.html
|
|
| |
In lugnet.general, Kerry Raymond writes:
>
> "Larry Pieniazek" <lpieniazek@mercator.com> wrote in message
> news:H5qp07.5wD@lugnet.com...
> > Maybe there should be a requirement that in order to be allowed to rate sets
> > you need to rate some sets good as well as some bad? Require that your
> > average rating across all sets you rated is at least 10 and no less than 90?
> > That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100.
<snip well thought out but complex system>
And someone had the temerity to actually agree when I called MY proposal
"possibly overengineered"??? :-)
I like it. I think it would work. I never know when Kerry's spoofing or not,
though. :-)
|
|
| |
Ken Godawa wrote:
>
> "Allan Bedford" <ExpertBuilder-DELETE-TO-REPLY@apotome.com> wrote in message
> news:H5qnsF.2ru@lugnet.com...
>
> <snipe>
> > I would say there are a number of non-moronic reasons that someone might
> > rate a set a zero. But in the overall scheme of things, I'm not sure it's
> > creating a problem. Two zeros for some of these sets really doesn't • reduce
> > the overall rating that much when many of the other votes are 100.
> >
> > I'm still curious though, what do you see as a solution? It's o.k. to
> > criticize (trust me, I'm an expert) but it is also appropriate in a case
> > like this to offer an alternative solution. Any thoughts on how to • rectify
> > the "zero problem"?
> >
> > All the best,
> > Allan B.
>
> Good post. I agree that some people over rate sets. But when the majority
> of the ratings of a set are either 100 or 90 then I don't see how that same
> set
> can receive a 0. I would bet that those who rate a popular set a 0 don't
> own that set. If they did, I would like to read why that person rated it a
> 0
> in the comments section.
>
> I would like to see a system where the top rating and bottom rating are
> thrown out.
> Similar to what is done in diving and ice skating scores. And depending on
> how many
> total ratings a set gets will determine how many of the top and bottom
> ratings are thrown out.
I haven't read to the end of this thread yet, but some of my thoughts
after reading a bunch:
Throwing out the highest and lowest score might not really help since
then it would just mean two people would have to decide to trash a set
or theme.
I kind of like Larry's suggestion of a limitation on the average of a
particular member's ratings. I would be inclined though to just require
the average to be at least 50 (and perhaps less than 90). Why? I think
it's fair to insist that you rate at least as many sets good as you rate
bad (this would force people to save their share of poor ratings for
sets they cared about as opposed to just trashing a theme they didn't
like - or they could trash a theme they didn't like and not get to rate
low the truly poor sets in their theme of interest). The reason I
wouldn't require the average to be in a small range in the center is
that many people would choose only to rate those sets they liked.
On this last part, perhaps a better rating system would be to ask people
to rate sets on a scale of 0-10 (why do we need 0-100?), then show the
distribution of ratings which are at least a 5. Also, show the total
number of people rating the set, and the total number of people rating
sets at all. Perhaps even give two rating for the set, one for parts,
and one for set quality (i.e. Larry might rate a Castle set as a 5 for
quality, but an 8 for parts). This would do a better job of showing the
sets which were truly good since they wouldn't be affected by the
naysayers. If a set really isn't that good, it's not going to have many
ratings 5 or higher. A set with 100 ratings 5 or higher which average 7
is probably a better set than one with 3 ratings which average 10 (and
you'll probably know if you fit in the niche that the 3 10s set perhaps
is particularly attractive to).
On the other hand, it's almost impossible to really get a good ratings
system since you really need to know how the likes of the people giving
the ratings match your own (that's why I tend to almost ignore movie
reviews - though I will pay some attention to the number of stars the
movie is rated with, but will completely ignore them if I have heard
good things about the film from friends, and consider it a movie I want
to see [for example a Star Wars movie - sure, the recent ones don't seem
as good as A New Hope, but then I realize I'm a different person, and
I've probably got a somewhat romanticized opinion of A New Hope]).
I've certainly had little problem figuring out which sets were
desireable without a ratings system.
Frank
|
|
| |
In lugnet.general, Kerry Raymond writes:
> If we have a 0-100 scale, then assuming some kind of normal distribution, we
> would expect the average rating to be about 50 with a standard deviation of
> (say) 20. Therefore, it becomes possible to moderate a member's ratings
> according. Basically you take the set of actual ratings they have contributed,
> and then calculate the actual mean and standard deviation of those ratings.
> Then you adjust each actual rating N as follows:
>
> X * N + Y
>
> where X is the relative standard deviation and Y is the relative mean. The
> effect of doing this is that the adjusted ratings should produce the desired
> mean and standard deviation. Thus *overall* set ratings will have a mean of 50
> and s/d of 20.
Heh-- I actually did something very similar for our company when we were
sending out our customer service surveys (rated 1-5). Obviously some clients
were overly thrilled with us and just gave us straight 5's. Some were mad at
us and gave us straight 1's. (In our company's defense we had far more
people who gave us all 5's than all 1's). One averaging system I had
effectively made these people's votes ALL turn into 3's. It was very
interesting. Needless to say I like the idea.
> Because the process is entirely mechanical, it requires only programming and >no human intervention.
I'm not entirely sure to what this is referring-- was there a system
elsewhere mentioned that was in part manual?
> Note. This system is still based on the assumption that a high rating
> indicates a set preferred by the user to a low rating. If people are
> reverse-rating sets (through error, malice, or a desire to deliberately
> distort the results in some way), then nothing can solve that problems, but
> the more extreme results will probably be scaled back to something more
> reasonable and the consequent impact on the overall rating of a set will be
> reduced.
To some extent this is deal-with-able with the "remove out-of-whack" votes
system. If a set gets 45 100's, 37 90's, 18 80's, 2 70's and 1 0 rating, the
0 rating is sufficiently out of the standard bell curve to automagically
throw it out. Other sets that didn't follow such nicely curved data would be
tougher for ruling out votes that didn't fit, but then again, if a set has
erratic data, it might very well have earned a 100 and/or 0 vote from someone.
Another system I had done some experimenting with was the "Remove the lowest
and highest vote for every N votes that exist for a given set, disregarding
the default '50' vote". Worked ok.
The real issue (I think) isn't in overal individual set ratings, however,
but in set *rankings*. In the above system, for example, the 0 vote brings
the mean down from 92.255 to 91.359. No big whoop. But when you show the
"Top X sets of all time based on Lugnet rankings", it's enough to bring it
down SIGNIFICANTLY. I've played around with the "remove N votes" system on
the Lugnet guide rankings with some interesting results. Some sets that
"should" be on the top 3 don't even make it to the top 10, etc. IIRC the
first time I tested (with all sorts of values for N), the Guarded Inn ranked
#1 the most times, but in the straight mean ranking (as shown on the Lugnet
page) it rated somewhere around #18. A more recent test (September 16th this
year) revealed the Black Seas Barracuda 'should' have been #1 but was ranked
#16 on Lugnet. Tsk tsk. I'd be curious to see what your suggested system
would reveal...
> I've argued elsewhere on LUGnet that most recent
> releases get rated (good/bad as appropriate), but only the older *better* sets
> get rated (and hence rated favourably) as everyone forgets about the also-ran
> sets of the past and doesn't bother to rate them. IMHO this is why older sets
> tend to overrank newer sets
I tend to rate according to:
- 40 points for piece selection, price per piece, etc
- 40 points for set design, set features, etc
- 20 points for set appeal (bonus points!)
I usually start each set off at 50, and go from there-- IE a 'regular' set
starts with 20 points for piece selection, 20 points for set design, and 10
points for appeal... But I will admit my own 'average' rating that I've
given is probably somewhere around 70.
One of the flaws of course being what you mention-- people don't rate junky
sets as much as they rate great sets. Personally, I only rate what I own.
And (obviously) if I think a set is junky, I'm not going to want to buy it,
and therefore I probably won't own it. I don't own any of the silly racers--
and if I did, I'd probably rate them at 0-20 each. But I don't own them, so
I won't rate them. As a result, my average (and others assumedly) set rating
is probably somewhere closer a 50 or 60, but we'll most likely never know...
DaveE
|
|
| |
In lugnet.general, Ken Godawa writes:
> I'm curious on why so many of the popular sets receive a "zero" rating.
> It just doesn't make any sense. Unless it's a valid reason, these ratings
> should be thrown out. Either these people are jealous or
> complete morons.
I just read through the majority of this thread. Here's my take...
I tend to regard the set ratings on the Lugnet database as mostly just a
curiosity. I don't take them that seriously, partly because I already know
what sets *I* like, and other people's rankings aren't likely to change
mine. But also, I have noticed that there are different groups of rankers
who fall into certain camps. Some people vote high on newer themes like SW
but the same people might vote low on older sets. Other people who are
nostalgic for older sets might vote higher for Classic Space or Town. And
finally, the ranking system is purely one of "What's your impression of this
set?" It does not ask the voter to consider price per piece, or to consider
the useful parts in a set where the main model is poor, or vise versa.
For these reasons I find the ratings interesting, but no basis for me
deciding what sets to buy. And for the same reasons, I think the comments
section for each set is far more valuable (and sometimes entertaining).
Curiously though, even the comments are not always ones I agree with either.
Take this set...
http://guide.lugnet.com/set/361_2
...The only comment listed is somewhat negative. However, I look at the
image of this set and think, "Wow, I'd love to have several of these for my
middle-class housing project in my Town." The only think I'd add is a trash
barrel (which wasn't available when the set was).
...However, if we really want a meaningful rating system, perhaps some
definition or example might be worthwhile, besides just "pick a number from
x-y". Maybe something like this:
100 = The price, piece selection, model, etc. are all so superb you simply
cannot imagine a set ever being better.
90 = One of your favorite sets. You wish you could buy more.
80 = A great set, but not quite as great as your favorites.
70 = A good compliment to other sets, or a good source of pieces.
60 = A decent set that could be better in some significant way.
50 = A mediocre set that is decent for those who really like the theme.
40 = A fair set for play or pieces or price, but only one such quality.
30 = A poor set that has one or two redeeming qualities like a rare piece.
20 = A set you think could have been improved on all levels of price, piece
selection, playability, etc.
10 = You think the rights for this set should have been sold off to a TLC
competitor it is so bad.
00 = That's a set?
Well, that's my thought...
-Hendo
|
|
| |
In lugnet.general, John P. Henderson writes:
> ...However, if we really want a meaningful rating system, perhaps some
> definition or example might be worthwhile,
I think part of the problem is people buy sets for different reasons. I (for
example) would've rated the set you mentioned a 70 because I'd love it for
part selection; though the model itself doesn't really intreuge me to buy
it. I think what we need is several different categories per set to rate:
-Price per piece (0-10)
-Piece selection (0-10)
-Model design (0-10)
-Model functionality (0-10)
-Model appeal (0-10)
Note model appeal referrs to something like "how badly do you want this set
based on what it is, regardless of design/functionality/etc?" For instance,
I wanted a Millenum Falcon. I personally think the price per piece ratio is
bad, the piece selection is ok, the model design is bad, and the
functionality is ok-to-bad. But I still want one, BECAUSE it's a Millenium
Falcon. Similarly, I want a Galaxy Explorer because it's nostalgic; not
because I think it's a great model or anything. AND similarly, someone who
hates, say, the pirate theme, could rate the BSB's appeal a '0', while still
agreeing that it was a good model design, had good pieces, etc. As if to say
"Pirates don't appeal to me, but if they DID, I'd buy this set".
That would probably help people a bit more, since they could fine-tune the
ratings a bit more to reflect what they'd really like. So if you were trying
to decide what to buy, and you shop based on part selection more than
functionality, you can look for sets that rated higher on that front.
Anyway, more ideas for the fire...
DaveE
|
|
| |
In lugnet.general, Kerry Raymond writes a really neat statistical analysis
thingie which I snipped:
My question here is "What are you using this information for which requires
such analysis?".
I mean do you base your set purchases on these figures? And if so, how many
times have you been burned by false "0" or "100" votes??
Come on people! Just get back to building!
My 2 cents
ROSCO
|
|
| |
In lugnet.general, Kerry Raymond writes:
>
> "Larry Pieniazek" <lpieniazek@mercator.com> wrote in message
> news:H5qp07.5wD@lugnet.com...
> > Maybe there should be a requirement that in order to be allowed to rate sets
> > you need to rate some sets good as well as some bad? Require that your
> > average rating across all sets you rated is at least 10 and no less than 90?
> > That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100.
>
> If we have a 0-100 scale, then assuming some kind of normal distribution, we
> would expect the average rating to be about 50 with a standard deviation of
> (say) 20.
I'm not sure that an average rating of 50 would be appropriate in this
context. One of the ongoing themes of discussion in LUGNET is that LEGO set
designs deteriorate over time. The apex of LEGO set design is generally
thought to be in the past, with opinions varying from around the mid 70's
through to about the early 90's. Some people would say the first release of
Star Wars was the apex, but I think we can happily dismiss them as being
caught up in the commercial hype of the whole Star Wars thing.
Therefore, we could not peg ratings to an average of 50. The average rating
should really be based on a mathematical formula based on year of release.
If you were going to say that an average of 50 applied over time, despite
commonly accepted deterioration over time, you would have to also accepted
an average of 50 over themes. Now, the sole Insectoids fan out there might
think this is good, because his ugly (and mostly blue) sets are going to be
miraculously lifted in the ratings, but consider the legion of people who
like blue boxes with wings who would going to see the Classic Space ratings
tumble.
> Therefore, it becomes possible to moderate a member's ratings
> according. Basically you take the set of actual ratings they have contributed,
> and then calculate the actual mean and standard deviation of those ratings.
> Then you adjust each actual rating N as follows:
>
> X * N + Y
>
> where X is the relative standard deviation and Y is the relative mean. The
> effect of doing this is that the adjusted ratings should produce the desired
> mean and standard deviation. Thus *overall* set ratings will have a mean of 50
> and s/d of 20.
>
> Individual set ratings can then be computed from these adjusted ratings
> provided by each LUGnet member (just do a normal simple mean).
>
> In simple terms what this means is that people who consistently over-rate sets
> high will have their ratings scaled down, and vice versa. If people are being
> stupid (e.g. allocating 100 to all their favourite sets and 0 to every thing
> else), the standard deviation will be too large and cause the ratings to be
> scaled back to about 70 and 30 respectively.
Ahhh... but you fail to appreciate that certain people are better judges,
who would - of course - only purchase better sets. Why should those train
fans be hampered in properly rating their sets, when no hopers who bought up
big on Aquazone Hydronauts get to award an average of 50 as well?
> Before anyone says this is ridiculous, I would point out that here in
> Queensland, we operate our tertiary admissions system on this basis.
<snip commentary on Queensland tertiaray admissions system>
> Because the process is entirely mechanical, it requires only programming and no
> human intervention.
>
> Note. This system is still based on the assumption that a high rating indicates
> a set preferred by the user to a low rating. If people are reverse-rating sets
> (through error, malice, or a desire to deliberately distort the results in some
> way), then nothing can solve that problems, but the more extreme results will
> probably be scaled back to something more reasonable and the consequent impact
> on the overall rating of a set will be reduced.
I think this part could be overcome by correlating set raters. If we
established a basket of LEGO sets representing a certain range, we could
calibrate all raters against it. We do this for meat grading in Australia -
we have a system were meat graders are regularly correlated with a common
eye group to ensure that all graders are calling the same results for the
same appearance of carcases.
Raters would be able to rate so that there average rating was above 50,
whilst preserving the integrity of the whole rating system.
> So a person who submitted all 0-ratings would get scaled back to 50-ratings on
> each set (i.e. no impact on the overall means). Similarly any person who
> submitted only 100-ratings would get scaled back to 50-ratings as well. This
> means that people need to equally willing to rate the sets they like and the
> sets they dislike, as otherwise their ratings will be scaled up/down to produce
> a more "average" mean.
While this sounds plausible on the surface, it does assume that sets per
reviewer will rate an average 50 over time, which will not work as I've
outlined above.
> I've argued elsewhere on LUGnet that most recent
> releases get rated (good/bad as appropriate), but only the older *better* sets
> get rated (and hence rated favourably) as everyone forgets about the also-ran
> sets of the past and doesn't bother to rate them. IMHO this is why older sets
> tend to overrank newer sets
This is partly true, but I think nostalgia (or selective memory) comes in.
If a set comes from whatever you think the golden age is (pick any 3-5 year
period from 1975 to 1994) it's got to be worth another 10-20 points right
away, right?
Of course, I think we should move on from subjective rating and go to an
objective system which looks at measurables (number of pieces, colour
consistency, building time, presence of Timmies or Jar Jars, number of sets
sold, etc) and derives a rating which can be repeated again and again.
Cheers
Richie Dulin
|
|
| |
> I only rate sets that I have built, and tend to buy only sets that I know
> I'll like. So, most of my ratings are 70-100. If a system like this is
> implemented, does this mean I'll have to poorly rate some sets that I don't
> own if I want the sets I have rated highly to retain the full grade that I
> think they deserve?
The whole point of the statistics is to moderate people's over-enthusiasm, so
therefore, the system should not allow sets to "retain the full grade that *I*
think they deserve".
Assuming you are rating all the sets (and not just your favourites), then by
the very meaning of the word "average", some must be above average and some
must be below average in terms of your personal experiences with some subset of
Lego's products. It is not meaningful for you to say "based on all the Lego
sets which I have experience, that they are all above average" as ratings
70-100 would suggest. Below average does not mean "bad".
I agree that each of us is self-selecting in terms of the sets with which we
acquire experience. An obvious example of this is preference for certain
themes. However, if everyone rated all the sets with which they had experience,
then the fact that not all of us rate all sets should not matter. So long as
there are a sufficiently large number of ratings of a particular set, the
statistical adjustments I have described tell us what we really need to know
... the general impression of people who bothered to buy/build this set in the
first place.
To take a concrete example, I am not a big fan of Bionicles. I do not buy them
and consequently do not rate them. If I was forced to rate them, I would put
them all close to 0. However, it is undeniable that many people do buy them,
and presumably have some view on which Bionicles are better than others. So if
a particular Bionicle set gets an average rating of 90 from those who bother to
buy/build the wretched things, I think we can safely conclude its one of the
best Bionicle sets. I doubt that a Pirates fan is likely to say, hey that
Bionicle at rating 90 is a better set than the Black Seas Barracuda which rated
at say 89, and then buy the Bionicle rather than the BSB. However, if a Pirates
fan says to themselves, "gee, those Bionicles seem pretty popular, maybe I
should give them a go", then it is presumably useful to know which are the
better Bionicles and which are the duds, or to discover that Roboriders
generally outrate Bionicles and maybe they'd be better to try a Roborider if
they want to try something of that general genre.
To some extent, when one is gathering statistics, it is useful at the outset to
know what questions people are trying to answer. If the question is "what is
the all-time best Lego set?", then frankly it's a dumb question as clearly
people's preferences are strongly influenced along theme lines, as we saw in
the recent Lego Legends poll, where some people voted for the Battrax, not
because they necessarily loved it, but because it was the only space set and
there needed to be a Space Legend, meanwhile others complained that the castle
vote was split by the presence of 2 castles etc.
What is a more reasonable question to hope to answer is "given some sets that I
think might be meaningfully compared against one another, how do they
compare?". It is then up to people to decide if it is meaningful for them to
compare a Roborider with a Bionicle, a Castle set with a Harry Potter set, a
large Pirate set with a small Pirate set, etc etc. If it is completely obvious
to you that you would love/hate a particular set because of something obvious
like its theme, there's not a lot of point looking at the ratings.
Kerry
|
|
| |
In lugnet.general, Ross Crawford writes:
> In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> thingie which I snipped:
>
> My question here is "What are you using this information for which requires
> such analysis?".
Use the information? Well, I guess you could.
But it's information, it is a thing of beauty to be treasured and preserved
for the benefit of future treasurers and preservers. A thing to be analysed
to learn deeper truths.
But you *could* use it:
The information is scorecard to be held up for your favoured set, and
confirmation of your low opinion of the set you most loath.
The information lets you see into the mind of the mob, to make sure the mob
agrees with you, and to make sure you agree with the mob.
Look into the statistics, and they will set you free.
<pause to make sure everyone's nodding>
But not much use for purchasing sets, no.
Richie
|
|
| |
> I think a revising of the rating might be a good way to go, perhaps you can
> have a system where you rate different aspects of the set, say the quality /
> quanitity of the parts, the quality of the model, etc. Then average those to
> get a final ranking. It might be a good way to balance some of the numbers.
I'm absolutely in favour of separating the ratings for the bricks versus the
model/instructions. How often have you heard someone say "lousy model but great
parts"?
Consider 6087 Witch's Magic Manor which rates on LUGnet at the moment at 45
(from 23 votes), clearly not a big favourite. The general consensus is that the
model is awful but the parts are quite useful. Similarly 7151 Sith Infiltrator
is a not rated highly among Star Wars sets, but everyone loved them as Grey
Plates Pack. Indeed, I wonder how many 10030 Imperial Star Destroyers have been
ordered just to part them out for the grey in them, people are already
prophesising that the price of grey will fall on bricklink as a result of this.
Lets face it. Anyone who buys multiple copies of any set is almost always doing
it for the parts, not the model. And many of us do this, so it would make sense
to have ratings for sets purely for their parts as well as for their models. Of
course, we can probably determine a parts rating for sets already, by comparing
the number of sets owned divided by the number of owners. The bigger the
number, the better it is as a parts pack. For example, the database shows 299
copies of the Guarded Inn owned by 130 people, suggesting people each have
about 2.5 copies of this set. It sounds like its a parts pack, right! Whereas
the Train Ideas book has a ratio of about 1 -- surprise, surprise, there are no
parts in this! Also, the Battrax which rates highly on LUGnet has a ration of
about 1, suggesting it's not a set to buy for its parts but for its model. Then
take a look at the 3033 Big Blue Tub -- again, loads of people have multiple
copies, including someone who claims to have over 700.
Kerry
|
|
| |
Hello!
In lugnet.general, Ross Crawford writes:
> In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> thingie which I snipped
Well done :-)
I couldn't follow these analyses anyhow. First I was always bad at maths and
secondly I didn't understand half of the sentences without looking up every
other word....
> My question here is "What are you using this information for which requires
> such analysis?".
Absolutely correct!
I think this set rating is supposed to be just fun. You can make a short
statement to a set and give a simple rating. That's it. And that's enough.
LEGO is our hobby and not our profession. So we should not make such an
earnest and strict science out of it.
A hobby is a very subjective thing, so one of the most rediculous statements
in this thread I read was that we shouldn't rate sets for subjective reasons.
The given rating mode is suposed to be subjective.
Any objective criteria for a set to be rated can be found elsewhere, as for
the piece count (directly at most sets' database sheet) price (ib.),
containing bricks (peeron) and the picture for everone to visualize (LUGNET,
BrickShelf). So why should we seperate the simple little rating into several
rating categories that only could repeat what the objective figures already
tell?
Also I can find nothing that spoke against rating under sentimental
influence. Most of us are adults and know LEGO and specific LEGO sets from a
child. So it is only humanly that fond memories one has of a set lead to a
very positive personal rating. Why not? When I love a set since so many
years then there can be thousand objective resons to rate it low: I will
rate it high nevertheless. I claim this freedom.
And if I want to rate a set a "100" than I'm doing so. I wouldn't be pleased
if I was prohibited to give a set a "100" unless I rate some other sets low
just for personal rating-ballance reasons. To prevent all too enthusiastic
ratings the automatic softener is installed. So there is no need for this
cut-the-highest-and-lowest-rating-thing. (Well, I wasn't able to follow the
mathematical analyses that were given in this thread though...)
All things considered I find no reason to change the existing rating system
in favour of a more complicated and bureaucratic system that only could be
more restrictive without any evident benefits.
Just my 2¢
Bye
Jojo
|
|
| |
In lugnet.general, Ross Crawford writes:
> In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> thingie which I snipped:
>
> My question here is "What are you using this information for which requires
> such analysis?".
>
> I mean do you base your set purchases on these figures? And if so, how many
> times have you been burned by false "0" or "100" votes??
>
> Come on people! Just get back to building!
I'm not sure information impracticality is really an excuse for inaccurate
or misleading information. I think the point is that if we *can* get more
accurate information, why not do it? Clearly this topic interests some
people who actively want better analysis (or who are intregued by the
analystics challenge), and doesn't interest others. In light of that, maybe
this thread should just go to lugnet.admin.general.
DaveE
FUT admin.general
|
|
| |
In lugnet.general, Ross Crawford writes:
> In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> thingie which I snipped:
>
> My question here is "What are you using this information for which requires
> such analysis?".
>
> I mean do you base your set purchases on these figures? And if so, how many
> times have you been burned by false "0" or "100" votes??
I think if you criticise this, you fail to, fundamentally, get the point
behind being obsessed with LEGO(r). Further, this mechanism really lets the
obsessive/compulsive among us obsess about yet another thing, and by arguing
against it you're taking all the fun out of it.
> Come on people! Just get back to building!
I did. And boy are my thumbs sore (had to say it!)
|
|
| |
> > Before anyone says this is ridiculous, I would point out that here in
> > Queensland, we operate our tertiary admissions system on this basis.
> We do this for meat grading in Australia -
> we have a system were meat graders are regularly correlated with a common
> eye group to ensure that all graders are calling the same results for the
> same appearance of carcases.
Hmm, I see a lot of potential for expanding your organisation's efforts from
carcases to live animals, to be specific, tertiary applicants. Given the way
the parents moan about all the statistical moderation that takes place, I think
a lot of them would be far happier if we just brought meat graders into the
schools and applied objective criteria (height, weight, colour consistency,
building time -- parents to advise, being Timmies or Jar Jars, etc).
Kerry
|
|
| |
> But not much use for purchasing sets, no.
On the contrary, I have used it for precisely this purpose. Having come out of
a long Dark Age, many themes (or subthemes) came and went in those 30 odd
years. Since then, I read LUGnet and see all the people going on and on about
Forestmen or Western etc. So, I think to myself that maybe it would be fun to
look out for some sets from that theme on ebay in order to experience their
delights and educate myself accordingly. Since I am probably not going to
acquire many sets from these past themes, I figure I might as well go after the
best ones rather than the poor/average ones, and LUGnet's ratings are a useful
indicator for this purpose.
However, I am comparing the ratings within a theme, and not comparing them
between themes (which is pretty much bogus for purchasing purposes).
Kerry
|
|
| |
In lugnet.general, Kerry Raymond writes:
> > But not much use for purchasing sets, no.
>
> On the contrary, I have used it for precisely this purpose. Having come out of
> a long Dark Age, many themes (or subthemes) came and went in those 30 odd
> years. Since then, I read LUGnet and see all the people going on and on about
> Forestmen or Western etc. So, I think to myself that maybe it would be fun to
> look out for some sets from that theme on ebay in order to experience their
> delights and educate myself accordingly. Since I am probably not going to
> acquire many sets from these past themes, I figure I might as well go after the
> best ones rather than the poor/average ones, and LUGnet's ratings are a useful
> indicator for this purpose.
Interesting.
My first measure is the price/piece - whether on ebay or new. I have an
upper limit, beyond which I will not purchase (although I'm always prepared
to reassess my upper limit), and I have a working limit, beyond which I will
not purchase, unless the set contains particular parts/figs (eg I'll pay no
more than 15c/part for a used pirate set, but if it contains a blue coated
soldier or two (or certain other elements), then the purchasing limit will
be raised).
My second measure is the comments on LUGNET. They are far more reliable than
ratings. If someone leaves a comment like 'This set is rubbish because it's
a pirate set' then the comment can quickly be discarded... but if I look at
a rating of, say, 20, I don't know whether that's because the rater thinks
it's rubbish because it's a pirate set, or there is some other (more valid)
reason for rating it so low.
My third measure is the parts inventory.
I've made purchases on the first, first & second and all three and not been
disappointed. We're I have been disapointed is where I've skipped the first
measure.
Cheers
Richie
|
|
| |
In lugnet.general, Larry Pieniazek writes:
> In lugnet.general, Allan Bedford writes:
> > In lugnet.general, Ken Godawa writes:
> > > I'm curious on why so many of the popular sets receive a "zero" rating.
> >
> > Can you give an example of one othe "popular sets" to which you refer? I'm
> > just curious to know which ones we're discussing.
> >
> > > It just doesn't make any sense. Unless it's a valid reason, these ratings
> > > should be thrown out. Either these people are jealous or
> > > complete morons.
> >
> > Perhaps it is a valid reason. I don't think very many folks around here are
> > morons. No moron could survive the LUGNET signup process. ;)
> >
> > As well... what would your solution to this problem be? Would you like to
> > see the zero rating removed? Should every set, no matter how good or bad,
> > get a minimum 10 rating?
>
> I would say probably not... some sets really ARE duds.
>
> > Or, like in some judged sports, should we remove
> > the highest and the lowest rating in order to better average out the
> > remaining votes?
>
> That's not a bad idea. It works for Figure Skating (and we know how fair and
> impartial THAT sport is!) Grin.... No seriously, it IS a good idea.
>
> I would say this, though.... how abou this... (and it may be overengineering
> the solution.) My supposition is that someone has went through and given a
> lot of sets as 0. In fact wasn't there a case early in ratings where it was
> known that someone went through and rated every single Pirate set with 0?
>
> Maybe there should be a requirement that in order to be allowed to rate sets
> you need to rate some sets good as well as some bad? Require that your
> average rating across all sets you rated is at least 10 and no less than 90?
> That is, no going and rating EVERYTHING a 0 or EVERYTHING a 100. This will
> encourage people to put some thought into what they like and dislike. You
> get 5 0s (or 5 100s) and then you have to go find something you like (or
> dislike, as the case may be) or else no more ratings are accepted until your
> average is within range.
>
> Seriously, if you think every single set LEGO ever made is worth only 0,
> what are you doing here at LUGNET? I know Allan and I have different
> opinions but we both can find sets we'd rate higher than 0, and sets we'd
> rate lower than 100... I am sure of it.
>
> Probably not really a good idea but maybe it will get people thinking?
> (deciding it's not really a major problem is an OK outcome)
>
> ++Lar
To take the 5 0's / 5 100's a little further, how about only including
ratings in the calculation for those people who actually own the set...that
way those who don't rate it are considering it a pass-able set, those who do
rate it either own it and so they have a credible opinion of the set, or
don't own it, and so can't really rate it (after all, how do you rate a set
which you haven't personally built, and so had a chance to consider the
'functions' of the set and/or the parts used and how they are used.) [by all
means let everyone rate them, but only consider ratings from owners of the set.]
To add to this, you could also require the average rating by a person over
all sets they have to have to be within a certain range. You're giving your
opinion of a set against something, after all. Alternatively, calculate for
the user what their average rating for all sets is, and then downgrade the
weighting of their ratings accordingly.
ie. if all sets are rated at 100%, then they're considering all the sets to
have equal value, so downgrade to an equivalent 50% in calculations, or if
half are 100% and others unranked, then downgrade to 75% and 25%
Anyway, just my 2 cents,
Benjamin Whytcross
|
|
| |
In lugnet.general, Johannes Koehler writes:
> Hello!
>
> In lugnet.general, Ross Crawford writes:
> > In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> > thingie which I snipped
>
> Well done :-)
Same for your comment, Jojo!
I could not agree more (pling!).
I think it is worth to spend a thought why people do rate some sets with
extreme votes, but I think, once we have realized they do so, we can easily
count it in. Any manipulation of the votes filters information out. I want to
know that some people have extreme feelings about sets.
If I look for e.g. at the rating of 375 (I own it and have rated it therefor
with 80% // it's too yellow to be more than 80), then I take a look at the
whole disdtribution of votings and not at the excact number of average rating.
I do not care if it is 91.3 or 87.6. But I see lots ob big red bars in the
top. So people obviously do love the set.....
Back to building!
Ben
|
|
| |
In lugnet.general, Benjamin Whytcross writes:
> To take the 5 0's / 5 100's a little further, how about only including
> ratings in the calculation for those people who actually own the set...that
> way those who don't rate it are considering it a pass-able set, those who do
> rate it either own it and so they have a credible opinion of the set, or
> don't own it, and so can't really rate it (after all, how do you rate a set
> which you haven't personally built, and so had a chance to consider the
> 'functions' of the set and/or the parts used and how they are used.) [by all
> means let everyone rate them, but only consider ratings from owners of the set.]
That is just silly, I have built a 10022 by myself but I do not own the set
(I was at a friend's place). I know what it looks like, how it is
constructed, the pieces used, etc... I can make an educated decision about
it. Also, what you propose is forcing us to say if we own a set or not. Fine
then, if I want to give my opinion and have it count, I can lie and say I
own every set.
> To add to this, you could also require the average rating by a person over
> all sets they have to have to be within a certain range. You're giving your
> opinion of a set against something, after all. Alternatively, calculate for
> the user what their average rating for all sets is, and then downgrade the
> weighting of their ratings accordingly.
> ie. if all sets are rated at 100%, then they're considering all the sets to
> have equal value, so downgrade to an equivalent 50% in calculations, or if
> half are 100% and others unranked, then downgrade to 75% and 25%
This I disagree with this because I do not have a tendency to buy and rate
sets that in my opinion merit 10% or less (Galidor for example), but I would
buy a Galaxy Explorer, Alpha 1 Rocket Base, Deep Freeze Defender, etc... if
they came at the right price (all of which get a 100% from me).
>
> Anyway, just my 2 cents,
>
> Benjamin Whytcross
That is my two pennies,
Jude
|
|
| |
In lugnet.general, Richie Dulin writes:
> In lugnet.general, Kerry Raymond writes:
> > > But not much use for purchasing sets, no.
> >
> > On the contrary, I have used it for precisely this purpose. Having come out of
> > a long Dark Age, many themes (or subthemes) came and went in those 30 odd
> > years. Since then, I read LUGnet and see all the people going on and on about
> > Forestmen or Western etc. So, I think to myself that maybe it would be fun to
> > look out for some sets from that theme on ebay in order to experience their
> > delights and educate myself accordingly. Since I am probably not going to
> > acquire many sets from these past themes, I figure I might as well go after the
> > best ones rather than the poor/average ones, and LUGnet's ratings are a useful
> > indicator for this purpose.
>
> Interesting.
>
> My first measure is the price/piece - whether on ebay or new. I have an
> upper limit, beyond which I will not purchase (although I'm always prepared
> to reassess my upper limit), and I have a working limit, beyond which I will
> not purchase, unless the set contains particular parts/figs (eg I'll pay no
> more than 15c/part for a used pirate set, but if it contains a blue coated
> soldier or two (or certain other elements), then the purchasing limit will
> be raised).
I always figured that no matter how good a set is, if I could by three other
poorer sets for the same price and could come up with something much
grander, then why the heck buy the overpriced set?
And getting blue-coated soldiers (red are harder, actually) for the right
price was simply a matter of patience.
>
> My second measure is the comments on LUGNET. They are far more reliable than
> ratings. If someone leaves a comment like 'This set is rubbish because it's
> a pirate set' then the comment can quickly be discarded... but if I look at
> a rating of, say, 20, I don't know whether that's because the rater thinks
> it's rubbish because it's a pirate set, or there is some other (more valid)
> reason for rating it so low.
This reply is really to reinforce this excellent observation. If you are
curious about a set, look at the ratings by all means, but read through the
comments that people have appended. They can tell you more about a set,
it's strengths and weaknesses. You'll have a much better idea if that is
the set for you than if you consider only simple raw numbers.
Bruce
|
|
| |
In lugnet.general, Bruce Schlickbernd writes:
> In lugnet.general, Richie Dulin writes:
> > My first measure is the price/piece - whether on ebay or new. I have an
> > upper limit, beyond which I will not purchase (although I'm always prepared
> > to reassess my upper limit), and I have a working limit, beyond which I will
> > not purchase, unless the set contains particular parts/figs (eg I'll pay no
> > more than 15c/part for a used pirate set, but if it contains a blue coated
> > soldier or two (or certain other elements), then the purchasing limit will
> > be raised).
>
> I always figured that no matter how good a set is, if I could by three other
> poorer sets for the same price and could come up with something much
> grander, then why the heck buy the overpriced set?
>
> And getting blue-coated soldiers (red are harder, actually) for the right
> price was simply a matter of patience.
Indeed. Patience can save a lot of money.
(Also, be aware that my 15c/part limit is Australian cents. 15 US cents way
too much:-))
Richie
|
|
| |
In lugnet.general, Kerry Raymond writes:
> > But not much use for purchasing sets, no.
>
> On the contrary, I have used it for precisely this purpose. Having come out of
> a long Dark Age, many themes (or subthemes) came and went in those 30 odd
> years. Since then, I read LUGnet and see all the people going on and on about
> Forestmen or Western etc. So, I think to myself that maybe it would be fun to
> look out for some sets from that theme on ebay in order to experience their
> delights and educate myself accordingly. Since I am probably not going to
> acquire many sets from these past themes, I figure I might as well go after the
> best ones rather than the poor/average ones, and LUGnet's ratings are a useful
> indicator for this purpose.
And how many times have you made such a purchase, without taking into
account 0/100 votes, and been disappointed?
ROSCO
|
|
| |
In lugnet.general, Larry Pieniazek writes:
> In lugnet.general, Ross Crawford writes:
> > In lugnet.general, Kerry Raymond writes a really neat statistical analysis
> > thingie which I snipped:
> >
> > My question here is "What are you using this information for which requires
> > such analysis?".
> >
> > I mean do you base your set purchases on these figures? And if so, how many
> > times have you been burned by false "0" or "100" votes??
>
> I think if you criticise this, you fail to, fundamentally, get the point
> behind being obsessed with LEGO(r). Further, this mechanism really lets the
> obsessive/compulsive among us obsess about yet another thing, and by arguing
> against it you're taking all the fun out of it.
On the contrary! I encourage those who are so inclined to study these
figures and do what they want with them! I just dont think it's necessary
for such analysis to be "built in" to Lugnet. Unless, of course, Todd finds
it interesting and *wants* to implement it 8?)
> > Come on people! Just get back to building!
>
> I did. And boy are my thumbs sore (had to say it!)
Just use your other fingers a bit! I'm mean I've heard the expression "all
thumbs" but I think you've taken it a bit far 8?)
ROSCO
|
|
|
