| |
Hey Todd,
I've been avoiding a direct and complete opinion about the rating system till
now, mainly because I wasn't sure of my stand on it. But now I know where I
stand... here're my answers:
> The first, original purpose for having ratings was to be able to lay the
> foundation for the later creation of variety of "what's hot" or "top X of
> group Y" listings for quick browsing -- something akin to the current
> Spotlight pages, only fully automated, instantly updating, and much more
> representative of collective opinion.
This is a good idea and I'll be glad to ee it implemented.
> The second original purpose was to
> lay the foundation for so-called "collaborative filtering" possibilities --
> the server learns (could learn) what types of things you prefer to read,
> and gives (could give) higher priority to you personally for messages rated
> higher by people with similar interests.
This, IMO, would work if (and only if) the rating were *not* based on numeric
rating, because the numbers can be perceived in oh-so-many ways. It would work
well, IMO, if there was a rating system based on written-out choices, for
example:
-this post is in the wrong NG (ie, off-topic)
-this post is OK, but not very useful
-this post is very informative
-this post features great MOCs/web sites/[etc]
You get the picture.
This could be very useful once (if?) the RSS/channels are implemented. I could
say, for example, that I want to see informative posts only; while Jeff (just
throwing around names) will ask to see MOC posts as well as informative ones;
and Eric will want to see all posts regardless... that kind of thing.
> These two main purposes become
> increasingly relevant as message traffic increases.
Right; but pure numbers aren't really helping. Categorizing posts
by "usefulness" is downright inappropriate and unhelpful. Because what's
useful and important to someone (e.g. info about a cool new mindstorms set
that is sighted in stores), will be useless to me and vice versa.
> It was never a purpose of the ratings system to make anyone ever feel bad
> or unwanted or unwelcome. It's core purpose is simply to highlight "neat or
> noteworthy stuff"
That's why I think that written-out statements are more useful than numbers.
Like Jeremy and others mentioned, a numeric scoring without a reason does not
help the person posting to realize what he/she has done wrong (or done
excellently well).
> but not to downgrade "un-neat or un-noteworthy stuff" or
> regular "fluff" (which there's nothing wrong with).
Right! "Fluff" is one of the things I like in lugnet, we are here day-in and
day-out; and there aren't Brad Justus posts every day, or MTT sightings every
week. Lugnet contains of a whole lot of fluff, and I personally like to read
it-- and see what's happening every day, regardless if it's highly "useful".
> Going with a scale 0 to 100, in retrospect, hasn't been any
> better from an overall morale point of view than if a scale -100 to +100 had
> been used.
Sorrowfully enough, no. :-(
Numbers are numbers, no matter what they are.
> 1. How would you feel (better or worse) if the numeric values of the ratings
> were not displayed to you unless you specifically requested (via some simple
> setting) that they be displayed to you?
Neither better nor worse.
> 2. How would you feel (better or worse) if the numeric values of the ratings
> were not displayed ever to anyone but collected and used by the server only
> for internal calculations, hotlist generation, and personal recommendations
> to you?
Better. I would like that.
> 3. How would you feel (better or worse) if the ratings were not even
> collected and collated in the first place? (i.e. the destruction of the
> feature altogether)
Neither better nor worse. I think it is a good idea but it needs to be re-
thought.
> 4. Have you ever felt victimized by the rating system? Have you posted
> something which has obtained a low rating and felt uncomfortable or unhappy
> about yourself or about LUGNET because of the low rating? How often?
Yes, to some extent. Not very often, though.
> 5. Have you ever felt victimized indirectly by seeing someone else's post
> get a high rating? How often?
I can't recall such a situation.
> 6. Do you feel that the article rating system makes it easier for you or
> harder for you to share your ideas? And does this bother you?
Neither, and no.
> 7. How does your initial reaction to the announcement of the article rating
> system compare to your current opinion of it?
Originally, I thought it was a great idea. After seeing it in action, I still
think it's a good idea; but it has to be reworked.
> 8. Do you feel that it is too early, too late, or the right time to address
> these issues?
Right time. Perhaps a tad too late, but it doesn't matter.
> 9. What other areas (besides news articles) can you imagine that a
> collaborative ratings system would be most helpful to you?
> LEGO sets?
Yes; but again, not pure numerical value.
> Websites? Individual web pages?
No way. Much, MUCH more potential for insults and bad feelings all around.
-Shiri
|
|
| |
Shiri Dori <shirid@hotmail.com> wrote:
> -this post is in the wrong NG (ie, off-topic)
> -this post is OK, but not very useful
> -this post is very informative
> -this post features great MOCs/web sites/[etc] • [snip]
> Right; but pure numbers aren't really helping. Categorizing posts
> by "usefulness" is downright inappropriate and unhelpful. Because what's
> useful and important to someone (e.g. info about a cool new mindstorms set
> that is sighted in stores), will be useless to me and vice versa.
> That's why I think that written-out statements are more useful than numbers.
Good point -- this is kind of what I was trying to say, although I'd taken
it to a further extreme. (Even less choices.)
--
Matthew Miller ---> mattdm@mattdm.org
Quotes 'R' Us ---> http://quotes-r-us.org/
Boston University Linux ---> http://linux.bu.edu/
|
|
| |
In lugnet.admin.general, Shiri Dori writes:
> Right; but pure numbers aren't really helping. Categorizing posts
> by "usefulness" is downright inappropriate and unhelpful. Because what's
> useful and important to someone (e.g. info about a cool new mindstorms set
> that is sighted in stores), will be useless to me and vice versa.
That's what the averaging effect is for -- to smooth that out. If the system
also could learn what you liked, you might find that helpful. (That's a long
way down the road, though.)
> [...]
> Right! "Fluff" is one of the things I like in lugnet, we are here day-in and
> day-out; and there aren't Brad Justus posts every day, or MTT sightings every
> week. Lugnet contains of a whole lot of fluff, and I personally like to read
> it-- and see what's happening every day, regardless if it's highly "useful".
Will you still feel that way when there are 4x the number of messages daily?
A year and a half ago, there were only 80-100 messages a day (on average).
Now there are 350-400 a day (on average). At some point, the fluff becomes
too much. And you may already have an unusually high liking or tolerance for
that sort of thing. Not everyone out there so much time to read everything.
Not trying to sound like a contrarian, just pointing out another POV.
--Todd
|
|
| |
Todd Lehman <lehman@javanet.com> wrote:
> > Right; but pure numbers aren't really helping. Categorizing posts
> > by "usefulness" is downright inappropriate and unhelpful. Because what's
> That's what the averaging effect is for -- to smooth that out. If the system
> also could learn what you liked, you might find that helpful. (That's a long
> way down the road, though.)
Smoothing what out, though? How does the system distinguish between "0: I
like posts about robots, but not in .castle" and "0: not interesting to me",
or "60: kinda funny if you're in the right mood" and "60: contains some
useful information but could be more complete"?
Also, I'm _very_ skeptical of the "match what you like" concept. It sounds
neat in practice, but I've never seen it implemented well.
homr/ringo/Firefly/bignote/launch/whateverthey'recallingthemselvestoday did
an ok job, but you'd have to do some serious language parsing/comprehension
stuff to make it work with news posts, even in such a narrow subject as
Lego.
--
Matthew Miller ---> mattdm@mattdm.org
Quotes 'R' Us ---> http://quotes-r-us.org/
Boston University Linux ---> http://linux.bu.edu/
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> In lugnet.admin.general, Shiri Dori writes:
> > Right; but pure numbers aren't really helping. Categorizing posts
> > by "usefulness" is downright inappropriate and unhelpful. Because what's
> > useful and important to someone (e.g. info about a cool new mindstorms set
> > that is sighted in stores), will be useless to me and vice versa.
>
> That's what the averaging effect is for -- to smooth that out.
Well- yeah. But at the current amounts of rating (most posts get no more than
two ratings) the averaging effect doesn't smooth much out.
I totally forgot to mention in my long post that I actually stopped regarding
a rating of a post as a factor of whether I'm going to read it or not. I just
ignore the rating, because I've noticed that the rating had nothing to do with
my perspective on the post whatsoever.
> If the system
> also could learn what you liked, you might find that helpful. (That's a long
> way down the road, though.)
Yes... the thought occured to me that perhaps, while posting, one could
(optionally) check off any number of boxes describing the post
(e.g. "MOC", "market", "set opinion"). Then each reader could specify what
things he would and would not like to see.
> > [...]
> > Right! "Fluff" is one of the things I like in lugnet, we are here day-in and
> > day-out; and there aren't Brad Justus posts every day, or MTT sightings
> > every
> > week. Lugnet contains of a whole lot of fluff, and I personally like to read
> > it-- and see what's happening every day, regardless if it's highly "useful".
>
> Will you still feel that way when there are 4x the number of messages daily?
> A year and a half ago, there were only 80-100 messages a day (on average).
> Now there are 350-400 a day (on average). At some point, the fluff becomes
> too much. And you may already have an unusually high liking or tolerance for
> that sort of thing. Not everyone out there so much time to read everything.
You're right. That hadn't occured to me. When I think of it, I spend more time
on lugnet than the average fan (1) and I check it every 10 minutes or so (more
often if I'm really bored). I guess I *do* tolerate fluff more than other
people, because lugnet for me is not only a mean for sharing my hobby (lego)
with other fans, but also a source of communication with people. (1)
I still think that word strings will provide better feedback and guidance (as
to what to read/what not) than numbers, though. But if the RSS ends up having
number ratings, I'd set my prefs to "show any message regardless of the
rating" because I, personally, like to read it all.
-Shiri
(1) What can I say? I'm in a bad social situation right now, my only friends
live 7 time-zones away from me and I mostly communicate with people (2) via
the net. I also have a lot of free time on my hands because I don't hang out
with my friends (it's a bit hard considering the geographical differences ;-).
(2) both my friends from Israel, and AFOLs.
|
|
| |
In lugnet.admin.general, Matthew Miller writes:
> Todd Lehman <lehman@javanet.com> wrote:
> > That's what the averaging effect is for -- to smooth that out. If the
> > system also could learn what you liked, you might find that helpful.
> > (That's a long way down the road, though.)
>
> Smoothing what out, though? How does the system distinguish between "0: I
> like posts about robots, but not in .castle" and "0: not interesting to me",
> or "60: kinda funny if you're in the right mood" and "60: contains some
> useful information but could be more complete"?
It can't (and doesn't actually need to) distinguish that so greatly -- the
bottom line (to it) would be that you disfavor posts about robots in castle
and things that are kinda funny or contain some useful info.
> Also, I'm _very_ skeptical of the "match what you like" concept. It sounds
> neat in practice, but I've never seen it implemented well.
> homr/ringo/Firefly/bignote/launch/whateverthey'recallingthemselvestoday did
> an ok job, but you'd have to do some serious language parsing/comprehension
> stuff to make it work with news posts, even in such a narrow subject as
> Lego.
I'm very skeptical about that form of collaborative filtering as well. But
there's a completely other form of it which is purely statistical correlation
based. It looks only at how your responses correlate to the responses of
others, without knowing (or having to know) anything at all about the content.
Then it makes a prediction about how you would feel about some brand new data
point based on how other people before you felt about that new data point.
Of course, it's a scheme which works better for things like record albums or
books or LEGO sets than time-sensitive things like news articles. If you're
always the first one to rate something, it couldn't help you out, but if
you're always the last one to rate something, then it could. That's the
theory, anyway. It works basically on the premise that everyone tends to
have opinions which can be approximated by a linear combination of other some
set of other people having multiple partially overlapping domains of input.
--Todd
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> In lugnet.admin.general, Matthew Miller writes:
> > Todd Lehman <lehman@javanet.com> wrote:
> > > That's what the averaging effect is for -- to smooth that out. If the
> > > system also could learn what you liked, you might find that helpful.
> > > (That's a long way down the road, though.)
> >
> > Smoothing what out, though? How does the system distinguish between "0: I
> > like posts about robots, but not in .castle" and "0: not interesting to me",
> > or "60: kinda funny if you're in the right mood" and "60: contains some
> > useful information but could be more complete"?
>
> It can't (and doesn't actually need to) distinguish that so greatly -- the
> bottom line (to it) would be that you disfavor posts about robots in castle
> and things that are kinda funny or contain some useful info.
Whoops -- :) -- I meant to say, "...that you disfavor posts about robots in
castle and _favor_ (somewhat) things that are kinda funny or contain some
useful info." But actually it would just look at the statistical correlation
between your rating and other people's ratings, etc.
--Todd
|
|
|
