| |
In lugnet.admin.general, Todd Lehman writes:
> [...]
> Clearly, these are very strong feelings being expressed by people. How many
> others feel this way? What would you like to see happen? Post your thoughts
> as a reply to this message (or reply privately if you prefer not to post your
> thoughts publicly).
> [...]
First, thanks to everyone who has taken the time to put their thoughts into
words, both publicly and privately. A clearer picture is beginning to emerge.
We'd like to try scaling things back (i.e., simplifying things) just a little
bit first before trying out any drastic fixes. Three things seem very clear:
(1) the current system of publicly showing composite numbers in the range 0 to
100 with a default of 50 is causing more overall harm than overall good, and
(2) changing the method of the scale might help, and (3) eliminating the
visible numeric scores altogether would definitely help (help relieve hurt
feelings, that is).
Ironically, the problem isn't the collection of data; the problem is
presenting back too much data -- too easy to see details or variations and
not know what they mean.
We're going to try a simple change to the way the scores are presented -- and
we'd like to keep this in place for at least 24 hours to collect feedback.
Maybe this improves things a bit, maybe it doesn't. Maybe it solves the
problems altogther, maybe not. We apologize in advance for the bumpy ride,
since the changes will be visible ones.
Changepoint 1
-------------
Remapped the range 0 to 100 to the range 0 to 5 and changed the default score
from 50/100 to 1/5 so that scores tend generally to climb rather than to fall.
Articles can still be marked down (toward zero), but seeing scores turn
downward would now be a rare rather than a common occurrence.
Thus, here's an old/new conversion table (just for illustration and
understanding -- not important to memorize):
OLD <==> NEW
--- ---
100 5
90
80 4
70
60 3
--> 50 <------------- old default
40 2
30
20 --> 1 <--- new default
10
0 0
And a comparison of the "marked down" and "marked up" ranges (which, again,
are a figment of the imagination, but it's effectively impossible to convince
people of that...so we have to live with that impression and compensate
for it):
OLD
-----
100 | ^
90 | |
80 | "marked up" (perceived as "good")
70 | |
60 | |
50 |<--starting point (default score)
40 | |
30 | |
20 | "marked down" (perceived as "bad")
10 | |
0 | V
NEW
-----
5 | ^
4 | |
3 | "marked up" (perceived as "good")
2 | |
1 |<--starting point (default score)
0 | "marked down" (perceived as "bad")
This was a very simple two-line code change on the server...the ratings engine
was designed to have its output mapped to other ranges besides 0-100, and the
default scores of 50 are/were never stored in the database (it was always
added on-the-fly at display-time) so changing this from 50 to 1 was trivial.
When casting input, the number of choices is also now decreased from 11
choices (corresponding before with 0,10,20,30,40,50,60,70,80,90,100) now to
6 choices (corresponding now with 0,1,2,3,4,5).
Changepoint 2
-------------
Instead of displaying ratings as numbers, display them now as a string of "+"
symbols (and a "-" symbol for 0)... i.e.:
SYMBOL NEW OLD
++++ 5 83-100
+++ 4 67-82
++ 3 50-66
+ 2 33-49
1 17-32
- 0 0-16
This was also a simple code change in a single location.
Changepoint 3
-------------
Simplifed the rating display in the "Brief" article view mode (the one which
shows an abstract or snippet of it, like what you see on the homepage):
OLD: Unrated: 50
NEW: (nothing)
OLD: Rated: 40 by 1
NEW: +
OLD: Rated: 63 by 5
NEW: ++
OLD: Rated: 96 by 14
NEW: ++++
Also simplified the rating display in the "All" (full) article view mode so
that the rating displays along with the other headers further to the left:
OLD: Rating: 50 / By: _0_
NEW: (nothing)
OLD: Rating: 40 / By: _1_
NEW: Rating: (_histogram_)
OLD: Rating: 40 / By: _1_
NEW: Rating: + (_histogram_)
OLD: Rating: 96 / By: _14_
NEW: Rating: ++++ (_histogram_)
These were also simple code changes each in a single location.
Changepoint 4
-------------
In a couple of places, flipped the direction of the symbol used for showing
articles you've rated from >> to << (only cosmetic to support the above).
Final notes
-----------
These changes are not switched "on" yet, but look for them about 10-15 minutes
after this "heads-up" message appears.
And the /news/rating-graph.cgi displays (the histograms) still show the raw
input table 0 to 100... Something to change later if this is worth pursuing.
--Todd & Suz
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> [...]
> Instead of displaying ratings as numbers, display them now as a string of "+"
> symbols (and a "-" symbol for 0)... i.e.:
>
> SYMBOL NEW OLD
> ++++ 5 83-100
> +++ 4 67-82
> ++ 3 50-66
> + 2 33-49
> 1 17-32
> - 0 0-16
> [...]
The minus signs still look damaging, and I don't think it's good that a
default of 1 (20 internally) can be lowered to 0 by a single person casting
a 0. (20+0)/2 = 10, which becomes 0, which becomes "-".
Might tweak this tomorrow, either to...
SYMBOL NEW OLD
+++++ 5 83-100
++++ 4 67-82
+++ 3 50-66
++ 2 33-49
+ 1 17-32
0 0-16
...or to...
SYMBOL NEW OLD
++++ 4 80-100
+++ 3 60-79
++ 2 40-59
+ 1 20-39
0 0-19
...and in either case with 0 as the default rather than 1 (or 20 internally).
0 (blank) just seems so much easier to start with and explain than partway
up the range. This way, also, nothing could _ever_ get a negative rating --
The worst that could happen to something is that it didn't have any symbol
shown next to it.
--Todd
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> SYMBOL NEW OLD
> ++++ 4 80-100
> +++ 3 60-79
> ++ 2 40-59
> + 1 20-39
> 0 0-19
>
> ...and in either case with 0 as the default rather than 1 (or 20 internally).
> 0 (blank) just seems so much easier to start with and explain than partway
> up the range. This way, also, nothing could _ever_ get a negative rating --
> The worst that could happen to something is that it didn't have any symbol
> shown next to it.
This looks like a much better rating system. And I agree that any given
article shouldn't be given a negative rating without further explanation. If
someone continuously receives negative ratings for their posts, they might
take it as their posts are not wanted and might cause them to stop posting at
all. In your old rating system, I assume anything less than 50 is a negative
meaning there is something wrong with the post. If there is anything wrong,
please explain further by leaving a reply instead of leaving a rating.
Leaving a low rating won't explain to the original poster what is wrong with
the article. Since the old rating was done anonymously, people might leave a
low rating for any reason even just because they don't like the original
poster.
On a side note I noticed that before the rating system was implemented I used
to scroll through at least 2-3 pages (by clicking on "show 100 more") of new
posts every morning and after the rating system was implemented, the number of
posts died down to a little more than 1 page.
Way back I thought about rating all links that I have on my page (almost 500
of them) but quickly discarded the idea since giving a lower rating than the
maximum to anybody will be like sending them a message that their "product" is
not good enough. This rating system would have done more harm than good.
And finally, the whole reason for having a rating system is to show the most
interesting articles? I thought that the purpose of having a Spotlight
section was to do this.
D.
|
|
| |
> Ironically, the problem isn't the collection of data; the problem is
> presenting back too much data -- too easy to see details or variations and
> not know what they mean.
Hmm. As far as I can see, the main problem is neither the collection _or_ the
representation of the data but the data itself. When members vote, they are
voting on different things eg:
Do I agree with that?
Was that worded well?
Was it interesting?
Is it of value to others?
etc?
Changing the way the data is represented, does not make the data any better.
If the rating system is to stay, I think the data would be a little more
meaningful if we knew how many people have read the message - i.e. if message
is has 4 votes and 4 readers, the rating may be meaningful... but if it has had
100 readers and only 4 votes that's not so good.
Lastly, members should not be able to rate their own posts (I assume they can
right now - but I'm not 100% sure).
Scott A
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> we'd like to keep this in place for at least 24 hours to collect feedback.
Well, since you asked, I thought I'd leave mine.
> Changepoint 1
> Remapped the range 0 to 100 to the range 0 to 5...
This change could be for the better. I was fine with the 100 scale (I think
the 5 scale gives less acurate results) but it's good if it makes rating easier
for people. Hopefully this will cause more people to rate. Under the old
0-100 system, I constantly was counting dots to see where "70" or "60" was.
under any system, I think it might be nice to somehow label each selection.
> and changed the default score from 50/100 to 1/5 so
This is much better than setting the default to zero as you seem to suggest
in http://www.lugnet.com/admin/general/?n=6288 I see a huge difference between
"not yet rated" and "rated down" and think that for ratings to work that we
need both.
> Changepoint 2
> ...display them now as a string of "+" symbols (and a "-" symbol for 0)
I don't mind going from 0-100 to 0-5, but I would MUCH rather see the number
than a number of symbols. It is much easier to quickly assess the rating of a
message when I don't have to count the symbols (and they are too close together
for me to be able to quickly tell the number by just looking). I wonder if
others feel the same about this change.
> Changepoint 3
> Simplifed the rating display in the "Brief" article view mode
As long as unrated messages are the only ones that don't have their
rating shown, this is fine. You mentioned changing the "-" to showing no
rating, and that makes it difficult to tell the difference between bad and
unrated messages. I prefer to see the "by 7" in all of the ratings views so
that I can tell how many have rated, but I would live on without that.
> Changepoint 4
> In a couple of places, flipped the direction of the symbol used for showing
> articles you've rated from >> to << (only cosmetic to support the above).
I didn't notice and I doubt that this will be the point that you get the most
complaints about. :)
Ben Roller
|
|
| |
Todd Lehman <lehman@javanet.com> wrote:
> SYMBOL NEW OLD
> ++++ 4 80-100
> +++ 3 60-79
> ++ 2 40-59
> + 1 20-39
> 0 0-19
How about
+++ 4
++ 3
+ 2
1
0
?
--
Matthew Miller ---> mattdm@mattdm.org
Quotes 'R' Us ---> http://quotes-r-us.org/
Boston University Linux ---> http://linux.bu.edu/
|
|
| |
In lugnet.admin.general, Dan Jezek writes:
> [...]
> And finally, the whole reason for having a rating system is to show the most
> interesting articles?
That's not the whole reason, no. It's one important reason, though.
> I thought that the purpose of having a Spotlight section was to do this.
That's one purpose of it, although the Spotlight section almost always ignores
auctions because it's more news- and MOC-focused. Before the ratings, it was
also produced from only a single person's input (in 99% of the cases), which
is a downside both in overhead and accuracy.
--Todd
|
|
| |
In lugnet.admin.general, Matthew Miller writes:
> Todd Lehman <lehman@javanet.com> wrote:
> > SYMBOL NEW OLD
> > ++++ 4 80-100
> > +++ 3 60-79
> > ++ 2 40-59
> > + 1 20-39
> > 0 0-19
>
> How about
>
> +++ 4
> ++ 3
> + 2
> 1
> 0
That's probably even better.
Gonna try out a related thing first -- changing the "+" symbols to "!" symbols.
The "!" symbol is a lot skinnier than "+", so it saves precious space, and the
count of symbols isn't really as important anyway as the overall bottom-line
visual draw. The "+" symbol (chosen over "*" for its higher legibility in
default Helvetica) still somewhat connotates "plus" or "better" even without
an accompanying "-" sign. (Do you get that feeling as well?) The idea behind
the exclamation point "!" is shouting "hey, look at this!!"
Changing the input-collection text from:
How would you rate this message? Low o o o o o o High o (No comment)
to:
Would you recommend this article to others? Yes! o o o o Yes!!!! o No
ought to alleviate ambiguity.
I always had reservations about using the word "rating" in context with
articles. Slashdot uses the word "score" but that also gives the feeling that
someone is "keeping track" or "keeping score." Other word options were
"voting" and "opinions" but according to their dictionary definitions, those
didn't quite match. (I don't recall how much in-depth the word choice was
discussed in the initial thread, but probably not enough.)
Migrating this away from the idea of "rating" and toward the idea of
"recommending" (that's what the ratings are supposed to do, in the final
analysis) seems like a wise idea.
--Todd
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> Gonna try out a related thing first -- changing the "+" symbols to "!" • symbols.
> The "!" symbol is a lot skinnier than "+", so it saves precious space, and the
> count of symbols isn't really as important anyway as the overall bottom-line
> visual draw. The "+" symbol (chosen over "*" for its higher legibility in
> default Helvetica) still somewhat connotates "plus" or "better" even without
> an accompanying "-" sign. (Do you get that feeling as well?)
No, but I see how other people might.
> The idea behind
> the exclamation point "!" is shouting "hey, look at this!!"
>
> Changing the input-collection text from:
>
> How would you rate this message? Low o o o o o o High o (No comment)
>
> to:
>
> Would you recommend this article to others? Yes! o o o o Yes!!!! o No
>
> ought to alleviate ambiguity.
<snip>
Yes, definitely yes.
If your post isn't recommended, it doesn't mean anything... just that it's not
recommended. No hard feelings (or at least less hard than "low ratings").
-Shiri
|
|
| |
In lugnet.admin.general, Shiri Dori writes:
> Yes, definitely yes.
> If your post isn't recommended, it doesn't mean anything... just that it's
> not recommended. No hard feelings (or at least less hard than "low ratings").
Also thinking of making a rating of 0 (lowest) not count -- i.e., be exactly
the same as inputting no opinion at all. In other words, there would be a
way not to recommend to read something (naturally) but no way to recommend
not to read something. This way, _only_ positive things could be "said" via
the input. Negative input would be the same as no input. This would
effectively discard the ability to penalize someone for what they've posted.
It would be a loss of useful functionality, but overall, we're probably better
off if that aspect of functionality doesn't exist, since (a) few people are
relatively emotionless and (b) this is just a hobby, after all. We're here to
have fun, to do positive things, not to begrudge each other accidentally or
on purpose.
--Todd
|
|
| |
Todd Lehman <lehman@javanet.com> wrote:
> default Helvetica) still somewhat connotates "plus" or "better" even without
> an accompanying "-" sign. (Do you get that feeling as well?) The idea behind
> the exclamation point "!" is shouting "hey, look at this!!"
Yeah, I agree that "+" has connotations of "better". But I think the "!"
seems to have a "Warning" feeling to it, especially in red. (It's a typical
icon in warning error message dialog boxes, for example.) That doesn't mean
it's necessarily a bad choice -- maybe it's just me. If it weren't for the
space issue, I'd pick "*".
> Changing the input-collection text from:
> Would you recommend this article to others? Yes! o o o o Yes!!!! o No
*grin* Other than reminding me of those awful Herbal Essences shampoo
commercials, looks good.
> Migrating this away from the idea of "rating" and toward the idea of
> "recommending" (that's what the ratings are supposed to do, in the final
> analysis) seems like a wise idea.
Agreed.
--
Matthew Miller ---> mattdm@mattdm.org
Quotes 'R' Us ---> http://quotes-r-us.org/
Boston University Linux ---> http://linux.bu.edu/
|
|
| |
In lugnet.admin.general, Matthew Miller writes:
> Yeah, I agree that "+" has connotations of "better". But I think the "!"
> seems to have a "Warning" feeling to it, especially in red. (It's a typical
> icon in warning error message dialog boxes, for example.) That doesn't mean
> it's necessarily a bad choice -- maybe it's just me. If it weren't for the
> space issue, I'd pick "*".
Welp, space isn't really _that_ much of an issue compared to the symbol itself.
All things being equal, a thinner character is better than a big fat character
like "+". Let's see what people think about "!" after seeing it for a little
while...I seem to remember visiting a discussion group site once which used a
small orange "!" image to mark something that was important -- it didn't seem
to me like a "don't read this!" but obviously a "read this!".
(Anyone else reading this -- if you think "!" is a terrible choice or have
another suggestion, please let us know! We can certainly try out "*" if "!"
sends the wrong impression.)
--Todd
|
|
| |
Todd Lehman <lehman@javanet.com> wrote:
> like "+". Let's see what people think about "!" after seeing it for a little
> while...I seem to remember visiting a discussion group site once which used a
> small orange "!" image to mark something that was important -- it didn't seem
> to me like a "don't read this!" but obviously a "read this!".
For what it's worth, slrn uses ! to mark highly-scored (via my own score
file) articles. So I'm certainly used to it. (I thought it kind of odd of
slrn too, but no one asked me for feedback there....)
--
Matthew Miller ---> mattdm@mattdm.org
Quotes 'R' Us ---> http://quotes-r-us.org/
Boston University Linux ---> http://linux.bu.edu/
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> This way, _only_ positive things could be "said" via
> the input. Negative input would be the same as no input.
"If you can't say anything nice, don't say anything at all". Gee thanks for
reminding us Mom...er..Todd. ;)
Ben Roller
|
|
| |
In lugnet.admin.general, Todd Lehman writes:
> (Anyone else reading this -- if you think "!" is a terrible choice or have
> another suggestion, please let us know! We can certainly try out "*" if "!"
> sends the wrong impression.)
I like + over *, but unless the !'s font is darker, it's worthless to me. I
have fairly good eyes, but I can barely even see that the article is rated.
Again, I introduce you to Mr. Dead Horse...Why not use 1-5 instead of !-!!!!!,
*-*****, +-+++++, or anything else. Yes, I know why not, but I like the
numbers better.
Ben Roller
|
|
| |
In lugnet.admin.general, Ben Roller writes:
> In lugnet.admin.general, Todd Lehman writes:
>
> > (Anyone else reading this -- if you think "!" is a terrible choice or have
> > another suggestion, please let us know! We can certainly try out "*" if "!"
> > sends the wrong impression.)
>
> I like + over *, but unless the !'s font is darker, it's worthless to me. I
> have fairly good eyes, but I can barely even see that the article is rated.
Ah, but that's the beauty of it! The font will be darker when the rating (or
recommendation) is higher; if the rating is low, you shouldn't even notice it.
A high-rated post will attract attention to itself; which is like a
recommendation-- the people recommending it (ie rating at higher) are helping
you notice it, etc.
I think it's great that way!
-Shiri
|
|
|
