To LUGNET HomepageTo LUGNET News HomepageTo LUGNET Guide Homepage
 Help on Searching
 
Post new message to lugnet.announceOpen lugnet.announce in your NNTP NewsreaderTo LUGNET News Traffic PageSign In (Members)
 Announcements / 555
554  |  556
Subject: 
Re: Opinions wanted: article rating harmful? (was: New feature: Article rating)
Newsgroups: 
lugnet.admin.general, lugnet.announce
Followup-To: 
lugnet.admin.general
Date: 
Fri, 21 Apr 2000 03:58:02 GMT
Highlighted: 
!! (details)
Viewed: 
967 times
  
In lugnet.admin.general, Todd Lehman writes:
[...]
Clearly, these are very strong feelings being expressed by people.  How many
others feel this way?  What would you like to see happen?  Post your thoughts
as a reply to this message (or reply privately if you prefer not to post your
thoughts publicly).
[...]

First, thanks to everyone who has taken the time to put their thoughts into
words, both publicly and privately.  A clearer picture is beginning to emerge.

We'd like to try scaling things back (i.e., simplifying things) just a little
bit first before trying out any drastic fixes.  Three things seem very clear:
(1) the current system of publicly showing composite numbers in the range 0 to
100 with a default of 50 is causing more overall harm than overall good, and
(2) changing the method of the scale might help, and (3) eliminating the
visible numeric scores altogether would definitely help (help relieve hurt
feelings, that is).

Ironically, the problem isn't the collection of data; the problem is
presenting back too much data -- too easy to see details or variations and
not know what they mean.

We're going to try a simple change to the way the scores are presented -- and
we'd like to keep this in place for at least 24 hours to collect feedback.

Maybe this improves things a bit, maybe it doesn't.  Maybe it solves the
problems altogther, maybe not.  We apologize in advance for the bumpy ride,
since the changes will be visible ones.


Changepoint 1
-------------

Remapped the range 0 to 100 to the range 0 to 5 and changed the default score
from 50/100 to 1/5 so that scores tend generally to climb rather than to fall.
Articles can still be marked down (toward zero), but seeing scores turn
downward would now be a rare rather than a common occurrence.

Thus, here's an old/new conversion table (just for illustration and
understanding -- not important to memorize):

   OLD  <==>  NEW
   ---        ---
   100         5
    90
    80         4
    70
    60         3
--> 50 <------------- old default
    40         2
    30
    20     --> 1 <--- new default
    10
     0         0

And a comparison of the "marked down" and "marked up" ranges (which, again,
are a figment of the imagination, but it's effectively impossible to convince
people of that...so we have to live with that impression and compensate
for it):

    OLD
   -----
   100 |     ^
    90 |     |
    80 |   "marked up" (perceived as "good")
    70 |     |
    60 |     |
    50 |<--starting point (default score)
    40 |     |
    30 |     |
    20 |   "marked down" (perceived as "bad")
    10 |     |
     0 |     V


    NEW
   -----
     5 |     ^
     4 |     |
     3 |  "marked up" (perceived as "good")
     2 |     |
     1 |<--starting point (default score)
     0 |  "marked down" (perceived as "bad")

This was a very simple two-line code change on the server...the ratings engine
was designed to have its output mapped to other ranges besides 0-100, and the
default scores of 50 are/were never stored in the database (it was always
added on-the-fly at display-time) so changing this from 50 to 1 was trivial.

When casting input, the number of choices is also now decreased from 11
choices (corresponding before with 0,10,20,30,40,50,60,70,80,90,100) now to
6 choices (corresponding now with 0,1,2,3,4,5).


Changepoint 2
-------------

Instead of displaying ratings as numbers, display them now as a string of "+"
symbols (and a "-" symbol for 0)... i.e.:

   SYMBOL  NEW   OLD
     ++++   5   83-100
      +++   4   67-82
       ++   3   50-66
        +   2   33-49
            1   17-32
        -   0    0-16

This was also a simple code change in a single location.


Changepoint 3
-------------

Simplifed the rating display in the "Brief" article view mode (the one which
shows an abstract or snippet of it, like what you see on the homepage):

     OLD:    Unrated: 50
     NEW:    (nothing)

     OLD:    Rated: 40 by 1
     NEW:    +

     OLD:    Rated: 63 by 5
     NEW:    ++

     OLD:    Rated: 96 by 14
     NEW:    ++++

Also simplified the rating display in the "All" (full) article view mode so
that the rating displays along with the other headers further to the left:

     OLD:    Rating: 50  /  By: _0_
     NEW:    (nothing)

     OLD:    Rating: 40  /  By: _1_
     NEW:    Rating: (_histogram_)

     OLD:    Rating: 40  /  By:  _1_
     NEW:    Rating: + (_histogram_)

     OLD:    Rating: 96  /  By: _14_
     NEW:    Rating: ++++ (_histogram_)

These were also simple code changes each in a single location.


Changepoint 4
-------------

In a couple of places, flipped the direction of the symbol used for showing
articles you've rated from >> to << (only cosmetic to support the above).


Final notes
-----------

These changes are not switched "on" yet, but look for them about 10-15 minutes
after this "heads-up" message appears.

And the  /news/rating-graph.cgi  displays (the histograms) still show the raw
input table 0 to 100...  Something to change later if this is worth pursuing.

--Todd & Suz



Message has 3 Replies:
  Re: Opinions wanted: article rating harmful? (was: New feature: Article rating)
 
(...) The minus signs still look damaging, and I don't think it's good that a default of 1 (20 internally) can be lowered to 0 by a single person casting a 0. (20+0)/2 = 10, which becomes 0, which becomes "-". Might tweak this tomorrow, either to... (...) (25 years ago, 21-Apr-00, to lugnet.admin.general)  
  Re: Opinions wanted: article rating harmful? (was: New feature: Article rating)
 
(...) Hmm. As far as I can see, the main problem is neither the collection _or_ the representation of the data but the data itself. When members vote, they are voting on different things eg: Do I agree with that? Was that worded well? Was it (...) (25 years ago, 21-Apr-00, to lugnet.admin.general)
  Re: Opinions wanted: article rating harmful? (was: New feature: Article rating)
 
(...) Well, since you asked, I thought I'd leave mine. (...) This change could be for the better. I was fine with the 100 scale (I think the 5 scale gives less acurate results) but it's good if it makes rating easier for people. Hopefully this will (...) (25 years ago, 21-Apr-00, to lugnet.admin.general)

Message is in Reply To:
  Opinions wanted: article rating harmful? (was: New feature: Article rating)
 
All, It seems at this point that the article rating feature -- intended to help -- is actually causing more harm than good to the community. It's difficult to gauge how much harm is being done when opinions are so varied, but it's clear that (...) (25 years ago, 20-Apr-00, to lugnet.admin.general, lugnet.general, lugnet.announce) !! 

309 Messages in This Thread:
(Inline display suppressed due to large size. Click Dots below to view.)
Entire Thread on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact

This Message and its Replies on One Page:
Nested:  All | Brief | Compact | Dots
Linear:  All | Brief | Compact
    

Custom Search

©2005 LUGNET. All rights reserved. - hosted by steinbruch.info GbR