Forum

  #11  
Old 04-06-2016, 02:14 PM
abra's Avatar
abra abra is offline
Senior Member
 
Join Date: Jun 2009
Posts: 148
Default

Wow, that one seems to have suffered, under the new ratings. I would think that a quote with a solve rate that low, should get an automatic Difficult rating.
Reply With Quote
  #12  
Old 04-06-2016, 05:44 PM
admin's Avatar
admin admin is offline
Administrator
 
Join Date: Feb 2008
Posts: 244
Default

Right now we have a three-tiered difficulty system, and one of the pitfalls of that system is that there are a lot of "edge cases". This is a prime example. We're marking as "Easy" all puzzles that fall into the 1-33 percentiles overall (based on a combination of completion rate, median solve time and user feedback). This puzzle here falls exactly on the 33rd percentile, so it literally straddles the line between Easy and Medium. It made it to "Easy" just by the hair on its chinny chin chin.

Most likely we'll beef up the scoring system so that we can break it down by 5-10 percentiles at a time (instead of 33 at a time) so that things will be a bit more reflective in the points department.

Thanks!
__________________
http://www.puzzlebaron.com/blog
Reply With Quote
  #13  
Old 04-06-2016, 06:37 PM
abra's Avatar
abra abra is offline
Senior Member
 
Join Date: Jun 2009
Posts: 148
Default

It's great to know that you're working on it. Thank you!
Reply With Quote
  #14  
Old 04-06-2016, 06:55 PM
nanrich's Avatar
nanrich nanrich is offline
Member
 
Join Date: Feb 2008
Posts: 37
Default

Yes, thanks Admin for the explanation. And also for having such a great site!
Reply With Quote
  #15  
Old 04-06-2016, 08:18 PM
LLapp's Avatar
LLapp LLapp is online now
Occasional Ranter
 
Join Date: Aug 2014
Posts: 255
Default Confusion about the 1-to-10 star rating system

Stephen,

Thank you for all the attention you've been giving this site lately. We players appreciate it.

Since you plan to further refine the difficulty ratings, I want to point out that there may be a critical glitch in how you calculate user feedback via average star ratings for Difficulty.

Based on reading many quote comments versus what you said recently about how you treat a quote's average star rating, I'm pretty sure that many (or even most) users have misinterpreted the meaning of the 1-to-10 star scale.

I get the impression that you read the meanings of our "Difficulty" ratings in terms of "Easy vs. Difficult," like this:

1 (extremely easy) -- 3 (easy) -- 5 (neither difficult nor easy) -- 7 (difficult) -- 10 (extremely difficult)

(Did I get that right?)

And meanwhile, I believe that many of us use that same "Difficulty" rating to rank the quote only for difficulty, literally -- like so:

1 (not difficult) -- 3 (somewhat difficult) -- 5 (pretty challenging) -- 7 (very difficult) -- 10 (nearly impossible; I only got here by googling the quote)

(Other players - Have you had this confusion?)

If I'm right about this disconnect, then you can see how it might lead you to misread many users' intended rating of "somewhat difficult" to mean "quite easy" -- which could result in you assigning a difficulty rating out of sync with reality. And it gets more complicated when you consider that the misunderstanding may be widespread but not universal.

This confusion about the star-rating scale is most apparent in how people respond to star ratings for "Offensive." When an innocent quote gets an average "Offensive" rating of 2 stars, for example, commenters typically remark, "How could anyone find this quote offensive?" The perception is that even one star equals "slightly offensive" -- whereas I think you are reading one star to mean "completely innocent." I suspect the same disconnect happens in the "Errors" rating: I might indicate a real-but-not-fatal error by clicking four stars, but now I wonder if you read that feedback to mean "Quote seems okay as is."

So, if you plan to use our star ratings for feedback data, I strongly encourage you to first update the feedback section of the solution page, to clarify the star rating scales. I bet you would get significantly different feedback after the scales are clarified.

Thank you for reading this long message!

Lisa
Reply With Quote
  #16  
Old 04-07-2016, 05:22 AM
LurkerSmurf's Avatar
LurkerSmurf LurkerSmurf is offline
Junior Member
 
Join Date: Jan 2010
Posts: 27
Default

As long as we're commenting on the star system, I've always had problems with putting "edgy" in the same category as "offensive". Edgy can mean clever, not just "on the border of good taste". I wonder how many so-called offensive quotes were removed because people rated them high for being somewhat rude but funny.
__________________
La, la, la - la - la - la....
Reply With Quote
  #17  
Old 04-07-2016, 05:39 AM
wvwoman's Avatar
wvwoman wvwoman is online now
Senior Member
 
Join Date: May 2012
Posts: 288
Default

Quote:
Originally Posted by LurkerSmurf View Post
As long as we're commenting on the star system, I've always had problems with putting "edgy" in the same category as "offensive". Edgy can mean clever, not just "on the border of good taste". I wonder how many so-called offensive quotes were removed because people rated them high for being somewhat rude but funny.
i've always thought the same thing. i don't think of edgy as a synonym of offensive--i think of it as offbeat, out of the ordinary, or wickedly funny.
Reply With Quote
  #18  
Old 04-07-2016, 06:31 AM
Annamariah's Avatar
Annamariah Annamariah is online now
Member
 
Join Date: Nov 2013
Posts: 33
Default

One more thing about the difficulty stars - if the solving rate is less than 10%, it means that most of the people who found the quote extremely hard never got the chance to give it any stars at all.
Reply With Quote
  #19  
Old 04-08-2016, 05:31 AM
montyb's Avatar
montyb montyb is offline
Senior Member
 
Join Date: Nov 2011
Posts: 175
Default For Lurker

"He whose inclination prompts him to cultivate your friendship of his own accord will love you more than one whom you have been at pains to attach to you."
Samuel Johnson

Lurker on 2012-09-26 14:41:53
So not fair that an "easy" one has a 14.8% solve rate.

The new system has this rated as "difficult" now.
__________________
"Great spirits have always encountered violent opposition from mediocre minds." ~ Albert Einstein
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT. The time now is 05:54 PM.


Powered by vBulletin® Version 3.6.6
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.

About Puzzle Baron

The Puzzle Baron family of web sites has served millions and millions of puzzle enthusiasts since its inception in 2006. From cryptograms to acrostics, logic puzzles to drop quotes, patchwords to wordtwist and even sudoku, we run the gamut in word puzzles, printable puzzles and logic games.

Questions or Comments?

The word 'bought' has how many letters in it?