Cryptograms Forums

Cryptograms Forums (http://www.cryptograms.org/forum/index.php)
-   Main Forum (http://www.cryptograms.org/forum/forumdisplay.php?f=2)
-   -   Difficulty Ratings (http://www.cryptograms.org/forum/showthread.php?t=817)

admin 03-31-2016 08:26 AM

Difficulty Ratings
 
Hi folks -

We've implemented a new difficulty rating system that should be quite a bit more accurate, based on real-world solving statistics. This means you will now start seeing more short puzzles that are ranked "Easy" and long puzzles that are ranked as "Difficult". The new formulations are based off median solve times (by quote length), success rates, and user feedback.

Thanks!

abra 03-31-2016 08:59 AM

Wow!! Stephen, you've heard us, and answered, This is great news.:)

Lurker 03-31-2016 09:19 AM

Thank you! Although, I'm going to miss all those points on short "difficult" quotes.

marnita 03-31-2016 12:15 PM

I agree, Lurker. Like many improvements, this one will take some getting used to.

gryhnd51 03-31-2016 09:28 PM

Now that we've had many quotes removed because some folks find them offensive, I've been paying attention to every single quote I solve and the ratings they've been given. I have to say that I've been truly amazed at the quotes deemed "offensive", especially since we are all given the opportunity to omit "possibly offensive quotes" in our settings. Therefore, I've taken every chance to negate the naysayers. I hope that the rest of you will take the small amount of time it takes to check them.
Also, the new scoring seems more arbitrary than rational...and based on a small percentage of folks who will always complain, no matter what. When I first started playing, those very long quotes were my "bread and butter" and the stepping stones to finally finding them more difficult and the shorter ones easier! I hope you'll all forgive me for saying that it feels a bit like "dumbing down" the site.
I know that Stephen is trying to be responsive to input, and I applaud him for that. I guess I'd just like to see a much larger sampling size before changes are made.
Ah well, I'm sure I'll get used to the changes and come to appreciate them.

abra 04-01-2016 12:09 PM

I thought it would be good to have the quotes rated according to the statistics. I know a lot of commenters had voiced that opinion as well.

But, so far I don't really care for the new system. Maybe, I just can't be pleased. :D

LLapp 04-03-2016 09:30 PM

Let's give it some time
 
I'm generally finding that the new difficulty ratings are more realistic. Yes, I still have trouble with some "Easy" ones (usually before coffee), and I still breeze through a few "Difficult" ones (usually after coffee), but for the most part the ratings are jibing with how hard the quote was.

Here is a great example of the new ratings being correct. This is a long quote I just solved that used to be ranked as "Easy" and that was upgraded last week to "Difficult." It was brutal, even though it was long, and the stats reflect this, and now so does the difficulty rating:
"Poker is to cards and games what jazz is to music. It's this great American thing, born and bred here. We dig it because everybody can play."

You solved this cryptogram in 76 seconds. That's worth 437 points!

This Cryptogram's Statistics:

Success Rate:
10.2%
Average Time:
176 seconds
Record Time: 29 seconds (nanrich)

This quote used to award no more than 149 points, and now it awards up to 449 points. I'd say that's much more realistic.

wvwoman 04-04-2016 02:20 AM

Quote:

Originally Posted by LLapp (Post 7030)
I'm generally finding that the new difficulty ratings are more realistic. Yes, I still have trouble with some "Easy" ones (usually before coffee), and I still breeze through a few "Difficult" ones (usually after coffee), but for the most part the ratings are jibing with how hard the quote was.

Here is a great example of the new ratings being correct. This is a long quote I just solved that used to be ranked as "Easy" and that was upgraded last week to "Difficult." It was brutal, even though it was long, and the stats reflect this, and now so does the difficulty rating:
"Poker is to cards and games what jazz is to music. It's this great American thing, born and bred here. We dig it because everybody can play."

You solved this cryptogram in 76 seconds. That's worth 437 points!

This Cryptogram's Statistics:

Success Rate:
10.2%
Average Time:
176 seconds
Record Time: 29 seconds (nanrich)

This quote used to award no more than 149 points, and now it awards up to 449 points. I'd say that's much more realistic.

wow--it surely is!

Altoid701 04-04-2016 03:39 PM

I'm all for this in theory, though I haven't yet played any quotes since the change was made so I'll have to get back to you with my real opinion! As someone who goes for speed rather than completion %, this'll likely mean I have fewer points overall, which I'm okay with as I don't play enough to make the top 20 in total points anyway.

nanrich 04-06-2016 12:05 PM

I'm having a hard time figuring out the new scoring system. Some quotes that have quite a low solve rate now receive much fewer points than under the old system. I've pasted in an example below. The difficulty ratings marked by players was quite low (3 something out of 10, if I'm remembering right), so that may be why the puzzle now gets a lower number of points. But it seems to me that the solve rate of 8.5% speaks for itself and that, compared to puzzles with a high success rate, a lot fewer players would even have the chance to give a difficulty rating. On the other hand, I've seen some puzzles that would previously have been rated "easy" based on their length that now get a lot of points even though the solve rate is over 50%. Anyone else noticing this?

You solved this cryptogram in 21 seconds. That's worth 146 points!
"Indifference, then, is not only a sin, it is a punishment."
Elie Wiesel
This Cryptogram's Statistics
Success Rate: 8.5%
Average Time: 92 seconds
Record Time: 10 seconds
Your Time: 21 seconds


All times are GMT. The time now is 08:42 PM.

Powered by vBulletin® Version 3.6.6
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.