Life In 19x19
http://prod.lifein19x19.com/

Seeking Input on Rating System - New Server
http://prod.lifein19x19.com/viewtopic.php?f=10&t=7634		 Page 1 of 1
Author:  illluck [ Sat Jan 12, 2013 5:46 pm ]
Post subject:  Seeking Input on Rating System - New Server
This is a question for FunNode (viewtopic.php?f=9&t=7603). Ravi is trying to find a rating system that would allow quick adjustment of new players' ratings. The current system is Elo with max rating change K = 32 which makes it quite tedious to reach a reasonable rating for those who are considerably stronger/weaker than the 1000 initial rating.

The main constraint here is that he is trying to keep some sort of consistency with the other games hosted and so isn't too thrilled about self-set ratings (though if there's no other reasonable choice I'd imagine it can be used).

Currently the suggestion seems to be a confidence-based scheme where the rating changes depend on the rating certainties of the two players (different K values for each player), but as none of us are all that knowledgeable in rating systems we'd like additional input. Is a confidence based system a reasonable choice in this case? If so, what should be the range and determinants of the K value?

Many thanks in advance :)
Author:  SmoothOper [ Sat Jan 12, 2013 6:27 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
illluck wrote:
This is a question for FunNode (viewtopic.php?f=9&t=7603). Ravi is trying to find a rating system that would allow quick adjustment of new players' ratings. The current system is Elo with max rating change K = 32 which makes it quite tedious to reach a reasonable rating for those who are considerably stronger/weaker than the 1000 initial rating.

The main constraint here is that he is trying to keep some sort of consistency with the other games hosted and so isn't too thrilled about self-set ratings (though if there's no other reasonable choice I'd imagine it can be used).

Currently the suggestion seems to be a confidence-based scheme where the rating changes depend on the rating certainties of the two players (different K values for each player), but as none of us are all that knowledgeable in rating systems we'd like additional input. Is a confidence based system a reasonable choice in this case? If so, what should be the range and determinants of the K value?

Many thanks in advance :)


I would do something like a runs based scan statistic with a poison assumption. IE if a player is playing other players at specific rank, is the proportion of games that the player is winning significantly different from 1/2 if so then the player is either higher or lower than the rank they are playing at, the reason it needs to be a scan statistic is so that you can compare windows of games that are of different lengths so that both incremental improvement over long periods of time and dramatic improvement with many wins in a short period will be rewarded. Maybe like Karlin and Altschul statistics.
Author:  gurujeet [ Sat Jan 12, 2013 7:39 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet
Author:  SmoothOper [ Sat Jan 12, 2013 8:06 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
gurujeet wrote:
The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet


Hmm... Normal distribution for modeling, players rating, explains much about the AGA.
Author:  Kirby [ Sat Jan 12, 2013 9:04 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
SmoothOper wrote:
gurujeet wrote:
The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet


Hmm... Normal distribution for modeling, players rating, explains much about the AGA.


Why don't you like the normal distribution in this case? I'm kinda interested in this topic.
Author:  Dusk Eagle [ Sat Jan 12, 2013 9:46 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
Well, I wouldn't go so far as to say it explains a lot about the AGA, but as for why a normal distribution could be considered a bad choice, take a look at this sensei's page on KGS's rank distribution. This curve is quite heavily left-skewed.
Author:  badukJr [ Sat Jan 12, 2013 10:13 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
But the AGA rating distribution is also skewed similarly.

http://senseis.xmp.net/?AGARatingHistogram

Maybe someone is defining normal distribution poorly :scratch:
Author:  Kirby [ Sat Jan 12, 2013 10:14 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
badukJr wrote:
But the AGA rating distribution is also skewed similarly.

http://senseis.xmp.net/?AGARatingHistogram

Maybe someone is defining normal distribution poorly :scratch:


Maybe we just need more ranks ;-)
Author:  yoyoma [ Sat Jan 12, 2013 10:14 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
The graph Dusk Eagle pointed to is the distribution of player's ranks. I think the cumulative normal distribution in the AGA ratings is related to the probability of one player beating another, given the difference in their rating. So if the difference is 0, its 50-50. And it moves away from 50-50 in a way defined by that distribution.

Picking normal vs some other distribution would involve doing some studies on which one fits the data the best I guess. I don't know much about that. I don't know how a Poisson fits in to all this, ratings are usually real numbers so it's not necessary to use a discrete probability distribution. Maybe SmoothOper prefers some method different from what Elo and Glicko use.
Author:  vash3g [ Sat Jan 12, 2013 11:41 pm ]
Post subject:  Re: Seeking Input on Rating System - New Server
Using the AGA's rank system I logged into the database and took this screenshot of the current rank distribution. I should point out that there is a large skewed due to so many at go congress artificially raising them to that point (most are not 1d). I will also point out that this system has been in place and has been tweaked slightly for the last 20 years or so. The math and other information on the background behind it can be found @ http://www.usgo.org/ratings

Image
Full Sized Image
Author:  SmoothOper [ Sun Jan 13, 2013 7:13 am ]
Post subject:  Re: Seeking Input on Rating System - New Server
Kirby wrote:
SmoothOper wrote:
gurujeet wrote:
The AGA rating system is documented on the usgo.org website and is open source. You might want to look at that.

- gurujeet


Hmm... Normal distribution for modeling, players rating, explains much about the AGA.


Why don't you like the normal distribution in this case? I'm kinda interested in this topic.


In this case there is no guarantee that there will be a significant difference between the ranks.
Author:  illluck [ Sun Jan 13, 2013 8:16 am ]
Post subject:  Re: Seeking Input on Rating System - New Server
It feels like the thread got a bit derailed (in fact, I reported SmoothOper's comment about AGA last night), but it's actually shaping out to be quite interesting, thanks to all for taking the time to respond. However, sorry if I wasn't clear, but the main question is about a way to let new users' ratings adjust quickly to "correct" levels.
Author:  pwaldron [ Sun Jan 13, 2013 8:21 am ]
Post subject:  Re: Seeking Input on Rating System - New Server
SmoothOper wrote:
Hmm... Normal distribution for modeling, players rating, explains much about the AGA.


The normal distribution is for a single player's performance around his true rating. The AGA system makes no assumptions about the distribution of ratings within the population as a whole.
Author:  HermanHiddema [ Sun Jan 13, 2013 10:53 am ]
Post subject:  Re: Seeking Input on Rating System - New Server
One system to consider is to use the first X games (say 15) as a basis for the initial rating. As long as the player has fewer than X games, calculate his rating as the rating that would not change given their score and their average opponent's rating. As soon as they have X games, their rating calculated as such is used as their starting rating, and from then on just use the usual system. Note that the calculation of this temporary rating is unstable as long as the player has only wins, or only losses. To fix that, add an additional virtual "game zero" with a draw against rating 1000 (this also elegantly results in a temporary rating of 1000 for players with no results yet.)
Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/