Life In 19x19
http://prod.lifein19x19.com/

Move over ELO, your days are numbered
http://prod.lifein19x19.com/viewtopic.php?f=8&t=1282
Page 1 of 1

Author:  CarlJung [ Wed Aug 04, 2010 2:27 pm ]
Post subject:  Move over ELO, your days are numbered

Copied from slashdot:

"Less than 24 hours ago, Jeff Sonas, the creator of the Chessmetrics rating system, launched a competition to find a chess rating algorithm that performs better than the official Elo rating system. The competition requires entrants to build their rating systems based on the results of more than 65,000 historical chess games. Entrants then test their algorithms by predicting the results of another 7,809 games. Already three teams have managed create systems that make more accurate predictions than the official Elo approach. It's not a surprise that Elo has been outdone — after all, the system was invented half a century ago before we could easily crunch large amounts of historical data. However, it is a big surprise that Elo has been bettered done so quickly!"

http://games.slashdot.org/story/10/08/0 ... e-Over-Elo

Author:  deja [ Wed Aug 04, 2010 2:32 pm ]
Post subject:  Re: Move over ELO, your days are numbered

And I thought this thread was about Electric Light Orchestra... Oh well, time for my Geritol and warm milk.

Author:  Fedya [ Wed Aug 04, 2010 6:21 pm ]
Post subject:  Re: Move over ELO, your days are numbered

deja wrote:
And I thought this thread was about Electric Light Orchestra... Oh well, time for my Geritol and warm milk.


Here you go, at least doing the instruments. :lol:

Author:  Li Kao [ Thu Aug 05, 2010 12:59 am ]
Post subject:  Re: Move over ELO, your days are numbered

Very good idea. The go community should follow that initiative IMO.
My bet would be on some some variant of Whole-History Rating. Their model functions (probability of rank change over time and win probability depending on ranks) probably need some tuning, but the general idea of it is good.
One thing pure WHR doesn't offer is that a certain rank at one time corresponds to the same strength a few years later. But since chess has competitive AIs which don't change their strength over time they can use those to anchor the ranks. So rank inflation and deflation can finally be removed quite well.
This anchoring is harder in go since there the AIs have more specialized flaws and are generally weaker.

Author:  Bantari [ Thu Aug 05, 2010 10:09 am ]
Post subject:  Re: Move over ELO, your days are numbered

I dunno...

Unless an algo can actually predict the result of a game with precision high enough so that the game does not need to be played at all, I'd say that the result of any given game is still mostly a function of what you had for breakfast that day. I see not that much point to increase the rank granularity - which would be necessary if the 'improved' rating algo were to have any effect on the rank. If we don't want the rating to be reflected in rank, then why bother?

For now, I think what we need is some general wide-margin 'pockets' of rank to give the players some idea how to find an even game or how to set the handi. Which we do have in the current kyu/dan system, and I think it works pretty well.

What I think is infinitely more important than tweaking the rating algo is to find a way to make the ranks more uniform around the world. That would really be worth putting effort into, imho.

Author:  prokofiev [ Thu Aug 05, 2010 10:23 am ]
Post subject:  Re: Move over ELO, your days are numbered

Bantari wrote:
I dunno...

Unless an algo can actually predict the result of a game with precision high enough so that the game does not need to be played at all, I'd say that the result of any given game is still mostly a function of what you had for breakfast that day. I see not that much point to increase the rank granularity - which would be necessary if the 'improved' rating algo were to have any effect on the rank. If we don't want the rating to be reflected in rank, then why bother?

For now, I think what we need is some general wide-margin 'pockets' of rank to give the players some idea how to find an even game or how to set the handi. Which we do have in the current kyu/dan system, and I think it works pretty well.

What I think is infinitely more important than tweaking the rating algo is to find a way to make the ranks more uniform around the world. That would really be worth putting effort into, imho.


This isn't really about increasing rank granularity (of course as you say there's a fair amount of randomness in who wins any given game). It's about increasing the accuracy of the ranks people actually have (e.g. fixing the 4k who's really 2k or the 2d who's really 1d). For example, there are "corner cases" (as Harleqin mentions in his signature) such as people who improve quickly or people who stop playing for a while and then come back (either stronger or weaker, depending on what happened while they weren't playing tournament/rated games).

It's nice if a rating system is better at deciding what handicap people should play with as then everyone gets to play more interesting games and fewer routs.

Author:  wms [ Thu Aug 05, 2010 10:51 am ]
Post subject:  Re: Move over ELO, your days are numbered

Bantari wrote:
I dunno...

Unless an algo can actually predict the result of a game with precision high enough so that the game does not need to be played at all, I'd say that the result of any given game is still mostly a function of what you had for breakfast that day.
You are implying that a rating algorithm's job is to predict who will win a game. The actual job is to predict the likelyhood of each player winning. It's a subtle difference but it's important.

Overall this is interesting. It doesn't take a genius to see that Elo was designed to be workable in a world where all calculations were done by hand, with at most a mechanical adder and a logarithm table to help you out. It's a brilliant system that works incredibly well even with that limitation, but today where vastly more complex calculations can be performed instantly and flawlessly it's not surprising that smart people can quickly come up with more accurate systems. When the content is over I'll be interested to see what the final "best" system is like. Seeing what approaches will be taken by the top systems seems to me to be the most interesting and useful part of the contest.

Edit: Your comment about syncing up the world's rating systems is on a different topic I think. Coming up with better ranking systems is an interesting and useful mathematical exercise. Syncing up the rating systems of the world is all politics. A completely different job, and probably a job that would be done by different people.

Author:  Bantari [ Thu Aug 05, 2010 11:36 am ]
Post subject:  Re: Move over ELO, your days are numbered

prokofiev wrote:
This isn't really about increasing rank granularity (of course as you say there's a fair amount of randomness in who wins any given game). It's about increasing the accuracy of the ranks people actually have (e.g. fixing the 4k who's really 2k or the 2d who's really 1d). For example, there are "corner cases" (as Harleqin mentions in his signature) such as people who improve quickly or people who stop playing for a while and then come back (either stronger or weaker, depending on what happened while they weren't playing tournament/rated games).

It's nice if a rating system is better at deciding what handicap people should play with as then everyone gets to play more interesting games and fewer routs.


I dunno...

Corner Cases are Corner Cases for a reason. They usually sit right between the ranks, and given the random factor to each of our games I am not sure if there is a great benefit of forcing them into this rather than that rank. Chances are - their actual playing strength jumps back and forth between these ranks anyways, on daily basis (sometimes from game to game.) Eventually they will learn enough (or forget enough) to find their way firmly into a rank. And if not, given the randomness of our play, I am not sure its really such a big problem.

wms wrote:
Edit: Your comment about syncing up the world's rating systems is on a different topic I think. Coming up with better ranking systems is an interesting and useful mathematical exercise. Syncing up the rating systems of the world is all politics. A completely different job, and probably a job that would be done by different people.


Very true. Or is it?

When you think about the servers, you are certainly right, because its usually up to the Chief to decide. But world is not just Go Servers, and I don't really think it is possible to have a decent algo on a server anyways - not unless you really take into account the game settings (like time controls, for example.) In real world, outside of the servers, math is one thing, but I have the feeling that the decision to implement a new algo for various federations would be done by many of the same people who would have to decide on unified world rating. And their reasoning (and process) might well be the same as well.

As a nice math exercise, you are all right, its a very cute and amazing thing. Thumbs up and toothy smile. ;)

Author:  prokofiev [ Thu Aug 05, 2010 1:39 pm ]
Post subject:  Re: Move over ELO, your days are numbered

Bantari wrote:
I dunno...

Corner Cases are Corner Cases for a reason. They usually sit right between the ranks, and given the random factor to each of our games I am not sure if there is a great benefit of forcing them into this rather than that rank. Chances are - their actual playing strength jumps back and forth between these ranks anyways, on daily basis (sometimes from game to game.) Eventually they will learn enough (or forget enough) to find their way firmly into a rank. And if not, given the randomness of our play, I am not sure its really such a big problem.


The real issue occurs when the disparity is more than one rank. This might seem rare, but, for example, many people have various short spurts where they improve a lot. If the rating system really lags behind for them for a while, that's a lot of mishandicapped games they have to play.

Author:  Bill Spight [ Thu Aug 05, 2010 3:13 pm ]
Post subject:  Re: Move over ELO, your days are numbered

wms wrote:
You are implying that a rating algorithm's job is to predict who will win a game. The actual job is to predict the likelyhood of each player winning.


For go I would say that the job is to predict the proper handicap and komi to make the game a toss-up. :)

Author:  Gresil [ Fri Aug 06, 2010 8:44 am ]
Post subject:  Re: Move over ELO, your days are numbered

deja wrote:
And I thought this thread was about Electric Light Orchestra... Oh well, time for my Geritol and warm milk.


"Don't Bring Me Down" started playing in my head when I saw the thread title...

Author:  deja [ Fri Aug 06, 2010 2:37 pm ]
Post subject:  Re: Move over ELO, your days are numbered

Gresil wrote:
deja wrote:
And I thought this thread was about Electric Light Orchestra... Oh well, time for my Geritol and warm milk.


"Don't Bring Me Down" started playing in my head when I saw the thread title...

Don't bring me doooowwwn, Brrruce - http://www.youtube.com/watch?v=qj8kMmUxkSE

Sorry, Carl :ugeek:

Author:  Mef [ Sat Aug 07, 2010 4:55 am ]
Post subject:  Re: Move over ELO, your days are numbered

Bantari wrote:
I'd say that the result of any given game is still mostly a function of what you had for breakfast that day.



The algorithm gives you the expected value, breakfast gives you the standard deviation (=

Page 1 of 1 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/