New revisions to this ranking system

The formation of Svift has prompted me to rethink a few small aspects of how this ranking system works. Regarding the rankings themselves, I’ve made two simple changes that, after a bit of testing, seem to have resulted in a better degree of accuracy in situations like the formation of Svift.

The first change is the simpler one. Because I’ve been keeping this system going for over a year now, I can extend the scope of time that the rankings cover. Up until now, the rankings were based on data from the last 300 on-record matches. This covered about seven months’ time. As it happens, that time-frame is small enough for people like Astt and Ma3la to have dropped off the list and become unranked, and therefore have a default score of zero.

Looking at how well Astt has been doing so far in scrims with Svift, it’s clear that seven months is too short a time to assume significant levels of rust. Therefore, I’ve extended the scope of the rankings to cover 500 matches instead of 300. At the moment, this covers almost exactly a year, and people like Astt and Ma3la still have a good amount of old data still in place.

A side-effect of this change is that the scores are higher because there are more people on the list due to the wider scope. The best players and teams now have scores north of a thousand when previously they were around 800.

Another effect of the wider scope is that it takes longer for new players, who don’t have many matches under their belts, to ascend up to the top of the rankings (if that’s where they deserve to be). An example is Kaidus, who only had his first on-record entries this season and is only now beginning to enter the elite tier in the rankings. Extending the scope from 300 to 500 matches means that players need to amass a few more entries before they can reach the same level.

The second change I’ve made helps to alleviate this, while also at the same time dampening the huge and unrealistic score shrinkage that’s seen when a good team picks up somebody who’s unranked.

The green column in the rankings represents the players’ hit-rates – the percentage of times they got gilded in their on-record matches. Whenever the rankings or projection machine have to factor in somebody who’s unranked, they obviously have zero entries and zero gildings, which amounts to a 0% hit-rate. Because their mileage is also zero, their score in the rankings is zero. Zero is, of course, the lowest score somebody can have, and the rankings and projection machine always use this number to represent somebody who is unranked. This assumption of zero is what makes teams suffer so much in score when an unranked player joins their camp.

I wanted to make a change that meant unranked players are actually assumed to be some way up from the bottom of the rankings, and they would only drop down if their opening matches don’t result in gildings.

The solution I’ve come up with involves weighting everyone’s hit-rate. The green column now represents this weighted hit-rate, which is itself the average of two aspects. The first of these is the real, pure hit-rate that we’re accustomed to. Someone who’s 2/4 has a pure hit-rate of 50%. The other aspect of this average functions the same, but it adds 1 to the player’s entries and gildings numbers. For someone who’s 2/4, this second aspect would take their hit-rate as if it were based on a record of 3/5, which would result in 60%. Because the green column takes the average of both these percentages, it would come out at 55% where previously it would have just been the pure value of 50%.

For the players who have loads of games under their belts, this change doesn’t modify their figure even by a single percentage point. The change has more influence the less experience a player has.

It’s biggest impact is with unranked players. People who are unranked are basically 0/0, and their old hit-rate would come out at 0%. With this change, however, their new weighted hit-rate is the average of 0/0 and 1/1. 1/1 is, of course, a 100% hit-rate, so the weighted value comes out at 50%.

A player’s final score is calculated based on how many players in the list have a worse weighted hit-rate than you, plus how many have a worse mileage (which is the entries and gilding numbers added together). People who are 0/0 still have a mileage of zero, but because their weighted hit-rate is now 50%, their score comes up quite significantly because there are many people in the list whose weighted hit-rates are lower than that.

Right now, the score of an unranked 0/0 player comes out at 452 rather than 0. That number will fluctuate up and down as the data pool for the rankings gradually changes. At the time of writing, a score of 452 places you 362nd out of the 588 players who are currently ranked. Whenever the rankings or projection machine are asked to account for an unranked player, it uses this score as a placeholder instead of zero like it used to.

Even with the scope widened to 500 matches, Svmzi remains a stranger to this system. Previously, he’d be contributing zero to Svift’s team score. Now he contributes 452, which significantly alleviates the pain caused by the team featuring an unranked player, so much so that they’re still ranked at about the same level as Lowpander. While I imagine most of us are expecting Svift to perform at an even higher level than this, it’s important to remember that the rankings have to assume somebody who’s unranked is just some random High player. We, however, have the benefit of knowing that Svmzi is notably more special than that. Meanwhile, under the old system, Svift were ranked behind even Nunya.

The way the rankings assess teams with fewer than six ranked players has also been adjusted a bit. Previously, it would simply take the average of every player in the rankings that’s a part of the team. If there were only four players in the list assigned to that team, it would take the average of them and them alone, and not substitute in zeroes for the remaining empty spots out of the six.

Now, factoring in any unoccupied spots is built in to the team rankings. If it can’t find six players assigned to the team, it fills in any empty spots with that default 0/0 score, currently 452. If it sees six or more, it just takes the average from everyone just like it did previously.

Something you might notice about this is that if a new team arises for their region’s top division, but none of their players have any on-record appearances yet, the whole team will be based on default scores and the team itself would have a preliminary score of 452 (again, that number might come out slightly differently with time). At present, such a score would actually leave them ranked ahead of the three lowest teams in the rankings. Previously, the team’s score would have been zero. If they have a poor opening match, though, their weighted hit-rate will naturally come down and so will their score. If their opening match is good and they generally get gilded, their score would improve. In many ways it’s like a placement match.

This all works to solve the problem of assuming anyone who’s unranked is worthless. Unranked players should instead be assumed to be roughly of the quality of a good High/IM/Div-2-tier player. With this new weighted hit-rate, unranked = better than the lowest of the ranked, which sounds about right for that goal.

The last changes I’ve made aren’t related to the rankings, and instead affect the projection machine. I’ve rebuilt it from the ground up for the second time. Its assessments are still based only on the teams’ scores in the rankings, but the route to it making a conclusion is now more advanced.

Previously, its impressions would be mostly based on the proportionality of the scores of the a two teams. If one team’s score was half that of the other, it would expect domination. This is over-simplified, but that was basically how it worked. This meant that a team with a low score would still be expected to dominate a team with a really low score, which never really turned out to be true.

The amendments I’ve made have prevented this from happening. To put it very basically, it now expects matches to be more and more close the further away the two teams’ average score is from the score of the world’s best team. This has the added bonus of it giving more decisive predictions between the top teams, who are usually only a few points from eachother. It used to be that it would expect very close matches between the three or four top teams, but it’s now much happier to lean more in one direction between two top teams even if there’s barely anything between them in score.

To give some examples of the effects of these changes in action, it used to expect that Lunatik would easily beat Fable 5-0. Now, it’s impression is 5.0-1.9. It used to think Jasmine Tea and No Safeword would be almost dead even in score. Now, it would expect the outcome to be clearer at 2.9-2.1 in favour of Jasmine.

I’d already made all these changes when Ma3la announced the revival of the Ascent brand, and that he was replacing Paddie at the team formerly called EVL. Under the old system, Ma3la had slipped into the realm of the unranked, and Ascent’s score in the team rankings would have plummeted due to his initial score of zero. The new expanded scope means this system still recognises Ma3la and his score is still high enough for him to be in the top 25% of the ranked players. This means the rankings are suspicious enough of him for it to see the loss of Paddie as a downgrade, but all the new changes to the rankings mean that Ascent has only lost about 40 points due to the change, and they’re still very much in touch with the world’s best teams.

Under the old system, Ascent would be ranked well out of position after losing Paddie. As it is now, they’ve only suffered a small setback that probably does a good job of simulating this little period of instability that the team is going through. They can claw the lost points back quickly if they keep finding success.

This isn’t a proper update post, but you can have a look at the newly-revised rankings here.

I’m going to be away for a week starting tomorrow, and it’ll take me another day or two to get back up to speed with the stuff I’ve missed afterwards. You can expect the next proper post to come the weekend after this one, in which I’ll cover all the matches to have happened since the previous update.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s