VoodooMike
Joined: Nov 07, 2010
|
  Posted:
Dec 13, 2011 - 10:44 |
|
I'll preface this first by saying this isn't an objection the CR, or a suggestion to replace it - this is merely an idea based around numbers I know to be available, and the frequent criticisms of systems like ELO as far as determining a player's "skill" in games that involve things like chance, opponent selection, and so on.
The base idea is this - using aggregated data for past racial matches, a baseline average for single-game performance is established. This means a different baseline for "Amazon vs. Chaos Dwarf" and "Khemri vs. Wood Elf" and so on. Most likely it would be based on the average number of touchdowns scored by each team in matchups, but maybe other things would work (maybe multiple things? TDs seem to be the primary "win" measurement).
Now, after each game the match result is compared with that baseline data, and a team's overall standardized Z score is modified by the average of the weighted Z distances (standard deviations) from the baseline data the two teams achieved.
In plainer english, if the average number of touchdowns in a match between, say, Goblins and Dwarfs is Goblins 0, Dwarfs 4, and you have a matchup with your goblin team against a dwarf team and earn a 1-3 loss, you've actually done BETTER than average and the opponent has done WORSE than average, which means that in spite of having lost the match, your rating goes up and his goes down.
The main issue with win-based ratings as far as assessing skill, is that it assumes balanced matches, and we know for a fact that teams aren't all balanced against one another. Similarly, since your rating always goes up with a win, and down with a loss, people are prone to avoiding matches they think they'll lose if they're concerned with their rating, especially if they're facing someone with a lower rating (where the penalty for loss is quite a bit higher than the benefit of victory).
So under a system like this you could actually rise to the top of the rankings as a halfling coach by simply doing much better with your halflings than most other people can, and you can CLPOMB a hundred lowball victories and it count for very little, which on some level seems to be what people think a rating of coaching skill should allow.
Anyway, this is mostly an idea for discussion. I don't expect it to be implemented (though feasibly we could create a mock-up of the system to explore it). |
|
|
Calthor
Joined: Jan 24, 2006
|
  Posted:
Dec 13, 2011 - 10:49 |
|
This is partly a good idea, imo. Except for the fact that if one team wins, and the other loses, the rating should go up for the winner, never down (an Elf team should not be punished for 'grinding' his way to a victory, so to speak).
Yet the influence of racial modifiers / racial match-ups (eg. the baseline idea) does have potential. |
|
|
Stonetroll
Joined: Jun 30, 2005
|
  Posted:
Dec 13, 2011 - 11:01 |
|
If you are trying to make a better CR formula by analyzing every micro aspect of the game, you should really count the number of KO/cas suffered against in there too. It takes much more skill and luck to win games when your players are being wiped off the pitch. But then again I really question if it is meaningful to try make a "perfect" coach rating, at the moment CR works as a nice ballpark approximate of player skill. There are more important stuff to do, like bombardiers and other missing skills. |
|
|
Mateuszzzzzz
Joined: Feb 26, 2006
|
  Posted:
Dec 13, 2011 - 11:19 |
|
You win game by scoring 1 more then opponent. I dont think scoring more is any measure of better skill. |
|
|
DanielM75
Joined: Jan 18, 2009
|
  Posted:
Dec 13, 2011 - 11:27 |
|
Agree with comments.
Take the recent FC match that was 8-7 vs a similar match that wasn't OTD silly as was a strategic loose cage 3-2 win.
You can't possibly assess coaching skill via that. |
|
|
VoodooMike
Joined: Nov 07, 2010
|
  Posted:
Dec 13, 2011 - 11:41 |
|
Calthor wrote: | This is partly a good idea, imo. Except for the fact that if one team wins, and the other loses, the rating should go up for the winner, never down (an Elf team should not be punished for 'grinding' his way to a victory, so to speak). |
But is it really a sign of skill for a Skaven team to score against a Halfling team? Is it a sign of skill for a Skaven team to beat a Halfling team? That's sort'v the point of racial comparisons - some teams you simply expect poor performance from, and that's not a matter of the coach being bad, or the opposing coach being good - it's a matter of some races being better than others.
Can we really say that a coach's skill is at fault if his Goblins lose to Chaos Dwarves? If we always assign a rating decrease with a loss, and an increase with a win, that IS what we're saying.. and because of it, we encourage those Chaos Dwarves to seek out Goblin teams to play against, since it builds their rating even if it results in a reasonably easy victory.
Stonetroll wrote: | If you are trying to make a better CR formula by analyzing every micro aspect of the game, you should really count the number of KO/cas suffered against in there too. |
Not trying for perfect, just brainstorming around the common complaints I see about ELO-style systems being used to determine coach skill. KO/cas is as valid a measure as any, but you do have to play to achieve casualties to get them, which would mean the system would penalize people who simply played to score. We can assume that everyone, on some level, plays to get more TDs than their opponent since that's what makes them the winner.
Stonetroll wrote: | There are more important stuff to do, like bombardiers and other missing skills. |
That's nothing you or I can do, but this is. They aren't mutually exclusive because of that.
DanielM75 wrote: | Take the recent FC match that was 8-7 vs a similar match that wasn't OTD silly as was a strategic loose cage 3-2 win.
You can't possibly assess coaching skill via that. |
When you have enough matches to use for your baseline, and you weight any changes to rating based on total games played by the team whose rating is being changed, then even if there are wild and unusual game results, they don't change things dramatically.
Remember also that if both teams score a ton of TDs, that will balance out in the calculation, since it is being compared to the typical ratio of TDs for that racial matching, and how far removed from that a specific match was. |
|
|
koadah
Joined: Mar 30, 2005
|
  Posted:
Dec 13, 2011 - 12:00 |
|
VoodooMike wrote: |
Anyway, this is mostly an idea for discussion. I don't expect it to be implemented (though feasibly we could create a mock-up of the system to explore it). |
The match data is freely available so anyone who is interested can create any ranking they want.
Someone could probably knock up a (greasemonkey?) script to suck it into the homepages of people who want it. |
_________________
O[L]C 2016 Swiss! - 19th June! ---- All Star Bowl XII - Teams of Stars - Sign up NOW! |
|
Hitonagashi
Joined: Apr 09, 2006
|
  Posted:
Dec 13, 2011 - 12:21 |
|
I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven.
The other issue is that, at least among most of the higher ranking coaches I watch, 2-0 is the ideal scoreline. Whatever race I'm playing with/against, I usually aim for a 2-0. 3-0 means that he sucked at offense, and higher usually means I blew him out with cas or dice. |
|
|
VoodooMike
Joined: Nov 07, 2010
|
  Posted:
Dec 13, 2011 - 12:42 |
|
Hitonagashi wrote: | I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven. |
It does mean, though, that you have to win with those goblins to be acknowledged by a win-only system as having some coaching skill, and to make that happen you'd have to be very selective about who you play (assuming you're given a choice).
Obviously there's no need to care an ounce about any sort of rating system, assuming there is no tangible benefit to having a high rating, but there will always be people who do care about any ranking, nomatter how silly it happens to be. Those people will be pushed to play only as high-success teams, and to choose (when possible) low-success teams to play against. |
|
|
dode74
Joined: Aug 14, 2009
|
  Posted:
Dec 13, 2011 - 12:45 |
|
Would you take TV into account? Baseline scores for different matchups will vary with TV. |
|
|
koadah
Joined: Mar 30, 2005
|
  Posted:
Dec 13, 2011 - 12:46 |
|
Hitonagashi wrote: | I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven. |
If they were reset for CRP some of the matchups are probably taking a while to fill with data.
Hitonagashi wrote: |
The other issue is that, at least among most of the higher ranking coaches I watch, 2-0 is the ideal scoreline. Whatever race I'm playing with/against, I usually aim for a 2-0. 3-0 means that he sucked at offense, and higher usually means I blew him out with cas or dice. |
That probably holds for most coaches. 1-0,2-1,2-0 are just fine. Anything else suggests to me that someone got lucky. |
_________________
O[L]C 2016 Swiss! - 19th June! ---- All Star Bowl XII - Teams of Stars - Sign up NOW! |
|
VoodooMike
Joined: Nov 07, 2010
|
  Posted:
Dec 13, 2011 - 13:05 |
|
dode74 wrote: | Would you take TV into account? Baseline scores for different matchups will vary with TV. |
There's no solid data to say that is the case, but I suspect it's true, given the overal TV difference/success trend though I hear that is likely the reverse itself any day/TV now!.
Ultimately, it'd be slicing the data pretty thin, and making for much wider Z ranges, which would make the rating distribution much flatter. Using an average across all TVs should cover the majority of games being played - if the ratios were routinely recalculated then it would begin taking into account higher TV differences (albeit weighted in favour of TV differences that most often occur) as they began being played more often, assuming they are.
koadah wrote: | That probably holds for most coaches. 1-0,2-1,2-0 are just fine. Anything else suggests to me that someone got lucky. |
That's the thing with luck - on a long enough timeline it will be against you about as often as it is for you, and that should balance things out. |
|
|
koadah
Joined: Mar 30, 2005
|
  Posted:
Dec 13, 2011 - 13:06 |
|
VoodooMike wrote: | Hitonagashi wrote: | I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven. |
It does mean, though, that you have to win with those goblins to be acknowledged by a win-only system as having some coaching skill, and to make that happen you'd have to be very selective about who you play (assuming you're given a choice).
Obviously there's no need to care an ounce about any sort of rating system, assuming there is no tangible benefit to having a high rating, but there will always be people who do care about any ranking, nomatter how silly it happens to be. Those people will be pushed to play only as high-success teams, and to choose (when possible) low-success teams to play against. |
Though if you get nothing for beating goblins etc many will refuse to play them in [R] and would give people the hump in [B ].
Maybe we're better off calling this something other than CR. MR?
The goals may not be the same as the goals for CR. |
_________________
O[L]C 2016 Swiss! - 19th June! ---- All Star Bowl XII - Teams of Stars - Sign up NOW! |
|
JimmyFantastic
Joined: Feb 06, 2007
|
  Posted:
Dec 13, 2011 - 13:10 |
|
To be fair to dode I think he meant chaos vs dwarfs where both teams are 2k+ chaos are big favourites whereas at 1k tv the dwarfs are favourites. Therefore someone could exploit your system by sweetspotting to take advantage of this kind of thing. |
_________________ Pull down the veil - actively bad for the hobby! |
|
pythrr
Joined: Mar 07, 2006
|
  Posted:
Dec 13, 2011 - 13:12 |
|
what's wrong with Christer's system? |
_________________
|
|
|
| |