42 coaches online • Server time: 15:32
Forum Chat
Log in
Recent Forum Topics goto Post Secret League Americ...goto Post DOTP Season 4goto Post Skittles' Centu...
SearchSearch 
Post new topic   Reply to topic
View previous topic Log in to check your private messages View next topic
VoodooMike



Joined: Nov 07, 2010

Post   Posted: Dec 13, 2011 - 10:44 Reply with quote Back to top

I'll preface this first by saying this isn't an objection the CR, or a suggestion to replace it - this is merely an idea based around numbers I know to be available, and the frequent criticisms of systems like ELO as far as determining a player's "skill" in games that involve things like chance, opponent selection, and so on.

The base idea is this - using aggregated data for past racial matches, a baseline average for single-game performance is established. This means a different baseline for "Amazon vs. Chaos Dwarf" and "Khemri vs. Wood Elf" and so on. Most likely it would be based on the average number of touchdowns scored by each team in matchups, but maybe other things would work (maybe multiple things? TDs seem to be the primary "win" measurement).

Now, after each game the match result is compared with that baseline data, and a team's overall standardized Z score is modified by the average of the weighted Z distances (standard deviations) from the baseline data the two teams achieved.

In plainer english, if the average number of touchdowns in a match between, say, Goblins and Dwarfs is Goblins 0, Dwarfs 4, and you have a matchup with your goblin team against a dwarf team and earn a 1-3 loss, you've actually done BETTER than average and the opponent has done WORSE than average, which means that in spite of having lost the match, your rating goes up and his goes down.

The main issue with win-based ratings as far as assessing skill, is that it assumes balanced matches, and we know for a fact that teams aren't all balanced against one another. Similarly, since your rating always goes up with a win, and down with a loss, people are prone to avoiding matches they think they'll lose if they're concerned with their rating, especially if they're facing someone with a lower rating (where the penalty for loss is quite a bit higher than the benefit of victory).

So under a system like this you could actually rise to the top of the rankings as a halfling coach by simply doing much better with your halflings than most other people can, and you can CLPOMB a hundred lowball victories and it count for very little, which on some level seems to be what people think a rating of coaching skill should allow.

Anyway, this is mostly an idea for discussion. I don't expect it to be implemented (though feasibly we could create a mock-up of the system to explore it).
Calthor



Joined: Jan 24, 2006

Post   Posted: Dec 13, 2011 - 10:49 Reply with quote Back to top

This is partly a good idea, imo. Except for the fact that if one team wins, and the other loses, the rating should go up for the winner, never down (an Elf team should not be punished for 'grinding' his way to a victory, so to speak).

Yet the influence of racial modifiers / racial match-ups (eg. the baseline idea) does have potential.
Stonetroll



Joined: Jun 30, 2005

Post   Posted: Dec 13, 2011 - 11:01 Reply with quote Back to top

If you are trying to make a better CR formula by analyzing every micro aspect of the game, you should really count the number of KO/cas suffered against in there too. It takes much more skill and luck to win games when your players are being wiped off the pitch. But then again I really question if it is meaningful to try make a "perfect" coach rating, at the moment CR works as a nice ballpark approximate of player skill. There are more important stuff to do, like bombardiers and other missing skills.
Mateuszzzzzz



Joined: Feb 26, 2006

Post   Posted: Dec 13, 2011 - 11:19 Reply with quote Back to top

You win game by scoring 1 more then opponent. I dont think scoring more is any measure of better skill.
DanielM75



Joined: Jan 18, 2009

Post   Posted: Dec 13, 2011 - 11:27 Reply with quote Back to top

Agree with comments.
Take the recent FC match that was 8-7 vs a similar match that wasn't OTD silly as was a strategic loose cage 3-2 win.
You can't possibly assess coaching skill via that.
VoodooMike



Joined: Nov 07, 2010

Post   Posted: Dec 13, 2011 - 11:41 Reply with quote Back to top

Calthor wrote:
This is partly a good idea, imo. Except for the fact that if one team wins, and the other loses, the rating should go up for the winner, never down (an Elf team should not be punished for 'grinding' his way to a victory, so to speak).

But is it really a sign of skill for a Skaven team to score against a Halfling team? Is it a sign of skill for a Skaven team to beat a Halfling team? That's sort'v the point of racial comparisons - some teams you simply expect poor performance from, and that's not a matter of the coach being bad, or the opposing coach being good - it's a matter of some races being better than others.

Can we really say that a coach's skill is at fault if his Goblins lose to Chaos Dwarves? If we always assign a rating decrease with a loss, and an increase with a win, that IS what we're saying.. and because of it, we encourage those Chaos Dwarves to seek out Goblin teams to play against, since it builds their rating even if it results in a reasonably easy victory.

Stonetroll wrote:
If you are trying to make a better CR formula by analyzing every micro aspect of the game, you should really count the number of KO/cas suffered against in there too.

Not trying for perfect, just brainstorming around the common complaints I see about ELO-style systems being used to determine coach skill. KO/cas is as valid a measure as any, but you do have to play to achieve casualties to get them, which would mean the system would penalize people who simply played to score. We can assume that everyone, on some level, plays to get more TDs than their opponent since that's what makes them the winner.

Stonetroll wrote:
There are more important stuff to do, like bombardiers and other missing skills.

That's nothing you or I can do, but this is. They aren't mutually exclusive because of that.

DanielM75 wrote:
Take the recent FC match that was 8-7 vs a similar match that wasn't OTD silly as was a strategic loose cage 3-2 win.
You can't possibly assess coaching skill via that.

When you have enough matches to use for your baseline, and you weight any changes to rating based on total games played by the team whose rating is being changed, then even if there are wild and unusual game results, they don't change things dramatically.

Remember also that if both teams score a ton of TDs, that will balance out in the calculation, since it is being compared to the typical ratio of TDs for that racial matching, and how far removed from that a specific match was.
koadah



Joined: Mar 30, 2005

Post   Posted: Dec 13, 2011 - 12:00 Reply with quote Back to top

VoodooMike wrote:

Anyway, this is mostly an idea for discussion. I don't expect it to be implemented (though feasibly we could create a mock-up of the system to explore it).


The match data is freely available so anyone who is interested can create any ranking they want.

Someone could probably knock up a (greasemonkey?) script to suck it into the homepages of people who want it.

_________________
Image
O[L]C 2016 Swiss! - April ---- All Stars - Anniversary Bowl - Teams of Stars - 13th March
Hitonagashi



Joined: Apr 09, 2006

Post   Posted: Dec 13, 2011 - 12:21 Reply with quote Back to top

I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven.

The other issue is that, at least among most of the higher ranking coaches I watch, 2-0 is the ideal scoreline. Whatever race I'm playing with/against, I usually aim for a 2-0. 3-0 means that he sucked at offense, and higher usually means I blew him out with cas or dice.
VoodooMike



Joined: Nov 07, 2010

Post   Posted: Dec 13, 2011 - 12:42 Reply with quote Back to top

Hitonagashi wrote:
I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven.

It does mean, though, that you have to win with those goblins to be acknowledged by a win-only system as having some coaching skill, and to make that happen you'd have to be very selective about who you play (assuming you're given a choice).

Obviously there's no need to care an ounce about any sort of rating system, assuming there is no tangible benefit to having a high rating, but there will always be people who do care about any ranking, nomatter how silly it happens to be. Those people will be pushed to play only as high-success teams, and to choose (when possible) low-success teams to play against.
dode74



Joined: Aug 14, 2009

Post   Posted: Dec 13, 2011 - 12:45 Reply with quote Back to top

Would you take TV into account? Baseline scores for different matchups will vary with TV.
koadah



Joined: Mar 30, 2005

Post   Posted: Dec 13, 2011 - 12:46 Reply with quote Back to top

Hitonagashi wrote:
I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven.


If they were reset for CRP some of the matchups are probably taking a while to fill with data.

Hitonagashi wrote:

The other issue is that, at least among most of the higher ranking coaches I watch, 2-0 is the ideal scoreline. Whatever race I'm playing with/against, I usually aim for a 2-0. 3-0 means that he sucked at offense, and higher usually means I blew him out with cas or dice.


That probably holds for most coaches. 1-0,2-1,2-0 are just fine. Anything else suggests to me that someone got lucky.

_________________
Image
O[L]C 2016 Swiss! - April ---- All Stars - Anniversary Bowl - Teams of Stars - 13th March
VoodooMike



Joined: Nov 07, 2010

Post   Posted: Dec 13, 2011 - 13:05 Reply with quote Back to top

dode74 wrote:
Would you take TV into account? Baseline scores for different matchups will vary with TV.

There's no solid data to say that is the case, but I suspect it's true, given the overal TV difference/success trend though I hear that is likely the reverse itself any day/TV now!.

Ultimately, it'd be slicing the data pretty thin, and making for much wider Z ranges, which would make the rating distribution much flatter. Using an average across all TVs should cover the majority of games being played - if the ratios were routinely recalculated then it would begin taking into account higher TV differences (albeit weighted in favour of TV differences that most often occur) as they began being played more often, assuming they are.

koadah wrote:
That probably holds for most coaches. 1-0,2-1,2-0 are just fine. Anything else suggests to me that someone got lucky.

That's the thing with luck - on a long enough timeline it will be against you about as often as it is for you, and that should balance things out.
koadah



Joined: Mar 30, 2005

Post   Posted: Dec 13, 2011 - 13:06 Reply with quote Back to top

VoodooMike wrote:
Hitonagashi wrote:
I believe racial factors and TV differences are taken into account in the current algorithm. Hence, while I lose CR for losing with goblins for dwarves, at least in the old system, I don't lose as much as I would if I was playing skaven.

It does mean, though, that you have to win with those goblins to be acknowledged by a win-only system as having some coaching skill, and to make that happen you'd have to be very selective about who you play (assuming you're given a choice).

Obviously there's no need to care an ounce about any sort of rating system, assuming there is no tangible benefit to having a high rating, but there will always be people who do care about any ranking, nomatter how silly it happens to be. Those people will be pushed to play only as high-success teams, and to choose (when possible) low-success teams to play against.


Though if you get nothing for beating goblins etc many will refuse to play them in [R] and would give people the hump in [B ].

Maybe we're better off calling this something other than CR. MR? Wink
The goals may not be the same as the goals for CR.

_________________
Image
O[L]C 2016 Swiss! - April ---- All Stars - Anniversary Bowl - Teams of Stars - 13th March
JimmyFantastic



Joined: Feb 06, 2007

Post   Posted: Dec 13, 2011 - 13:10 Reply with quote Back to top

To be fair to dode I think he meant chaos vs dwarfs where both teams are 2k+ chaos are big favourites whereas at 1k tv the dwarfs are favourites. Therefore someone could exploit your system by sweetspotting to take advantage of this kind of thing.

_________________
Pull down the veil - actively bad for the hobby!
pythrr



Joined: Mar 07, 2006

Post   Posted: Dec 13, 2011 - 13:12 Reply with quote Back to top

what's wrong with Christer's system?

_________________
Image
Image
Display posts from previous:     
 Jump to:   
All times are GMT + 1 Hour
Post new topic   Reply to topic
View previous topic Log in to check your private messages View next topic