Tuesday, February 25, 2014

A measure of effectiveness.

       Several of my recent posts have generated rather heated discussions.  Both because of my defense of unconventional builds, and my habit of seeing things differently.  But there is one common thread.  When you design a build to do damage, you are wrong because high damage means sloppy mech destruction.  A lower damage count, many times, results in more kills.  This cannot be denied.

       The other side of the coin is that when you look at kills, that number may only be high because you are stealing kills from teammates, or you are running a glass cannon, and despite your impressive count, your team loses.  This also is true.  Your win/loss ratio could be used as a measurement, but to us PUGgers, due to the variability of teammates, and random disconnects, it really doesn't mean much.  So where to begin?

       While the tendency is to make a build, play test it, and bounce it off other players for opinions, that method is far from scientific.  I want something based on gathered data, that can conclusively show a build, mech, or pilot, is good, or bad.  There are more variables to cope with than can possibly be gathered together, so which ones should be the center of focus.

       In my line of work I deal with several regulatory agencies.  My department must keep track of several metrics, each of which tell only a part of the story, but together, tell everything.  Preventive maintenance goes up, correctives go down.  Things like that.  It would seem, at first, that the same dynamic could be captured here.

       I have previously drawn up spreadsheets with odd combinations of metrics expressed at ratios, because such things can be quite useful.  Damage per second / heat per second, is a good example of this.  Damage per ton is another.  I felt I could come up with some magic ratio that would indicate your effectiveness as a pilot, your skill as a mech designer, or at least the effectiveness of a build.

       After all, the kills/deaths ratio doesn't tell the story completely.  And, as it has been painfully pointed out to me, the job of some mechs is to wreak havoc and die noisily in the center of the field.  Drawing attention to yourself, while your team either picks apart the enemy, or captures objectives.

       At first glance it would seem like kills/damage might work.  Using a great game and a terrible one, to maximize the spread, I came up with (7 kills / 1208 damage {I know right, great game.  Not typical though unfortunately} x 100 for usability) yields 0.579 for a match in my "terrible" atlas build.  The same math for a bad match in my shiny new CN9A(C) yields (1 kill /200 damage, x 100) 0.500

      Clearly this ratio does not tell much of anything, since one of my best matches, and one of my worst, are separated by such a small amount.  Maybe that few hundredths (Ten thousandths, really) means a lot.  My experience likely is not indicative of everyone else's.  Also, matches can be won or lost on the capture of objectives.  Wins pay better, all else being equal.  So that guy in the Jenner didn't kill anyone, he did win you the match.  His ratio couldn't even be calculated, but without him, you all lose.

       The more I thought about it, the more it seems it can't be done.  When you think about it, if there was one ratio by which an effective assault pilot could be rated, would that ratio be valid for a light pilot?  If the effectiveness of a medium is judged by the kills/damage ratio, does that hold up for a heavy, or a light?

       In the absence of any reasonable, or even mildly irrational, metric to measure effectiveness as a pilot, or the effectiveness of a build, there is only one, long-term, way to measure your efficacy.  The win/loss ratio.  If you have been doing your part, regardless of poor-performing teammates, you will win matches.  This is because bad players will, over the long-run, appear on both teams, more or less, equally.

       But then ELO is always seeking to keep that win/loss ratio close to 1.  Whether it works or not; it does invalidate the win/loss ratio as a usable metric.  So here we are, back at the beginning, bouncing ideas off of each other, and having no real numbers.  Not for the PUG world anyway.

No comments:

Post a Comment