+ Reply to Thread
Results 1 to 17 of 17

Thread: Interpreting Parri's results and methodology

  1. #1
    Join Date
    Nov 2008
    Posts
    177

    Interpreting Parri's results and methodology

    This thread linked below is very long but there's so much data, I wanted to start a discussion about how to properly interpret it, as well as ask if there are potential problems with the methods he used.

    World of Warcraft (en) Forums -> Pre 3.1 Exhaustive tank comparisons.

    I'm a little pressed for time (here at work) but here's basically what I came away from the article with:

    --Druids are currently beasts at tanking, have a longer Time-To-Live by a factor of almost 1.5 to 2 against the bosses that hit hardest, and their "worst-case scenario" isn't bad at all, since they're not bound by cooldowns very much.

    --DK avoidance is very high (he claims as high as Sunwell-level but doesn't offer numbers) and in most situations their mitigation is on par with druids. Their worst-case scenario though is as bad as warriors and paladins, albeit much less likely to occur.

    --Paladins and Warriors have a slight edge on mobs that hit fast and not all that hard. IMO it's not really the easy, soft-hitting mobs that I should worry about.

    --Threat and gimmick fights like Sarth + 3 were not discussed and I don't really want to discuss them here, either.

    What I'm looking for from replies is (dis)agreement with my analysis and a better understanding of how he tested this. I don't know how this ThinkTank program works and how applicable its results are to the actual game.

    Pylae

    P.S. Part of the reason I'm so interested in this is that I do have a druid sitting at 70. If druids have twice the time-to-live without a heal that warriors do, that IMO would justify leveling the druid. Especially since I saw a blue bost saying shield block won't be fixed in the 3.1 timeframe.

  2. #2
    Join Date
    Jun 2008
    Location
    USA
    Posts
    114
    I too would be interested in seeing some discussion on this. I read over his thread a few days back and didn't have time to fully dive into the charts and methodology.

  3. #3
    Join Date
    Aug 2008
    Posts
    340
    Seems like very interesting work. I wish I had time (or possibly cared enough) to look into it in more detail.

    One thing I would say though: if your motive is to find the ultimate tank to play - ultimate defined by time-to-live - then, inspite of your interest in this very sophisticated analysis - I think you're barking up a tree that doesnt exist. Blizzard have no interest in creating the best tank - if you prefer one style over another, then go for it. Thats what I do. Time to live is a nice concept, but it doesnt say everything there is to say about real gameplay. You need to factor in time-to-live comparisons with many other complex & subtle factors - the biggest of which are the human ones over which blizzard has no control. Oh - and if thats not enough, remember the game is always changing. Feral druids may (will) look very very different 9 months from now.
    Last edited by Stearns; 03-24-2009 at 03:08 PM.

  4. #4
    Join Date
    Jul 2008
    Posts
    248
    Wow, so after 2 hours of reading, analyzing, and scratching my head...

    I think I understand what this all means. Now. With that said. This is also your warning, wall-o-text commencing. I'll try and make it as legible as possible.

    In summary, I think it means:

    Bear
    DK
    Pali
    Warrior

    With obvious exceptions to the rule, IE: trash...

    It's one hell of a start on a possible fix for block, and by the patch notes, they are already addressing the DK Avoidance-off-the-DR issue with Blade Barrier, and working on druids vastly superior TTL factor, with their armor/hp nerf.

    While I don't agree that blizzards fixes are for the best, it's a step. The problem is with newer content, and an expanding gear repuitare, static data may become skewed. A warrior's armor values, along with palies, is going to go up, as our BnB, our shield, is only on the first tier. This (armor) is a very important factor in determining a classes TTL. A druid/DK armor values will go up also, due to higher ilvl budgets, but the point being is the gap will close.

    Now. Obviously there are so many more factors in all content, as stated not all fights are static like patchwerk. You get multiple factors, and as stated, fights like 25Sarth3D aren't even mentioned. This makes a HUGE factor for guilds wanting to clear ALL content, in picking their tanking core. For anyone who listened to the podcast recently, it's not about tanking the big boss, it's about who everyone else wants too. And no matter how good of a tank you are, if your guild is set on the goal for clearing all content, their going to pick the best tank for the job, especially progression. Bears are FAR superior in ALL aspects of this. Sure, we can let the warrior tank it when we have it on farm, but that's not what we want. As tanks, we want to be the first to tank it for our guild, we want pride in knowing our guild trusted us with that duty.

    So we work toward creating a baseline. Obviously some classes are going to outperform, or if not outperform, make those first few attempts easier on certain bosses. But the problem I'm seeing is we are not buffing/nerfing according to the correct baseline. If your going to balance around a baseline, that baseline cannot be at the top, or certain tanks are going to fall short, and you cannot balance around the bottom, or certain tanks are going to painfully outperform others.

    Balance must be maintained around the middle. A good chunk of content must be tuned to this middle, not all, but probably a majority. And the tanks must be tuned accordingly also, to this baseline. So that yes, certain tanks will be better suited for specific encounters. However on the opposite end of the spectrum, you don't have any tanks who are simply not capable of performing.

    It is blizzards stated goal of balance between the tank classes, however currently, that point of balance is nowhere to be found. Not saying that there isn't some balance! I'm saying the baseline, to where they are balancing too, is off.

    In order to properly balance, you must first set your baseline. For both encounters, and for tanks. Once this is done, the work is not done. If one tank should exceed, or fall short, of these baseline aspects, how to fix them is of the utmost importance. You cannot say, "Oh, these two classes are unable to tank this boss, lets nerf the encounter." All this creates is problems at the other end of the spectrum, and then the tanks at the top get nerfed down because they are now performing WAY over the baseline. The answer is not always in the baseline of the encounter perhaps, as some bosses are designed to be harder than others, IE: Sarth is harder than Vault. So you must then balance around class baseline.

    Now, in balancing around class baseline, you must pick a gray area, where is to be your mean. If a tank is performing outside of this mean, with excess of two times better, or worse, then moving the mean isn't always the option. You don't nerf the hell out of one class at the top, I'm alright with them performing on this encounter better. You bring them down slightly, bring the lower classes UP slightly, and get everyone within the mean baseline again.

    Now the true problem lies in overall balance, you fix one encounter to get everyone within the baseline, now with those fixes, what happened to the other 20 boss fights? Did that class that received a buff now outperforming others? Is the tank you nerfed un-capable of tanking that encounter now? This is why 'nerfing' or 'buffing' a class never works. You change too much, and well; you've changed TOO MUCH! Your overall balance is going to be thrown off.

    My proposal is this. Start with what we have. We've had one hell of a beta so far. Now lets work on it ^_^. Developers should have a decent baseline now, so adjustment needs to be done by Blizzard. For the encounters, tune the tank baseline around that, and do small tweaks for the future. Large class changes alienate people, hell, my warrior isn't even the same character as he was pre 3.0. It's not bad, it might not be good, but it is what it is. Moving forward is key yes, but lets not forget what we left behind.

    Ladies and gentlemen, we are having a repeat of BC, essentially. For tanks, bears were OP at the start, and then became the worst due to nerf's and the mechanic of crushing blows. At high gear levels, avoidance became too powerful, allowing classes to hit the uncrushable cap, or what is now known as the block cap. We have a new tank in our midst, which is really making warriors and palies rethink what the concept of avoidance is, and bears are going to be learning about block.

    While each class should have it's strengths and weaknesses, whether it be HP, armor, avoidance, TTL, or cooldowns, there must be a balance.

    The answers lies in where we balance these classes, and HOW we balance them. Blizzard doesn't want to lose it's customer base, and big changes hurt your community.
    November 23, 2004 8:27:03 AM - Glomgore 10+ Years of tanking? Priceless

  5. #5
    Join Date
    Dec 2007
    Posts
    2,055
    Fundamentally, the results are tainted as he used Rawr to set up his profiles without knowing how to use Rawr, or recognizing that rawr is not WoW. (Rawr's class models are not 100% accurate.) He made stupid mistakes like putting the 26 agi to 1H enchant on shields, etc.

    With his ~base~ being this screwed up, I fail to perceive how he could accurately craft a TTL simulation as he doesn't know enough about the game mechanics to recognize his own mistakes in rawr, and without solid grasp of game mechanics any simulation he crafts will not be accurate.

    When I finished reviewing the base information and the methodology, I discarded the results out of hand. Flawed test is flawed.

  6. #6
    Join Date
    Nov 2008
    Posts
    177
    If he did make mistakes like that it's cause for concern, but I don't reach the conclusion that his results have no merit. Or I'd want to see more indications of errors, particularly in the combat simulation.

    Let's also use common sense...do his test results jive with what's actually happened/is happening? Asking if druids and DKs are better on Sarth + 3 and Malygos are really biased questions, but who is getting the best results tanking on the PTR (although also biased since that has changes in it).

    Pylae

  7. #7
    Join Date
    Dec 2007
    Posts
    2,055
    Well, it doesn't help that 3.1 is changing the way druid stam works, removing DK's Str: parry conversion, giving paladins an extra 3% mitigation outright and altering warrior stances.

  8. #8
    Arguing about his RAWR setup is one thing. Arguing about the ThinkTank program is quite another. The sub-half-percent differences in starting attributes is not significant if his program is accurate, given the results that he summarized. Seeing as how he made the source for it available to look at, we need feedback specifically at whether or not the assumptions and calculations that result are to be trusted. Unfortunately I can't really help in that department, but I thought maybe restating/refocusing the discussion might help others.

  9. #9
    Join Date
    Jul 2008
    Posts
    248
    Quote Originally Posted by Pylae View Post
    If he did make mistakes like that it's cause for concern, but I don't reach the conclusion that his results have no merit. Or I'd want to see more indications of errors, particularly in the combat simulation.

    Let's also use common sense...do his test results jive with what's actually happened/is happening? Asking if druids and DKs are better on Sarth + 3 and Malygos are really biased questions, but who is getting the best results tanking on the PTR (although also biased since that has changes in it).
    ^^^^^ This.

    100% accurate or not, they do provide a potential eye opening discussion. Even if the concepts are only partially true, it's still something I think can be addressed.

    As far as the PTR goes, I can only speak from a warriors standpoint of, the only change to my stances is a 5% threat increase from the lessened neg % on the defensive stance. Warriors are gonna need ALOT more help than with threat if his testing has any even close to accurate numbers.
    November 23, 2004 8:27:03 AM - Glomgore 10+ Years of tanking? Priceless

  10. #10
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    3,466
    His methodology is flawed. He only models physical fights, which naturally propels druids forward. This isn't wrong as long as you note it, which he doesn't -- he even claims it's "the most in-depth comparison that has ever taken place", which is blatantly false.
    He disregards the extra avoidance a dual-wielding boss has been known to grant without any hard proof that it is gone.
    His Warrior build is rather bad. He claims 15/5/51 builds are an "aberration", yet his own build has less mitigation than a standard 15/5/51 build, as it doesn't have Improved Thunder Clap.
    And, as Alent notes, he uses Rawr poorly.
    風林火山陰雷

    ಠ ,ಠ
    Dovie'andi se tovya sagain - it's time to roll the dice

  11. #11
    Join Date
    Aug 2007
    Posts
    913
    Well, the biggest problem I have with his benchmarks is that his metrics seem to be inherently flawed. When time to live is significantly less than the length of a global cooldown, the survival of any tank will depend on pure luck that heals happen to land at the right time. Of course, with the amount of incoming damage he describes, (lowest) TTL shouldn't be that low: Even with parry haste, two consecutive boss attacks can't possibly occur that fast at the slower attack speeds, and there's nothing else in the model he describes where a tank can be killed except by consecutive auto-attacks. So, the numbers don't add up.

    In general, his assumption of a boss that only ever uses auto-attack is flawed, too. Practically every boss has instant attacks, either physical or magical.

    I'm also a bit put off by his combative tone, which mixes a lot of (aggressively expressed) opinion in with his data.

  12. #12
    Join Date
    Nov 2008
    Posts
    177
    I think one of his metrics is lowest TTL in four days. I'm guessing that means that in a simulation of the boss beating on you for a solid 96 hours. To me that basically means "how long can you live when you're smacked as hard as the boss can hit without avoiding anything while all your cooldowns are down."

    I won't have much time at work today but if anyone has thoughts on just what the columns in those spreadsheets mean I'd like to hear em. I did notice that some of the TTLs are like 700 (ms) which you're right, doesn't make any sense.

    Pylae

    P.S. Is a swing time of 1.0 mean an even second between swings?

  13. #13
    Join Date
    Dec 2007
    Posts
    2,055
    Quote Originally Posted by Ghladum View Post
    Arguing about his RAWR setup is one thing. Arguing about the ThinkTank program is quite another. The sub-half-percent differences in starting attributes is not significant if his program is accurate, given the results that he summarized. Seeing as how he made the source for it available to look at, we need feedback specifically at whether or not the assumptions and calculations that result are to be trusted. Unfortunately I can't really help in that department, but I thought maybe restating/refocusing the discussion might help others.
    There's no refocusing necessary. If he can't setup rawr correctly, he can't set up ThinkTank correctly. I agree, in some cases the differentiation between his rawr files and reality is somewhere in the range of 10~50 hp. big deal. It's that his misuse of one tool means he probably misused the other tool.

    Also, Roana describes problems with the testing very well:

    Quote Originally Posted by Roana View Post
    In general, his assumption of a boss that only ever uses auto-attack is flawed, too. Practically every boss has instant attacks, either physical or magical.
    In short - the only boss that meets the criteria of his testing is Patchwerk - and to be generous maybe Maexxna. Thorim is the closest example of this kind of damage in Ulduar from what I've heard and he has an instant cast attack fowling things up - unbalancing strike. By discarding everything that makes a raid boss a raid boss, he's created a hypothetical scenario that cannot exist.

    Also, as Norrath points out, he's not a very good warrior. He considers threat builds an abberation and he tanks with Defender's code.

  14. #14
    Join Date
    Nov 2008
    Posts
    177
    Here's a good question...do we have links to any better testing?

    Pylae

    P.S. And since Patchwerk is the one giving me trouble, I consider that relevant! (Although it arguably has more to do with teaching the entire Construct-25 wing to PUGs--not so excited about that.)

  15. #15
    Join Date
    Dec 2007
    Posts
    2,021
    It doesn't matter if we have better testing or not. Taking anything, if it is flawed (not arguing if it is or isn't), is not a good route to take. Kind of like how two wrongs don't make a right.

  16. #16
    Join Date
    Nov 2008
    Posts
    177
    I agree if it's flawed it's flawed, but with 11 million people playing the game there must be some other testing out there, probably even on tankspot.

  17. #17
    Join Date
    Dec 2007
    Posts
    2,055
    Quote Originally Posted by Pylae View Post
    I agree if it's flawed it's flawed, but with 11 million people playing the game there must be some other testing out there, probably even on tankspot.
    As memory serves, Rak's guild did extensive TTL testing on the Patchwerk tank test dummies on PTR. I do not remember where that was posted, but it was referenced heavily in the most recent tanking topics post.

+ Reply to Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts