+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 22

Thread: An attempt for a tanking simulation

  1. #1
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29

    An attempt for a tanking simulation

    This thread concerns a tanking simulator I'm programming. It is directly inspired by this thread http://www.tankspot.com/showthread.p...g-mechanics%29 which is an incredibly interesting discussion on the relative value of EHP (effective health points) versus avoidance. Basically I implemented survival talents for every tanking class, a boss will lands several attacks on the tank while some healers will try to leave him in life.


    Progress status : Not everything is ok, the code is still awful (and worth, in french :P), contains eventually some errors. Statistical models are not precise enough but still the whole has some value I think.


    What it can do : Basically, it evaluates the number of deaths under certain circumstances. It can be :
    • Variation of classes (Paladin, DK, Warrior, Druid)
    • Variation of armor, Hps, dodge, parry, strength(used for blocking), block value, agility.
    • Variation of heals, particularly introducing failed heals when the healer must move to survive for example (thanks to Kojiyama)
    • Variation of boss attacks
    So we can simulate a fight to obtain an approximation of :
    • Stamina versus other characteristics, or more globally every stat versus another in a certain context.
    • The tanking value of a class versus another in predetermined context.
    • The evolution of stamina and other stats with increased or decreased boss attacks, variation of heals...
    • Nothing prevents at first glance to implement healing classes in the simulation, and thus improve the results or help determine (not empirically) which one fits the most with a boss.




    Although I do not have high opinion of my code, the main problem actually concerns data-values. By being precipitated, I made some errors (the kind I can have for hours in front of my eyes if no one else tell me) when using RAWR (which is by the way really helpful). I don't want Aggathon work and the whole discussion to be interrupted by my precipitation as results might differ a lot with small variations like the ones I'm doing here.
    (Basically, I took accidentally the same dodge rating for druid and warrior which, with diminishing returns conducts to great failure.)
    Heals and Hps are with +25% buff from ICC and dodge is reduced by 20%.


    (90000 to 14000 depending on trial)+-10000 every 1,2+-0,2 seconds
    (65000 to 90000 depending on trial)+-5000 every 4+-0,2 seconds
    20000+-0 every 1+-0,2 secondes

    Heals are following :
    10000+-2000 every 1,3+-0,2 seconds
    20000+-4000 every 2+-0,2 seconds
    3000+-500 every 0,5+-0,2 seconds
    20000+-4000 every 2,5+-0,2 seconds


    A sample of what we get if a druid/DK/paladin/warrior is subject to different boss knowing that 20% of healing is interrupted due to any cause. Marge of error due to statistic and randomness in the simulation differs from the trials. It is generally +-0,1. These results are indicating the relative equivalency between one gem of stamina and one gem of dodge while the boss attack harder and harder (in axis)


    tanking_20%.jpg




    Note that results are extremely dependent on dodge rating. I know this is confusing, but this set of result is only valid for a 600 dodge rating class, which is roughly ilvl264. (200 dodge and 2100 agility buffed for druid).


    In this trial I wanted to know if the ratio stamina/dodge under 20% failed heals (which represents movements) was evolving with the intensity of boss attacks. It doesn't seem so on these curves, in contrary of DK (0%) (absolutely no interruption) which decreases significantly. It simply means that, in Patchwerk style fights the tendency is that stamina becomes less and less important while the damage output increases.
    Keep in mind that my boss hits very hard for statistical interpretation reasons, so the values displayed are not real ones.




    You have to be vigilant with all these results as they are not confirmed. By publishing them, I simply hope to receive criticism. There's too much to think for me for the moment.

  2. #2
    Join Date
    Jan 2009
    Location
    United Kingdom
    Posts
    1,644
    Without knowing the exact calculations, it's hard to make specific comments. However, I think it's obvious how the interruption death element vs. the eventual worst-case timing convergence death element gives very different results.

    With the 0%, you pretty much see a linear relationship between the value of Stamina and the percentage of the delta margin between remaining health and the random variance in the worst case scenario. With interruption becoming the likely cause of death, avoidance gains in comparison to Stamina (in comparison to the non-interrupted case), as they both then provide an effective way to avoid the death condition.

    The interesting thing in the chart, which is often mirrored in Burst Time graphs, is the non-linear nature of the value of Stamina in scenarios where it interacts with Avoidance to increase survival. In the case where basically only Stamina matters (the 0% graph) the value of Stamina consistantly drops as the ability to survive N extra death conditons decreases.

    However, in the interruption-driven model, while the base value of Stamina relative to Avoidance is lower, it does not decrease in value at the same rate as in the Stamina-centric condition. In fact, you can see the value of Stamina actually increase in many of the graphs where it would have only decreased before. That seems to represent more realistic results, as the effect of adding more Stamina and/or Avoidance is rarely a linear result.

    Anyhow, I'm sure the results will evolve with time as you iterate on the testing method. Looking forward to seeing how it ends up and perhaps some more details on the calculations.
    Maintainer of Rawr.ProtWarr theorycrafting tool. Feel free to PM suggestions or feature requests!

  3. #3
    Join Date
    Aug 2009
    Posts
    7
    Looks interesting, but I'd like to raise a few general concerns :
    -this simulator is built to test specific situations, and I don't think you will be able to make general statements based on the results (unless you spend A LOT of time simulating to get a good array of results

    -I'm afraid introducing a lot of random will make it very hard to obtain precise results, even with millions of samples, as converging speed will be greatly reduced as you add parameters or mechanics. Please try to measure precision (I don't see any simple way to do it)

    -pseudo-random mechanics : Kojiyama explained it better than I could, but beware of events that may seem random, but really aren't and discretely skew your data

    -Healing & Tanking reaction simulation : Do you intend to model proactive or reactive cooldown usage for healing and tanks? I don't know how you built your simulator, but it may not be easy to implement once it's done, think about it!

    Aside from those concerns, if the simulator is complete enough, it could be used to determine relative values on a fight-by-fight basis, which would be crazy, but there are LOTS of caveats on this track.
    On your numbers, 20% failed heals seems like a lot, wouldn't 5 or 10% be more realistic? the issue with this being random is that in reality, you often die when healer-disrupting and burst abilities stack, which this won't account for.

    Otherwise, I'm ready to help you if you need (I can survivre french code although I find it a bad idea in the first place)

  4. #4
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    To answer to Kojiyama, you're well in advance on me in interpretation of statistical results. I thank you as you said what I'd have said (I hope) far later. :P I'm currently watching the EHP/avoidance model for protection warrior on RAWR (:P) at the moment to see if our analyzes converge.


    To answer Agarwaen on each point :
    • Right, I think that even it's possible it will take some time. I'm currently simulating 15000 hours of fight per minute (without parallelism), and even with 15 minutes run, there can be some variation. I have not the knowledge for measuring precision non empirically although I can ask some friends for help.
    • Right again. Since the beginning I introduced variations in frequency of heals and boss attacks to break “reproduced sequences” but I know it may not be sufficient.
    • Concerning proactive healing and tanking, I asked a few members of my raid to share their healing experience but it's hard for me to traduce it in term of algorithm. For proactive tanking, I implemented DK deathstrike, after some testing, I did find that the best moment was 30-35% of his life. In my simulation I use 10 seconds for its cooldown. Honestly for the moment, other classes use their CDs as soon as they can. I know this is biased so I will correct it.
    I'm using another model also for failed heals. Instead of seing 20% of heals interrupted, there's a 1% chance every 0,1 second to have a 2 seconds window with no healing. I need to create HOT structure also as they are still active in movement. Results change so there's a lot of work determining which model adequate the most “reality”.


    Thank you for you commentaries.

  5. #5
    Join Date
    Jan 2009
    Location
    United Kingdom
    Posts
    1,644
    Yeah, really a more accurate healing model would need to emulate the standard behavior of each healing class.

    e.g.

    Disc Priest:
    1) Keep PW:S off cooldown (would need to implement rewards)
    2) Use Penance on cooldown, but only if the tank is below 60% health or so
    3) Consider using Prayer of Mending on movement (or on cooldown, depending on set bonus)
    4) Use Flash Heal otherwise
    5) Random interruptions that happen when PW:S or PoM are available have a smaller impact (e.g. 0.5s instead of 1.5s, since you'd use the instant while moving)

    Holy Paladins would be much simpler to model, with the ability to use Holy Shock on movement events, if it was off cooldown. However, this would dramatically lower their throughput (just not negate it).

    Resto Druids would maintain HoTs, and thus be able to heal through movement events resonably well. Swiftmend could be saved for extreme low health conditions (say < 20% for a ballpark) or moderate low-health conditions during movement (say < 50-60%).

    Obviously a lot of work to do something like that, but it would be simulating a much more normal environment. This would actually allow you to do a much more frequent chance for movement without it having as dramatic of an effect. It could also allow you to input extended movement periods if needed, if you wanted to model specific fights. (e.g. Professor Putricide, you have X% chance of a short movement every Y seconds for goo targets, slime puddles, or gas canisters, and then a 4% chance every time an orange add shows up to need to kite it--instants only--for Z seconds.)

    Getting into complex stuff, so if you want to be able to model such things, be sure to build your simulator code in a well-structured way that will allow extensibility.
    Maintainer of Rawr.ProtWarr theorycrafting tool. Feel free to PM suggestions or feature requests!

  6. #6
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    Well, as I “play” holy priest (I know they are not ideally suited for MT ^^), at least I have one, I was thinking a little on how to implement it. I need too much knowledge for other classes. Furthermore, it looked easier, I insist on the “look” . There is
    • a shield every 15 secondes
    • Fast heal
    • renovation
    until I realized I needed to implement almost all talents to be accurate, not only surge of light etc. Tanking is far easier as I only need survivals ones.
    I think I'll eventually give up simulating completely healers and only use things like class/spells and characteristics like critic/haste/intensity. Maybe one day I will implement things like surge of light and another (never say never) the rest of the talents.


    As I'm really ignorant of druid/chaman/Paladin and disc priest (although detailed above), any help would be welcome concerning healing spells used, in which order and frequency etc...


    I think I have found some interesting tools for my statistics. I learned how to compute the pearson coefficient. (For stats fans, linear regression are around 0,995, which will simplifies everything as it is a straight) and the standard error is relatively low which means I will not need hours of computing for each test.


    Anyway, I will provide some tests and interpretation (with simple healing) as soon I will have finished my code with "proactive tanking" and verified talents implementation.

  7. #7
    Join Date
    Jan 2009
    Location
    United Kingdom
    Posts
    1,644
    I don't think the healing simulation has to be 101% accuate, but more as long as it resonably approximates what the healer would generally attempt to do it should be close enough to give a decent result.
    Maintainer of Rawr.ProtWarr theorycrafting tool. Feel free to PM suggestions or feature requests!

  8. #8
    Hey I started something very similar to this awhile back and what I determined I needed to do to get real world outcomes was to have "modules" for specific bosses and replicate their actual attacks. I used Festergut for my test module and simulated tanking the encounter. Since the critical moments of death are usually boss specific it's why I was trying to head in that direction. I had a hard time making useful information on pure damage reduction over time type scenarios since those don't usually help since it's not deadly and there is typically lots of overhealing.

    Perhaps some spike damage type scenarios of different types worked in would be helpful.
    Mkdk - Chambers of Shaolin
    deathknight.info for DK forums | Follow me on Twitter

  9. #9
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    I wish to provide two sets of data.
    In the first healing an boss attacks remain constant, while the healing failed factor is evolving. Healing failed factor is the percentage each 0,1 second that a no healing status occurs for the next two seconds. The four values are 0,2, 0,4, 0,6 and 0,8 which roughly means that 4/8/12/16% of healing is failed.


    14_07_evolution.jpg
    The curve is the ratio between the coefficient adding one gem of stamina and one gem of dodge. Saying otherwise, the value represents the interest of stamina versus dodge. A value of 1 indicates that they are equal.
    I'm still looking for explanation, I believe that druid is less affected by diminishing return when gemming dodge. High value of DK seems to come from his health based abilities such as connexion or death strike.
    Note that boss attacks are not realistic of what wow involves. My tanks are dying roughly every 90 seconds which is important and the healing system is not realistic either. Don't charge me if you change your gems and dies more frequently.
    Standard error is 0,1.




    The second graph is a class comparison with constant healing and varying boss attacks(those starts relatively weak to end at a moderate intensity). I took the 16% healing failed. (with 0 results are different). The value is the number of death.
    14_07_evolution_class.jpg


    It seems that the DK, with its cooldown, his deathstrike provide an interesting survivability although the difference in these conditions is not very high. Ardent defender doesn't make the paladin very strong as the frequency of death is still toward one for every 90 seconds. I still need to improve proactive tanking.


    I will try to collect data on boss from the different logs I can find from worldoflogs. Real data are always better.

  10. #10
    Join Date
    Sep 2009
    Posts
    5,532
    Can you explain a little bit more what your various axis represent? It's a little confusing to me.
    "If the world is something you accept rather than interpret, then you're susceptible to the influence of charismatic idiots." -Neil deGrasee Tyson

    Twitter @Aggathon || @Tankspot || Twitch.Tv/Aggathon

  11. #11
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    Sure, on the first graph, axis represents failed heal although we have (blame on me I don't really know how to use openoffice :/) to multiply by four to obtain an estimation of failed heal. 1 on the axis means 4 and 4 means 16.


    On the second graph, I will detail how the boss attacks. He have three attacks with different frequency. One of 60k, one of 47,5k and one of 25k of physical damage. For each trial (basically if we increment axis by one), there is a 5k gain on the first and a 2,5k on the second attack. So on the far right we have 60+9*5=105k attacks and 47,5+9*2,5=70k attacks. The third attack doesn't change.





    Another very important thing to do is to implement magical damages and I will simulate a Festergut fight (without a good model of healing though).

  12. #12
    Join Date
    Jun 2008
    Posts
    718
    I have been programming a DK simulator for quite a while now which is quite different to rawr in that what I am doing is reversing alot of calculations I have this gear what spec best suits it. DK's have some of the wildest talents and what the sim does is attempt alter talents to cover shortcomings in gear.

    So if my gear is say lacking in threat stats a slightly higher threat build may actually work better than a straight survival based spec. What I end up with is a 3 axis graph EH(survival) vs Threat vs Mitigation.

    The tricky part is weighting them so you can resolve the 3 spline equations :-)

    For threat for example if you go below 5K TPS it uses a log table of the diffence as you will have huge issues and similarly once you get above 12K TPS threat weights in inverse log.

    EH is nice and linear but mitigation is a nightmare and took ages.

    Results look promising so far but I need more data and TIME!!!!!!

    It sort of the opposite to what you are doing as well as you have a DK represented on a single line I would have multiple lines depending on what talents they had in or out. I understand that at the end of the day you end up with a HP and and armor value which you are using but those values are also changable by talents and those talents have other effects as well.
    ** Remember Warcraft players fail in directions you never thought possible.

  13. #13
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    threat is linked to Survivability for a DK and you're right, this would be interesting to include it in the weighting.



    I've implemented a Festergut fight with values I've found on internet.
    Basically Festergut hits for 75k, and gains 30% attack speed and intensity each time he inhales blight. Furthermore, I gave him a magic attack of 15k every two minutes and an aura of 4k every 3 seconds. Tanks are ilvl264 geared with raid buffs.
    Here is the average number of death for each 5 minutes of fighting (repeated 100000 times), knowing that if the tank dies, he comes back immediately with full health but without CDs.
    In the first sample healing corresponds of two full time healers (they are weak in term of healing output tough). No interruption of healing for all the trials I've done.
    Druid : 2,79 death by fight
    DK : 1,89 death by fight
    Warrior : 4,37 death by fight
    Paladin : 3,11 death by fight.


    Adding a healer, results change significantly :
    Druid : 0,25 death by fight
    DK : 0,22 death by fight
    Warrior : 0,95 death by fight
    Paladin : 0,3 death by fight.


    The main evolution concerns the paladin, I would say that its death counter decrease exponentially due to the mechanism of ardent defender. “the less he dies, the more the CD is up and he dies even less”.
    Note that the warrior has some powerful survival CDs such as shield wall he is using specifically the first time Festergut has 90% haste and power bonus but seems to dies the second very often.
    Survivability of druids/Paladin/DK seems to be equivalent. It brings the question of the pertinence of deathcount as the value interpreted. I mean paladin will always performs well when it is low and performs poorly when it's high. It's biased.


    Now, concerning ratio stamina versus dodge with some magical attacks. This time magical attacks are 20k/2 minutes and 4k/3seconds in the first trial. The standard error is around 0,1.
    Druid : 2,25 gems of dodge/gem of stamina.
    DK : 5,58
    Warrior : 2,48
    Paladin : 2,15


    It's 30k/2 minutes and 6k/3 seconds in the second trial.
    Druid : gem of stamina = 1,84 of dodge.
    DK : 4,6
    Warrior : 2,4
    Paladin : 2,13


    Interpreting this results, we might be surprised that the ratio stamina/dodge seems to be higher with less magical damage (at least for the DK). But we have to keep in mind what my trials tended to show before, that in stressful situations, avoidance is better than stamina. Of the two tendencies (magical means stamina is better than avoidance and more damage means avoidance is better) stressful conditions are a little more important in this data set.


    The actual question is to find the stressfulness of real wow fighting as it influences the ratio stamina/avoidance. Another thing to keep in mind is that we may consider avoidance more useful for high end raiding, but stamina doesn't suffer diminishing return whereas avoidance does. So for ivl277 (where dodge is higher), there's contrary tendencies to be explored.

  14. #14
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    I'm implementing healing classes in the simulator. I have done disc priest at least all of its spells but I'm lacking real values. I've seen many of it differing sometimes by 20% or more. (I'm using fantasy values at the moment : 10k for the shield. 4,5 for penance 6k for prayer of mending and 7k for flash heal. I'm not sure if I have to use renovation) I guess I will have to compute them myself unless someone know where to find them. I know there are good references on classes here or on EJ but I can't exploit them.
    [EDIT] I had forgotten RAWR... :P so I think I have now what I was looking for.


    Another thing I'm looking at is performance issue. The execution time of my simulations have increased by a factor two or three when I added healing classes. There's a lot of if-conditions (some of them I could eventually remove) and although the complexity (I mean number of instruction) should remain constant, I'd like to know if I can help my compiler a little in someway. Forgive me if it looks obvious but my computer formation is beginning to date and it's not my job either. So if you have links or advises to share, I would be grateful.
    Last edited by Hypathia; 07-17-2010 at 07:43 AM.

  15. #15
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    I think that all previous results and interpretations are flawed for a reason I will explain, but I need to come back on what I've done previously.
    There was at first a simulator with not so realistic values. I choose them to diminish the standard error. Indeed the more the number of death (that's my criteria) is high the more derived statistics are precise. Essentially due to tank CDs, I mean ardent defender etc... if I take high value of death counter by minute, those tanking CD will have less impact, introducing a bias.
    Now with a way more complete simulator, and realistic boss (In fact only Festergut), I think the number of death should be between 0,1 and 0,4 by fight. This means I will need a lot more simulations for each trial than I thought as the error seems to be in square root of number of deaths.


    In conclusion, this means that there will surely be no public release for the executable of the core simulator as it's unusable (Who would find fun to run several hours a simulation to have a ratio on a specific boss ?). My idea is to “publish” computed results, which would be interpreted by a script to obtain a wider data base dynamically. Technically results obtained by parametrization of a simulation should be boss formulation and simulation. The condition of possibility is to find statistical laws to move from one result to another. Lot's of work in perspective so.

  16. #16
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    Hello, I'm still alive.


    Since the 4,0 patch I have been rewriting my simulator with a lot of enthousiasm, trying to avoid precipitation. I now have a concern : On one hand I don't want to begin a “Who's the better tank” war on a public forum. On the other I'm really disappointed by terrible equivalence between classes. I know some have advantage in one domain not the other, or under precise circumstances.


    Mastery is I believe the source of all the problems. I won't go further to avoid controversy, as I may also be wrong by extrapolating values at level 80 (4,0) to Cataclysm. Maybe the best thing to do is letting Blizzard finish their job.


    So my question is : Is it a good Idea to discuss here the numbers I obtain from simulations comparing different classes and impact of mastery on their survivability or is it a little to early ?

  17. #17
    Join Date
    Jun 2009
    Location
    Belgium
    Posts
    2,361
    Discuss please, with a giant WARNING on top.

    The more numbers/models out there, the better Blizzard can see what they're doing wrong
    Quote Originally Posted by Ion
    Damn old people, screwin' with my grind.
    Mists of Pandaria Protection Warrior Spreadsheet
    Warlords of Draenor One Minute Field Guides

  18. #18
    Join Date
    Dec 2007
    Posts
    2,055
    I would think as long as you were working with beta server "live" values and real itemization quantities, it would be reliable and fascinating discussion.

  19. #19
    Join Date
    Oct 2008
    Posts
    4,930
    The key accent here should be that this is a comparison of actual values and not a discussion of what people think is best.

    Considering the full scope of strengths and weaknesses of each tank class is a value for raids, raid leaders, and strategic planning.

    People arguing about how [that other class] is over powered or their own class is gimped is fruitless and likely to get a (well-deserved) lock. The more factual the post, the less whiny or arbitrary, the better.
    The (Old) Book on Death Knight Tanking
    The New Testament on Death Knight Tanking
    -----------------------------------------
    Quote Originally Posted by Horacio View Post
    Who f-ing divided by zero?!?

  20. #20
    Join Date
    Jun 2010
    Location
    Grenoble, France
    Posts
    29
    Actually, my 4.x simulator was a rewriting of 3,3,5, with many improvements and my mind remained the same. In others words, I was comparing resistance to spike damage for different classes. There were large differences between classes for sure. But it is mostly irrelevant as I have realized that lack of healing more than spike damage was the primal cause of death.


    As I'm a little narrow minded (^^), I will continue to explore spike damage as It can be useful for heroic mode. But my priority now is to understand the 4,x way healing will be done, with class specificity, healing interruptions and the very complex mana management.

+ Reply to Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts