Data Analysis Report

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    Wondering how the new magic phase feels like? Try it yourself! The Behind The Scenes blog gives you enough to playtest it, including spells of four paths of magic, all hereditary spells and the Dwarven runes!

    • New

      Calisson wrote:

      I did not say that slide 2 would fix all issues. What I meant is that seeing the US impact, not too large but not too small, would help to alleviate some of US concerns.
      DA+TouS do a wonderful job of collecting all the data they can and presenting it all, they have never been the issue. The issue comes from the people "interpreting" it, and there are dozens of screenshots and chunks of skype calls that have illustrated this issue. So unless you have fired large chunks of the staff recently it doesn't really alleviate anything, because, as noted, the issue isn't on the data collection/analysis side.
      “You can never know everything, and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyways.” -Lan Mandragoran, EotW

      UD Army Community Support

      Playtester

      Supreme Death Cult Hierarch

      Dovie’andi se tovya sagain.
    • New

      Nicreap wrote:

      fire large chunks of the staff
      That's one way to solve the issue. :muaha:
      But not what I had in mind.
      I'd rather do some interpretation on my own, but inerpreting in the opposite direction as nay-sayers.
      For that, I do like everyone: I quote data and say things which seem to make sense for someone who knows nothing about stats. :saint:

      Social Media Team

      UN Coordinator, aka UNSG

      - druchii.net contribution: The 9th Age - Dread Elves
    • New

      Calisson wrote:

      Nicreap wrote:

      fire large chunks of the staff
      That's one way to solve the issue. :muaha: But not what I had in mind.
      I'd rather do some interpretation on my own, but inerpreting in the opposite direction as nay-sayers.
      For that, I do like everyone: I quote data and say things which seem to make sense for someone who knows nothing about stats. :saint:
      Well, it's the only way the data report alleviates the problem (indirectly of course) :P

      The issue is the nay-sayers can use whatever they wish and don't have to justify it to anyone. So it doesn't matter what the data does or does not say, because their view will always be the right one, and if the data disagrees the data is bad. Got screenshots of that too :D
      “You can never know everything, and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyways.” -Lan Mandragoran, EotW

      UD Army Community Support

      Playtester

      Supreme Death Cult Hierarch

      Dovie’andi se tovya sagain.
    • New

      Cyprinus wrote:

      DanT wrote:

      I like the distribution plots too, and the popularity plots.
      When I did my own analysis, these 2 (and the fig 12 type plot) were the 3 metrics I used.

      Some of the discussions during the 2.0 update were about the abilities of different armies to perform at the top tables relative to the bottom tables.
      So the distribution plots are very interesting to look at.
      Perhaps the number of times an army ends up in the top 5 compared to the amount of times it ends up in the last 5?
      Alternatively, one could generate a distribution plot, with bins at 10% intervals... this would generate a spread similar to the placement graph, but then less detailed (as we use 10% intervals. Still, it should show how armies place accros the board.

      EDIT:
      As for top table to lower table. It would be good to know the number of times a player is used in hse results. For example, one excellent player wins a lot with Army X and goes to many tournaments, the skill of the player (not the army strength) will influence the top results.

      Then again, this already happens in all of the analyses. And unless we can ‘average’ each players results, we have to rely on greater numbers to even these out for us
      Yeah - I think there are issues here with armies and player correlating. I'm not sure whether I consistently come bottom 5, but I certainly come bottom quarter quite consistently. I also am fairly loyal to my HBE. Compare me to say Craig on the UK circuit who's been playing beasts pretty solidly for a while now and he'll consistently come top 5. I'm pretty confident he'd make top 5 with whichever army you'd choose to give him (and same goes for the top 5-10 players on the circuit), so if they stick with the same army then they will bias the results...and that's ignoring the underlying reasons why we all choose our armies of course...such a minefield...

      @DanT - I have a question for you, as you are both on BLT and also a statistician. If the data report (or non-report, but analysis, such as is presented) basically doesn't give a meaningful answer to the question of external power levels, which is what I believe @Nicreap is suggesting - and broadly I trust him as much as anyone on this as I followed his arguments last time round when he wrote and defended the report then - how did BLT use the data? Surely BLT couldn't use data under these circumstances, and is reduced to - for better or worse - use their experience and a logical approach to qualitative rather than quantitative problems? Now I don't necessarily see this as a problem - for that matter I think it would be very useful to have points people involved with the design phase (ADT and ACS guys) so that appropriate designs can be matched to points in a cohesive way, rather than chopping them apart and then down the line having to piece together elements that no longer fit together. But the impression I got was that BLT spends hours and hours agonising over what to do based on data available - and from this analysis it suggests that this isn't really a meaningful approach? I'm sure I must have got the wrong end of the stick somewhere though - hence my rather garbled question!
      Join us on Ulthuan.net
    • New

      ferny wrote:

      @DanT - I have a question for you, as you are both on BLT and also a statistician. If the data report (or non-report, but analysis, such as is presented) basically doesn't give a meaningful answer to the question of external power levels, which is what I believe @Nicreap is suggesting - and broadly I trust him as much as anyone on this as I followed his arguments last time round when he wrote and defended the report then - how did BLT use the data? Surely BLT couldn't use data under these circumstances, and is reduced to - for better or worse - use their experience and a logical approach to qualitative rather than quantitative problems? Now I don't necessarily see this as a problem - for that matter I think it would be very useful to have points people involved with the design phase (ADT and ACS guys) so that appropriate designs can be matched to points in a cohesive way, rather than chopping them apart and then down the line having to piece together elements that no longer fit together. But the impression I got was that BLT spends hours and hours agonising over what to do based on data available - and from this analysis it suggests that this isn't really a meaningful approach? I'm sure I must have got the wrong end of the stick somewhere though - hence my rather garbled question!
      Unofficial brief BLT answer (because I don't have time to formally vet the answer with the whole team, but I believe it to be fairly representative):

      As it happens, the data report broadly agreed with BLT thoughts already, so in some sense we proceeded according to our own experience anyway.
      We also think the main conclusion that can be drawn from the DA report is that broadly the game is pretty well balanced externally.
      It is not perfect, but it never will be (which won't stop us trying of course).

      In addition, the core rules changes going into 2.0 mean that the external balance is more difficult to act upon correctly.

      For these two reasons, BLT focussed more (but not exclusively) on internal balance (which isn't to say we got it right of course :P )

      That said, we had an approximate goal in our heads, which was to balance to the upper middle of 1.3.
      So, the best 2-3 armies will come down a touch, the next 3ish will stay broadly the same, and the others will all increase in power to varying extent.

      Of course, there is a big error on this attempt, because of the core rules changes.
      We expect there to be more of a focus on external balance during the public beta test.

      Balancing team

      EoS Community Support

      "Two things gamers hate most is change and the way things are" - Stygian
    • New

      @DanT, I agree the 2.0 rules shift is a big deal. My OnG experience has been that 2.0 armybook revisions have mainly been tiny tweaks of pretty minimal impact, other than one big price change.

      On the other hand the changes to the core rules will have a massive impact. Bows, horde formations, frenzy update, magic, spears... each of these will have a bigger impact that all of the changes in the armybook put together (ok, a slight exaggeration, but it is broadly correct).