Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

The latest issue of the 9th Scroll is here! You can read all about it in the news.

  • Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Results are here:


    Windelov wrote:

    Purpose and methods:

    The fantasy tabletop gaming scene has diversified over the last 4 years since the discontinuation of Warhammer Fantasy Battles (WHFB) as a tabletop game by Games Workshop. The purpose of this survey is to evaluate some of the major qualities and preference for the game design and rules of the 4 major games incl. the discontinued WHFB in its latest edition, Age of Sigmar, 9thage (first edition, current official edition, and beta edition), 9thage Quickstarter and Kings of War.

    Display Spoiler

    Logistical factors such as cost, number of models required, number of players, duration of the games, randomness, the use of terrain and visual appearance of the tabletop will not be covered specifically.

    A survey was started on June 22 with an expected duration of 7-14 days. The survey can be downloaded as a pdf on XXX. No changes to the questions or options were made after the first participant had started the survey.

    No formal sample size was calculated as there was no data, including measures of variation, to build assumptions on. No minimal relevant difference was established as the survey was not validated (although this should be done for future use of the survey). A sample of 200 responders was deemed adequate. Unblinded data was visible only to @Windelov . Data was presented blinded in updates during collection missing one gaming system at random to avoid confounding participating and results.

    There were no restriction on having to be a member of a specific forum to gain as broad feedback as possible. Likewise, there were requirement of having played all the games in the survey but to gain information representing some level of experience, the responders had to have played 5 games to provide feedback on the qualities of the game.

    To ensure proper data quality, double posts were identified by degree of variation in answers given over a short period. Likewise, replies with the same value given to several games (e.g. assigning 0 or 5 to ALL qualities for several game systems) were excluded.

    Inference testing will be done with paired t-tests (i.e. only testing responders who played both the games evaluated) for data following an approximate Gaussian distribution, a non-paired t-test will be used as a sensitivity analysis (due to the many players only playing one or a few of the games). There will not be correction for multiple testing. Significance will be claimed for p<0.05, and p-values will be truncated to the third decimal.



    Results:
    A total of 200 people participated during the first 8 day where after the survey was closed. A total of 7 responders were excluded as obvious double posts. Furthermore, some entries for one or more games were deleted e.g. as stating “0” for all but a single game (obviously mistaken 0 for not having played the games). Excluding them did not affect the overall results. All numbers are presented in table 1.


    GameValid responses "Qualities"Valid responses "Preferences"Double or fake* posts removed
    WHFB1531499
    AoS81769
    9th age V1-v1.1 (first edition)12011711
    9th age V1.3 (current official edition)11511411
    9th age v. 2 beta13013012
    KoW49484
    Quickstarter18NA7



    *"fake posts" denote entries where values are given without variation, often all values stated as either 5 or 1. This happened mostly in conjunction with double posts (same results given within a short time period often <1 min).

    Most responders were linked to the survey from the-ninth-age.com, 40%, but with a substantial portion of responders linked from other 9th age sites and tabletop sites, 25% each, with the remaining part being from referred by other friends or others, note that only slightly more than half completed this part as not mandatory.



    The preference for games by how fun they are to play (so not related to the community, availability of players, clubs, etc) can be seen in figure 1 (bars indicate mean values and whisker denote 95% confidence intervals).



    When considering the preference by time of when the different systems were launched, WHFB was replaced by AoS which was significant less preferred by responders (p<0.001) both when considering players who played both systems (paired test) and if considering the replies independently (unpaired). The existing alternative KoW was preferred to AoS (p=0.03, paired) but was slightly less preferred than the existing WHFB although this was not significant (p=0.68).

    The first version of the 9th age (v.1.0 -1.1) was significantly more preferred than the WHFB among those that played both system and when considering all players (p=0.03, paired; p=0.01 unpaired). It was significantly more preferred than AoS (p<0.001 for both paired and unpaired) and KoW (p<0.001 for both paired and unpaired).

    With the update to the current official version 1.3 the preference as compared to the first version dropped significantly (p<0.001 paired and unpaired) and the 9th age v.1.3 was not significantly preferred compared to WHFB (p=0.28, favor of WHFB) and AoS (p=0.18, favor of 9th Age), although still preferred to KoW (p=0.04, paired).

    The Beta of 9th age covered the ground lost with the v1.3 and was equally preferred to the v1.0-1.1 (p=0.55in favor of beta edition). The beta edition was not significantly preferred to WHFB in the paired analysis (p=0.05) but was superior when comparing all responders (p<0.01 unpaired), the difference was presumably due to a larger group responders having played both WHFB and the V1.0 of FB:9th age than the Beta edition. ). The beta was significantly more preferred than AoS and KoW (p<0.001 for all, paired and unpaired).


    Gaming qualities:
    Responders were asked to rate 6 game qualities:
    • Diversity - The number of options /lack of restrictions in units and their equipment/abilities that you can bring to the table
    • Easy to learn and remember - The time spent learning and maintaining knowledge of the game to play at a competitive level (spending a lot of time gives a low score)
    • Efficient game play - The proportion of game time engaged in interesting actions of the game .
    • Balance - Internal and external balance of the armies to ensure that the majority of units are viable and that you fight other armies on equal terms.
    • Robust rules - The lack of ambiguity and need to discuss or consult FAQs for answer to rules
    • Immersion - The feeling of being part of story of fantastic battles, feeling connected to the characters and units.
    Each quality was assessed by a score of 0-5, with "0" being none at all; 1 being very little/barely any; 3 being moderate/average; and 5 being very high/perfect.



    When considering the gaming scene after the discontinuation of WHFB with only the v1 of the 9th age being available, it was clear that none of the successors could claim the same level of immersion as WHFB, with AoS > 9th Age > KoW.
    While AoS and KoW score vastly higher (and similar) on easy to learn but lost on Diversity where 9th age preserved the score of WHFB. All three successor games were more efficient in terms of gameplay, but KoW most so. For Balance and Robust Rules 9th age and KoW distanced themselves from both games (former and current) of Games Workshop.



    When then considering the updates of FB:9th age, there was a loss of immersion and diversity from the first full Version (1.0 and 1.1) to the current official version (1.3), and perhaps a slight loss in rules robustness. None of the parameters improved with the v1.3.

    With the introduction of beta, the "Diversity" and "Robust Rule" loss was somewhat restored while the "easy to learn" deteriorated even more. Balance was even more improved now to the level of KoW.



    As the Quickstarter is intended as bridge to the FB:9th age, it is not to compete with it but with other games. When comparing Quickstarter to KoW and AoS, there is a marked loss of "diversity" to both games while also less "easy to learn". Quickstarter is inferior to AoS on immersion and inferior to KoW on balance. The Quickstarter does not score highest on any parameters.

    KoW seems superior to AoS, efficicent game play, balance and rules robustness while AoS is superior to KoW on Diversity and Immersion with not difference on "easy to learn".


    Full histograms and measures of variation will be added for qualities. A subgroup analysis will be conducted to evaluate if results were different for responders who were linked from the the-ninth-age.com vs other 9th age sites vs other table top sites.


    Original first post
    Display Spoiler

    Hi all,

    This is a survey on the qualities of different fantasy tabletop games focusing on the evolution of the FB:9th age.

    The idea for building spider diagrams on core attributes was made by @JimMorr and picked up by me and @DJWoodelf for making a survey to get more general feedback from the community. The survey is anonymous, please complete only once. I will do a sensitivity analysis where I remove all those who seem to be double posting :)

    Feel free to distribute in your local gaming forum/Facebook page. The survey will close when we 200 have completed it! Which with the current pace will be in 6-7 days :) on July 14th, so in three weeks from now and I will report results the in the week after.

    Link to survey

    For those 'who might not be using Spider diagrams on a daily basis, it is simply an easy way to get a good sense of the qualities of product/people/processes and allows for matching them to different needs/end users/customer.
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:

    The post was edited 2 times, last by Windelov ().

  • Survey on Qualities of Fanatasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Hoffa wrote:

    I think a clear definition of what the categories mean are needed.


    Which ones do you think are not clearly enough defined in the survey?


    “The survey evaluates games by 6 qualities:

    • Diversity - The number of options /lack of restrictions in units and their equipment/abilities that you can bring to the table
    • Easy to learn and remember - The time spent learning and maintaining knowledge of the game to play at a competitive level (spending a lot of time gives a low score)
    • Efficient game play - The proportion of game time engaged in interesting actions of the game .
    • Balance - Internal and external balance of the armies to ensure that the majority of units are viable and that you fight other armies on equal terms.
    • Robust rules - The lack of ambiguity and need to discuss or consult FAQs for answer to rules
    • Immersion - The feeling of being part of story of fantastic battles, feeling connected to the characters and units.
    Each quality is assessed by a score of 0-5, with "0" being none at all; 1 being very little/barely any; 3 being moderate/average; and 5 being very high/perfect. Do not score game systems that you haven't played at least 5 times. “
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • Thanks for all the replies so far. Below are the blinded result of 6 of the 7 games under evaluation. I will include the 7th and unblind data when the survey is complete.

    Target is 200 responders.

    If you are eager to see which games score highest (or lowest), share the link to the survey on your local tabletop game FB site or Forum and encourage your friends to complete it as well. Please do not submit more than once, I will delete all double entries but it is tedious work.

    Thx
    Nis





    Number of respondees so far and predicted
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • I found filling in this survey fascinating.

    I rated a game that I quite like, poorly on many of these categories.

    I think I have learnt that the average score in these categories is not a good proxy for whether I will like a game :P
    New rules:
    (1) I will do my best to answer your criticisms, particularly of RT, but don't forget to thank one of the unsung heros who hold this project together: rules clarity team, lectors, website admin, background etc...
    (2) If you tag me and I don't answer you, its because I'm busy, sorry :( . If you still want an answer ~4 days later then tag me again and I will try to do better :)
  • DanT wrote:

    I found filling in this survey fascinating.

    I rated a game that I quite like, poorly on many of these categories.

    I think I have learnt that the average score in these categories is not a good proxy for whether I will like a game :P
    I agree

    That is why the last bit is there. What game would you prefer. Despite you not rating the game you like high for the different qualities you may still like it and that is what counts.

    If the a game is not rated highly overall, then we might can use the answers on how it compares to the others on qualities (and which qualities were rated as lacking by those not liking it).

    For the iterations of the same game we can use the change in qualities to evaluate if the game is improving on the expense of other parts.
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • Kristian wrote:

    I gave up on this.
    The questions are biased and the quality of a wargame includes all kinds of intangibles not covered in this survey.
    Why do you think the questions are biased and towards what?

    Of course there is all kinds of intangibles not covered. Being intangible they are difficult to represent in tangible questions.

    Best
    Nis
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • Update after almost 60 responders running ahead of expected rate. I will unblind when we hit 200 :)

    Complete if you havent already done so, and share at your local facebook/forum table top site :)

    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • Hi @DJWoodelf

    It ask to which extend the qualities are fulfilled by each game as. It doesn’t state bad or good.

    This provides the input needed for evaluating how games differ. It does not allow for saying whether one game is better than another.

    Some may think that just having the largest average sum will provide a meaningful value for good or bad. That is NOT the case.

    For that, we would have to ask different questions that take the personal opinion of the player into account. See @DanT comment above.

    Neutral questions are great but often, I find, difficult to interpret but I would happy to work on it. What type of questions do you think we should ask then?
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • Well it has been derived from SWOT analysis on obtaining new players, so it may be missing qualities that are completely neutral. Randomness for example - some like it, some hate it, some like it in the middle. Amount of models used. Amount of terrain used. Probably many more.
    Homebrew: Hetmanate of Ukray_____________Report your battle results using mobile app: T9A Magic Flux!
  • Tsk. I'd fill this out, but I'm not allowed to :P

    Never played five or more games of any single edition of Ninth Age. They change too fast for that, for someone who even at the best of times plays maybe once a month.
    "Inter arma enim silent leges."

    True Heirs of Avras: Vermin Swarm Auxillary Army books
    Some call it shameless Wishlisting! Some call it an Unplayable Mess! Some ask what is the Point of This! Is it an Auxillary Book? A Copy-paste Ripoff? A Fan Version of an existing Book? See for yourself, citizens!
  • Survey on Qualities of Fanatasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    JimMorr wrote:

    Well it has been derived from SWOT analysis on obtaining new players, so it may be missing qualities that are completely neutral. Randomness for example - some like it, some hate it, some like it in the middle. Amount of models used. Amount of terrain used. Probably many more.


    Agree those are good questions and very worthy of asking.!

    For e.g. randomness would that be 1) little -> moderate -> a lot (neutral) or 2) too little -> just right -> too much (good/bad)?

    For 1, how would you use the answers ?
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:
  • The use is simply to inform the discussion.
    Lambert, sure I will add measures of variation

    We were off to a great start but over the last 48 h, participation has dropped to a crawl.

    I will give it another 24 hours and stop it. This will probably not be enough for some of the interesting subgroup analysis but should give us a good level of understanding for main objective (compare qualities of games)
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup:

    The post was edited 2 times, last by Windelov ().

  • I think PR would want to see the result of this first poll before endorsing further ones.

    Let’s unpack this one soon and review the results to learn more and then we can perhaps do another endorsed by PR.
    Thanks to all who participated in the Survey, see the results here: Results of survey on preference and qualities of Fantasy Tabletop Games (FB:9th, Quickstarter, AoS, WHFB, KoW)

    Join the work on a community driven suggestion for a KoE rewrite: KoE community suggestions for rewritten army book, first draft
    :thumbsup: