When I posted my initial link to the questionnaire, I said I had a hypothesis, but not what it was. I actually had two hypotheses:
- Pagans score lower on Compliance than average. (This was the ‘stereotype hypothesis’ in my mind.)
- Pagans score lower on Steadiness than average. (This hypothesis was supposed based on my experience resolving disputes as ADF’s Member’s Advocate.)
So, what were my results for these two hyoptheses? Statistically insignificant, and not corroborated, respectively. Only one person out of my 90 useable responses self-identified as a non-ADF member; their data actually pushed against my hypotheses but did not produce a singificant difference. Here’s the statistics, on which I’ve left far too many significant digits:
For the first hypothesis regarding Compliance, the big picture here comes from the standard deviation. When used to construct a confidence interval, the null hypothesis that ADFers are much the same as “average”, which is to say a 25% split four ways, is not rejected. (This is even the case for 50% confidence, let alone the usual standard of 95%)
For my second hypothesis, what jumps out at me is that my average result for Steadiness was actually higher than 25%! This throws my second hypothesis out the window. I will note it isn’t a statistically significant deviation, though.
Some Issues Worth Pondering
If I was doing this again, I would likely use my own instrument to tally the data. Not only would it be easier, but there were about ten submissions I had to reject because folks had written, from the confusing results chart, percentages for A, B, C, and D, which were associated with the biggest, second biggest, etc. value. Additionally I’d get access to the numbers as-computed, rather than a percentage rounded to two digits. One of my checks on data entry was to add the values; there were a number that totalled to 99 or 101 rather than 100.
I did note a number of cases where D > I > S > C and wondered if it was cases of this — but even if so, they would have pushed things toward my hypotheses, not away. The high value for S also argues against that being a major issue.
Of course, I’d prefer to do such with a Big Five approach anyhow. An ADF member with experience sent me links to some instruments that would involve not too many questions if I decide to do this again.