Foundations for a Statistical Mindset
Preface
How we think about data about how we think
Not Another Teen Movie Psych Stats Book
- Why another intro psych stats book?
- We’ve been teaching it for a long time
- Something is rotten in the state of Denmark. Still big fundamental problems to the way psychology research is done.
- Increasingly other good resources available (links as aside), but most of them at a more advanced level - pitched at correcting misconceptions from intro stats rather than being the intro stats.
- This book comes out of two strong opinions:
Psychologists Need Quantitative Literacy
- The field nominally recognizes this by requiring all psych majors take a stats class, but it’s often treated as the unwanted stepchild of the psych curriculum, just a hurdle to jump over before you get to the meat of the scholarship.
- But all the problems in the literature suggest that better training is needed.
- Kelvin quote on something isn’t a science unless it can be quantified. Science isn’t the only valid way of knowledge about the universe and the human condition, but if we claim to be a psychological science, we need to walk the walk of that approach.
- Some argue that all science teams should have a stats consultant to do this work - don’t trust yourself to do the complicated stuff. Stats consultants are heroes in the academic world but it is still problematic to offload the statistical thinking to them because statistical models are the theories we are testing - if you can’t formalize your hypothesis as a model and clearly understand the assumptions of it, you don’t understand your data or your theory very well. Also, bad design decisions get made before a stats person comes onto the team, wasting research resources on a study that can’t answer anything useful.
Quantitative Literacy Doesn’t Need to Start with Math Stats
- This may be a surprising thing to hear considering what was in the first opinion.
- Math stats classes get to the heart of the tools used in stats, with theorems and derivations. A lot of students learn well from that and come into scientific practice with a strong quantitative foundation.
- …And a lot don’t. Math is abstract and it is not always clear how it connects to the “why” of doing it. This feeds the psych researcher’s treatment of it as magic spells to chant to give them permission to publish. And if a student doesn’t become a researcher, they may not always connect how marble drawing problems in their probability class connects to risk assessment about vaccines in their own life.
- A different approach to teaching students to value quantitative skills is to start with the personally-relevant problems and the logic of answering them. In psych stats, that means talking a lot about concepts like uncertainty, making predictions, data quality, etc. before ever calculating a variance statistic.
- This also means the instructor doesn’t need to commit to one school of statistics at the beginning. Whether you are Bayesian or Frequentist, you have similar research goals that are served by different approaches/assumptions. Conversely, there are lots of other research approaches that traditionally fall outside of the psych stats curriculum but that are worthy of understanding and use.1 An intro curriculum can teach the variety of ways to pose and answer a statistical question, and why psychologists would care about these, without getting into deep math right away. Afterwards, a student can pursue the more formal math training with a foundational understanding of its importance and a stronger intrinsic motivation for it.2
Foundations of this Book
With those positions in mind, this is the book’s main learning goals:
- know your estimand
- there is uncertainty, variation in a variable we want to know about
- information from other variables can help us reduce that uncertainty (model building)
- research questions focus on different reasons for doing this: documenting the association (description), reducing the uncertainty as much as possible (prediction), understanding how important predictors change uncertainty in the outcome (explanation)
- different levels of specificity for hypotheses: any sort of uncertainty reduction (NHST), comparing different options for least uncertainty (model comparison), making point predictions
And it’s strategy to achieving them:
- Organized around types of questions/needs, not specific tools or frameworks
- Model-based rather than collection of tools – build understanding. Leaves out some things that might otherwise appear in an intro stats class like chi-square tests, but builds a foundation that more flexibly prepares for future research possibilities with modeling.
- De-emphasize NHST
- Interpretation more than calculation
- Doing stats, with code
- Learning stats with domain-specific guidance
Who this Book is For:
Undergraduates or early grad students who wish to read and conduct their own research. Brain & behavior topics (neuro, psych, cog sci, behavioral science, etc.); non-academics who wish to be more critical consumers of cognitive & behavioral research.
Intended Workflow:
Read, practice inline code, do problem sets, keep this for reference.
Assumed Knowledge:
Mathematics up through algebra, familiarity with intro psych content