Foundations for a Statistical Mindset

Preface

How we think about data about how we think

Not Another Teen Movie Psych Stats Book

Every university program in the psychological and brain sciences requires its students to take a statistics class at some point. The students dutifully comply in order to get their degree, though some with more enthusiasm than others.

In my own psychological training, years ago, the only thing I remember of my undergraduate stats class was that I didn’t really want to be there. It felt like a book of magic spells I was supposed to memorize in order to be allowed to do what I was actually in interested in, studying the basics of why people do what they do.

It wasn’t until half way through graduate school when I enrolled in a statistical modeling class, and things finally made sense. The logic of what statistical models mean, why they are useful, and how they are deeply relevant to testing scientific theories (indeed, are the theories) was finally clear. Not only did I finally understand statistics better, I became more passionate about it.

I don’t think this is an uncommon experience for psychology students. I have been consulting on and teaching data analysis methods to undergraduate students for many years now, and very frequently new students ask me “what kind of test can I use on my data?” while looking at a drop down list from a stats software menu, without understanding what that test is testing or whether it would even answer their research question. Worse, sometimes they don’t even have a quantifiable research question and are just designing datasets around a memory that analysts are supposed to compare two groups to each other somehow.

It is also common in the wider psychological research community to see fundamental problems with the ways people perform and interpret their data analyses, despite the fact that statistics has been required as part of undergraduate and graduate training for a century¹.

I think this lack of understanding stems from the way statistics is traditionally taught to psychology students. It is often taught as a collection of tools, focusing on hand calculation with toy data (and thus limited to analyses that are hand calculable.) It emphasizes population inference early and elevates null hypothesis testing as the best way to answer research questions. But this doesn’t really connect with the reason students are in the classroom in the first place: to understand the “how” and “why” of human cognition and behavior.

My gamble is that we can teach psychological statistics in a way that connects with and satisfies this fundamental motivation more directly - by focusing on data generation processes more than specific tests; seeing data emerge through simulation rather than only viewing it after it’s written down; and exploring different types of research questions about the data generation process instead of limiting our imagination to a null hypothesis. I believe this approach inspires more persistence and passion in further analytic training and eventually better theory building in the field.

I am not alone in this thinking, and this book makes reference to many other excellent resources that take such an approach. However, most of these resources are pitched for advanced students, written to help them overcome misconceptions they learned previously. I haven’t seen as many good resources written for introductory students, aimed at instilling modern statistical instincts from the beginning.

That is the genesis of this book. Readers should not expect it to be a thorough reference for all statistical modeling approaches or a deep dive into the mathematical derivations of modeling tools. However, I hope it can function as a sort of statistical “stem cell,” giving students foundational ways of thinking about models and how they are useful. It will then motivate and facilitate further training in modern statistical modeling and data analysis.

Core Ideas of this Book

With that background in mind, these are the book’s main learning goals:

• be specific: about your research focus, question, measurements, population, hypotheses
• there is uncertainty, variation in a variable we want to know about
• information from other variables can help us reduce that uncertainty (model building)
• research questions focus on different reasons for doing this: documenting the association (description), reducing the uncertainty as much as possible (prediction), understanding how important predictors change uncertainty in the outcome (explanation)
• different levels of specificity for hypotheses: any sort of uncertainty reduction (NHST), comparing different options for least uncertainty (model comparison), making point predictions

And its strategy to achieving them:

• Organized around types of questions/needs, not specific tools
• Model-based rather than collection of tools – build understanding. Leaves out some things that might otherwise appear in an intro stats class like chi-square tests, but builds a foundation that more flexibly prepares for future research possibilities with modeling.
• De-emphasize hypothesis testing
• Interpretation more than calculation
• Doing stats, with code
• Learning stats with domain-specific guidance

Who this Book is For

Undergraduates or early grad students who wish to read and conduct their own research. Brain & behavior topics (neuro, psych, cog sci, behavioral science, etc.); non-academics who wish to be more critical consumers of cognitive & behavioral research.

Intended Workflow

Read, practice inline code, do problem sets, keep this for reference.

Assumed Knowledge

Mathematics up through algebra, familiarity with intro psych content

---

1. Psychologist and philosopher of science Paul Meehl put it harshly but perhaps not incorrectly: “The failure of some in our profession to recognize solutions to quantitative problems or to apply them to controversies is partly due to self-selection in the social sciences for poor quantitative talent; but in psychology it is due even more to the lamentable lack of mathematical education of psychology majors.”