Welcome back! This week our Digital Methods class has continued our discussion of data, but with a focus on shapes of data. While there are many aspects of this week’s readings that made me really think about how I’m using and looking at data, I found the takeaway of one article especially profound. This article, as indicated by the name, focuses on why “All Models Are Wrong” (Richard Jean So 2017).
While this title and message may seem jarring and contradictory at first, as was meant by the author’s undergraduate statistics professor, this idea actually means the opposite of what you might think. Of course we know that models are never perfect. This can be seen in research within the social sciences where we are attempting to represent a population with a smaller sample of said population – because how can we ever really know that a sample is representative? Often, it is clear that the only way to be truly representative is to create a study of an entire population. But why bother creating studies when models are always wrong?
This is because, as Richard Jean So (2017) points out, understanding how something is wrong can better help us understand either issues with our original hypothesis, methodology, or even our understanding of the problem we are trying to approach or solve. For example, in forensic anthropology, it can be hard to compile large study samples because, frankly, there aren’t that many skeletal collections – in comparison to the entire population at least. But does that mean we should not do any research with them? I don’t think so. I think it just means that we have to be careful about what conclusions we draw and be aware of what types of factors may be influencing our model – and making it “wrong.” These factors may limit us in some sense, but are essential to guiding future research.
In fact, I think Catherine D’Ignazio and Lauren Klein’s (2018) book, Data Feminism, can really speak to this – especially in their chapter titled “Numbers Don’t Speak For Themselves.” This chapter focuses on the importance of context when approaching and using data – i.e., data is never just a set of numbers, but a reflection of the research or research that compiled it. I think this idea goes in tandem with the limitations of models – often times it is the ignorance of data context that can create models that are wrong. Context is so important because it can provide information on why certain data was included, or what data wasn’t. In many ways, the best way to understand our data is to understand what is missing and how that affects the model or study in total.
Overall, research isn’t perfect. But that doesn’t mean that it should be abandoned or halted altogether. Instead, we must make sure to look at our data in a few different angles to better understand what it is and is not telling us.
