And this week, I have finished looking through the material on Text Analysis! While, it was a great experience working through the material, I have decided to focus my blog this week on the article by Nguyen and colleagues (2020) titled “How We Do Things With Words: Analyzing Text as Social and Cultural Data.” The authors used this article to talk about their own experience using computational text analysis to look at cultural and social concepts, with the intent purpose of providing guidance on potential roadblocks, opportunities, and areas for collaboration between researchers. I found that this article is especially insightful for guiding new research with Text Analysis, as the authors walk through a variety of scenarios one might face.
One insightful section in the paper focuses on “units of interest” and how researchers need to approach how exactly they are handling their data. While it is common practice to tokenize documents in order to get down to the unit of words, single words may detract from the true meaning of a passage. For example, the authors bring up examples such as “black hole,” two words that when together have a much different meaning than either word alone. And even beyond word combinations, the authors comment on differences within one word, such as the difference between “apple” and “Apple.” As tokenization changes all words to lowercase, some inherent meanings are lost. Therefore, it is really important to consider not only the data we want to examine, but how exactly we are examining it and how this affects our analyses and conclusions.
And while I talked in depth about stopwords in my last blog, the authors reintroduce this topic. Rather than just focusing on how many words should be removed from an analysis, Nguyen and colleagues (2020) instead reflect on the definition of common words and how this shifts depending on your analyses and the types of documents you are examining. In one particular example, the authors describe a scenario where court documents are examined and words commonly considered uncommon, such as prisoner, are very frequent and add little to our understanding of the document. For example, in my line of work, words such as “forensic,” “biological,” or “anthropology” may not provide much extra pieces of information.
Nguyen and colleagues (2020) conclude the article by addressing how unexpected results are not always a sign that our model was ineffective, but rather that there may be some misconceptions between what patterns we believe we are seeing and how the program we are using is able to interpret patterns. If our analysis does not match our initial hypothesis, this does not mean that, for example, a certain set of topics cannot be grouped together, but that our current system cannot do so, likely because of unaddressed assumptions in the model. This then can lead to new types of questions, such as why the model approaches the data in a certain way and what that means for analysis on these set of topics. For example, would extra guiding by the research help refine the model – or would we bias the model too much towards what we expect?
Focusing on the why of how our model is working is essential to a well informed and robust analysis. If we are using methods blindly, it will surely be reflected in our research and take away from what we hope to bring to a wider audience.
