Crowdsourcing Critical Discourse Analysis: Using Amazon’s Mechanical Turk to Explore Readers’ Uptake of Comments about Language on RateMyProfessors.com

Abstract

Critical discourse analysis (CDA) studies how social dominance and power are discursively enacted through, for example, discourse’s influence on attitudes, beliefs, and ideologies. Yet, various critics have charged that CDA’s generalizations, drawn from textual analysis, conflate analysts’ own interpretations with those of ‘typical’ readers. We examine one example of this: Subtirelu’s (2015) study of comments about instructors’ language and ethnicity on RateMyProfessors.com. We use Amazon.com’s Mechanical Turk to test Subtirelu’s claim that ostensibly neutral or positive comments about language are taken up negatively by readers. Our experiments find that comments in which instructors’ accents are mentioned but not disparaged (e.g., ‘She has an accent, but…’) lead readers to be slightly less willing to take a course from the instructor than when information about the instructor’s accent is withheld. We also present a post hoc analysis designed to examine whether other textual features might explain the differing reactions to this information about accent which we observed. We hope the study will serve as an example of the type of work that can be done in CDA not only to address methodological criticisms but also to lead to more nuanced theory about the effects of discourse on audiences.