COp-Ed: Measuring Criticism of the Police in Local Media Coverage Using Large Language Models

A series of high-profile incidents of police violence against Black citizens over the past decade, has spawned a contentious public debate in the USA on the role of police. This debate has played out prominently in the news media and has led to a perception that media outlets have become more critical of the police. There is, however, little empirical evidence supporting this perceived shift. We construct a large dataset of local news coverage of the police from 2013-2023 in a politically diverse set of 10 largeU.S. cities. Leveraging advanced natural language models, we measure media criticism by analyzingwhether coverage supports or is critical of two contentions: (1) that the police protect citizens and (2) thatthe police are racist. To validate this approach, we also collect human annotations from members ofdifferent political parties. We find that contrary to public perceptions, local media criticism of the police hasremained relatively stable along these two dimensions over the past decade. While critical reporting spikes in the aftermath of high-profile police killings these events did not produce sustained increases innegative police news. Furthermore, we find only small differences in coverage between outlets in more conservative and more liberal cities. This consistency in content contrasts with large observed differencesin perceived criticism between Republican and Democratic annotators. Thus, while local reporting on thepolice is not meaningfully more critical over the past decade, the impact of coverage likely depends on the viewpoint of the reader consuming it. Taken as a whole, the result suggest local media coverage has not been infected with the partisanship that objective reporting aims to avoid.