Home // HUSO 2015, The First International Conference on Human and Social Analytics // View article

Computationally Detecting and Quantifying the Degree of Bias in Sentence-Level Text of News Stories

Authors:
Clayton J. Hutto
Dennis J. Folds
D. Scott Appling

Keywords: bias detection; bias quantification; linguistic model; text processing

Abstract:
Fair and impartial reporting is a prerequisite for objective journalism; the public holds faith in the idea that the journalists we look to for insights about the world around us are presenting nothing more than neutral, unprejudiced facts. Most news organizations strictly separate news and editorial staffs. Bias is, unfortunately, ubiquitous nevertheless. It is therefore at once both intellectually fundamental and pragmatically valuable to understand the nature of bias. To this end, we constructed a computational model to detect bias when it is expressed in news reports and to quantify the magnitude of the biased expression. As part of a larger overall effort, we conducted a survey of 91 people to investigate factors that influence the perception of bias in fictitious news stories. During this process, subjects provided ground-truth gold standard ratings for the degree of perceived bias (slightly, moderately, or extremely biased) for every sentence across five separate news articles. In this work-in-progress, we analyze the efficacy of a combination of linguistic and structural information for not only detecting the presence of biased text, but also to construct a model capable of estimating its scale. We compare and contrast 26 common linguistic and structural cues of biased language, incorporating sentiment analysis, subjectivity analysis, modality (expressed certainty), the use of factive verbs, hedge phrases, and many other features. These insights allow us to develop a model with greater than 97% accuracy, and accounts for 85.9% of the variance in human judgements of perceived bias in news-like text. Using 10-fold cross-validation, we verified that the model is able to consistently predict the average bias (mean of 91 human participant judgements) with remarkably good fit.

Pages: 30 to 34

Copyright: Copyright (c) IARIA, 2015

Publication date: October 11, 2015

Published in: conference

ISSN: 2519-8351

ISBN: 978-1-61208-447-3

Location: St. Julians, Malta

Dates: from October 11, 2015 to October 16, 2015