Graphical integrity issues in open access publications: Detection and patterns of proportional ink violations

Academic graphs are essential for communicating complex scientific ideas and results. To ensure that these graphs truthfully reflect underlying data and relationships, visualization researchers have proposed several principles to guide the graph creation process. However, the extent of violations of these principles in academic publications is unknown. In this work, we develop a deep learning-based method to accurately measure violations of the proportional ink principle (AUC = 0.917), which states that the size of shaded areas in graphs should be consistent with their corresponding quantities. We apply our method to analyze a large sample of bar charts contained in 300K figures from open access publications. Our results estimate that 5% of bar charts contain proportional ink violations. Further analysis reveals that these graphical integrity issues are significantly more prevalent in some research fields, such as psychology and computer science, and some regions of the globe. Additionally, we find no temporal and seniority trends in violations. Finally, apart from openly releasing our large annotated dataset and method, we discuss how computational research integrity could be part of peer-review and the publication processes.
Author summary Scientific figures are one of the most effective ways of conveying complex information to fellow scientists and the general public. While figures have the great power to leave a strong impression, they can also confuse and even mislead readers. Visualization researchers suggest several rules to avoid these issues. One such rule is the Principle of Proportional Ink (PPI) which suggests the size of shaded areas in graphs to be consistent with their corresponding quantities. The extent of violations of this principle in scientific literature is unknown, and methods for detecting it are lacking. In this article, we develop a deep learning-based method to perform such tasks on bar charts. An analysis of hundreds of thousands of figures revealed that around 5% of bar charts have violations of PPI. We found differences in the prevalence of these issues across fields and countries. We discuss the implications of these results and the opportunities and challenges posed by technology such as the one proposed here.