Dark Data Project

Introduction

Identifying misinformation is an increasingly daunting challenge for organizations with an urgent imperative to reduce consequent harms, not least because there exists a growing technological capacity to generate disinformation at unmanageable volume. Common mitigations to this sort of content, such as collaborative hash databases and end-user participation, are methodologies of last resort, with limited efficacy and their own problematic consequences. The greatest impediment to better algorithmic triage of misinformation may be the nuanced nature of truth itself. While there is a rich literature of metaphysics pertaining to theories of truth and information, scholarship at this level of abstraction simply offers too little practical value to those on the forefront of misinformation management. A "field guide" is required which broadly classifies assertions in a manner that facilitates mitigations appropriate to a particular taxa of truth.

This guide is also intended to inform the development of artificial intelligence, natural language processing technologies and structured tasking tools, all of which will be essential to implement mitigations at scale. As important as it is for human beings to better understand the particular species of an assertion, it seems impossible that machines will be able to effectively mitigate negative outcomes without some sort of taxonomic scaffolding for misinformation.

Demonstrable Fact

Examples

"The world is round." / "The world is flat."

Analysis and Mitigation

A demonstrable fact is any assertion which one can, in principle, determine for oneself to be true or false. Demonstrable facts are the underpinning of objective reality, without which we would be unable to function, and are rarely (but not never) subject to debate.

The Greek mathematician and astronomer Eratosthenes established in 3 BC that the Earth was a sphere, a feat he accomplished using just the light of the sun and two vertical sticks spaced widely apart. Today, anyone can perform the same experiment, or can travel around the world with a compass and eventually return to their starting point without having stumbled off the edge of a pancake-shaped world. The mitigation strategy for demonstrable fact, therefore, is demonstration, and the greater burden lies not in establishing truth but in communicating it.

Deductive Fact

Examples

"Smoking can cause cancer." / "Human industrialization can cause climate change."

Analysis and Mitigation

Deductive facts are similar to demonstrable facts, in that they are often well enough established to be considered a reasonable measure of objective reality, but nonetheless require a logical inference to prove or disprove an assertion. This inference will often take the form of statistical correlation.

Because the mechanism by which the nicotine in a cigarette causes an increased likelihood of terminal lung cancer is not as simple as, say, the causality of a fatal gunshot wound, deductive assertions are much more likely to be debated regardless of any amount of evidence. Tobacco companies successfully delayed legal consequences for decades, causing unnecessary widespread loss of life, and climate denialism borrows from the same playbook.

Proving the truth or falsehood of deductive facts requires both a consensus of data and analysis, and a means of replicability, which is the cornerstone of the scientific method. As with demonstrable fact, the heaviest burden here is communication: no matter how obvious a correlation may seem, those who are disinclined to review and understand the data will resort to anecdotal and lived experience ("My grandmother smoked for ninety years." / "How can there be global warming when this is the coldest winter in years?").

Historical Truth

Examples

"Twenty children were shot and killed at Sandy Hook Elementary School on December 14, 2012." / "The Cambodian genocide of 1974-79 resulted in millions of deaths at the hands of the Khmer Rouge."

Analysis and Mitigation

Like demonstrable facts, historical truths reflect previously documented assertions and generally aren't reliant on inference, although events which are far enough in the past to be sparsely documented can sometimes be leveraged as examples of false inference. The fragmentary fossil record of early hominid lineages, for example, have been used to sow doubt about human evolution. Even relatively recent, well documented events will be debated if they serve ideological ends; Holocaust denialism is a particularly virulent example of this.

Unlike demonstrable fact, a historical truth usually cannot be proven or disproven from firsthand experience. Citing a consensus of data and analysis, particularly contemporaneous data and replicable scholarship, is therefore an essential mitigation strategy. As with any assertion, it's important to ensure that unreasonable detail isn't employed as a bad faith attempt at refutation. One need not know the exact number of dead to establish the truth of a genocide.

Marquee Truths

Examples

"Laughter is the best medicine." / "Chance favors the prepared." / "Liberals want big government, and will increase your taxes to pay for it." / "Conservatives value individual liberty over collective welfare."

Analysis and Mitigation

Big "marquee" truths are assertions which are so broad that they cover a wide range of circumstances, and therefore are vulnerable to reframing into countertruths. Laughter may indeed be the best medicine – unless you have tuberculosis, in which case you're better off with antibiotics. All things being equal, it's better to be prepared for misfortune – unless you choose to spend your life in a concrete bunker.

While these sorts of truths often take the form of homespun wisdom and harmless cliches, they can have a more malignant impact when used to generalize about politics, nationality, ethnicity, gender or sexual orientation. Not every liberal is a "tax and spend" liberal, and not every conservative is a "slash and burn" capitalist.

The best mitigation to umbrella assertions like these is specificity: an assertion should be reduced to the atomic level before being evaluated for truth or falsehood.

Moral Truth

Examples

"Killing is wrong." / "An eye for an eye." / "Abortion is a sin regardless of circumstance." / "We shouldn't experiment with nature, even to solve climate change."

Analysis and Mitigation

Like marquee truths, moral truths are often extremely broad and are as likely to contain their own countertruths. Killing may be generally perceived to be wrong, but there are myriad exceptions of the rule which many people find palpable, such as warfare, self-defense and punishment. While broad moral truths may lack mechanisms for verifiability, their lack of specificity also makes them unlikely vehicles for misinformation. Conversely, specific assertions such as the morality of abortion, gender transitioning and legalization of drugs may certainly be employed for persuasion based on misinformation, and should therefore be mitigated with the same strategies used to establish deductive fact: a preponderance of evidence from well established and non-partisan sources. It should be noted that assertions made on moral grounds are generally more deeply held than those simply based on lived experience.

Religious and Cultural Truth

Examples

"In the beginning God created the heaven and the earth." / "The Inuit goddess Sedna determines when certain sea creatures will be available for hunting and fishing."

Analysis and Mitigation

Religious and cultural truths can be alternatively interpreted as metaphorical, moral and/or literal truths, even though they may seem in direct contradiction to demonstrable or deductive fact. As such, they have the potential to be repurposed for malignant purposes, like the 1989 fatwa morally justifying the (unsuccessful) murder of writer Salman Rushdie, or the belief by the Heaven's Gate cult that mass suicide would transport them to a passing spaceship.

Fortunately, as with moral truths, religious and cultural truths are often framed broadly enough not to be a vehicle for misinformation. Rather, the goal of identifying this sort of truth is to avoid inadvertently applying mitigations more appropriate to demonstrable or deductive fact. Content moderators and algorithms shouldn't erroneously mistake a metaphorical religious truth for a refutation of demonstrable fact.

Ideology

Examples

"The insurrectionists who breached the US Capitol in 2020 were attempting to overthrow democracy." / "The freedom fighters who breached the US Capitol in 2020 were acting in defense of democracy." / "Our lives are controlled by the Deep State." / "The world is run by the Illuminati, the Trilateral Commission, the Bilderberg Group, etc."

Analysis and Mitigation

The Churchillian cliche that history is written by the victors offers little solace in the days before history is written. Although ideological truth can often be easily identified by its reliance on "us versus them" framing, mitigation is particularly difficult when assertions involve conspiracy theories that are inherently unprovable. These sorts of assertions can also blur the line with religious or cultural truths, since the "us" and "them" may represent religious or cultural group identities.

An ideological truth is often a historical truth framed through the intent of external or unseen agents ("these people did this thing for that particular purpose"). The best path to establishing veracity, therefore, is to check whether the assertion remains the same without the presence of intent or unseen agents. If one asserts that the US government is hiding the wreckage of a UAP which crashed near Roswell in 1947, a reasonable analysis can negate the malignant role of MIB and instead focus on actual documented crash evidence to ascertain the likelihood that the wreckage was of extraterrestrial origin.

Empiracal Truth

Examples

"I feel unseen." / "No one wants to hear what I have to say."

Analysis and Mitigation

Most formal theories of truth hinge on the distinction between objective and subjective truth – the reality of a shared world vs the reality of individual experience. Classical thought held that objective reality was the only valid measure of truth, whereas individual experience was informed by caprice and conventionality. Such stark distinctions have thankfully fallen out of fashion, replaced with a more nuanced understanding of truth as filtered through individualization and group identity. This mirrors a fundamental change in how we scientifically see the world; the clockwork precision of Aristotle's world has become a fuzzier, quantum mechanical universe in which the nature of reality cannot be excised from the perception of that reality.

This presents us with a form of empirical truth: the truth of experience. The perception of truth can itself be a form of truth. This isn't mere wordplay. If a member of a marginalized population asserts that they feel unseen, the truth of the assertion may little to do with photons and much to do with shared cultural experience. Things like implied bias nonetheless exist in environments where no "objective" measures of bias are being captured.

As with moral or religious truth, the primary value of understanding empirical truth is not to identify a specific vehicle for misinformation. Rather, we should evaluate assertions that reference perceived experience with an awareness of social and cultural context. This avoids inadvertently applying mitigations more appropriate to other types of truth. Just because everyone in the room objectively had the same opportunity to speak doesn't mean that everyone was equally heard.

Summary

• Identifying, triaging and mitigating the impacts of misinformation and disinformation is heavily dependent on an understanding of what constitutes truth

• Many content moderators and algorithms approach truth from a monolithic perspective, interpreting assertions solely on a factual basis

• In fact, there are several different types of truth, each of which should be triaged according to its own criteria; applying standards of factual truth to moral, cultural, religious or empirical assertions will invariably result in faulty analyses

• Although there will undoubtedly be forms of truth not discussed here, this guide is intended as a preliminary tool for creating more nuanced workflows and automation