TL;DR: Suppose Person A states a claim and Person B asks, "What does that mean?" What type of info is B requesting, and how can they describe that to A?
ELABORATION: When I read, hear, or otherwise encounter a claim that seems to warrant skepticism, one useful heuristic I often try as a first pass is to contemplate two simple questions:
- What does that mean? (WDTM)
- How do we know? (HDWK)
Answering those requires (much) more work than asking them but usually leads to insight into the claim's (de)merits. In this post I'll just focus on the first question, which seems vital to ensuring that both/all parties are discussing largely the same thing.* Here are two aspects of WDTM I'd like to understand better:
- What type of information about the claim does WDTM request?
- How can WDTM's request be described succinctly to a "typical person" without getting bogged down in linguistic, philosophical, communication-theoretic, or other specialized concepts or terminology?
To help clarify those two questions about WDTM and stimulate discussion, I'll just list five example claims and mention initial thoughts about what WDTM may mean. To be clear, I'm not particularly interested in these specific claims and could've used any of countless others here instead (e.g., many adages, aphorisms, etc. like Claim 4).
- Claim 1: Religious people are judgmental.
- Claim 2: Skeptics are closed-minded.
- Claim 3: Many associates of the Clintons die mysteriously.
- Claim 4: Better safe than sorry.
- Claim 5: Science harms more than it helps.
When I ask WDTM about these or many other claims, I tend to think in terms of
operational definitions: ways to express the claim more precisely and observably, such as we might if we intended to obtain relevant evidence to support or refute it -- perhaps in an ideal study with infinite resources; this seems related to falsifiability.** For instance, regarding Claim 1 I'd want to know things like who qualifies as "religious" or "judgmental" (e.g., classification rules, measurable attributes), whether this tacitly refers to a comparison (e.g., religious vs. not, more vs. less judgemental), the focal population's spatial or temporal scope (e.g., global vs. geographically limited, current vs. historical), and more. I'd want to know similar things about the other four claims and will just note that Claim 3 intrigues me in that simply trying to answer WDTM raises core problems with this conspiracy theory (i.e., Clinton body count).
Finally, although my operational-definition approach to WDTM often helps my own reflection on a claim, it seems inadequate for broader use in discussions with others, especially folks who aren't familiar or comfortable with scientific inquiry. In particular, for casual conversations or brief interactions wherein someone makes a claim that seems worth questioning, explaining operational definitions seems overly cumbersome and is probably confusing or otherwise off-putting to many folks. So I'd like to find better ways to express succinctly what WDTM means that would more likely further such conversations than derail or halt them, maybe in the spirit of a tool for one's Baloney Detection Kit. Surely others have encountered this challenge and developed more effective strategies.
Does that make sense? Thoughts?
*As topics related to this post we could examine HDWK more closely, consider other simple skeptical questions about claims (e.g., Why does it matter? What should we do?), or track down better skeptical heuristics -- I wouldn't be surprised to find writing/thinking about this from several decades or even centuries ago.
**My tendency is probably influenced by educational and professional training and experience in quantitative psychology, statistics, and research methodology. It also relates to strategies in the problem-formulation stage of evidence-based approaches in various disciplines, such as
PICO and its kin. But other disciplines may offer better strategies for everyday discussions, such as the humanities or legal scholarship.
