I have a dangerous understanding of statistics.
About 20 years ago, I wanted to include cognitive psychology in my research. But I'd never taken a course on statistics, so I taught myself using a textbook. This enabled me to make use of cognitive psychology research, which reports conclusions of their research using statistics.
But I am unable to evaluate the quality of the conclusions based on the methodologies. I understand what the statistical terms mean, but I'm not qualified to evaluate the quality of the work that led to the conclusion.
Instead, I stick to sources that have had experts other than the author review the research to ensure its quality.
To put this point another way, some of the things that cognitive psychologists claim make intuitive sense to me. I read the conclusion and it matches my experience, and so I expect it to be true. But that doesn't mean it is.
As science has demonstrated over and over again, many times it's the explanations that don't make sense to us that are the correct ones. The history of scientific inquiry is littered with examples of explanations that sounded wrong, but turned out to be right.
Though germ theory (the view that microorganisms we cannot see with the naked eye make us sick) has been around for 500 years or more, it was routinely rejected until the end of the 19th century when Robert Koch was able to connect a specific germ to a specific disease.
Before then, germ theory didn't make sense based on the current understanding. And yet, the methodologies of science demonstrated that this was in fact the case.
When Barry Marshall and Robin Warren first suggested that bacteria were the cause of peptic ulcers, this explanation was also met with skepticism. Two decades later, they won a Nobel Prize for the research that demonstrated that this was in fact the case.
Relying on what makes sense or sounds right is dangerous even if it has statistical claims attached to it.
Just because it makes sense doesn't mean it's true.
Just because I like the explanation doesn't make it accurate or doesn't mean that it matches reality.
Instead, what we need are experts (be it in biology, psychology or any other study of our world) who review the claims based on their expertise in that area. In specialized areas of inquiry, I can't rely on myself to discern what's true from what's false without the help of experts.
So when you encounter someone who tells you that there's “overwhelming evidence” of some conclusion about our world, you should check their sources. Because the evidence that matters is not the evidence you agree with, but the evidence that experts have scrutinized and endorsed.
This is particularly important for claims and conclusions with which you already agree.
Psychological science has produced overwhelming evidence that we tend to pay too much attention to claims and conclusions that support what we already believe, and not enough to the claims and conclusions we disagree with even when they're accurate.
But don't take my word for it. You should look it up.