The One Reason Your Correlation Results Are Probably Wrong

I'm sure you all know how to do a correlation analysis to test if one variable is related to another. You probably even know when to use a Spearman correlation test over a Pearson correlation test.

But did you know that the answers from these tests are probably wrong?

The one reason your correlation results are probably wrong

When you have a pair of variables, finding a statistical relationship between the pair of them is pretty straightforward. It’s easy to see that where you have maybe a dozen variables you can analyse them pairwise using standard correlation tests and find out which relationships exist in your dataset, and which do not.

It's so simple it's enough to make you feel smug and self-satisfied - maybe even a stats hotshot.

Not so fast, cowboy – it’s not quite that simple!

You see, when you analyse a pair of variables using univariate tests, you’re testing to see whether there is a relationship between these variables without taking into account any other potential factors.

There are loads of ways in which your variables might be interacting with and influencing each other, so when you have a significant p-value from univariate analysis you can’t be sure that the answer you get is correct.

Let me make it easy for you.

If you get a non-significant p-value (larger than 0.05), you can be pretty sure (actually, 95% sure) that there is not a direct relationship between your variables. That’s not to say that one does not influencethe other indirectly, it may do, but there is not likely to be an independent relationship between them.

On the other hand, if you get a significant p-value (smaller than 0.05), the best you can say is that there may be a relationship between them. The relationship might be independent, but equally it might not.

This flow chart might help you a little:


Feeling a little less smug now, aren’t we?

So if univariate tests don’t give us the answers we need, where do we go from here? Well, the univariate tests are still useful to us. Remember that univariate tests are pretty good at telling us when there isn’t a direct relationship between a pair of variables.

This is useful information and allows us to narrow the field of possibilities between what might be related to your main variable (aka hypothesis variable) and which ones aren’t.

So, in turn you test each variable against your hypothesis variable to see which of them are notrelated. Then you discard them. What remains are the variables that might be related to it.

The next step gets tricky because we now need to test the relationship between the hypothesis variable and all of these variables whilst taking into account all the possible interactions between them. Sounds scary!

We’re now dipping our toes into the world of multivariate analysis.

I'm not going to go into detail about univariate and multivariate correlations here because I explain all about them in my ​FREE eBook Beginner's Guide to Correlation Analysis.

In this book you'll learn:

  • the difference between correlations, associations and statistical relationships
  • how to analyse your data to get a better understanding of what it's trying to tell you
  • how to use both univariate and multivariate statistical tests to pin down the correct story of your data first timeevery time

You can get your copy of this book right here:

Download your Free Ebook Correlation is not Causation

I will give you a little advice though: do univariate analyses on your data first to get a good understanding of the underlying patterns of your data, then confirm or deny these patterns with the more powerful multivariate analyses. This way you get the best of both worlds and when you discover a new relationship, you can have confidence in it because it has been discovered and confirmed by two different statistical analyses.

When pressed for time I’ve often just jumped straight into the multivariate analysis. Whenever I’ve done this, it has always ended up costing me more time – I find that some of the results don’t make sense and I have to go back to the beginning and do the univariate analyses before repeating the multivariate analyses.

 I advise that you think like the tortoise rather than the hare – slow and methodical wins the race…

Find out the one reason your correlation results are probably wrong. #correlation #statistics #freeebooks

What happens if your correlation results are wrong? Find out how to tell if they are in this blog post. #correlation #statisticsFind out the one reason your correlation results are probably wrong. #correlation #statistics #freeebooks
Find out the one reason your correlation results are probably wrong. #correlation #statistics #freeebooks

Discover the world of correlation analysis

Beginner's Guide to Correlation Analysis
Get your FREE download when you become a subscriber and learn how to look at correlations with a focus on understanding the data and choose the right ways to analyse it.

By submitting this form you are granting us permission to send you Chi-Squared Innovations Newsletter. You can unsubscribe at any time. Privacy Policy