Tag Archives: Depression

The perils of scientists who go fishing (Is Oscar the cat making me depressed)

27 Feb

photo

My usual focus here is on materials that are generic, and thus anyone could use them, but I’ve been thinking this week about research methods and thus this idea might really only be applicable to those teaching rational thinking at undergraduate level or above.

If you’ve  read anything else I’ve written here you’ll know that I have a strong focus on students understand the power of the scientific method to find ‘answers’. The current big idea of ‘big data’, and researchers ‘mining’ that data is somewhat alien to me. The  example of ‘big data’ that is currently exercising the British press is the government’s idea to make the data the National Health Service holds available to researchers.

I should say that there are clearly very good reasons to make such data available to researchers. Ben Goldacre has written an excellent article in the Guardian explaining why ‘big data’ is so valuable. My worry about ‘big data’ is that it encourages researchers to go ‘fishing’ through mountains of data without firstly progressing through making an observation, finding a theory and deriving a hypothesis from that theory. I’ve seen a great example of this data ‘fishing’ this week that rather sums up my fears. ‘Describing the Relationship between Cat Bites and Human Depression Using Data from an Electronic Health Record‘ by Hanauer etal was published in the peer-reviewed PLOS ONE journal, and using a very large ‘data mining’ sample produces evidence for exactly what the title describes.

Now, it may well be that this paper has discovered a great unrecognised source of depression (and god knows I’m no stats genius), but I’m really bewildered by the idea of using null-hypothesis testing when you didn’t actually have a hypothesis before you started the study. My worry about this sort of thing was further compounded by reading an excellent article in a recent edition of Nature that gets to the heart of the idea that many scientists really don’t understand the statistics they are using. In particular the article looks at the impact on statistical significance if one factors in the plausibility of the initial hypothesis. Maybe cats really do cause depression (my own cat certainly annoys me when he decides to wake up at 5AM), but just how plausible is this hypothesis, and thus how much can we trust the statistically significant result that has been found.

Here endeth this odd detour into the land of statistics, I shall return to rants about the popular press undermining rational thinking in the near future.

Advertisements
%d bloggers like this: