Archive | February, 2014

The perils of scientists who go fishing (Is Oscar the cat making me depressed)

27 Feb


My usual focus here is on materials that are generic, and thus anyone could use them, but I’ve been thinking this week about research methods and thus this idea might really only be applicable to those teaching rational thinking at undergraduate level or above.

If you’ve  read anything else I’ve written here you’ll know that I have a strong focus on students understand the power of the scientific method to find ‘answers’. The current big idea of ‘big data’, and researchers ‘mining’ that data is somewhat alien to me. The  example of ‘big data’ that is currently exercising the British press is the government’s idea to make the data the National Health Service holds available to researchers.

I should say that there are clearly very good reasons to make such data available to researchers. Ben Goldacre has written an excellent article in the Guardian explaining why ‘big data’ is so valuable. My worry about ‘big data’ is that it encourages researchers to go ‘fishing’ through mountains of data without firstly progressing through making an observation, finding a theory and deriving a hypothesis from that theory. I’ve seen a great example of this data ‘fishing’ this week that rather sums up my fears. ‘Describing the Relationship between Cat Bites and Human Depression Using Data from an Electronic Health Record‘ by Hanauer etal was published in the peer-reviewed PLOS ONE journal, and using a very large ‘data mining’ sample produces evidence for exactly what the title describes.

Now, it may well be that this paper has discovered a great unrecognised source of depression (and god knows I’m no stats genius), but I’m really bewildered by the idea of using null-hypothesis testing when you didn’t actually have a hypothesis before you started the study. My worry about this sort of thing was further compounded by reading an excellent article in a recent edition of Nature that gets to the heart of the idea that many scientists really don’t understand the statistics they are using. In particular the article looks at the impact on statistical significance if one factors in the plausibility of the initial hypothesis. Maybe cats really do cause depression (my own cat certainly annoys me when he decides to wake up at 5AM), but just how plausible is this hypothesis, and thus how much can we trust the statistically significant result that has been found.

Here endeth this odd detour into the land of statistics, I shall return to rants about the popular press undermining rational thinking in the near future.

The Daily Express can’t make the numbers add up !!

18 Feb


The front page of yesterday’s Daily Express managed to combine glorious unintended irony with some really questionable reporting of statistics. AS you’ll see from the image above the lead story reported an opinion poll where 70% of respondents wanted all immigration to be stopped immediately. It’s quite entertaining that of three people pictured on the front page one was born in India and a second is the daughter of Russian immigrant parents (presumably of the three it is Simon Cowell that the Express are happy to have as a citizen !).

Despite the glorious irony of the pictures, it’s the stats that I find really interesting. The stories headline is very clear that ‘70% say we must ban new migrants’, and yet the very first line of the story says ‘Almost three out of four Britons want immigration to be reduced or stopped completely, a poll shows’ (My BOLD). So, within the space of one line we’ve gone from 70% wanting a complete ban to 70% wanting a reduction !!! If you then look at the actual survey on which the story is based you discover that rather than 70% of respondents wanting an immediate ban on immigration the actual figure was 21%, with a rather 49% wanting immigration to be reduced.

This seems like a lovely example to introduce students be being discerning consumers of the media, after all a contradiction between the headline and the very first line of the story ought to be easy to spot.

(P.S. It’s just been pointed out to me that the mother of Simon Cowell’s son is American, and thus it may be three of the four people pictured on the front page that the Express have an issue with !!!)

Does pretending to be Voldermort really make you bad ????

10 Feb


The news media is awash today with reports of a study that demonstrates that playing a ‘baddie’ in a video game can have a impact in your behaviour in the real world. From the Daily Mail in the UK to The Indian Express ‘Know Thy Avatar: The Unintended Effect of Virtual-Self Representation on Behaviour’ has had huge media coverage. (Sadly the original article is behind a paywall, but a decent summary can be found here).

What’s interesting about this story, from the rational thinking perspective, is that their is very little wrong with the science, in that it’s very concisely reported in a highly reputable journal. It does however serve as a nice example for getting students to think about drawing conclusions from studies, and whether studies really have the ‘real world’ implications that the popular press suggest. In the case of this study, what it demonstrates is that the psychological impact of playing either ‘Voldermort’ or ‘Superman’ in a video game for 5 minutes lingers long enough for those playing the baddie to decide to give a future participant a dose of chilli sauce to eat (The ‘Baddie’ response), rather than some chocolate to eat (The ‘Goodie’ response).

I have two questions that I’ll be asking my students about this study:

1) Is it really that surprising that role-playing has an impact on behaviour immediately after the role-playing ? I’m thinking about a Shakespearian actor playing Richard III. It seems entirely plausible to think that playing that role for 3 hours per night might impact on an actors behaviour the minute they left the stage.

2) Isn’t the really interesting question ‘How long does the impact of playing Voldermort hang around for ? The authors of this study have demonstrated clearly that role playing has a significant effect on behaviour immediately afterwards, but surely what is really interesting is whether that role-playing has any longer term impact. If it was to be found that the impact dissipates after 5 minutes, it would still be an interesting piece of work, but hardly ground breaking.

All in all this seems like a really nice illustration for students that they need to engage their rational thinking skills even with a perfectly respectable piece of research. (I also wonder how long it will take those who are on a ‘computers are damaging our brains’ crusade to latch onto this story)

Learning Styles: A topic in need of some rational thinking

7 Feb


As any group of students if they know anything about ‘Learning Styles’, and you’ll be surprised at how pervasive the concept is. The idea that we might all have our on personal ‘best way’ of learning is very intuitively appealing, and I suspect this is exactly why educational policymakers latched on to ‘Individualised Learning’ so quickly. If you haven’t come across Learning Styles before you can find a decent summary here, but in essence it’s the suggestion that we each have a favoured method of learning : one might be a visual learner (favouring pictures/ diagrams etc or an auditory learner, favouring spoken explanations etc)

This idea gained huge traction in education. I can remember meetings where university management explained that I needed to take into account student learning styles when designing teaching, and if you ask any students with kids you soon find that Learning styles are still around in schools. What’s really intriguing from a rational thinking perspective about Learning Styles is that as soon as you think about the idea it starts to fall apart.

By far the best demolition of learning styles can be found in Daniel Willingham’s work, but you can see what’s wrong with it merely by thinking about how you’d design an experiment to test it. Imagine two groups of participants, one made up of auditory learners and the other made up of visual learners. You give each group a sheet of paper with a list of words to learn, and you read out a second list of words to learn. You’d predict that the visual learners would remember more of the paper list and the auditory learner would learn more of the list read aloud. Of course, if you do that study you don’t find any difference at all.

I like to use this a an example for students who have already understood the importance of the scientific method. They can then easily come up with the experiment suggested above for themselves. Equally, I think it’s a lovely example of yet another idea that once lodged in the public consciousness is very difficult to dislodge.

%d bloggers like this: