Must See Film!

I wonder if this film is any good:

Read more!

Gender Differences: Need More Data!

I was asked about this last week by a colleague, and now it's hit the blogosphere, so I thought I would publicly leap into a dispute about sexism in science. And make a plea for people to actually look at their data.

This was all started by a group of biologists who have been working at NCEAS on publication biases in ecology (the biggest bias is, of course, that not enough of my papers get accepted straight away). They managed to get their latest results published in TREE.

The received wisdom is that there is a bias against women in science. One area where this might be seen is in acceptance of papers for publication – referees and editors might have a bias (conscious or subconscious) against women. If this is true, the proportion of papers published by women should be higher in journals where the gender of the author is not known.

For this and other reasons there have been suggestions floating around that journals shift to a system of double-blind reviews. At the moment most journals have single blinding: the authors' identities are known to the referees, but the referees' identities are not revealed to the authors (unless the referees wish to do so). In double blinding, the referee doesn't know the identity of the author. Hence, any bias due to gender of the author should be removed. So, if a journal shifts from single blinding to double blinding, the proportion of papers by female authors should increase.

In 2001 the journal Behavioural Ecology moved to double blinding. But did this change the proportion of female authors? Or, more exactly, was there a bias against women that was removed? After all, the proportion of female authors might be changing in the rest of science – the null expectation is that the change in Behavioural Ecology should be the same as in similar journal, rather than there should be no change. So, the group gathered data on the number of papers by male and female first authors from before and after Behavioural Ecology switched to double blinding for five similar journals too. And then they compared the change in proportion of female authors in Behavioural Ecology to that in the other journals.

Err, no.

What they did was to compare the change in the proportion of female authors in each journal to zero. They found was that Behavioural Ecology and Biological Conservation. had increases that were significantly different, but not the other journals. They therefore concluded that there was an effect of double blinding, and that the increase in Biological Conservation must have been due to other factors. Oddly, though, at no point did they seem to make a direct comparison. It is not clear that they looked at the data either. Had they done so, they would have seen this:



The lines show the change from before Behavioural Ecology went double blind to afterwards. The vertical lines are the standard errors. Behavioural Ecology is the thick black line. We see that the proportion of female authors increases in all of the journals, but also that it is greatest in Behavioural Ecology. But is that increase significantly (in any sense) greater than in the other journals? Well, comparing it to zero obviously inflates the estimate of significance, because the other journals are all also increasing.


We can get an idea about if the data show anything with a more focussed analysis. This is also simplified, but I an ignoring some variation, and a more sophisticated analysis (=too much hassle to explain) comes to the same conclusion (and yes, for those who have read the paper, so does including the "don't knows").

What we can do is calculate the difference between the before and after proportions of female authors for the “control group”, and estimate the distribution of differences that would be expected if there was no double blinding implemented. Then we can ask if the difference in the proportion for Behavioural Ecology falls so far outside this distribution that it would be unlikely to explain the change.

These are the differences:

Journal	Percentage Before	Percentage After	Difference (%)
Behavioural Ecology	23.7	31.6	7.9
Behavioral Ecology & Sociobiology	25.1	26.3	1.3
Animal Behaviour	27.4	31.6	4.2
Biological Conservation	13.8	20.6	6.8
Journal of Biogeography	14.4	16.5	2.0
Landscape Ecology	19.5	23.4	3.9

For the journals in black, the mean difference is 3.65%, with a standard deviation of 2.15%. If these were exact, then there would be a 95% chance that the change for another, similar, journal would be between -0.6% and 7.9%. So, Behavioural Ecology is right on the edge.

But it assumes that the variance is known. In reality it is estimated, and only estimated from 5 data points (i.e. not a lot). If we take this into account, we find that the prediction for a journal would fall between -2.3% and 9.6% (with 95% probability). Now Behavioural Ecology is reasonably well inside the limits. Even someone wanting to do a one-sided test will find it inside.

So, the analysis shows little evidence for any effect of double blinding. But there are a couple of caveats, which could have opposite effects. The first is simply that there is not a lot of data – only 6 data points. We would really need more journals to be able to come to any conclusion. In particular, there may have been some other changes at Behavioural Ecology that could have had an effect.

The second caveat is more subtle. Suppose you were a journal editor, and you introduce a rule that authors have to admit that statisticians are the highest form of life in their acknowledgements. After a couple of years, you notice that the proportion of authors called Fisher has increased. You wonder if this is because of the new rule. So, you compare it with other journals, and find no increase. You therefore declare that Coxes appreciate statisticians, but other people don't. But what about all those other effects you didn't see? What about the changes in numbers of Boxes, Coxes, and Nelders? Humans are very good at detecting patterns, but very bad at judging whether they are random. And using the same data from which you spotted a pattern to assess whether it is real is naughty – of course you're going to see an effect, because you've already noticed it in the mass of all possible things that could happen. Now, I don't know if the authors are guilty here – they don't say how they came to decide to examine this particular aspect of the data, but the introduction is a bit arm-wavy about the effect of double-blinding on sex ratio.

Of course, the solution to both caveats is simple – get more data. Anyone fancy trawling through the literature this weekend?

EDIT: Oops. I should have hat-tipped Grrlscientist for her post, which encouraged me to write this. Hedwig - I'm sorry. Please don't set Orpheus onto me...

Reference
BUDDEN, A., TREGENZA, T., AARSSEN, L., KORICHEVA, J., LEIMU, R., LORTIE, C. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23(1), 4-6. DOI: 10.1016/j.tree.2007.07.008
Read more!

Damn Lies in So Many Languages

The ISI Glossary of Statistical Terms

Isn't this just great? You too can find out what the Afrikaans is for heteroscedasticity.

Seriously, this is the sort of resource that is really useful for a small number of people. The web is ideal for it. If only it was easy to find with Google.

I wanted to use the site to find out what the Finnish for "ordinal regression" is. Of course, it's one of the few terms they don't have. So, please tell me! Or tell me what it is in any other language that might be useful - help the ISI!

Powered by ScribeFire.

Read more!

The Truth Revealed

On the way home this evening I was musing about quantum mechanics. Now, I know little more than a layman, but I know enough to be aware that nobody really understands it. It struck me that it must have destroyed several promising academic careers - for someone with an analytical mind, it looks like something that you don't understand, but if you just learn a little ore, and try to put the different bits together then eventually the penny will drop, and it will all become clear.

In reality, of course, what happens is that you get sucked into it more and more, until you realise that there is no way you or anyone else will understand it. If you're lucky, that's after you have retired from a successful career as a theoretical physicist. But then the truth hit me...

One of the points they make about quantum mechanics is that it happens at very small scales, so we can never see it directly. Hence, they have to use instruments to measure the effects. With these, all sorts of bizarre and counter-intuitive result are found. But they are never observed directly. Perhaps this is no accident. Perhaps it was designed.

Here's my just-so story theory. Everyone knew that physicists were the only real scientists, finding Great Truths, and telling us that philately will get us nowhere (*). So, as a joke some poor technician decided to set up the latest instrument they were toiling away at to produce non-sensical results. The experiment was performed, and the results published. Thus was quantum mechanics began.

Of course, the technician couldn't admit to it, but that didn't matter. He just persuaded his friends and colleagues to play along with the joke, and build ever bigger instruments to produce nonsense and allow them to laugh behind the backs of these great Nobel-winning scientists. Then, someone must have mockingly suggested digging out the inside of a Swiss alp. And they even fell for that.

So, now we have a whole subject that makes absolutely no sense, because it is a fiction brought about by a conspiracy of instrumentalists. They can't admit it, because they have invested too much in the scam (as, come to that, have several governments). Instead, they go on building their machines to produce random results, safe in the knowledge that whatever odd squiggle is seen as an output, some theoretical physicist will be able to explain it with some weird theory (just stringing us along, eh?).

Of course, biologists would never fall for something like this, would they? After all, what could we be given by a technician? A machine that magically produces all the DNA we want? Who would fall for that?

(*) There. Two obscure references for the price of one.
Read more!

Dyed in a blogging accident

This post is for the dyslexic xkcd fans.


At the moment, Google gives about 1,650 hits for "died in a blogging accident", but it seems nobody has yet changed colour as a result of a mishap whilst posting their thoughts to the web.
Read more!

Black Hole Found by Finns

A Finnish team makes the news:
BBC NEWS | Science/Nature | Huge black hole tips the scales

Yes, I was the one who told the where to find Scunthorpe. Read more!

The On-Flight Service Was Terrible

Finally, a good reason to work/live in a skyscraper.

I must admit to occasionally wanting to stand outside the World Trade Centre in Helsinki and throw paper aeroplanes at it. Read more!