Was the Intelligent Design Challenge Intelligently Designed?

A couple of days ago, Ian Musgrave put up The Intelligent Design Challenge at The Panda's Thumb. He gave 6 DNA sequences, and the challenge was to work out which ones were designed by humans. What I thought was interesting about this was not so much finding the answer, but working out what Ian Musgrave was trying to show, and whether the intelligent design community would pass the test he was really setting. Or indeed if they even spotted it.

The reaction of the professional IDers was, apparently, to not enter. Bill Dembski posted the challenge at Uncommon Descent, but went no further (at least publically). Casey Luskin, attack-mouse of the Disco Institute (Casey, any chance of you sending mea signed photo I can put up here?) told us in the UD thread

Dembski’s methods of design detection can discriminate between informational patterns that are produced by chance/law, or alternatively were produced by intelligence. When there is real design to be detected, Dembski’s methods of design detection can work regardless of whether the designer was human or non-human.

and then complained that the other sequences might be of non-human design, so the method won't work. Of course, he doesn't actually try to apply the method, and then note that there might be false positives. Neither are apparently prepared to use Dembski's explanatory filter for anything other than making divine coffee.

One of the amateur IDers, Patrick, did some detective work. He discovered that four of the six sequences are found in the synthetic bacterium Mycoplasma genitalium. The rest of the UD crowd are ecstatic and declare victory. Oh, and complain about how it's an awful challenge. Even funnier, when Musgrave clarifies the challenge, they really pile on, claiming total victory using an appalling cricket analogy (my conclusion is that Patrick didn't read the final delivery and was undone by the wrong'un).

Now, before Musgrave reveals all, I'll try and make an intelligently designed prediction, based on my extensive research (5 minutes BLASTing the sequences). BLAST is an algorithm for efficiently searching DNA sequences for close matches - you give it a sequence, and it tells you what sequences are similar. There is a huge database called GenBank, which should contain all publically available sequences. If you plug in Ian Musgrave's sequences in (as the ironically named Teleological has also done), you find that the same four sequences as Patrick are in there as coming from the synthetic bacterium. But, the final sequence, number 6, is also found in a natural M. genitalium, so it isn't artificially designed. Nobody at UD seems to have noticed this yet.

Two points amused me. Firstly, I was not surprised that there was a little trap - what's the point in setting a challenge like this if you make it easy? The second one is the way Patrick found the solution. Before his post, DLH had pointed to Venter's work on making an artificial bacterium. Patrick acknowledges that he has to use extra information to decide which is designed, and indeed argues that he needs to have this extra information. In other words he needs to know the identity of the designer. This is not good because ID claims that it is possible to detect design without knowing anything about the designer. There is a strong insistence that one shouldn't say anything about the identity of the designer. A cynical interpretation of this is that it is an attempt to avoid admitting that the designer is their Christian God, so that ID is creationism, and hence religion. So, Patrick has demonstrated an important point - in practical design detection one immediately tries to use as much information as one has about the putative designers. And why not?

Oh, silly me. Politics. Read more!

Big Grant Deadline Today!

Today is the deadline for applications to the Finnish Academy, so scientists all over the country are busy writing abstracts and trying to find an amusing acronym for their project. The bad news for them is that they are up against fierce competition.




Yes, the beast will soon send off the description of his project on Integrated Pest Management For The Removal of the Coarse-haired Wombat (Vombatus ursinus) from Finland.

35% is a reasonable proportion of the grant for me to charge as an overhead, don't you think? Read more!

Not a bluff

I got home late today (I might blog about why later), and found two postcards from the post office through my letterbox. For those of you who don't know, if the Finnish post office can't deliver a package, they put a postcard through your letterbox telling you to pick it up.
Anyway, I was expecting one package, with some books in it (CRC Press/Chapman and Hall was having a sale). But two? Anyway, I wandered down to the post office, just before it closed and picked the up. One was large and light, the other smaller but heavier and presumably contained the books. How wrong I was! The larger package was full of statistical goodness, and clear air. The smaller package had several copies of the February issue of Bluff Europe, evidently a poker magazine (and not as exciting as Buff Europe...). I don't even play poker - it's too close to work.
So, if anyone in Helsinki wants some hot tips from Sammy Farha - the coolest man in poker (huh? I thought that was someone else), just get in contact with me. You too can follow Phil Hellmuth's Fantasy Final Table!

Me? I'm just a lawnmower. You can tell me by the way I walk. Read more!

Must See Film!

I wonder if this film is any good:

Read more!

Gender Differences: Need More Data!

I was asked about this last week by a colleague, and now it's hit the blogosphere, so I thought I would publicly leap into a dispute about sexism in science. And make a plea for people to actually look at their data.

This was all started by a group of biologists who have been working at NCEAS on publication biases in ecology (the biggest bias is, of course, that not enough of my papers get accepted straight away). They managed to get their latest results published in TREE.

The received wisdom is that there is a bias against women in science. One area where this might be seen is in acceptance of papers for publication – referees and editors might have a bias (conscious or subconscious) against women. If this is true, the proportion of papers published by women should be higher in journals where the gender of the author is not known.

For this and other reasons there have been suggestions floating around that journals shift to a system of double-blind reviews. At the moment most journals have single blinding: the authors' identities are known to the referees, but the referees' identities are not revealed to the authors (unless the referees wish to do so). In double blinding, the referee doesn't know the identity of the author. Hence, any bias due to gender of the author should be removed. So, if a journal shifts from single blinding to double blinding, the proportion of papers by female authors should increase.

In 2001 the journal Behavioural Ecology moved to double blinding. But did this change the proportion of female authors? Or, more exactly, was there a bias against women that was removed? After all, the proportion of female authors might be changing in the rest of science – the null expectation is that the change in Behavioural Ecology should be the same as in similar journal, rather than there should be no change. So, the group gathered data on the number of papers by male and female first authors from before and after Behavioural Ecology switched to double blinding for five similar journals too. And then they compared the change in proportion of female authors in Behavioural Ecology to that in the other journals.

Err, no.

What they did was to compare the change in the proportion of female authors in each journal to zero. They found was that Behavioural Ecology and Biological Conservation. had increases that were significantly different, but not the other journals. They therefore concluded that there was an effect of double blinding, and that the increase in Biological Conservation must have been due to other factors. Oddly, though, at no point did they seem to make a direct comparison. It is not clear that they looked at the data either. Had they done so, they would have seen this:



The lines show the change from before Behavioural Ecology went double blind to afterwards. The vertical lines are the standard errors. Behavioural Ecology is the thick black line. We see that the proportion of female authors increases in all of the journals, but also that it is greatest in Behavioural Ecology. But is that increase significantly (in any sense) greater than in the other journals? Well, comparing it to zero obviously inflates the estimate of significance, because the other journals are all also increasing.


We can get an idea about if the data show anything with a more focussed analysis. This is also simplified, but I an ignoring some variation, and a more sophisticated analysis (=too much hassle to explain) comes to the same conclusion (and yes, for those who have read the paper, so does including the "don't knows").

What we can do is calculate the difference between the before and after proportions of female authors for the “control group”, and estimate the distribution of differences that would be expected if there was no double blinding implemented. Then we can ask if the difference in the proportion for Behavioural Ecology falls so far outside this distribution that it would be unlikely to explain the change.

These are the differences:

Journal	Percentage Before	Percentage After	Difference (%)
Behavioural Ecology	23.7	31.6	7.9
Behavioral Ecology & Sociobiology	25.1	26.3	1.3
Animal Behaviour	27.4	31.6	4.2
Biological Conservation	13.8	20.6	6.8
Journal of Biogeography	14.4	16.5	2.0
Landscape Ecology	19.5	23.4	3.9

For the journals in black, the mean difference is 3.65%, with a standard deviation of 2.15%. If these were exact, then there would be a 95% chance that the change for another, similar, journal would be between -0.6% and 7.9%. So, Behavioural Ecology is right on the edge.

But it assumes that the variance is known. In reality it is estimated, and only estimated from 5 data points (i.e. not a lot). If we take this into account, we find that the prediction for a journal would fall between -2.3% and 9.6% (with 95% probability). Now Behavioural Ecology is reasonably well inside the limits. Even someone wanting to do a one-sided test will find it inside.

So, the analysis shows little evidence for any effect of double blinding. But there are a couple of caveats, which could have opposite effects. The first is simply that there is not a lot of data – only 6 data points. We would really need more journals to be able to come to any conclusion. In particular, there may have been some other changes at Behavioural Ecology that could have had an effect.

The second caveat is more subtle. Suppose you were a journal editor, and you introduce a rule that authors have to admit that statisticians are the highest form of life in their acknowledgements. After a couple of years, you notice that the proportion of authors called Fisher has increased. You wonder if this is because of the new rule. So, you compare it with other journals, and find no increase. You therefore declare that Coxes appreciate statisticians, but other people don't. But what about all those other effects you didn't see? What about the changes in numbers of Boxes, Coxes, and Nelders? Humans are very good at detecting patterns, but very bad at judging whether they are random. And using the same data from which you spotted a pattern to assess whether it is real is naughty – of course you're going to see an effect, because you've already noticed it in the mass of all possible things that could happen. Now, I don't know if the authors are guilty here – they don't say how they came to decide to examine this particular aspect of the data, but the introduction is a bit arm-wavy about the effect of double-blinding on sex ratio.

Of course, the solution to both caveats is simple – get more data. Anyone fancy trawling through the literature this weekend?

EDIT: Oops. I should have hat-tipped Grrlscientist for her post, which encouraged me to write this. Hedwig - I'm sorry. Please don't set Orpheus onto me...

Reference
BUDDEN, A., TREGENZA, T., AARSSEN, L., KORICHEVA, J., LEIMU, R., LORTIE, C. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23(1), 4-6. DOI: 10.1016/j.tree.2007.07.008
Read more!

Damn Lies in So Many Languages

The ISI Glossary of Statistical Terms

Isn't this just great? You too can find out what the Afrikaans is for heteroscedasticity.

Seriously, this is the sort of resource that is really useful for a small number of people. The web is ideal for it. If only it was easy to find with Google.

I wanted to use the site to find out what the Finnish for "ordinal regression" is. Of course, it's one of the few terms they don't have. So, please tell me! Or tell me what it is in any other language that might be useful - help the ISI!

Powered by ScribeFire.

Read more!

The Truth Revealed

On the way home this evening I was musing about quantum mechanics. Now, I know little more than a layman, but I know enough to be aware that nobody really understands it. It struck me that it must have destroyed several promising academic careers - for someone with an analytical mind, it looks like something that you don't understand, but if you just learn a little ore, and try to put the different bits together then eventually the penny will drop, and it will all become clear.

In reality, of course, what happens is that you get sucked into it more and more, until you realise that there is no way you or anyone else will understand it. If you're lucky, that's after you have retired from a successful career as a theoretical physicist. But then the truth hit me...

One of the points they make about quantum mechanics is that it happens at very small scales, so we can never see it directly. Hence, they have to use instruments to measure the effects. With these, all sorts of bizarre and counter-intuitive result are found. But they are never observed directly. Perhaps this is no accident. Perhaps it was designed.

Here's my just-so story theory. Everyone knew that physicists were the only real scientists, finding Great Truths, and telling us that philately will get us nowhere (*). So, as a joke some poor technician decided to set up the latest instrument they were toiling away at to produce non-sensical results. The experiment was performed, and the results published. Thus was quantum mechanics began.

Of course, the technician couldn't admit to it, but that didn't matter. He just persuaded his friends and colleagues to play along with the joke, and build ever bigger instruments to produce nonsense and allow them to laugh behind the backs of these great Nobel-winning scientists. Then, someone must have mockingly suggested digging out the inside of a Swiss alp. And they even fell for that.

So, now we have a whole subject that makes absolutely no sense, because it is a fiction brought about by a conspiracy of instrumentalists. They can't admit it, because they have invested too much in the scam (as, come to that, have several governments). Instead, they go on building their machines to produce random results, safe in the knowledge that whatever odd squiggle is seen as an output, some theoretical physicist will be able to explain it with some weird theory (just stringing us along, eh?).

Of course, biologists would never fall for something like this, would they? After all, what could we be given by a technician? A machine that magically produces all the DNA we want? Who would fall for that?

(*) There. Two obscure references for the price of one.
Read more!