“We need to reward politicians who give us better data and we need to persecute those who dare to use slight-of-hand and mislead us.”
In his piece on Newsnight on Wednesday night (4th February) Ben Goldacre discussed the need for evidence-based decision marking, especially in the political arena – and how democracy needs evidence, not just principles and ideas. This raised a number of interesting concepts that may not have been fully considered at this stage: including the punishment of politicians who misuse statistics and encouraging politicians to create better evidence of what works and what fails.
“Without good quality evidence we are all flying in the dark”
One idea mooted to create better evidence of what works and what fails is to perform randomised trials of policies.
When introducing any new policies, we could randomly split people into two groups – one group for whom the new policy would apply, the other group would remain with the status quo. There would need to be clear measures of success in place before the changes were made. This would be a definite improvement on the state how policy changes are evaluated.
However, with any trials involving people, we, as researchers, need to get ethical approval. Who would provide ethical approval for this? People participating in studies also need to give informed consent and have the ability to withdraw from a study. In terms of implementing unpopular policy changes this would make finding people to willingly participate in these randomised studies of policy efficacy would be extremely difficult. Should we be willing to bend the standard ethical framework for research in order to measure the efficacy of policies?
In 2013 Iain Duncan Smith (the Secretary of State for Work and Pension) wrongly claimed that official government statistics showed that the coalitions’ benefits cap had got 8000 extra people back into work – briefing journalists secretly before the official data publications. The UK Statistics Authority pointed out exactly how this claim was incorrect “in luxurious detail” (Andrew Dilnot’s official letter to Iain Duncan Smith is here) – that being the extent of the reprimand.
So, how should we appropriately reprimand politicians and others in power how misuse statistics? As opposed to the vicious punishment proposed by Ben Goldacre, I would counter that perhaps additional lessons in the use of statistics would be more appropriate. My, slightly ad-hoc punishment and re-education scheme would be something along these lines:
- 1st Offence: A two hour seminar (or equivalent) with an online test afterwards
- 2nd Offence: A full day course (or equivalent in the number of hours) with an online test with the results of the test published online.
- 3rd Offence: The equivalent of a level one course in statistics (not one aimed at UG mathematics or statistics) with a subsequent essay on why they were wrong (in each of the previous offences) that is published online for public comment which would be graded (pass or fail) by statisticians from the UK statistics authority.
- 4th (and subsequent) Offence(s): Fines – they’ve had their chances; these fines would go towards the funding of the courses for the previous three offences.
These punishment / rehabilitation schemes for misusers of statistics would, of course, be open to refinement and updating as needs be. People would be referred into the scheme for re-education (although some people may consider it to be a punishment) by the UK statistics authority. Those referred into the system would need to repeat attendance at courses until they pass the tests.
To be decided 