The Natcen report on disability benefits is disappointing

The DWP declined to publish the Natcen report on The uses of health and disability benefits.  It’s now been released to the Work and Pensions Committee, and is available here.  The report is presented as a qualitative study; wisely, the authors have avoided numbers.  However, the form of the report is disappointing, and I cannot suppose that it is an adequate reflection of the work that was done.

Qualitative social research works, or needs to work, on two guiding principles.  The first is that what people tell you is evidence.  That is often derided by people in love with quants, but it is fundamental to the nature of evidence.  Courts of law judge evidence by looking for corroboration – probabilities and statistics aren’t enough.  As a broad proposition, evidence is corroborated when two or three witnesses  say the same things, confirming what the others have said.  This report doesn’t do that. It consists very largely of the researchers’ summaries of what people told them.  In a 79-page report, there are 15 ‘case illustrations’, and only 27 direct quotations from respondents.  That means, simply put, that while there are plenty of judgments, there is hardly any evidence given for those judgments.

The second principle is voice: what people tell you, and the way they tell it, matter.  Direct quotations, right or wrong, have a purpose and a moral authority.  Researchers have an ethical duty to report what people are telling them.  The way the respondents express themselves is fundamental to any adequate qualitative social research.

It may well be that this format was demanded by the DWP.  (I’ve been asked by other commissioners, in the past, to dump direct quotations and to just say what I think instead – and I’ve refused to do so.)  It’s very likely that the researchers intend to provide the evidence for this report, and to reflect the voice of the respondents,  in a separate publication.  However, they will need the DWP’s permission for this.  Their report, as it stands, does not do what it needed to do.

Leave a Reply