Why should managers read research?

Why should managers read research?   There has been much talk about the `two cultures’ of management and research – the one focused on today’s pressing problems and pragmatic solutions, the other presenting 200 page reports hedged with qualification and abstraction some years after the first question was posed.  Initiatives have been set up to try to bridge the gap – providing `good enough’ research to inform decisions by managers, whether reconfiguring local maternity care or decommissioning inpatient addiction services.  There is no doubt that managers and researchers inhabit different worlds.  But management without research is anecdote.   And research without use is archive.

In making the case for research, we often stress its instrumental importance – providing hard evidence of the effectiveness and cost-effectiveness of care.  This is important – but what has struck me is the power of research to change the way you think as well as what you do.  When I recently asked a `research-friendly’ trust chief executive what research had most influenced him, he mentioned works on theories of leadership and organisational culture, as well as shifting care out of hospital.   The best research can illuminate, provoke and challenge – as well as inform.  

What follows are a personal selection of papers which I think (for different reasons) arelikely to become modern classics.   In separate blogs, I have extracted the key findings for each, with a personal commentary and sense of what they add to what is already known – plus a cheeky one-line summary for those who really don’t have time for more.  There are a range of methods – from a systematic review of trials to a five-year ethnographic study of one practice.  These include:

Mary Dixon-Woods’ theory-rich evaluation of a major patient safety intervention http://taralamont.blogspot.co.uk/2012/06/why-change-can-be-hardest-word-of-all.html

John Gabbay and Andree Le May’s deep study of a practice to explore how doctors use evidence http://taralamont.blogspot.co.uk/2012/06/what-doctors-really-do-and-why.html;

Sasha Shepperd’s systematic review of hospital at home schemes http://taralamont.blogspot.co.uk/2012/06/as-well-as-or-instead-of-beds-evidence.html

At the Health Service Research Network symposium in Manchester this week, we heard from the likes of Mike Cooke and Liz Mear why all trusts – not just the biomedical beacons – want to engage with research and the new opportunities opening up with AHSNs.  Every trust is now asked for metrics on recruiting patients to trials – and there may be real incentives at a time of pressure in building up research capacity to attract new funds.  But for managers there are also personal rewards in taking the time to read more widely.  Each of these pieces of research are important but also a pleasure to read.  Today’s managers are asked to flex their thinking muscles as never before, to think critically about what we do now and what could be done tomorrow.   This may be a good place to start.

As well as or instead of beds (the evidence on hospitals at home)…

Shepperd S, Doll H, Broad J, Gladman J, Iliffe S, Langhorne P, Richards S, Martin F, Harris R. Hospital at home early discharge. Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: CD000356. DOI: 10.1002/14651858.CD000356.pub3. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000313.pub3/full

Why read this?

Pressure on acute beds leads managers to look for alternatives.    Hospital at home schemes have developed rapidly in different parts of the country, providing care from clinical staff for a fixed period for people who would otherwise need to stay in hospital.   This particular study brings the robust Cochrane methods of systematic review to bear on the important service question – do hospital at home schemes for early discharge lead to better outcomes for patients?  And do they cost less than inpatient stays?   This study published in 2009 looked at 26 published randomised trials (mainly from the UK) comparing hospital at home schemes for early discharge with usual care in hospital for patients with stroke, those recovering from surgery and older people with a mix of conditions.   As well as the results from these individual trials, the researchers led by Sasha Shepperd carried out a meta-analysis, pooling the individual patient data from 13 of these trials to answer the common question – does it work?

This study is important because complex service interventions, like hospital at home schemes, have not always been subject to rigorous evaluation and scrutiny.   Many new ideas spread rapidly across the health service, many with inflated claims of cost savings and benefits – from virtual wards to rapid response teams.  This study is as good as it gets in terms of applying `gold-standard’ research methods to address important service questions.  But it also highlights the limitations of any such study – it is basically as good as the research that is out there.  The small number of robust studies to review and the heterogeneity of interventions, making it very difficult to compare like with like.   In some cases, these were hospital outreach services led by specialists, in other these were community-based services.  Not all provided 24 hour care and the input varied, from specialist or dedicated nurses to range of physiotherapy and occupational therapy staff.  Context is all-important in understanding complex services, but these were not always well described or indeed able to be adjusted for in this method of review.   But this is still a `go-to’ source document for managers reviewing options to relieve pressure on hospital beds.

What does the study say?

For older people and those with a stroke, there was no difference in outcome and quality of life for those in hospital at home schemes compared with usual care.  Patients seemed to prefer hospital at home initiatives and there was no reported increase in burden to carers.   However, the readmission rates to hospital were higher for older people and those with a stroke in hospital at home schemes.  At the same time, there was a lower chance of such people being in residential care at follow-up.  Comparing hospital at homes with inpatient activity, there was evidence of increased total length of stay for patients using hospital at home.    

Overall, there was no evidence that hospital at home schemes generated cost savings.  

The authors took care to discuss and interpret some of the complex messages in this piece of work.  There are no categorical answers – for instance, on the critical cost-effectiveness question, the picture looks much stronger for hospital at home schemes if restricted to patients with mild disability (or where the running costs of local hospitals is high).   There is an argument that numbers for hospital at home schemes are still so small (around 1-2% of hospital patients) that it would be difficult to realise cost savings at scale.   Where local schemes are in place for older people with COPD or stroke, the evidence is not strong enough to discontinue these, given the likely benefits.  But these should probably best be considered as an adjunct to hospital care.  From the evidence to date, the review suggests no compelling case for hospital at homes acting as a substitute for inpatient care.

Bite-size messages for managers

Hospital at home schemes unlikely to justify closing a hospital ward...yet.

What doctors really do (and why guidelines are only part of the answer)…

Gabbay J, le May A (2004).  Evidence based guidelines or collectively constructed `mindlines’?  Ethnographic study of knowledge management in primary care.  British Medical Journal; 329: 1013-1016.http://www.bmj.com/content/329/7473/1013

Why read this?

This is a powerful piece of research, based on more than five years close observation of a high-performing general practice.  This kind of ethnography is often used to great effect, from the jungles of Borneo to the street life of Chicago, to describe culture and practice in different groups.  The authors wanted to find out how clinicians actually use knowledge and where they look to for information and advice on everyday problems.   They immersed themselves in the daily life of the practice, from patient consultations to range of practice meetings with different staff, and checked out their emerging findings on how knowledge is formed with short visits to other practices and settings.

This study from two experienced researchers (both with clinical backgrounds – one medical, one nursing) shows the benefits of careful, embedded research over a period of years, reaching the parts that surveys and interviews alone cannot reach.   These are the realities of a busy Monday morning surgery when the partner is off sick and the locum hasn’t arrived, as opposed to scenario decision-making in the abstract.   As well as fascinating anecdote and case study, this piece of work benefits from wider knowledge of social science theory, from the new discipline of implementation science (pioneered in this country by Martin Eccles and others) to older work on professional knowledge and decision making. This short and powerful article (expanded in a readable book) provides a new way of thinking about evidence-based practice.   Rather than berate clinicians for not `following the book’, it provides illuminating insights into how people really make decisions and what influences practice.

  

What does this study say?

Looking at typical issues, from prescribing statins to screening for early kidney failure, the authors found that clinicians don’t make direct use of guidelines or formal research when they are practising.  But nor do they ignore them – instead, they follow `clinical mindlines’, which is made up of experience (both their own and colleagues in the practice) and a mix of formal and informal evidence.   This includes `tacit knowledge’ – patterns of thinking which may be deeply embedded and almost instinctive.   Several examples are given of these implicit clinical reasoning, such as the decision to admit a woman complaining of feeling `ice in her chest’.  But rather than opposing views of knowledge being codified (textbooks and guidelines) or tacit (intuition based on experience and decision rules), the authors argue that clinicians draw on both in a clinical mindline.  They highlight the importance of loose professional networks within and outside the practice to create and reinforce knowledge.  These informal links were important go give clinical knowledge the `social life’ which would ensure it was used – from coffee room chat with partners in the practice to wider diabetes networks.   Similarly, research showed the importance of stories and anecdotes in embedding knowledge in practice – including the power of real life `near-miss’ stories of patient harm.  In a successful practice, they also observed a collective clinical mindline, where they developed `ways of doing things’ by challenging and making sense of formal evidence, discussing anomalies from experience and refining and updating their knowledge of diseases and how to manage them.

We already knew from Trevor Sheldon and others http://tinyurl.com/8yjacnl more than ten years ago of the limited uptake of NICE guidelines, particularly where messages are complex …  Jeremy Grimshaw http://tinyurl.com/cv4eekv has shown in careful examination of evidence from controlled trials the limited effect of passive diffusion of guidelines.  

What does this mean for managers, who may not engage with clinical decision-making directly?   It throws light on the messy world of how knowledge is used – in fact, as pointed out in another classic research paper (Walshe and Rundall http://tinyurl.com/c63au2v), even more tricky for managers with a dispersed, loosely defined body of social science literature than clinicians where at least the formal knowledge is more organised, with defined hierarchy of evidence and mechanism for extracting recommendations (via NICE).  In making complex decisions like whether to decommission inpatient eating disorder services, managers are much more likely to be influenced by what other trusts have done, advice from experts near to hand, experience with similar specialist services locally and other forms of tacit knowledge than to consult a Cochrane review.   Managers, like doctors, are influenced by `worst last case’ – particularly if it risks ending up on the front page of the local newspaper.

Bite-size take-home message for managers (with apologies to the authors)

Who you know IS what you know.

Why change can be the hardest word of all...

Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ (2011).  Explaining Michigan: Developing an Ex Post Theory of a Quality Improvement Program.  Milbank Quarterly: 89(2); 167-205 http://www.kliniskbiokemi.net/pdfiler/DixonWoods2011.pdf

Why read this?

There is little good evidence on how or why service improvement initiatives work – or don’t work.   This paper highlights key findings from one of the most successful patient safety initiatives.  This was a campaign to reduce (and, in some cases, eliminate) harm in a hundred intensive care units in Michigan.  It resulted in remarkable reductions in central line infections and other avoidable harm.   The programme had a clear protocol and documented resources which others can use.   But as the authors note, programmes rarely keep to their original plans.  As other countries and regions try to replicate these results, greater understanding is needed of the real reasons for success in Michigan.

What distinguishes this review is the power of the analysis and elegance of the writing, combining rigorous theoretical knowledge with insights from the clinicians who led the initiative.  The paper combines the talents of two leading medical sociologists with an interest in patient safety, Mary Dixon-Woods from Leicester and Charles Bosk from the US (whose ethnographic account in the 1970s of why trainee surgeons make mistakes has become something of a classic and well worth a read), with the experience and insight from  the participating programme leaders, including Peter Pronovost.    This generates findings which have the texture of lived experience (and battle scars), as well as the benefits of a rich social science hinterland.  

The paper is also interesting for its methods.  Ideally, ambitious initiatives like Michigan should be accompanied by real-time evaluation, including observation of the programme in action.  However,  this was not possible.  Instead, the authors updated the programme’s initial theory of change with wider theoretical learning from social science and insights from key players – both those creating the programme and local participating intensive care units.   This provides a grounded reality to their findings – plus some useful methodology for others to adopt in retrospective evaluations (although general readers may want to skip the parts on ex post theorisation and Bayesian logic). 

What does this study say?

The key findings from this exercise combining theory, experience and more theory to understand how and why Michigan worked apply to many quality improvement initiatives.   Firstly, once enough organisations sign up to a programme of change, more will follow because of known patterns of peer influence (`isomorphic pressures’).   This may sound obvious, but has implications for new programmes – better to engage a large number of organisations at once, than have a few beacon or showcase sites at the start. 

Secondly, success at Michigan depended on strong horizontal links between units to generate the energy and momentum of a social movement.  These relationships were key in exerting peer pressure and supporting each other through change.  

An interesting third point (which probably could not have been articulated by the leaders themselves) was the need to construct blood-borne infections as a social problem that could be fixed.  At the time, infection was seen as inevitable and tricky to solve (`the problem of many hands’) – the programme re-framed this as an avoidable problem, using variation in infection rates mong participating units as evidence of the potential for improvement.   This was put into practice by the project leaders (or `vertical core’) who provided robust scientific expertise and credibility while also able to inspire and engage others – and respond to challenges from sceptics.  To win hearts and minds, the programme combined hard data and robust evidence-based practice with stories – the 18 month old toddler who died following a catheter infection.   This helped to disrupt existing beliefs about harm and potential for change. 

The fourth element identified by the authors was a few powerful interventions at the sharp end to change practice and culture, with much discussion of the checklist.   Popular accounts of the Michigan project reduce its success to the use of a five-point checklist of evidence-based practice (from hand hygiene to use of chlorhexidine before inserting a line).   This paper gives a careful account of the checklist’s social functions, making visible the discrepancies between actual and ideal practice and licensing challenge from nurse to doctor in a way which subverts usual practice.  The authors use theoretical knowledge to explain how a checklist institutionalises good practice by making it routine, or even ritual. 

A fifth key factor was the use of data collected systematically by the central team  (unlike many system improvements where it is generated locally in often haphazard ways) to measure and benchmark. 

Lastly, the programme provided hard as well as soft tactics to ensure engagement, and deter non-compliance, including sanctions by programme leaders.

(Aside – do the authors make enough of the effect of charismatic leaders?  From a few patient safety events, I can testify to the rockstar effect of clinical champion, Peter Pronovost – named as one of Time Magazine’s 100 most influential people in 2008 and holder of a `genius grant’ (McArthur fellowship).  Not many of them around..)

Bite-size take-home message for managers (with apologies to the authors)

Successful programmes need stories AND statistics.

 

Why stories matter

I was very struck by a good piece of service research from Addenbrook’s presented at today’s HSRN symposium (www.hsrlive.org).  Although the session was focused on productivity, the research from Dr Mai Wong really highlighted the power of patient narratives (and clinical storytelling) in understanding demand.  Her work focused on frequent attenders to emergency departments – often the focus of policy initiatives, with claims of excess service use, but poorly understood.  Dr Wong showed the heterogeneity of this patient population.  The clever thing was the way she translated the descriptive HES data and clinical casenote reviews into archetypes - living, breathing pen portraits.

We were told about Kelly, a young woman with COPD and anxiety who visited A&E ten times a year.  And about Bob, well known to emergency staff, homeless and prone to self-harm and substance misuse,  attending A&E every other week.  These personal profiles helped her to understand the different demands of the moderately frequent attender (where brief psycho-educational interventions might help) as opposed to the extreme frequent attender (Bob), requiring intensive case management.

Why does this matter?  John Seddon (www.thesystemsthingreview.co.uk) punctures myths about lean management (or bastardised versions of it).  One is that standardising service operations is exactly the wrong thing to do if we don’t understand the nature of demand.   Our healthcare systems often multiply demand and activity, as needs of the individual patient are not met upfront.  The work of Christiansen also highlights the need to frontload each care pathway with high quality assessment by teams of skilled clinicians – get it right first time.

So why do we need patient stories?   If we don’t understand how people use services and their underlying needs we won’t improve productivity.

HSRN symposium Manchester, 19 June 2012