How numbers help – from weather to walk-in clinics

Off last week to flood-bound Exeter, for a stimulating two-day conference led by Martin Pitt at Peninsula Medical School (http://www.hsrlive.org/events/change-by-design-systems-modelling-and-simulation-in-health-care).  It was designed to bring together clinicians, managers and patients with researchers practising those strange sciences of systems modelling and simulation.  These techniques have been under-used in health, but there was a palpable sense of excitement over these two days that this was an approach whose time had come.   

This is not new – health planners in the 1950s were using primitive modelling methods for booking outpatient systems.    But latest techniques embrace the complexities of health and social care, the uncertainties and the multiple interests of commissioners, range of providers and patients.  It is no longer – and perhaps never has been – a two dimensional numerical exercise.

We heard inspiring stories of how the particular techniques of operational research had been brought to bear on tricky NHS problems.   These include using queuing theory to allocate and share scarce specialist mental health assessment slots between teams; applying stochastic modelling techniques to predict ambulance response times and plan rosters; using scenario planning to allocate capacity between medical, surgical and cardiac beds on `service lines’ in paediatric intensive care; and using system dynamics to re-model the entire unscheduled and emergency care systems in one locality. 

There was a great presentation from Paul Harper, using software animations to illustrate the dangers of planning capacity on averages.  If you fail to build in variability, a given in most systems dependent on human behaviour, your estimated average wait of 30 minutes in a walk-in centre becomes two hours.  Check out his youtube presentation (http://www.profpaulharper.com/home/research/research-materials) .  This made me think of a brilliant book I read recently on the dangers of relying on `common sense’ by the US engineer turned sociologist, Duncan Watts (http://www.amazon.com/Everything-Obvious-Common-Sense-Fails/dp/0307951790).  A common sense planner would schedule outpatient waits based on average times from reception to work-up with  nurse to doctor.  This would be wrong.   A quote by Watts - “the whole trick is to know what variables to look at and then know how to add” – could itself be an epigraph for operational research.

One of the best parts of the two day event was a sandpit exercise where small groups of service leaders and operational researchers quickly worked up bids for new projects.  These were pitched to the room, dragon den style.   The outputs were impressive - from using location analysis to site diagnostic services across one region to modelling how best to implement NICE guidelines for DVT care.

I ended the day talking to a paediatrician who had stumbled on the event, with no prior knowledge of systems modelling, and was inspired to get analytic help when making a business case for a new specialist epilepsy nurse and pathway redesign.  There is a tension though between the very applied, local problem-driven analytics and a more lasting body of knowledge.  Sally Brailsford (mathematician, turned nurse, turned health modelling academic) had pointed to the paradox – we have a huge body of evidence, but few generaliseable outputs.  She had identified 1008 individual papers on re-modelling emergency department flows.   Were all these necessary?  How can we learn from the best?   As well as the embedded local analysts within a health organisation focused on particular problems, we need high quality research studies to generate national learning, by testing and validating models and carrying out robust evaluations of impact.

And so a long return from Exeter, with rather trying transport arrangements given the flood damage.   During discussion, some had raised the old argument that healthcare was just too complex to lend itself to mathematical techniques.   The same of course used to be said of weather forecasting, where predictions of more than three days  were notoriously inaccurate.  But today’s weather modelling techniques, using historic data from multiple sensors and understanding the interplay of solar activity, land masses, water temperatures and wind flow are much better.   Applied to health, techniques such as system dynamics can build in uncertainties (such as patient preferences) and variability (patient and clinician behaviours), with more sophisticated understanding of interactions (through network analysis and other) to predict more accurately how services might be used and savings could be made.   Scenario planning can also present various `what-ifs’ to integrate strategic uncertainties – a given in the NHS – into the planning process.  Numbers themselves are not enough.  But at a time of ever tighter financial pressures, can we afford to ignore the weathermen?

From Bletchley Park to NICE

What can Bletchley Park teach us about organisations and organisational life?  I have just read an excellent book http://tinyurl.com/9q3gyff by Christopher Grey, Decoding Organization  (great title), who brings his academic experience in organisational studies, together with a deep delve into archives, to the iconic site of Bletchley Park.  Much has been written on this, but it is strangely under-studied as an organisation.  Grey provides rich insights while debunking myths.  Yes, there really were chess-playing tweedy mathematical geniuses recruited from Cambridge colleges in Hut 6 – but at the same time it was a complex organisation of 10,000 staff, three quarters of whom were women.  

In many ways the story of Bletchley Park challenges all modern precepts of successful organisations – clear leadership, open culture, shared objectives and feedback to staff.  At Bletchley Park, it was not clear who was in charge, with continuous friction between the different agencies at the helm , from the Admiralty to the Foreign Office.    The chain of command was obscure - a US navy liaison officer arriving in 1942 was amazed to find no organisational chart.  This despite the workplace having grown to a complex web of listening stations, interception, decoding and intelligence functions.  There were no shared work goals - beyond the overarching mission and unifying force of war, which Grey does not discount.   Textbooks on successful management would have chief executives get up at staff meetings and tell rousing stories of what the organisation has achieved.   At Bletchley Park, there was little shared information – indeed, many or most of those working there knew nothing of the breaking of Enigma ciphers, a momentous act which experts reckon shortened the war by two years.  Instead, the organisation was characterised by secrecy and highly compartmentalised units.   It has been described as a “multiple series of concentric circles”, with 47 fairly autonomous sections.  One particularly telling story which Grey tells is of a couple who had both worked at Bletchley Park but never revealed the fact to each other until thirty years later. 

Modern management stresses the need for a strong organisational culture with shared values and beliefs.  But Bletchley Park appears to be “a multiplicity of different agencies with potentially competing interests.”   Grey argues that the organisation was formed from conflicts and negotiations between very different cultures - from the military and civilian, the dons and the clerks.     

So how did it work?  Grey describes the dense web of friendship connections between these individuals, including the shared backgrounds of many and informal recruitment through universities.    While antithetical to modern notions of equal opportunity, some of the core activities were supported by high levels of trust and interconnectedness on a personal basis, something which Grey calls informal `micro-networks’.  This enabled for instance competing heads of naval and army `huts’, who had been friends at university, to work out solutions for competing demand for rare resources – in this case, use of analytic devices or bombes.  Strong pre-existing personal and social  connections helped to avoid institutional conflicts.  They also provided a cultural structure that overcame the incoherent organisational structure of Bletchley Park.   In this way, it most closely resembles more recent kinds of knowledge-intensive organisations in Silicon Valley.

Many accounts celebrate the eccentricities and amateurism of Bletchley Park – one source notes that a high-ranking foreign visitor was appalled at the (effective) indexing system housed in shoe boxes.   But Grey favours the notion of `organised anarchy’ – Bletchley Park was charcterised both by efficient rule-based standardisation of work (associated with formal bureaucracy), much of it routine, as well as reliance on personal initiative, networks and discretion.  Not either, but both.   Similar arguments have been made for the craft (or art?) of medicine, where evidence-based guidelines do not substitute for professional judgement. 

It is interesting that Bletchley Park is recognised as more successful than its equivalents in Germany or US. This is partly because it brought together for the first time the separate functions of interception, cryptanalysis and intelligence, creating a new kind of organisation.   But there were also different ways of working, mobilising large numbers of staff on a temporary basis for particular projects, enabling a degree of flexibility and innovation which more established military and administrative structures in other countries may have inhibited.  Something of the spirit of the Olympics Games Makers perhaps. 

Why does it matter?  This thought-provoking study brings academic rigour – with Grey’s broad hinterland of organisational theory - together with a narrative of a time and place which still fascinates us.  It makes me think about some high-performing hospitals, with tight networks of semi-autonomous specialist coteries, and `light touch' general management.  Grey's insight is that Bletchley Park's success may be because of its organisational chaos and porosity and the tensions between diverse units - not despite it. 

But it is also his methods and storytelling which excite.  Could we use this power of analysis to learn more about NHS organisations and our recent history?  Could we for instance start to decode the success of NICE as a unique British institution?  What particular confluences led to the creation of this new institution in 1999 – including the momentum of evidence-based medicine, the Child B case and other headlines on postcode lottery  generating the need for political distance and a process to manage the demand for expensive new drugs and treatments?  How much did the continuity and traits of the `three at the top’ (Mike Rawlins, Andrew Dillon, Peter Littlejohns) contribute to its longevity?  What are the tensions between the rational enterprise of evidence-based decision-making and the competing interests of different parties (industry, patients, clinicians and politicians) and how were these played out in some of the big stories (tamiflu or drugs for kidney cancer or Alzheimer’s disease)?   There are rich seams of structure, agency, culture and political process to mine here. 

An aside – I found out  that Christopher Grey shares my enthusiasm for the under-rated and deeply unfashionable novels of C P Snow.  Snow was himself a key figure in recruitment at Bletchley Park, having moved from scientist to war-time director of the Ministry of Labour.  I can’t think of a writer who describes better the emotional intensity of working life – from political intrigue (Corridors of Power, The Masters) to scientific fraud (The Affair).   Our anorak passion is shared by Muir Gray, I found in a recent twitter exchange, who insists his registrars read Snow’s novels to understand how policy and organisations work.  Time for a revival?

What does a good hospital look like?

The biggest question in health care at the moment is how to reduce costs without compromising quality.  But we still know very little about the relationship between inputs or costs and outcomes for patients.  There is some research – largely qualitative – on characteristics of high-performing hospitals.  And it is mainly hospitals, rather than other kinds of provider organisations, which are studied.  But even this literature is characterised by a circularity of argument – good facilities attract good staff which treat patients well…  And is it true?

I was thinking about this when reading a paper by Veena Raleigh and colleagues looking at patient-reported experience across services http://tinyurl.com/dxqstho. They found that some hospitals consistently performed better than others.  Their analysis showed that foundation and teaching status and proportion of white inpatients were positively associated with high patient ratings – deprivation, not so much.   In other words, some hospitals in tough places still managed to perform consistently well on patient measures such as dignity, respect, cleanliness.

What else do we know?  Jha and colleagues in large quantitative analyses in the US looked at the relationship between efficiency and structural characteristics, such as nursing levels and outcomes http://content.healthaffairs.org/content/28/3/897.abstract.  They found that low-cost hospitals perform worse on quality indicators and patients were less satisfied.  So much, so predictable perhaps – but worth stating in a climate where payers may look only at the bottom line and seek out low-cost alternatives.  A more recent study by Stukel and co in Canada http://jama.jamanetwork.com/article.aspx?articleid=1105068, confirming these findings, provide further insight into why hospital costs may be related to patient outcomes.  Their careful work showed for instance that patients in high-spending hospitals received a higher intensity of nursing care and more visits from specialists.

There is also an interesting debate about performance by specialty or whole-institution – Dr Foster and all, take note.   This is a large and contested area, but research for instance by Shwartz and colleagues in the US http://mcr.sagepub.com/content/68/3/290.abstract suggest that hospitals who perform well on a composite measure were often not in the top quintile for individual measures.  That is, there may be pockets of excellence from clinical teams or specialties in otherwise poor performing trusts.

There does not appear to be a simple blueprint for successful organisations.  For instance, we know that larger is not always better – an evidence review by Rod Sheaff some time back found no consistent relationship between the size and performance of an organisation (over and above the relationship between volume and quality for specialist procedures). http://www.netscc.ac.uk/hsdr/projdetails.php?ref=08-1318-055  The review similarly found no consistent or strong relationship between performance and other factors such as leadership style or economic environment.

So where do we go from here?  Veena Raleigh’s work suggests system-level determinants of good patient experience – and that some trusts can deliver against the odds.   We know something – largely from North America – about the relationship between organisational input and performance.  But, given the importance of the question, the paucity of high quality research in this area is striking http://www.ncbi.nlm.nih.gov/pubmed/22871420.   It would be good to see ambitious studies in the UK which tackled these big questions.   What does good look like – and do we get what we pay for?

So where are the doctors? (...in patient safety research)

Bob Wachter, a leading US clinical researcher/leader (of `hospitalist’ fame) over here on a sabbatical last year mentioned in passing his personal roll-call of influential figures from this side of the water on patient safety research.  Jim Reason, Charles Vincent, Mary Dixon-Woods… all social scientists.  Where were the doctors?  In the US, the leading lights combine research and clinical leadership – Atul Gawande, Peter Pronovost, David Bates, Lucian Leape, Don Berwick. 

A few exceptions come to mind – Liam Donaldson, responsible for setting the agenda at a national (and international) level.  Tony Avery (GP) and his work on prescribing errors, Peter McCulloch (surgeon) and his crowd-pleasing studies on operating theatres using Formula One handover techniques.  And other professional groups have their research luminaries – particularly pharmacy, thinking of the work of Nick Barber and Bryony Dean Franklin, from medication safety in care homes to evaluation of electronic prescribing.  Nurse leaders have been prominent in safety campaigns and initiatives (for instance around infection control) – perhaps less notable in research and setting the framework for debate. 

But the absence of prominent medics as patient safety researchers and thinkers is puzzling.  This may be part of a broader issue – few trust chief executives in this country have a clinical background.   In the US, Goodall’s work http://www.ncbi.nlm.nih.gov/pubmed/21802184 showed a positive association between high-performing healthcare facilities and leadership by a physician.  My quick googling of chief executives of high-performing trusts (QUEST) on quality/safety markers show none with an obvious medical background, one from nursing.   There’s a whole other debate around medical leadership and the interesting hybrid medical/manager role from the likes of Peter Spurgeon and Chris Ham.

Does it matter?  Flip another way and you could cite patient safety research as an example of social scientists leading the way – from Jim Reason’s analysis of latent threats and system weaknesses to Charles Vincent and others (Sari, Hogan) measuring the rate of harm to robust evaluations of complex safety interventions (Dixon-Woods, Benning).  It has been exciting to see other, newer disciplines outside health come to the fore – human factors (Rhona Flin), design and ergonomics (Peter Buckle).   Plus the important contribution of researchers with an understanding of organisational culture and sense-making.  I particularly like the paper by Graham Currie and Justin Waring, whose observational study http://tinyurl.com/dy4weak of hospital incident reporting systems showed how doctors determined what counted as safety incidents – for instance, dismissing non-sterilisation of instruments as an issue.   In this way, we know that top-down safety initiatives which overlook issues of professional and instutitutional cultures and hierarchies (pace Mintzberg) are doomed to failure.

So there is a good foundation for patient safety research in this country, driven by social scientists http://tinyurl.com/d24xx38.  But Atul Gawande’s great insights http://gawande.com/complications into medical and surgical practice show so elegantly the dilemmas of doctors trained for a world that no longer exists.  Today’s clinicians need teamworking and communication skills (and checklists) to navigate complex healthcare systems – and an understanding of how those systems work.  This kind of insight comes from the inside out.  So where are the UK’s  Atul Gawandes who will shape the patient safety debates of the future?

Why should managers read research?

Why should managers read research?   There has been much talk about the `two cultures’ of management and research – the one focused on today’s pressing problems and pragmatic solutions, the other presenting 200 page reports hedged with qualification and abstraction some years after the first question was posed.  Initiatives have been set up to try to bridge the gap – providing `good enough’ research to inform decisions by managers, whether reconfiguring local maternity care or decommissioning inpatient addiction services.  There is no doubt that managers and researchers inhabit different worlds.  But management without research is anecdote.   And research without use is archive.

In making the case for research, we often stress its instrumental importance – providing hard evidence of the effectiveness and cost-effectiveness of care.  This is important – but what has struck me is the power of research to change the way you think as well as what you do.  When I recently asked a `research-friendly’ trust chief executive what research had most influenced him, he mentioned works on theories of leadership and organisational culture, as well as shifting care out of hospital.   The best research can illuminate, provoke and challenge – as well as inform.  

What follows are a personal selection of papers which I think (for different reasons) arelikely to become modern classics.   In separate blogs, I have extracted the key findings for each, with a personal commentary and sense of what they add to what is already known – plus a cheeky one-line summary for those who really don’t have time for more.  There are a range of methods – from a systematic review of trials to a five-year ethnographic study of one practice.  These include:

Mary Dixon-Woods’ theory-rich evaluation of a major patient safety intervention http://taralamont.blogspot.co.uk/2012/06/why-change-can-be-hardest-word-of-all.html

John Gabbay and Andree Le May’s deep study of a practice to explore how doctors use evidence http://taralamont.blogspot.co.uk/2012/06/what-doctors-really-do-and-why.html;

Sasha Shepperd’s systematic review of hospital at home schemes http://taralamont.blogspot.co.uk/2012/06/as-well-as-or-instead-of-beds-evidence.html

At the Health Service Research Network symposium in Manchester this week, we heard from the likes of Mike Cooke and Liz Mear why all trusts – not just the biomedical beacons – want to engage with research and the new opportunities opening up with AHSNs.  Every trust is now asked for metrics on recruiting patients to trials – and there may be real incentives at a time of pressure in building up research capacity to attract new funds.  But for managers there are also personal rewards in taking the time to read more widely.  Each of these pieces of research are important but also a pleasure to read.  Today’s managers are asked to flex their thinking muscles as never before, to think critically about what we do now and what could be done tomorrow.   This may be a good place to start.

As well as or instead of beds (the evidence on hospitals at home)…

Shepperd S, Doll H, Broad J, Gladman J, Iliffe S, Langhorne P, Richards S, Martin F, Harris R. Hospital at home early discharge. Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: CD000356. DOI: 10.1002/14651858.CD000356.pub3. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000313.pub3/full

Why read this?

Pressure on acute beds leads managers to look for alternatives.    Hospital at home schemes have developed rapidly in different parts of the country, providing care from clinical staff for a fixed period for people who would otherwise need to stay in hospital.   This particular study brings the robust Cochrane methods of systematic review to bear on the important service question – do hospital at home schemes for early discharge lead to better outcomes for patients?  And do they cost less than inpatient stays?   This study published in 2009 looked at 26 published randomised trials (mainly from the UK) comparing hospital at home schemes for early discharge with usual care in hospital for patients with stroke, those recovering from surgery and older people with a mix of conditions.   As well as the results from these individual trials, the researchers led by Sasha Shepperd carried out a meta-analysis, pooling the individual patient data from 13 of these trials to answer the common question – does it work?

This study is important because complex service interventions, like hospital at home schemes, have not always been subject to rigorous evaluation and scrutiny.   Many new ideas spread rapidly across the health service, many with inflated claims of cost savings and benefits – from virtual wards to rapid response teams.  This study is as good as it gets in terms of applying `gold-standard’ research methods to address important service questions.  But it also highlights the limitations of any such study – it is basically as good as the research that is out there.  The small number of robust studies to review and the heterogeneity of interventions, making it very difficult to compare like with like.   In some cases, these were hospital outreach services led by specialists, in other these were community-based services.  Not all provided 24 hour care and the input varied, from specialist or dedicated nurses to range of physiotherapy and occupational therapy staff.  Context is all-important in understanding complex services, but these were not always well described or indeed able to be adjusted for in this method of review.   But this is still a `go-to’ source document for managers reviewing options to relieve pressure on hospital beds.

What does the study say?

For older people and those with a stroke, there was no difference in outcome and quality of life for those in hospital at home schemes compared with usual care.  Patients seemed to prefer hospital at home initiatives and there was no reported increase in burden to carers.   However, the readmission rates to hospital were higher for older people and those with a stroke in hospital at home schemes.  At the same time, there was a lower chance of such people being in residential care at follow-up.  Comparing hospital at homes with inpatient activity, there was evidence of increased total length of stay for patients using hospital at home.    

Overall, there was no evidence that hospital at home schemes generated cost savings.  

The authors took care to discuss and interpret some of the complex messages in this piece of work.  There are no categorical answers – for instance, on the critical cost-effectiveness question, the picture looks much stronger for hospital at home schemes if restricted to patients with mild disability (or where the running costs of local hospitals is high).   There is an argument that numbers for hospital at home schemes are still so small (around 1-2% of hospital patients) that it would be difficult to realise cost savings at scale.   Where local schemes are in place for older people with COPD or stroke, the evidence is not strong enough to discontinue these, given the likely benefits.  But these should probably best be considered as an adjunct to hospital care.  From the evidence to date, the review suggests no compelling case for hospital at homes acting as a substitute for inpatient care.

Bite-size messages for managers

Hospital at home schemes unlikely to justify closing a hospital ward...yet.

What doctors really do (and why guidelines are only part of the answer)…

Gabbay J, le May A (2004).  Evidence based guidelines or collectively constructed `mindlines’?  Ethnographic study of knowledge management in primary care.  British Medical Journal; 329: 1013-1016.http://www.bmj.com/content/329/7473/1013

Why read this?

This is a powerful piece of research, based on more than five years close observation of a high-performing general practice.  This kind of ethnography is often used to great effect, from the jungles of Borneo to the street life of Chicago, to describe culture and practice in different groups.  The authors wanted to find out how clinicians actually use knowledge and where they look to for information and advice on everyday problems.   They immersed themselves in the daily life of the practice, from patient consultations to range of practice meetings with different staff, and checked out their emerging findings on how knowledge is formed with short visits to other practices and settings.

This study from two experienced researchers (both with clinical backgrounds – one medical, one nursing) shows the benefits of careful, embedded research over a period of years, reaching the parts that surveys and interviews alone cannot reach.   These are the realities of a busy Monday morning surgery when the partner is off sick and the locum hasn’t arrived, as opposed to scenario decision-making in the abstract.   As well as fascinating anecdote and case study, this piece of work benefits from wider knowledge of social science theory, from the new discipline of implementation science (pioneered in this country by Martin Eccles and others) to older work on professional knowledge and decision making. This short and powerful article (expanded in a readable book) provides a new way of thinking about evidence-based practice.   Rather than berate clinicians for not `following the book’, it provides illuminating insights into how people really make decisions and what influences practice.

  

What does this study say?

Looking at typical issues, from prescribing statins to screening for early kidney failure, the authors found that clinicians don’t make direct use of guidelines or formal research when they are practising.  But nor do they ignore them – instead, they follow `clinical mindlines’, which is made up of experience (both their own and colleagues in the practice) and a mix of formal and informal evidence.   This includes `tacit knowledge’ – patterns of thinking which may be deeply embedded and almost instinctive.   Several examples are given of these implicit clinical reasoning, such as the decision to admit a woman complaining of feeling `ice in her chest’.  But rather than opposing views of knowledge being codified (textbooks and guidelines) or tacit (intuition based on experience and decision rules), the authors argue that clinicians draw on both in a clinical mindline.  They highlight the importance of loose professional networks within and outside the practice to create and reinforce knowledge.  These informal links were important go give clinical knowledge the `social life’ which would ensure it was used – from coffee room chat with partners in the practice to wider diabetes networks.   Similarly, research showed the importance of stories and anecdotes in embedding knowledge in practice – including the power of real life `near-miss’ stories of patient harm.  In a successful practice, they also observed a collective clinical mindline, where they developed `ways of doing things’ by challenging and making sense of formal evidence, discussing anomalies from experience and refining and updating their knowledge of diseases and how to manage them.

We already knew from Trevor Sheldon and others http://tinyurl.com/8yjacnl more than ten years ago of the limited uptake of NICE guidelines, particularly where messages are complex …  Jeremy Grimshaw http://tinyurl.com/cv4eekv has shown in careful examination of evidence from controlled trials the limited effect of passive diffusion of guidelines.  

What does this mean for managers, who may not engage with clinical decision-making directly?   It throws light on the messy world of how knowledge is used – in fact, as pointed out in another classic research paper (Walshe and Rundall http://tinyurl.com/c63au2v), even more tricky for managers with a dispersed, loosely defined body of social science literature than clinicians where at least the formal knowledge is more organised, with defined hierarchy of evidence and mechanism for extracting recommendations (via NICE).  In making complex decisions like whether to decommission inpatient eating disorder services, managers are much more likely to be influenced by what other trusts have done, advice from experts near to hand, experience with similar specialist services locally and other forms of tacit knowledge than to consult a Cochrane review.   Managers, like doctors, are influenced by `worst last case’ – particularly if it risks ending up on the front page of the local newspaper.

Bite-size take-home message for managers (with apologies to the authors)

Who you know IS what you know.

Why change can be the hardest word of all...

Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ (2011).  Explaining Michigan: Developing an Ex Post Theory of a Quality Improvement Program.  Milbank Quarterly: 89(2); 167-205 http://www.kliniskbiokemi.net/pdfiler/DixonWoods2011.pdf

Why read this?

There is little good evidence on how or why service improvement initiatives work – or don’t work.   This paper highlights key findings from one of the most successful patient safety initiatives.  This was a campaign to reduce (and, in some cases, eliminate) harm in a hundred intensive care units in Michigan.  It resulted in remarkable reductions in central line infections and other avoidable harm.   The programme had a clear protocol and documented resources which others can use.   But as the authors note, programmes rarely keep to their original plans.  As other countries and regions try to replicate these results, greater understanding is needed of the real reasons for success in Michigan.

What distinguishes this review is the power of the analysis and elegance of the writing, combining rigorous theoretical knowledge with insights from the clinicians who led the initiative.  The paper combines the talents of two leading medical sociologists with an interest in patient safety, Mary Dixon-Woods from Leicester and Charles Bosk from the US (whose ethnographic account in the 1970s of why trainee surgeons make mistakes has become something of a classic and well worth a read), with the experience and insight from  the participating programme leaders, including Peter Pronovost.    This generates findings which have the texture of lived experience (and battle scars), as well as the benefits of a rich social science hinterland.  

The paper is also interesting for its methods.  Ideally, ambitious initiatives like Michigan should be accompanied by real-time evaluation, including observation of the programme in action.  However,  this was not possible.  Instead, the authors updated the programme’s initial theory of change with wider theoretical learning from social science and insights from key players – both those creating the programme and local participating intensive care units.   This provides a grounded reality to their findings – plus some useful methodology for others to adopt in retrospective evaluations (although general readers may want to skip the parts on ex post theorisation and Bayesian logic). 

What does this study say?

The key findings from this exercise combining theory, experience and more theory to understand how and why Michigan worked apply to many quality improvement initiatives.   Firstly, once enough organisations sign up to a programme of change, more will follow because of known patterns of peer influence (`isomorphic pressures’).   This may sound obvious, but has implications for new programmes – better to engage a large number of organisations at once, than have a few beacon or showcase sites at the start. 

Secondly, success at Michigan depended on strong horizontal links between units to generate the energy and momentum of a social movement.  These relationships were key in exerting peer pressure and supporting each other through change.  

An interesting third point (which probably could not have been articulated by the leaders themselves) was the need to construct blood-borne infections as a social problem that could be fixed.  At the time, infection was seen as inevitable and tricky to solve (`the problem of many hands’) – the programme re-framed this as an avoidable problem, using variation in infection rates mong participating units as evidence of the potential for improvement.   This was put into practice by the project leaders (or `vertical core’) who provided robust scientific expertise and credibility while also able to inspire and engage others – and respond to challenges from sceptics.  To win hearts and minds, the programme combined hard data and robust evidence-based practice with stories – the 18 month old toddler who died following a catheter infection.   This helped to disrupt existing beliefs about harm and potential for change. 

The fourth element identified by the authors was a few powerful interventions at the sharp end to change practice and culture, with much discussion of the checklist.   Popular accounts of the Michigan project reduce its success to the use of a five-point checklist of evidence-based practice (from hand hygiene to use of chlorhexidine before inserting a line).   This paper gives a careful account of the checklist’s social functions, making visible the discrepancies between actual and ideal practice and licensing challenge from nurse to doctor in a way which subverts usual practice.  The authors use theoretical knowledge to explain how a checklist institutionalises good practice by making it routine, or even ritual. 

A fifth key factor was the use of data collected systematically by the central team  (unlike many system improvements where it is generated locally in often haphazard ways) to measure and benchmark. 

Lastly, the programme provided hard as well as soft tactics to ensure engagement, and deter non-compliance, including sanctions by programme leaders.

(Aside – do the authors make enough of the effect of charismatic leaders?  From a few patient safety events, I can testify to the rockstar effect of clinical champion, Peter Pronovost – named as one of Time Magazine’s 100 most influential people in 2008 and holder of a `genius grant’ (McArthur fellowship).  Not many of them around..)

Bite-size take-home message for managers (with apologies to the authors)

Successful programmes need stories AND statistics.

 

Why stories matter

I was very struck by a good piece of service research from Addenbrook’s presented at today’s HSRN symposium (www.hsrlive.org).  Although the session was focused on productivity, the research from Dr Mai Wong really highlighted the power of patient narratives (and clinical storytelling) in understanding demand.  Her work focused on frequent attenders to emergency departments – often the focus of policy initiatives, with claims of excess service use, but poorly understood.  Dr Wong showed the heterogeneity of this patient population.  The clever thing was the way she translated the descriptive HES data and clinical casenote reviews into archetypes - living, breathing pen portraits.

We were told about Kelly, a young woman with COPD and anxiety who visited A&E ten times a year.  And about Bob, well known to emergency staff, homeless and prone to self-harm and substance misuse,  attending A&E every other week.  These personal profiles helped her to understand the different demands of the moderately frequent attender (where brief psycho-educational interventions might help) as opposed to the extreme frequent attender (Bob), requiring intensive case management.

Why does this matter?  John Seddon (www.thesystemsthingreview.co.uk) punctures myths about lean management (or bastardised versions of it).  One is that standardising service operations is exactly the wrong thing to do if we don’t understand the nature of demand.   Our healthcare systems often multiply demand and activity, as needs of the individual patient are not met upfront.  The work of Christiansen also highlights the need to frontload each care pathway with high quality assessment by teams of skilled clinicians – get it right first time.

So why do we need patient stories?   If we don’t understand how people use services and their underlying needs we won’t improve productivity.

HSRN symposium Manchester, 19 June 2012