Misusing numbers to sell project results

The more I know about statistics, the less confident I am about my own use of numbers and the more suspicious I am of any publicised data. The oracle on this topic is Dr Ben Goldacre, author of Bad Science, one of my favourite reads.  A new book out by Charles Seife called Proofiness; the dark art of mathematical deception is along similar lines.

When it comes to scaling up the results of improvement work across a wider population, the danger of inflating the opportunity is significant. These are some of the numerical problems that have come my way in the past few week:

1. Hospital a got a 30% improvement, therefore if we multiply this across the whole county, this 30% improvement will give us a £1.3 billion saving.  Humpf? So I need to know how representative of the national picture was Hospital A. I need to know the numbers and not the percentages behind this statement. I need to know how generalisable the improvement process is. And that is before we get to understanding how the savings are calculated.
2. Pathway b is delivering 212.3 more patients in a shorter time of 8 days.  Humpf? More than what and shorter than what? The 212.3 gives me the wobbles because I've never seen 0.3 of a patient. Seife discusses the art and deceptiveness of the specific number in his book - see a relevant excerpt.
3. 87% of hospitals have adopted the use product y. 87% of what population - national, region, the city? What counts as use - have they used it once or are they using it all the time. Did they try it out or have they made it part of their regular routine? Did just one ward test it once or have more (all?) wards tested it and are using it all the time?  These questions are important in understanding the true nature of what has, or has not, been spread throughout a system.

Are there any headlines of numbers that make you wonder?

Connection is key for innovation

A little luck and a lot of perspiration is no longer the prerequisite for innovation. Nowadays connections, relationships and networks are key.

Steven Johnson's next book is about Where Good Ideas Come From. You can watch him in action on this TED video (which starts with a picture and talk about the Grand Cafe in Oxford).



A couple of key points include:

1. Innovation is the result of the brain making new connections and these connections mirror workplace connectivity.  The better the networks and more complex the relationships then the more likely it will be that good ideas arise. His premise is that innovation is an interactive process.
2. The architecture of space is important if ideas are to be generated and spread - he has looked into what environments have contributed to innovation. If we want innovation we need to design spaces that enable connections.
3. Great ideas that appear to spark from nowhere are most likely the result of a long period of incubation which may not be obvious and are most likely cobbled together from a variety of existing ideas.. He argues that ideas are networks and the way of thinking about them as sparks, illumination etc is no longer relevant.

Patient Experience; report to help Boards build patient and public experience into quality fo care

If you are steeped in the healthcare quality improvement methods you will know how important it is to understand the patient experience. Scaling up project results to a wider audience does not mean the patient's views are no longer important. While the theory is obvious the practice is not. It can be difficult to engage non-executives and Board members with the patient experience and a new report on the Patient Experience  issued by Dr Foster in the UK covers the following:


Foreword 
Key messages 
The scope of this report 
What is patient experience? 
Good intelligence guide: The basics 
Good intelligence guide: Measuring what matters 
Good intelligence guide: Building an intelligent report 
Good intelligence guide: Advice from the sharp end
Board briefing: Intelligence types and uses 
Board briefing: 2010 policy directions 
Board briefing: Glossary 
Board briefing: The current state of play 
References 
Acknowledgements 

It provides advice on how Boards can produce a meaningful intelligence report on the patient experience. I particularly like the checklists and the brevity with which it is written.  They have nicely avoided the temptation to burst into theoretical concepts.

I found this report via Jocelyn Cornwall's blog at the Kings Fund - another good source of information.

Learning networks or social movements? Which is best for large scale change?

I'm not sure there is a specific answer as to whether learning networks or social movements are best for large scale change.  However, my personal preference is for learning networks and these are my reasons:

1. learning networks connect the people who are interested in solving the problem, and they do so in an interactive, participative and empowered way. In comparison social movements have the image of someone wanting to make something happen and in manipulating the joining in process and to an agenda they have set.
2. learning networks are about the exchange of information, discussion and both personal and group learning. Personal behaviour may change as a result of learning about something new from within a peer group. When a group discovers new knowledge and feel the creative process of doing so, they may become quite committed to then implementing changes.
3. learning networks can be direct and specific. They can be topic based or professional grouping based.
4. there is a sense of an output with learning networks. Social movements are useful in that they are purpose driven, however, learning networks to me are more concrete, pragmatic and easier to associate with outcomes and results.
5. we know learning networks work in healthcare while social movement theory is in its infancy. I am a proponent of using the existing knowledge we have to best effect.

This doesn't mean social movements are not useful, rather that I am sceptical about their ability to deliver results, especially when we know learning network can have an impact.

There is an excellent paper on Learning networks for sustainable, large scale change which is worth a read. 

Options for large scale spread; excellent new report by WHO, IHI, Harvard

A new report about large scale change and spread of simple, high impact interventions is available now. And it is a really good summary of methods old and new. I highly recommend it for anyone planning simple large scale spread, especially if you think you know the answers. This report summarises a variety of methods and is very pragmatic in it's use of examples.

The World Health Organization Patient Safety Programme and the Harvard School of Public Health commissioned the United States Agency for International Development’s Health Care Improvement Project (HCI), managed by University Research Co., LLC (URC), to present its understanding of and experience with the effective adoption of simple, high-impact interventions, such as the Surgical Safety Checklist. URC is joined in this effort by the Institute for Healthcare Improvement, which also has decades of experience in this field.

Readers should note this report and advice covers the spread of simple, high impact changes - namely it does not cover complex, system related changes.

The report covers:

III. Spreading Evidence-based Interventions
IV. The Scientific Basis for Spread
A. Framework for Spread
B. Individual Adoption and Behavior Change
C. Positive Deviance
D. Factors that Influence the Rate of Spread
E. Understanding the Social System
F. Integrating Content into Process Design
G. Testing and Implementing Change
H. Executing for System-level Results
V. Approaches for Large-scale Spread
A. Natural Diffusion Approach
B. Executive Mandates
C. Extension Agents Approach
D. Emergency Mobilization Approach
E. Affinity Group Approach
F. Collaborative Approach
G. Virtual Collaborative
H. Wave Sequence Approach
I. Campaign Approach
J. Hybrid Approaches
K. Lessons Learned from Large-scale Spread
VI. Which Approach Should Be Used to Disseminate Checklists?

(And thank you to the authors of the report for referencing my book on how good practice doesn't spread)

5 checks to test whether your conference call is productive

Conference calling is often touted as the more productive way to have a meeting. In many cases it is, however, some calls are so badly organised, chaired and processed, that it may have been better to meet face to face - or not meet at all.

1. Don't start on time.
"We'll wait another couple of minutes for Dean and Tam."
silence
"Is Mike on the line?"
silence
"Janet can you call Mike on his mobile to check he is dialling in"
Silence

If 20 people wait 3 minutes past the start time for the people who have not joined the meeting it means 1 hour has been wasted. If 100 people wait 3 minutes it is the equivalent of 5 hours wasted by the organisation.

A good way round this problem is to predictably open the line 5 minutes before the session starts and then predictably start on time - every time. You wouldn't keep the meeting room door locked until the advertised start of the meeting.




2. Use slides
"For those of you with Internet connections, you'll see the agenda. What? Oh Mary, yes, I'm sorry, the agenda is so up to date it couldn't be circulated beforehand."
...
"Can everyone see the slides?"
"Next slide please. No, go back one. Yes, that's it."
...

Slides are used mostly as personal notes for the person speaking. As some people don't have access to the computers when they dial in, any necessary slides should be circulated before the call (rather than afterwards). Some learning styles work better when they have had time to read and then contribute.

3. Rely only on audio
Meetings of any sort work best when there is trust. Unless the session is an online seminar when your role is to push out information via some slides to a large group, then video is essential. Even in the seminar option, the speaker needs to use video.  The more participants who use video, the more everyone will pay attention and not do their email at the same time.  Multitasking is proven not to be productive.  A 90 minute conference call with everyone doing two tasks at once is unproductive. It could be replaced by a 30 minutes, focused, video-face-to-face committed time discussing what matters.

When presenters refuse to use the video option they lack the ability to influence using body language. The same goes for listeners. This makes the call ineffective and unproductive as messages have to be repeated.

4. Don't use names
'Thanks for that..um.. sorry, who was that who agreed to review the budget?"
...

Audio, especially without visual, is anonymous - which may account for why some groups enjoy using it - it feels like they are having meetings yet no much happens to move the work forward. Anonymity is not a good name as it means listeners have to work hard to keep connected.

Each time you speak, say your name: '"Sarah here, I like what you're proposing Simon, however, I am not sure about item 3."  Try and say the other person's name as often as possible to let them know you are connecting with what they are saying.

5. Have a call when an email would have done
Holding a meeting is an act of power and control, and one to be used with care.  If the intent of the call is to share information then consider first whether an email or discussion group would be a better and more productive option.  If group discussion is required - then a call is good. If information needs to be shared and not much discussion is expected or required, then another method is far more productive. Many calls can be replaced by the use of efficient and productive discussion groups and document storage systems. Huddle is a great one, though Google groups/documents is a good start.  The real issue here is about power and control; relinquishing the weekly conference call and replacing it with an online discussion group means someone has lose control of the process.