Business learning from walking: 2. Knowing where you are is more important than knowing where you are going

Picture from simonmainwaring.com

It's a truth that you have to start where you are. Whether it's starting a walk along a trail or a new change initiative to streamline the repeat prescribing process, you begin where you are - not where you think you are.  Anyone who has run a process mapping event will know that participants love to create maps and plans about what they want to happen, where they want to be. Focusing on the reality of what actually happens seems far more difficult. It's fairly boring too. To start your walk you need to be able to place your finger on the map and say "We are here". To start a change program you need to be able to use all the data you have to state your current position.

Would you start walking a linear trail, like the Thames Path or Hadrian's Wall, without knowing where you are starting from and whether you are at the place you expected to be?  I may decide to start walking the Thames path but if my actual start point is 8 miles from the predicted start point, then it's goign to be a very long day and probably one with many disappointments.

After you've started moving, you need to keep track of where you are. In business we do this by measuring our progress. These measurements need to be close in time to the actions and decisions. If I walked a route saying I would check the map at every hour on the hour (monthly reporting?) then I could quite easily waste time and energy by going in the wrong direction.  Every decision point needs a check between plan and actual progress.

I can talk a lot about where I want to end up with my walks, as Many talk a lot about what the results fo their change program will be. But in the end, those results and goals are dependent on a system and practice of knowing where you are, at any point along the way.

Innovative changes to care pathways can increase hospital admissions

Innovative changes to care pathways can increase hospital admissions - really? Well, a report from the Nuffield Trust in March 2011 suggests there is little or no evidence that community interventions lead to a reduction in hospital use.

The report is a good one with a firm research founding - in the absence of any randomised control data. It points out that redesigning pathways can discover unmet need which may account for an increase in hospital attendance.

What caught my attention was that using their own data, each of the eight interventions assessed demonstrated a reduction in hospital use. However, when compared to control groups, there was in fact an increase. This leads me to one of the ongoing issues I have with "innovation" or "improvement" projects. It's easy to come up with a measurement system and set of goals and sample size that has inbuilt biases to ensure good results - and win prizes. But in the end, improvement needs to be tested against control groups.

I recommend you read the full research report, if only to grasp the seriousness of this issue.

Free SPC Templates for Excel

There are a  number of free templates available for your statistical process control data and charts.  It goes without saying that you should check that the template meets your needs and is the right type of SPC chart for your data.  I've not checked the accuracy of the following so please use at your own risk.

1. From Vertex42, a straightforward, simple spreadsheet with no complicated macros or other things to stymie your system
2. Statistical Solutions provide the entire range of SPC charts for free, though you do need to register. X Bar/S, I/MR, PChart, NPChart, Xbar-R Generator, XBar-S Generator, IChart Generator, APQP/PPAP.
3. There' are pages and pages of spreadsheets available here - of which SPC is one. I liked the Six Sigma Template Kit.

Run Charts: How to create and interpret

Managing data in an improvement project often feels like it is some magical craft, with spells, secrets and strange ingredients. But this isn't true. A run chart is the most basic and the most important analysis tool in your improvement armoury.

BMJ Quality & Safety have a very useful article with instructions on how to create a run chart and how to interpret it: click here to go to the article

Measuring large scale change

Many complex programs are designed to deliver large scale change. A key concept is knowing when a change is being made, and whether it is in the right direction.  Large system change is different to process change in that it seldom has a clear beginning and end, has multiple causal factors (some of which we will never know), and the result is often separated from the action in time and space.

Differentiating the types of measures from each other helps, as well as estimating and checking connecting between them.  The following categories may be useful to you in deciding how to measure what.

Inputs: a bit like a baseline measure though perhaps a bit more active. It could be the number of patients not attending their appointments or the % of staff committed to a new organisational vision.

Activities: this counts how much is done of something designed to engender change. IN the case of large scale change there may be a variety of activities underway at the same time. This could be the % increase in number of people attending a workshop, the number of patients

Outputs: the results of the specific activities. So if the number of employees attending a patient experience workshop increased, and the workshop had an aim of improving staff satisfaction, then an output would be the amount of increase in staff satisfaction (and perhaps compared to areas where employees had not attended the workshop.

The above three measures often look alike. What is key is to understand what large scale change is being measured and to think through, and perhaps map, the links between the identified measures.

Outcomes: this differs from outputs in that it moves up a higher level - more long term, bigger impact. For example, if patient satisfaction increases then an outcome may be more patients returning, more income etc.

Impacts: this is the final level of measure. Perhaps the organisation reaches a new public grading, patients in the local area experience better health as a result of the improved services etc. Reduced health inequalities is another example.

Identifying the measures is only half of the learning from measuring for large scale change.A key step is to find a way to map out the linkages between the measures. To do this at the start of a program is helpful as learning from the actual measures can be replotted. This will help identify whether movement to the large scale change is underway as a result of the current activities - or not.

Choosing the right chart to show your project progress and performance

Essential to any improvement project and spread / adoption campaign is a system to measure progress. An Excel produced bar or line chart on the regulation maroon and blue is suggestive of either a lazy monitoring process or one which is not turning data into useful information.

In large programs fundholders often like to see comparisons between one sub-project or site and another. Or maybe you are interested in how the results are stacking up as a whole; for example, are all the sites adopting all the elements of the new project? Do you know the best way to show a trend of your data? What about relationships between different data?

A favourite place to download Excel templates for your charting needs can be found at the Chart Chooser. Best of all, you can work out which template best suits your needs using their diagnostic process.

Do you have any templates for charts used to monitor the spread / adoption process or project improvement that you would like to share? Leave your comments below.

Fear of failure? Never Events need a target of zero

So how would you feel if you were the patient of a surgical team whose improvement target for the "Never" event of wrong site surgery was - to reduce by 20%? A "Never" event is something that, by definition, should never happen. The NHS in England have a list of 8 that are so well evidenced they should be adopted for implementation with no further discussion.

Why then are we seeing targets for Never Events that are not zero? My recent experiences suggest the following dynamics at play:

• zero is too difficult a target to achieve in healthcare (try telling that to the patient who has just had their wrong kidney removed or the suicide that happened when on 1-1 watch and using non-collapsible rails). Admittedly in some healthcare processes zero may be a tough tartget, however, Never Events mean never.
• we may not meet the target (fear the failure and the knock on consequences for individuals, teams and project work); when I encounter this I realise I am working with an individual or group who fear the failure of improvement greater than they feal the failure of harming a patient.
• it's not worth all the changes for zero (the costs of the change outweight the cost of full redeuction of a never event); this is, of course, a judgement call and my hope is it is made with full data analysis and consultation of those involved. Some never events happen so seldom it may be difficult to justify the changes required.
• we don't believe the research is good enough; the NHS in England (National Patient Safety Agency) have streamlines a varietyof Never Event lists to determine a core 8 which are well proven in all aspects. More research I suggest is not required.
• we will take a while to get to zero; that's ok, then let's see a desired outcome of zero and some leeway to reduce over time. The trick is perhaps not to design for a 20% reduction in year 1 but rather to design for zero in year 3 and monitor progress over time.

Never events need targets of Zero - 0. They should never happen.

Why use percent and average for improvement projects?

It seems that every healthcare quality improvement presentation I hear nowadays uses averages and percents as the mechanism for demonstrating an improvement was made. I have some problems with this:

1. There seems to be a confusion between performance management and measurement for improvement. Yes, the organisation may need to report on the % of x - done mostly so someone in charge can make a comparison. However, this is no reason to replicate this measurement in the project. For instance, if you're working on length of stay (LOS) then you may find mode (most frequently occurring number) is more helpful in demonstrating a change consequent to your improvement activities and it has the benefit of indicating the experience for most patients.
2. When a nurse pointed out to be that they do not have .32 of a bed then I took notice. Of course she was right. So some stats that showed an average of 12.32 beds were to be shifted each month (a specific project) I could see this was nonsense. Statistically it could be argued this made sense, though as a mechanism for engaging staff, working in whole numbers, whole beds, whole patients, tends to make more sense.
3. So ward A delivers a 4.18 average length of stay. Management now want all wards to achieve this (let's assume most are higher than this). So the processes and procedures underway in A is replicated to others (or attempted). The difficulty is their case mix may be different, their problem may be one where only their long stayer need to be addressed (they have the same mode but ave stats skewed the total figures) etc. In aiming for an average do they figure out the mathematics of LOS - x number can be 4 days, y number can be 5 days etc? For this is what the average leads them to - to game.
4. An average, is, by definition, an abstract concept that assumes half the values will be above the line and half below the line. Do you really want to have half your experiences, interventions etc be more than the agreed number?
5. One more measurement challenge. What happened to the 100% (or 0%) target? The common version is 98% of y or 95% of z. My logic suggests we are designing systems for a percentage failure. This is a tough call in healthcare. Which 2% of patients will you choose not to have optimal care for their diabetes?

Building awareness is key

Most people keep their focus on the amount and extent of implementation; how many people have adopted the new practice or what benefits have been gained. This is important. However, I maintain that the amount of ultimate benefit you end up with, the number of people who end up adopting a change, comes from the pool of the people who decided to make a change. And not everyone goes through with the chnage process. Then the number of people who decide to make a change, comes from the pool of people who are aware - of two things. Firstly there is a problem they need to resolve, or maybe they just need to know there is a solution worth implementing - or maybe a bit or both. Regardless, they need to have awareness.

So when you next design your programme to rollout the benefits of a set of changes or want to get a large group of people to adopt some changes, my recommendation is to concentrate on the awareness phase. It helps to measure this as you then begin to have an idea of the slope of your adoption curve.

I could go on about this in some detail, but all I really wanted was to make you aware of the issue...

3 tips for measuring the spread of good practice

1. The most basic spread / adoption chart is one which measures the number of people adopting a good practice over time; that is, people on the vertical axis, time on the horizontal axis. If you're not prepared to do this check then do comment in this blog and let me know how else you keep track of progress.

2. Calculate your total population who you would like to adopt the good practice. Work out how long this might take. Then pick yourself up off the floor.... Now work out that from this total amount, what might be a realistic target for the timescale and resources that you have for the project.

Why does this matter? Look at the picture above. Without having thought about the total intent of the spread programme, it is quite possible for a shorter period to show results like those under the red lines. When the project then ends, and measurement continues, then the results look like they take a nosedive. This is when measurement usually stops.

One of the reasons for working out the total intent before starting the spread programme, is to provide for some creative thinking. You may just go about the process in a different way if you knew that eventually there were going to be 25,000 staff using the new method, across 14 locations and it was likely to take seven years.

3. Einstein was on to something with his work on time and space. We've no need to get that deep into the mathematics when working on spreading good practice. However, the issue of space is an important one; or maybe you think of it easier as geography. I've seen some teams use excellent maps and nowadays there are interactive electronic maps on which data can be tagged. Think and get creative. Learn about how and why different practices spread in different ways, not just because of the peopel involved but also because of the space and geography.

Three basics to remember for spread measurement.

People.

Time.

Space.

(c) 2008, Sarah Fraser