Featured post

Horizon scanning and the big picture challenges affecting health and social care

The health, social care and public health system is facing a number of overarching and interlinked challenges.  These include:

  • a possible increase in demand resulting from the ageing population
  • uncertainty over future funding arrangements
  • a need to ensure the system provides high-quality care.

At the CfWI, we have identified 11 issues of this nature and have called them big picture challenges: http://www.cfwi.org.uk/publications/big-picture-challenges-for-health-and-social-care-implications-for-workforce-planning.

Our horizon scanning programme is analysing these big picture challenges to draw out their workforce implications. We published our first report at the beginning of February, and we used Health Education England’s education outcomes framework (EOF) to put forward thought provoking questions about how Health Education England and other organisations involved in workforce training and education could help to address these challenges.

We are currently developing this work to look at the wider workforce implications of the big picture challenges. The Francis Report put forward the need for a ‘fundamental change’ in how the NHS operates while current government policy is to promote greater working in the community.

As the report made clear, the workforce will be crucial to addressing these issues, as will education and training. We have identified a number of workforce implications that need to be considered, including how bands one to four can be utilised to improve workforce productivity and meet demand, and how we can recruit and retain sufficient domiciliary care workers to meet future demand.

Some of the issues include:

  • How can we ensure that professionals are adequately trained to work in the community?
  • What skills will the workforce need to meet conditions such as dementia?
  • How can we ensure the workforce possesses the values and behaviours required to deliver high quality patient-centred care?

Share with us your thoughts, comments and concerns of what the future workforce needs to look like.

Expert elicitation at the CfWI: lessons learnt and improvements to the current process

Since last year, the CfWI has used the SHeffield ELicitation Framework (SHELF) as its key method of elicitation – seeking to obtain knowledge from experts where no formal data exists.  The process was developed by Professors Tony O’Hagan and Professor Jeremy Oakley of The University of Sheffield.

Thus far, we have used the full SHELF method at three workshops for the Horizon 2035 programme. However, time constraints on this process has meant the number of variables that required elicitation and the complexity of workforce groups have not always been possible to elicit.  We have therefore been working with Professor O’Hagan to explore alternatives to the current process, which is discussed in detail in Technical Paper 14: Updated approaches to elicitation.

Exploring elicitation with Professor O’Hagan

SHELF accommodates several different protocols for elicitation, known as ‘quartiles’, ‘roulette’ or ‘tertiles’. We initially chose the quartiles variant. A quartile is a way of dividing a set of observations into four defined intervals, but we found when we used this method, it took a long time to ensure that participants understood what they had to do.

We explored with Professor O’Hagan whether there are alternatives to quartiles that experts will be more comfortable with. We agreed to adopt the tertiles protocol, which involves dividing the distribution into three equal parts, and also found that tertiles led to more accurate elicitations, as they do not suffer as much from overconfidence and anchoring.

In addition, we looked at the possibility of developing training material on elicitation, so that participants fully understand the process and statistical terms. This would also save time at workshops, ensuring that we can spend the maximum amount of time working with our experts in their respective fields of knowledge. We will be exploring a number of options around this on future projects, including written and web training.

Lessons learnt

In the new elicitation technical paper, it was important to review our tools for elicitation in order to make it a more useful, practical tool for our stakeholders. We use sophisticated statistical terms to capture the knowledge of experts in diverse (often non-statistical) fields, and it is vital to maintain an approach that is rigorous, but that works for the people who take part in our workshops.

Our discussions with Professor O’Hagan have helped to enhance the elicitation process, so that it is easier and useful for participants. More importantly, considering we have usually one day to capture the valuable knowledge of experts, we don’t want to spend half of that time explaining statistical terms, so the suggested areas for improvement in this technical paper streamlines that process.

I am looking forward to conducting elicitations using the revised methodology in the future.  If you want to find out more about this work please e-mail the team at enquiries@cfwi.org.uk.

System dynamics: supporting better policy decisions through deeper understanding

This blog post was originally posted on the Horizon Scanning Hub.

‘Complex systems defy intuitive solutions’ –  Jay W. Forrester, Professor Emeritus of Management in System Dynamics, Massachusetts Institute of Technology

System dynamics as an analytical approach is well suited to issues in health and care, where policy actions need to take into account complex interdependencies, and delays between action and consequence.  I recently attended the 33rd ISCDC in Boston where I presented a poster on the Horizon 2035 programme, our system-wide analysis of the health and care system in England. The conference included over 150 presentations describing how the system dynamics approach is being used to make more robust decisions within complex policy areas.

System dynamics applied to health and care

Health and care topics were big talking points in Boston. For example, there were a significant number of presentations discussing public health initiatives, such as screening for cancers, vaccinations for infectious diseases and new drug development.  This included a number of authors who discussed modelling Ebola, an area that CfWI has also been supporting with its work for the World Health Organisation.

Best practice

Interestingly, several of the presentations reviewed best practice in system dynamics modelling. For example, Jeremy Sato and Bob Eberlein both discussed the importance of using an appropriate architecture for demographic modelling to avoid ‘cohort blending’ leading to misleading results.  It was reassuring to note that the demographic modelling in the Horizon 2035 model is consistent with their recommendations.

Agent based modelling

Another interesting area of discussion was broadening the application of system dynamics so that it is integrated with other analytical approaches.  For example, integrating system dynamics with agent based modelling (ABM) which enables a greater level of detail to be applied when considering an individual’s behaviour.  Kimberly Thompson discussed the use of ABM to represent outbreaks of polio and measles in the Amish community.

ABM could be worth exploring further at the CfWI, especially when considering individual actions and choice, such as decisions around training, assessing the impact of public health initiatives or outbreaks of infectious disease.  However we would need to bear in mind that ABM models have a much higher data requirement than more aggregate modelling such as system dynamics, and it can be more difficult to validate model behaviour.

Horizon 2035

Whilst attending the conference I had numerous thoughts about how we could extend the analytical scope of our projection models for Horizon 2035.  One particular area was in the modelling of specific public health interventions by the health and care workforces.  Public health interventions exhibit classic “worse before better” dynamics, whereby an initial investment must be made and there can then be a significant delay before the benefits are realised.  For example, workforce effort applied to improving the diet of the population may not result in reductions in associated prevalence of diseases such as diabetes and coronary heart disease for a long time.  Therefore when assessing the cost benefit of the intervention, it is necessary to take the long view, something that system dynamics modelling is ideally suited to do.

For example, the diagram below illustrates the impact that public health skills would have on population behaviour using a stock and flow diagram and the Horizon 2035 taxonomy. The taxonomy is a rigorous conceptualisation of the different skills used in the health and care system which has been validated with system stakeholders.  The impact of public health interventions are represented in the diagram using the prevent skill.  The population is segmented into those exhibiting low, medium and high risk health lifestyles.  Prevent, which in the case of lifestyle advice could be implemented through marketing campaigns or interventions by healthcare practitioners, would act to reduce the net flow of people moving from low risk health behaviours through to high risk behaviours.

sion blog

The conference presented me with many ideas that could be integrated into the workforce modelling carried out at the CfWI, and I look forward to exploring these over the coming months.

You can find more information about the conference at http://conference.systemdynamics.org/.  Many of the abstracts and papers from the conference are available from http://conference.systemdynamics.org/current/upload/schedbythread.html.

Making an impact: Horizon 2035 at the International Conference of the System Dynamics Society

This blog post was originally posted on the Horizon Scanning Hub.

Dr Siôn Cave attended the International Conference of the System Dynamics Society in Massachusetts in July. He shares his experience of the conference here.

I have recently returned from the 33rd International Conference of the System Dynamics Society, in Boston, the spiritual home of system dynamics, where I presented a poster on Horizon 2035. The project has been commissioned by England’s Department of Health and considers different workforce futures for health, social care and public health 20 years from now.  Its aim is to inform decision-makers in order to help ensure the health and care workforce in 2035 can meet the demand challenges facing them.

In Horizon 2035 we use a whole-system skills-based approach to model diverse workforce groups and to understand the relative scale of competing demand pressures.  The project applies the CfWI’s robust workforce planning framework, which is based on systems thinking principles and system dynamics simulation.

Presenting the poster at the conference provided an opportunity to explore our use of system dynamics in Horizon 2035 with other expert practitioners.  The poster illustrated how we have integrated a series of systems thinking techniques in the project.

Feedback from our peers was very complimentary.  Everyone I spoke to was impressed with the scale of the work, and that it is being used to meet a real need to adopt a whole system approach to planning for the future of the health and care system in England.  A number of people were also interested in the structures we had used, and how we had been able to integrate analysis of uncertainty into our projections using the Sheffield Elicitation Framework (SHELF) method.

The feedback I received confirmed that Horizon 2035 is an important piece of research, and we are making the most that the system dynamics approach has to offer. On reflection, the conference presented me with many ideas that could be integrated into the workforce modelling carried out at the CfWI.  We anticipate publishing the initial results of this work later this year at www.horizonscanning.org.uk

There is more information about the conference at http://conference.systemdynamics.org/.  Many of the abstracts and papers from the conference are available from http://conference.systemdynamics.org/current/upload/schedbythread.html.

Understanding the wider public health workforce

This post has been taken from the CfWI and Royal Society for Public Health’s Understanding the wider public health workforce report.


We are pleased to introduce this first collaboration between the Centre for Workforce Intelligence (CfWI) and the Royal Society for Public Health (RSPH), bringing together the CfWI’s skills in quantitative workforce analysis with the RSPH’s experience of workforce training and access to employer and community stakeholders.

This report comes at a time when the reorganisation of the public health workforce in England, the presence of public health within local government, and ongoing demands on the health system, offer a unique opportunity to encourage the ‘non-traditional’ public health workforce to have a greater role in improving the nation’s health and wellbeing.

This is wholly consistent with the Five Year Forward View published by NHS England and partner organisations in 2014 which sets out a challenging vision for the future of the NHS and the public health system, including a call for a ‘radical upgrade in prevention and public health’.

Innovative approaches to prevention, and designing new models of care are an important step, but an upgrade in public health is unlikely to become a reality without engaging the wider public health workforce with the right skills and competences, values, and behaviours to deliver innovation in practice across health and wellbeing in England.

The wider workforce can be found across the public, private and third sectors, working in a broad range of organisations and professions. This workforce is interwoven into the fabric of local communities, from unpaid volunteers, social care providers, police and fire officers, housing and education staff, to a wide breadth of healthcare staff, such as allied health professionals, midwives and pharmacists.

This diverse range of personal contacts offers many opportunities to promote healthy messages and initiate or embed behaviour change through having a ‘healthy conversation’ with a customer, client or patient, or signposting someone to a relevant service. The wider workforce is undoubtedly an instrumental part of the new public health landscape – we hope this report demonstrates the scale of the opportunity and the benefits already being delivered.

Therefore, we believe this report is timely and look forward to working with agencies across the public health system and beyond to help the wider workforce have as much impact as possible.

Read the full report, and accompanying infographic on the CfWI website.

Future health IT skills and competences in Europe and beyond…

This blog post was originally posted on the Horizon Scanning Hub.

John Fellows, Senior Consultant in Horizon Scanning, considers the potential impacts of technology on future healthcare workforces, looking at the scale of the workforce required, as well as changes to function and location. But how do these factors relate to other potential changes…?

I attended the eHealth week in Riga, Latvia in May, where I was invited to speak at a session on Healthcare Workforce Development and IT Skills as part of the EU-US eHealth/Health IT Cooperation Assembly. My presentation was on Horizon scanning for future health IT skills and competences in Europe.

The other presenters in this session (Rachel Kaye and Rachelle Blake: A digital solution to the EU/US skilled eHealth/Health IT Worker shortage; and Mary Cleary: Building on the EU-US MoU Roadmap) were all participants in the Workforce Development Group of the EU-US Memorandum of Understanding (MoU).

The MoU is a 2010 agreement between the European Commission and the US Department of Health and Human Services to support cooperation in the area of health-related information and communication technologies. Kaye, Blake and Cleary described the production of the HITCOMP (Health IT Competences) tool – a database of Health IT competences which can be used to create, for example, job descriptions or education programmes for health workers.

It’s very interesting to consider the potential impacts that technological and international developments may have on future health workforces; and how these changes can be taken into account in workforce models.

Flexibility and technology effects

A useful framework for considering these changes is provided by Atkinson’s model of labour flexibility. I most recently came across this in a thought-provoking article from Susan Nancarrow on Six principles to enhance health workforce flexibility and will use three parts of Atkinson’s classification (the external numerical, functional and locational flexibility aspects) to outline some of the interesting challenges in understanding technology effects on future health workforces.

Firstly, information and communication technologies may alter the locational flexibility of health workers in the future, depending on the types and volume of tasks that are amenable to being carried out remotely. Functional flexibility (or which roles are aligned to which competences) may also be affected by technological developments, through the use of clinical decision support systems for example.

Both of these types of labour flexibility then have potential impacts on the external numerical, or changes to intake, of health workers. If remote technologies are shown to be effective, safe and that people wish to use them, then theoretically there may be some reasonably-sized productivity increases.  If clinical decision support systems are effective, then the number of years of workforce training required for certain tasks may decrease in certain areas.

However, drivers of change very rarely all point in the same direction at once and the most obvious driver pointing in an opposite direction concerns the complexity of multimorbidity and the extent to which the types of demand coming from future populations will be amenable to the types of technologically-enabled flexibility outlined above.

Horizon scanning at the CfWI

In the Horizon 2035 project we have considered these types of productivity effects and the drivers pushing and pulling in different directions by level of health and care workforce in England, and conducted an elicitation workshop with experts on this topic so that we could quantify these effects. We’re currently working on the second of three publications planned for the Horizon 2035 project, and plan to share the initial results from these projections in the coming months.

We’ll also be researching and writing on the skills implications of developments in health IT and eHealth as part of the European Union Joint Action on Health Workforce Planning and Forecasting as it’s crucial to further develop our collective understanding of the potential Member State and European-level effects of health IT and eHealth on future health workforces.

All the presentations from the eHealth week 2015 can be found through the summit website and are well worth exploring.

Brussels 2015: Demographics, innovation and healthy ageing

This blog post was originally posted on the Horizon Scanning Hub.

Earlier this month I presented at the European Summit on Innovation for Active and Healthy Ageing. The central question was: How can technology, innovation and new ways of cooperation help Europe to turn its ageing population into an opportunity?

The summit also looked at integration of health and care services across Europe; the role of education; emerging technologies; and how to accelerate innovations.

Future skills

I participated as a panellist on the ‘health workforce’ stream, moderated by Michel Van Hoegaerden from the EU Joint Action (JA) with colleagues Antoinette de Bont from Erasmus University, Rotterdam and Silvia Gomez Recio from the European Federation of Nurses.

Undoubtedly our health and care systems, the workforce, our expectations and how care is accessed and be effective will be very different in 20 years’ time. My presentation shared our EU JA research on the future skills and megatrends we are examining, using horizon scanning to produce a report on future skills and competencies. Examples include demographic changes, skills supply, the future patient, and personalisation.

Colleagues’ presentations outlined the different perspectives of future skills needed, a widening continuum of care that must be thought about and the evidence base our respective programmes are seeking to establish. We discussed and received questions which centred on the future demographic shifts we see ahead, key shifts and workforce flexibilities and developments that will be required. As in our Horizon 2035 research and scenarios in the UK, we also considered wellbeing skills and our approach to applying a much broader definition of ‘workforce’ that includes unpaid carers and volunteers.

Engaging and empowering patients

I was also very pleased to attend a session on engaging and empowering patients chaired by David Somekh from the European Health Futures Forum, a partner organisation of our EU research. The session included powerful testimony from patients such as Eric Slachmuylders. He highlighted how better co-ordination and ways of working, enabled by technology and data between health and care professionals, could realise very significant improvements. You can see the presentations here.

What futures do we face?

The future is not certain and there are risks ahead that threaten health system and workforce resilience if we only think of a single’ business as usual’ future. There is great potential across the EU to better understand the different multiple futures we face, simulate them and consider how we can collectively respond to better understand our future population health and care needs – something close to our hearts at the CfWI.

The good news is that the foundations are in place, but we must continue at pace to further our thinking of what might be required longer term as well as short term. This conference highlighted much of the potential that exists and it was clear that there is a strong future for health and care to be at the centre of creating employment opportunities and economic growth as well as leading on health system innovations. People and the workforce will be at the centre of all our futures, indeed they will be the difference between success and failure.

All the presentations can be found through the summit website and are well worth exploring.

You can learn more about our EU work and our recently published Horizon 2035 scenarios at www.horizonscanning.org.uk.

You can follow Matt (@meds888) and the Horizon Scanning team (@horizonscanning) on twitter.

Delivering knowledge and intelligence in public health

The CfWI has published a review of the public health knowledge and intelligence workforce identifying an estimate of the size of this crucial workforce and the skills and competences they offer to society. CfWI Senior Researcher Tom Speller discusses the report’s findings in this post. 

The shape of the public health workforce was changed in April 2013 when staff transferred from the NHS to Public Health England (PHE) and local authorities. An important part of this workforce are those who deliver knowledge and intelligence (K&I). This discipline is defined by the Faculty of Public Health (FPH) as involving the “surveillance, monitoring and assessment of health and the determinants of health, plus the development of the public health evidence base and knowledge”.

The CfWI first noted the importance of K&I staff in its Mapping the Core Public Health Workforce report, which was published in October 2013. We were commissioned by PHE to do a review of those who work in K&I roles off the back of our core workforce study, and this was published today (date).

Through this research, we are seeking to shine a light on this group, so that anybody can understand how people come to work in this profession, what factors are currently informing their work, and what areas may require interventions to facilitate further sustainable development of this workforce over the next five to ten years.

Over the last year we have engaged with a number of stakeholders to influence the contents of the report. Through a series of workshops across England and extensive research, it is clear to us that there is a lot of value held in the work done by this workforce but a lot of uncertainty remains around the career development on offer in pursuing a career as an employee in this field.

We know that the system is still taking shape following the changes to the public health system two years ago, and given this uncertainty, people working in public health knowledge and intelligence have adapted positively to their new environment. People with whom we engaged during the project often underlined the exciting nature of their work, being valued by their employers, and their ability to work in new areas such as housing and environmental health.

These are encouraging findings, and demonstrate that there is a lot of potential for public health knowledge and intelligence teams to help improve the public’s health. With the increasing profile of public health following the late 2014 publication of the NHS’s Five Year Forward View, if marshalled effectively public health knowledge and intelligence teams could be at the forefront in helping save pounds… both on people’s waistlines, and on financing the NHS.

Through our report, the CfWI has identified some areas for possible development to ease feelings of uncertainty, or concerns around workforce mobility and we are working closely with PHE, Health Education England and the Department of Health, to support understanding of our reports in the development of future public health workforce studies.

We hope that our report helps stakeholders to fully understand the public health knowledge and intelligence workforce is knowing who they are, what they do, how they come to work in public health, and how they can be better supported is vital for ensuring we have the intelligence to help improve the health of the population as a whole. Ensuring a high quality public health workforce will be increasingly vital in the future, especially given the ambitions of the Five Year Forward View, and we encourage you to contribute to the debate. I would personally like to thank all of the stakeholders who worked closely with me and the CfWI public health team in helping to prepare this report.

Find out more about our public health work on our website; if you would like more information please contact Bree Rankin, Head of Public Health (bree.rankin@cfwi.org.uk).

Horizon Scanning at the Business Systems Laboratory Symposium 2015

This blog post was originally posted on the CfWI Horizon Scanning Hub.

“Across every sector of society, decision makers are struggling with the complexity and velocity of change in an increasingly interdependent world.” Klaus Schwab, Founder and Executive Chairman, World Economic Forum, 2012

The Centre for Workforce Intelligence’s (CfWI) robust workforce planning approach, which we use in all our workforce studies, is based on the application of systems thinking principles in order to understand the complexity of the health and social care system. This enables us to create better quantitative workforce demand and supply projections that consider the inherent uncertainty of the future. The planning approach integrates a set of systems thinking methods and techniques such as causal loop diagramming, scenario generation and system dynamics models.

We are always keen to explore our use of systems thinking methods with other expert practitioners. We like to learn about different techniques that could generate additional insights to our workforce planning approach. With this in mind we presented at the third Business Systems Laboratory Symposium in Perugia, Italy. The symposium draws together multidisciplinary practitioners from around the world, and provides an excellent platform for exchanging new ideas. This year the symposium included over 80 presentations from more than 150 authors. I was delighted to be able to support the CfWI with three presentations at the conference, along with colleagues Jack Lawrence and Dan Gioe. Our presentations covered how we are using scenarios in policy analysis, the use of system dynamics modelling in developing robust demand and supply projections, and how we have integrated a series of systems thinking techniques in the Horizon 2035 project.

The health and social care system contains a number of features that benefit from systems thinking techniques. The system is complex and interdependent, with significant delays between intervention and impact. It also consists of multiple organisations and stakeholders with differing perspectives and agendas. It was therefore not surprising to find that a number of authors presented on issues affecting the health and social care system around Europe.  Mike Jackson, noted systems thinking author, discussed how systems methods could be used to better understand the underlying causes of the recent accident and emergency crisis in England, and Kazimierz Sliwa explored recent changes that have occurred within the Polish health care system.

All three of our CfWI presentations were very well received and generated lively discussion, along with some challenging debate. It is always good to be asked difficult questions – it shows you are being listened to! We received a number of compliments about our work. For example, it was noted a number of times that the way we were integrating systems thinking techniques meant that we are always using ‘the right tool for the right job’.  The audience was also very impressed that we were able to demonstrate that systems thinking approaches were being used to inform important national policy decisions undertaken by the UK government.

Thoughts for future systems thinking analysis at the CfWI

A number of authors discussed the development of very high-level semi-quantitative models that can be used to generate strategic insights. Methods that were discussed included Bayesian belief networks, fuzzy logic, and the sensitivity model developed by Vester. Each of these methods has been applied based on high level causal loop diagrams developed with system stakeholders. We have previously explored the use of high level system maps using Vester’s sensitivity model, and this work was reported in ‘Technical Report 6 – Horizon scanning: Analysis of key forces and factors’. It would be worthwhile re-evaluating these system maps in light of the new insights generated during the Horizon 2035 project and attempting these other methods for casual loop diagram quantification.

Example of causal loop diagram

Another area that was discussed in a number of presentations was the use of the viable systems model (VSM). The viable systems model is an organisational representation of the elements and interactions considered essential for any system to be viable. A viable system is one that is organised and operates efficiently within its changing environment. The health and social care sector can certainly be described as existing within a changing and dynamic environment. It would be interesting to apply the VSM to the health and social care system functions associated with workforce planning. This could highlight organisational improvements that would benefit the system as a whole.


Health & Social care system as of 2013 (Image courtesy of Department of Health,


All in all, the symposium offered much food for thought, new people to discuss systems theory with, and a number of promising new avenues to explore that could add value to the robust workforce planning framework. It also confirmed that others certainly think we are using the right approach!

There is more information about the symposium at http://bslab-symposium.net/3rd-international-symposium-perugia-2015/.

Rome 2014: A major step forward in uniting Europe’s healthcare workforce planning agenda

Last week I, along with Horizon Scanning Consultant John Fellows and CfWI Managing Director Greg Allen, attended the second conference of the EU Joint Action on Health Workforce Planning and Forecasting in Rome, hosted by the Italian Ministry of Health.

This conference had the theme of Improving planning methodologies and data across Europe and was well attended by a wide range of policy makers, workforce planners and professional organisations.

The conference was opened by Beatrice Lorenzin, minister of health for Italy, followed by Dr Andrzej Rys from the European Commission DG SANCO, responsible for health and consumer protection.  There was strong mention of future drivers of healthcare and the workforce – ageing, chronic disease, technology and the economy.

The EU Commission shared how the economies across Europe are recovering but are still fragile.  Youth unemployment remains a critical issue and how we tackle challenging workforce imbalances for the future will be key.  The tough questions considered off the back of this included how to balance the challenge of short-term budgets and long-term planning needs.

Workforce planning methods across Europe

For us it was a big deal as we launched the CfWI’s first major EU Joint Action publication, User guidelines on qualitative methods in health workforce planning and forecasting.

This work describes the health workforce planning and forecasting approaches for Belgium, Finland, Germany, Hungary, Netherlands, Spain and the United Kingdom.  It is an update on the EU Commission’s Feasibility study on EU level collaboration on forecasting health workforce needs, workforce planning and health workforce trends.

Also our sister work package led by Italy presented the emerging findings of their work and the production of the handbook on quantitative planning methodologies. More to come in the near future from them.

Future skills and the future workforce

John and I presented Insights on future skills and competences at the largest breakout session to share our ongoing research on future trends and drivers that will affect the workforce.  This is an important part of our work across Europe as part of the Joint Action, with a focus on what drivers will impact skills and competencies in the future, out to 2035. It will identify the mega trends that are of particular note for policy makers and workforce at local, national and international levels.


We also heard from a wide range of excellent speakers from across Europe, from Ireland to Moldova, Finland to Italy and Scotland to Romania.  Topics ranged from workforce mobility, universal healthcare overage and the associated health workforce challenges. We also heard important updates from the Global Health Workforce Alliance and interesting work by Prof Bob Elliot that is aiming to improve the evidence base of skill mix outcomes.

The main message I heard over and over was the need now more than ever to plan for the future of our respective populations, their needs, the potential of innovations we might realise and the dynamics of workforces and skills nationally, across Europe and globally.

I would encourage you to have a look at the many excellent presentations and breakouts available at http://euhwforce.weebly.com/141204-rome-conference.html.

Our thanks to the Italian Minister of Health, her ministry, the EU Commission, the Global Health Workforce Alliance, EU JA partners and Michel Van Hoegaerden.

You can view Matt’s slides from the conference here:

Putting expert elicitation into practice: Lessons learned by the CfWI

Over the last year, we ran a series of elicitation workshops that aimed to quantity the factors that were needed to shape our Horizon 2035 model. Elicitation is the process of obtaining knowledge from one or more experts for uncertain futures.

For the purposes of Horizon 2035, elicitation is used to quantify critical model parameters with an unknown future value. There is no alternative to eliciting expert knowledge as the future is inherently undefined.

At the CfWI, we have decided to use the SHeffield ELicitation Framework (SHELF) (http://www.tonyohagan.co.uk/shelf/) developed by Professor Tony O’Hagan an Emeritus Professor, Department of Probability and Statistics (University of Sheffield).

This form of elicitation replaces our online Delphi panel exercises. This enhancement is a natural evolution of modernising our methodology to fit future needs. Limitations from the incumbent panel method used at the CfWI included the longer length of the process, no open debate, losing engagement of experts between the rounds, and rationales often being reduced to short sentences. The purpose of using SHELF is to improve on such concerns.

The SHELF method involves two rounds of debate on a particular question that is pertinent to an area under consideration. After the first round of debating, experts are asked for a series of values on the question. The median, which is the value the expert believes to be most probable, is determined first. This is followed by the upper and lower bounds which are values the experts believe are possible but extremely unlikely. Finally, the lower and upper quartiles are asked for, which are the twenty-fifth and seventy-fifth quartiles between the upper and lower bounds. These are used to obtain a probability distribution curve for all experts. An example of a probability distribution curve is shown below in Figure 1.

Figure 1: An example of a probability distribution curve. Green line showing the lower quartile, the blue line showing the upper quartile. The peak of the curve representing the median, the most likely value.


These graphs are then shown to the group and a second round of debate takes place in order for a consensus to be reached. For Horizon 2035, we have run three workshops considering physical long-term health conditions, mental long-term health conditions and workforce productivity. The workshops were one day long each, which gave the participants enough time to understand the SHELF approach and to answer our questions.

One lesson that we have learnt is that in order to have a successful elicitation, it is vital that you invite experts that are relevant to the discussion, and provide them with a comprehensive brief beforehand. This is particularly important if you want to save time on the day as there is a lot of content to get through. For our workshops, we sent out briefing packs which explained our project, the process and the questions we will be asking.

The structure of the day was broken down into three sections. The beginning of the day was devoted to explaining our project, what we were hoping to achieve from Horizon 2035, and how the experts could help us. Here we gave the experts a chance to ask questions on the project itself to develop a stronger sense of understanding of the project and its importance. This led to increased levels of engagement.

The second section of the workshop was spent on training the experts in the SHELF method, during the training we had ran through various examples. The experts were explained probability distributions at first. Once these concepts of SHELF had been grasped we were able to move on to the third section of the workshop.

In the third part of the workshop we moved on to asking the experts the questions we needed answering for our project. A running theme in all the workshops, which was interesting to note, was the discussion on whether the question being asked was of the right one. However, for each question being asked, only a few minor tweaks were made to them. For example in long-term mental health, we changed the scope to include alcohol and substance abuse. This showed the levels of engagement of the experts which was a positive sign. Once the questions were agreed by all experts to be the right questions we had started the SHELF process.

The most interesting part of the day was to see the experts debate the questions we had presented them. This showed their expertise in the matter and difference of opinion. For instance, in our workshops we were able to get a consensus on all questions except one where there was a split opinion. This was fine as it presented two differences in opinion on a certain future. In the workshops we may have wanted to consider extra time as asking three questions was challenging and in the last workshop we weren’t able to ask the third question due to time constraints. Looking back at the workshops we could even have spread them over two days to give us enough time for all our questions to be answered, though we recognise that this is a lot of time away from usual jobs for our participants and is not always possible.

All in all the SHELF method has proven to be a great way of gaining quantities which were not possible to gain elsewhere. Going forwards, we will try to improve the process ensuring learning from our previous workshops. All readers are welcome to contact the team with their thoughts or questions to horizonscanning@cfwi.org.uk.