(This article was first posted on LinkedIn)
Keeping aside the buzz and hype about utilization of A.I.* in various disciplines and what mistakenly many assume it can accomplish, there are real merits in governments and decision-makers setting out strategies for adoption of A.I. in health service delivery. In this article, I will review the economic benefits of the application and in later articles, the other advantages.
Of those health systems analyzed by the Commonwealth Fund in their performance rankings, people in the U.S. and Australia had the highest out-of-pocket costs when accessing healthcare. This issue has arisen not for the lack of investment in healthcare by governments in these countries. In 2016, the healthcare spending in the U.S. increased by 4.3 percent to attain a figure of US$3.3 trillion or about US$10,348 per person. Of the US$3.3 trillion, US$1.1 trillion was spent on hospital care, US$92 billion on allied health services and about US$162.5 billion on nursing care facilities and retirement communities. Together these expenditures constituted approximately 45% of the 2016 health spend. Of the total health expenditure, individuals/households contribution matched the government expenditure (28% of the total health expenditure). In Australia in 2015-16, the total healthcare spend was AU$170.4 billion (a AU$6 billion increase compared to 2014-15). Of this, the government expenditure on public hospital services was AU$46.9 billion and on primary healthcare AU$34.6 billion. Expenditure by individuals accounted for 52.7% of non-government expenditure or 17.3% of total health expenditure. While governments continue to increase spending this hasn't really made a serious dent on out-of-pocket costs.
One of the key component of annual healthcare spending and pertinent to this article is the recurrent healthcare expenditure. Recurrent healthcare expenditure does not involve acquisition of fixed assets and expenditure on capital but largely expenditure on wages, salaries and supplements. In Australia, recurrent healthcare expenditure constitutes a whopping 94% of the total expenditure and in the US, about US$664.9 billion was spent on physician and clinical wages in 2016. Considering it is unlikely for wages to go down; it is hard to imagine recurrent healthcare expenditure decreasing and consequently total healthcare spend decreasing .
A.I. technology, which has been around for decades but has only recently received wide-spread attention, is increasingly being applied in various aspects of healthcare (primarily in the U.S.). While in an earlier article, I have argued how A.I. can never totally replace human clinicians**, many number of American hospitals are using A.I. technology to leverage their consultants expertise where in some cases the A.I applications are outperforming them. I won't discuss the technologies and application here (as it will be covered in my book chapter^ and in subsequent articles) but I will discuss the costs of development of these technologies from a healthcare point of view.
With traditional software development, the usual phases include discovery and analysis phase, prototype implementation and evaluation phase, minimum viable product and followed by product release. The costs associated with these phases, depending on the project sizes and complexity of the software, can constitute anywhere from US$10,000 to US$100,000^^. However, development of AI programs (here I will consider Machine Learning based programs not robotic applications, which adopt a different development model and consequently different cost models) have distinctive features to be considered in their development.
These aspects include acquisition of large data sets to train the system and fine-tuning the algorithms that analyze the data. Where significant data sets cannot be obtained, data augmentation can be considered. Costs will be incurred in acquiring the data sets if not available in prior. However, in the context of healthcare government agencies and hospitals can provision this data for developers at no costs (if the A.I. program is being developed/customized for their exclusive use). So the most cost impacting factor is whether the data is structured or not. Data doesn't have to be structured, there are several machine learning algorithms that are trained to analyze unstructured data. However, developing programs to review unstructured data incurs more costs. Even when structured data is available, there are processes like data cleansing and data type conversion, which add to the costs. The next distinctive feature is fine-tuning/customizing the machine learning algorithm to suit organization's requirements. As the healthcare context requires the program to have a high degree of accuracy (less false negatives and high true positive identification epidemiological speaking), many round of refinements of the algorithm will be required.
Even considering these distinctive features, which will add to the baseline costs ranging from US$50,000 to US$300,000^^; you are looking at a range of total costs of US$60,000 to US$500,000^^ (depending on the organisation requirements and complexity of the A.I. software). If we consider in the U.S. about US$664 billion (2016) and in Australia that AU$64 billion (2015-2016) was spent on hospital recurrent expenditure alone, a mere 0.016% allocation of the spend on developing A.I. technologies could fund development of at least 18 (hospital focused advanced machine learning based) applications per annum. The ROI is not just economic but also improvement in patient outcomes because of avoidance of medical errors, improved medical/laboratory/radiological diagnosis and predictability of chronic disease outcomes. Considering the rapid advances machine learning based program have made in medical prediction, diagnosis and prognosis^ , governments and healthcare organizations should seriously consider focus on supporting the development and deployment of A.I.technologies not only for the serious dent these applications can make on recurrent health expenditure but also how they can significantly improve patient access.
* I have a distaste for the term 'A.I.' which I have explained the reasons for in an earlier article but use this term as it is widely recognizable and accepted to portray computational intelligence based products.
^ 'Use of Artificial Intelligence in Healthcare' (book working title: E-Health, ISBN 978-953-51-6136-3; Editor: Thomas.F.Heston, MD)
^^ This estimation does not take into consideration deployment, insurance and marketing costs.
Eric Topol, a Professor of Genomics and Cardiologist, in his 2015 NYTimes bestseller 'The Patient Will See You Now', foresees the demise of the current form of hospital based acute care delivery (with a shift to delivery of care at homes of patients) along with replacement of human delivered clinical services by smart systems/devices in the coming future. I would not go so far as Professor Topol in his assessment about the replacement of human clinicians but can easily foresee the automation of a large part of human led clinical care in the coming decades.
While I do not remotely purport to be an expert on automated medical systems (that will take some more time and neurons); having undertook months of research to complete a book chapter on the use of Artificial Intelligence in healthcare (book working title: E-Health, ISBN 978-953-51-6136-3; Editor: Thomas.F.Heston, MD), just completed a hands-on AI programming course delivered by Microsoft while undertaking a scoping review of the use of recurrent neural networks in medical diagnosis and preparing the use case of a non-knowledge based clinical decision support system, I can rationally state where the automation of healthcare delivery is heading to.
Before I analyse the 'artificial intelligence take-over of medical care' scenario further, I would like to here express my slight distaste for the term 'Artificial Intelligence'. This term derives from the incorrect assumption intelligence has been primarily a biological construct so far and any intelligence that is now being derived outside the biological domain is artificial. In other words, intelligence is framed exclusively in reference to biologically derived intelligence. However, intelligence is a profound entity of it's own with defined characteristics such as learning and reasoning. Human intelligence is not the most intelligence can be and as trends go, computing programs with their increasingly advanced algorithms can potentially in the next decade or so exceed the learning and reasoning powers of humans in some aspects. Therefore, a source based terminology for intelligence would be more appropriate to frame intelligence. In other words, computational intelligence would be more appropriate than artificial intelligence.
Healthcare has been a fertile domain for computational intelligence (CI) researchers to apply CI techniques including artificial neural networks, evolutionary computing, expert systems and natural language processing. The rise in interest and investment in CI research has coincided with the increasing release of CI driven clinical applications. Many of these applications have automated the three key cornerstones of medical care: diagnosis, prognosis and therapy. So it is not hard to see why commentators, including clinical commentators, are predicting the replacement of human clinicians by CI systems. While it is indeed rational, based on current trends in CI research, to imagine automation of many human clinician (for convenience sake, I am focusing on physicians rather than other professions such as nurses, allied health professionals) led tasks including interpretation of laboratory and imaging results (CI applications are already matching the radiologist's accuracy in interpretation of MRI, CT and Radiological images), predicting clinical outcomes (CI applications have successfully predicted acute conditions by reviewing both structured and unstructured patient data) and diagnosing various acute conditions (in fact the earliest CI applications, dating back to the 70's, already had this ability); it is hard to imagine CI systems completely replacing human clinicians in conveying diagnosis and discussing complex treatment regimens with patients, especially with high risk patients. There are some other areas where it is equally hard to foresee automation of clinical tasks such as some complex procedures and making final treatment decisions. So what is to come?
I foresee a co-habitation model. A model that accepts the inevitable automation of a significant number of tasks that are currently performed by human clinicians (the at-risk areas are where there is less human interaction and where there is a structured process in implementing the task; structure means algorithms can be developed easier you see) but allows for human clinicians to make the final decision and be the lead communicator with patients.
E. Topol, The Patient Will See You Now: The Future of Medicine is in Your Hands. New York: Basic Books, 2015.
J. N. Kok, E. J. W. Boers, W. A. Kosters, P. Van Der Putten, and M. Poel, “Artificial Intelligence: Definition, Trends, Techniques, and Cases,” 2013.
D. L. Poole and A. K. Mackworth, Artificial Intelligence: Foundations of Computational Agents, 2nd Edition. Cambridge University Press, 2017.
B. Milovic and M. Milovic, “Prediction and Decision Making in Health Care using Data Mining,” Int. J. Public Heal. Sci., vol. 1, no. 2, pp. 69–76, 2012.
K. L. Priddy and P. E. Keller, Artificial Neural Networks: An Introduction. Bellingham: SPIE Press, 2005.
S. C. Shapiro, Encyclopedia Of Artificial Intelligence, 2nd Editio. New York: Wiley-Interscience, 1992.
R. Scott, “Artificial intelligence: its use in medical diagnosis.,” J. Nucl. Med., vol. 34, no. 3, pp. 510–4, 1993.
We all have come across poorly done evaluations. While program evaluation methodologies can get complex reflecting the real world complexities, there are fundamental steps all evaluators should consider in their evaluation. As a prelude to my book 'Evaluation Made Easy' (to be published next year), I have prepared this video. The video discusses the basic expectations for a good program evaluation.Please feel free to use and share the video. to edit.
Quite often researchers and managers confuse or conflate program evaluation and performance audit as a single concept. Surprisingly, this happens too often. Notwithstanding the theme of ‘assessment’ across these two methodologies there are vital differences. This article outlines the differences between the two approaches at a high level.
First, let us define the two methodologies. While there are numerous ways of defining performance audits, a simple definition is ‘determination of the compliance of programs, activities and functions with predetermined standards’. Program evaluation, on the other hand, is ‘the systematic assessment of the effectiveness and efficiency of a program’. The definition’s itself set out the differences in the methodologies.
Further differences arise in the implementation of the two approaches. Whereas, performance audits does not so much concern itself with research theories and is focused on answering the question ‘does the service reach a predetermined standard?’; program evaluation, which has evolved from social science utilises a theoretical base to judge the quality of the program. In other words, program evaluation answers the question ‘what standard does this program achieve?’
Audits are designed to be observational and not interventional. They are set-up to provide assurance that the service quality and delivery meets acceptable standards. However, audits are not designed to provide you with the answer ‘how and why’. This is addressed by program evaluations, which in addition to addressing the ‘how and why’, also if designed well answer ‘what and what next’? Ideally, evaluations are to be experimental and utilise a counter-factual too. Thus, evaluation designs adopt a more comprehensive approach. This is not to undermine the value of performance audits. In most instances, when only an assurance exercise is required in a time sensitive context, performance audits are very suitable. Also, audits are more amenable to be implemented by non-experts. However, as with any scientific approaches, they have to be designed by experts.
So the choice between a ‘performance audit’ and a ‘program evaluation’ is what does the commissioner require to know about the program? If it is just to measure compliance, the choice will be an audit. If it is to know how a program has performed and why so, the choice will be a program evaluation.
HRA. (2013). Defining research. NHS Health Research Authority, London.
Hurd, I. (1993). Linkages between audit and evaluation in Canadian Federal Departments. Treasury Board Secretariat. Government of Canada.
Davis, FD. (1990). Do you want a performance audit or a program evaluation? Public Administration Review.50: 35-41
(I originally published this article in Linkedin)
Recently, I was invited to be a panel speaker at a Healthcare Summit (this is to occur next month in Melbourne). A topic that has been suggested for discussion is "What's next for healthcare and where do we see ourselves in 15-20 years?" I have thought about this subject many times in the past but this invite reignited my dormant cogitation. So, I spent some time revisiting this issue through the prism of recent trends and published evidence. I now provide a summation of the analysis in this article.
At the onset, I must state I believe the future of healthcare is very much yoked to current healthcare investment, planning and delivery-the state of which presents both a pessimistic and optimistic picture.
Pessimistic, because I find a significant amount of focus on delivery of healthcare guided by political, pecuniary, professional and parochial interests. In many instances, planning to cater future demand for healthcare is in-sapient and neophobic. With the quantum of accessible big data, published epidemiological evidence and identified variability and disparity in healthcare service delivery, it does boggle one's mind how investment into healthcare service delivery can be so poorly channelled and implemented? Accepting there are financial, political, workforce and infrastructural limitations that pertain to each country's health system, there are yet fundamental activities (as they relate to access, minimisation of waste/variability in care and a patient centred approach) that each health system can easily adopt. However, a reactionary state of affairs, rather than a proactive and preventative approach, reigns in many countries. An OECD report released in January this year identified that of every dollar spent on healthcare in OECD countries, 20 cents is wasted. What this means the governments could have spent twenty percent less yet improved patient's outcomes. Also, the European Parliament has estimated that an estimated 1.4% of the GDP of European countries is lost because of health inequities (the amount lost is approximately equal to the Defence spending share of the GDP). While life expectancy increases across the globe, health disparities in both developing and developed countries persist at a significant scale. In a study published in Health Affairs this year, 38% of the US population were found to report "poor health" because of inability to access healthcare services. Further, the WHO estimates a shocking 150 million per year across the globe face catastrophic health costs because of user fees. So there is a lot to be concerned about how healthcare is being delivered across the world.
However, we do see a glimmer of hope when noting trends in health literacy, patient advocacy, investment in public health interventions, rapid uptake of mobile/pervasive technology and increasing use of shared assessment platforms. In a study published in Lancet earlier this year, which reviewed 'Healthcare Access and Quality (HAQ)' indices in 195 countries found nearly all countries improve their indices between 1990 and 2015. Also, the study found the Global HAQ improved from 40.7 in 1990 to 53.7 in 2015. In the same journal, a study assessing 33-health related Sustainable Development Goal (SDG) indicators in 188 countries across 25 years found pronounced progress with modern contraception, under-5 mortality, and neonatal mortality. There is also increasing investment in public health by both developing and developed countries. An economic evaluation of South Africa's investment in public health care over the period 2005 to 2014 identified that a direct association with significant improvement in under-five mortality rate. With the rapid increase in the use of mobile phones especially in developing countries ( for example, 283 million in China 125 million in India, 46 million in Indonesia, 35 million in Ghana) and the accelerated development of mobile health applications and cloud-based data accessible on mobiles, the WHO has actively been pushing a Global Digital Health agenda to capitalise on this momentum. The use of smartphone technology in the delivery of healthcare is increasingly reducing the cost of healthcare delivery. The US Food and Drug Administration just last year approved more than 35 digital health applications and the UK's NHS has pledged to incorporate digital innovation in its strategy for delivery of healthcare into the future.
Coming back to the original question, where do I see healthcare delivery heading into the future? When you analyse countries that show the strongest performance in their health indices you will see some common threads: Universal Health Cover, robust investment in primary/preventative health care, timely access to hospital care, and minimal variability in the delivery of healthcare services. In a systematic review of the Return on Investment (ROI) of national public health interventions published last month (in the Journal of Epidemiology and Community Health), it was found that the median ROI was 27.2 and cost benefit ratio was 10.3. Also, several studies have identified that schemes that minimise fee burden on patients not only improve their access to healthcare but also improve health outcomes of the community as such. Provisions or guarantees for timely access to hospital services are now written into legislation, for example California and Canada. Further, in a report commissioned by Philips where more than 25,000 patients and 2600 healthcare professionals across 13 countries were polled for their perceptions of their views on healthcare found patients and healthcare professionals highly value integrated seamless healthcare i.e. healthcare centred around the patient's needs. Based on the above factors, I outline key principles that should guide design and delivery of healthcare into the future.
At the System Level (DEFA):
Demographics: The demographic dividend that resulted from post world war in developed countries has now because of low fertility rates (it is estimated that 60% of the world's population is presenting this trend) transformed to a demographic recession. This means unless there is a replacement population strategy, an inverted population pyramid with an increasingly ageing population (because of increasing longevity) serviced (through taxes or otherwise) by a younger, but smaller, populace will emerge. Even in populous countries, like China, the dependency ratio for retirees is increasing rapidly (In China it is said to balloon to 44% by 2050). How does a healthcare system constrained by resources cope with this demographic trend? As assumed incorrectly by many, providing healthcare to an ageing population will not lead to rocketing of costs. A report published by the Australian Government some years ago identified that annual percent of health costs due to ageing alone would reduce from 2020 onwards. However, it is known that the aged rely on the government more than many other age groups and require complex care, which results in greater utilisation of health services than other demographic groups. So attuning of health service delivery to this demographic will be key in ensuring a responsive health service.
Epidemiology: While dividing epidemiology into 'chronic disease' and 'infectious disease' camps is problematic, the scale of the global chronic disease burden (estimated by WHO to be a lost productivity of approximately US $84 billion) merits a specialised focus on its distribution and causation. A significant number of chronic diseases emerge through behavioural risk factors (smoking, poor nutrition, physical inactivity, excess alcohol consumption..etc). These risk factors are amenable to change through appropriate early interventions. A research report analysing chronic disease epidemiology in 23 countries that accounted for 80% of the global disease burden stated that just a mere 2% decrease in chronic disease death rates per year in these countries over the next 10 years would result in 24 million deaths averted and US $8 billion saved. Thus, health systems have to incorporate evidence backed public health interventions in their design and delivery and link them to the wider government's investment into social determinants like housing, education and transport.
Financing: Who needs to bear the responsibility of healthcare financing is a very emotive and political issue that immediately yields a bewildering array of opinions. However, I write here with the premise that studies have shown countries with universal health care systems perform better than heavily privatised health systems. Healthcare Financing is an important lever to achieve universal healthcare. Generally, healthcare financing is achieved through government payments, private health or social insurance plans and out-of-pocket payments. If we consider the role of the government in financing the health system, their ability is impacted by demographics, epidemiology, technological and workforce costs and performance of their country's economy. Noting current trends, these factors will continue to place 'stress' on the government's ability to finance healthcare. It will be easy for the government to shift the 'responsibility' to the users (social insurance) and/or the private sector. While there may be some pro's with the shift in the responsibility, there is significant evidence that spreading costs of healthcare to all production components and financing universal health care through a combination of progressive taxation of income supported by insurance and out-of-pocket payments not only yields the best chances for health system sustainability but also efficiency and equitable outcomes.
Access: in the healthcare context can be simply stated as the ease with which an individual can receive healthcare. However, there are several dimensions to this indicator. Authors Levesque, Harris and Russell in their article in the International Journal for Equity in Health outline 5 dimensions: Approachability, Acceptability, Availability, Affordability and Appropriateness. Contrary to generally held opinion, economic factors alone do not determine a patient's ability access to health services. Social, cultural, and geographic factors all play a role in access to health services. However, financial reasons outplay other factors in its scale, significance and relevance across health systems in determining access to healthcare. Studies have shown governments have the most influential role in ameliorating the barriers financial considerations place. The government also has a consequential role in ensuring equitable access to all segments of the population irrespective of their location, race, religion and culture. Yet, time and again, governments fail in ensuring timely access, equitable and appropriate care. While we can be quick to admonish governments for their failures, it is unrealistic to expect access in its truest sense to be achieved in a resource constrained and diverse geographical environment. Compromises and rationalisation need to be accepted along with a focus on preventing the conditions that create demand on stretched resources.
At the individual level (PPC):
Personalised Healthcare: This term encompasses more than personalised medicine, which involves the use of genetics and genomics to guide and deliver healthcare. Personalised healthcare is about using biological information to predict the risk of acquiring a disease or how a patient will respond to treatment. Very often lack of personalisation of care results in medical errors and in appropriate treatment. Not to mention the waste and costs. A personalised approach not only will result in the use of appropriate treatments but also considerably reduce the costs of care. The other advantage of personalised healthcare is involvement of patients in devising treatment plans. Because of the involvement of the patient in planning and sharing of information about their therapy, personalised healthcare also achieves higher compliance than many other models of care.
Proactive Preventative Healthcare: Many health systems across the world have become reactionary systems based on episodic, acute care models. The focus is on diagnosing and treating illnesses rather than preventing them happening in the first place. It is a 'wait and react' model i.e. wait for the patient to become sick and then treat. With the availability of big data, epidemiological data, technological innovations and evidence for the efficacy of early interventions, the excuses to continue with this model are fast running out. I am not blind to the challenges that emerge when shifting from a reactionary model to a proactive and preventative care model. The challenges include financing models that favour secondary and tertiary care, the barriers to integrating healthcare across multiple levels (horizontal and vertical) and the time taken to visualise results of investments. However, with ballooning healthcare costs and increasing uncertainty of current healthcare delivery models, there really needs to be a radical approach to shift the thinking around healthcare service delivery. Macintosh, Rajakulendran, Khayat and Wise in their 'Transforming Health Market' report discuss the catalysts for this shift to occur. The enablers include a decentralised approach to deliver healthcare, focus on health outcomes rather than outputs, collaboration across the system and empowered patients.
Customised Healthcare: With this approach, healthcare design and delivery is centred around the patient. There has been extensive discourse about patient-centered care (PCC) in the past many years to the point this term now sounds cliche. Yet, the shift of healthcare delivery to a patient centred approach has been slow-going. Patients continue to face challenges in navigating the healthcare system and receive timely and appropriate treatment. Some Scandinavian countries have established a seamless system to share patient data across providers (and to the patient themselves). The health care delivery model in these countries presents a great narrative as to how a publicly funded but privately delivered healthcare can achieve excellent health outcomes. In Sweden, a decentralised customised healthcare delivery model with controlled costs is the envy of the world. Healthcare Organisations do not need to wait for their country to adopt a Swedish model. A study of more than 3000 hospitals in the United States by the Armstrong Institute for Patient Safety and Quality identified elements common to hospitals, which adopted customised healthcare. These components include Hourly Rounds, Communication Boards in Patient Rooms, Bedside Shift Report, Discharge Folders, Post-Discharge Phone Calls, Multi-disciplinary Rounds, and Transparent Standards of Performance. As one can see these measures do not require a great amount of investment and can be mostly adopted with existing resources.
I end this article with this quote from Dr Louis Hugo Francescutti " Unilaterally cutting cost won’t eliminate the inefficiencies, unnecessary procedures and avoidable burdens on the system. But if we work together with fresh thinking and strategically-smarter spending, we can recast into a more and humane and efficient health care system and the lower overall costs to governments will follow."
(I originally published this article in Linkedin)
Program Evaluation is a well-established methodology to assess the effectiveness and efficiency of programs. The methodology to undertake program evaluations has become diverse and complex over the years. However, program evaluation approaches can be grouped into two main categories: method-driven evaluation and theory-driven evaluation. With the method-driven approach, the emphasis is on the methods (qualitative or quantitative or mixed methods) employed to assess the success of the program. In contrast, the theory-driven evaluation emphasises the centrality of the program theory with methods being determined on their suitability to test the theory.
Each approach has its strengths. With the method is driven evaluation, which is commonly used; the strengths are the approach is usually faster to implement and provides stakeholders easier to understand results. With the theory-driven evaluation, the results are context specific; thus, more accurate. However, each approach has its limitations too. The method driven approach, which in most instances does not emphasise the context or analyse the intervention provides results that are somewhat generic. The theory-driven approach takes a longer time to implement and provides results that are sometimes abstract or complex for stakeholders to understand and action. Also, some have criticised theory-driven approaches as too academic in its approach and not stakeholder friendly.
II. Integrated Approach:
The author through this paper presents an integrated model, which employs the best of the traditional method-driven and theory-driven approaches while addressing the limitations of each approach. The principle behind the development of the integrated model is to ensure stakeholders gain the greatest value from the commissioning of program evaluation. At the moment, most of the current evaluation approaches come with inherent limitations in addressing stakeholder needs. This is very unhelpful as all commissioned evaluations need not only to assess program but also present solutions to issues identified. By presenting weak or abstract results, that cannot be followed up; current models do not necessarily benefit stakeholders. To address this major limitation, the integrated model utilises components from both evaluation approaches that have, over the years, been attested as practical to assess programs. The model also includes innovative elements to ensure results delivered by implementing this model will be useful for stakeholders.
The model includes traditional aspects of program evaluation such as program logic components: Inputs, Outputs, Outcomes and Key Performance Indicators (KPI). Also, it includes theory-driven elements such as Context and Program Theory, which are often ignored in traditional program evaluation models at the detriment of the validity of the results. By incorporating practical and useful components and leaving out esoteric concepts that indulge academics more than stakeholders, the integrated model ensures deployment of this model can be done in realistic time frames.
The key focus areas of the IMoE are as follows:
A. Program Theory: is a causal statement outlining the expected outcomes as a result of the program intervention in a particular context. The program theory is developed in ahead of the program assessment by consulting with stakeholders, literature and other sources. The program theory is then tested throughout the course of the evaluation using various appropriate methods. Incorporation of the program theory component in the IMoE ensures views of stakeholders are taken in advance, and a theory is developed about how the program is working or not. The program theory also considers the context in which the program was introduced thus tailoring assessment and solutions to be context specific. By refining or revising the theory later, the evaluation ensures assumptions are tested, and appropriate solutions are presented if the program is not working.
B. Context: describes the situation in which the program has been introduced and is operating. The context includes geopolitical, economic and other scenarios that influence the program’s implementation and outcomes. The context component is often ignored in many generic evaluation approaches leaving assessments either incomplete or invalid. Program’s don’t succeed or fail merely because of the resources or change brought in by the program but also because of the context in which they were introduced; ignoring the context in assessment lessens the credibility of evaluation results. Therefore, the IMoE includes and emphasises description and assessment of context.
C. Intervention: includes the resources and outputs being introduced through the program. Inputs and outputs are data collected through regular evaluations but the IMoE groups them under the ‘intervention ‘category to distinguish it from the ‘Change’ and ‘Outcomes’ component of the IMoE model.
D. Change: incorporates variations that have occurred as a result of the program intervention. The changes can be positive or negative. Positive changes are those support program objectives, and the negative changes are those deter achievement of program objectives. The changes are to be stated in the preliminary program theory and assessed during the evaluation.
E. Outcomes: are the end results of a program i.e. the objectives or the goals the program set to achieve. Depending on the duration of the program, short-term or mid-term or long-term outcomes are considered in the evaluation. The outcomes are not directly assessed but assessed through key-performance indicators (KPI) developed by the evaluator in consultation with stakeholders. The KPI can be included in the program theory, but this is optional.
As it can be gathered, the emphasis of the IMoE is not the methods or academic interests of the evaluator, but the usefulness of the evaluation results to stakeholders i.e. are the results valid and can they be acted upon? This is achieved through incorporation of the program theory and emphasis on the context, intervention and change. Also, the emphasis on the program theory means stakeholders are involved in the very onset while providing an opportunity for their assumptions to be tested through a vigorous approach. As the IMoE incorporates context in the construction of the program theory; it ensures the results are tailored to the particular program, organisation and scenario. Further, by incorporating program logic elements the IMoE ensures the implementation is practical, results are understandable to stakeholders, and the model is not restricted to impact assessments only.
The IMoE has several strengths to it as it brings together the best of the traditional and theory-driven approaches of program evaluation. While it incorporates several components from both the approaches, the enjoining does not result in a complex or unwieldy model. In fact, a streamlined stakeholder-centric process is constructed. If there is a limitation to the IMoE, it is conceptual at this stage and is yet to be implemented. However, steps are being taken to employ IMoE in various healthcare settings.
1) WCSRM. Introduction to Evaluation. Web Centre for Social Research Methods.2017. Available from: https://www.socialresearchmethods.net/kb/intreval.php
2) Chen H, Rossi PH. Introduction: Integrating theory into evaluation practice. In: Chen H, Rossi P, editors. Using theory to improve program and policy evaluations. Westport, CT: Greenwood Press; 1992. p. 2–11.
3) Chen H. Theory-driven evaluations. Newbury, CA: Sage Publications; 1990. p.328.
4) Hansen MB, Vedung E. Theory-based stakeholder evaluation. Am J Eval. 2010;31(3):295–313.
5) Marchal B, Van Belle S, Westhorp G, Peersman G. Realist Evaluation Approach [Internet]. 2015. Available from: http://betterevaluation.org/approach/realist_evaluation.
6) Porter S. Realist evaluation: an immanent critique. Nurs Philos [Internet]. 2015;16(4):239–51. Available from: http://doi.wiley.com/10.1111/nup.12100.
7) Cojocaru S. Clarifying the theory-based evaluation. Rev Cercet si Interv Soc. 2009; 26(1): 76–86.
8) White H. Theory-based impact evaluation: principles and practice. Journal of Development Effectiveness. 2009. p.271–84.
9) Christie C A, Alkin MC. The User-Oriented Evaluator’s Role in Formulating a Program Theory. Evaluation. 2003; 24(3): 373–85.
(I originally published this article in Linkedin)
I initially titled this article as "Why most of the current hospitals will be obsolete shortly?" but I decided the statement as negative and hence the above title. However, the content of this article still revolves around the old title.
Currently, most hospitals in most countries operate as insulated or disengaged structures within the community. Even within the hospitals, many departments/divisions work as silo structures with minimum links (depending on protocols and need) with other units in the hospital. Both of these aspects have contributed to increasing costs and risks to patients. The burgeoning tide of an aged populace, patients with chronic diseases and rising costs of medical technology and specialised workforce has led to a perfect storm for many hospitals.
So where does this take us to? There are various solutions from the elimination of waste, better use of existing resources and update of clinical protocols to minimise costs and decrease risks to patients. However, there is so much these measures can do to avoid making many hospitals outmoded imminently. These actions have to be expanded to encompass a radical rethink of how hospital services are delivered and how the hospital is configured within.
For the sake of brevity, I will focus on three essential elements:
1) The first measure is to reconfigure the current set-up, which has units in hospitals organised by disciplines i.e. Medicine, Surgery, Paediatrics. This separation of groups by disciplines has passed its due date. This configuration has led to miscommunication amongst staff, barriers to effective treatment and waste of meagre resources. The new approach is to organise units around diseases or body systems. For example, Cardiac Sciences or Diabetes or Respiratory Sciences.., etc. Setting up units/departments as such will bring together clinical professionals of different backgrounds to focus on a united purpose and allow for best use of equipment and resources common to all.
2) The second measure and perhaps 'the definitive' element of a hospital of the future is to have integrated care in it is the truest sense. What this means is the co-location of health services at all levels in the same campus. This means the provision of general practice, allied health and acute level services in the same vicinity with linked electronic records. This will not only ensure continuity of care but also decrease inconvenience and expenses for patients. The other benefits are decreased risks to patients that come with the transfer of records and handover from a different system. Further, provision of such integrated care will lead foster an environment of innovation.
3) The third aspect of planning and delivery of hospital services in the future is likely to be controversial. This measure involves closing down or amalgamation of hospital services in many regional or rural centres. The operation of hospital services in many regional or rural locations has led to unsustainable expenditure and compromises with patient safety. There is a need to move away from a mindset of delivering impractical hospital services in such centres. The hospitals here are to be replaced by a strong primary care set-up accompanied by referral centre to transition/transfer patients who require secondary or tertiary care.
I would like to hear what you think about these propositions? Please comment below.
(I originally published this article in Linkedin)
The World Health Organisation in one of its definitions of 'Healthcare Systems' describes it as thus "A health system consists of all organisations, people and actions whose primary intent is to promote, restore or maintain health. This includes efforts to influence determinants of health as well as more direct health-improving activities. A health system is, therefore, more than the pyramid of publicly owned facilities that deliver personal health services. It includes, for example, a mother caring for a sick child at home; private providers; behaviour change programmes; vector-control campaigns; health insurance organisations; occupational health and safety legislation. It includes inter-sectoral action by health staff, for example, encouraging the ministry of education to promote female education, a well-known determinant of better health."
Here most of us who are conversant with 'Systems Thinking' understand a system comprises components but what most of us do not realise components by themselves does not maketh a system. The inter-connectivity and common standards coupled with the components is what completes a system. Bearing this in mind; it is hard to describe many countries health service structures as systems. Most health care is delivered in countries through 'cottage industry' frameworks where each healthcare organisation operates according to its standards with little connectivity to other healthcare services. Very few, if any, countries have all their health service delivery units (public and private) connected and provide standardised care (taking into account the different levels of provision of healthcare). Hence the use of the term 'Healthcare System' is a misnomer. Though for lack of a better terminology and as an aspirational term, we will continue to use this phrase.
(I originally published this article in Linkedin):
Program Evaluation is ubiquitous, and a lot of people conduct program evaluations without having training in program evaluation nor having a grasp of the fundamentals. This is a pity because these evaluators assess important programs and relay findings which do not even touch upon the basic elements of program evaluation (program logic and program theory)- thus squandering time and resources. While there are a bewildering array of evaluation methodologies/approaches, fortunately, the fundamentals underpinning program evaluation remain the same across these approaches.
I have been a learner regarding program evaluation (and will continue to be). In this learning journey, I have come across two sources which have benefited me immensely.
The first source is written to be light in its language but not in the message it is trying to convey i.e. to ensure program evaluation is conducted effectively and efficiently:
The second one is a text available for free to download at the below link and covers program evaluation comprehensively (it is light on theory-driven evaluation approaches but will forgive the authors for it is otherwise comprehensive):
I would encourage those who are interested in program evaluation to start off with the first attachment and progress to the text, which is also quite easy to read.
Far from certain outcomes, healthcare reforms often lead to unexpected results. So why do governments frequently engage in healthcare reforms?Are healthcare reforms the necessary solution? What kind of reforms should governments engage in? These and other issues will be addressed in a journal article. I will provide the link here once published.