Measuring quality of care for all women and newborns: how do we know if we are doing it right? A review of facility assessment tools - Chuyên Trang Chia Sẻ Kiến Thức Thời Trang Mới Nhất

Existing facility assessment tools provide a valuable way to assess quality of maternal and newborn care as one element within the national measurement toolkit. Guidance is clearly needed on priority measures and for better harmonisation across tools to reduce measurement burden and increase data use for quality improvement. Targeted development of measurement modules to address important gaps is a key priority for research.

Five surveys met our inclusion criteria: the Service Provision Assessment (SPA), developed for the Demographic and Health Surveys programme; the Service Availability and Readiness Assessment, developed by WHO; the Needs Assessment of Emergency Obstetric and Newborn Care developed by the Averting Maternal Death and Disability programme at Columbia University; and the World Bank’s Service Delivery Indicator (SDI) and Impact Evaluation Toolkit for Results Based Financing in Health. The proportion of quality measures covered ranged from 62% for the SPA to 12% for the SDI. Although the broadest tool addressed parts of each of the 31 quality statements, 68 (25%) of 274 input and process quality measures were not measured at all. Measures of health information systems and patient experience of care were least likely to be included.

We identified assessment tools through a purposive sample that met the following inclusion criteria: multicountry, facility-level, major focus on maternal and newborn health, data on input and process indicators, used between 2007 and 2017, and currently in use. We matched questions in the tools with 274 quality measures associated with inputs and processes within the WHO standards. We excluded quality measures relating to outcomes because these are not routinely measured by many assessment tools. We used descriptive statistics to calculate how many quality measures could be assessed using each of the tools under review. Each tool was assigned a 1 for fulfilling a quality measure based on the presence of any or all components as indicated in the standards.

Ensuring quality of care during pregnancy and childbirth is crucial to improving health outcomes and reducing preventable mortality and morbidity among women and their newborns. In this pursuit, WHO developed a framework and standards, defining 31 quality statements and 352 quality measures to assess and improve quality of maternal and newborn care in health-care facilities. We aimed to assess the capacity of globally used, large-scale facility assessment tools to measure quality of maternal and newborn care as per the WHO framework.

Existing health facility assessment tools, which collect data at the facility, provider, and patient level to assess the provision of services, might also capture the measures for quality-of-care standards, albeit not designed for this purpose specifically. These tools, although not universal in content or use, are widely used to assess health system readiness and performance.Given the current priority of maternal and newborn health and the growing focus on measuring and improving quality of care in health-care facilities,it becomes important to identify which components of WHO’s standards are being measured with existing global tools and to know whether they allow for the assessment of quality care which, in turn, can help support countries’ quality improvement efforts. In this paper, we assess the capacity of these currently used tools to measure quality of maternal and newborn health as defined in the WHO framework.

Assessing improvements in the quality of care requires measurement of a baseline level of quality and requires tracking progress against a well defined benchmark of reference.To our knowledge, no unified global tool is available that can capture the comprehensive list of indicators for maternal and newborn health set forth by WHO. In fact, many facility assessment tools exist that collect large amounts of data.In addition, we understand that there might be other country-level processes that capture some components of quality of maternal and newborn care, yet these measures might be used for quality improvement processes at the local level and not reported at the global level.

These standards can be used to guide multiple activities at the national and facility level, such as creating national standards of care, measuring the quality of care, providing benchmarks for ensuring high-quality services at the facility level, and informing health facility audits.An example of such an activity is the Quality, Equity, Dignity (QED) Network, comprising ten countries (Bangladesh, Côte d’Ivoire, Ethiopia, Ghana, India, Malawi, Nigeria, Sierra Leone, Tanzania, and Uganda), which was established with the goal of halving maternal and neonatal mortality and improving experience of care by 2022.With the standards as the basis, this network focuses on strengthening national leadership, accelerating action to improving quality, sharing knowledge and generating evidence throughout, and establishing accountability mechanisms through a monitoring framework.In addition, WHO has released standards for improving quality of care in health-care facilities for children and young adolescents.

Maternal and newborn survival is linked to the quality of the care they receive in health-care facilities. Appropriately measuring the level of quality available to pregnant women and newborns is the first step towards ensuring that all women and newborns are provided the best quality of care. Available evidence has shown that focusing on assessing input measures alone or availability of essential interventions or resources will fall short of understanding what actually happens when women and newborns receive facility-based care. Our findings support the need for harmonising existing measurement tools with global standards and filling gaps in measurement. Finally, we note the urgent need for guidance on how to adapt the standards at the country level and establish priorities for measurement, especially in resource-poor settings. The available evidence points to the value of existing tools in measuring components of quality of care as one element in the quality improvement toolkit, while pointing to the gaps by measuring what needs to be measured.

Our work contributes to the growing body of literature on quality measurement by assessing existing widely-used tools that assess both input and process measures. Only three of the tools reviewed completely assessed some of the quality statements in the standards, yet they mostly did this incompletely. Most of the quality measures typically assessed by the existing tools relate to input measures and availability of physical resources. Our work identifies important gaps in measuring experience of care and with regards to health information systems. We also bring attention to existing misalignment between what existing tools measure and what the standards call for regarding quality of maternal and newborn care. Our study can provide support for further research in those weaker areas of measurement: in validating data use for improved clinical care and use of new and existing tools to ensure woman-centred, respectful care.

We searched PubMed for literature related to health facility assessments and quality of maternal and newborn care using the following search terms, (((quality of care) AND ((((maternal OR pregnancy OR obstetric OR delivery OR intrapartum OR antenatal OR postnatal OR postpartum))) OR ((maternal health) OR newborn health)))) AND (((measuring) OR measurement) OR standardized tools OR facility assessment). Previous research classified health facility assessment indicators by type of quality measure and found a preponderance of input measures only. Studies from the past 5–10 years have shown poor alignment between global indicators for maternal and newborn quality care and health facility assessments, and these mostly focused on the Service Provision Assessment and Service Availability and Readiness Assessment tools. Given the present context, in which quality measurement and improvement are receiving unprecedented global attention, and with the release of the WHO standards for quality maternal and newborn care in 2016, understanding the extent to which existing tools can be used to measure quality is imperative.

In an effort to understand and improve quality of maternal and newborn care, WHO developed a framework and associated global standards for health-care facilities.The framework covers eight domains focusing on both the provision and experience of care, in which a standard is developed for each. The WHO framework for improving quality of maternal and newborn care includes eight overarching standards ( figure 1 ). These standards are defined as, “a description of what is expected to be provided to achieve high-quality care around the time of childbirth” and are broken down into 31 quality statements (“concise statements of priorities for measurably improving quality of care around childbirth”), which are then organised into 352 quality measures (“criteria for assessing, measuring and monitoring the quality of care as specified in the quality statement”) according to the three categories of input, process, and outcomes.Each quality statement comprises multiple measures designed to monitor quality of care. For example, standard 5, “women and newborns receive care with respect and preservation of their dignity” is composed of three quality statements: 5.1, “all women and newborns have privacy around the time of labour and childbirth, and their confidentiality is respected”; 5.2, “no woman or newborn is subjected to mistreatment, such as physical, sexual or verbal abuse, discrimination, neglect, detainment, extortion or denial of services”; and 5.3, “all women have informed choices in the services they receive, and the reasons for interventions or outcomes are clearly explained.” Statement 5.1 is assessed by eight quality measures: three correspond to input, two to process, and three to outcome measures.

High-quality care during pregnancy and childbirth is crucial to improving health outcomes for women and their newborns.Despite an increase in institutional births and skilled birth attendance globally, only a few countries were able to achieve the Millennium Development Goals of reducing maternal mortality ratio and infant mortality rate.Although access to institutional care has increased, it is still poor for a substantial proportion of women and many avoidable deaths occur within health-care institutions or despite the presence of a skilled birth attendant. The Sustainable Development Goals (SDGs) and the Global Strategy for Women’s, Children’s, and Adolescents’ Health (2016–30) have identified issues of equity and quality as inherent to addressing this shortcoming, especially in resource-limited settings.

We used descriptive statistics to calculate how many quality measures could be assessed using each of the tools under review. Each tool was assigned a 1 for fulfilling a quality measure based on the presence of any or all components as indicated in the standards. For quality measures with multiple subcomponents, at least one of the subcomponents had to be measured (eg, a quality measure might list several medicines and the tool might measure a subset of medicines on the list) unless the quality measure explicitly required that all subcomponents be present for the measure to be met (eg, provision of essential newborn care required four elements; tools had to assess all four). We then calculated the percentage of quality measures captured through each tool—ie, what percentage of quality measures of a given quality statement were being assessed with a specific tool.

First, we selected the modules for each of the tools that included questions related to quality: facility-based audits, provider and patient or exit interviews, and direct observation. Second, we established whether the questions included in the tools assessed the quality measures in WHO’s standards by matching questions in the tools to each of the measures, which required ensuring all questions in the tools and all measures were considered. For example, availability of uterotonics was captured both under service delivery components of some tools and under availability of essential medicines.

We identified instruments used for large-scale surveys through a purposive sample of tools that met inclusion criteria: multicountry, facility-level, major focus on maternal and newborn health, data on input and process indicators, used between 2007 and 2017, and currently in use. We excluded surveys used only in research studies, or some region-specific tools, because the purpose of this exercise was to identify tools that had already been applied at a national scale in multiple contexts, including several low-income and middle-income countries.For each selected tool, we identified the most comprehensive version implemented to date. After a preliminary analysis, we decided to exclude all quality measures relating to outcomes because these are not routinely measured by all of these assessment tools. Instead, we assessed only input and process indicators, which are crucial components of the process of quality. We reviewed each questionnaire and matched questions in the tools with quality measures associated with the standards.

Of the 274 quality measures that were included in the analysis, 68 (25%) were not measured by any of the tools (see appendix pp 97–102 for a complete listing). A breakdown of the measures that were not assessed shows that the standards that are being captured the least by the tools are the ones relating to health information systems (standard 2) and the ones relating to the experience of care, including effective communication (standard 4), respect and preservation of dignity (standard 5), and emotional support (standard 6). Standard 2 was only measured by two tools—SPA, which assessed 42% of the measures in this standard and AMDD, which assessed 63%. Standards 4 and 6 were only measured partially by two tools—SPA and AMDD, ranging from 14% to 67%. For standard 5, only the SPA tool partially covered all of the measures (an average of 41%). Conversely, standards relating to the availability of infrastructure and essential medicines (standard 8) and appropriate referrals (standard 3) were covered more comprehensively across tools; while across tools 49% of measures in standard 8 and 34% of measures in standard 3 were captured on average, all tools addressed at least some measures of each quality statement.

Our analysis indicates that the SPA tool partially assessed all 31 quality statements, while covering 62% of the 274 quality measures. The AMDD tool partially covered 30 of 31 statements, while covering 57% of the quality measures. Next comes the RBF survey, partially covering 24 of 31 statements, yet only 18% of all quality measures. The SARA tool covered 23 of 31 quality statements, while it captured 30% of quality measures. The SDI survey only assessed 15 of 31 quality statements and 12% of the quality measures. Figure 4 depicts how well each of the tools measured each of the eight standards and figure 5 shows how well each tool was able to measure the 31 quality statements.

Green boxes indicate complete fulfilment of all measures included in that quality statement. Yellow boxes indicate partial fulfilment of the quality statement. Red boxes indicate the tool does not include any measurement of this quality statement. AMDD=Averting Maternal Death and Disability. RBF=Impact Evaluation Toolkit for Results Based Financing in Health. SARA=Service Availability and Readiness Assessment. SDI=Service Delivery Indicator. SPA=Service Provision Assessment. *These proportions reflect whether at least one of the subcomponents of the statement was measured, unless the quality statement expressly states requirement of all subcomponents.

An overview of each of the selected tools is included in the table , showing the purpose, number of countries covered, data collection methods, sampling, and specific maternal and newborn care data collected. Most of the data these tools collect on maternal and newborn care are related to medical and equipment supply, especially for emergency obstetric care. Although most of these tools are designed for repeated data collection (every 1–2 years for SARA, every 5 years for SPA), administration in practice is often less frequent, based on country and donor interest.

We used the SPA questionnaires administered in Malawi and Nepal (because Malawi included a module on observation during labour and delivery, and Nepal a module on exit interviews with postpartum women), the standard SARA forms, the standard AMDD tool, the SDI questionnaire used in Tanzania, and the RBF questionnaire used in Cameroon. We looked at facility-based surveys (SPA, SARA, AMDD, SDI, RBF), health-care provider interviews (SPA, AMDD, SDI, RBF), and—if available—direct patient observation during labour and delivery (SPA) and post-partum patient interviews (SPA). We included in our analysis a total of 274 input and process quality measures. Figure 2 depicts the standards, statements, and measures under review. The appendix (pp 1–96) gives a complete listing of all 274 measures under review and the operationalisation of each of them.

Discussion

Our analysis included five survey tools and the proportion of measures covered ranged from 62% (SPA, including delivery observation and post-partum exit interview) to 12% (SDI). Although the broadest tool addressed parts of each of the 31 quality statements, 68 of 274 input and process quality measures were not measured at all. Across all tools, 206 (75%) of 274 input and process indicators associated with the standards were captured. This work provides the first assessment and comparison of the ability of five widely used facility assessment tools to measure quality of maternal and newborn care as per WHO’s standards.

Sprockett A

Review of quality assessment tools for family planning programmes in low- and middle-income countries.

, 15

Kruk ME
Gage AD
Arsenault C
et al.

High-quality health systems in the Sustainable Development Goals era: time for a revolution.

, 18

Gergen J
Josephson E
Coe M
Ski S
Madhavan S
Bauhoff S

Quality of care in performance-based financing: how it is incorporated in 32 programs across 28 countries.

Leslie HH
Sun Z
Kruk ME

Association between infrastructure and observed quality of care in 4 healthcare services: a cross-sectional study of 4,300 facilities in 8 countries.

Leslie HH
Zhou X
Spiegelman D
Kruk ME

Health system measurement: Harnessing machine learning to advance global health.

Bohren MA
Titiloye MA
Kyaddondo D
et al.

Defining quality of care during childbirth from the perspectives of Nigerian and Ugandan women: a qualitative study.

Bohren MA
Vogel JP
Fawole B
et al.

Methodological development of tools to measure how women are treated during facility-based childbirth in four countries: labor observation and community survey.

Completeness of available measurement was highest for standard 3, addressing resources for referral, and standard 8 on essential physical resources for care. This finding supports previous assessments identifying a preponderance of indicators on inputs to care in health facility assessment tools.Input measures provide incomplete insight on the quality of care provided;research has suggested that analytical approaches to shorten measurement of health facility readiness are available, but that clear guidance from standards is needed to inform this work.Conversely, standards on health information systems (HIS) and patient experience were much less likely to be measured: only two tools provided a partial assessment of HIS and three captured no relevant information at all. Validation of measures to assess data use, particularly timely use to inform clinical care, is an important area for implementation research. Tools or modules that are better able to measure perceived experience of care, such as respect and dignity, are also needed; although resource-intensive, capturing the voice of women is crucial to improving quality of care.WHO publishedthe development of two tools (labour observation and community survey) to measure mistreatment during childbirth; it is also a call for researchers and health-care providers to use and implement these tools to inform the development of more woman-centred, respectful maternity health-care services.

Sprockett A

Review of quality assessment tools for family planning programmes in low- and middle-income countries.

, 12

Nickerson JW
Adams O
Attaran A
Hatcher-Roberts J
Tugwell P

Monitoring the ability to deliver care in low- and middle-income countries: a systematic review of health facility assessment tools.

, 23

Moxon SG
Guenther T
Gabrysch S
et al.

Service readiness for inpatient care of small and sick newborns: what do we need and what can we measure now?.

, 24

Sheffel A
Karp C
Creanga AA

Use of Service Provision Assessments and Service Availability and Readiness Assessments for monitoring quality of maternal and newborn health services in low-income and middle-income countries.

Kruk ME
Gage AD
Arsenault C
et al.

High-quality health systems in the Sustainable Development Goals era: time for a revolution.

Madaj B
Smith H
Mathai M
Roos N
van der Broek N

Developing global indicators for quality of maternal and newborn care: a feasibility assessment.

Many of the shortcomings of the facility assessment tools identified in this analysis are in line with those identified in other studies, the main one being that none of the existing tools is able to effectively and comprehensively measure quality of sexual and reproductive health care, inclusive of maternal and newborn health, in addition to the fact that these tools are somewhat limited to assessing input measures.These shortcomings pose a challenge given the costly endeavour taken on by each of these globally used tools; if they are not adequately measuring different domains of quality, they are not measuring essential components of health care.Similarly, if they are measuring items that do not correspond to these standards for quality care, then resources might be spent inefficiently. Other researchers have attempted to assess the feasibility of measuring quality based on previous guidance from WHO using facility-level data in select countries.Their findings, consistent with ours, show the limitations of the currently available instruments used in global health.

Rosado C
Molla YB
Strachan M
Rawlins B
Khadka N

Review of newborn indicators in maternal and child survival program-supported countries. USAID: Washington DC.

Moxon SG
Guenther T
Gabrysch S
et al.

Service readiness for inpatient care of small and sick newborns: what do we need and what can we measure now?.

Banke-Thomas A
Wright K
Sonoiki O
et al.

Assessing emergency obstetric care provision in low- and middle-income countries: a systematic review of the application of global guidelines.

Diamond-Smith N
Sudhinaraset M
Montagu D

Clinical and perceived quality of care for maternal, neonatal and antenatal care in Kenya and Namibia: the service provision assessment.

Several other studies looked at the ability of globally used tools to measure quality of care. A review of maternal and child survival programmes supported by USAIDalso highlighted the misalignment between what is being measured and what global standards and metrics require. The authors emphasise the need for better guidance in operationalising global standards at the country level. A reviewof three of the tools included in our analysis, looking at their ability of capturing quality of newborn health, found similar deficiencies. A systematic reviewof emergency obstetric care identified important limitations in measurement methods included in existing guidelines. Another study,looking at emergency obstetric care in two African countries with the SPA survey, found that women’s perception of quality did not always correspond to what the tools attempted to measure, highlighting the difficulty in measuring quality of care with the currently available tools.

The gap between existing tools and the standards underscores the need for efficiency in identification of prioritised measures mapping to these standards. It further calls for harmonisation of global tools to ensure that core data can be captured and used for health system assessment and improvement. This same approach would also be useful for guidance on outcome measures, which are not collected consistently across these tools but are crucial to assess effect. In the short term, harmonisation across tools in the content and details of measurement would facilitate baseline assessment and comparison across facilities and surveys while reducing duplicative data collection and underused data.

Kruk ME
Gage AD
Arsenault C
et al.

High-quality health systems in the Sustainable Development Goals era: time for a revolution.

WHO

Quality, equity, dignity: a network for improving quality of care for maternal, newborn and child health. Monitoring framework. Geneva: World Health Organization.29

WHO

What is the Quality of Care Network?.

The priority for improving measurement is clarity, coordination, and action rather than yet more measurement for the sake of measurement.Although the WHO standards offer good direction, the challenge of identifying which are the most crucial or relevant to facilities and countries remains, signalling the need for guidance as to what can be considered a minimum standard for the purposes of measurement. Efforts are underway by the QED Network and the Health Data Collaborative to refine standards for more efficient measurement. The QED Network, for example, is leading efforts in monitoring quality of care through a set of common measures that would be collected at a national level and a catalogue of performance and quality improvement measures collected at the district and facility level using more granular data, all identified from the measures associated with the WHO standards.We welcome these efforts and hope our analysis informs this type of work.

Our study has some limitations. First, we matched items in the tools with quality measures when at least one component was present, which might be considered a low requirement in many cases (eg, for cases in which a list of essential medicines was promoted and the tools only assessed a fraction of these listed medicines). Second, existence of measures was equated with fulfilment, which we know is a potentially strong assumption. Even when the measure exists, it is not always used to assess the correct treatment or service—eg, availability of oxytocin as an input measure does not assess whether its use was correctly timed for preventing or treating post-partum haemorrhage. Third, as no clear guidance is available on how to prioritise among the measures provided, we considered each measure within a statement equally in calculating the proportion of measures collected; the percentages shown are not intended to capture the relative value of the information assessed. Finally, although these surveys are designed to provide a common instrument applicable across multiple countries, the specific modules or tools selected can vary by country, meaning that our findings are specific to the versions of the tools analysed here, which are the most comprehensive versions used to date.

Through our analysis of the largest service readiness assessments using the most up-to-date guidance set forth by WHO in 2016, we identified potentially valuable research and policy implications. First, we call for better guidance and simplified recommendations to operationalise and prioritise these measures for large-scale surveys, such as the ones assessed in this analysis. Additionally, we call for the different agencies leading multicountry surveys to coordinate better to reduce redundancy and inconsistencies, while also allowing for a more precise, standardised, and routine approach to measurement of quality of maternal newborn health care, including a more accessible way in which countries can understand and use these data, and a mechanism through which tools can easily adapt to emerging evidence. Research is particularly needed on measurement of actionable information systems and patient experience of care. Lastly, our analysis aims to encourage countries, organisations, and researchers committed to ensuring that quality standards are met to use the best available tools, to seek new measures for the gaps identified, and to strengthen data use for quality improvement at the district and facility levels. These implications are especially important given the urgency to improve quality of care.

Existing facility assessment tools provide a valuable way to assess quality of maternal and newborn care as one element within a national measurement toolkit. Some of the WHO standards are more widely covered in these tools than others. Better guidance on a minimum set of priority measures at the global level and increased harmonisation between existing tools, and the development and validation of new and improved modules to address important gaps, are urgent priorities. Improved guidance, harmonisation, and modules would be a crucial step towards improving the quality of maternal and newborn health and achieving the SDGs by 2030.