The release of two reports from the National Institute of Medicine (1999, 2001) focused the nation's attention on the quality of hospital care and the problem of patient safety. These reports found that errors in hospital care were more common than the public had realized and recommended that health care delivery be reorganized to improve the quality of care. In response to the reports, federal and state governments, insurers, regulators, and health care providers are implementing health care-indicator initiatives to promote improvement in health care quality. Public reporting of quality indicators can help guide consumer choice among hospitals and assist businesses and insurers make purchasing and reimbursement decisions. However, most of the indicators included in public reporting initiatives reflect medical processes. Moreover, quality incentive programs for hospitals, generally known as pay-for-performance or value-based purchasing, are focused exclusively on physician-driven activities and medical outcomes (Centers for Medicare & Medicaid Services [CMS], 2007a). Under these programs, hospitals demonstrating good outcomes and efficient health care practices receive incentives, such as higher reimbursement rates, than hospitals with lesser performance. Recently, CMS announced that it will not provide reimbursement for care related to hospital-acquired complications (Centers for Medicare & Medicaid Services, 2007b).

A third report from the Institute of Medicine (2004) stated that, operationally, nurses have a critical role in securing patient safety. With 2.4 million practicing registered nurses (RNs) in the United States, nursing is the largest of the health care professions. Although nurse staffing and indicators of nursing-sensitive outcomes (patient outcomes that vary in response to changes in nurse staffing) are included in some public reporting initiatives, nursing indicators represent a small proportion of the pool of quality indicators. They are absent altogether from value-based purchasing initiatives. Because nurses are the most prevalent care providers in hospitals, the promotion of health care quality through public reporting and value-based purchasing cannot be comprehensive unless nursing's contributions are incorporated.

Mandating nurse-to-patient staffing ratios is one alternative public policy approach to promoting nursing quality that has been considered by several states and adopted by at least one. The focus on staffing ratios for nursing is consistent with research literature that shows an influence of nursing hours of care on various patient outcomes. However, use of staffing ratios may be an insufficient policy response as to date, literature has been limited in terms of the number of nursing workforce characteristics or attributes available for the study of quality of care. There may be other workforce characteristics that are as influential in promoting quality of care as nurse staffing ratios.

This article reviews what is known from previous nursing outcomes research and identifies gaps in the current state of knowledge. It then describes the contribution to outcomes research that can be made through the extensive data on nursing workforce characteristics available in the National Database of Nursing Quality Indicators (NDNQI). Next it presents findings from a NDNQI study describing the relationship of nursing workforce characteristics to patient fall rates and the rate of hospital-acquired pressure ulcers. The article concludes with a discussion of implications from this study for both nurse administrators and health policy officials involved in outcomes-based, quality-improvement initiatives.

Review of Previous Nursing Outcomes Studies

This section will present the growing body of evidence that describes the relationship between hospital nursing workforce attributes, such as nurse staffing levels, and patient outcomes. Because many of these studies have had significant limitations in conceptual framework, design, and nursing workforce attributes, this section will also discuss the limitations of these studies.

Previous Studies Relating to Workforce Characteristics and Patient Outcomes

The Agency for Healthcare Research and Quality (AHRQ) recently published a comprehensive and systematic review of the literature on the relationship between workforce characteristics, such as nursing hours and ratios, and patient outcomes (Kane, Shamliyan, Mueller, Duvai, & Witt, 2007). The AHRQ review identified 97 observational studies published between 1990 and 2006 and included 94 of these reports in a meta-analysis. This meta-analysis found strong and consistent evidence that higher registered nurse (RN) hours were related to lower patient mortality rates, lower rates of failure to rescue, and lower rates of hospital-acquired pneumonia. There was evidence that higher, direct care RN hours was related to shorter lengths of stay. Higher total nursing hours also were found to result in lower hospital mortality and failure to rescue rates, and in shorter lengths of stay. Based on fewer studies, the review found evidence that the prevalence of baccalaureate-prepared RNs was related to lower hospital mortality rates, that higher RN job satisfaction and satisfaction with workplace autonomy were related to lower hospital mortality rates, and that higher rates of nurse turnover were related to higher rates of patient falls. The conclusion of the meta-analysis was that higher nurse staffing was associated with better patient outcomes, but that the association was not necessarily causal. Further, the associations varied by service line and unit type.

A recent study by Needleman, Buerhaus, Stewart, Zelevinsky, and Soeren (2006) demonstrated the business case, i.e. the cost effectiveness, for increasing the proportion of nursing hours supplied by RNs, without increasing total nursing hours. The cost of increasing RN's proportion of nursing hours was less than the cost that would have resulted from adverse events, such as failure to rescue, urinary tract infections, hospital-acquired pneumonia, upper gastrointestinal bleeding, shock, and cardiac arrest. More than 90 percent of the cost savings was associated with reduced length of stay.

Limitations of Previous Studies

Significant gaps remain in nursing outcomes research literature. These gaps need to be addressed to strengthen the case for including nursing quality indicators in public reporting and value-based purchasing initiatives and to provide guidance to nurse executives regarding staffing models. Work is needed in the specification of theoretical or conceptual models, including the analysis of unit-level, rather than hospital-level, data. A number of authors have also noted the need to examine additional work-related, structure measures. Finally appropriate data sets for the analysis are also needed. These limitations are addressed in the following sections.

Conceptual framework limitations. Nursing outcomes research typically is based on Donabedian's (1988, 1992) conceptual framework, or derivations thereof, in which the structure of care influences the processes of care, and both in turn influence the outcomes of care. Because this framework supports many variations in actual model specification, many different organizational characteristics have been investigated. For example, different nursing workforce characteristics have been used as measures of the structure of hospital care; and the outcomes of a variety of different health conditions have been studied.

The Donabedian framework implies a hierarchical analysis model, in which patients are embedded in hospital units that have both structural characteristics and processes, and units are embedded within hospitals that have both structural characteristics and processes. Only a few studies, particularly studies published since the 1990s, had access to datasets that supported a hierarchical analysis. Failure to use a hierarchical model of analysis results in mis-estimation of the relationship between nursing workforce characteristics and patient outcomes. Harless & Mark (2006) demonstrated that relationships in many previous research studies may have been attenuated by having access only to hospital-level nurse staffing data and not unit level data.

It is important to note that some valuable studies have used the hospital service line (e.g., medical or surgical patients) as the unit of analysis (Needleman et al. 2001). In a different approach Whitman, Kim, Davidson, Wolf, and Wang (2002) have argued for the patient care unit, including unit specialty, as the unit of analysis because it is the operational level with the responsibility for care. A few authors have actually used the patient care unit as the unit of analysis (e.g., Blegen, Goode & Reed, 1998; Dunton, Gajewski, Taunton & Moore, 2004). Studies with data for service lines or unit types have demonstrated that specific aspects of the nursing workforce may be significant for some service lines or unit patient outcomes and not for others (e.g. Needleman et al., 2006).

Nursing workforce characteristic limitations. Although most previous research on the relationship between nursing workforce characteristics and patient outcomes has used nursing hours or patient-to-nurse ratios, a few studies have examined other characteristics, such as education, job satisfaction, or turnover. Work-related structure measures for which researchers have recommended further research include organizational factors, such as those affecting nursing processes (Mick & Mark, 2005), measures of hospital commitment to quality (Kane, et al., 2007), measures of longer-term organizational strategies and processes (Covaleski, 2005), and measures of hospital leadership (Bradley et al., 2006).

Data quality limitations. Additional measures of characteristics of the nursing workforce, such as measures of nursing processes, are needed, as are improvements in data quality, including larger sample size, reduced bias, and reduced measurement error. However, the nursing workforce should simultaneously be characterized in terms of supply (hours); knowledge, expertise, and experience; job satisfaction; and fitness (fatigue). Theoretically based measures of nursing processes, such as assessment, surveillance and monitoring, nursing interventions, communication with other health care providers, and patient education, should also be included in analyses.

The data available for nursing outcomes research have generally come from three types of sources. First, analysts have used large national data sets, such as hospital discharge abstracts or Medicaid costs reports, and matched those with nurse staffing data from selected states. Generally, such data sets are limited to information for the largest states and do not have data at the unit level. As a consequence, measures of the nursing workforce cannot distinguish between nurses in direct patient care or those involved in administrative or outpatient activities. While these data sets have information on a large number of patient outcomes, the nursing workforce indicators are quite limited. Second, analysts have obtained data from individual states or subsets of hospital surveys, administrative data, or hospital primary data collections. The California Nursing Outcomes Coalition Database and the Veterans Administration Nursing Outcomes Database are good examples of datasets that have unit-level information on both a variety of nursing workforce characteristic and patient outcomes for a subset of the nation's hospitals. Third, some analysts have collected data from convenience samples of a small number of hospitals to which they have access. It is questionable whether findings from these convenience-sample studies can be generalized to larger populations.

Finally, most studies are based on cross-sectional data sets. These data sets do not allow the analyst to study trends or estimate lagged effects. Understanding these trends or lagged effects could contribute to a causal understanding of the relationship between nursing indicators and patient outcomes.

In summary, advancing our knowledge of the relationship between nursing workforce attributes and patient outcomes will come from the use of data sets which support hierarchical analyses; additional attributes of the nursing workforce; unit-level data; and large, representative, longitudinal data sets.

NDNQI as a Data Resource for Nursing Outcomes Research

The American Nurses Association (ANA) established the National Database of Nursing Quality Indicators (NDNQI) in 1998 with the twin goals of (a) providing acute care hospitals with comparative information on nursing indicators that could be used in quality improvement projects, and (b) developing a database that could be used to examine the relationship between aspects of the nursing workforce and nursing-sensitive patient outcomes (National Database of Nursing Quality Indicators, n.d.). The NDNQI was developed in a way that addresses many of the limitations encountered by researchers working with other data sets as described above. The NDNQI will support hierarchical models of multiple nursing workforce indicators and patient outcomes. It is a large, longitudinal database, with unit-level data and national, although not representative, coverage. The next section will discuss strengths, limitations, and data collection methods of the NDNQI.

Strengths of NDNQI

NDNQI is a large database. Over 1,100 hospital report quarterly data on nursing workforce characteristics, including process measures, and patient outcomes. NDNQI also conducts an annual RN survey, which collects additional information on nursing workforce characteristics. In 2006, over 175,000 RNs responded to the survey.

NDNQI is a longitudinal database. Data were first reported to NDNQI for the third quarter of 1999 by 23 hospitals, and the number of reporting hospitals has grown steadily over the ensuing 31 quarters. The RN Survey data have been collected annually since 2002.

Data are collected for eight unit types: critical care, step down, medical, surgical, combined medical-surgical, rehabilitation, pediatric, and psychiatric. RN Survey data are collected for all hospital unit types, including outpatient and interventional units.

NDNQI contains many structure, process, and outcomes indicators. Measures of hospital structure include staffed-bed size, ownership, metropolitan/rural location, teaching status, and Magnet status. Measures of hospital structure include staffed-bed size, ownership, metropolitan/rural location, teaching status, and Magnet status. Measures of unit structure include unit type and over two dozen characteristics of the nursing workforce, including but not limited to: nursing hours per patient day (total, RN); skill mix; contract staff nursing hours; RN education; certification; years of experience in nursing; percent of RNs that float; shift type; intent to stay on the job; opinion on quality of care provided on the unit; RN satisfaction with RN to RN communication, with RN to MD communication, and with professional development; and RN age. Measures of nursing process include percent of patients with a risk assessment and, for those at risk, whether a prevention protocol was in place. Outcome measures include the patient fall rate, injury fall rate, hospital-acquired pressure ulcer rate, psychiatric patient injury assault rate, prevalence of pediatric IV infiltration, completeness of the pain assessment cycle for pediatric patients, and restraint prevalence.

Indicators included in NDNQI have good measurement properties. Data are collected on 8 of the 15 National Quality Forum Consensus measures, which have demonstrated reliability and validity. NDNQI conducts a reliability study on an indicator each year; the most recent study on pressure ulcers supported the reliability of NDNQI hospital identification and staging of pressure ulcers (Gajewski, Hart, Bergquist & Dunton, 2007; Hart, Bergquist, Gajewski, & Dunton, 2006). The reliability of satisfaction data from the RN survey is confirmed annually. The average response rate is 64 percent.

Limitations of NDNQI

Hospitals in every state and the District of Columbia participate in NDNQI, but participation is voluntary. Hospitals choose to participate in NDNQI because of their interest in the quality of nursing care and because they have the staff, data, and economic resources to participate. Therefore, NDNQI hospitals are a self-selected sample, and are not representative of all hospitals in the United States.

To better understand the limitations on representativeness of the NDNQI sample, NDNQI hospital characteristics were compared with data from the American Hospital Association's (AHA) Annual Survey. Due to differences in variable definitions and reference time period, however, the comparisons are not definitive. As with the NDNQI, the AHA database relies on self-reported data.

When compared to all hospitals in AHA's 2005 survey, NDNQI hospitals are significantly different on a number of characteristics (Table 1). Although the large sample sizes result in even minor differences achieving statistical significance, many of the characteristics are substantively different as well.

Hospitals of various sizes participate in NDNQI, with 12% having less than 100 staffed beds and 18% having more than 500 beds. On average, NDNQI hospitals were significantly larger than all hospitals in the AHA database. Over 80% of NDNQI hospitals were non-governmental, not-for profit facilities. Fewer NDNQI hospitals were for-profit than all hospitals in the AHA database. Approximately 15% of NDNQI hospitals were recognized as American Nurses Credentialing Center (ANCC) Magnet facilities, a higher percentage than for all AHA hospitals.

NDNQI Data Collection Methods

After a hospital joins the NDNQI, the facility is assisted by NDNQI staff in correctly classifying its units into unit types. After taking an on-line tutorial and passing quizzes on the key aspects of standardized data collection guidelines, hospital staff may enter their quarterly nurse staffing and patient outcomes data into web-based forms or submit their data through an XML batch upload. NDNQI quarterly data are collected via a secure website. Each hospital uses a code and password for access to the NDNQI system. Permissions for all hospital users except the site coordinator are reset quarterly. The website provides hospitals with data review tools, error reports, and immediate feedback on a number of common data entry errors. Submitted data are reviewed each quarter by NDNQI statisticians for outliers or significant changes across months in the quarter. Suspected errors are reviewed by hospitals and corrected. If suspected errors are not corrected, the data are deleted. Reports are downloaded from the NDNQI website in PDF and Excel files. Site coordinators in each facility are asked to review their reports for accuracy and completeness and notify NDNQI if they find errors, which are then corrected.

The RN survey data, the source of many nursing workforce characteristics, are collected primarily via a web interface. Each facility is guided through a two month preparation period and given materials, such as announcements and reminder cards, to promote a satisfactory response rate. Hospital survey coordinators have access to a live, web-based, unit-specific response rate, so they can tailor efforts to reach out to collect data from all units. From 2002 through 2006, a few hospitals (<50 per year) in which staff who had limited access to web-linked computers were allowed to collect survey data using paper surveys and Scantron sheets. However, this form of data collection activity was discontinued in 2007. Data are cleaned for illogical and out of range responses prior to report production. For privacy reasons, data are reported only for units with at least five responses and a 50% response rate. Survey reports are downloaded by hospitals via a secure web-connection and survey coordinators are asked to review their reports for apparent errors and report such to NDNQI.

A Study to Assess the Economic Value of Nursing Staff and RNs

A recent study was conducted using NDNQI data to assess the value of nurses in terms of averting patient falls and hospital-acquired pressure ulcers. The analysis file, the analytic approach, and the findings of this study will be described and discussed below. This study was the first NDNQI study to include the workforce characteristic of RN experience. All data were collected under protocols approved by the University of Kansas Medical Center's institutional review board.

Analysis File

Annualized measures were calculated from the quarterly data for the period from July 1, 2005, through June 30, 2006. RN characteristics from the RN survey were matched to quarterly data on staffing and outcomes on the basis of the quarter in which the survey month occurred. The hospital unit was the unit of analysis and included 1,610 critical care, step down, medical, surgical, combined medical-surgical, and rehabilitation units.

Analytic Approach

The analysis proceeded in two phases. First, an exploratory analysis using regression trees examined the relationship between several nursing workforce characteristics and the adverse events of patient falls and hospital-acquired pressure ulcers (HAPUs). The models included five hospital characteristics (staffed bed size, teaching status, metropolitan location, Magnet status, and ownership), six unit types, and 20 nursing workforce attributes. Regression trees sequentially identified independent variables most highly related to the dependent variable, in this case the fall rate or HAPU rate. The regression trees were used to narrow the number of indicators to be included in the formal modeling, comprising the second phase of the analysis. The formal modeling was conducted using mixed linear models, which are hierarchical and account for the dependencies among units within the same hospital. Each patient outcome was related to three hospital characteristics, six unit types, and eight nursing workforce characteristics (Table 2).

Findings: Evidence of the Value of Nursing from NDNQI Data

The results indicated that lower fall rates were related to higher total nursing hours (including RN, LPN/LVN, and unlicensed nursing assistants) per patient day, a higher percentage of nursing hours supplied by RNs, and a higher percentage of nurses on a unit with more than 10 years experience in nursing.

• For every increase of one hour in total nursing hours per patient day, fall rates were 1.9% lower.
• For every increase of 1 percentage point in the percent of nursing hours supplied by RNs, the fall rate was 0.7% lower.
• For every increase of a year in average RN experience, the fall rate was 1% lower.
• Fall rates were highest on rehabilitation units and lowest on critical care units.
• Fall rates in Magnet facilities were 10.3% lower than rates in non-Magnet facilities.

To promote the lowest fall rates, nurse managers could simultaneously optimize total nursing hours and both percentage of hours supplied by RNs and RNs with longer experience in nursing. For example, by increasing nursing hours from 6 to 7 hours per patient day, increasing the percentage of hours supplied by RNs from 60% to 70%, and increasing the average experience of RNs by 5 years, the fall rate would, on average, be reduced by 7.7%.

Lower HAPU rates were related to fewer total nursing hours per patient day, a higher percentage of hours supplied by RNs, and a higher percentage of RNs with 10 or more years of experience in nursing.

• For every increase of 1 hour in total nursing hours per patient day, HAPU rates were 4.4% higher. Although the analysis controlled for unit type, which is accepted as a proxy for patient acuity, this anomalous result may indicate inadequate risk adjustment or acuity adjustment. That is, net of hospital size, teaching status, Magnet status, and unit type, units with sicker patients at risk of pressure ulcers may have higher levels of nurse staffing.
• For every percentage point increase in the percentage of nursing hours supplied by RNs, HAPU rates were 0.7% lower.
• For every increase of a year in average RN experience, the HAPU rate was 1.9% lower.
• HAPU rates are highest on critical care units and lowest on the combined floor units, i.e. step down, medical, surgical, and combined medical-surgical units.

Nurse managers could promote the lowest HAPU rates if they would simultaneously increase the percentage of hours supplied by RNs from 60% to 70% and increase the average experience of RNs by 5 years. If managers arranged the staffing in this way, the HAPU rate could be reduced by an average of 11.4%.

Limitations

The findings from this study are limited in two ways. First, the results are generalizeable only to NDNQI facilities, which are self-selected for their interest in nursing quality indicators and their ability to participate in a national database. These facilities are larger, less likely to be for-profit, and more likely to be Magnet facilities than all hospitals in the AHA database. Second, the anomalous relationship between total nursing hours per patient day and HAPU rates suggests that more specific controls for patient acuity or risk should be included in the formal models.

Discussion of Study Implications

The findings from this analysis of the relationship between nursing workforce characteristics and the two patient outcomes of patient fall rates and HAPU rates not only confirmed, but also expanded, previous research insights regarding the importance of nurses in achieving safe patient outcomes. The significant relationship between nursing hours and skill mix and observed fall rates had been established previously. This analysis expanded the list of influential nursing workforce characteristics to include RN experience. Having a higher percentage of experienced RNs on the unit was related both to lower fall rates and lower HAPU rates. The effect sizes of experience were larger than those for skill mix. This particular finding provides salience to the argument that ...retaining experienced nurses on patient care units is paramount in the provision of high quality nursing care. retaining experienced nurses on patient care units is paramount in the provision of high quality nursing care. The significance of RN experience demonstrates the importance of looking beyond nursing hours or patient-to-nurse ratios in the promotion of safe patient outcomes.

The results of this study underscore the importance of including multiple characteristics of the nursing workforce in public reporting of the quality of nursing care. Nursing administrators and managers can apply the results of this study to promote quality of care by incorporating all three characteristics, i.e., nursing hours, skill mix, and experience in hiring and unit staffing decisions. In addition, businesses, insurers, and governments engaged in the design and implementation of value-based purchasing programs can use these findings by enhancing the proportion of nursing staff having greater skill and experience and by increasing the number of nursing hours.

 This study also emphasizes the importance of assessing and tracking the quality of nursing care at the patient care unit level.This study also emphasizes the importance of assessing and tracking the quality of nursing care at the patient care unit level. The odds of an adverse event occurring vary by unit type, reflecting differing patient populations. Future research is needed to determine if the relationships between nursing workforce characteristics and patient outcomes vary across unit type-patient outcome combinations.

Data from NDNQI will be a valuable tool for researchers interesting in nursing systems research. Data from NDNQI will be a valuable tool for researchers interested in nursing systems research. The large sample size, unit detail, longitudinal scope, and array of nursing workforce measures will support the examination of many and varied research questions.

Conclusion

Characteristics of the nursing workforce have been shown in this article to be important factors promoting the quality of safe and effective hospital care.To be comprehensive, quality improvement initiatives, such as public reporting and value-based purchasing, should incorporate nursing workforce measures.  To be comprehensive, quality improvement initiative...should incorporate nursing workforce measures. Previous research has demonstrated that nursing hours and RN hours of care are important factors. The study reported in this article has demonstrated that additional characteristics, such as years of experience, also are influential. The broad array of nursing workforce characteristics in the NDNQI database will support many future analyses of the role of nursing in achieving high quality patient care.

Authors

Nancy Dunton, PhD
E-mail: ndunton@kumc.edu

Dr. Nancy Dunton is an Associate Professor in the University of Kansas Medical Centers School of Nursing, with a joint appointment in the School of Medicine's Department of Health Policy and Management. She received a Ph.D. in Sociology from the University of Wisconsin-Madison. Dr. Dunton has been the director of the National Database of Nursing Quality Indicators since it was established in 1998. She has made numerous presentations on NDNQI to nursing groups across the United States and internationally. Dr. Dunton has more than 25 years of experience in helping organizations use outcome indicators in decision making. She has been the principal investigator on more than 20 health and social services research projects.

Byron Gajewski, PhD
E-mail: bgajewski@kumc.edu

Dr. Byron Gajewski is an Assistant Professor at the University of Kansas Medical Center, Schools of Nursing and Allied Health, and member of the Center for Biostatistics and Advanced Informatics. He has a PhD in Statistics from Texas A&M University-College Station. Dr. Gajewski has been an investigator on numerous health services and health behaviors research grants and contracts. He has conducted analyses on NDNQI data since 2002.

Susan Klaus, PhD, RN
E-mail: sklaus@kumc.edu

Dr. Susan Klaus received her PhD in Gerontology from the University of Kansas-Lawrence. She is an Instructor at the University of Kansas Medical Center, School of Nursing. She has been an investigator on the NDNQI project since 2002 and is the director of quarterly data processing

Belinda Pierson, MA
E-mail: belinda_pierson@yahoo.com

Belinda Pierson was a data analyst with the NDNQI project. She has a Master's degree in mathematics.

Centers for Medicare & Medicaid Services. (2007a). CMS Manual System, Pub 100-04 Medicare Claims processing, Change Request 5499.