The escalating complexities of health care delivery systems and changing sociopolitical market forces compel the profession to confront the issues of promoting documentation of initial competence of students and new graduates and the continuing competence of experienced and certified practitioners. Educators, administrators, regulators, and association leaders have begun to discuss and debate these problems and the consequences of action and non-action. This article provides an overview, context and rationale for competency-based education and practice and selected methods relevant to performance assessment and related contemporary issues. The author suggests a structure for developing alternative that are flexible, efficient and effective across the broad spectrum of nursing roles and responsibilities.
Key words: assessment, competence, competency-based, nursing education, outcomes, performance evaluation
The escalating complexities of health care delivery systems and changing sociopolitical market forces compel the profession to confront the issues of promoting documentation of initial competence of students and new graduates and the continuing competence of experienced and certified practitioners. Educators, administrators, regulators, and association leaders have begun to discuss and debate these problems and the consequences of action and non-action. This article provides an overview, context and rationale for competency-based education and practice and selected methods relevant to performance assessment and related contemporary issues. The author suggests a structure for developing alternatives that are flexible, efficient and effective across the broad spectrum of nursing roles and responsibilities.
Redesigning Expectations for Initial and Continuing Competence For Contemporary Nursing Practice
Competence, assessment of competence, documentation of competent performance, and mandatory continuing competence are controversial topics receiving increasing emphasis in current manuscripts and conferences of all types and at all levels. This focus on competence is directed both to those in academic and service sectors, across the spectrum from small colleges and enterprises to universities and mega-industries, and from local, to national, to global endeavors. Concerns about competence are not new; with increasing urgency, however, perceptions and expectations related to competence are being reconceptualized and redesigned to be more responsive to escalating changes and associated consequences for education, health care and other industries. These broad societal changes are explored, for example, in recent publications by Huston & Fox (1998), Lenburg (1999a), and McBride (1999), who urge nurse educators to integrate them into curricula content and learning experiences.
This article presents a broad overview of some relevant factors in the historical and contemporary context of the movement to promote, or require, initial and continuing competence; it also suggests potential rationales for mandating competence, and is focused primarily on faculty, students, and practitioners of nursing. It highlights some relevant initiatives taken in the past and some new possibilities for redesigning expectations for competence in both education and service environments. This background context is designed to prepare readers for the more specific subsequent articles in this issue.
New expectations for contemporary practice competencies are emerging due to broad societal changes. Managed care, repeated mergers of once independent health care institutions, and related factors have precipitated significant reorganization of which services that can be provided, how and by whom, how much time can be spent providing them, how they are paid for, and who makes these critical decisions. Waves of new knowledge, technology and related requirements overtake previous ones before they can be absorbed and put into use. These changes are causing a continual transformation of practice environments and the priority competencies required, how they should be learned, and how individuals should be held accountable for competent performance. Other factors also have precipitated higher expectations of competence: the public is more savvy and assertive about the competence of providers, the consequences of incompetence are extraordinarily dire and expensive, and changes in the global economy demand more productivity and efficiency.
These changes require concomitant modifications in policies, position descriptions, and evaluation of skills of essentially all personnel, including professional nurses. This, in turn, has necessitated that employers seek nurses (and other providers) who are effective, flexible and creative critical thinkers, collaborative team players, safe and effective practitioners, and efficient managers in uncertain and demanding circumstances. These expectations present quite a different set of competencies and consequences from those of the past, for both academic and service sectors.
Employers are experiencing a widening gulf between the competencies required for practice and those new graduates learned in their education programs. Employers report the need to spend an increasing amount of time and resources to orient and teach new nurses the competencies required in today's workplace. This situation has precipitated an increased emphasis on initial and continuing performance competencies of new graduates, even though this is an unwelcome added expense. For more than a decade del Bueno and colleagues (Anthony & del Bueno, 1993; del Bueno, 1995a; del Bueno, 1995b) have challenged hospital-based nurse administrators and managers to implement competency-based practice models, such as their Performance-Based Development System. Specific performance examinations and nationally standardized tests are being used more frequently to document the readiness of new employees for full practice responsibilities.
Competent practice ... is more essential and mandatory than ever, as employers, consumers, insurance companies, and health care conglomerates expect more and different skills than in the past ...
Competent practice, therefore, is more essential and mandatory than ever, as employers, consumers, insurance companies, and health care conglomerates expect more and different skills than in the past, which often goes far beyond those the faculty in schools of nursing currently emphasize, expect, or reward. The gap in expectations precipitates the urgent need for increased mutual respect, shared accountability and collaborative partnerships between those in nursing education and service more than ever before. Instead of each sector independently focusing just on its own responsibilities, which falls short of the collective need, educators and service leaders are urged to work collaboratively to establish core practice competencies and assessment methods and then hold those engaged in health care delivery accountable for them.
Historical and Contemporary Contexts
In recent years, a critical mass of society's leaders, policy makers, gatekeepers, and stakeholders in health care and other disciplines has become increasingly concerned and ready to engage in serious debate about competency issues, in the interest of the public good and global competition. In fact, some essential aspects of competency reform were initiated nationally in the 1970s and continued to evolve progressively through the 1980s and 1990s, especially the standards for health care organizations and academic institutions. An early classic study, conducted by Grant and Associates (1979) and supported by the Fund for Improvement of Post-Secondary Education, was reported in the book On Competence: A Critical Analysis of Competence-Based Reforms In Higher Education. This study documents concerns about, and recommendations for, reforms in education to promote competence substantiated by evidence. Interestingly, it was published 20 years ago, but is just as applicable today.
Perhaps the most immediately plausible explanation for [the] concern over competence is that levels of competence once thought at least tolerable for a society moving at a slower pace are quite inadequate for a society whose internal management is growing steadily more complicated. Consider, too, the amount of knowledge that individuals must master to conduct themselves competently and to have the self-confidence that comes from believing themselves competent.
Today, in fact, belief in one's own competence is no longer enough, and a demand for demonstrated competence now motivates much of education. This demand underlies much of the insistence on continuing education and even relicensure in the major professions, and it aims both to uncover cases of self-delusion about one's competence and to prevent apathetic resignation about maintaining one's competence. There is more to know in general; there is more to know about the specialty on which one focuses; and there is more continuous production of new knowledge that requires sifting, even if much of it must be discarded as unproven or redundant (pp. 19-20).
Grant's definition of competency-based education is relevant and very useful in both academic and non-academic programs: It is " a form of education that derives a curriculum from an analysis of a prospective or actual role in modern society and that attempts to certify student progress on the basis of demonstrated performance in some or all aspects of that role. Theoretically, such demonstrations of competence are independent of time served in formal education settings" (Grant, 1979, 6).
Later, at the 1986 conference of National Governors' Association, actions were taken to change the meaning and methods of accreditation and accountability. The emphasis was shifted from reporting resources (number of doctorates and library holdings) to documenting student learning, from instructional improvement to institutional accountability, and from the orientation of student completion to graduate competence. Ewell (1987, 1991, 1994) in his role with the Education Commission of the States, traces the development, implementation, issues, and consequences of these changes. Outcomes and outcomes assessment began to refocus the purpose and methods of learning and practice. Spangehl (1987) explored concerns about assessment and reminded academics that expected benefits require time, patience and leadership.
The 1990 September/October issue of Change, journal of the American Association of Higher Education, focused on the competency movement, with special emphasis on assessment of competence (Hutchins & Marchese, 1990). Subsequently, concepts, philosophy and methods of continuous quality improvement (TQM, CQI, QI) began to be integrated into education as it was in business and industry. The Baldridge Award, prestigious in the commercial world, began to be sought by academics as a testimony of effective quality education (Seymour, 1994). Marchese's (1991) summary of six key components of quality improvement and their application to education are very useful.
These reforms also focused on basic issues of faculty reform, including teaching-learning methods and responsible assessment of students' actual competence. The challenge was clear: teachers needed to shift intentionally from traditional patterns of giving information to actively engaging students in real-time, collaborative learning focused on developing and expanding such competencies as assessment, creative and critical thinking, communication, and leadership. The roles and expectations of teachers and students had to change substantially, from lectures and traditional content to strategies that promote the competence and confidence graduates are expected to use in the evolving workplace.
During the 1990s, terms related to outcomes were introduced more specifically into accreditation criteria for academic and health care institutions. The educational councils of the National League for Nursing (NLN, 1992) revised program evaluation to include outcome criteria. Interestingly, the practice-related competencies students and graduates must achieve are not even mentioned in these documents. Gradually, language associated with competence crept into official documents and was used more frequently by nurse leaders, regulatory and accrediting authorities, such as the American Nurses Association (ANA), 1991; NLN, 1993; and the American Association of Colleges of Nursing (AACN), 1998. NLN also initiated pre-convention conferences focused on competency assessment (Lenburg, 1991). The criteria for accreditation, with more emphasis on competence, were changed further by the creation of the NLN Accreditation Commission (NLNAC) in response to recommendations from the US Department of Education in 1996. Other changes for BSN and higher degree programs evolved in the late 1990s through the newly created accreditation affiliate of the American Association of Colleges of Nursing (AACN). The revised Essentials of Baccalaureate Nursing Education Practice (AACN, 1998) and web pages for these organizations contain the most current criteria and guidelines: www.nln.org and www.aacn.nche.edu.
Other influential leaders and groups in the health care industry, such as the Pew Health Professions Commission (1995, 1998), specifically outlined the need for nursing and health care providers to find ways to validate continuing competence. To some extent, medical students and physicians, teachers, and other groups also have begun the process of incorporating concepts of outcomes, competencies, and performance assessment into the rubric of expected standards. For example, four medical school consortia recently received substantial grants from the Josiah Macy Jr. Foundation (Report, 1996, 22-25) to establish clinical skills assessment programs. For several years, the Joint Commission on the Accreditation of health care Organizations (JCAHO) has required specific validation of competence of health care providers for institutional accreditation (JCAHO, 1996).
This increasing emphasis on competence is being applied to the entire nation's workforce, as manifested by the high-profile White House Summit held on January 12, 1999 in Washington, DC.
At the request of the American Nurses Association (ANA), Lenburg was asked to be the representative for nursing. She participated and briefly presented during breakout sessions some relevant core competencies clearly applicable across the broad spectrum of education and practice. The unmistakable message from the Summit is that leaders at all levels and in all influential groups throughout the nation (including those in education, nursing and health care) need to refocus their perspectives on the multiple complex dimensions of contemporary work-related competencies and accountability. This includes changes in education and training programs and the performance expectations of current and would-be employees, administrators and managers, educators, and those responsible for accreditation and regulation. Documented competence of the workforce, individually and collectively, is a growing necessity, not a luxury endorsed by those with extra resources.
Beginnings of the Paradigm Shift to Competence
During the 1970's and 1980's, two colleges emerged as pioneers by responding to the challenge and developed competency-based programs in liberal arts, nursing, business and other majors, albeit using very different approaches. The New York Regents External Degree Programs (now named Regents College) and Alverno College provide seminal examples of contemporary competency-based education that requires verification of predetermined outcomes and performance expectations using specific and psychometrically-sound assessment methods. Their nursing programs paved the way for other schools to adopt and adapt methods of combining traditional cognitive norm-referenced testing with specifically designed actual and simulation performance examinations (criterion-referenced) to validate competencies required for graduation and nursing practice (Lenburg, 1979, 1983, 1984, 1990; Nolan, 1998; Alverno College Faculty, 1994). Their web pages provide extensive current information and references: www.regents.edu and www.alverno.edu.
Alverno College required college-wide emphasis on designated competencies and core values for all students in the college regardless of major. Its faculty continues to provide national leadership in faculty development, offering annual extensive seminars and workshops related to competencies and assessment methods. Alverno faculty are frequent contributors at conferences and in publications, with considerable focus on their program of evaluation research and positive educational outcomes. The Alverno web page contains multiple and useful resources and links.
Between 1972 and 1975, the faculty and professional nursing staff of the New York Regents External Degree Nursing Programs conceived and designed the first associate degree nursing (ASN) program based entirely on assessment of knowledge and skills. After it was completed and accredited in1975, they began development of the BSN degree, through additional funding from the W.K. Kellogg Foundation. After considerable effort it also was accredited by NLN in 1981. (Approximately 14 other degree programs also were developed, all under the auspice of The University of the State of New York-USNY.) The overall nursing faculty (official curriculum and policy makers), various faculty subcommittees, and the staff of nurse educators and psychometricians developed the comprehensive series of nationally standardized norm-referenced tests and the innovative criterion-referenced clinical performance examinations, based on actual and simulation methods to verify essential competencies.
During the 1970's and 1980's ASN and BSN programs that included both performance and written examinations were implemented. Additionally, a national network of Regional Performance Assessment Centers and highly structured orientation and training programs required for all MSN-prepared clinical examiners were implemented (Lenburg, 1979, 1983, 1984). Regular monitoring and update sessions were required to insure that all program-related policies, practices, and assessment methods were rigorously maintained. The quality and efficiency of the nursing examinations and related study materials were revised periodically based on a program of systematic ongoing research (Lenburg, 1990). These innovations are cited to illustrate some methods by which required practice competencies can be defined, specifically evaluated, and revised through ongoing research, even in a national external outreach program.
As a result of these years of creative and successful developments, and the increasing emphasis on documenting practice competencies, education and service leaders across the country (and abroad) became more receptive to their own needs to make substantive change. Throughout the 1990s presentations related to competency-based education and performance assessment were made at a wide array of local, regional, and international conferences. These included research conferences, faculty development conferences, nursing and other association and organization conventions, and the International Congress of Nurses (ICN). These ideas also were adapted and presented to allied health educator groups and to the American Academy of Family Physicians at a special conference on competency assessment (Lenburg, 1994). Another indicator of growing interest in competency education is reflected in the multiple short and long term consultations leaders in the competency-based movement have been asked to conduct with nursing faculties across the country (Luttrell, et al, 1999; Redman, et al, this issue). The rapidly increasing number of publications, conferences, and assessment instruments emphasize the remarkable paradigm shift now taking place.
Initiatives to Promote Continuing Competence
In his keynote address at the First NLN Education Summit, held in Chicago, September 1998, O'Neil reviewed current health care trends and the challenges posed to the nursing profession by the Pew Commission. He urged the profession to "aggressively communicate its strengths, and bust out of the box and begin anew." He also stated his conviction that "the certification and licensing of health professionals of the future must be tied to demonstration of continuing competence" (O'Neil, 1999, p. 13). He recommended that nursing education and nursing practice must become more reintegrated, in keeping with changes in health care delivery and societal changes. His perspectives merit thoughtful reflection, as do responses of leaders of NLN educational councils whose comments also were published in Nursing and Health Care Perspectives. Since the Summit, as reported in the NLN 1998 Annual Report, the Board of Governors has initiated a project entitled Transforming the Nursing Education Landscape, to focus of these issues.
In this author's opinion, O'Neil's observations are patently relevant. Nurse educators, whether responsible for classroom or clinical learning, are most effective when they engage in some form of clinical practice through which they adapt past skills to current circumstances and learn new ones now required in complex health care environments. Those who practice little, or not at all, are ill-equipped to promote competence among students in the ever-changing contemporary workplace. It is difficult enough for teachers to keep up with changes in the academic institution, curriculum and accreditation requirements, course content, logistics of clinical placements, and demands for scholarship. Similarly, it is difficult for practitioners to keep up with changes in standards, knowledge developments, research findings, and their applications in practice.
It is increasingly legitimate to ask: Are nurses as competent as they need to be?
These changes are powerful incentives for those in academic institutions and organizations, and regulatory agencies to design reform systems that insure initial and continuing competence of providers, with resulting benefits for consumers, students, providers, institutions, and the profession. As the gap between education (basic and advanced) and current practice expectations grows increasingly wider, it is more imperative than ever that those in the education and practice sectors formulate and implement functional and collaborative relationships that creatively integrate the best each has to offer. Working collaboratively, they can identify more effectively the essential competencies needed for practice and create the structures and methods to validate their achievement. The initial and continuing competence of students and graduates at all levels, depend on these shared responsibilities for effective and purposeful communication and respectful interactive partnerships.
The escalating demands for competence, coupled with limited resources and resistance to change, result in the need for designing flexible alternatives to fit diverse conditions. These considerations have sparked a movement involving many segments of the nursing profession to promote verified continuing competence for experienced nurses as well as novices, and to prepare for the consequences for initial and continuing licensure and certification. As one example, at the ANA 1998 national conference of the Nursing Organization Liaison Forum (NOLF) and National Federation of Specialty Nursing Organizations (NFSNO), Lenburg conducted a workshop on competency outcomes and performance assessment for the 200 leader- participants from more than 70 nursing specialty organizations. They all were grappling with multiple and complex issues related to competency outcomes, performance assessment methods, and continuing competence for recertification, in relation to their individual and collective organizations. The conference was designed to stimulate innovative and collaborative thinking and the creation of alternative strategies to promote continuing competence across the array of specialty groups. Another major conference on credentialing, scheduled for November 1999, is sponsored by American Nurses Credentialing Center (ANCC) and the Institute for Research, Education, and Consultation (IREC) in cooperation with 18 other national organizations involved in some aspect of competence and/or certification.
Various nursing specialty associations, regulators, and employers in many states recently have conducted similar exploratory discussions, debates, and planning conferences. Discussions related to the demonstration of continuing competence as a requirement for relicensure and/or certification, and/or continued employment are extraordinarily controversial and generate a wide span of reactions and opinions. They have resulted in considerable anxiety and conflict, as well as a coming together in many instances, between and among individual nurses and various agencies, organizations, and regulators. Clearly, substantive reforms in academic and continuing education and in credentialing requirements are needed to accommodate consumer protection, technological innovations, sociodemographic and market forces, and the rising incidence of litigation related to health care. The question is how to accomplish this goal most effectively while minimizing unacceptable and damaging consequences.
Redesigning Competencies and Expectations
Redesigning expectations for competent practice directly influences education and practice, faculty and students, administrators and staff, employers and employees, and regulators and associations. Various levels and categories of personnel and agencies expect their own special considerations. These variations make the process of identifying required competencies seem almost impossible, unless the participants are willing to step out of the box and apply four essential leadership characteristics: Creativity, flexibility, sensitivity and persistence. Implementing these attitudes and ways of responding to uncertainty and ambiguity make it possible to consider alternatives not previously imagined. They invite a comprehensive, reflective, exploratory approach to discovering or inventing options that transcend the personal and emotional attachment to habits and routine ways of thinking. Leaders in the profession who embrace these traits are more likely to invent effective and unifying structures that systematically hold its members and novices accountable for competent performance.
As a result of many years of study, experiences, and associated research, Lenburg developed such a comprehensive model to promote the transition from a traditional curriculum to one focused on Competency Outcomes and Performance Assessment (COPA). The COPA Model incorporates multiple components including philosophical, educational, and psychometric concepts; strategic transitional planning; faculty and student reorientation; curriculum, policy, and administrative considerations; and various types and methods of evaluation. It is applicable to the development, implementation and assessment of both initial and continuing competence. The perceptions of Grant and others cited earlier contributed to the identification of four essential questions that are basic to the framework used in the COPA Model. Responses to these questions contribute to the development of specific details unique to each particular purpose, group, level, or settings. The questions are:
- What are the essential competencies and outcomes for contemporary practice?
- What are the indicators that define those competencies?
- What are the most effective ways to learn those competencies? And,
- What are the most effective ways to document that learners and/or practitioners have achieved the required competencies (Lenburg, 1990, 1999b, Luttrell, et al, 1999)?
These questions and interrelated concepts for each one, formulate the organizing framework and processes that are the essential components of Lenburg's Model (1999b). The framework, concepts, interactive links between and among the components, the methods and processes are summarized in other articles in this issue. The eight core practice competencies in the Model, fundamental to all components, are:
- assessment and intervention,
- critical thinking,
- human caring relationships,
- leadership, and
- knowledge integration.
The COPA framework and methods potentially are applicable for documenting either initial or continuing competence. They apply to academic and clinical practice environments, to newly licensed nurses or to certified nurses who need to confirm continuing abilities to engage in competent and contemporary practice. The Model can be adapted and applied to a diverse array of environments (clinical, classroom, or labs), levels and types of nurses in different practice roles (clinical, educational, administrative, or others); it also applies to continuing education, academic or non-academic courses. A few examples of performance assessment methods include actual or simulation clinical performance examinations, portfolio development, specific projects, poster presentations, demonstrations of diverse abilities, peer or expert review, or a combination of methods.
Ultimately, all competency assessments pertain to one or more of the eight core practice competencies. Flexibility in the particular method, or the combination of methods, facilitates application to diverse settings and levels of practice. The essential requirements are the unequivocal specification of competency outcomes, specific indicators of competence, the most effective ways they can be learned, and the most effective methods by which competence will be assessed against the pre-established standards.
Ultimately, all competency assessments pertain to one or more of the eight core practice competencies.
Another example of a contemporary approach to promote competence among clinicians is illustrated in the recent work reported by the American Association of Critical Care Nurses Certification Corporation (AACN). Its leaders and members developed the Synergy Model as part of its reconceptualization of certified nursing practice. This Model is predicated on core concepts of the interface between patient needs and characteristics and nurses' competencies and the resulting influence on patient outcomes. Curley (1998), past chairman of the AACN Board, recently described the basics of the Model and subsequently AACN sent out a request to members of the American Academy of Nursing to provide scholarly critiques of the Model to foster collaboration in revisions and implementation. (Reprints available from: ivcReprint@aol.com) Other approaches are described in diverse nursing publications.
As the new millennium approaches, the nursing profession is struggling with increasing urgency to change its culture by creating and implementing the level of competence in professional practice required by consumers, policy-makers, employers, and other stakeholders. Documented competence is becoming essential -- not optional -- and is likely to become mandatory in the near future for initial and continuing licensure and certification, and perhaps even for employment. Because the issues that surround this movement are so complex and far-reaching in scope and consequences, a systematic, objective, consistent, and defensible framework and related processes must be created and implemented with rational and multi-faceted input.
This is the time for increased collaboration, creativity and flexibility among all segments of the profession and related other groups. It is the time for sharing experiences and works-in-progress, both the successful and the not-so-successful, that focus on similar overall issues, problems and needs. Mechanisms for integrating the most effective and efficient learning and assessment methods need to be implemented at local, regional and national levels, and among the broad array of constituencies. The American Nurses Association and multiple other organizations already have begun this process, as reported in this and other articles in this issue and elsewhere. This is the time for all members of the profession to engage in thoughtful reflection and creative and participatory problem solving, without becoming paralyzed by the magnitude of the efforts required to change the existing system to one that insures competent practice and quality of care for consumers.
Examples of Internet Sites Relevant to Competence
The following list of web page sites are intended as examples to locate useful information or links to information about competencies, assessment, and related requirements or regulations. Too many others exist, and change is too frequent to allow listing more sites.
www.aacn.nche.edu American Association of Colleges of Nursing
www.aahe.org American Association for Higher Education
www.accrediting-comm-nlnac.org NLN Accreditation Commission
www.alverno.edu Alverno College
www.gnacademy.org Global Network Academy; focus on distance learning
www.josseybass.com Assessment Update and other assessment resources
www.ncsbn.org National Council of State Boards of Nursing
www.nln.org National League for Nursing
www.nursce.com National Center for Continuing Education
www.NursingWorld.org American Nurses Association; OJIN & related sites
www.regents.edu Regents College: external nursing and other degrees
www.Nursingworld.org/ce ANA/ANF continuing education modules
Carrie B. Lenburg, EdD, RN, FAAN
Dr Lenburg, Loewenberg Chair of Excellence in the School of Nursing, University of Memphis from 1997-1999, worked with the nursing faculty to convert the BSN program to the competency outcomes and performance assessment model and methods. She also is engaged similarly with the nursing faculty of the University of Colorado Health Science Center to integrate the model to its range of four degree programs (BSN, MSN, ND, and PhD), and into all UC-SON Internet courses. She also is ongoing consultant to the newly developing BSN program at King College (Bristol TN), implementing the COPA Model from the outset.
Article published September 30, 1999
Alverno College Faculty. (1994) Student Assessment-as-Learning at Alverno College (Third Edition). Milwaukee: Alverno College.
American Association of Colleges of Nursing. (1998). The essentials of baccalaureate education for professional nursing practice. Washington, D.C.: Author.
American Nurses Association. (1991). Nursing's agenda for health care reform. Kansas City, MO: American Nurses Association.
Anthony, C. E., & del Bueno, D. (1993). A performance-based development system. Nursing Management, 24(6), 32-34.
Cohen, E.E. and Cesta, T.G. (Eds.). (1999, in progress). Case management: From concept to evaluation, 3rd edition. St Louis: Mosby.
Curley, M.A.Q. (1998). Patient-nurse synergy: Optimizing patients' outcomes. American Journal of Critical Care 7 (1), 64-72.
del Bueno, D. J. ( 1995a). Spotlight on...Ready, willing, able? Staff competence in workplace. Journal of Nursing Administration, 25(9), 14-16.
del Bueno, D. J. (1995b). Why can't new grads think like nurses? Nurse Educator, 19(4), 9-11.
Ewell, P.T. (1987). Assessment: Where are we? The implications of new state mandates. Change 19 (1), 23-28.
Ewell, P.T. (1991). Assessment and public accountability: Back to the future. Change 23(6), 12-17.
Ewell, P.T. (1994). A matter of integrity: Accountability and the future of self-regulation. Change 26(6), 24-29.
Grant and Associates. (1979). On competence: A critical analysis of competence-based reforms in higher education. San Francisco: Jossey-Bass.
Huston, C.J. & Fox, S. (1998). The changing health care market: Implications for nursing education in the coming decade. Nursing Outlook 46, 109-114.
Hutchins, P. and Marchese, T. (1990). Watching assessment: Questions, stories, prospects. Change 22 (5), 12-38. (See entire issue for related articles.)
Joint Commission on Accreditation of health care Organizations. (1996). Comprehensive accreditation manual for hospitals: The official handbook. Oakbrook, IL: Author.
Lenburg, C.B. (1979). Emphasis on evaluating outcomes: The New York Regents External Degree Program. Peabody Journal of Education 56, 212-221.
Lenburg, C.B. (1983). Expanding the options through the external degree and regional assessment centers. In B. Bullough, V. Bullough, & M.C. Soukup, editors. Nursing issues and strategies for the eighties. New York: Springer.
Lenburg, C.B. (1984). An update on Regents External Degree Program. Nursing Outlook, 32, 250-254. (early development of very nontraditional program)
Lenburg, C.B. (1990). Do external degree programs really work? Nursing Outlook, 36, 234-238.
Lenburg, C.B. (1991). Assessing the goals of nursing education: Issues and approaches to evaluation of outcomes. In M. Garbin (Ed.), Assessing education outcomes. New York: NLN Press.
Lenburg, C.B. (1994). Transformation to a competency-based performance assessment system. Proceedings from the special conference on the assessment of primary care competencies, June 4, 1994, Kansas City, MO.
Lenburg, C.B. (1999a). Contemporary issues in nursing education. In B. Cherry and S.R. Jacob (Eds.), Contemporary nursing: Issues, trends and management (p. 66-97).
Lenburg, C.B. (1999b, in process). The competency outcomes and performance assessment model applied to nursing case management systems. In E.L. Cohen & T.G. Cesta, Case management: From concept to evaluation, 3rd edition. St Louis: Mosby.
Luttrell, M.F., Lenburg, C.B., Scherubel, J.C., Jacob, S.R., & Koch, R.W. (1999). Redesigning a BSN curriculum: Competency outcomes for learning and performance assessment. Nursing and Health Care Perspectives, 20, 134-141.
Marchese, T. (1991). TQM reaches the academy. AAHE Bulletin 44 (3), 3-9.
McBride, A.B. (1999). Breakthroughs in nursing education: Looking back, looking forward. Nursing Outlook 47,114-119.
National League for Nursing. (1992). Criteria and guidelines for the evaluation of baccalaureate and higher degree programs in nursing. New York: NLN.
National League for Nursing. (1993). A vision for nursing education. NY: NLN.
Nolan, D.J. (1998). Regents College: The early years. Virginia Beach, VA: Donning Publishers.
O'Neil, E. (1999). The opportunity that is nursing. (Keynote paper in Special Report of NLN's Education Summit, with responses from education councils). Nursing and Health Care Perspectives 20 (1), 10-17.
Pew Health Professions Commission. (1995). Performing Health Care Workforce Regulation: Policy Considerations for the 21st Century. San Francisco: University of California San Francisco Center for the Health Professions.
Pew Health Professions Commission. (1998). Strengthening Consumer Protection: Priorities for Health Care Workforce Regulation. San Francisco: University of California San Francisco Center for the Health Professions.
Report of the Josiah Macy, Jr. Foundation: July 1, 1995 through June 30, 1996. (1996). New York: Josiah Macy, Jr. Foundation, 22-25.
Seymour, D. (1994). The Baldrige cometh. Change 26 (1), 16-27.
Spangehl, S.D. (1987). The push to assess: why it's feared and how to respond. Change 19 (1), 35-39.