Author: Sportsman, Susan; Schumacker, Randall E; Hamilton, Patti
Date published: July 1, 2011
JOINING FORCES, a school of nursing in a liberal arts university in north-central Texas, a nearby community college nursing program, and a 359-bed regional medical center obtained a Texas Higher Education Coordinating Board (THECB) grant in 2004 for the purpose of developing a Regional Simulation Center (RSC). The primary goal was to increase by 39 percent the total number of students admitted to both entry-level nursing programs. To enable faculty to pursue other activities, including teaching additional students, scenario-based practice with high-fidelity manikins took the place of traditional, hospitalbased clinical education with actual patients. Responsibilities for competency education and validation were delegated to four clinically strong, baccalaureate-prepared, laboratory mentors (Sportsman et al., 2009).
When the RSC was established, the Texas Board of Nursing (TBON) had no regulations regarding the substitution of clinical simulation for traditional clinical experience. However, it did require that nurse faculty hold a master's degree or higher. Because this project used BSN-prepared laboratory mentors in the simulation center, the project director requested and received a waiver to that rule. Based on the evaluation of the threeyear project, reported annually, the TBON in 2008 made the decision to remain silent on limitations to the number of hours Texas students can participate in simulation as part of their clinical experience.
This article reports on the evaluation for the grant as reported to the TBON. The purpose was to determine if participation in scenario-based, high-fidelity patient simulation in the RSC for a portion of the clinical experience traditionally supervised by faculty would influence student outcomes. The research questions for this descriptive, longitudinal study focused on several areas: students' sense of their own clinical competence; anxiety regarding school performance; attitudes about and interest in learning opportunities; motivation to learn; concentration during learning activities; and satisfaction with the clinical learning environment. Finally, the study evaluated the impact on seniors' graduating grade point average (GPA) and scores on a standardized exit examination.
Literature Review A report by del Bueno (2005) suggested that only 35 percent of new graduates meet entry expectations for clinical judgment when evaluated by the Problem Based Development System (PBDS), which uses video simulation to assess nurses' competence. According to del Bueno, a major cause of this failure is the emphasis in nursing programs on teaching greater amounts of content, rather than the application of knowledge. Assuming del Bueno's findings are correct, providing greater opportunity to apply clinical knowledge through the use of high-fidelity patient simulators (HFPS) may lead to greater competence among new graduates as reflected in traditional academic markers.
Multiple benefits of patient simulation have been identified in the literature, including: a) the ability to administer care to high-acuity, low-frequency cases, using actual clinical equipment without the potential to harm patients; b) skill development in procedures that require eye-hand coordination and ambidextrous maneuvers; c) the enhancement of confidence; and d) improved decision-making, teamwork, and communication in times of crisis (Jha, Duncan, & Bates, 2001; Medley & Horne, 2005; Patow, 2005). A number of researchers have used randomized controlled trials to determine if learners who train with simulators demonstrate greater knowledge and competence in the skill(s) being presented compared to learners using more traditional educational processes (Grantcharov et al., 2004; Steadman et al., 2006). In general, researchers have found that the use of simulation, particularly HFPS, provides learning outcomes that are at least as good, and perhaps better, than other learning methodologies, including practice in an actual patient situation.
Issenberg, McGaghie, Petrusa, Gordon, and Scalese (2005) reviewed and synthesized existing evidence in 109 studies from five literature databases to answer the question, "What are the features and uses of high-fidelity medical simulations that lead to most effective learning?" They found that published research, while generally weak, suggested that such simulations facilitate learning under the right conditions (providing feedback, repetitive practice, integration of simulation into curricula, a range of levels of difficulty, multiple learning strategies, controlled environment, individualized learning, defined outcomes, and simulator validity). Issenberg et al. concluded that while research regarding simulation in medical education needed improvement in terms of rigor and quality, high-fidelity simulations are educationally effective and simulation-based education complements other forms of education.
Weaver (2011) analyzed studies published since 1998 on the use of HFPS in undergraduate nursing education. The review found that this teaching-learning strategy benefits students in the areas of knowledge, value, realism, and learner satisfaction. Specifically, it was found that the hands-on learning and "real" decision-making implemented across the nursing curriculum add value to nursing curricula. Findings were mixed in terms of student confidence, transfer of knowledge, and stress reduction.
Despite the weakness of evaluative research, nurse educators have continued to use scenarios programmed for high-fidelity patient simulators as a substitution for clinical supervision in care settings to address limited clinical opportunities. Nehring and Lashley (2004) confirmed the increased use of HFPS through a survey of nursing programs in 18 universities and 16 community colleges. Their survey explored curricular use, faculty time and use, student opinions, evaluation, and other uses of HFPS in research and continuing education. It was found that community colleges used simulation more frequently than universities, and simulation was most often used to teach physical assessment and critical instances. A slight majority of the sample used this teaching-learning experience as part of required clinical experiences.
Recent attempts to determine the impact of the use of HFPS on nursing student performance have been inconclusive. A 2007 pilot study by Radhakrishnan, Roche, and Cunningham identified nursing clinical practice parameters influenced by simulation by evaluating the clinical performance of 12 senior second-degree BSN students in a complex two-patient, simulated assignment. They found that students who participated in patient simulation, in addition to their usual clinical training, had significantly higher scores than the group that only participated in usual clinical training on patient identification and on assessing vital signs. The remainder of their clinical performance was not significantly different from that of the control group.
The Regional Simulation Center Experience The RSC, located in a 3,410 square foot renovated nursing unit at the hospital partner, was equipped with six Laerdal SimManTM high-fidelity patient simulators (four adult and two infant/child manikins) and sufficient equipment to provide nursing care in a variety of scenarios. Faculty developed learning objectives for scenarios consistent with course objectives and worked with the lab mentors to develop realistic patient care scenarios.
The amount of clinical time (CT) students spent in the center varied, depending on the clinical objectives of the course. Courses in which students were assigned to the RSC included health assessment (100 percent of CT), fundamentals (50 percent of CT), two medical-surgical courses (each, 25 percent of CT), childbearing families (8 percent of CT), pediatrics (13 percent of CT), and the final capstone course (20 percent of CT). Students were not assigned to the RSC during their psychiatric experience or, in the case of BSN students, their community health clinical experience. Initially, faculty for first-year courses developed the majority of the simulation scenarios. As the project progressed and faculty and lab mentors increased their expertise, participation in simulation was included in a number of senior-level courses.
Over the three years of the project, both nursing programs, as well as the partner hospital, used the RSC for competency education/validation. In fact, the center posted more than 20,000 visits during the 34 months of the grant. BSN students represented 80 percent of the total visits; ADN students, 14 percent; and learners from the hospital, 6 percent. In addition, the number of students admitted to both nursing programs exceeded the goal set in the grant. As positive as those outcomes were, they provided no information regarding the impact of participation in scenario-based HFPS upon the metrics for student success used by academia.
The Evaluation THEORETICAL FRAMEWORK When identifying metrics of success for nursing graduates, variables that influence success must be considered. The Baramee and Blegen (2003) model was used as a theoretical framework for all of the research related to the project, including the research reported here. Baramee and Blegen studied variables that influenced graduating grade point average (GPA) and the perception of clinical competence among 885 students graduating from BSN programs in Thailand. Variables believed to influence GPA and sense of clinical competence were categorized into input variables (student background/characteristics, school/faculty characteristics) and process variables (use of learning strategies identified in the Learning and Study Skills Inventory [LASSI], e.g., anxiety, motivation, attitude).
Baramee and Blegen (2003) concluded that five variables produced a viable model to explain students' perceptions of their own clinical competence: perception of one's own hardiness, effort in studies, clinical learning environment, student-faculty relationship, and graduating GPA. Path analysis indicated that perception of one's own efforts, clinical learning environment, and graduation GPA had direct effects on students' perceptions of clinical competence. Sense of one's own hardiness had an indirect effect on perceptions of clinical competence through its impact on students' perceptions of their efforts, clinical learning environment, and student-faculty relationship.
DATA COLLECTION INSTRUMENTS Data were collected on outcome variables, such as scores on a standardized exit examination and GPA, students' perceptions of their clinical competence, and process variables, such as anxiety regarding school performance, attitudes about and interest in learning opportunities, motivation to learn, concentration during learning activities, and satisfaction with the clinical learning environment. Four data collection instruments, including a demographic data sheet, were used: the Clinical Competence Appraisal Scale (CCAS), four subscales from the Learning and Study Skills Inventory (LASSI), and the Clinical Learning Environment Scale (CLE). Exit examination scores and graduating GPA were used to quantitatively answer the research questions. The data collection instruments are described in Table 1 on following page.
DATA COLLECTION PROCESS Following approval by the university Human Subjects Review Committee and the opening of the RSC in January 2005, quantitative data were collected in January 2005, 2006, and 2007 from the baccalaureate (BSN) and associate degree (ADN) nursing students (juniors) who had successfully completed their Fundamentals course the previous fall. Data were also collected in April 2005, 2006, and 2007 from the BSN and ADN students (seniors) who were expected to graduate each May. The number of respondents during the six data collection periods totaled 895.
As student cohorts progressed through their course of study over the 34 months of the grant, each group had increasingly more experience in the RSC. For example, 2005 seniors had little or no experience in the RSC prior to data collection at the end of their course of study. In contrast, the 2006 seniors had three semesters of RSC experience before graduation, and the 2007 seniors had up to five semesters. Table 2 (on following page) illustrates the amount of simulation experienced by each cohort before data were collected in their junior and senior years.
The standardized exit examination scores for BSN and ADN seniors (2006 and 2007) were collected from each school during the summer of 2007. Scores for the 2005 seniors were not collected; the reporting mechanism for the examination changed between 2005 and 2006, making comparison difficult. Investigators also collected graduating GPA from student transcripts in the summer of 2007.
DATA ANALYSIS Student responses were compared across the 34 months of the grant to evaluate the impact of participation in scenario- based simulation on identified student metrics of academic success.
Results DEMOGRAPHIC DATA Females accounted for 85 percent of the sample. Sixty-three percent of the total respondents in both schools were between 19 and 29 years old; 25 percent were 30 to 39 years old; 10 percent were in their 40s; and 2 percent were age 50 or older. Slightly less than half had health care experience prior to entering their nursing program. The type of health care experience was not explicitly defined, but anecdotally it was known that respondents had professional experience as licensed practical/vocational nurses or as unlicensed assistive personnel. This variation in the definition of health care experiences may be a limiting factor in this research.
FINDINGS FOR RESEARCH QUESTION 1: What is the impact of participation in scenario-based simulation upon students' sense of their own clinical competence, their anxiety regarding school performance, attitude about and interest in learning opportunities, motivation to learn, concentration during learning activities, and satisfaction with the clinical learning environment? The combined data for both nursing programs were compared by year to assess the impact of participation in scenario-based simulation. The amount of simulation the student groups received increased each year of the grant. Table 2 illustrates this increase by semester of the project for each group.
Junior Responses Mean scores on the Psychomotor Skills Performance subscale (PSP) of the CCAS were compared by year, without consideration for the type of nursing program. Mean scores dropped over the three years of the study, from 44.41 to 40.71. Juniors in 2005, who had no simulation experience before completing the CCAS, rated their competence in psychomotor skills performance significantly higher (n = 482, f = 15.616, p = .0001) than 2006 and 2007 juniors, who had already participated in simulation.
Senior Responses Senior responses on all subscales of the CCAS, four subscales of the LASSI, and the CLE were compared for the three years of the grant. There was no significant difference in the mean score of the PSP or three CCAS subscales (Teaching/Collaboration, Planning/ Evaluation, Interpersonal Relations/Communications) among the three groups of seniors. The only CCAS subscale for which there was a significant difference (n = 308, f = 8.723, p = .0001) was Leadership.
For the LASSI, the mean scores on the Attitude, Concentration, and Motivation subscales remained essentially the same over time. The mean score of the Anxiety subscale for 2005 seniors was significantly higher, indicating lower anxiety, than for 2006 and 2007 seniors (n = 327, f = 4.249, p = .015). The mean scores on the CLE for 2005 seniors were significantly lower (n = 339, f = 7.633, p = .001) than for 2006 and 2007 seniors.
FINDINGS FOR RESEARCH QUESTION 2: What is the impact of participation in scenario-based simulation upon seniors' graduating grade point average and scores on the standardized exit examination? The mean scores on the standardized test, as well as the mean graduating GPA, of the three years of seniors were compared. No significant differences were found in either graduating GPA or test scores for these students.
Discussion Two findings from this study had the greatest impact on the organization of clinical experience for nursing students. First, despite the level of simulation the group experienced, there was no significant difference in mean exit examination scores between 2006 and 2007 seniors when the two types of programs were compared. Results of this examination have been correlated with success on the NCLEX-RN licensing exam (Lauchner, Newman, & Britt, 2006). Similarly, for the three groups of seniors from both schools, there was no significant difference in mean scores for the CCAS Psychomotor Skills Performance, Teaching/Collaboration, Planning/Evaluation, or Interpersonal Relations/ Communication subscales. Therefore, the substitution of scenario-based HFPS for time spent in clinical agencies (ranging from 8 percent, Childbearing Families, to 100 percent, Health Assessment) did not negatively affect performance on the standardized exit examination. Nor did it make a difference in seniors' perceptions of their own clinical competence. Both of these findings argue for the substitution of simulation experience for some traditional clinical experiences as a strategy for increasing student admissions when there are limited clinical experiences available to schools.
The significant difference in the mean scores on the PSP subscale for 2005 juniors compared to 2006 and 2007 juniors suggests that simulation early in students' clinical courses may give students a chance to implement psychomotor skills in a dynamic, simulated clinical situation. This is far different from performing a skill correctly in a static situation. The internal and external feedback that students receive in the simulation may provide a glimpse of their actual clinical skill, thereby reducing their perception of competence on psychomotor skills. By participating in simulated patient events in a safe environment, students may recognize the limits of their competence and the implications for patient safety. This could provide an impetus for greater commitment to learning for these students.
Differences were also found among the mean responses of seniors to one component of the CCAS. Seniors who had little or no experience in simulation perceived themselves to be less competent in leadership skills than those seniors who had participated in simulation throughout their course of study. During the teaching scenarios, small groups of students worked together to provide care, playing various roles including charge nurse, recorder, family member, and/or primary nurse. Perhaps this opportunity to role play enhanced their sense of their competence in leadership activities.
The anxiety of 2005 seniors, who had participated in little or no simulation, was significantly lower than that of 2006 and 2007 seniors. While investigators hoped that participation in simulation of high-risk patient situations would increase students' sense of competence, thereby reducing their anxiety, these experiences seemed to increase their anxiety. While this was not the desired outcome, the greater anxiety level may serve as a motivating influence as new graduates begin to practice.
The anxiety subscale of the LASSI was designed to assess respondents' sense of their anxiety in relation to current learning opportunities. The instrument does not adequately discriminate between anxiety related to educational experiences in the classroom and anxiety related to their clinical experiences. In addition, the respondents' general anxiety level was not evaluated. This is a limitation of the study. It is important to note that the evaluation of students in the scenario-based simulation was formative, rather than summative; the students did not receive a grade for each simulation. As a result, anxiety related to simulation grades was not a factor in the students' course evaluations.
Despite the elevated sense of anxiety in the 2006 and 2007 graduating seniors who had participated in simulations, the mean scores on the CLE for these students were significantly higher than the mean scores for the 2005 seniors. This suggests that simulation may positively influence students' perceptions of the clinical environment where they were assigned during the last semester of their course of study.
Several factors limit the generalizability of the study findings. The study was implemented in a RSC developed in partnership with a university, community college, and hospital, and located in a renovated nursing unit. Student experiences were directly supervised by BSN-prepared, nonfaculty lab mentors. While both of these characteristics positively influenced the success of the center, there are few centers in the United States with a similar organizational structure. This may reduce the external validity of the study.
There were also threats to internal validity. All students enrolled in both types of programs over the three years participated in the study. Juniors and seniors participating in the 2005 data collection were used as a control group because of their limited participation in simulation at that time. However, over the course of the three-year study, the operation of the RSC improved and the staff gained in expertise in developing scenario-based simulations. Thus, the student experience in 2006 and 2007 may be very different beyond the difference in the number of semesters in which students participated in simulation activities. For example, by the third year of the grant, the simulation scenarios were far more complex and the processes for managing the scenarios were more developed than in the first or second year. Similarly, other factors unrelated to the simulation center, including changes in faculty members and clinical agencies, may have affected student experience over time.
Recommendations for Further Study Because of threats to internal and external validity, this study should be replicated using random assignment and a control group that does not participate in the simulation experience. In addition, specific findings should be evaluated in more depth. For example, graduating seniors who participated in simulation activities were more anxious, as measured by the LASSI, than those who had not participated. A more thorough evaluation of the level of student anxiety throughout the course of study should be done to determine if participation in simulation, among all the stressors of nursing school, was the cause of the anxiety. A more sensitive instrument to measure both state and trait anxiety is strongly recommended.
The BSN students as an aggregate were assigned to the RSC far more than the ADN students were. The impact of the specific hours individual students frequently spent in the RSC upon their sense of competence and anxiety should be considered. Finally, this study explored the relationship between participation in simulation and scores on a standardized exit examination. This correlation should be extended to include NCLEX-RN scores.
In this study, scenario-based HFPS was delivered by BSN-prepared lab mentors. Although this is an excellent model for providing simulation to large numbers of students, maximizing the use of master's and doctorally prepared faculty for other teaching or scholarly activities, faculty members do monitor the simulation experience in most nursing programs. The difference in outcomes, if any, of these two approaches should be explored.
The study evaluated the impact of participation in scenario-based high-fidelity patient simulation upon students' perceptions of clinical competence. The impact of participation in HFPS on clinical competence, as evaluated by faculty and/or practicing nurses, should also be determined. In addition, while there is evidence from other research that participation in scenario-based HFPS is equally or more effective in supporting learning than other teaching methods, there is no evidence that this improved learning has an impact on the delivery of patient care. The impact of participation on patient safety needs evaluation.
As the use of HFPS grows in nursing education, this study provides initial evidence of its impact on the academic success of ADN and BSN students in north-central Texas. The findings point to the need for additional research as simulation becomes an integral component of many nursing students' experience.
Baramee, J., & Blegen, M. (2003). New graduate perception of clinical competence: Testing a causal model. International Journal of Nursing Studies, 40, 389-399.
del Bueno, D. A. (2005). Crisis in critical thinking. Nursing Education Perspectives, 26(5), 278-282. doi: 10.1043/1536-5026(2005)026[0278 :ACICT]2.0.CO;2
Dunn, S., & Burnett, P. (1995). The development of a clinical learning environment scale. Journal of Advanced Nursing, 22, 1166-1173.
Grantcharov, T. P., Kristiansen, V. B., Bendix, J., Bardram, L., Rosenberg, J., & Funch-Jensen, P. (2004). Randomized clinical trial of virtual reality simulation for laparoscopic skills training. British Journal of Surgery, 91(2), 146-150. doi:10.1002/bjs.4407
Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systemic review. Medical Teacher, 27(1), 10-28. doi: 101080/0/4215905000046924
Jha, A., Duncan, B., & Bates, D. (2001). Simulator-based training and patient safety. In Making health care safer: A critical analysis of patient safety practices (pp. 510-517). (Evidence Report/Technology Assessment: Number 43). Rockville, MD: Agency for Healthcare Research and Quality
Lauchner, K., Newman, M., & Britt, R. (2006). Predicting licensure success with a computerized comprehensive nursing exam: The HESI exit exam. CIN: Computers, Informatics, Nursing, 24 (Suppl 3), 4S-9S.
Medley, C., & Horne, C. (2005). Using simulation technology for undergraduate nursing education. Journal of Nursing Education, 44(1), 31-34.
Morrison, S., Adamson, C., Nibert, A., & Hsia, S. (2004). HESI exams: An overview of reliability and validity. CIN: Computers, Informatics, Nursing, 22(4), 220-226.
Nehring, W., & Lashley, F. (2004) Current use and opinions regarding human patient simulators in nursing education: An international survey. Nursing Education Perspectives, 25(5), 244-248. doi: 10.1043/1094-2831(2004)025/0244: CUAORH\2.0.CO;2
Patow, C. (2005, March-April). Advancing medical education and patient safety through simulation. Patient Safety and Quality Healthcare. Retrieved from www.psqh.com/marapr05/ simulation.html
Radhakrishnan, K., Roache, J., & Cunningham, H. (2007). Measuring clinical practice parameters with human patient simulation: A pilot study. International Journal of Nursing Education Scholarship, 4(1), 1-11. doi: 10.2202/1548-923x.1307.
Schwirian, P. M. (1978). Evaluating the performance of nurses: A multidimensional approach. Nursing Research, 27, 347-351.
Sheetz, L. J. (1999). Baccalaureate nursing student preceptorship programs and the development of clinical competence. Journal of Nursing Education, 28, 29-35.
Sportsman, S., Bradshaw, P., Bolton, C., Lee, M., Townley, N., Watson, M., & Close. (2009). A regional simulation center partnership: A collaboration to improve staff and student competency. Journal of Continuing Education in Nursing, 40(2), 67-73.
Steadman, R., Coates, W., Huang, Y., Matevosian, R., Larman, B., McCullough, L., et al. (2006). Simulation-based training is superior to problem- based learning for the acquisition of critical assessment and management skills. Critical Care Medicine, 34(1). 151-157. doi: 10.1097/01CCM00001906.42013.94
Weaver, A. (2011). High-fidelity patient simulation in nursing education: An integrative review. Nursing Education Perspectives, 32(1), 27-43). doi: 10.5480/1536-5026-32.1.37
Weinstein, C., Schulte, A., & Palmer, D. (2002). Learning and study strategies inventory (LASSI) users manual (2nd. ed.). Clearwater, FL: H & H Publishing.
About the Authors Susan Sportsman, PhD, RN, ANEF, was dean of the College of Health Sciences and Human Services, and Minnie Rhea Wood Professor of Nursing, Midwestern State University, Wichita Falls, Texas, when this study was conducted. She will assume the position of consultant at Elsevier on August 15, 2011. Randall E. Schumacker, PhD, is a professor in the College of Education, University of Alabama, Tuscaloosa. Patti Hamilton, PhD, RN, is dean of graduate studies, Midwestern State University. For more information, contact Dr. Sportsman at email@example.com.