by Kimberly Garcia, DNP, FACNM, CNM, WHNP ↓ & Julie Gee, PhD, MSNEd, RN ↓

Pretest-Posttest Evaluation of Change in Knowledge and Knowledge Retention after Educational Activities with Low-Literacy Lay Midwives in Urban Guatemala

Abstract

Aims: The primary aim was to assess lay midwives’ knowledge about preeclampsia after an educational activity. The secondary and third aims were to assess lay midwife knowledge retention about preeclampsia and about obstetrical emergencies.

Design: A pretest-posttest evaluation was used to assess lay midwife knowledge and knowledge retention about preeclampsia and obstetrical emergencies.

Methods: Through a partnership among three organizations, an evidence-based educational activity was conducted to enhance lay midwives’ knowledge about preeclampsia. Activities included repetition, role plays, storytelling, hands-on practice, and return demonstrations. Preeclampsia Reminder Cards were modified to increase visual literacy. A 17-item pre-posttest was used to evaluate changes in lay midwives’ knowledge. Knowledge retention was assessed about obstetrical emergencies and preeclampsia.

Results: Eleven lay midwives participated in the preeclampsia study. The average pretest score was 32% compared to the average posttest score of 88%. All midwives returned a year later. The average knowledge retention score was 66%. Seventeen lay midwives participated in an obstetrical emergency study with nine returning. The average knowledge retention score was 73% on a six-item survey compared to 51% on the pretest and 85% on the posttest.

Conclusions: Culturally relevant educational activities increase lay midwife knowledge and knowledge retention about preeclampsia and obstetrical emergencies.

Introduction

Guatemala is one of the few Latin American countries in which approximately 50% of the people self-identify as indigenous (United States Agency for International Development [USAID], 2022). Heath inequities persist among indigenous and nonindigenous people, particularly in women of reproductive age (Every Mother Counts, 2022). The maternal mortality rate (MMR) in Guatemala is 95 deaths per 100,000 births which is ranked 75th in the world and second in Central America, after Nicaragua (Central Intelligence Agency, 2020).

Approximately 90% of rural births (Summer, 2019) and 60 to 70% of all births in Guatemala occur at home (Zelter, 2018) in part due to lack of institutional capacity, health care associated with financial (Goldman, 2001) and geographical constraints, and cultural and linguistic marginalization (Every Mother Counts, 2022). The majority of the home births are attended by lay midwives (Juarez, 2020), also known as comadronas in Spanish. Because lay midwives come from the same communities they serve and speak the local language, they are trusted and respected; therefore, play a significant role in women’s health (Goldman, 2001).

While lay midwives are affordable practitioners who play a pivotal role in assisting pregnant women to birth at home in Guatemala (Goldman, 2001); lay midwives often lack knowledge about obstetrical emergencies (Lang, 1997, Roost et al, 2004, Walsh, 2006), such as preeclampsia, which is the second leading cause of maternal death in the country after postpartum hemorrhage (PPH) (USAID, 2017).

The Guatemala Ministry of Health (MOH) had offered trainings since 1955 for lay midwives, however, many issues are evident with the trainings (Goldman & Glei, 2000, Lang, 1997). Lay midwife trainings are offered sporadically, taught in Spanish with written material even though many lay midwives are indigenous, speak Mayan dialects (including Kaqchikel), and have low literacy (Fahey, 2013: Hernandez, 2017, Kestler et al., 2013, Maupin, 2009; Walker, 2015).

Additionally, MOH registered nurses (RNs) with a minimum of a one year of experience, not necessarily in birth, teach the trainings, (Goldsman, 2001) sometimes criticizing lay midwife practices (Goldman & Glei, 200; Greenberg, 1982, Maupin, 2008). The trainings focus on recognizing “the signs of danger” and transferring the pregnant patients to hospitals (Foster et al., 2004; Zelter, 2018) even though many barriers exist for transfer.

Further, no evaluations occur to determine if the trainings change lay midwife knowledge, practice, and ultimately improve maternal and newborn outcomes (Lang, 1997; Foster et al., 2004). Unfortunately, the MOH’s approach to training lay midwives has not improved outcomes (Bailey et al, 2002) and amplifies health inequities for women who birth at home, as evidenced by the Maternal Mortality Rate (MMR) in Guatemala not decreasing statistically in more than 20 years (Chary et al., 2012).

Four articles regarding MOH trainings found lay midwives did not use information from the trainings because their knowledge and practices were dismissed during the trainings (Greenburg, 1982; Maupin, 2008; Hinojosa, 2004), or because the information was not provided in their language (Lang, 1997). Two other articles found lay midwife practice is guided more by spirituality than information provided in MOH trainings (Roost, 2004; Walsh, 2006).

All six articles recommended providing information in the native language of speakers, orally for low-literacy audiences with educational activities, such as role plays, hands-on activities, repetition and return demonstrations, and to respect and integrate lay midwife knowledge and practices. These suggestions were implemented in three studies where lay midwife knowledge about PPH and obstetrical emergencies changed significantly after participating in culturally sensitive educational activities (Garcia et al., 2012; Garcia et al., 2018, Garcia, 2022).

The culturally appropriate education was tailored and framed from the perspective of the target population—engaging with local culture is important to determine how culture influences behavior (Thomas et al., 2004).  For example, the primary investigator (PI) designed a Preeclampsia Reminder Card (Appendix A) by combining drawings from “Signs of Danger” in the MOH’s Birth Log and the Home-Based Life Saving Skills (HBLSS) curriculum from the American College of Nursing Midwives (ACNM), which has been field tested throughout the developing world (Sibley, et al., 2010).

Lay midwives in past educational activities understood simple color drawings of the “Signs of Danger” from a MOH Birth Log better than complex, black and white drawings from the HBLSS (Garcia, 2022). Thus, reminders of the signs and symptoms of preeclampsia were created from the MOH drawings, and reminders of interventions came from the HBLSS, as the MOH does not offer visual representations of interventions.

Other lay midwife training methods that have demonstrated improved outcomes among pregnant women and neonates include providing adequate consultation for lay midwives (Juarez et al., 2020), lay midwives teaching lay midwives (Hernandez et al., 2017), and intensively training lay midwives (Foster, 2004; Thompson et al., 2014). Components of all of these studies included providing information orally in the native language of participants, and respecting and integrating lay midwife knowledge and practices.

 A fifth study found intensive training of lay midwives decreased postpartum complications, but did not improve lay midwives’ detection and hospital referral for obstetrical emergencies (Bailey et al., 2002). This study did not mention if the training was provided orally in the native language of participants and if lay midwife knowledge and practices were respected.

We focused on culturally sensitive educational activities presented orally in the native language as participants, as these methods enjoyed the most evidence in the literature.

Also critical to improving lay midwife knowledge is consistent educational reinforcement (Mosby et al., 2015; Weiss et al., 2009). Even though this project focused on presenting new content to the lay midwives, we were also able to assess knowledge retention a year after the educational activity about preeclampsia and three years after a prior educational activity about obstetrical emergencies. 

This project, which was a partnership among the MOH, expert faculty in the College of Nursing at the University of Utah (UU), and the non-profit organization Refuge International, took place in San Raymundo, an urban municipality in the highlands of northeast Guatemala.

Aims

The primary aim for the project was to improve low-literacy lay midwives’ knowledge about preeclampsia in San Raymundo using culturally relevant educational activities. A secondary aim was to assess knowledge retention about preeclampsia a year after the original educational activity. A final aim was to assess knowledge retention three years after a previous culturally appropriate educational activity with lay midwives about recognizing obstetrical emergencies.

Methods

Design

A cross-sectional pre-posttest evaluation was used to examine the effect of an evidence-based learning activity on lay midwife knowledge about preeclampsia. Participants received laminated Preeclampsia Reminder Cards to reinforce information presented in the educational activity. The project also included a posttest to evaluate lay midwife knowledge retention about preeclampsia a year after the original teaching, and a posttest to evaluate knowledge retention about obstetrical emergencies three years after a learning activity about obstetrical emergencies. A UU Internal Review Board granted the study permission. A checklist for the Strengthening Reporting of Observational Studies (STROBE) was followed in reporting this study. The study met all nine of the ACNM global competencies and skills (ACNM Division of Global Engagement, 2017).

Sample

The target population consisted of 11 lay midwives at a Refuge International health clinic in San Raymundo. No sampling method was used. The lay midwives were recruited for participation by the MOH RNs who informed the lay midwives about the preeclampsia educational activity (See Educational Activity Box.) and the project a month beforehand. Upon arrival at the health clinic, the lay midwives were informed in detail about the study. They were advised that the preeclampsia education would not be withheld whether or not they chose to participate in the study; that willingness to complete the survey served as consent for study participation; and that they were free to leave at any time. Any participant who identified as a midwife and spoke Spanish or Kaqchikel was invited to participate. All information was provided in Spanish and translated into Kaqchikel.

Of the 11 lay midwives attending the educational activity, all agreed to participate in the-post evaluation project in 2021 and 2022. All questionnaires were completed; thus, no forms were discarded. To address the final aim of knowledge retention regarding obstetrical emergencies, nine midwives who had attended an educational activity in 2018, voluntarily completed an eight-item posttest in 2021.

Table 1 displays demographic information. Age range of participants was 30 to 75 years with an average of 53 years. Experience as a lay midwife ranged from one to 50 years with an average of 22 years. Formal education ranged from none to 15 years with an average of 6.35 years. Participants reported attending none to 13 births a month, for an average of four. Four participants could read, write, and count. Two could read and write. Two could write. Two could not read, write, or count. One could read. The groups consisted of 10 Spanish speakers and one Kaqchikel speaker. Other information asked included comfort transferring pregnant patients with obstetrical emergencies to hospitals. Three participants were comfortable, two were slightly comfortable, and three were slightly uncomfortable.

Data Collection

Data was collected face-to face on paper forms. RNs from the MOH and Refuge International volunteers assisted low-literacy lay midwives complete all forms. The questionnaire included two parts, a six-item demographic profile and a 17-item pretest/posttest questionnaire about preeclampsia knowledge. The 17-item questionnaire included 14 fill-in-the blank questions about normal blood pressure, high blood pressure, six signs and symptoms of preeclampsia, three actions to take if a patient has preeclampsia, and three results that can occur if the patient with preeclampsia doesn’t get needed medical attention. The 17-item questionnaire also included three questions about the correct way to take blood pressure. Two answers, one correct and one incorrect, were offered for these three questions. Participants were invited to circle the correct answer to each of these three questions. The six-item survey about obstetrical emergencies came from the HBLSS. (See Survey Boxes).

The PI, who has had a program of study since 2009 with Guatemalan lay midwives in partnership with the MOH and Refuge International, developed the demographic profile and surveys based on a literature review about Guatemalan lay midwives and obstetrical emergencies. A judge panel of three experts established face validity of the tools and a Fog Index of 0.35. The Fog Index measures the readability of the writing sample. The resulting score is an approximation of the number of years of formal education required to understand the tool on first reading. In other words, participants with less than a year of formal education should understand the questions on the demographic profile and the questionnaires.

Data Analysis

The evaluation relied on frequencies and measures of central tendency to analyze demographic data. Scores on the pretest and posttests were based on the total number of correct answers from a questionnaire about preeclampsia and obstetrical emergencies.

Results

The average preeclampsia pretest score was 5.36 out of 17 (31.52%). The average preeclampsia posttest score was 15 (88.23%). The average knowledge retention score about preeclampsia was 11.18 (66%). The average knowledge retention score about obstetrical emergencies was 4.4 (73.33%) compared to 51% on the pretest and 85.5% on the posttest. (See Results Box.)

Most participants had a simplistic understanding of preeclampsia. Before the educational activity, they knew what normal blood pressure was, that headaches and blurred vision were signs of preeclampsia, likely because these signs are taught in the MOH classes, and that they needed to transfer patients with high blood pressure to the hospital. However, most participants lacked a more in-depth comprehension of preeclampsia that could help them save lives,

After the educational activity, most participants knew what high blood pressure was. They could recite other signs and symptoms or preeclampsia, such as nausea and vomiting, right upper quadrant pain, and peripheral edema. They also understood they needed to place the patient on her left side, and that the patient could seize, and she and her fetus could die, if the patient did not receive medical treatment.

Information participants were most likely to forget was that hyper reflexes can be a sign of preeclampsia, and they shouldn’t put anything in the patient’s mouth. Further, lay midwives could recall the correct mechanics for taking a blood pressure, such as the patient should sit without her legs crossed, and she should have her feet on the ground. However, participants fumbled a bit with the mechanics of taking a blood pressure, which is normal when this skill is introduced.

In addition to the laminated Preeclampsia Reminder Cards, all participants were given a backpack with a stethoscope, sphygmomanometer and two birth kits that included blue pads to put under patients, exam gloves, umbilical cord clamps, scissors, razors, soap, iodine, bulb suctions, tape measures, and newborn onesies, and knitted baby hats and sweaters.

Participants were encouraged to continue practicing taking blood pressures with family members and friends at home. MOH RNs asked the lay midwives to bring blood pressure instruments to monthly meetings at the MOH so RNs could continue providing guidance on this skill. The PI also requested that participants keep data on the number of obstetrical emergencies they see to share during future studies to evaluate if the educational activities are changing lay midwife practice.

Regarding knowledge retention, participants retained more information about obstetrical emergencies (73%) than preeclampsia (66%), despite the time frame being longer between testing periods, four compared to one year, likely because MOH RNs frequently teach lay midwives about obstetrical emergencies. A closer timeframe for retesting about obstetrical emergencies was precluded by the COVID pandemic.

On a positive note, knowledge retention scores among lay midwives in San Raymundo were better than knowledge retention scores among rural lay midwives regarding PPH (Garcia & Dowling, 2018). In 2016, ten lay midwives from remote Sarstun retained 21% of eight steps to address PPH on a pretest and 65% of steps on a posttest after a second educational activity when evaluated seven years after an initial educational activity in 2009 (Garcia & Dowling, 2018). By comparison, twelve lay midwives, including ten from 2016, identified 17.5% of steps on a pretest and 60% of steps on a posttest to address PPH after an educational activity in 2009 (Garcia et al., 2012).

Ideally, Sarstun lay midwives should have been evaluated for knowledge retention at a closer interval that seven years. These lay midwives still had more knowledge about PPH in 2016 than they had before an initial educational activity in 2009. Further, Sarstun lay midwives likely had lower retention scores than San Raymundo lay midwives as Sarstun midwives were retested at a longer interval, and Sarstun lay midwives are less literate and live more remotely than San Raymundo lay midwives.

Discussion

Results from this study reinforce recommendations in the literature to provide information in the native language of participants, orally for low-literate audiences, with educational activities and drawings to reinforce knowledge retention, and to respect and integrate knowledge and practices from participants. Results from this study demonstrate that the knowledge of low-literacy participants will improve and be retained when these suggestions are followed.

Limitations of the study

This study took place during the COVID-19 global pandemic, which further alienated already marginalized groups, such as the indigenous. Thus, the sample size was smaller than ideal and may not be generalizable to lay midwives in urban Guatemala. Lay midwives who attended the study during COVID-19 were likely to be more affluent than most lay midwives, which could have led to sampling bias. Further, only one indigenous lay midwife participated in this educational activity compared to five indigenous among 17 lay midwives who participated in a 2018 educational activity about obstetrical emergencies. Finally, most lay midwives answered the questions correctly that had to be circled in the preeclampsia survey which may have led to question bias.

Conclusion

Clearly teaching low-literacy audiences who speak indigenous languages in Spanish with written material is not effective. Instead, low-literacy audiences learn best with educational activities, such as roll plays, repetition, storytelling, hands-on practice, and return demonstrations, presented orally in their native languages.

Knowledge retention cards about obstetrical emergencies, such as preeclampsia, should be created with cultural humility to increase visual literacy. Simple, color drawings from inside the culture are more effective than complex, black and white drawings from outside the culture. Evaluation of knowledge retention should occur yearly at the very least. Data should be consistently collected on lay midwife responses to obstetrical emergencies to evaluate if the educational activities are changing practice, and ultimately decreasing the MMR in Guatemala.

Ethical Aspects and Conflict of Interest

The study was conducted with the approval of the UU Internal Review Board. Issues of confidentiality and anonymity were addressed. The study encompassed has no conflicts of interest.

Funding

This study did not receive a grant or funding from a public, commercial or non-profit source.

References

  • American College of Nurse Midwives Division of Global Engagement. (2018). ACNM Global Health Competencies and Skills. Retrieved from https://www.midwife.org/acnm/files/cclibraryfiles/filename/000000007496/Global%20Health%20Competencies.pdf
  • Bailey PE, Szászdi JA, Glover L. Obstetric complications: does training traditional birth attendants make a difference? Rev Panam Salud Publica. 2002 Jan;11(1):15-23. doi: 10.1590/s1020-49892002000100003. PMID: 11858126.
  • Central Intelligence Agency. (2020). Central America: Guatemala. Retrieved from https://www.cia.gov/the-world-factbook/field/maternal-mortality-ratio/country-comparison
  • Chae, SY, Chae, MH, Kandulas, S, Winter, RD. (2017). Promoting Improved Social Support and Quality of Life with CenteringPregnancy Group Model of Prenatal Care. Arches of Women’s Mental Health, 20(1) 209-220. doi: 10.1007/s00737-016-0698-1
  • Chary A, Díaz AK, Henderson B, Rohloff P. The changing role of indigenous lay midwives in Guatemala: new frameworks for analysis. Midwifery. 2013 Aug;29(8):852-8. doi: 10.1016/j.midw.2012.08.011. Epub 2013 Feb 12. PMID: 23410502.
  • Every Mother Counts (2022). Guatemala: A Deeper Dive. Retrieved from https://everymothercounts.org/grants/guatemala-a-deeper-dive/
  • Fahey, JO, Cohen, SR, Holme, F, Buttick, ES, Dettinger, JC, Keslter, E, Walker, DM. (2013). Promoting Cultural Humility During Labor and Birth. Journal of Perinatal and Neonatal Nursing, 27(1), 36-42. http://DOI:10/1097/JPN.0b013e31827e478d
  • Foster J, Anderson A, Houston J, Doe-Simkins M. A report of a midwifery model for training traditional midwives in Guatemala. Midwifery. 2004 Sep;20(3):217-25. doi: 10.1016/j.midw.2004.01.004. PMID: 15337277.
  • Garcia, K, Dowling D, Mettler, G. (2018). Teaching Guatemalan traditional birth attendants about obstetrical emergencies. Midwifery, 61, 36-38. https://doi.org.10.1016/j.midw.2018.02.012
  • Garcia, K, Morrison, B, Savrin, C. (2012). Teaching Guatemalan Midwives about Postpartum Hemorrhage. MCN: American Journal of Maternal/Child Nursing, 37, 42-47. http:// doi: 10.1097/NMC.0b013e3182387c0a.
  • Garcia, M, Chismark, EA, Mosby, T, Day, S. (2010). Development and Validation of Nutritional Educational Pamphlet for Low Literacy Pediatric Oncology Caregivers in Central American. Journal of Cancer Education, 25(4), 512-517. Doi:10.1007/s13187-010-0080-3
  • Garcia, K. (2022). An Observational Study of Teaching Methods with Low-Literacy Comadronas in Urban Guatemala. Canadian Journal of Midwifery Research and Practice, Volume 21, Number 3.
  • Goldman, N. Glei, D.A., Pebley, A.R., & Delgado. H (2001). Pregnancy Care in Rural Guatemala: Results from the Encuestra Guatemalteca de Salud Familiar. Rand. https://www.rand.org/content/dam/rand/pubs/drafts/2008/DRU2642.pdf
  • Greenberg L. Midwife training programs in highland Guatemala. Soc Sci Med. 1982;16(18):1599-609. doi: 10.1016/0277-9536(82)90290-8. PMID: 7146937.
  • Hernandez, S, Bastos Oliveria, J, Shitazian, T. (2017). How a Training Program is Transforming the Role of Traditional Birth Attendants from Cultural Practitioners to Unique Health-care Providers: A Community Case Study in Rural Guatemala. Frontiers in Public Health, 5(111), 1-8. http://doi.10.3389/fpubh2017/00111
  • Hinojosa SZ. Authorizing tradition: vectors of contention in Highland Maya midwifery. Soc Sci Med. 2004 Aug;59(3):637-51. doi: 10.1016/j.socscimed.2003.11.011. PMID: 15144771.
  • Juarez, M, Juarez, Y, Coyote, E, Nguyen, T, Shaw, C, Hall-Clifford, R, Gillard, G, Rohloff, P. (2010). Working with Guatemalan Lay Midwives to Improve the Detection of Neonatal Complications in Rural Guatemala. British Medical Journal Open Quality, 9(1), e000775. doi: 10.1136/bmjoq-2019-000775
  • Juarez M, Juarez Y, Coyote E, Nguyen T, Shaw C, Hall-Clifford R, Clifford G, Rohloff P. Working with lay midwives to improve the detection of neonatal complications in rural Guatemala. BMJ Open Qual. 2020 Jan 23;9(1):e000775. doi: 10.1136/bmjoq-2019-000775. PMCID: PMC7011902.
  • Kestler, E, Walker, D, Bonvecchio, A, Saenz de Tejada, S, Donner, A. (2013). A matched pair cluster randomized implementation trail to measure the effectiveness of an intervention package aiming to decrease perinatal mortality and increase institution-based obstetric care among indigenous women in Guatemala: a study protocol. BMC Pregnancy & Childbirth, 13(73), 1-11. http://doi: 10.1186/1471-2393-13-73.
  • Lang, JB, Elkin, ED. (1997). A study of the beliefs and birthing practices of traditional midwives in rural Guatemala. Journal of Nurse Midwifery, 42(1), 25-31.
  • Maupin JN. Remaking the Guatemalan midwife: health care reform and midwifery training programs in Highland Guatemala. Med Anthropol. 2008 Oct-Dec;27(4):353-82. doi: 10.1080/01459740802427679. PMID: 18958785.
  • Mosby, TT, Hernandez Romero, AL, Molina Linares, AC, Challinor, JM, Day, SW, Caniza, M. (2015). Testing efficacy of teaching food safety and identifying variables that affect learning in a low-literacy population. Journal of Cancer Education, 30 (1) 100-7.doi: 10.1007/s13187-014-0666-2.
  • Roost, M, Johnsdotter, S, Liljestrand, J, Essen, B. (2004). A qualitative study of conceptions and attitudes regarding maternal mortality among traditional birth attendants in rural Guatemala. British Journal of Obstetrical Gynaecology, 111, 1372-1377.
  • Sibley, L, Tebben Buffington, S, Beck, D, Armuster, D. (2010). Home Based Life Saving Skills: Promoting Safe Motherhood Through Innovative Community-Based Interventions. Journal of Midwifery & Women’s Health, 46(4), 258-266. https://doi.org/10.1016/S1526-9523(01)00139-8
  • Summer, A, Walker, D, Guendelman, S. (2019). A Review of the Forces Influencing Maternal Health Policies in Post-War Guatemala. World Medical and Health Policy, 11(1), 59-82. http://doi.10.1002/wmh3.292
  • Thomas, S. B., Fine, M. J., & Ibrahim, S. A. (December, 2004). Health disparities: the importance of culture and health communication. American Journal of Public Health., 94(12): 2050. doi: 10.2105/ajph.94.12.2050
  • Thompson LM, Levi AJ, Bly KC, Ha C, Keirns T. Premature or just small? Training Guatemalan birth attendants to weigh and assess gestational age of newborns: an analysis of outcomes. Health Care Women Int. 2014 Feb;35(2):216-31. doi: 10.1080/07399332.2013.829066. Epub 2013 Oct 18. PMID: 24138160; PMCID: PMC3925468.
  • United States Agency for International Development (2017). Ending Preeclampsia: Focus Country Guatemala. Retrieved from https://knowledgecommons.popcouncil.org/departments_sbsr-rh/620/
  • United States Agency for International Development (2022). Indigenous Peoples. Retrieved from https://www.usaid.gov/guatemala/indigenous-peoples
  • Walker, DM, Holme, F, Zelek, ST, Olvera-Garcia, M, Montoya-Rodriguez A, Fritz, J, Fahey, J, Lamadrid-Figueroa, H, Cohen, S, Kestler, E. (2015). A process evaluation of PRONTO simulation training for obstetric and neonatal emergency response teams in Guatemala. BMC Medical Education, 15(117), 1-8. http: doi: 10.1186/s/12909-015-041-7
  • Walsh, L. (2006). Beliefs and Rituals in Traditional Birth Attendant Practice in Guatemala. Journal of Transcultural Nursing, 17(2), 148-154. http://DOI:10.1177/1043659605285412
  • Weiss-Laxer, NS, Mello, MJ, Nolan, PA. (2009) Evaluating the education component of a hospital-based child passenger safety program. Journal of Trauma, 67. DOI: 10.1097/TA.0b013e3181a93512
  • Weiss, HB, Bouffard, SM, Bridglall BL, Gordon, WE. (2009). Reframing Family Involvement in Education: Supporting Families to Support Educational Equity. Equity Matters, Research Review, No. 5.
  • Zelter, N. (2018, December, 13) A Sacred Calling: Midwifery in Guatemala. Midwifery Around the World, 12/13/2018. https://medium.com/midwifery-around-the-world/a-sacred-calling-midwifery-in-guatemala-539768907dc7

Box: Educational Activity about Preeclampsia

The educational activity about preeclampsia began by asking lay midwives the following questions in a group format: what are the signs and symptoms of pre-eclampsia, what is normal blood pressure what is high blood pressure, what is the proper way to take a blood pressure, what should they do if a patient show signs of pre-eclampsia, what are the consequences of pre-eclampsia for the patient and fetus.

Information came from the group first and was provided if the group was lacking knowledge or provided incorrection information, in a Centering-Pregnancy style fashion, which improves participant engagement (Chae et al, 2017). Information was repeated back, multiple times throughout the educational activity. Lay midwives also participated in roll plays with an actor pretending to be a patient with pre-eclampsia and in return demonstrations of taking a correct blood pressure.

Participants practiced taking blood pressures on each other for an hour. Questions were asked during return demonstrations, such as should the patient cross her legs or have them flat on the ground? Should the patient be sitting, standing, or lying down? At what level should the arm be when taking a blood pressure.

Laminated, Preeclampsia Reminder Cards were given to participants to support knowledge retention of the signs, symptoms and interventions. The Reminder Cards were modified to improve visual literacy. Reminders for the signs and symptoms of preeclampsia came from MOH drawings of obstetrical emergencies. Reminders for interventions regarding preeclampsia came from the ACNM’s HBLSS curriculum.

Table One: Demographic Data, N=11

DataRangeAverage
Age30-7653
Years as Lay Midwife1-5022
Years of Formal Education0-156.35
Primary Language10 Spanish1 Kaqchikel
Number of Births per Month0-144
Comfort with Hospital Transfers4 comfortable 2 slight comfortable 3 uncomfortable 4 slightly comfortable Literacy 
Literacy4 Read, write, count 2 Read, write 2 Write 2 Does not read, write, count 1 Reads 

Survey Box: Preeclampsia Pretest & Posttest

QuestionsCorrect AnswersPretest ScoreCorrect ScoreRetention Score
What is normal blood pressure?120/808/1111/1111/11
What is hypertension?140/901/1110/118/11
What is the correct way to take blood pressure? Circle the correct responses.Patient should be seated or lying down. Patient should cross her legs or patient should not cross her legs. Patient’s legs should be on the floor or its not important where patient’s legs are.1/11   1/11       1/117/11   9/11       8/115/11   4/11       5/11
What are six signs and symptoms of preeclampsia?6. headache 7. blurred vision 8. nausea & vomiting 9. right upper quadrant pain 10. swelling that’s worse in the morning 11. hyper reflexes10/11 10/11 4/11 5/11 1/11 3/11 0/1111/11 11/11 9/11 8/11 9/11 8/11 3/1111/11 10/11 4/11 3/11 5/11 6/11 1/11
What three actions should be taken if a patient has preeclampsia?12. Put her on her left side 13. Don’t put anything in her mouth 14. Transfer her to the hospital0/11 0/11   11/1110/11 6/11   11/116/11 3/11   11/11
What three things can happen if a patient with preeclampsia doesn’t receive proper treatment?15. She can seize. 16. She can die. 17. Her baby can die.1/11 1/11 1/1111/11 11/11 11/1110/11 10/11 10/11

Survey Box: Correct Examples of Obstetrical Emergencies for Pretest & Posttest Example

AnswersPretest ScorePosttest scoreRetention Score
Bleeding14/1717/179/9
Infection of uterus, breast, urinary tract6/1714/173/9
Preeclampsia5/1712/274/9
Birth Delay8/1715/178/9
Vaginal Infection or Malaria6/1713/275/9
Grand Multiparous Patient13/1716/279/9

Knowledge Retention Results Box

Teaching ContentYearPretestPosttestKnowledge Retention & Year
Preeclampsia202132%88%66% 2022
Obstetrical Emergencies201851%85.5%73% 2022
Postpartum Hemorrhage200921%65%60% 2016
by Candace Chow, PhD ↓; Meganne N. Ferrel, MD ↓; Emily M. Graham, MD ↓ & Megan Fix, MD ↓

Women Leaders in Academic Medicine: A Qualitative Study Addressing the Leaky Pipeline

Published in Cureus: Chow C J, Ferrel M N, Graham E M, et al. (April 10, 2024) Perspectives From Students and Faculty on How Women Achieve Leadership Roles in Academic Medicine: An Exploratory Qualitative Study . Cureus 16(4): e57969. doi:10.7759/cureus.57969

Abstract

Introduction: The glass ceiling in academic medicine is characterized by lower pay, fewer career advancement opportunities, and fewer tenured faculty positions held by women despite increasing numbers of women entering medicine. Creating change relies on preparing early-career women for positions of leadership, but most leadership programs focus on faculty, not trainees. The present study investigates how to prepare women medical students to be leaders in academic medicine.

Methods: This qualitative study used the theory of gendered organizations as a conceptual framework. Focus groups with women medical students and faculty were conducted at an academic medical center in the West. A total of 25 individuals (10 students and 15 faculty) participated. Recordings of focus groups were transcribed and coded using thematic analysis until saturation of themes was achieved.

Results: Codes were organized into three themes: obstacles, support systems, and self-presentation. Obstacles identified included the subthemes microaggressions, macroaggressions, lack of women role models in leadership, and personal characteristics such as ability to self-promote and remain resilient. Support systems included sponsorship, allyship, mentorship, networking, and gender-specific role modeling subthemes. Self-presentation involved learning behaviors in demonstrating leadership and exuding confidence, being strategic about career moves, resiliency, and navigating social norms.

Discussion: The key themes of obstacles, support systems, and self-presentation are targets for systemic and individualistic improvement in leadership development. The theory of gendered organizations underscores the importance of mitigating obstacles and increasing support from at the systemic, rather than individual, level.

Introduction

The glass ceiling is a familiar concept in medicine characterized by lower pay, fewer career advancement opportunities, and fewer tenured faculty positionsheld by women physicians across the nation, despite increasing numbers of women entering medicine.1-5 It is not for lack of trying either – research demonstrates that women are not given the same opportunities as men, such as for networking and serving on hiring committees – which translates to fewer opportunities for career advancement.6Likewise, we see attrition of women leaders, or a “leaky pipeline,” which is partially attributed to a lack of positive mentors and role models, discrimination, and gender bias.1,7  While there are programs designed to help women circumvent these obstacles, most of these programs are designed to help women that are already in positions of leadership.8,9 These programs, while needed, do not reach women at the beginning of their medical careers. Given that more women are starting to enroll in medical school than men, it is important to understand how to prepare early-career women, including students and trainees, to become leaders and break the glass ceiling. 

Two of the authors on this team are former leaders of a grassroots women’s leadership group named WE WILL (Women Empowering Women in Leadership) at the Spencer Fox Eccles School of Medicine at the University of Utah. WE WILL aims to create space and promote women leaders in medicine utilizing mentorship and professional development through networking events and skill-building workshops for women students and physicians. Preliminary results from WE WILL programming show success in empowering women for leadership positions after networking and skill workshop events.10  Based upon WE WILL outcomes and greater University support, we think developing and implementing a leadership curriculum will motivate more women to choose careers as academic leaders in medicine and better prepare them to succeed in those roles. Ultimately, by better-preparing women for positions of leadership within medicine, important perspectives and representation will be added to our healthcare system, along with enhanced patient care and decision-making processes.11-14

Conceptual Framework

The Theory of Gendered Organizations15-17 assumes that the workplace was created and is divided along gendered lines. Instead of arguing that gender discrimination exists, this theory helps us understand how gender discrimination happens and is reproduced. Acker15,16 explains that organizations are gendered with respect to divisions along gendered lines (e.g., men assume positions of power while women are tasked with unskilled work). Symbols, such as clothes, help to reinforce these gender divisions. In turn, the way in which individuals express themselves at work is also gendered. For example, suits, which are considered typical workplace attire, are traditionally masculine. Finally, organizations are also gendered with respect to how individuals interact with one another (e.g., turn-taking in meetings).

Research Questions

This qualitative study was conducted to understand the challenges women medical students and faculty face in academic medicine to inform the creation and implementation of a leadership curriculum for women medical students. The study explored the support systems and resources participants have used to buffer these challenges. Using the Theory of Gendered Organizations as the conceptual framework,15 we are specifically interested in understanding how these challenges are reflective of gendered organizations such as academic medical centers and what support systems participants utilize to overcome these challenges.

Methods

This project was deemed exempt by the University of Utah Institutional Review Board. Focus groups were conducted with University of Utah women medical students and physician faculty holding positions of leadership. Student participants self-selected to participate after an open recruitment email was distributed to the school of medicine. One of the faculty authors assisted with identifying women faculty in positions of leadership to engage in purposive sampling of women leaders. As this study was focused on women, only women-identifying medical students and faculty were asked to participate. Three student and three faculty focus groups were held. A total of 25 individuals participated (10 students and 15 faculty). Student focus groups were conducted by MNF and EMG, and faculty focus groups by MF. Participant consent was obtained by the individual conducting the focus group. Questions for the semi-structured focus groups were pilot tested by the authors. Questions were designed to inquire about individual determination, aspiration, support, and obstacles in becoming a leader in medicine. Focus groups were conducted and analyzed until there was a saturation of themes.11 To ensure focus group leaders understood participants correctly, accuracy was corroborated during the focus groups through reflexive feedback.

Focus groups were transcribed verbatim and transcripts were coded and analyzed using thematic analysis.12 The primary investigator reviewed all transcripts in detail and developed an initial codebook. Other authors then coded the transcripts. Disagreements were resolved by consensus and revisions were made to the codebook. Authors (CJC, MNF, EMG) then reviewed all transcripts to ensure the data were well-described by this revised codebook.

Positionality of Researchers

The authors include one physician, one medical education researcher, and two medical students (both of whom are now residents). Having both faculty and students on the team afforded us rapport with the two participant groups of focus (faculty and students) and allowed us to explore faculty and student perceptions of the data.

Results

Codes were organized into three themes: obstacles, support systems, and self-presentation. In the subsequent paragraphs, we share exemplary quotes that illustrate these themes, many of which reflect gendered processes at work. We use ‘S’ to denote when a quote is from a student focus group and ‘F’ when a quote is from a faculty-student group. The number is used to distinguish between the student and faculty focus groups.

Obstacles

Participants reported various obstacles in their careers. Most obstacles are reflective of the ways in which gendered organizations continue to reinforce differences and disparities between men and women. Obstacles included microaggressions, macroaggressions, a lack of women in leadership, lack of role models, and personal characteristics.

Participants shared about various microaggressions they had experienced as women. A common refrain was not being recognized as the physician: “when you’re a petite female … I used to always get, ‘Hey nurse, am I going to see a doctor?’” (F4). Participants shared about how they were viewed differently because they were mothers: “whatever we were juggling, I always contributed, I always worked hard or as hard, if not harder, I think, than a lot of people, and to have little barbs thrown at you because you were on maternity leave or because you went to your child’s kindergarten…performance” (F3). Several faculty members commented on the “mind clutter, the worry you have to go through about ‘what am I going to wear today?’” (F4). Students shared similar concerns: “…we have a patient presentation and they just say, “Dress professionally.” If you look around the room, the men just get to wear a button up shirt and slacks, and that is professional and it’s easy. And the women have to figure out, ‘Is this dress long enough? Is this dress too low cut? Am I going to get looked at because my shoulders are out? Is a pencil skirt unprofessional?’ … These are things that are both internal and external barriers” (S2).

Participants also shared macroaggressions. Some reflected on hiring conversations: “sometimes more overt comments about like ‘are we trying to hire a woman or are we trying to hire the best person for the job?’” (F3). Another shared: “…I found out recently that there was a discussion that there should be a non-compete on my contract when I came here because I was going to get pregnant…but [it didn’t happen because] …somebody else had spoken up and said, “No, you can’t do that, that’s gender discrimination” (F3). Participants also disclosed instances of sexual harassment: “I have had one patient where I had to stop standing at a certain part of his bed because he could reach my leg and he would like grab and pat my leg…I just started standing on the far end of his bed where he couldn’t reach it” (S2).

The lack of women in leadership was a major concern shared among participants: “we still aren’t seeing very many women leaders… there’s systemic discrimination against women,” (S1). Faculty shared the same sentiment: “I think this is an amazing institution, but … some of the things that frustrate me here are not indirectly related to [the fact that… department chair is] the only woman in that room where those decisions get made right now and that’s just a problem” (F1).

Participants also mentioned how a lack of role models made their entry into medicine difficult. Whether it was because they did not “have any physicians in [their] family” (S3) or because “no one in [their] family…is in academia” (F4), these students and faculty discussed having to navigate a new field of study on their own. Others said they had a hard time picturing themselves in medicine because they had not seen themselves reflected in role models: “growing up, I never saw a woman who’s a minority in places that I wanted to be. I never saw a Latino doctor… I had so many questions growing up. Like ‘how do I even get to this place?’” (S1).

Participants also discussed how their own characteristics, which they had been socialized to adopt, got in the way of themselves. One student reflected: “… a man is going to … take charge, and people will maybe listen to him more, and that’s a fear in the back of my mind…it’s definitely something I consider when I’m like, oh should I go after this position? Maybe it’s not a good fit for me, and I think I tie that into being a female” (S3). Faculty experienced similar internal struggles: “[My] biggest barrier has been myself. But going back to what [she] … said is, self-advocating. If you look at studies, women have been known to be, to undervalue themselves” (F2).

Support Systems

Participants spoke at length about support systems that helped them overcome the obstacles mentioned in the section above. Much of this support involved navigating and even circumventing the gendered system, including sponsorship, allyship from men in their lives, mentorship from colleagues, and gender-specific role modeling.

Faculty recalled how important sponsorship was to their success. They shared that sponsors are not necessarily long-term mentors but are individuals who “put themselves on the line and give you an opportunity” (F3). One shared: “The most important factor for me was a sponsor…He had just, he only spent two years in our department, but I happened to be his chief resident. And he sponsored me. I would have never had the job I had without him. I am 100 percent sure” (F2). Similarly, another participant said, “it was actually my immediate…supervisor within my division who saw my potential from my clinical interests to take over and was very great about…guiding me…into that role” (F3). Another faculty member said that their network was key to opportunities: “the value of the relationships has always been really important to me and has created mentorship and sponsorship [for me]” (F2).

Participants also shared how mentorship played a key role in their success. Several faculty across focus groups reflected on how mentorship programs advanced their careers, like the Association of American Medical College’s (AAMC) Executive Leadership in Academic Medicine (ELAM) program and other career-advancement programs designed specifically for women.18 Others explained that specific mentors helped pave their paths: “He also helped me say no to things and be like “You’re too busy.” So that was also, I think, a very important trait of a mentor, being interested and aware of everything that I was doing and to help me make the best choices” (S2). Participants noted that “it wasn’t necessarily people who looked like me or had the same path” (F1) that mentored them, but people who recognized their potential and provided encouragement.  

More than one participant commented on how men colleagues were key to their success: “…[he] was actually really important for me because I didn’t have female mentors because they didn’t exist” (S2). Others spoke about the importance of having men who understood the discriminatory system as mentors: “… it’s kind of put me in this place where finding male mentors who recognize that it was harder to be a woman [is helpful]” (S2). Participants also shared about the significance of having supportive life partners: “it’s hard enough to have to go learn all these things. I do Step studying. I didn’t do anything useful. I didn’t do the dishes. I didn’t my laundry. My husband did it all for me. … I think it’s huge” (S1).

Finally, students and faculty spoke about gender-specific advice they had received. One shared about lessons she had learned from her mother, who did not go to college until having children: “my mom would always say things like, “You’re going to college,” not like, “Are you going to college?…She was like, “…you’re going to college, and you’re doing it before you have kids. And you’re not getting married before you graduate college’” (S2). A faculty member also reflected on the importance of her mother’s example: “my mom was always the boss in my house, and she was always…the leader in any groups…And I would always see her running the show, and that was what I would look up to” (F3). Participants also shared about learning to respond to sexism from women role models: “I think the most helpful thing for me has been to see the behavior modeled by my attendings and residents to show how they handle it. Because unfortunately, it happens to them on rounds in front of the entire team, just as much as it happens to me alone in an exam room before I go get the attending” (S2).

Self-Presentation

Participants (mainly the faculty) shared that another way to overcome obstacles was to learn how to act in ways that demonstrated leadership and/or exuded the confidence needed to succeed in academic medicine. These included learning to be strategic in their careers, be resilient, and navigate social norms. Participants also discussed how important it is for women to advocate for one another.

Participants shared how they strategized to get where they were. This included knowing what you wanted: “Find what motivates you and then go for it. Totally unabashedly. Go for it.” (F1). This also included learning to “gracefully self-promote” themselves (F4) to share their accomplishments. Others mentioned being comfortable with themselves: “embrace…knowing who you are and being authentic to who you are. Don’t try on somebody else’s thing. And there’s not a type of person that is a leader; the effective people are the ones that know themselves well and are comfortable with being that person in that leadership space” F3.

Resilience was also mentioned multiple times: “roll with the punches instead of feeling flattened and demoralized” (F4). Others felt being challenged was what helped them succeed: “I guess my own resilience and persistence has kind of kept me in the game. I’m also a little rebellious like, ‘You think I can’t do this? Well, let me show you, I actually can’” (F3). Similarly, someone shared: “I think we faced all these different challenges, but as I reflect, I think they’ve made me stronger, more resilient. All these little battles that you have to fight. All these little obstacles on the road, and you kind of just keep plowing through” (F2). Students also said that reflecting on what they had accomplished was helpful: “looking back and see how far you’ve come can be a huge thing. Just remembering where you started and how hard it was just to get into medical school and how hard it was just to get through college…And just remembering that you can do hard things and you’re totally capable” (S1).

Faculty shared how they learned to navigate the social norms. In response to the discussion about constant criticism about how to dress, faculty discussed strategies for not having to make time-consuming decisions about what to wear: “I went through this process where I got rid [of it]… I only wear black, gray, and blue because then I only need black belts and black shoes” (F1). Another participant shared: “I went to Singapore and I got a tuxedo made. And I wore tuxedos for events for a number of years” (F4). Faculty also shared that handling the “whole guilt thing” (F4) related to being a working mother involved prioritizing their children and their well-being. One elaborated: “pay other people to do all of that menial stuff for you, because you will be saner and calmer at the end of the day..If you were trying to do all the other stuff, you’d just be exhausted and frazzled and frustrated. [This way] you’re fulfilling your potential.” (F4).

Finally, participants spoke about the importance of women advocating for each other. One said that hearing about other women’s paths was helpful: “[Someone told me ‘This is going to be hard. There’s no such thing as a good time for when you have kids. There’s no such thing, … you just do it when you’re ready. And that was really reassuring to me…” (F3). Another said having women confidantes was key: “…I had a …situation where incidentally, a male in a position of leadership said, ‘Don’t ask for that. Don’t ask for too much; just be grateful of what you’re getting.’ And I went to some other women colleagues, and they said, ‘Ask for it; you need to make sure you get what you’re worth and ask for these different opportunities in your leadership position,’ and I took the advice I wanted and asked for it.” (F3). Others said women “have to identify and help each other and point out ways to improve and be better so that everybody can reach their maximum potential of where they want to be” (F2).

Discussion

This qualitative study explored the external and internal challenges women medical students and faculty in academic medicine face to ultimately assist in creating a leadership curriculum for women medical students. A curriculum that addresses women’s gendered barriers may more effectively prevent career attrition and foster a break through the “glass ceiling” of academic medicine. Three key themes were identified within the student and faculty discussions: obstacles, support systems, and self-presentation. Obstacles were further subdivided into external and internal factors, with external factors including microaggressions, macroaggressions, few women in existing leadership appointments, and a lack of career role models, whereas internal factors included personal characteristics such as self-doubt and feelings of imposter syndrome. The support systems theme included positive influences such as sponsorship, allyship from men within and outside medicine, and role modeling from women within academia. The final theme, self-presentation, encompassed introspective and extrospective factors such as knowledge of self-identity, clear goals, resilience amid adversity and setbacks, and strategizing how colleagues and society perceive them. Understanding these primary themes and their subdivisions will likely assist medical schools in charting a curriculum for women students.

Obstacles

Overt macroaggressions and microaggressions are public health threats that medical schools and healthcare institutions should eliminate.19 As a multifaceted issue, numerous strategies have been proposed to improve women physicians’ situation. Some of those solutions include creating a clear mission statement that supports pluralism and diversity and providing support programs to minority students.20-22 Partnering with organizations such as TIME’S UP, which aims to improve the safety, power, and equity for all women in healthcare and other organizations, may be one way institutions can demonstrate their commitment to diversity and equity.23 Maintaining a zero-tolerance policy for sexual assault and gender discrimination for all staff members is another useful way institutions can demonstrate allyship.24 Institutions should also consider providing annual or regular training to students, faculty, and staff on recognizing and combating explicit and internal bias.25 Finally, for women students who experience macroaggressions and microaggressions, additional resources that teach coping and communicating skills to combat such events may also be warranted since aggressions will likely stem from both patients and healthcare team members.26

Another barrier to the advancement of women in academic medicine identified by our focus groups was the lack of women role models and mentors. While nearly every academic physician would argue the need for mentorship and coaching for personal and career growth, discussion ensues on maximizing mentoring relationships. As a hot topic in academic medicine, new studies highlight that one size does not fit all for mentoring, especially when it comes to mentoring women.27-29 This sentiment was echoed by participants in this study. Unlike men, women mentees frequently prefer and perform better with women mentors.28,29 For example, a woman mentee may feel more comfortable discussing clothing items for conferences and meetings with a woman mentor. Similarly, a woman mentor might be more approachable to a woman mentee who wants to discuss the struggles of balancing societal, social, and professional expectations and norms. Men and women have also been shown to network differently, which may add further support the importance of having at least one mentor of the same gender.30 Thus, in addition to enforcing equitable hiring practices, institutions should prioritize establishing formal mentoring programs and matching mentees and mentors of similar backgrounds and career goals.27 These formal mentoring relationships would likely benefit medical students, residents, and junior faculty members. Regularly evaluating mentoring relationships is also warranted to ensure mentor-mentee satisfaction and fit.  

As highlighted in the discussion groups, multiple students and faculty members expressed moments of self-doubt that often negatively influenced their actions and career trajectory. These internal obstacles may be best encompassed by the term ‘imposter syndrome’, an experience identified roughly 50 years ago and encompasses feelings of anxiety, undervaluing of one’s career potential, and lack of belonging in the field.31 Imposter syndrome is also linked to poor performance and faulty decision-making, which may produce significant career detriments.32,33 Although students of all backgrounds may experience imposter syndrome to some degree during their training and career, studies show that women are more likely to carry and persist in these feelings. Several argue that this phenomenon originates from women constantly overcoming societal norms and gender discrimination to be perceived as competent.34,35 Many cite that being authoritative, decisive, and loud, especially in high-acuity settings, are generally socially acceptable traits for men but are usually perceived as negative traits for women, who are better perceived when they are agreeable and approachable. However, regardless of the cause of the high incidence of imposter syndrome in women trainees, learning how to address negative internal dialog will likely benefit women aspiring towards academic leadership roles.36 In addition to teaching mindfulness strategies, this obstacle may serve as another motivation to partner women medical students with women faculty members. Witnessing first-hand how women faculty members lead and organize teams may empower women students to lead boldly. Other strategies include peer-to-peer and/or student-to-faculty meetings to create space for discussing failures and emotionally charged events and identifying external and internal areas for improvement.

Support Systems

Sponsorship and allyship were identified as crucial components of the support systems theme in this study and are essential themes commonly identified for career growth in the literature.37-39 However, notwithstanding the importance of sponsorship and allyship, without a broad audience and network, it can be challenging for women to find sponsors who advocate for local and/or national promotion. Additionally, women have been shown to network differently from men, and simply being present at meetings is unlikely to help women foster the kind of relationships needed for promotion.30,39 In a recent multi-institutional study exploring men and women’s experiences with networking within academic medicine, participants of all genders stated that gendered networking practices exist and are catered to men, citing the “boys’ club” as a source of promotion for men and a disadvantage for women.40 Through gendered networking events, men in academic medicine obtain research opportunities, letters of recommendation, and contact with individuals in prominent leadership positions. These advantages increase professional accomplishments and, subsequently, more opportunities for promotion and leadership appointment.40 Women, conversely, have fewer networking opportunities and are significantly less likely to ask for professional favors or opportunities from those within their network in fear that it would be unethical to leverage a personal relationship for professional advancement.30,39,40 There are a few avenues for combatting this phenomenon. Individuals in positions of power should strive to host inclusive, gender-neutral activities and be cognizant of all voices when making decisions that will impact group practice. When leadership positions open, all faculty should be notified of the opportunity. Individuals in leadership positions should also be aware of how they sponsor women sponsees. For example, gender bias is common in reference letters and when colloquially describing women sponsorees.41,42 Program and section leaders should strive to avoid describing women’s physical attributes or gendered adjectives and instead use agentic descriptors and focus on highlighting products and results. Institutions can also demonstrate allyship broadly by employing equitable hiring practices and equal compensation for starting salaries and bonus pay structures. Institutions should also provide protected maternity and paternity leave with each child, daycare resources, and flexibility when childcare services are needed to alleviate child-rearing burden for women students and faculty.43,44 Other strategies include allowing parents to bring their families to work social events and providing child care resources at national meetings.43,44 Women, in turn, should be vocal about their career goals and aspirations. Directly asking leaders and sponsors for endorsement when leadership opportunities arise may also be needed for women trainees and faculty members. Thus, skills-building resources for communication and emotional intelligence would also likely be useful in a leadership curriculum.

Another related topic to allyship and sponsorship highlighted by this study is the importance of male allyship. One female faculty commented the following on how a male leader provided support: “…he texted back to me, ‘You are no longer allowed to have imposter syndrome. Trust me; you’re the real deal.’ … I needed someone else to say that to me, even today, to get me over this hump that like, I don’t belong here in these leadership roles” (F3). This study example echoes the findings from the well-cited article by Pololi et al.,45 which found that women frequently report a lower sense of belonging and fewer, poorer relationships at work.45 Although this sentiment is likely multifactorial in origin, since most academic appointments are held by men, the importance of male allyship is imperative to improve feelings of belonging. It should also be noted that women are not seeking validation or approval from their male colleagues, but are instead seeking support from their professional peers and supervisors.46,47 Creating a supportive community may be especially important for women in fields where men predominantly occupy academic appointments; such as surgical specialties. This was also noted in our study with many women faculty highlighting the importance of male mentors and sponsors in their career.46,47 Other studies have also echoed the importance of this cross-gendered mentoring.48  In addition to backing from male colleagues, women also expressed the importance of allyship within the home. Regarding domestic responsibilities, several female students and faculty members commented that they could better focus on professional goals when their male significant others performed most of the domestic work. These findings echo those of others and support that traditional, gendered home roles are dissolving and are becoming more equal.49,50 These pooled study findings affirm that women who are supported by their peers, leaders, and life partners are more likely to report satisfaction at home and work.

Self-Presentation

            As displayed in this study and others, proficiency in setting short- and long-term goals is crucial to attaining career advancement and overall satisfaction.51,52 While various studies define short- and long-term goals, short-term goals generally comprise action items that can be measurable within 6- to 12 months and work in concert to achieve overarching long-term goals. Interestingly, study participants did not pinpoint how to set effective short- and long-term goals; instead, they pointed out why goals should be set, and focused on the significance of authenticity and self-awareness. Accurately estimating one’s ability and regularly examining one’s motivation have recently been explored as key factors for improving overall resiliency and preventing burnout.53,54 In fact, understanding and reviewing one’s purpose and ‘why’ in medicine is considered so valuable that medical schools and healthcare systems throughout the nation are dedicating educational and training resources to increase self-consciousness in trainees and faculty members alike.55-57 This is not entirely surprising since individuals at the medical school level and beyond generally already have proficiency in setting and achieving goals. In many instances, the leaky pipeline appears more attributable to losing focus on why the goals and lists of tasks were set.58 Thus, with the ever-increasing burdens placed on women medical students and physicians, systems and programs that help keep women’s ‘why’ forefront may be essential to prevent burnout and provide encouragement as they climb the academic ladder. Maintaining a clear vision of self-identity, purpose, and ability will also enable women to appropriately select which academic avenues to pursue and allow more ‘all-in’ focus on what matters most. Minimizing distractions and turning down opportunities that do not contribute to one’s overall ‘why’ or overarching career goals will also promote resiliency and decrease burnout.

            Navigating societal and professional norms can cause significant decision-making fatigue and distress for women physicians. In addition to the previously mentioned solutions, women faculty emphasized the importance of delegating minute tasks and nonessential decisions whenever possible to lighten daily cognitive burdens. Some strategies faculty members recommended include minimizing their professional wardrobes to coordinating pieces, asking other healthcare team members to carry out administrative duties that don’t require a physician-level education to perform, and hiring others to perform household responsibilities. Appropriate delegation of tasks and decisions may greatly reduce women’s pressure to be full-time physicians and caretakers. Learning to delegate also has positive implications for leadership attainment, as efficient delegation is critical to creating effective teams.59 In addition to delegating, societal and professional pressures can also be better managed by supporting and relying on other women in medicine to be mentors and sponsors.60 This point mentioned by women faculty members reiterates the importance of having at least one mentor of the same gender and effectively using one’s network to promote other women in medicine. In addition to providing individual support, solidarity among women faculty and trainees may also serve as a catalyst for improving systemic barriers.60

Limitations

            One study limitation is the inclusion of participants from a single center, which may differ from the experiences of women from other institutions and regions. However, since this study was ultimately performed to assist with the creation of a women medical school leadership curriculum at the Spencer Fox Eccles School of Medicine at the University of Utah, results are significant for study investigators and will likely be useful to others with similar goals.

Conclusion

This exploratory study of women medical students and faculty identified three key themes to remedy the leaky pipeline in academic medicine: obstacles, support systems, and self-presentation. Obstacles were further defined by a lack of mentors and role models, discrimination, and gender bias. Support systems emphasized the importance of mentorship, sponsorship, gender-neutral networking, and male allyship. Self-presentation focused on the gravity of being authentic, keeping one’s purpose forefront while pursuing short- and long-term goals, appropriate delegation of tasks, and solidarity with other women in medicine. These themes and their respective subdivisions are areas for structured improvement for women to circumvent the effects of the leaky pipeline and ultimately break the glass ceiling of academic medicine.

References

  1. Berlingo L, Girault A, Azria E, Goffinet F, Le Ray C. Women and academic careers in obstetrics and gynaecology: aspirations and obstacles among postgraduate trainees – a mixed-methods study. BJOG. 2019;126(6):770-777.
  2. Butkus R, Serchen J, Moyer DV, et al. Achieving Gender Equity in Physician Compensation and Career Advancement: A Position Paper of the American College of Physicians. Ann Intern Med. 2018;168(10):721-723.
  3. Jena AB, Khullar D, Ho O, Olenski AR, Blumenthal DM. Sex Differences in Academic Rank in US Medical Schools in 2014. JAMA. 2015;314(11):1149-1158.
  4. Kane L. Medscape Physician Compensation Report 2019. Medscape Business of Medicine; Apirl 10, 2019 2019.
  5. Colleges. AoAM. Distribution of Chairs by Department, Gender, and Race/Ethnicity, 2015. Washington, DC: AAMC;2015.
  6. Farrugia G, Zorn CK, Williams AW, Ledger KK. A Qualitative Analysis of Career Advice Given to Women Leaders in an Academic Medical Center. JAMA Network Open. 2020;3(7):e2011292-e2011292.
  7. Edmunds LD, Ovseiko PV, Shepperd S, et al. Why do women choose or reject careers in academic medicine? A narrative review of empirical evidence. Lancet. 2016;388(10062):2948-2958.
  8. Helitzer DL, Newbill SL, Morahan PS, et al. Perceptions of skill development of participants in three national career development programs for women faculty in academic medicine. Acad Med. 2014;89(6):896-903.
  9. Richman RC, Morahan PS, Cohen DW, McDade SA. Advancing women and closing the leadership gap: the Executive Leadership in Academic Medicine (ELAM) program experience. J Womens Health Gend Based Med. 2001;10(3):271-277.
  10. Weaver M, Brecha, F., Sreekantaswamy, S., Neville, R., Nguyen, S., Fix, M. WE WILL: Women Empowering Women in Leadership A Model for Facilitating Successful Networking and Community Building: a Poster. 2019, MWIA Centennial Meeting. American Medical Women’s Association.
  11. Jefferson L, Bloor K, Birks Y, Hewitt C, Bland M. Effect of physicians’ gender on communication and consultation length: a systematic review and meta-analysis. J Health Serv Res Policy. 2013;18(4):242-248.
  12. Mehrotra A, Morris M, Gourevitch RA, et al. Physician characteristics associated with higher adenoma detection rate. Gastrointest Endosc. 2018;87(3):778-786 e775.
  13. Tsugawa Y, Jena AB, Figueroa JF, Orav EJ, Blumenthal DM, Jha AK. Comparison of Hospital Mortality and Readmission Rates for Medicare Patients Treated by Male vs Female Physicians. JAMA Intern Med. 2017;177(2):206-213.
  14. Wallis CJ, Ravi B, Coburn N, Nam RK, Detsky AS, Satkunasivam R. Comparison of postoperative outcomes among patients treated by male and female surgeons: a population based matched cohort study. BMJ. 2017;359:j4366.
  15. ACKER J. HIERARCHIES, JOBS, BODIES::A Theory of Gendered Organizations. Gender & Society. 1990;4(2):139-158.
  16. Acker J. Gendering organizational theory. Classics of organizational theory. 1992;6:450-459.
  17. Balmer DF, Courts KA, Dougherty B, Tuton LW, Abbuhl S, Hirshfield LE. Applying the Theory of Gendered Organizations to the Lived Experience of Women with Established Careers in Academic Medicine. Teaching and Learning in Medicine. 2020;32(5):466-475.
  18. Jagsi R, Spector ND. Leading by Design: Lessons for the Future From 25 Years of the Executive Leadership in Academic Medicine (ELAM) Program for Women. Acad Med. 2020;95(10):1479-1482.
  19. Acholonu RG, Cook TE, Roswell RO, Greene RE. Interrupting Microaggressions in Health Care Settings: A Guide for Teaching Medical Students. MedEdPORTAL. 2020;16:10969.
  20. Gomez LE, Bernet P. Diversity improves performance and outcomes. J Natl Med Assoc. 2019;111(4):383-392.
  21. Hilberman E, Konanc J, Perez-Reyes M, Hunter R, Scagnelli J, Sanders S. Support groups for women in medical school: a first-year program. J Med Educ. 1975;50(9):867-875.
  22. Stanford FC. The Importance of Diversity and Inclusion in the Healthcare Workforce. J Natl Med Assoc. 2020;112(3):247-249.
  23. Acholonu R, Mangurian C, Linos E. TIME’S UP Healthcare: Can we put an end to gender inequality and harassment in medicine? BMJ. 2019;364:l987.
  24. Halkitis PN, Alexander L, Cipriani K, et al. A Statement of Commitment to Zero Tolerance of Harassment and Discrimination in Schools and Programs of Public Health. Public Health Rep. 2020;135(4):534-538.
  25. Sabin J, Guenther G, Ornelas IJ, et al. Brief online implicit bias education increases bias awareness among clinical teaching faculty. Med Educ Online. 2022;27(1):2025307.
  26. Sakowski SA, Feldman EL, Jagsi R, Singer K. Energizing the Conversation: How to Identify and Overcome Gender Inequalities in Academic Medicine. J Contin Educ Health Prof. 2020;40(4):274-278.
  27. Myers PL, Amalfi AN, Ramanadham SR. Mentorship in Plastic Surgery: A Critical Appraisal of Where We Stand and What We Can Do Better. Plast Reconstr Surg. 2021;148(3):667-677.
  28. Loethen J, Ananthamurugan M. Women in Medicine: The Quest for Mentorship. Mo Med. 2021;118(3):182-184.
  29. Shen MR, Tzioumis E, Andersen E, et al. Impact of Mentoring on Academic Career Success for Women in Medicine: A Systematic Review. Acad Med. 2022;97(3):444-458.
  30. Szell M, Thurner S. How women organize social networks different from men. Sci Rep. 2013;3:1214.
  31. Gomez-Morales A. Impostor Phenomenon: A Concept Analysis. Nurs Sci Q. 2021;34(3):309-315.
  32. Bravata DM, Watts SA, Keefer AL, et al. Prevalence, Predictors, and Treatment of Impostor Syndrome: a Systematic Review. J Gen Intern Med. 2020;35(4):1252-1275.
  33. Cader FA, Gupta A, Han JK, et al. How Feeling Like an Imposter Can Impede Your Success. JACC Case Rep. 2021;3(2):347-349.
  34. Heise L, Greene ME, Opper N, et al. Gender inequality and restrictive gender norms: framing the challenges to health. Lancet. 2019;393(10189):2440-2454.
  35. Stamarski CS, Son Hing LS. Gender inequalities in the workplace: the effects of organizational structures, processes, practices, and decision makers’ sexism. Front Psychol. 2015;6:1400.
  36. Ludwig DS, Kabat-Zinn J. Mindfulness in medicine. JAMA. 2008;300(11):1350-1352.
  37. Ayyala MS, Skarupski K, Bodurtha JN, et al. Mentorship Is Not Enough: Exploring Sponsorship and Its Role in Career Advancement in Academic Medicine. Acad Med. 2019;94(1):94-100.
  38. Keating JA, Jasper A, Musuuza J, Templeton K, Safdar N. Supporting Midcareer Women Faculty in Academic Medicine Through Mentorship and Sponsorship. J Contin Educ Health Prof. 2022;42(3):197-203.
  39. Levine RB, Ayyala MS, Skarupski KA, et al. “It’s a Little Different for Men”-Sponsorship and Gender in Academic Medicine: a Qualitative Study. J Gen Intern Med. 2021;36(1):1-8.
  40. Murphy M, Callander JK, Dohan D, Grandis JR. Networking practices and gender inequities in academic medicine: Women’s and men’s perspectives. EClinicalMedicine. 2022;45:101338.
  41. Filippou P, Mahajan S, Deal A, et al. The Presence of Gender Bias in Letters of Recommendations Written for Urology Residency Applicants. Urology. 2019;134:56-61.
  42. Khan S, Kirubarajan A, Shamsheri T, Clayton A, Mehta G. Gender bias in reference letters for residency and academic medicine: a systematic review. Postgrad Med J. 2023;99(1170):272-278.
  43. AM AL, Gosling CM, Khasawneh E, McKenna L, Williams B. Challenges Faced by Female Healthcare Professionals in the Workforce: A Scoping Review. J Multidiscip Healthc. 2020;13:681-691.
  44. Newman C, Templeton K, Chin EL. Inequity and Women Physicians: Time to Change Millennia of Societal Beliefs. Perm J. 2020;24:1-6.
  45. Pololi LH, Jones SJ. Women faculty: an analysis of their experiences in academic medicine and their coping strategies. Gend Med. 2010;7(5):438-450.
  46. Rimmer A. Senior male leaders must act as allies to help women succeed, says report. BMJ. 2019;364:l1099.
  47. Sinha MS, Dzara K, Mueller SK. How Male Allies Can Support the Advancement of Women in Academic Medicine. Acad Med. 2022.
  48. Bickel J. How men can excel as mentors of women. Acad Med. 2014;89(8):1100-1102.
  49. Isaac C, Petrashek K, Steiner M, Manwell LB, Byars-Winston A, Carnes M. Male Spouses of Women Physicians: Communication, Compromise, and Carving Out Time. Qual Rep. 2013;18:1-12.
  50. Graham EM, Ferrel MN, Wells KM, et al. Gender-based Barriers to the Advancement of Women in Academic Emergency Medicine: A Multi-Institutional Survey Study. West J Emerg Med. 2021;22(6):1355-1359.
  51. Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol. 2002;57(9):705-717.
  52. Steinmann B, Klug HJP, Maier GW. The Path Is the Goal: How Transformational Leaders Enhance Followers’ Job Attitudes and Proactive Behavior. Front Psychol. 2018;9:2338.
  53. Hashem Z, Zeinoun P. Self-Compassion Explains Less Burnout Among Healthcare Professionals. Mindfulness (N Y). 2020;11(11):2542-2551.
  54. Monroe C, Loresto F, Horton-Deutsch S, et al. The value of intentional self-care practices: The effects of mindfulness on improving job satisfaction, teamwork, and workplace environments. Arch Psychiatr Nurs. 2021;35(2):189-194.
  55. Benbassat J, Baumal R. Enhancing self-awareness in medical students: an overview of teaching approaches. Acad Med. 2005;80(2):156-161.
  56. Tokumasu K, Obika M, Obara H, Kikukawa M, Nishimura Y, Otsuka F. Processes of increasing medical residents’ intrinsic motivation: a qualitative study. Int J Med Educ. 2022;13:115-123.
  57. Souba WW. Academic medicine and the search for meaning and purpose. Acad Med. 2002;77(2):139-144.
  58. Linnenbrink-Garcia L, Perez T, Barger MM, et al. Repairing the Leaky Pipeline: A Motivationally Supportive Intervention to Enhance Persistence in Undergraduate Science Pathways. Contemp Educ Psychol. 2018;53:181-195.
  59. Linney BJ. The art of delegation. Physician Exec. 1998;24(1):58-61.
  60. Sharma M, Rawal S. Women in Medicine: The Limits of Individualism in Academic Medicine. Acad Med. 2022;97(3):346-350.
by Benjamin Drum, MD, PhD ↓; Bair Diamond, MD ↓; Rachel Hess, MA ↓; Matthew Szadkowski, MD ↓ & Matthew Steimle, DO ↓

Increased Pediatric Resident Confidence in Point-of-Care Ultrasound after an Ultrasound Elective

Abstract

Introduction
While point of care ultrasound (POCUS) education has been demonstrated as a need in pediatric residency programs from a resident and program director perspective, few pediatric residency programs integrate POCUS education into their curricula.

Methods
We designed and implemented a two week-long pediatric elective designed to increase resident confidence in cardiac, lung, and abdominal POCUS examinations. We evaluated residents’ confidence using a Likert scale survey before the elective, after the elective, and at least 18 months after completion of the elective.

Results
Globally, resident confidence across all POCUS domains increased significantly and persisted for at least eighteen months after elective completion. Qualitative analysis of course feedback revealed resident perception of strength in POCUS training with opportunities for expanding the elective.

Discussion
This study of a novel POCUS curriculum in a pediatric residency program demonstrates increased confidence with POCUS skills which persisted post-residency graduation. The described curriculum could serve as a model for other programs and institutions seeking to incorporate an efficient method of teaching this important skill.

Introduction

Point-of-care ultrasound (POCUS) is a growing field in medical management and education, and several specialties including emergency medicine, internal medicine, and general surgery have integrated POCUS into resident training. [1-4]. POCUS uses no radiation, can be done at the bedside, and is generally well tolerated, making it an ideal imaging modality for pediatrics. However, in pediatric residency programs, utilization varies widely between institutions [5]. Despite strong evidence displaying POCUS’s benefit in routine conditions, as well as a belief among a majority of surveyed residency programs that residents should be trained in POCUS, POCUS education has had minimal implementation at most pediatric residency programs of emergency departments and ICUs. [6-7]. Among the many utilities for POCUS in pediatric patients, strong evidence exists for its use to rule in intra-abdominal injury (IAI), to diagnose pneumonia and pneumothorax, and to identify patients with bronchiolitis with severe airway disease and likely prolonged oxygen requirements, all with equivalency to (and in some cases superiority over) more traditional imaging modalities [8-12].

The AAP has acknowledged POCUS’s utility, specifically as a useful method of ruling in pathology [13]. Due to the lack of established clinical guidance around POCUS in pediatrics, a group of European and North American pediatric intensivists and cardiologists convened to compose evidence based guidelines for critically ill children, and the 20 expert panelists strongly agreed on over 20 recommendations including that POCUS has utility in detection of pneumonia and pleural effusion, multiple line placements and procedures, as well as monitoring conditions such as acute respiratory distress syndrome (ARDS) [14].

Currently, few pediatric POCUS curricula have been published with data supporting its efficacy. Brant et al. at the University of Colorado pediatrics residency program devised a curriculum for all interns comprised of three half-days with both didactics and hand-on application [15]. Comparing pre- and post-course test results revealed a statistically significant increase in scores, and the participants’ post-course objective structured clinical exam (OSCE) scores demonstrated high proficiency in obtaining and interpreting key POCUS views on the FAST exam. Recently, Sabnani et al developed a longitudinal, multimodal POCUS curriculum for pediatric residents consisting of an online question bank combined with in-person learning sessions [16]. Evidence of comparably structured curricula exists sub-specialty pediatric; however, the data gathered is similarly promising [17-19].

Our goal was to develop a consolidated, easily replicable and efficient POCUS curriculum to increase confidence with and proficiency of POCUS skills, primarily within the pediatric inpatient setting.

Methods

Personnel and Location
This curriculum was devised by two pediatric emergency medicine (PEM) physicians who work primarily at a large, tertiary academic children’s hospital, and who regularly work with pediatric residents. The curriculum was designed as an elective, available to second- and third-year residents who expressed interest in POCUS, and whose planned career paths would benefit from a working knowledge of POCUS (i.e., PEM, critical care, neonatal critical care, hospital medicine, and rural general pediatrics). This elective was first offered to pediatric residents in 2017, with a total of 10 participating residents to date, 7 of which completed all the surveys making up our data set (3 residents did not complete the post-survey). Upon graduation, the residents largely went into careers that could incorporate POCUS (4 pediatric emergency medicine, 1 pediatric hospital medicine, 1 global health, 2 child abuse).

Course Curriculum
In the first week of the elective, residents completed online didactics including a structured series of video modules and associated multiple-choice quiz evaluations through an POCUS education program. Topics included ultrasound basics as well as cardiac and extended focused assessment with sonography in trauma (eFAST) exams. There were no formal lectures or didactics during the second elective week to maximize faculty efforts in direct scanning sessions.

The second elective week started with an initial simulator session using an advanced US-capable medical manikin. Residents completed 10 cardiac and 10 eFAST cases on a high-fidelity manikin, guided by a PEM US faculty member. The remainder of the hands-on portion took place during 3-4 hands-on sessions in the Primary Children’s Hospital Emergency Department, under the supervision of the previously mentioned PEM faculty, who had POCUS privileges. Scans were performed on patients with consenting families, who were identified by ED staff as clinically stable and accommodating enough to allow an educational scan, and all scanning sessions lasted two hours in duration. If any pathology was found or suspected, it was saved into the picture archive and communication system (PACS) and documented into the EMR. The provider taking care of the patient was informed so they could process this information into the patients care and make any additional follow up recommendations. Additionally, all educational scans were saved in the machine for 1-2 months. Each scan was also logged on paper and scanned into the residents’ electronic academic file for purposes of the elective. Residents were expected to attend a quality assurance session reviewing scans with faculty during their week. Creating and presenting a five-to-ten-minute presentation on a POCUS interesting case or topic found in the literature was the final component of the course.

Ultrasound Indications
Students were taught lung views to assess for pleural effusion, pneumothorax, and A line vs. B line profile, cardiac views of parasternal long, parasternal short, apical four chamber, and sub-xyphoid to assess for gross cardiac function and pericardial effusion, IVC view to assess volume status, and abdominal views of right upper quadrant, left upper quadrant, and pelvis to assess for free fluid and bladder volume. Students also had the opportunity to practice ultrasound needle guidance using a Blue Phantom task trainer made of SimulexUS tissue. These indications were aligned with national standards for POCUS, adapted for the resident level [13-14].

Course Objective
The course administrators’ goal was to increase confidence and competency of the above POCUS indications.

Survey Data Collection
A 15-item Likert scale survey (Table 1) was utilized to assess confidence in the specific POCUS skills and indications learned during the course. This survey was administered before (pre-), immediately after (post-), and at least 18 months after completion of the course and 6 months after the resident had completed residency training (18 m). To obtain the post-residency timepoint, residents were contacted by last phone or email on record with brief explanation of the study and asked for permission to send the survey.

To obtain qualitative data, residents also completed an exit survey immediately following the elective exploring resident thoughts about the structure, content, and suggestive improvements for the course.

Survey Data Analysis
Given the non-parametric nature of our data with a Likert scale survey, data was analyzed using medians both at the individual question and globally by using the median score of all fifteen Likert scale questions per participant. Significance was analyzed using the Wilcoxon paired match test, applied between pre- and post- data, pre- and 18-m data, and post- and 18-m data. A p value of <0.05 was used as a cut off for significant difference.

Qualitative Analysis
Survey data was grouped into topics and analyzed using a thematic coding system in which post-course survey responses were coded based on frequency and emphasis that respondents placed on answers to survey questions [20]. Topics emerged from the analysis of the qualitative data. Following the final round of coding, the codes were classified into overarching themes in preparation for interpreting the data.

IRB Approval
This study was approved by the University of Utah under IRB study 00157801.

Results

Resident confidence across all POCUS domains assayed in the Likert survey increased after the course with pre-survey question median = 2.0 and interquartile range = 1.0 compared to post-survey median 4.0 and interquartile range = 0.5, (n = 7, p = 0.016, response rate 100%). In addition, this increase persisted for at least eighteen months after elective completion with 18-month survey median = 3.0 and interquartile range = 1.5 (n = 7, p = 0.031 compared to pre-survey and p = 0.99 compared with post-survey, response rate 100%) (Figure 1).

In addition, 14 of the 15 questions were also found to be significantly improved after the elective with persistent increase in 13 questions eighteen months after elective completion. The only question which was not found to have a significant increase after the elective was “I am confident finding and obtaining vascular access with POCUS”. Notably residents did have time with a vein simulator but did not have the opportunity to place any peripheral or central access during the elective. This question was also found to be non-significant at the six-month survey time point, in addition to the question “I am confident in my ability to diagnose shock with POCUS” (Table 2).

Our analysis of post-survey responses yielded two major themes. These include: (1) strengths of the elective and (2) participant derived proposals for curriculum development. Descriptive quotations for these themes are included below.

Strengths of the elective were described in the following ways:

  • “I feel way more confident in my abilities at the end of the course then I would have ever thought. Also I feel like I know so much more about when to use ultrasound and how amazing it can be in practice.”
  • “I appreciated that all scanning was done with a faculty member, and that I received real- time feedback on how to improve the quality of my images”

Participant derived proposals for curriculum development include the following sample responses:

  • “Could be cool to follow the sonographers around to get a few more hours of scanning and see how people who do US all the time may do things differently.”
  • “More time scanning, less time to complete video lectures…Maybe two more days for scanning and only 3 days for the video lectures”

Discussion

Our study highlights that a short elective, which could be implemented with minimal resources, can have a lasting impact on pediatric residents’ confidence in performing POCUS. Furthermore, resident confidence in ultrasound skills persists after completion of residency. Our elective has attempted to minimize faculty time through the use of online resources, does not require purchase of handheld ultrasounds for the residents, and can fit seamlessly into the current educational structure of most pediatric residency programs.

This study highlights the importance of designing measurable education data at specific time points with POCUS education and training; at the onset of training, after training and at delayed time points, to measure persistence of confidence in performing POCUS. A hybrid model of online education coupled with simulated training and real patient scanning with a skilled instructor achieves long-term resident retention and maintains a 1:1 student/teacher ratio, which residents qualitatively found beneficial, while minimizing faculty resources. Studies involving POCUS education and knowledge retention are needed to determine the most useful data to collect in terms of measuring retention of POCUS knowledge and skill as well as timing of checking retention of skill to maintain POCUS mastery and competency.

Limitations
Several limitations were present in designing and executing this curriculum and study. This study is limited by small sample size and was conducted at a single institution; therefore, it is not clear if this model is broadly generalizable. With a very small number of POCUS-privileged attending physicians, the course was instructor intensive and dependent on their schedules which restricted the number of residents able to participate. As a result, our data was collected over several years of offering the elective in order to obtain the sample size, and it is possible that the sample is heterogeneous given temporal separation. Currently our institution does not offer dedicated funding directed to a resident-oriented POCUS program. As a result, our assessment methods are also limited to resident surveys rather than more formalized skills-based assessments of resident skill.

Using online self-guided didactic material prior to the elective did limit the amount of faculty non-scanning time needed to implement this elective and reduced teaching commitment to only one week; however, it also limited the customization of the curriculum. With more resources available, we would explore the ideal instructor to student ratio (currently 1:1) as well as assess the efficacy of in-person, live-taught didactics versus online, self-directed modules. Faculty teaching time was roughly 9 hours per elective, spread between 3 different faculty equaling approximately 3 hours per resident. While not a major time commitment, this is still a significant demand to ask of faculty when it is unfunded time.

Next Steps
We are currently exploring several next steps. As our study only explores resident confidence, next steps including changing from the confidence survey to a competency-based quiz in alignment with the Kirkpatrick model of evaluation. Increasing the amount of PEM US faculty would further dilute the teaching requirement and may also allow expansion of the elective to more residents, although faculty funding remains a significant barrier. In terms of scheduling, our residency has recently adopted an “X+Y” format with built-in elective weeks every fourth week. Future work will explore partnering with other US departments (PICU, PHM, NICU) to try and develop a longitudinal curriculum which could occur in resident elective weeks. Incorporating other departments could even allow for specialization of the longitudinal experience based on the resident’s intended career path.

Conclusion

This novel two-weeklong intensive elective offered to second and third year pediatric residents increased resident confidence in both obtaining and interpreting ultrasound images in several cardiac and eFAST indications. While other electives have been implemented in pediatrics, this is the first which is only two weeks in duration. In addition, this is the only study we have found which shows persistence in ultrasound confidence for at least eighteen months after elective completion and at least six months after the completion of residency. Thus, despite its short duration, this course appears to be effective in its POCUS content delivery, and could serve as a guide for other pediatric programs to implement ultrasound into their residency curricula.

References

  1. McGahan JP, Pozniak MA, Cronan J, et al.: Handheld ultrasound: Threat or opportunity?. Applied Radiology. 2015, 20-25. doi: 10.37549/AR2166
  2. American College of Emergency Physicians. Ultrasound Guidelines: Emergency, Point-of-Care and Clinical Ultrasound Guidelines in Medicine. Ann Emerg Med. 2017;69:e27-e54. doi: 10.1016/j.annemergmed.2016.08.457
  3. Galen B, Conigliaro R. The Montefiore 10: a pilot curriculum in point-of-care ultrasound for internal medicine residency training. J Grad Med Educ. 2018;10:110-111. doi: 10.4300/JGME-D-17-00683.1
  4. Beal E, Sigmond B, Safe-Silski L, Lahey S, Nguyen V, Bahner D. Point-of-care ultrasound in general surgery residency training. J Ultrasound Med. 2017;36:2577–2584. doi: 10.1002/jum.14298
  5. Lee L, DeCara JM. Point-of-Care Ultrasound. Review Paper. Current Cardiology Reports. 2020;22(11):1-10. doi:10.1007/s11886-020-01394-y
  6. Good R, O’Hara K, Ziniel S, Orsborn J, Cheetham A, Rosenberg A. Point-of-Care Ultrasound Training in Pediatric Residency: A National Needs Assessment. Hospital pediatrics. 2021;11(11): 1246-1252. doi:10.1542/hpeds.2021-006060
  7. Gutierrez P, Berkowitz T, Shah L. et al. Taking the Pulse of POCUS: THe State of Point-of-Care Ultrasound at a Pediatric Tertiary Care Hospital. POCUS Journal 2021; 6(2):80-87. doi: 10.24908/pocus.v6i2.14781
  8. Liang T, Roseman E, Gao M, Sinert R. The Utility of the Focused Assessment With Sonography in Trauma Examination in Pediatric Blunt Abdominal Trauma: A Systematic Review and Meta-Analysis. Pediatric Emergency Care. 2021;37(2):108-118. doi:10.1097/PEC.0000000000001755
  9. Alzahrani SA, Al-Salamah MA, Al-Madani WH, Elbarbary MA. Systematic review and meta-analysis for the use of ultrasound versus radiology in diagnosing of pneumonia. Critical ultrasound journal. 2017 Dec 2017;9(1):6. doi:10.1186/s13089-017-0059-y
  10. Shah V, Tunik M, Tsung J. Prospective evaluation of point-of-care ultrasonography for the diagnosis of pneumonia in children and young adults. JAMA pediatrics. 2013;167(2):119-125. doi:10.1001/2013.jamapediatrics.107
  11. Alrajab S, Youssef AM, Akkus NI, Caldito C. Pleural ultrasonography versus chest radiography for the diagnosis of pneumothorax: review of the literature and meta-analysis. Critical care 2013;17(5): R208. doi:10.1186/cc13016
  12. Hopkins A, Doniger S. Point-of-Care Ultrasound for the Pediatric Hospitalist’s Practice. Hospital pediatrics. 2019;9(9):708-718. doi:10.1542/hpeds.2018-0118
  13. Le Coz J, Orlandini S, Titomanlio L, Rinaldi V. Point of care ultrasonography in the pediatric emergency department. Italian journal of pediatrics. 2018;44(1):87. doi:10.1186/s13052-018-0520-y
  14. Singh Y, Tissot C, Fraga M, et al. International evidence-based guidelines on Point of Care Ultrasound (POCUS) for critically ill neonates and children issued by the POCUS Working Group of the European Society of Paediatric and Neonatal Intensive Care (ESPNIC). Critical care. 2020;24(1):65. doi:10.1186/s13054-020-2787-9
  15. Brant J, Orsborn J, Good R, Greenwald E, Mickley M, Toney A. Evaluating a longitudinal point-of-care-ultrasound (POCUS) curriculum for pediatric residents. BMC medical education. 2021;21(64). doi:10.1186/s12909-021-02488-z
  16. Sabnani R, Willard CS, Vega C, Binder ZW. A longitudinal evaluation of a multimodal POCUS curriculum in pediatric residents. POCUS Journal 2023;8(1):65-70. doi:10.24908/pocus.v8i1.16209
  17. Good R, Orsborn J, Stidham T. Point-of-Care Ultrasound Education for Pediatric Residents in the Pediatric Intensive Care Unit. MedEdPORTAL : the journal of teaching and learning resources. 2018;14: 10683. doi:10.15766/mep_2374-8265.10683
  18. Bhargava V, Haileselassie B, Rosenblatt S, Baker M, Kuo K, Su E. A point-of-care ultrasound education curriculum for pediatric critical care medicine. The ultrasound journal. 2022;14(44). doi:10.1186/s13089-022-00290-6
  19. Harel-Sterling M, McLean L. Development of a blended learning curriculum to improve POCUS education in a pediatric emergency medicine training program. CJEM. 2022;24(3):325-328. doi:10.1007/s43678-022-00264-6
  20. Knox S, Schlosser L. Consensual qualitative research: A practical resource for investigating social science phenomena. Claire E Hill (ed): 2012.

Figures & Tables

Figure 1: Median Likert scale of each participant across all fifteen questions at the time points of pre (prior to elective), post (immediately after elective), and 18 months (at least 18 months after completion of elective). Each participant is connected with a line across each time point (n = 7). P-values: pre- to post- = 0.016, post- and 18 m = 0.99, pre and 18 m = 0.031.

For each of the following statements, please mark the number that best corresponds to your response:

 Strongly DisagreeDisagreeNeutralAgreeStrongly Agree
I am confident adjusting the basic knobs such as gain and depth12345
I am confident at choosing the correct probe for a given patient and purpose12345
I am confident finding and obtaining vascular access with POCUS12345
I am confident at performing either a FAST exam and/or an EFAST exam12345
I am confident that I can diagnose pericardial effusion with POCUS12345
I am confident that I can diagnose tamponade with POCUS12345
I am confident that I can diagnose a pneumothorax with POCUS12345
I am confident that I can diagnose fluid in the peritoneum such as hemoperitoneum or ascites with POCUS12345
I am confident in my ability to obtain basic cardiac views with POCUS12345
I am confident in my ability to assess left ventricular function with POCUS12345
I am confident in my ability to perform a thoracic examination with POCUS12345
I am confident in my ability to evaluate right ventricle function and volume overload with POCUS12345
I am confident in my ability to evaluate volume responsiveness with POCUS12345
I am confident in my ability to diagnose shock with POCUS12345
I am confident in my ability to acquire images, interpret them, and clinical integration using POCUS: “putting it all together” 12345
Table 1: Likert scale survey
QuestionsPre-Post-18 mpre/post p-valuepost/18m p-valuepre/18m      p-value
Q1. I am confident adjusting the
 basic knobs such as gain and depth		1540.0160.2500.016
Q2. I am confident at choosing the correct probe for a given patient and purpose1540.0160.5000.016
Q3. I am confident finding and obtaining vascular access with POCUS2330.0631.0000.125
Q4. I am confident at performing either a FAST exam and/or an EFAST exam2430.0160.3130.016
Q5. I am confident that I can diagnose pericardial effusion with POCUS2440.0161.0000.031
Q6. I am confident that I can diagnose tamponade with POCUS2440.0161.0000.031
Q7. I am confident that I can diagnose a pneumothorax with POCUS2430.0160.3750.031
Q8. I am confident that I can diagnose fluid in the peritoneum such as hemoperitoneum or ascites with POCUS2430.0161.0000.016
Q9. I am confident in my ability to obtain basic cardiac views with POCUS2440.0160.8130.031
Q10. I am confident in my ability to assess left ventricular function with POCUS1430.0161.0000.016
Q11. I am confident in my ability to perform a thoracic examination with POCUS2430.0160.7500.031
Q12. I am confident in my ability to evaluate right ventricle function and volume overload with POCUS1330.0160.5310.031
Q13. I am confident in my ability to evaluate volume responsiveness with POCUS1340.0160.7810.016
Q14. I am confident in my ability to diagnose shock with POCUS1320.0160.5310.063
Q15. I am confident in my ability to acquire images, interpret them, and clinical integration using POCUS: “putting it all together”1430.0160.2500.031
Table 2: Data for each Likert scale POCUS question.
Medians of the responses of the participants to each question are reported at the time points of prior to the course (pre-), immediately after the course (post-), and at least 18 months after completion of the elective (18 m) along with p-values calculated using the Wilcoxon paired values test (n = 7 for all data points).
by Bryan ↓

2023 Journal of the Academy of Health Sciences: A Pre-Print Repository

by Angela Coker, MD ↓; Deirdre Caplin, PhD. MS ↓; Brian F. Flaherty, MD ↓; Stefanie Pease-Romero, MSN, RN, CHSE ↓; Wendy Dahmen, MSN-Ed, RN ↓; Angela P. Presson, PhD, MS ↓; Zhining Ou, MS ↓ & Jared W. Henricksen, MD, MS-HPEd ↓

Simulation Stress: Positive Challenge or Negative Threat?

Introduction

Simulation-based education (SBE) is used to teach healthcare providers clinical knowledge, communication skills, crisis resource management, and team dynamics [1–4]. However, SBE may be stressful to participants as it exposes educational gaps and often requires performance in front of peers [5,6]. Stress can enhance learning when perceived as positive and challenging or impair learning when perceived as negative and threatening [7–10]. While stress may be inherent in SBE, it remains largely unknown how participants experience stress, which participants are at higher risk of negative stress, and what the best practice is for SBE facilitators to manage participants’ stress. A better understanding of SBE-related stress is needed to identify participants at risk for negative stress and to guide SBE facilitators to prioritize emotional recovery or learning during simulation sessions.

One approach to understanding the impact of stress is examining participants’ self-appraisal of stress known as cognitive appraisal. Individuals appraise a situation in two ways: 1) as a challenge if conscious and subconscious resources outweigh the demands of a task at hand, or 2) as a threat if the demands are greater than available resources [11–15]. Challenge appraisal is associated with benefits likely to impact SBE participants’ learning in a positive way such as increased positive affect and effective adaptive physiologic responses [16,17]. Threat appraisal is likely to negatively impact SBE participants’ learning experience and is incongruent with the principles of psychological safety in simulation environments [18–22].

In this prospective cohort study, we will assess challenge and threat cognitive appraisal among SBE participants before and after a simulation scenario. We will evaluate SBE participant factors associated with their cognitive appraisal including demographics, prior SBE experience, global perceived stress, and baseline risk for anxiety disorders. We hypothesize that most SBE participants will appraise their stress as a challenge rather than a threat, where participants earlier in their career or training and those with less simulation experience being more likely to appraise their stress as a threat. We will also examine whether SBE facilitators are able to identify participants who experience SBE as a threat by using a psychological distress identification tool [5]. We hypothesize that participants assigned higher psychological distress scores will be more likely to appraise their stress as a threat.

Methods

This study was approved by the University of Utah institutional review board. Simulation participants at the Primary Children’s Hospital Simulation Center going through their regularly-scheduled simulation training sessions between March to September 2020 were recruited to take surveys before and after one of their simulation events as part of this study. Most sessions were multidisciplinary and included practicing nurses, patient care technicians, respiratory therapists, and providers such as an attending physician, fellow, resident, nurse practitioner (NP), or physician’s assistant (PA). Some sessions had only nurses, and there were no student participants in this study. The sessions were formative and confidential. All two-hour simulation sessions consisted of a prebrief, a short brief before two simulation scenarios with high-fidelity mannikins each followed by a debrief, and a summary at the end. The sessions were facilitated by Intermountain Health-certified simulation facilitators with a simulation specialist present to operate the mannikin. An unrestricted variety of simulation scenarios were used with variable learning objectives, all chosen from a scenario library by the facilitator to suit the needs of the participant group which ranged from novice to expert. All participants were shown a consent cover letter indicating participation in the study was voluntary prior to accessing the study surveys. Survey responses were collected via REDCap, an internet-based database and survey tool [23]. Surveys were accessed by a QR code linking to REDCap where participants entered a simulation session event number to identify simulation type, and an individual study number assigned to them when they entered the simulation center to keep participant responses anonymous.

Participants were surveyed at two time points: 1) after the session prebrief and before the first scenario brief, and 2) after the first scenario was complete and before the first debrief. The pre-scenario survey included questions about demographics, healthcare role, prior simulation experience, and the following three surveys: the National Institute of Health (NIH) Perceived Stress Scale (PSS), the Generalized Anxiety Disorder screening tool (GAD-7), and the Acute Stress Appraisal survey (ASA). See appendix A to view these study surveys.

  1. NIH PSS: This survey quantifies the degree of perceived stress over the past month by assessing how much control respondents feel they have over their lives. For this 10-item survey, an uncorrected T-score of ≤40 indicates low levels of stress while a score ≥60 suggests high levels of perceived stress [24,25].
  2. GAD-7: This survey quantifies baseline anxiety through a 7-item survey assessing anxiety symptoms over 2 weeks to measure severity of symptoms and risk of having an anxiety disorder. A total score of 0-4 indicates no anxiety disorder, 5-9 indicates a mild anxiety disorder, 10-14 a moderate anxiety disorder, and more than 14 indicates a severe anxiety disorder [26,27].
  3. ASA: The Acute Stress Appraisal survey (ASA) determined the cognitive appraisal ratio of threat or challenge and was used with permission from the developer, Wendy Mendes, PhD of University of California San Francisco [28]. This survey is commonly used in psychology research and asks respondents about their capability to handle a stressful task. The survey has two parts with 12 questions answered before a task is completed (ASA pre) and 10 questions answered after the task is done (ASA post) to assess the demands on the participant and resources available to the participant before and after task completion. A threat ratio is then calculated by dividing the demand score by the resource score. A threat ratio >1 indicates the stress associated with a task is threatening (demands exceed resources) while a ratio ≤1 indicates the stress associated with a task is challenging (resources exceed demands). Each participant filled out the ASA pre that was part of the first survey before the brief of the first simulation scenario, and filled out the ASA post in the second survey after the scenario was complete and before the debrief was done.

Session facilitators were all clinical personnel who regularly facilitate simulation sessions, but none were psychologists or psychiatrists. At the end of the simulation session, the session facilitator assigned each participant a psychological distress level (PDL). This level was determined by using the Simulation Psychological Distress Algorithm, a measure designed by Henricksen, et al. to assist facilitators in recognizing and beginning to assist any simulation participant who may be in a psychologically distressed state [5]. The algorithm has four levels of distress: 0 (none), 1 (mild), 2 (moderate), and 3 (severe). Levels are based on facilitator-observed behavior that may indicate psychological distress such as participant withdrawal from the group, tearfulness, anger or raised voice. The PDL was matched to participant surveys in REDCap by the assigned individual study number. Every facilitator scored simulation participants on the PDL scale at the time of the simulation session and were blinded to all participant survey results such as baseline anxiety levels and stress appraisal. Each facilitator had immediate access to select study authors (Coker and Dahmen) to collaborate regarding PDL scoring of each participant to minimize inter-rater scoring differences. Each session only had one facilitator.

Statistical Analysis. Participants were excluded from analysis if they did not complete both the pre and post scenario surveys. Demographics and clinical outcomes of interest were summarized using mean and standard deviation (SD) or median and interquartile range (IQR) for continuous variables. For categorical variables, counts and percentages were reported. Participants were categorized into one of four outcome groups based on the stress state from the ASA pre and post surveys: Challenge/Challenge, Challenge/Threat, Threat/Challenge, and Threat/Threat. Demographic, PSS, and GAD-7 data were compared between these four groups using the Wilcoxon rank sum test for skewed continuous variables and chi-squared or Fisher’s exact test for categorical variables. Univariable logistic regression models evaluated the association between the dichotomized outcome Challenge/Challenge versus the other three combinations (Challenge/Threat, Threat/Challenge, and Threat/Threat) for each variable of interest. Variables with p-value<0.10 were included in a multivariable logistic model. Odds ratios (ORs) and 95% confidence intervals (CIs) were reported. Statistical significance was assessed at the 0.05 level. PDL and ASA scores were compared in a cross-tabulation table using Fisher’s exact tests. Statistical analyses were implemented using R v. 4.0.3 (R Core Team, 2020).

Results

Five-hundred and fifty-one pediatric healthcare personnel participated in the study. Four hundred eighty-five (485) participants completed both surveys and sixty-six (66) participants completed only one of the surveys and were therefore excluded due to incomplete data. A summary of demographic information, job variables, simulation experience and type, and PSS, GAD-7, and ASA pre/postsurvey scores are shown in Table 1. The majority of participants were nurses (n=347, 71.5%). Forty-three (43) medical trainees participated in the study, and 40% were in their first year of training. The majority (92.2%) of participants had completed at least one simulation session in the past two years. Almost half (48.7%) of simulation scenarios were for non-critical care groups working in the medical/surgical unit, same day surgery center, or post-anesthesia care unit. The vast majority (99.4%) of participants scored 40 or less on the PSS, indicating low perceived stress over the last month (median 32, IQR 30-34). On the GAD-7, 80.3% of participants scored in the “no” or “mild” anxiety disorder range.

The four ASA state groups are compared in Table 2. The majority of participants (n=325, 67.6%) appraised their stress as a challenge before and after the simulation scenario (Challenge/Challenge). Eighty-one participants (16.7%) appraised their stress as a threat before the scenario and a challenge after the scenario (Threat/Challenge). 9.9% and 5.8% were in the Challenge/Threat and Threat/Threat groups, respectively. There was a statistically significant difference between the four ASA state groups when looking at job type, simulation type, years in practice and training, and anxiety disorder scores. There was no statistically significant difference in prior SBE participation, gender, and chronic stress scores between the four groups. Results of logistic regression analysis comparing the Challenge/Challenge group to the other three ASA state groups are shown in Table 3. Univariable logistic regression showed that nurses and assistive personnel (patient care technicians and respiratory therapists) were more likely to appraise their stress as a challenge rather than a threat compared to advanced providers (attending physicians, NPs, and PA’s). Trainees (residents and fellows) were analyzed separately from attending physicians. When compared to advanced providers, assistive personnel were 10 times more likely to be in the Challenge/Challenge group (p<0.001, OR 10.31172[3.15-38.05]), and nurses were 4 times more likely to be in the Challenge/Challenge group (p=0.012, OR 4.09[1.42-13.38]). Simulation type and chronic stress scores were not significant in univariable regression analysis. Prior simulation experience was variably significant, with those who attended three or more simulation sessions in the past two years ultimately being more likely to appraise stress as a challenge in multivariate analysis. In multivariable logistic regression analysis, nurses, assistive personnel, and participants with lower anxiety disorder scores continued to be significantly more likely to appraise stress as a challenge.

Table 4 displays results comparing participant ASA scores with facilitator-assigned PDL scores. Of 23 participants in the Threat/Threat group, one was given a distress score 1 or higher. Thirteen participants were given a distress score 1 or higher out of 282 participants in the Challenge/Challenge group. Overall, there was no association between the PDL and ASA scoring (p>0.99).

Discussion

Our survey study of cognitive appraisal in SBE participants primarily aimed to categorize simulation-induced stress as challenging or threatening and to identify participant and simulation factors predictive of cognitive appraisal. This is the largest and only multidisciplinary study of cognitive appraisal in SBE.

Other studies have used the challenge and threat stress state to characterize simulation participants. Carenzo, et al. showed that residents in a simulation-based competition performed better when they were in a challenge state with a high level of resources until demands increased, shifting towards a threat state [29]. In their study, a greater level of training and higher self-confidence were associated with challenging stress, and state anxiety levels were not. Harvey, et al. showed that threat appraisal is associated with more difficult simulation scenarios and higher salivary cortisol levels [30]. In contrast to these studies, our study did not show that simulation type was significantly associated with the challenge or threat state, perhaps indicating there were participants who appraised their stress as a challenge and threat in every type of simulation despite any level of simulation difficulty. Also, our study used the GAD-7 which measures risk for an anxiety disorder rather than state anxiety levels and we made no attempt to correlate participant physiology to psychology.

This study defined four groups of simulation participants depending on their stress appraisal before and after simulation: Challenge/Challenge, Challenge/Threat, Threat/Challenge, and Threat/Threat. We found that 68% of participants appraised their stress surrounding a simulation scenario as a challenge rather than a threat before and after simulation (Challenge/Challenge group). Nurses and assistive personnel had higher odds of appraising simulation stress as a challenge compared to advanced providers. Lower anxiety disorder screening scores and more simulation experience were also associated with the Challenge/Challenge appraisals.

Advanced providers were less likely to be in the Challenge/Challenge group than nurses and assistive staff. This may be because advanced providers are often expected to take the team leader roles and may have more expectations placed on them during a scenario either by themselves or by other participants. They may worry about a loss of respect from the rest of the multidisciplinary team when knowledge gaps are revealed. Trainees were not significantly different from advanced providers on univariable analysis.

Our findings support our hypothesis that most SBE participants appraise simulation-induced stress as a challenge. Challenge appraisal indicates participants feel they have the experience, knowledge, and skills (i.e., resources) to confront the simulation scenario even if the content is unfamiliar to them. Challenging stress is associated with improved performance and personal growth [13,29].

It seems intuitive that some (17% in our study) who appraise their stress as a threat before simulation were relieved when the simulation was done and appraised their stress as a challenge after simulation (Threat/Challenge group). We recognize that less than 6% of participants appraised their stress as a threat before and after simulation (Threat/Threat group), and some (~10%) appraised their stress a challenge before simulation and switched to threat after simulation (Challenge/Threat group). These last two groups are intriguing and warrant more study.

We further aimed to evaluate facilitators’ abilities to accurately identify participants whose ASA scores would indicate they are in a threatened stress state by having the facilitators assign PDL scores. Contrary to our hypothesis, there was no relationship between PDL and ASA scores. This indicates facilitators are not able to accurately associate observed simulation participant behaviors with a participant’s appraisal of stress. Henricksen, et al previously showed that PDL scores > 0 are rare in simulation, occurring in <1% of participants [5]. In our study, 4% of participants had a PDL of 1 or higher. Yet, despite a higher incidence of PDL scores, there was no correlation to ASA scoring. Unless a participant is having an exceptional emotional response, it may be near impossible to know who is in a threatened state.

Healthcare providers are trained to work in stressful conditions and can internally regulate their emotions while remaining functional in their jobs [31]. What is perceived as distress by a facilitator may be excitement or distraction for a participant. All personnel have impression management skills that can defy facilitators’ skills in detecting most threatening stress [32,33]. Thus arises a debriefing conundrum where most SBE participants are in a challenge state and ready to learn, but a small percentage of participants exist in a hidden threatened state.

We propose that pragmatic debriefing with emotional surveillance be used to help the majority of simulation participants obtain maximal simulation benefit. Most particpants are likely to be in a challenge state and all participants have impression management skills that may defy facilitator identification of psychological distress. Thus, those in a threatened state are hidden amongst simulation participants. By focusing on learning objectives as well as discovering what happened and what consequences of actions occurred during simulation, debriefing can concentrate on improving performance. Being watchful for signs of psychological distress will help every facilitator maintain awareness that simulation is threatening to some participants, and there may be a participant that needs special attention. Monitoring participants for psychological distress and offering assistance when appropriate is what we can do to help those in a threatened state, given the difficulty of identifying them. Incidentally, helping each team member improve may help them psychologically rather than exposing their threatened state to every participant.

This study does have limitations. Generalizability of these results may be limited since this is a single center study. Employees providing direct patient care at Primary Children’s Hospital are required to attend simulation sessions with their units at least once a year and are frequently exposed to the concept of psychological safety. Our institutional culture may make SBE participants more comfortable in the simulation center than in an institution where simulation is less routine, and may have affected study results. Institutional simulation culture may need to be understood better to understand stress in simulation. Our study population may also affect generalizability. Nurses were well represented, but other types of healthcare providers such as resident and fellow trainees were a smaller proportion of the study population making it difficult to draw absolute conclusions about this group. Although study authors were available to assist facilitators in assigning PDL scores to each study participant, inter-rater reliability was not measured because of study authors’ availability to every facilitator. Thus, assigned PDL scores were given from the perspective of two consistent study personnel and a session facilitator who all observed the study participants and the scores may be subject to any observer bias they may have had. Further research with a more balanced study population would make it easier to assess differences between advanced practitioners, nurses, trainees, and assistive personnel. Also, we did not study the effect of debriefing on cognitive stress appraisal. We assessed stress states immediately after a simulation scenario but before debriefing. It is possible that debriefing can provide support to SBE participants and change their cognitive appraisal to favor a challenge state. Future studies could repeat the ASA post survey after debriefing to investigate if debriefing changes a participant’s cognitive appraisal. Different debriefing strategies could be studied using this ASA survey as well. Finally, data collection was interrupted by quarantine procedures during the start of the COVID-19 pandemic, which limited study participation and had an unknown effect on the baseline stress and anxiety of the participants.

This was a prospective cohort study assessing cognitive stress appraisal in participants engaged in multidisciplinary simulation scenarios. The results show that SBE stress is mostly appraised as a challenge, particularly for nurses, respiratory therapists, and patient care technicians compared to attending physicians, NPs, and PAs. More prior simulation experience and lower anxiety disorder risk scores were also associated with greater likelihood of challenge appraisal whereas time in healthcare role, chronic stress, and simulation type were not significant on multivariable analysis. Self-reported participant threatened stress appraisal was not correlated to facilitator-assigned distress scores, indicating facilitators are not likely to detect threatened participants by their behavior. The debate on whether to concentrate on the learning and improvement of the challenged majority or the emotional evaluation of the hidden and threatened minority may begin.

References

  1. Cheng A, Lang T, Starr S, Pusic M: Technology-enhanced simulation and pediatric education: a meta-analysis. Am Acad Pediatrics 2014; Accessed September 5, 2019. https://pediatrics.aappublications.org/content/133/5/e1313.abstract
  2. Dunn W, Dong Y, Zendejas B, Ruparel R, Farley D: Simulation, mastery learning and healthcare. The American journal of the medical sciences 2017; 353(2):158-165.
  3. Lopreiato JO, Sawyer T: Simulation-based medical education in pediatrics. Academic Pediatrics 2015; 15(2):134-142.
  4. Mileder LP, Schmölzer GM: Simulation-based training: The missing link to lastingly improved patient safety and health? Postgraduate Medical Journal 2016; 92(1088):309-11.
  5. Henricksen JW, Altenburg C, Reeder RW: Operationalizing healthcare simulation psychological safety. Simulation in Healthcare 2017; 12(5):289-97.
  6. Gouin A, Damm C, Wood G, et al: Evolution of stress in anaesthesia registrars with repeated simulated courses: An observational study. Anaesthesia, critical care & pain medicine 2017; 36(1):21-6.
  7. Joels M, Pu Z, Wiegert O, Oitzl MS, Krugers HJ: Learning under stress: how does it work? Trends in cognitive sciences 2006; 10(4):152-8.
  8. de Kloet ER, Oitzl MS, Joëls M: Stress and cognition: are corticosteroids good or bad guys? Trends in Neurosciences 1999; 22(10):422-6.
  9. Schwabe L, Wolf OT: Learning under stress impairs memory formation. Neurobiology of Learning and Memory 2010 ;93(2):183-8.
  10. Yerkes RM, Dodson JD: The relation of strength of stimulus to rapidity of habit-formation. Journal of Comparative Neurology and Psychology 1908; 18(5):459-82.
  11. Lazarus RS, Folkman S: Stress, Appraisal, and Coping, Springer publishing company, 1984.
  12. Lazarus RS, Folkman S: Transactional theory and research on emotions and coping. European Journal of Personality 1987; 1(3):141-69.
  13. vonRosenberg J: Cognitive appraisal and stress performance: The Threat/Challenge matrix and its implications on performance. Air Medical Journal 2019; 38(5):331-333.doi:https://doi.org/10.1016/j.amj.2019.05.010
  14. Hulbert-Williams NJ, Morrison V, Wilkinson C, Neal RD: Investigating the cognitive precursors of emotional response to cancer stress: Re-testing Lazarus’s transactional model. British Journal of Health Psychology 2013; 18(1):97-121. doi:10.1111/j.2044-8287.2012.02082.x
  15. Gaab J, Rohleder N, Nater UM, Ehlert U: Psychological determinants of the cortisol stress response: the role of anticipatory cognitive appraisal. Psychoneuroendocrinology 2005;30(6):599-610. doi:https://doi.org/10.1016/j.psyneuen.2005.02.001
  16. Tomaka J, Blascovich J, Kibler J, Ernst JM: Cognitive and physiological antecedents of threat and challenge appraisal. Journal of Personality and Social Psychology 1997;73(1):63-72. doi:10.1037/0022-3514.73.1.63
  17. Blascovich J, Tomaka J: The biopsychosocial model of arousal regulation. Advances in Experimental Social Psychology 1996; Vol. 28. Academic Press; 1996:1-51.doi:10.1016/S0065-2601(08)60235-X
  18. Gardner R: The role of debriefing in simulation-based education. Seminars in perinatology 2013; 37(3):166-174. doi:10.1053/j.semperi.2013.02.008
  19. Gardner R: Introduction to debriefing. Seminars in perinatology 2013; 37(3):166-174.doi:10.1053/j.semperi.2013.02.008
  20. Rudolph JW, Raemer DB, Simon R: Establishing a safe container for learning in simulation: the role of the presimulation briefing. Simulation in Healthcare 2014;9(6):339-349. doi:10.1097/SIH.0000000000000047
  21. Rudolph JW, Simon R, Dufresne RL, Raemer DB: There’s no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simulation in Healthcare 2006; 1(1):49-55. doi:10.1097/01266021-200600110-00006
  22. Brett-Fleegler M, Rudolph J, Eppich W, et al: Debriefing assessment for simulation in healthcare: development and psychometric properties. Simulation in Healthcare 2012;7(5):288-294. doi:10.1097/SIH.0b013e3182620228
  23. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 2009;42(2):377-381. doi:https://doi.org/10.1016/j.jbi.2008.08.010
  24. NIH Toolbox® Scoring and Interpretation Guide for the IPad NIH Toolbox ® Scoring and Interpretation Guide for the IPad.; 2006.
  25. Cohen S, Kamarck T, Mermelstein R: A global measure of perceived stress. Journal of Health and Social Behavior 1983; 24(4):385-396.
  26. Spitzer RL, Kroenke K, Williams JBW, Löwe B: A brief measure for assessing generalized anxiety disorder: The GAD-7. https://jamanetwork.com/
  27. Löwe B, Decker O, Müller S, et al: Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Medical Care 2008;46(3):266-274. http://www.jstor.org/stable/40221654
  28. Mendes W (UCSF). Acute stress appraisals. 2017;(March):1-3.
  29. Carenzo L, Braithwaite EC, Carfagna F, et al: Cognitive appraisals and team performance under stress: A simulation study. Medical Education 2020; 54(3):254-263.doi:10.1111/medu.14050
  30. Harvey A, Nathens AB, Bandiera G, Leblanc VR: Threat and challenge: Cognitive appraisal and stress responses in simulated trauma resuscitations. Medical Education 2010; 44(6):587-594. doi:10.1111/j.1365-2923.2010.03634.x
  31. Thompson NJ, Corbett SS, Welfare M: A qualitative study of how doctors use impression management when they talk about stress in the UK. International Journal of Medical Education 2013; 4:236-246. doi:10.5116/ijme.5274.f445
  32. Murphy NA: Appearing Smart: The impression management of intelligence, person perception accuracy, and behavior in social interaction. Personality and Social Psychology Bulletin 2007; 33(3):325-339. doi:10.1177/0146167206294871
  33. Bell E, McAllister S, Ward PR, Russell A: Interprofessional learning, impression management, and spontaneity in the acute healthcare setting. Journal of Interprofessional Care 2016; 30(5):553-558. doi:10.1080/13561820.2016.119

Tables

Table 1. Summary of demographics and survey scores

Current Job Position/Credentials Chronic stress/PSS scores 
Advanced providers: Attending + NP + PA15 (3.1%)≤40 low stress482 (99.4%)
Assistive personnel: Tech + Resp Therapist + other80 (16.5%)40-60 moderate stress3 (0.6%)
Nurse347 (71.5%)Mean (SD)31.9 (3.3)
Trainees: Resident + Student + Fellow43 (8.9%)Median (IQR)32.0 (30, 34)
Current Year in Training (PGY) 1  17 (39.5%)Range(10, 43)
210 (23.3%)Chronic anxiety/GAD-7 scores 
  0-4 minimal anxiety203 (41.9%)
36 (14%)5-9 mild anxiety186 (38.4%)
45 (11.6%)10-14 moderate anxiety74 (15.3%)
5+5 (11.6%)15+ severe anxiety22 (4.5%)
Years in Practice Mean (SD)5.9 (4.4)
<2123 (27.8%)  
2-595 (21.5%)Median (IQR)5.0 (2, 8)
5-1093 (21%)Range(0, 21)
10+131 (29.6%)ASA pre scores <1 stress is a challenge  376 (77.5%)
Mean (SD)7.6 (8.3)>1 stress is threat109 (22.5%)
Median (IQR)5.0 (1.5, 10)Mean (SD)0.8 (0.4)
Range(0, 40)Median (IQR)0.8 (0.6, 1)
SBE participation (# sessions in past 2 years) Range(0.1, 2.6)
038 (7.8%)  
165 (13.4%)ASA post scores <1 stress is a challenge  409 (84.3%)
2192 (39.6%)>1 stress is threat76 (15.7%)
381 (16.7%)Mean (SD)0.7 (0.3)
440 (8.2%)Median (IQR)0.7 (0.5, 0.9)
5+69 (14.2%)Range(0.1, 2.6)
Gender   
Female427 (88%)  
Male58 (12%)  
Simulation type   
Dental Clinic13 (2.7%)  
ED/Obs unit68 (14%)  
PICU Fellows Boot Camp15 (3.1%)  
Med/Surg + Outpatient Procedures/PACU236 (48.7%)  
ECMO16 (3.3%)  
NICU14 (2.9%)  
Nurse Residency56 (11.5%)  
PICU67 (13.8%)  
Summary of demographics and job characteristics of participants, Generalized Anxiety Disorder-7 (GAD) results for likelihood of anxiety disorder, Perceived Stress Scale (PSS) results of amount of stress, Acute Stress Appraisal (ASA) score results before and after simulation for cognitive appraisal, and simulation type. All results are expressed as mean and percentage unless otherwise indicated.

Table 2. Summary of baseline variables stratified by ASA scores before and after a simulation scenario

 All: N=485Challenge/Challen ge (C/C): N=328Challenge/Thre at (C/T): N=48Threat/Challen ge (T/C): N=81Threat/Thre at (T/T):p-value
  Current Job Position/    N=28  <0.001s
Credentials      
Attending Physician + NP + PA15 (3%)5 (1.5%)6 (12.5%)3 (3.7%)1 (3.6%) 
Tech + Resp Therapist +80 (16%)67 (20.4%)3 (6.2%)6 (7.4%)4 (14.3%) 
Other Nurse  347  233 (71%)  33 (68.8%)  66 (81.5%)  15 (53.6%) 
 (72%)     
Resident + Student + Fellow43 (9%)23 (7%)6 (12.5%)6 (7.4%)8 (28.6%) 
Current Year in Training (PGY)     0.048f
117 (40%)6 (26.1%)1 (16.7%)3 (50%)7 (87.5%) 
210 (23%)8 (34.8%)0 (0%)2 (33.3%)0 (0%) 
36 (14%)4 (17.4%)1 (16.7%)1 (16.7%)0 (0%) 
45 (12%)2 (8.7%)2 (33.3%)0 (0%)1 (12.5%) 
5+5 (12%)3 (13%)2 (33.3%)0 (0%)0 (0%) 
Years in Practice     0.029s
<212376 (24.9%)19 (45.2%)19 (25.3%)9 (45%) 
 (28%)     
2-595 (21%)67 (22%)2 (4.8%)21 (28%)5 (25%) 
5-1093 (21%)70 (23%)9 (21.4%)13 (17.3%)1 (5%) 
10+13192 (30.2%)12 (28.6%)22 (29.3%)5 (25%) 
  SBE participation (#(30%)      0.26s
sessions in past 2 years)      
038 (8%)21 (6.4%)9 (18.8%)5 (6.2%)3 (10.7%) 
165 (13%)40 (12.2%)6 (12.5%)12 (14.8%)7 (25%) 
2192135 (41.2%)16 (33.3%)34 (42%)7 (25%) 
 (40%)     
381 (17%)58 (17.7%)4 (8.3%)13 (16%)6 (21.4%) 
440 (8%)28 (8.5%)4 (8.3%)6 (7.4%)2 (7.1%) 
5+69 (14%)46 (14%)9 (18.8%)11 (13.6%)3 (10.7%) 
Gender: Female427284 (86.6%)41 (85.4%)77 (95.1%)25 (89.3%)0.16f
  Chronic stress/PSS scores(88%)      0.30k
Median (IQR)32 (30,32 (30, 33.2)32 (29, 34)33 (30, 35)32 (30, 34) 
  Chronic anxiety/GAD-734)      0.004k
scores Median (IQR) Simulation type5 (2, 8)5 (2, 8)5 (2, 8.2)6 (4, 8)9 (5, 12)  <0.001s
Dental Clinic13 (3%)8 (2.4%)1 (2.1%)4 (4.9%)0 (0%) 
ED/Obs Unit68 (14%)52 (15.9%)4 (8.3%)10 (12.3%)2 (7.1%) 
PICU Fellows15 (3%)7 (2.1%)4 (8.3%)1 (1.2%)3 (10.7%) 
Med/Surg + Outpatient236167 (50.9%)12 (25%)40 (49.4%)17 (60.7%) 
Procedures/PACU(49%)     
ECMO16 (3%)6 (1.8%)3 (6.2%)5 (6.2%)2 (7.1%) 
NICU14 (3%)10 (3%)0 (0%)4 (4.9%)0 (0%) 
Nurse Residency56 (12%)28(8.5%)16(33.3%)8(9.9%)4(14.3%) 
PICU67 (14%)50(15.2%)8(16.7%)9(11.1%)0(0%) 
s Chi-squared test by Montecarlo simulation, f Fisher’s exact test, k Kruskal-Wallis test.
 
Summary of participant characteristics and survey scores grouped by combination of cognitive appraisal of challenge or threat before and after a simulation scenario. Results are expressed as mean and percentage unless otherwise indicated.

Table 3. Univariable and multivariable logistic regression comparing Challenge/Challenge to all others

UnivariableMultivariable
 OR (95% CI)p-valueOR (95% CI)p-value
Current Job Position/Credentials    
Attending Physician + NP + PAReference   
Tech + RT + other10.31<0.00116.4 (4.58,66.09)<0.001
 (3.15,38.05)   
Nurse4.09 (1.42,13.38)0.0124.9 (1.58,17.08)0.008
Resident + Student + Fellow2.3 (0.69,8.45)0.18––
Current Year in Training (PGY)    
1Reference   
27.33 (1.33,60.29)0.034––
33.67 (0.55,32.7)0.20––
41.22 (0.13,9.56)0.85––
5+2.75 (0.36,25.76)0.33––
Years in Practice    
< 2Reference   
2-51.48 (0.84,2.64)0.181.14 (0.6,2.17)0.69
5-101.88 (1.05,3.45)0.0371.5 (0.76,2.97)0.24
10+1.46 (0.87,2.47)0.161.52 (0.83,2.8)0.18
SBE participation (# sessions in past 2 years) 0    Reference   
11.3 (0.57,2.92)0.531.65 (0.66,4.23)0.29
21.92 (0.93,3.9)0.072.2 (0.92,5.33)0.08
32.04 (0.91,4.57)0.082.68 (1.04,7.02)0.042
41.89 (0.75,4.88)0.182.96 (1.02,8.9)0.049
5+1.62 (0.72,3.66)0.242.73 (1.05,7.24)0.04
Gender – Male1.58 (0.86,3.08)0.16––
Chronic stress/PSS scores0.97 (0.91,1.03)0.28––
Chronic anxiety/GAD-7 scores0.95 (0.91,0.99)0.0110.93 (0.89,0.98)0.005
Simulation type    
Dental ClinicReference   
ED/Obs Unit2.03 (0.55,7.01)0.27––
PICU Fellows0.55 (0.11,2.44)0.43––
Med/Surg + Outpatient Surg/PACU1.51 (0.44,4.7)0.48––
ECMO0.38 (0.08,1.65)0.20––
NICU1.56 (0.31,8.28)0.59––
Nurse Residency0.63 (0.17,2.11)0.46––
PICU1.84 (0.5,6.31)0.34––
Univariable and multivariable analysis of Challenge/Challenge group to all others combined. Variables that were statistically significant in the univariable analysis were included in the multivariable analysis.

Table 4: Cross-tabulation of Psychological Distress Level (PDL) versus Acute Stress Appraisal (ASA) scores

 PDL      ASA     
    C/C + C/T + T/CT/TC/T + T/C + T/TC/C
 No distress (=0)39123
1		132282
 Distress (=1/2/3)18613
Cross-tabulation of Participant Distress Level (PDL) assigned by simulation facilitator with Acute Stress Appraisal scores indicated by participants.

Appendix

GAD-7
Author: Wendy Berry Mendes, PhD. March 2017. Wendy.Mendes@ucsf.edu
Author: Wendy Berry Mendes, PhD. March 2017. Wendy.Mendes@ucsf.edu
by Wendy L. Hobson-Rohrer, MD ↓; Rachel Gordon, MD ↓; Deborah L. Cabaniss, MD ↓ & Boyd F. Richards, PhD ↓

Improving the Promotions Dossier with the Enhanced CV

Article has been published.

Journal of Continuing Education in the Health Professions
PubMed

Citation

Hobson, W. L., Gordon, R. J., Cabaniss, D. L., & Richards, B. F. (2022). Documenting Educational Impact in the Promotion Dossier with an Enhanced Curriculum Vitae. The Journal of continuing education in the health professions, 42(1), 47–52. https://doi.org/10.1097/CEH.0000000000000386