The safeTALK suicide training: an evaluation of attitudes and actions among medical students

BACKGROUND safeTALK is a half-day gatekeeper training programme on recognizing persons at risk of suicide and intervening appropriately. Primary care clinicians have been increasingly targeted for suicide intervention training; however, evidence surrounding the effectiveness of safeTALK is lacking, particularly among medical learners. The aim of this study was to assess whether safeTALK training by medical learners enhanced suicide literacy and intervention skills. METHODS Undergraduate medical students from a university in Ontario were invited to complete an online survey regardless of whether or not they had taken safeTALK training as part of their curriculum. Suicide literacy was measured using the Literacy of Suicide Scale (LOSS) and intervention skills were measured using the Suicide Intervention Response Inventory (SIRI). RESULTS The majority believed that suicide risk assessment training was very important to undergraduate medical education. Although limitations were noted, this study did not demonstrate that safeTALK training significantly improved medical students’ suicide literacy levels or suicide intervention skills. CONCLUSIONS A more comprehensive programme that includes the epidemiology of suicide and mental health disorders, in addition to intervention skills, is recommended to ensure medical learners are equipped to dispel the stigmas surrounding suicide and offer the appropriate care and follow up to their patients in future practice.


INTRODUCTION
Suicide is a steadily growing public health concern that robs communities, families, and society. Rates vary greatly worldwide, but an estimated 3728 lives in Canada and 41 149 lives in the United States were lost to suicide in 2011 and 2013 respectively. [1][2][3][4] It is estimated that up to 45% of persons that died by suicide saw their primary care clinician in the previous 30 days. It is therefore imperative that primary care clinicians are skilled in the detection and intervention of those at risk of suicide. 5,6 Studies have shown that family physicians cite fear of heightening a patient's suicidal feeling, offending a patient, experiencing discomfort, and lacking confidence in intervening as reasons why they do not consistently inquire about suicide. 7 One study demonstrated that among a host of different professionals, including general practitioners, medical students, teachers, and police officers, general practitioners were least motivated for suicide prevention, possibly attributable to their perceived lack of competence in the topic. 8 As primary care physicians commonly encounter high-risk suicidal patients and report low levels of confidence in intervening, they are unique targets for suicide prevention programmes. 3,6,9,10 A large systematic review that focused on suicide prevention strategies identified physician education in depression recognition and treatment as an effective means to reduce the rates of suicide. 3 Thus, confidence in assessing suicidality in the primary care setting is essential to identify those at risk of suicide and to intervene early and appropriately. 3,11 A strategy that has been employed in suicide prevention efforts is termed "gatekeeper training". 2 Gatekeeper training teaches trainees to identify those at high risk of suicide and to manage the situation by coordinating a referral where appropriate. 2,12,13 Gatekeeper training programmes focus on recognition of risk factors, availability of resources, and efforts to reduce stigma. 3 Applied Suicide Intervention Skills Training (ASIST) is a gatekeeper training programme by LivingWorks Education that is widely used and can lead to improvements in attitudes and confidence of participants. 13 Other gatekeeper training packages include Question Persuade and Respond (QPR), Yellow Ribbon International for Suicide Prevention, STORM Skills Training, and safeTALK. 13 The safeTALK programme by LivingWorks Education, is a half-day training programme that teaches participants to recognize those at risk of suicide, engage with them, and connect them with someone trained in suicide intervention. 14 Many gatekeeper training programmes show positive results, but the long-term effectiveness of gatekeeper training programmes is questionable, indicating the need for regular refresher courses. 13 Gatekeeper training in institutional settings, such as in the military and in schools, has shown some promise in preventing suicide. 2 Training in other populations, such as in the workplace and in small communities, has shown mixed results. 2 Brief gatekeeper training of novice trainees has demonstrated positive changes in knowledge and attitudes about suicide, 12 while ASIST training in a remote on-reserve First Nations community did not have a significant impact of selfreported preparedness in suicide intervention and participants were no more likely to engage in gatekeeper behaviours. 15 Implementing gatekeeper training programmes in structured settings, such as those offered by institutions, is the mainstay of programme delivery, but such programmes could also be implemented for the general population if further research demonstrates their effectiveness across different populations. 2 Negative attitudes towards suicide prevail among medical students, which may influence their treatment of patients in suicidal crises throughout their studies and, later, in practice as physicians. 10,16 Inquiries into suicide literacy, attitudes, and stigmatization of suicide among medical students revealed that they have poor literacy and understanding of suicide, high levels of stigmatization, and judgmental attitudes, most notably in the early years of their training. This is likely attributable to the lack of emphasis on mental health and clinical exposure during the early stages of medical training. 16,17 It is important to address these negative attitudes and poor suicide literacy levels through education to improve skills and competence in dealing with suicidal patients. 10,[16][17][18] The greatest potential to alter attitudes and enhance therapeutic skills may exist at the undergraduate medical level and, as such, it may be most beneficial to introduce initial suicide prevention training programmes to students at this level. 9,10,17 One such training programme was studied and the findings indicated that medical students' attitudes, stigmatization, and negative appraisal of suicide changed significantly following training. 10 A much shorter training programme was used with medical residents, and results indicated a significant improvement of confidence, attitudes, and behaviour. 19 Despite the lack of uniform and standardized suicide prevention education, medical students, residents, and physicians alike stress the importance of an interactive, skillsbased suicide prevention curriculum as opposed to a strictly didactic curriculum that will enhance their confidence and competency in dealing with patients at potential risk of suicide. 6,9,10,16,18,20,21 The lengthy time commitment required of some gatekeeper programmes, some of which are up to five days long, can discourage enrolment. Flexible course structures, such as safeTALK, may encourage attendance, especially among busy individuals including general practitioners and medical students. 13 safeTALK is a half-day training programme that teaches participants to recognize those at risk of suicide, engage with them, and connect them to those trained in suicide intervention within the community. 14 A six-month pilot of safeTALK was initiated in Scotland in 2006 and subsequently evaluated prior to implementing a Scotland-wide programme of training. 14 The participants in the pilot were from a wide variety of professions, thus limiting generalizability. Additional evidence surrounding the effectiveness of safeTALK is lacking, with one smallscale evaluation of veterinary undergraduates and staff in Scotland reporting increases in knowledge, confidence, and awareness immediately after the course. 13,22 A medical school at a university in Ontario recently implemented safeTALK as a mandatory component of the phase 1 curriculum in response to the high rate of suicides in the north and among First Nations youth. The purpose of this study is to assess medical students' knowledge and opinions of the safeTALK workshop and its value to their future practice as physicians, thus evaluating the importance of such a programme to medical curriculum. The efficacy of safeTALK in increasing suicide literacy and/or preparing students for training in rural and remote communities was explored through an online survey to medical students. We specifically sought to examine whether suicide intervention skills and suicide literacy were enhanced by safeTALK and whether medical students believed the training was of value to their potential future practice as primary care physicians.

Participants
The participant sample consisted of undergraduate medical students enrolled at a medical school in Ontario. Necessary permissions were granted to contact the students and recruitment was facilitated through the Learner Affairs Department. Invitations to participate were emailed to the student body with a link to an online survey developed by the authors using Remark Survey Software. A brief description of the study was outlined in the email with further information presented online prior to participation. Participants could not commence the survey unless they read through the information and consented to the terms of the study; after which they were brought to the beginning of the survey. Data collection took place over the course of one month, with two reminder emails sent to the students.

Survey and outcome measures
The survey consisted of background information questions to solicit demographic information such as age, gender identity, year of study, intended medical specialty, culture, educational background, and whether they had completed the safeTALK workshop. The importance of suicide risk assessment in undergraduate medical education and the perceived value of safeTALK was also evaluated through a series of open-ended questions. The survey was intended to be brief and engaging using established techniques in online survey development and data collection. 23 All web pages prominently displayed crisis intervention phone numbers and web links to local mental health support services in case participants were negatively triggered by the survey questions on suicide. Suicide intervention skills were measured with the Suicide Intervention Response Inventory-I (SIRI), a 25-item self-report instrument that assesses skills in responding to a patient at risk of suicide. 24 In the present context, research has demonstrated that SIRI can detect improvement in caregiver skills resulting from focused training in suicide management. 19,24 Respondents were instructed to indicate which reply, A or B, would be more facilitative to a hypothetical suicidal patient remark. Scores consist of summed totals of the correct, or more facilitative, responses, potentially ranging from 0 to 25. The SIRI-I has demonstrated good construct and convergent validity, good test retest reliability and high internal consistency. [24][25][26] Internal consistency in the present study was low (α = 0.20) with several items removed due to zero variance from the small sample size.
Levels of suicide literacy among the respondents were gauged with the abbreviated Literacy of Suicide Scale (LOSS; 12-item) constructed by Calear et al., (unpublished). Each of the items included in the LOSS are scored as "True" or "False". Correct responses are given a score of one, while incorrect are given a score of zero. Scores were summed, with higher scores indicating higher suicide literacy. Each of the questions touched on one of the four domains of suicide literacy: signs and symptoms, risk factors, causes and triggers, or treatment and prevention. 27 As the items in the scale have correct and incorrect answers, previous validation of the scale's reliability was performed using item-response theory (IRT) 17 and yielded adequate reliability (α = 0.71). Internal consistency for the current study is not reported as a small sample size and five items with zero variance yielded a small, negative reliability coefficient. 28

Analytical plan
A descriptive analysis of the independent variables was performed using SPSS 20. Independent samples ttests were used to compare the mean scores of the LOSS and the SIRI across different within-group samples; primarily (i) having taken safeTALK training; versus (ii) have not taken safeTALK training. The subthemes from the LOSS were examined in depth to determine whether literacy levels were stronger in some areas over others. A thematic analysis of open-ended questions was performed using NVivo 10 to gain a greater understanding of the student's perspective on safeTALK and the need for suicide risk assessment training as medical learners. This study received approval from the University's Research Ethics Board.

RESULTS
From a potential pool of 175 participants, we received survey responses from 30 individuals yielding a 17% response rate. As shown in Table 1, two-thirds (66.7%) of participants were women, with an average age of 26.8 years (standard deviation [SD] = 4.5). Participants' undergraduate education was primarily in health sciences and natural sciences and close to half were intending to specialize in family medicine. Nearly all the participants (90%) had completed the safeTALK workshop when it was offered through the medical school curriculum. One-third (33.3%) had previous suicide intervention training (other than safeTALK) such as Applied Suicide Intervention Skills Training (ASIST). The majority of participants (66.7%) believed that suicide risk assessment training was "very important" to undergraduate medical education. During placement or clerkship, half of participants (56.7%) were exposed "not very often" to persons at risk of suicide, while 16.7% of participants were exposed "fairly often" to persons at risk of suicide. 37% of participants felt "neutral" that their safeTALK training prepared them for exposure to persons at risk of suicide. Among participants that were exposed "fairly often" to persons at risk of suicide, 3.8% felt "well prepared" from the safeTALK training. Since taking the safeTALK workshop, half of participants (50%) "agree" that they felt more comfortable discussing the topic of suicide with others, while 13.3% "disagree" or "strongly disagree" that their comfort level improved. Lastly, 40% of participants "agree" that the safeTALK workshop is of value to their future medical practice, while 10% "disagree" or "strongly disagree" that safeTALK was of value.

RES MEDICA
The effectiveness of the safeTALK workshop was examined through the 12-item Literacy of Suicide Scale (LOSS) and the 25-item Suicide Intervention Response Inventory (SIRI-I). Table 2 shows the percentage of correctly answered responses to the items in the LOSS and their corresponding theme. The mean LOSS score was 4.8 (SD = 0.7) of a maximum score of 12. Scores across the domains for the LOSS varied, with participants performing best in recognizing the signs and symptoms of suicide (63.3% correct, SD = 22.1), yet scoring lower in recognizing the causes of the nature of suicidality (1.7% correct, SD = 6.3). LOSS scores were subsequently compared by whether participants had taken safeTALK training. Participants that did not take safeTALK training scored higher on the LOSS (μ = 5.3, SD = 0.6) than participants that took the training (μ = 4.7, SD = 0.7); however the difference between groups was not significant (t(28) = 1.45, p = 0.16), with 3.6% of the variance in LOSS scores being explained by safeTALK training. No significant between-group differences in LOSS scores emerged across gender, cultural self-identification, or previous suicide intervention skills training. Students with psychiatry as their intended specialty performed better on the LOSS than other specialties, however the differences in average LOSS scores were not significant.
In Males and females performed equally well on the SIRI and there were no significant between-group differences in SIRI scores across gender, cultural selfidentification, or previous suicide intervention skills training. Slight differences in SIRI scores emerged across medical students' intended specialty, with psychiatry students showing higher average scores (μ = 24.5, SD = 0.58) than other specialities, however the differences did not reach statistical significance.
A thematic analysis was undertaken for two openended questions: (i) In your opinion, how important is suicide risk assessment training to undergraduate medical education? and (ii) In your opinion, what other mental health or suicide risk assessment training should be part of undergraduate medical education? Table 3 highlights the main themes emanating from the respondents' answers. Respondents felt that suicide risk assessment training was important as a medical learner. They additionally noted the importance of these skills for interacting with family, friends and classmates, as well as during placements or with future patients. One respondent underlined the importance of being aware and cognizant of the effect that the pressures of medical school can have on classmates and peers.   In response to the question about other mental health or suicide risk assessment training in undergraduate medical education, respondents identified specific topics (e.g. depression, anxiety, the mental health system), as well as additional training that increases their capacity for suicide risk assessment. Respondents were very interested in learning how to ask the necessary questions, how to effectively help patients in crisis, and assess when patients are safe for discharge.
"[A] critical skill to develop in an MD program is the ability to answer the question 'is this person safe to send home?'. We need to be able to ask the right questions to know whether to admit a patient or to feel safe sending them home after some brief counselling." Respondents also used this question as an opportunity to express a general dissatisfaction with the content from safeTALK training, citing that it was too simple and unhelpful at the undergraduate medical education level. Another theme that emerged was the need for more advanced training, identifying the more intensive ASIST programme as a potential alternative. Repeating this training each year was also proposed to increase confidence and reduce the stigma associated with mental health issues. "[s]afeTALK is too "layman" for medical students.
We are generally very aware of mental health and suicide, and the level of intervention in this workshop is too shallow. I think that ASIST would be a more appropriate workshop for students because it provides an actual framework on how to address suicide and intervene."

DISCUSSION
This study adds to a small, yet growing body of research investigating suicide intervention skills and literacy amongst medical learners. Previous studies, as well as this paper, clearly indicate that there is a great need for an interactive, skills-based, standardized suicide awareness and prevention curriculum at the undergraduate medical education level. 6,9,10,16,18,20,21 A medical school in Ontario sought to do this by implementing safeTALK as a mandatory requirement for all medical students to complete. This study investigated the effectiveness of the safeTALK workshop and value of the training using two validated measures that address suicide literacy and suicide intervention skills.
While most participants reported that suicide risk assessment was very important to undergraduate medical education, the scores on the LOSS that measure suicide literacy were quite low. The mean score on the LOSS was 4.8 out of a possible 12, with students scoring especially low on the questions that addressed the causes of the nature of suicidality, while performing best in recognizing the signs and symptoms of suicide. These findings differ from the Batterham et al. 29 community-based study using the 12-item LOSS. Participants in the current study scored lower than Batterham's community sample, yet had less difficulty with items related to signs and symptoms of suicide. In the current study, only 3.6% of the variance in LOSS scores were explained by the safeTALK training, suggesting that the training itself had a very small effect on medical students' suicide literacy.  There are many occasions in which students may encounter patients in the community setting who are suicidal.  It's something we need to know how to handle if a situation arises in any of our placements.  Many students will practice in rural areas, where they may be the only resource for such assessments.

Others at risk
 It is important we receive this training so we can help our peers. Further, it is important for any one of your friends/family/acquaintances who may be having suicidal thoughts, and of course for your future patients down the line.

As physicians
 It is very important for future physicians to be trained in assessing suicide risk as we will ultimately be making the decisions whether or not to [admit] patients.

Building capacity
 We need more opportunities where suicide awareness and mental health awareness are discussed, so we can be constantly reminded that there are people out there at risk.  [A] critical skill to develop in an MD programme is the ability to answer the question "is this person safe to send home?"

Advanced training
 ASIST would be a more appropriate workshop for students because it provides an actual framework on how to address suicide and intervene.  Something more comprehensive and repeated each year until it feels natural to use the skills.

Limitations of safeTALK
 safeTALK was not designed for medical students or health care providers. It is designed for the lay person.  We need something that gives us further steps than just safeTALK Of notable interest were the zero variance items in the LOSS that were incorrect, such as "most people who suicide are psychotic", and "a suicidal person will always be suicidal and entertain thoughts of suicide". This could possibly suggest that medical learners are confusing the term "psychotic" with "psychiatric" rather than as an episode of psychosis. Additional reasons may include the need for increased attention to mental health disorders and the epidemiology of suicide in medical curriculum, which may fall outside the parameters of safeTALK training.
Unlike the LOSS scores, scores of the SIRI assessing intervention skills were more promising. Participants performed well with an average score of 23.3 out of 25. These scores are on par with previous studies using the SIRI that sampled crisis line trainees and experienced crisis counsellors. 25 No significant between-group differences were found in SIRI scores by whether or not medical students took the safeTALK training. Although the mean scores were higher on this measure of suicide intervention skills, there was no statistical indication that the safeTALK training had any effect on SIRI scores. Although the levels of suicide literacy and suicide intervention skills were not significantly enhanced by safeTALK, 40% of participants recognized the value of the safeTALK workshop to their future medical practice and 50% reported enhanced comfort discussing suicide following the training. With close to half of the participants intending to specialize in family medicine, suicide risk assessment and intervention skills are paramount.
Thematic analysis of participants' responses highlighted the importance of suicide risk assessment to undergraduate medical education, to identifying those at risk, and as future physicians. Respondents discussed the importance of acquiring such a skill set throughout undergraduate medical education so that students are equipped with the necessary tools to help address suicide and immediate risk. While many learning and practice opportunities are available in rural and remote areas, students and practitioners may feel less supported in these settings in terms of adequate risk assessment. Students stressed the importance of learning to identify those at risk not only as future physicians, but as medical students among family, friends, and classmates.
The desire for more mental health training designed specifically for healthcare professionals surfaced through the open-ended questions. safeTALK, which is primarily designed for the general population, may be too basic for medical students, as many want a more comprehensive risk assessment programme. The results from this study, as well as previous studies that investigate the effectiveness of gatekeeper training programmes, suggest additional training, possibly in the form of regular refresher courses, should be provided to students to enhance confidence and competencies in this domain. 13 There is an appetite among medical students to build their capacity in mental health, suicide risk assessment, and intervention skills. As reflected in the low suicide literacy scores and the open-ended responses, the undergraduate medical education community should consider a social epidemiological focus on suicide and mental health disorders to balance their familiarity of the topic, in addition to developing clinical intervention skills and counselling.
The low response rate of 17% and small sample size makes it difficult to generalize the results to other population-based studies. The limited availability of time to collect the data was likely the primary reason for the low response rate. Although the email invitation was sent out to students across all four years of study, the mean year of study among participants was 1.6 years. The student investigator on this project was also a student in her first year of study, which may have resulted in a greater proportion of students from that cohort participating in the study. Additionally, those that chose to participate may have had a more explicit interest in suicide education compared to those who did not. Additional research is needed prior to implementing a new suicide prevention and intervention programme. However, results of the current study indicate that suicide literacy levels are not on par with suicide intervention skills. Future curriculum development could investigate the value of striking a balance between suicide and mental health literacy levels and intervention skills through regular refresher courses and a more comprehensive approach

CONCLUSION
As primary care physicians commonly come into contact with patients at high risk of suicide, there is a growing interest in providing suicide intervention skills training as part of undergraduate medical education. Although limitations were noted, this study did not demonstrate that safeTALK significantly improved medical students' suicide literacy levels or suicide intervention skills. Results revealed that medical learners require a more comprehensive programme than safeTALK alone. Future research should examine the long-term impact of suicide prevention programmes in addition to classes on the epidemiology of suicide and mental health disorders to best equip medical learners with the literacy and confidence to assess and respond to patients at risk of suicide.
Enhancing skills in this domain of care is of the utmost importance in the context of the suicide public health crisis affecting several northern, rural and remote communities in Canada. Medical students must be trained appropriately so they report greater confidence in suicide intervention during clinical placements and in their future practice as primary care physicians.