Formative Evaluation of an eLearning Approach to Suicide Prevention Training

Training Acceptability - System Usability Scale for Client Bot Emily

Time Frame: Once, immediately after engaging with each of the training technologies

The System Usability Scale, a self-report measure using a Likert-type rating scale, will assess acceptability of the two technologic training components: Client Bot Emily and Lyssn Advisor. Client Bot Emily is a virtual patient/chat bot with the persona of a suicidal patient. Nurses chat with Emily and practice skills of asking open-ended questions and making reflections about statements Emily makes. Lyssn Advisor is an online platform that automatically codes transcripts of audio or video recorded role-plays for counseling skills; nurses then log in to the platform to see their scores, which provides feedback on their counseling performance. Scores on the SUS range from 0-100, which higher scores indicating greater usability or acceptability.

Training Acceptability Self-Report - Post-Training

Time Frame: Immediately after completing the training

A 4-item self-report measure created for this study will assess satisfaction with all of the training components using a Likert-type scale of 0 = Not at all satisfied to 4 = Completely satisfied.

Training Acceptability - System Usability Scale for Lyssn Advisor

Time Frame: Once, immediately after engaging with each of the training technologies

The System Usability Scale, a self-report measure using a Likert-type rating scale, will assess acceptability of the two technologic training components: Client Bot Emily and Lyssn Advisor. Client Bot Emily is a virtual patient/chat bot with the persona of a suicidal patient. Nurses chat with Emily and practice skills of asking open-ended questions and making reflections about statements Emily makes. Lyssn Advisor is an online platform that automatically codes transcripts of audio or video recorded role-plays for counseling skills; nurses then log in to the platform to see their scores, which provides feedback on their counseling performance. Scores on the SUS range from 0-100, which higher scores indicating greater usability or acceptability.

Training Acceptability Self-Report - 6-Month Follow-up

Time Frame: 6-months after completing the training

A 4-item self-report measure created for this study will assess satisfaction with all of the training components using a Likert-type scale of 0 = Not at all satisfied to 4 = Completely satisfied.

Training Engagement With Client Bot User Data - Post-Training

Time Frame: Immediately after participants indicate they have completing the training activities

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Client Bot. The unit of measurement will be the number of participants who accessed the Client Bot during the training period.

Participant Retention

Time Frame: 6 Months after completing the training

Percent of participants retained throughout study follow-up

Lyssn Advisor Training Engagement Self-Report - 6 Month Follow-up

Time Frame: 6-months after participants indicate they have completed the training activities

Participants will self-report whether or not they accessed any of the training technologies during the follow-up period.

Training Engagement With Lyssn Advisor User Data - Post-Training

Time Frame: Immediately after participants indicate they have completing the training activities

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Lyssn Advisor. The unit of measurement will be the frequency of times participants visits the Lyssn Advisor webpage.

Training Engagement With Client Bot User Data - 6-Month Follow-up

Time Frame: 6-months after participants indicate they have completed the training activities

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Client Bot. The unit of measurement will be the number of participants who logged on and interacted with the Client Bot after the training period and before completing the 6-month survey.

Training Engagement With Lyssn Advisor User Data - 6-Month Follow-up

Time Frame: 6-months after participants indicate they have completed the training activities

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Lyssn Advisor. The unit of measurement will be the frequency of times participants visits the Lyssn Advisor webpage during the 6-month follow-up period.

Training Engagement Self-Report - Post-Training

Time Frame: Immediately after participants indicate they have completed the training activities

Investigators will collect self-reported completion of the training activities using a 4-item measure. Participants will report on a sliding scale from 0-100, what percent of each training activity they believe they completed.

Client Bot Training Engagement Self-Report - 6 Month Follow-up

Time Frame: 6-months after participants indicate they have completed the training activities

Participants will self-report whether or not they accessed any of the training technologies during the follow-up period.