Pulsed tVNS Protocol and Reinforcement Learning
- Conditions
- Vagus Nerve Stimulation in Reward Learning
- Interventions
- Device: sham stimulationDevice: taVNS stimulation
- Registration Number
- NCT06205108
- Lead Sponsor
- University of Bonn
- Brief Summary
The aim of this study is to investigate the potential of a phasic taVNS stimulation protocol for reinforcement learning. The investigators will disentangle its effects on learning actions and outcomes through the administration of pulsed stimulation during different stages of learning (stimulation during action vs. stimulation during outcome). This will provide insights into optimal stimulation timing and help determine whether pulsed vagal stimulation can be more effective when paired with instrumental actions or rewarding feedback. Developing a tool that non-invasively improves value-based decision-making by using pulsed stimulation would redefine the application options of taVNS. It will enable tVNS to act as a teaching signal comparable to physiological signals in reward-based learning. In the long run, this may inform targeted interventions for individuals with altered reward function, a key symptom in a range of mental disorders.
As part of the study, The investigators will test three hypotheses:
H1 - Instrument Learning Task: Participants will show improved action-outcome learning when positive feedback after a cue is paired with an effective high-intensity stimulation compared to sham stimulation (sham/taVNS).
H2 - Instrumental Learning Task: Participants will show improved action-outcome learning when the action leading to a reward with higher probability (i.e., correct choice) is stimulated with high intensity stimulation. Again, this will only be observable for active but not sham stimulation (sham/taVNS).
H3 - Functional Magnetic Resonance Imaging (fMRI): Behavioral gains in learning of the cues in the high-intensity active stimulation condition are correlated with higher signals in the midbrain and dorsal striatum during feedback (reward presentation) or action.
- Detailed Description
The study follows a single-blind randomized cross-over design with only 1 within-subject factor (stimulation condition, 2 levels: taVNS, sham) leading to 2 experimental sessions per participant.
Previous studies mainly focused on administering tonic stimulation, applied continuously with a 30s ON 30s OFF stimulation protocol for the full task duration. More recently, protocols delivering short taVNS bursts have been developed to more closely mimic vagal signals. However, optimal parameters (e.g., stimulation length, frequency) to elicit phasic signals are not yet determined. To test which combination of parameters most successfully elicits phasic responses, The investigators conducted a pilot data collection with a set of parameters comparing taVNS and sham stimulation. As a proxy for phasic signaling, the investigators investigated pupil dilations mediated by the locus coeruleus, a main target of the vagus nerve. Ten participants were invited for 2 sessions (taVNS, sham) during which they received pulsed stimulation with the following settings: stimulation duration of 1, 2, or 3s, stimulation frequency of 20, 25, or 30Hz. As a result of this pilot, the investigators found that most pilot participants (7 out of 10) showed the strongest taVNS-induced response vs. sham when stimulation for 1s with 20 Hz frequency. Therefore, this combination of settings will be implemented during the main study. For stimulation, the tVNS Health - R® (tVNS Health, Erlangen, Germany) is used and for eye-tracking the SR Research Eyelink 1000Plus®.
For the main study, participants will be invited to three experimental sessions. The investigators observed considerable interindividual variability in the pupil response to stimulation and similar variability has been previously described for taVNS-induced brain responses. This variability might be partly related to different anatomical characteristics. Therefore, participants will first complete an eye-tracking session to evaluate their pupil response to short taVNS pulses. The investigators then only invite participants responding to taVNS to the neuroimaging sessions to test whether short stimulation pulses can induce phasic signals in principle. In the first session, informed consent is obtained. The pupil response to stimulation with the chosen stimulation settings (1s and 20Hz, 45 trials, \~25min) comparing taVNS and sham stimulation (two blocks separated by a 5-minute break following a randomized order) will be tested to confirm that they elicit a robust pupillary response. taVNS at the right cymba concha for active taVNS and sham stimulation at the earlobe will be used. For the two neuroimaging sessions, the investigators plan to invite participants who showed larger pupil dilation for taVNS vs. sham stimulation during the first session. The investigators will administer taVNS in one session and sham in the other. Session order will be randomized. During calibration, the stimulation intensity will be adjusted to correspond to a mild pricking sensation for taVNS and sham. As previously reported, the intensity will be increased from 0.1 mA in 0.1 mA increments until the participant reports experiencing this sensation. To ensure effective blinding, the investigators will pair non-winning cues (for the blocks in which stimulation is paired with rewarding feedback) or actions associated with the incorrect stimulus (the cue less likely to give a reward in the blocks where stimulation is paired with an action) with a low-intensity (0.1 mA) stimulation. This serves as an additional control condition to compare the effects of the actual stimulation (high intensity) with stimulation that should not activate vagal afferents (low intensity). Participants will be informed that they will receive stimulation with varying intensity in all trials, even though the stimulation won't always be perceived.
Before entering the scanner, participants will complete a short practice of the instrumental learning task to become familiarized with the structure and procedure of the task. Then, participants will enter the fMRI scanner. Here the investigators will conduct a calibration of the stimulation intensity as well as another short practice of the task to calibrate the grip force device used during the task. After this second practice, an anatomical image will be acquired, followed by a resting-state measurement for functional connectivity analyses. During the measurement, a movie will be shown to reduce movement and prevent the participant from falling asleep (Inscapes). To conclude the session, participants will perform the instrumental learning task (described in the section below) with concurrent taVNS or sham stimulation (\~1h of stimulation).
Instrumental Learning Task A probabilistic learning task to evaluate the potential of pulsed taVNS on instrumental learning will be used where participants must learn which actions are related to higher probabilities of reward based on specific cues. Participants will be presented with pairs of abstract stimuli and will learn through trial and error which cue is more likely to yield a reward. For each pair of stimuli, participants will be instructed to select one stimulus by pressing a grip force device for 3 consecutive seconds. They will hold one grip force device in each hand and are instructed to press the grip force corresponding to the side of the selected stimulus.
To provide varying levels of difficulty, the win probabilities of the "correct" stimulus will differ between easy and hard trials. In easy trials, the win probability of the correct stimulus will be 0.75, meaning that choosing the associated action will result in a reward 75% of the time. In hard trials, the win probability of the correct stimulus will be slightly lower at 0.65. The win probability of the other stimulus (the "incorrect" stimulus) will be calculated as 1 minus the probability of the correct stimulus.
The task will consist of eight cue pairs divided into four blocks. Each block contains two pairs, one with a high difficulty and one with a low difficulty. During two blocks, participants will be stimulated with high-intensity stimulation whenever they receive a reward, regardless of the chosen action. During the other two blocks, stimulation will be administered whenever participants perform the action associated with the correct stimulus (the cue more likely to give a reward), regardless of the outcome. As an additional control condition in addition to sham, the intensity will be varied between low-intensity taVNS that is insufficient to induce vagal activation (constant between all participants) and high-intensity stimulation as determined by individual calibration. For each trial, the correct choice will receive high-intensity stimulation, while the incorrect choice will be paired with low-intensity stimulation.
Recruitment & Eligibility
- Status
- RECRUITING
- Sex
- All
- Target Recruitment
- 40
- Age between 18 and 35
- Body-Mass-Index between 18.5 and 30.0 kg/m2
- Providing written informed consent
- Normal or corrected-to-normal vision
acute:
- skin lesions at the stimulation site (e.g., wounds, inflammation),
- earrings or piercings on the left or right ear which cannot be removed,
- implants (pacemaker, cochlear implant, cerebral shunt),
- required permanent use of hearing aid,
- pregnant or nursing,
- other contraindications for MRI (e.g. claustrophobia) lifetime:
- brain injury,
- schizophrenia,
- bipolar disorder,
- severe substance use disorders,
- coronary heart disease,
- stroke,
- diabetes,
- epilepsy,
- asthma 12-month prevalence:
- current mood or anxiety disorder (excluding specific phobias),
- obsessive-compulsive disorder,
- trauma- and stressor-related disorders,
- somatic symptom disorder,
- eating disorder.
Study & Design
- Study Type
- INTERVENTIONAL
- Study Design
- CROSSOVER
- Arm && Interventions
Group Intervention Description sham stimulation sham stimulation Stimulation at the earlobe that is not innervated by the vagus nerve with the same parameters (short bursts, 20Hz, 1s, 400µs pulse widths, delivered in parallel to the action or the feedback). Stimulation strength is individually calibrated. Stimulation lasts \~1h throughout the study. Stimulation lasts \~1h in the session. taVNS stimulation taVNS stimulation taVNS is a non-invasive technique to stimulate the auricular branch of the vagus nerve. Transcutaneous electrodes are placed in the cymba concha of the ear and short bursts (20Hz, 1s, 400µs pulse widths) of stimulation are delivered either in parallel to the action or the feedback. Stimulation strength is individually calibrated. Stimulation lasts \~1h in the session.
- Primary Outcome Measures
Name Time Method Choice accuracy 1 hour Quantified by the accuracy of action upon cue presentation during the Reinforcement Learning Task. Between-condition, within-subject effects (taVNS vs. Sham stimulation). Stimulation order and stimulation intensity will be used as covariates.
Reward sensitivity from reinforcement learning model 1 hour Individual choices will be used to derive the individual reward sensitivities by modeling reinforcement learning using Q-learning. We will perform model comparisons (WAIC) to determine whether separate reward sensitivities for tVNS and sham explain the data better. Additionally, we will use model comparisons to assess whether modulations of the reward sensitivity by taVNS specifically during action, feedback, or both will explain the data better. Moreover, we will use bootstrapping to compare reward sensitivities between conditions (sham vs. tVNS). We will use Bayesian Hierarchical Modeling (STAN) for model estimation and model comparisons (WAIC). Stimulation order and stimulation intensity will be used as covariates.
Changes in brain response during cue presentation 1 hour taVNS-induced changes reward-related brain activity (3T). We will assess changes (tVNS vs. sham, full factorial in SPM) in during cue anticipation. We will assess changes in a mask including the striatum and midbrain defined with the Harvard Oxford Atlas including an improved midbrain.
Response times 1 hour Time taken by a participant to perform an action after cue presentation. Stimulation order and stimulation intensity will be used as covariates.
Learning rate from a reinforcement learning model 1 hour Individual choices will be used to derive individual learning rates by modeling reinforcement learning using Q-learning. We will perform model comparisons (WAIC) to determine whether separate learning rates for tVNS and sham explain the data better. Additionally, we will use model comparisons to assess whether modulations of the learning rate taVNS specifically during action, feedback, or both will explain the data better. Moreover, we will use bootstrapping to compare learning rates between conditions (sham vs. tVNS). We will use Bayesian Hierarchical Modeling (STAN) for model estimation and model comparisons (WAIC). Stimulation order and stimulation intensity will be used as covariates.
Changes in brain response associated with reward prediction errors 1 hour taVNS-induced changes reward-related brain activity (3T). We will compare changes (tVNS vs. sham, full factorial in SPM) in the model-based fMRI contrasts representing reward prediction errors at feedback presentation. Since behavioral changes in model parameters affect model-based regressors, they may introduce changes unrelated to neural effects. We will first compare models (BIC) to determine whether individual-level parameters or group-level parameters explain the neural data better. We then test taVNS-induced changes in the winning model and report changes in the other one as sensitivity analyses. We will assess correlated brain activation in a mask including the striatum and midbrain defined with the Harvard Oxford Atlas including an improved midbrain.
- Secondary Outcome Measures
Name Time Method Stimulation intensity 2 minutes post stimulation self-rated rating (visual analogue scale 1-100) of the stimulation intensity as potential moderator of tVNS effects.
Changes in brain response associated with expected value during cue presentation 1 hour taVNS-induced changes reward-related brain activity (3T). We will compare changes (tVNS vs. sham, full factorial in SPM) in the model-based fMRI contrasts representing the expected value of the chosen option at cue presentation. Since behavioral changes in model parameters affect model-based regressors, they may introduce changes unrelated to neural effects. We will first compare models (BIC) to determine whether individual-level parameters or group-level parameters explain the neural data better. We then test taVNS-induced changes in the winning model and report changes in the other one as sensitivity analyses. We will assess changes in correlated brain activation in a mask including the striatum and midbrain defined with the Harvard Oxford Atlas including an improved midbrain.
Changes in brain response win vs. loss 1 hour taVNS-induced changes reward-related brain activity (3T). We will compare changes (tVNS vs. sham, full factorial in SPM) in the model-free fMRI contrast comparing win feedback with loss feedback. We will assess tVNS-induced changes in a mask including the striatum and midbrain defined with the Harvard Oxford Atlas including an improved midbrain.
Trial Locations
- Locations (1)
Section of Medical Psychology, University Hospital Bonn
🇩🇪Bonn, Germany