Towards efficient, personalized anesthesia using continuous reinforcement learning for propofol infusion control

We demonstrate the use of reinforcement learning algorithms for efficient and personalized control of patients´ depth of general anesthesia during surgical procedures - an important aspect for Neurotechnology. We used the continuous actor-critic learning automaton technique, which was trained and tested in silico using published patient data, physiological simulation and the bispectral index (BIS) of patient EEG. Our two-stage technique learns first a generic effective control strategy based on average patient data (factory stage) and can then fine-tune itself to individual patients (personalization stage). The results showed that the reinforcement learner as compared to a bang-bang controller reduced the dose of the anesthetic agent administered by 9.4% and kept the patient closer to the target state, as measured by RMSE (4.90 compared to 8.47). It also kept the BIS error within a narrow, clinically acceptable range 93.9% of the time. Moreover, the policy was trained using only 50 simulated operations. Being able to learn a control strategy this quickly indicates that the reinforcement learner could also adapt regularly to a patient´s changing responses throughout a live operation and facilitate the task of anesthesiologists by prompting them with recommended actions.