Agent-based Simulation for Blood Glucose Control in Diabetic Patients

This paper employs a new approach to regulate the blood glucose level of type I diabetic patient under an intensive insulin treatment. The closed-loop control scheme incorporates expert knowledge about treatment by using reinforcement learning theory to maintain the normoglycemic average of 80 mg/dl and the normal condition for free plasma insulin concentration in severe initial state. The insulin delivery rate is obtained off-line by using Qlearning algorithm, without requiring an explicit model of the environment dynamics. The implementation of the insulin delivery rate, therefore, requires simple function evaluation and minimal online computations. Controller performance is assessed in terms of its ability to reject the effect of meal disturbance and to overcome the variability in the glucose-insulin dynamics from patient to patient. Computer simulations are used to evaluate the effectiveness of the proposed technique and to show its superiority in controlling hyperglycemia over other existing algorithms

Authors:

• Sh. Yasini
• M. B. Naghibi-Sistani
• A. Karimpour

Keywords:

• Insulin Delivery rate
• Q-learning algorithm
• Reinforcement learning
• Type I diabetes.

References:

[1] American Diabetes Association, "Standards of medical care for patients
with diabetes mellitus," Diabetes Care, vol. 26, pp. S33-S50, 2003.
[2] B. Topp, K. Promislow, G. De Vries, "A model of β-cell mass, insulin,
and glucose kinetics: Pathway to diabetes," Journal of Theoretical
Biology. Vol. 206, 2000, pp. 6.5-619.
[3] "The effect of intensive treatment of diabetes on the development and
progression of long-term complications in insulin-dependent diabetes
mellitus," N. Eng. J. Med, vol. 329, pp. 977-986, 1993, DCCT.
[4] A. Albisser, B. Leibel, T. Ewartz, Z. Davidovac, C. Botz, W. H. Zingg,"
artificial endocrine pancreas," Diabetes Care, vol. 23, 1974, pp. 389-
396.
[5] J. Jaremco, O. Rorstad, "Advances toward implantable artificial pancreas
for treatment of diabetes," Diabetes Care, vol. 21, 1998, pp. 444-450.
[6] M. Nalecz, J. Wojcicki, and I. Zawicki, "Control in artificial pancreas,"
IFAC Control Aspects Biomed Eng, 1987, pp. 123-137.
[7] R. S. parker, G. L. Bowlin, and G. Wnek, Eds. "Insulin delivery,"
Encyclopedia of Biomaterials and Biomedical Engineering. New Yourk:
Market Dekker, 2004, pp. 857-866.
[8] F. P. Kennedy, "Recent development in insulin deliver technique:
current status or future potential," Drugs, vol. 42, 1991, pp. 213-227.
[9] DC. Klonoff. "Continuous glucose monitoring: roadmap for 21st.
century diabetes therapy," Diabetes care Vol. 28, 2005, pp. 1231-1239.
[10] C. Amaral, B. Wolf, "Current development in non-invasive glucose
monitoring," Journal of Medical Engineering and Physics, 2007,
Elsevier.
[11] TM. Gross, BW bode, D. Einhorn, "Performance evaluation of the
minimed continuous glucose monitoring system during patient home
use," Diabetes Technology Theoretical. vol. 2, 2000, pp. 49-56.
[12] J. Li, Y. Kuang, C. C. Mason, "Modeling the glucose-insulin regulatory
system and ultradian insulin secretory oscillations with two explicit time
delays," J. of Theoretical Biology, vol. 242, 2006, pp. 722-735.
[13] A. De Gaetano and O. Arino, "Mathematical modeling of the
intravenous glucose tolerance test", J. Math. Biol. Vol. 40, 2000, pp.
136-168.
[14] R. N. Bergman, L. Philips, and C. Cobelli, "Physiological evaluation of
the factors controlling glucose tolerance in man," Journal of Clinical
Investigation, vol. 68, 1981, pp. 1456-1467.
[15] J. T. Sorenson, "A physiological model of glucose metabolism in mans
its use to design and assess improved insulin therapies for diabetes," ",
Ph.D. Dissertation, Chem Eng. Dep., Massachusets Inst. Technol,
Cambridge, 1985
[16] Ch. Li and R. Hu, "Simulation study on blood glucose control in
diabetics", Proc. IEEE Int. Conf. on Biomed. and Bioinf Eng, 2007, pp.
1103-1106.
[17] F. Chee, T. L. Fernando, A. V. Savkin, and V. Heeden, "Expert PID
control system for blood glucose control on critically ill patients," IEEE
Trans. Biomed. Eng., vol. 7, No. 4, 2003, pp. 419-425.
[18] Z. H. Lam, J. Y. Hwang, J. G. Lee, J. G. Chase, and G. C. Wake, "Active
insulin infusion using optimal and derivative-weighted control," Medical
Engineering & Physics. vol. 24, 2002, pp. 663-672.
[19] K. H. Kientiz and T. Yoneyame, "A robust controller for insulin pupms
based on H-infinity theory" IEEE Transaction on Biomedical Eng., vol.
40, 1993, pp. 1133-1137.
[20] Sh. Yasini, A. Karimpour, M. B. Naghibi-Sistani, S. Ghareh., "An
automatic insulin infusion system based on H-infinity control
technique," Procceding of the 2008 IEEE, CIBEC-08, 2008, pp. 1-5,
Cairo, Egypt.
[21] F. Chee, AV. Savkin, TL Fernando, S. Nahavandi, "optimal H-infinity
insulin injection control for blood glucose regulation in diabetic
patients," vol. 52, No. 10, 2005, pp. 1625-1631.
[22] M. S. Ibbini, M. A. Masadeh, and M. M. Bani Amer, "A semi closed
loop optimal control system blood glucose level in diabetics", J. Medical
Eng. & Tech. Vol. 28., 2004, pp. 189-196.
[23] M. E. Fisher, "A semi closed-loop algorithm for control of blood glucose
levels in diabetics", IEEE Trans. On Biomed. Eng, vol.38, 38. No. 1,
1991.
[24] R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction,
1989, MIT Press.
[25] R. N. Bergman, D. T. Finegood, M. Ader, "Assessment of insulin
sensitivity invivo", Endocrine Reviews, Vol. 6, No. 1, 1985, pp. 45-85.
[26] C. Neatpisarnvanit, JR Boston, "Estimation of plasma insulin for plasma
glucose. IEEE Transaction on Biomedical Engineering," vol. 49, No. 11,
2002, pp.1253-1259.
[27] C. Watkins, Learning from delayed rewards, Ph.D. Dissertation
Cambridge University, 1998.