Mathematical modeling of an optimal control problem for combined chemotherapy and anti-angiogenic cancer treatment protocols

Document Type : Research Article

Authors

• Y.A. Mahaman Nouri ¹
• S. Bisso ²

¹ Department of Fundamental Sciences, National School of Engineering and Energy Sciences, University of Agadez, Niger.

² Department of Mathematics and Computer Science, Faculty of Science and Technology, Abdou Moumouni University, Niger.

10.22067/ijnao.2025.92904.1628

Abstract

We formulate and analyze an optimal control problem for combined chemotherapy and anti-angiogenic therapy. The model couples tumor burden, vascular support, and a logistic surrogate for healthy tissue that encodes homeostasis and drug-induced depletion. The cost functional balances tumor reduction and drug sparing with toxicity mitigation: Beyond terminal terms and quadratic control regularization, it includes a trajectory reward for healthy tissue and a smooth, differentiable below-threshold penalty based on a softplus construction. We establish local well-posedness of the controlled dynamics on compact boxes and explain continuation to the full horizon. Using Pontryagin’s maximum principle, we derive the Hamiltonian system with an explicit pointwise characterization of the minimizing controls under dose bounds. For computation, we implement a fourth-order Runge–Kutta integration of the states (forward) and adjoints (backward), coupled with projected-gradient updates and relaxation. Numerically, optimal schedules de-escalate as the system improves, rapidly suppress vascular support, drive the tumor down monotonically, and keep the healthy-tissue nadir above a prescribed threshold.

Keywords

• Optimal control problem
• Cancer treatment strategies
• Tumor growth model with healthy cell dynamics
• Pontryagin’s Maximum Principle

Main Subjects

• Modeling and Simulation

---

References

[1] Alimirzaei, I. Malek, A. and Owolabi, K.M. Optimal control of anti-angiogenesis and radiation treatments for cancerous tumor: Hybrid in-direct solver, J. Math. 2023 (2023), 5554420. 
[2] Arnold, V.I. Ordinary differential equations, Springer-Verlag, 1992.
[3] Bodzioch, M., Belmonte–Beitia, J., and Foryś, U. Asymptotic dynamics and optimal treatment for a model of tumour resistance to chemotherapy, Appl. Math. Model. 135 (2024), 620–639.
[4] Clarke, F.H. Functional analysis, calculus of variations and optimal control, Springer, 2013.
[5] Cohen, A.D. and Shapiro, H. Optimal control of drug delivery in cancer therapy: A review, Appl. Math. Comput. 392, (2021), 125697.
[6] Feng, Z. and Liu, W. Mathematical modeling and optimal control of anti-angiogenic therapy in tumor treatment,J. Theor. Biol. 540, (2022), 110166.
[7] Ghosh, P. and Mukherjee, D. Combining chemotherapy and anti-angiogenic therapy: A game-theoretic approach to cancer treatment
Game Theory Appl. 9(2) (2023), 45–62.
[8] Hahnfeldt, P., Panigrahy, D., Folkman, J. and Hlatky, L. Tumor development under angiogenic signaling: A dynamical theory of tumor growth, treatment response, and postvascular dormancy, Cancer Res. 59 (1999), 4770–4775.
[9] Hale, J.K. Ordinary differential equations, Krieger, 1980.
[10] Hanfeld, J., Carash, C. and Spanish, E. Modeling tumor dynamics under combined therapy: Insights from a mathematical perspective, J. Theor. Biol. 370 (2015), 203–215.
[11] Huang, Y. and Zhou, H. Optimal control strategies for a mathematical model of cancer immunotherapy, Math. Biosci. 319 (2020), 108309.

[12] Jarrett, A.M., Faghihi, D., Hormuth II, D.A., Lima, E.A.B.F., Virostko, J., Biros, G., Patt, D., Yankeelov, T.E. Optimal Control Theory for Personalized Therapeutic Regimens in Oncology: Background, History,
Challenges, and Opportunities, J. Clin. Med. 9(5) (2020), 1314.
[13] Joorsara, Z. Hosseini, S.M, Esmaili, S. Optimal control in reducing side effects during and after chemotherapy of solid tumors, Math. Methods Appl. Sci. 47(8) (2024), e10049.
[14] Katz, S.C. and Henson, M. Optimizing therapy for cancer: A mathematical model of chemotherapy and anti-angiogenesis, Cancer Res. 72(10) (2012), 2541–2550.
[15] Krebs, R.M. and Lichtenstein, H. Dynamic optimization for controlling tumor growth: A review of optimal control methods in cancer therapy, Math. Med. Biol. 36(1) (2019), 1–27.
[16] Lecca, P. Control theory and cancer chemotherapy: how they interact. Front. bioeng. biotechnol. 8 (2021), 621269.
[17] Ledzewicz, U., Schättler, H. and Friedman, A. Optimal control for combination therapy in cancer, Proceedings of the 47th IEEE Conference on Decision and Control, Cancun, Mexico, (2008), 3783–3788.
[18] Li, M., Grigas, P. and Atamtürk, A. On the softplus penalty for large-scale convex optimization, Oper. Res. Lett. 51(6) (2023), 666–672.
[19] Liberzon, D. Calculus of Variations and Optimal Control Theory, Princeton University Press, 2011.
[20] Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V. and Mishchenko, E.F. The Mathematical Theory of Optimal Processes, Interscience Publishers, 1962.
[21] Rosen, J.B. The gradient projection method for nonlinear programming, J. Soc. Ind. Appl. Math. 8(1) (1960), 181–217.
[22] Sharp, J.A., Burrage, K., Simpson, M.J. Implementation and acceleration of optimal control for systems biology, J. R. Soc. Interface. 18(181) (2021), 20210241.

[23] Wang, L., Xu, Y. and Chen, J. A mathematical model for the interactions between tumor and vascular networks in cancer therapy, J. Math. Biol. 77(3) (2018), 761–795.