Orso Forghieri

I am Orso Forghieri, a researcher in Reinforcement Learning and Applied Mathematics. My work focuses on Reinforcement Learning, Dynamic Programming, Markov Decision Processes, state abstraction, hierarchical methods, and scalable decision-making.

My PhD thesis, Hierarchical Reinforcement Learning for Large Scale Problems, was prepared at École polytechnique / Institut Polytechnique de Paris, within CMAP, under the supervision of Erwan Le Pennec, Hind Castel-Taleb, and Emmanuel Hyon.

I have also worked on sequential and explainable decision-making methods for systematic equity trading at Qube Research & Technologies, and on reinforcement-learning approaches for edge-computing service placement in collaboration with Orange Gardens / CMAP.

I hold a Master’s degree from École normale supérieure Paris-Saclay and an Engineering Diploma from École polytechnique.

Research interests

• State abstraction.
• Approximate dynamic programming.
• Markov aggregation/disaggregation.
• Scalable MDP solving.
• Hierarchical reinforcement learning.
• Planning.
• Stochastic optimization.

Applications

• Large-scale planning.
• Resource allocation.
• Network optimization.
• Edge computing and service placement.
• Market forecasting.
• Railway-delay propagation.

Selected work

• Hierarchical Reinforcement Learning for Large Scale Problems, PhD thesis, 2025.
• Faster Latency Constrained Service Placement in Edge Computing with Deep Reinforcement Learning, IFIP Networking 2025.
• State Abstraction Discovery from Progressive Disaggregation Methods, EWRL 2024.
• Progressive State Space Disaggregation for Infinite Horizon Dynamic Programming, ICAPS 2024.
• Selected research software for state-space disaggregation, MDP solving, Gymnasium environments, and applied forecasting.

Research and applied collaborations

I am interested in research collaborations on scalable decision-making, reinforcement learning, approximate dynamic programming, network/resource optimization, and sequential decision problems in applied domains.

Academic service

• Doctoral representative on the Laboratory Life Committee, CMAP, École polytechnique, 2024-2025.
• Reviewer for ROADEF 2025.
• Alpha tester for the Marmote MDP solver.
• Mathematics interviewer for the MSc X-HEC Data Science for Business, Feb. 2024.
• Outreach speaker at an IHES meeting with Bachelor students from École polytechnique, Nov. 2023.
• Member of CNRS GDR ROD working groups through MyGDR.

Contact / profiles

• Email: orso.forghieri@gmail.com.
• GitHub.
• LinkedIn.
• Google Scholar.
• DBLP.