Policy Gradients for Cumulative Prospect Theory in Reinforcement Learning
Quick summary
arXiv:2410.02605v5 Announce Type: replace-cross Abstract: We derive a policy gradient theorem for Cumulative Prospect Theory (CPT) objectives in finite-horizon Reinforcement Learning (RL), generalizing the standard policy gradient theorem and encompassing distortion-based risk objectives as special cases. Motivated by behavioral economics, CPT combines an asymmetric utility transformation around a reference point with probability distortion. Building on our theorem, we design a first-order policy gradient algorithm for CPT-RL using a Monte Carlo gradient estimator based on order statistics. We
Key takeaways
- arXiv:2410.02605v5 Announce Type: replace-cross Abstract: We derive a policy gradient theorem for Cumulative Prospect Theory (CPT) objectives in finite-horizon Reinforcement Learning (RL), generalizing the standard policy gradient theorem and encompassing distortion-based risk objectives as special cases.
- Motivated by behavioral economics, CPT combines an asymmetric utility transformation around a reference point with probability distortion.
- Building on our theorem, we design a first-order policy gradient algorithm for CPT-RL using a Monte Carlo gradient estimator based on order statistics.
Why it matters
“Policy Gradients for Cumulative Prospect Theory in Reinforcement Learning” may affect what data AI products can use and where accountability sits. Product teams should watch compliance duties, rights holders should watch enforcement, and users should watch transparency and appeal mechanisms.
Member comments