We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives. In this framework, an agent aims to optimize a risk measure of the distribution of objective values, where the objective function depends on the frequency of visitation of states induced by the agent’s policy. First, we motivate, propose, and formalize risk-aware GUMDPs, which enable agents and decision makers to trade off expected performance by risk aversion while benefiting from the rich set of objectives that can be cast under the framework of GUMDPs. We focus our attention on the entropic risk measure (ERM). Second, we show how we can solve risk-aware GUMDPs with ERM objectives by resorting to online planning techniques. In particular, we propose an approach based on Monte Carlo Tree Search (MCTS) to provably solve risk-aware GUMDPs up to any desired accuracy. Third, we provide a set of experimental results showcasing that our approach is successful when optimizing for a spectrum of risk-aware behaviors in the context of GUMDPs under diverse tasks (standard MDPs, maximum state entropy exploration, imitation learning, and multi-objective MDPs).
@inproceedings{santos2026ra-gumdp,title={Risk-Aware General-Utility Markov Decision Processes},author={Santos, Pedro P. and Vital, Fábio and Sardinha, Alberto and Melo, Francisco S.},year={2026},eprint={2607.09298},archiveprefix={arXiv},primaryclass={cs.LG},url={https://arxiv.org/abs/2607.09298},}
Conversational repair is a mechanism used to detect and resolve miscommunication and misinformation problems when two or more agents interact. One particular and underexplored form of repair in emergent communication is the implicit repair mechanism, where the interlocutor purposely conveys the desired information in such a way as to prevent misinformation from any other interlocutor. This work explores how redundancy can modify the emergent communication protocol to continue conveying the necessary information to complete the underlying task, even with additional external environmental pressures such as noise. We focus on extending the signaling game, called the Lewis Game, by adding noise in the communication channel and inputs received by the agents. Our analysis shows that agents add redundancy to the transmitted messages as an outcome to prevent the negative impact of noise on the task success. Additionally, we observe that the emerging communication protocol’s generalization capabilities remain equivalent to architectures employed in simpler games that are entirely deterministic. Additionally, our method is the only one suitable for producing robust communication protocols that can handle cases with and without noise while maintaining increased generalization performance levels.
@inproceedings{vital2024implicit,title={Implicit Repair with Reinforcement Learning in Emergent Communication},author={Vital, Fábio and Sardinha, Alberto and Melo, Francisco S.},booktitle={Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems},pages={2115-2124},year={2025},isbn={9798400714269},publisher={International Foundation for Autonomous Agents and Multiagent Systems},address={Richland, SC},keywords={emergent communication, multi-agent reinforcement learning, reinforcement learning, representation learning},location={Detroit, MI, USA},series={AAMAS '25},numpages={10},}
We present Perceive-Represent-Generate (PRG), a novel three-stage framework that maps perceptual information of different modalities (e.g., visual or sound), corresponding to a series of instructions, to a sequence of movements to be executed by a robot. In the first stage, we perceive and preprocess the given inputs, isolating individual commands from the complete instruction provided by a human user. In the second stage we encode the individual commands into a multimodal latent space, employing a deep generative model. Finally, in the third stage we convert the latent samples into individual trajectories and combine them into a single dynamic movement primitive, allowing its execution by a robotic manipulator. We evaluate our pipeline in the context of a novel robotic handwriting task, where the robot receives as input a word through different perceptual modalities (e.g., image, sound), and generates the corresponding motion trajectory to write it, creating coherent and high-quality handwritten words.
@inproceedings{9981788,author={Vital, Fábio and Vasco, Miguel and Sardinha, Alberto and Melo, Francisco},booktitle={2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},title={Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories},year={2022},volume={},number={},pages={5855-5860},keywords={Robot motion;Visualization;Scalability;Pipelines;Communication channels;Trajectory;Task analysis},doi={10.1109/IROS47612.2022.9981788},}
