Reinforcement Learning: A New Approach to Economic Policy Development
In our recent paper published in Nature Human Behaviour, we present a groundbreaking proof-of-concept demonstration showcasing the capability of deep reinforcement learning (RL) in identifying economic policies that garner majority support in a simple game. This study addresses a pivotal issue in AI research – the challenge of training AI systems to align with human values.
Imagine a scenario where a group of individuals come together to invest money. After the investment pays off, a profit is generated. The question arises – how should the returns be distributed among the investors? Should it be divided equally, proportional to the initial investment size, or based on relative contributions considering the wealth disparities among participants?
This dilemma of resource redistribution has long been a topic of debate among various academic disciplines. In our study, we leverage deep RL as a tool to explore potential solutions to this complex problem.
To address this challenge, we designed a simple game involving four players, with each game session comprising 10 rounds. Players were allocated funds in each round, and they had a choice to either keep the funds or invest them in a common pool with guaranteed growth but uncertain redistribution rules. The players voted for a referee who determined the distribution of proceeds. One referee followed a predefined redistribution policy, while the other was determined by our deep RL agent.
When we analyzed the votes cast by new players, we observed that the policy devised by the deep RL agent was more favored compared to traditional baselines. This highlights the potential of AI systems in learning policies that align with human values and preferences.
The deep RL agent’s approach to redistributing funds based on relative contributions and incentivizing generous contributions reflects a mix of established human theories on resource allocation. By training the AI to optimize human votes, we ensure that AI systems generate solutions that resonate with human values.
By incorporating the principle of majoritarian democracy through voting mechanisms, we aim to capture the collective preferences of individuals. However, further research is needed to strike a balance between majority preferences and minority representation in policy decisions.