VMAS Voting: Iterative Spatial Voting for Collective Decision-Making

A Multi-Agent Reinforcement Learning (MARL) environment for evaluating Iterative Spatial Voting (ISV) in collective decision-making using the Vectorized Multi-Agent Simulator (VMAS).

Overview

This project presents a controlled MARL environment where agents "dance" in a 2D preference space, each with a personalized Gaussian mixture preference distribution. Through iterative movements and interactions, agents collectively determine an aggregate outcome position using various social choice aggregation methods (mean, median, geometric median, etc.).

The goal is to investigate whether agents can learn cooperative strategies through decentralized, iterative adaptation that approximate optimal social choice solutions, even when individual preferences remain private.

Key Features

• Synthetic Preference Generation: Each agent receives an individual preference landscape defined by a mixture of Gaussian components, enabling flexible and multi-modal preferences
• Multiple Aggregation Methods: Supports various social choice rules including:
  • Mean/Centroid
  • Median (componentwise)
  • Geometric Median
  • L_p Norm aggregation
  • L_∞ Norm (center of smallest enclosing circle)
  • Weighted Mean (with optional conviction voting)
  • Min/Max (extreme positions)
• Dynamic Preferences: Optional Perlin noise-based evolution of preference landscapes over time
• Physics-Based Mobility: Inertial mobility model with configurable physics parameters, or classical unrestricted mobility
• Convergence Detection: Automatic detection of Nash equilibrium when agents reach best-response positions
• Efficiency Metrics: Quantifies the gap between emergent outcomes and ideal points using:
  • Price of Anarchy
  • Average Social Index (ASI)
  • Compromise Cost
  • Unfairness measures
• Visualization: Rich visualization with topology heatmaps, side plots for individual agent preferences, outcome path history, and ideal point markers

Installation

# Clone the repository
git clone https://github.com/lucaslopes/vmas-voting.git
cd vmas-voting

# Install dependencies using uv (recommended)
uv venv
uv pip install -e '.[dev]'

# Or using pip
pip install -e .

Quick Start

import vmas
from scenarios.voting import Scenario

# Create the environment
env = vmas.make_env(
    scenario=Scenario(),
    num_envs=1,
    device="cpu",
    continuous_actions=True,
    max_steps=500,
    # Scenario configuration
    n_agents=4,
    aggregation_method="mean",
    topology_view="average",
    show_side_plots=True,
    dynamic_preferences=True,
    seed=1234,
)

# Run a simple simulation
for step in range(500):
    # Generate random actions (in practice, use a trained policy)
    actions = [
        env.scenario._torch_randn((1, 2), device=env.device) * 0.1
        for _ in range(env.n_agents)
    ]
    obs, rews, dones, infos = env.step(actions)
    
    # Check for convergence
    if env.scenario.converged[0]:
        print(f"Converged at step {env.scenario.step_count}")
        break
    
    env.render(mode="human", env_index=0)

Configuration Options

World Parameters

• n_agents: Number of voting agents (default: 3)
• xdim, ydim: Arena dimensions (default: 1.0)
• grid_spacing: Resolution for preference sampling (default: 0.05)
• wrap_borders: Enable toroidal wrapping (default: False)
• collisions: Enable collision detection (default: True)

Preference Generation

• n_gaussians: Number of Gaussian components per agent (default: 3)
• cov: Covariance of Gaussian components (default: 0.15)
• negative_gaussian_ratio: Ratio of negative (aversion) Gaussians (default: 0.3)
• dynamic_preferences: Enable time-evolving preferences (default: False)
• preference_drift_speed: Speed of preference evolution (default: 0.01)
• perlin_noise_scale: Spatial frequency of noise (default: 0.15)
• perlin_noise_speed: Time evolution speed (default: 0.1)

Aggregation Methods

• aggregation_method: One of "mean", "median", "geometric_median", "l_p", "l_infinity", "weighted_mean", "min", "max" (default: "mean")
• aggregation_p_value: p-value for L_p norm (default: 2.0)
• conviction_weighting: Enable conviction voting for weighted mean (default: False)

Physics & Mobility

• inertial_mobility: Enable physics-based movement (default: True)
• impulse_strength: Strength of action impulses (default: 0.4)
• damping_coefficient: Velocity damping (default: 0.98)
• restoration_strength: Velocity restoration (default: 0.01)
• max_velocity: Maximum velocity cap (default: None)

Convergence Detection

• convergence_threshold: Position change threshold (default: 0.01)
• convergence_window: Number of steps for stability check (default: 10)
• equilibrium_check_interval: Steps between equilibrium checks (default: 5)
• ideal_outcome_method: Method for computing ideal point ("maximin", "social_welfare", "nash", etc.)

Visualization

• topology_view: "average", "single", or "overlay" (default: "average")
• show_side_plots: Display individual agent preference plots (default: True)
• show_outcome_point: Display current outcome position (default: True)
• show_outcome_path: Display outcome trajectory history (default: True)
• show_ideal_outcome: Display ideal point marker (default: True)

Research Applications

This environment is designed for:

1. Learning Convergence Policies: Discovering algorithms that enable agents to find cooperative policies producing outcomes approximating ideal collective choices
2. Scaling to Higher Dimensions: Extending to N-dimensional preference spaces where top-down optimization becomes intractable
3. Real-World Transition: Benchmarking voting rules before deployment in:
   • Decentralized Autonomous Organizations (DAOs)
   • Participatory budgeting systems
   • Political systems
   • Other collective decision-making contexts

Citation

If you use this code in your research, please cite:

@techreport{vmas-voting,
  title={Emergent Consensus from Private Preferences via Reinforcement Learning in Iterative Spatial Voting},
  author={Felipe, Lucas Lopes},
  year={2025}
}

References

• VMAS: Vectorized Multi-Agent Simulator
• Original Proposal

License

GNU General Public License v3.0

Acknowledgments

This scenario is based on the VMAS sampling scenario and adapted for collective decision-making research.