Skip to content

Jmarinis/mongo_cluster

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
config
config
 
 
scripts
scripts
 
 
systemd
systemd
 
 
README.md
README.md
 
 

Repository files navigation

MongoDB Replica Set Cluster for Raspberry Pi 3B

High-availability MongoDB cluster across 4x Raspberry Pi 3 Model B nodes, tuned for 1GB RAM and 100Mbps Ethernet.

Architecture

┌─────────────────────────────────────────────────────────┐
│                    MongoDB Replica Set                   │
│                                                          │
│  Node 1 (192.168.1.101)  ──  Primary ⭐                 │
│  Node 2 (192.168.1.102)  ──  Secondary                  │
│  Node 3 (192.168.1.103)  ──  Secondary                  │
│  Node 4 (192.168.1.104)  ──  Arbiter ⚖️ (or Secondary)  │
└─────────────────────────────────────────────────────────┘

Connection String:
mongodb://192.168.1.101:27017,192.168.1.102:27017,192.168.1.103:27017/?replicaSet=rs0

Requirements

  • 4x Raspberry Pi 3 Model B (1GB RAM)
  • Raspberry Pi OS 64-bit (Debian Bookworm) - required for ARM64 MongoDB
  • SSH access to all nodes
  • Static IPs assigned to each node
  • External SSD recommended (better IOPS than SD cards)

Quick Setup

1. Prepare All Nodes

On each Pi, ensure you're running 64-bit OS:

uname -m  # Should return: aarch64

If not, flash Raspberry Pi OS 64-bit to your SD card/SSD.

2. Configure Node IPs

Set environment variables on the machine where you'll run scripts:

export NODE1_IP="192.168.1.101"
export NODE2_IP="192.168.1.102"
export NODE3_IP="192.168.1.103"
export NODE4_IP="192.168.1.104"
export REPLICA_SET_NAME="rs0"

3. Install MongoDB on Each Node

Copy the scripts and config to each node, then run:

# On Node 1
NODE_IP=192.168.1.101 sudo -E bash scripts/install-mongodb.sh

# On Node 2
NODE_IP=192.168.1.102 sudo -E bash scripts/install-mongodb.sh

# On Node 3
NODE_IP=192.168.1.103 sudo -E bash scripts/install-mongodb.sh

# On Node 4
NODE_IP=192.168.1.104 sudo -E bash scripts/install-mongodb.sh

Or use SSH to automate:

for i in 1 2 3 4; do
  NODE_IP="192.168.1.10${i}" \
  ssh pi@"192.168.1.10${i}" \
  'bash -s' < scripts/install-mongodb.sh
done

4. Initialize Replica Set

Run this ONCE on Node 1 (the node you want as primary):

bash scripts/init-replica-set.sh

This will:

  • Verify connectivity to all nodes
  • Initialize the replica set configuration
  • Wait for primary election
  • Apply performance tuning

5. Verify Cluster Health

bash scripts/cluster-health.sh

For continuous monitoring:

bash scripts/cluster-health.sh --watch

Manual Setup (Alternative)

If you prefer to set up manually without scripts:

On Each Node

# Install MongoDB
curl -fsSL https://www.mongodb.org/static/pgp/server-7.0.asc | \
  sudo gpg --dearmor -o /usr/share/keyrings/mongodb-server-7.0.gpg
echo "deb [signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg arch=arm64] https://repo.mongodb.org/apt/debian bookworm/mongodb-org/7.0 main" | \
  sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list
sudo apt update
sudo apt install -y mongodb-org

# Create tuned config (replace 192.168.1.101 with each node's IP)
sudo tee /etc/mongod.conf << 'EOF'
storage:
  dbPath: /var/lib/mongodb
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.25
      journalCompressor: snappy
    collectionConfig:
      blockCompressor: snappy
    indexConfig:
      prefixCompression: true

systemLog:
  destination: file
  path: "/var/log/mongodb/mongod.log"
  logAppend: true

net:
  port: 27017
  bindIp: 192.168.1.101,127.0.0.1  # CHANGE THIS
  maxIncomingConnections: 100

replication:
  replSetName: "rs0"
  oplogSizeMB: 512

processManagement:
  fork: false
  timeZoneInfo: /usr/share/zoneinfo
EOF

sudo systemctl enable mongod
sudo systemctl start mongod

On Node 1 (Initialize Replica Set)

mongosh --eval '
rs.initiate({
  _id: "rs0",
  members: [
    { _id: 0, host: "192.168.1.101:27017", priority: 3 },
    { _id: 1, host: "192.168.1.102:27017", priority: 2 },
    { _id: 2, host: "192.168.1.103:27017", priority: 1 },
    { _id: 3, host: "192.168.1.104:27017", arbiterOnly: true }
  ]
})
'

Systemd Hardening (Optional)

Apply memory limits and security restrictions:

sudo systemctl edit mongod
# Paste contents of: systemd/mongod.override.conf
sudo systemctl daemon-reload
sudo systemctl restart mongod

Performance Tuning

The configuration is already tuned for Pi 3B:

Setting Value Reason
cacheSizeGB 0.25 Leaves room for OS + other processes
oplogSizeMB 512 Enough for low-write workloads
maxIncomingConnections 100 Prevents connection overload
MemoryMax 700MB Hard limit to prevent OOM
CPUQuota 80% Prevents thermal throttling

Additional Tuning Tips

  • Use external SSD: Dramatically improves I/O performance
  • Disable swap: sudo dphys-swapfile swapoff && sudo systemctl disable dphys-swapfile
  • Set CPU governor: echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
  • Monitor temperature: vcgencmd measure_temp - throttle if > 80°C

Connecting Your Application

Node.js / mongoose

const mongoose = require('mongoose');

mongoose.connect(
  'mongodb://192.168.1.101:27017,192.168.1.102:27017,192.168.1.103:27017/myapp?replicaSet=rs0',
  {
    readPreference: 'secondaryPreferred',  // Read from secondaries when possible
    maxPoolSize: 10,                        // Keep connections low for 100Mbps
  }
);

Python / pymongo

from pymongo import MongoClient

client = MongoClient(
    'mongodb://192.168.1.101:27017,192.168.1.102:27017,192.168.1.103:27017/?replicaSet=rs0',
    readPreference='secondaryPreferred',
    maxPoolSize=10,
)

db = client['myapp']
collection = db['documents']

Monitoring

Check cluster health

# One-time check
bash scripts/cluster-health.sh

# Continuous watch
bash scripts/cluster-health.sh --watch

Manual checks

# Replica set status
mongosh --eval "rs.status()"

# Replication info
mongosh --eval "rs.printReplicationInfo()"
mongosh --eval "rs.printSecondaryReplicationInfo()"

# Server status
mongosh --eval "db.serverStatus()"

System monitoring

# Memory usage
free -h

# MongoDB process memory
ps aux | grep mongod

# Disk I/O
iostat -x 1

# Network throughput
iftop

Troubleshooting

MongoDB won't start

# Check logs
sudo journalctl -u mongod -f

# Check data directory permissions
ls -la /var/lib/mongodb
sudo chown -R mongodb:mongodb /var/lib/mongodb

Replica set election failing

  1. Ensure all nodes can reach each other (firewall rules)
  2. Verify /etc/mongod.conf has correct bindIp on each node
  3. Check NTP sync across all nodes: timedatectl

High replication lag

# Check oplog window
mongosh --eval "rs.printReplicationInfo()"

# If oplog is too small, increase it (on primary):
mongosh --eval "
  use local
  db.oplog.rs.drop()
  db.adminCommand({replSetResizeOplog: 1, size: 1024})
"

OOM killer terminating MongoDB

# Apply systemd memory limits
sudo systemctl edit mongod
# Add: MemoryMax=700M
sudo systemctl daemon-reload
sudo systemctl restart mongod

# Or reduce WiredTiger cache in /etc/mongod.conf
# storage.wiredTiger.engineConfig.cacheSizeGB: 0.2

Adding Authentication (Production)

After the cluster is running, enable authentication:

# Generate keyfile
openssl rand -base64 756 > /tmp/mongodb-keyfile
chmod 400 /tmp/mongodb-keyfile

# Copy to all nodes
for ip in 101 102 103 104; do
  scp /tmp/mongodb-keyfile pi@192.168.1.${ip}:/etc/mongodb-keyfile
  ssh pi@192.168.1.${ip} "sudo chown mongodb:mongodb /etc/mongodb-keyfile"
done

# Create admin user (on primary, before enabling auth)
mongosh
use admin
db.createUser({
  user: "admin",
  pwd: "strong_password_here",
  roles: [{ role: "root", db: "admin" }]
})

# Uncomment security section in /etc/mongod.conf on all nodes
# security:
#   authorization: enabled
#   keyFile: /etc/mongodb-keyfile

# Restart all nodes (one at a time, starting with secondaries)

File Structure

pg_cluster/
├── config/
│   └── mongod.conf              # MongoDB configuration template
├── scripts/
│   ├── install-mongodb.sh       # Automated installation
│   ├── init-replica-set.sh      # Replica set initialization
│   └── cluster-health.sh        # Health monitoring
├── systemd/
│   └── mongod.override.conf     # Systemd resource limits
└── README.md                    # This file

Expected Performance

On Raspberry Pi 3B (1GB RAM, 100Mbps Ethernet, external SSD):

Metric Value
Insert throughput 200-500 docs/sec
Query throughput 500-2000 queries/sec
Replication lag < 100ms (typical)
Failover time 10-30 seconds
Memory usage ~300-500MB per node

Next Steps

  • Set up automated backups: mongodump on secondary nodes
  • Configure alerts for failover events
  • Add application-level retry logic for failover handling
  • Monitor disk growth and oplog size

License

MIT - Use at your own risk

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages