Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .assets-revision
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
# is a git revision (branch name like `main`, a tag, or a specific commit
# sha). Override at runtime with the ASSETS_REVISION env var.
repo: ChilleD/WebHarbor
revision: 8d8e4069588ef55594622fee1ab9fa51c2011d07
revision: 68798983027ceaf17df62f6167625baae38a160a
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A coding agent (Claude Code, Cursor, Aider, Codex, ...) is reading this. Read on

## What it is

19 Flask mirror websites (Amazon, GitHub, BBC News, ...) packaged into one Docker image, plus a control plane on `:8101` for resetting per-site state. Used as a deterministic offline environment for web-agent benchmarks. ~3 GB image.
20 Flask mirror websites (Amazon, GitHub, BBC News, ...) packaged into one Docker image, plus a control plane on `:8101` for resetting per-site state. Used as a deterministic offline environment for web-agent benchmarks. ~3 GB image.

Two repos:
- **code** (this one) — Flask apps, control plane, scripts.
Expand Down Expand Up @@ -48,17 +48,17 @@ Inside the image, sites live at `/opt/WebSyn/<site>/`. The path predates the ren
# fresh clone
./scripts/fetch_assets.sh # pulls assets from HF
./scripts/build.sh # docker build -t webharbor:dev .
docker run -d -p 8101:8101 -p 40000-40018:40000-40018 webharbor:dev
docker run -d -p 8101:8101 -p 40000-40019:40000-40019 webharbor:dev
```

Or use the published image directly:

```bash
docker run -d -p 8101:8101 -p 40000-40018:40000-40018 \
docker run -d -p 8101:8101 -p 40000-40019:40000-40019 \
battalion7244/webharbor:latest
```

Sites are on `40000`-`40018` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:
Sites are on `40000`-`40019` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:

| Method | Path | Purpose |
|--------|---------------------|-------------------------------------------|
Expand Down Expand Up @@ -136,13 +136,13 @@ python3 -m py_compile sites/<site>/app.py

# 3. run on alt ports (don't collide with anything you already have running)
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41018:40000-40018 webharbor:dev
-p 8201:8101 -p 41000-41019:40000-40019 webharbor:dev

# 4. control plane healthy, all sites alive
curl -s http://localhost:8201/health | python3 -m json.tool | head

# 5. every site renders 200
for p in $(seq 41000 41018); do
for p in $(seq 41000 41019); do
curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
done

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ The full agent guide is loaded above via `@AGENTS.md`. The notes below apply onl

## Existing containers

If a container is already running on `:8101` / `:40000-40018`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41018`).
If a container is already running on `:8101` / `:40000-40019`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41019`).
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ git clone https://github.com/<you>/webharbor && cd webharbor
./scripts/fetch_assets.sh # pull current assets
./scripts/new_site.py mywebsite # OR edit an existing site
./scripts/build.sh && docker run -d --rm \
-p 8101:8101 -p 40000-40018:40000-40018 webharbor:dev
-p 8101:8101 -p 40000-40019:40000-40019 webharbor:dev
# iterate locally...

./scripts/extract_assets.sh ../webharbor-static-pr/ # split assets out
Expand Down
5 changes: 3 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# WebHarbor — slim, self-contained image.
# 19 Flask mirror sites + control plane on :8101.
# 20 Flask mirror sites + control plane on :8101.

FROM python:3.12-slim-bookworm

Expand All @@ -14,6 +14,7 @@ RUN pip3 install --no-cache-dir \
Flask-Login==0.6.3 \
Flask-WTF==1.2.2 \
Flask-Bcrypt==1.0.1 \
bcrypt==5.0.0 \
Werkzeug==3.1.3 \
Jinja2==3.1.4 \
SQLAlchemy==2.0.36 \
Expand All @@ -40,6 +41,6 @@ COPY control_server.py /opt/control_server.py
COPY site_runner.py /opt/site_runner.py
RUN chmod +x /opt/websyn_start.sh

EXPOSE 8101 40000-40018
EXPOSE 8101 40000-40019

CMD ["/opt/websyn_start.sh"]
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,17 @@ WebHarbor takes a different approach. We leverage coding agent (e.g., Claude Cod
- **Deep features unlocked** — carts, checkouts, accounts, all fully testable
- **Evolving** — harder tasks drive richer mirrors; the environment grows with agents
- **RL-ready** — sub-second database resets between rollouts
- **Community-driven** — 19 sites today, scaling to 100+ together
- **Community-driven** — 20 sites today, scaling to 100+ together

## 🚀 Quickstart

One command to run all web environments:

```bash
docker run -p 8101:8101 -p 40000-40018:40000-40018 battalion7244/webharbor:latest
docker run -p 8101:8101 -p 40000-40019:40000-40019 battalion7244/webharbor:latest
```

Then point your agent at `http://localhost:40000` through `http://localhost:40018` to explore 19 local mirrors of webvoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, and Target`.
Then point your agent at `http://localhost:40000` through `http://localhost:40019` to explore 19 local mirrors of webvoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, and Target`.

For sub-second reset between rollouts, expose the control plane and call `/reset/<site>`:

Expand Down
2 changes: 1 addition & 1 deletion agent_demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export OPENAI_BASE_URL=https://api.openai.com/v1 # or your Azure / vLLM endpoi

## Run a task

WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40018:40000-40018 battalion7244/webharbor:latest`).
WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40019:40000-40019 battalion7244/webharbor:latest`).

Run a single task from a site's `tasks.jsonl`:

Expand Down
1 change: 1 addition & 0 deletions agent_demo/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ description = "Minimal browser-use ReAct agent + LLM-as-judge demo for WebHarbor
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"bcrypt==5.0.0",
"browser-use>=0.10.1",
"openai>=1.50.0",
"playwright>=1.56.0",
Expand Down
Loading