diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 34df2ec..5a2ac11 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,8 +1,6 @@ # Contributing to Major League GitHub -Thank you for your interest in contributing to [Major League GitHub](https://www.mlg.soccer)! This is an open-source, standalone side project and contributions of all kinds are welcome — bug fixes, new features, documentation improvements, and more. - -> **Repository:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) +Thank you for your interest in contributing to **Major League GitHub**! This is an independent, open-source side project and contributions of all kinds are welcome — bug reports, documentation improvements, feature requests, and code changes. --- @@ -10,185 +8,173 @@ Thank you for your interest in contributing to [Major League GitHub](https://www - [Code of Conduct](#code-of-conduct) - [Getting Started](#getting-started) -- [Project Structure](#project-structure) -- [Development Workflow](#development-workflow) -- [Two Backend Profiles](#two-backend-profiles) +- [Development Environment](#development-environment) +- [Branch Naming](#branch-naming) +- [Commit Messages](#commit-messages) +- [Pull Request Process](#pull-request-process) - [Code Style](#code-style) -- [Environment Variables](#environment-variables) -- [Submitting a Pull Request](#submitting-a-pull-request) -- [Security Vulnerabilities](#security-vulnerabilities) +- [Security Guidelines](#security-guidelines) +- [Reporting Issues](#reporting-issues) --- ## Code of Conduct -Be respectful and constructive. This project follows standard open-source community norms — harassment, discrimination, or hostile behavior of any kind will not be tolerated. +This project follows standard open-source community norms. Be respectful, constructive, and welcoming. Harassment of any kind will not be tolerated. --- ## Getting Started -### Prerequisites +### 1. Fork and Clone + +```bash +git clone https://github.com/flamingo-stack/major-league-github.git +cd major-league-github +``` + +### 2. Set Up the Development Environment + +**Prerequisites:** | Tool | Minimum Version | |------|----------------| -| Java (JDK) | 21 | -| Maven | 3.9+ | +| Java JDK | 21 | +| Apache Maven | 3.9+ | | Node.js | 18+ | | npm | 9+ | -| Redis | 7+ (or Docker) | -| Docker | 24+ | -| Git | 2.40+ | +| Redis | 6+ | -### Fork and Clone +**Backend setup:** ```bash -# Fork the repository on GitHub, then: -git clone https://github.com//major-league-github.git -cd major-league-github -git remote add upstream https://github.com/flamingo-stack/major-league-github.git +cd backend +GITHUB_TOKENS=ghp_your_token_here mvn spring-boot:run ``` -### Start Redis +The backend starts on port **8450** (REST API) by default. + +**Frontend setup:** ```bash -docker run -d -p 6379:6379 --name mlg-redis redis:7 +cd frontend +npm install +BACKEND_API_URL=http://localhost:8450 npm run dev ``` -### Run the Backend Service +The frontend dev server starts on port **3000**. -```bash -export GITHUB_TOKENS="ghp_yourTokenHere" -export SPRING_REDIS_HOST="localhost" -export SPRING_REDIS_PORT="6379" +> **No Redis?** Use the disk cache for simpler local development: +> ```bash +> GITHUB_TOKENS=ghp_your_token_here CACHE_IMPLEMENTATION=disk mvn spring-boot:run +> ``` -cd backend -mvn spring-boot:run -Pbackend-service -``` +See the [Local Development Guide](./docs/development/setup/local-development.md) for full details including debug configurations for IntelliJ IDEA and VS Code. -Health check: +--- -```bash -curl http://localhost:8450/actuator/health -# Expected: {"status":"UP"} -``` +## Development Environment -### Run the Cache Updater +### IDE Recommendations -```bash -# In a new terminal -cd backend -GITHUB_TOKENS=$GITHUB_TOKENS \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pcache-updater -``` +**Backend (Java / Spring Boot):** +- **IntelliJ IDEA** (recommended) — enable Lombok annotation processing in Settings → Build, Execution, Deployment → Compiler → Annotation Processors +- **VS Code** — install Extension Pack for Java, Spring Boot Extension Pack, and Lombok Annotations Support -### Run the Frontend Dev Server +**Frontend (React / TypeScript):** +- **VS Code** (recommended) — install ESLint, Prettier, TypeScript extensions +- **WebStorm** — excellent TypeScript and React support out of the box -```bash -cd frontend -npm install -BACKEND_API_URL=http://localhost:8450 npx webpack serve -``` +### Environment Variables -Open [http://localhost:8450](http://localhost:8450). See [Local Development](./docs/development/setup/local-development.md) for the full guide. +Set these before running the backend: ---- +```bash +# Required +export GITHUB_TOKENS="ghp_your_token_here" -## Project Structure +# Optional — use disk cache instead of Redis +export CACHE_IMPLEMENTATION="disk" -```text -major-league-github/ -├── backend/ # Java 21 + Spring Boot 3.4 backend -│ └── src/main/java/cx/flamingo/analysis/ -│ ├── controller/ # REST API controllers -│ ├── service/ # Business logic services -│ ├── cache/ # Cache abstraction + implementations -│ ├── config/ # Spring configuration -│ ├── graphql/ # GitHub GraphQL query builder -│ ├── model/ # Domain models -│ ├── rate/ # GitHub token rate management -│ └── exception/ # Exception handling -├── frontend/ # React 19 + TypeScript frontend -│ └── src/ -│ ├── components/ # UI components -│ ├── hooks/ # Custom React hooks -│ ├── services/ # API integration layer -│ └── types/ # TypeScript type definitions -└── docs/ # Documentation +# Optional — Redis connection (defaults to localhost:6379) +export SPRING_REDIS_HOST="localhost" +export SPRING_REDIS_PORT="6379" + +# Optional — frontend API URL (for separate frontend/backend) +export BACKEND_API_URL="http://localhost:8450" ``` --- -## Development Workflow +## Branch Naming + +Use descriptive branch names with a prefix indicating the type of change: -### Branch Naming +| Prefix | When to use | +|--------|-------------| +| `feat/` | New feature | +| `fix/` | Bug fix | +| `docs/` | Documentation changes | +| `refactor/` | Code refactoring (no behavior change) | +| `chore/` | Tooling, dependencies, build config | +| `test/` | Adding or improving tests | -Use descriptive, lowercase, hyphen-separated branch names: +**Examples:** ```text -feat/add-team-filter-pagination -fix/cache-miss-on-empty-results -docs/update-quick-start -chore/bump-spring-boot-version +feat/hiring-profile-caching +fix/rate-limit-token-rotation +docs/api-endpoint-reference +refactor/city-service-loading ``` -Prefixes: -- `feat/` — new feature -- `fix/` — bug fix -- `docs/` — documentation only -- `chore/` — dependency updates, tooling, non-functional changes -- `refactor/` — code restructuring without behavior change - -### Workflow - -1. **Create a branch** from `main`: - -```bash -git checkout main -git pull upstream main -git checkout -b feat/your-feature-name -``` +--- -2. **Make changes** — keep commits focused and atomic. +## Commit Messages -3. **Verify your changes:** +Write clear, concise commit messages that explain *what* and *why*: -```bash -# Backend: compile without tests -cd backend && mvn compile -DskipTests +- Use the imperative mood: "Add cache invalidation" not "Added cache invalidation" +- Keep the first line under 72 characters +- Optionally add a body for context -# Backend: run tests (if available) -cd backend && mvn test +**Examples:** -# Frontend: lint -cd frontend && npx eslint src/ +```text +feat: add Haversine proximity filter for MLS teams -# Frontend: build check -cd frontend && npx webpack --mode production -``` +fix: handle empty GITHUB_TOKENS env var gracefully -4. **Push and open a PR:** +docs: document cache mode configuration options -```bash -git push origin feat/your-feature-name +refactor: extract scoring formula into separate method ``` -Then open a pull request at [https://github.com/flamingo-stack/major-league-github/pulls](https://github.com/flamingo-stack/major-league-github/pulls). - --- -## Two Backend Profiles +## Pull Request Process + +1. **Create a branch** from `main` using the naming convention above +2. **Make your changes** — keep PRs focused and small where possible +3. **Verify locally:** + - Backend compiles and starts: `mvn spring-boot:run` + - Frontend builds and lints: `npm run build` and `npx eslint src/` + - The app functions at `http://localhost:3000` +4. **Write a clear PR description** explaining what changed and why +5. **Open the PR** against the `main` branch at [https://github.com/flamingo-stack/major-league-github/pulls](https://github.com/flamingo-stack/major-league-github/pulls) +6. **Address review feedback** — be responsive to comments -The backend runs as **two separate services** from one codebase, controlled by Maven profiles: +### PR Checklist -| Service | Maven Profile | Port | Role | -|---------|--------------|------|------| -| Backend Service | `backend-service` | 8450 | Serves the REST API | -| Cache Updater | `cache-updater` | 8451 | Runs scheduled cache refresh jobs | +Before submitting: -Always test changes against the relevant profile. If you modify caching logic, test both profiles. +- [ ] No secrets, tokens, or passwords committed to source +- [ ] No hardcoded URLs that should be configurable +- [ ] Backend compiles cleanly (`mvn clean package -DskipTests`) +- [ ] Frontend lints cleanly (`npx eslint src/`) +- [ ] No wildcard CORS (`allowedOrigins("*")`) introduced +- [ ] New endpoints or behavior changes are documented +- [ ] `ApiError` responses do not expose raw stack traces --- @@ -196,126 +182,101 @@ Always test changes against the relevant profile. If you modify caching logic, t ### Backend (Java) -- **Style:** Standard Java conventions; Lombok annotations (`@Data`, `@Builder`, `@RequiredArgsConstructor`) are used throughout. -- **Annotation processing:** Must be enabled in your IDE (IntelliJ: Settings → Compiler → Annotation Processors → Enable). -- **Constructor injection:** Prefer constructor injection over field injection for dependencies. -- **Thin controllers:** Controllers should delegate to services — no business logic in controllers. -- **Logging:** Use SLF4J. Log request parameters, cache state, and rate-limit status. **Never log token values or raw API responses containing personal data.** +- **Java 21** — use modern Java features where appropriate +- **Lombok** — use `@Data`, `@Builder`, `@Slf4j`, etc. for boilerplate reduction +- **Spring conventions** — follow standard Spring Boot layering (Controller → Service → Repository/Cache) +- **Thin controllers** — business logic belongs in services, not controllers +- **Structured GraphQL** — use `GitHubQueryBuilder` for GitHub API queries; never build GraphQL strings manually +- **Async** — use `@Async` for long-running tasks; configure thread pools in the `Configurations` module ### Frontend (TypeScript / React) -- **ESLint:** Configured in `frontend/eslint.config.js`. Run before committing: - -```bash -cd frontend && npx eslint src/ -``` +- **TypeScript strict mode** — all props and state should be typed +- **React hooks rules** — follow the Rules of Hooks; ESLint will enforce this +- **URL-driven state** — use the `useUrlState` hook for filter state; do not use component-local state for URL-persisted filters +- **Axios service layer** — all API calls go through `frontend/src/services/`; do not call the backend directly from components +- **Material UI** — use MUI components consistently; avoid inline styles where a theme solution exists -- **Formatting:** Prettier is configured. Enable format-on-save in your editor. -- **Strong typing:** All API responses should be typed against the interfaces in `src/types/`. Avoid `any`. -- **URL state:** Filter state is managed via the `useUrlState` hook. Do not store filter state in component state or context — keep it URL-driven. -- **Path aliases:** Use `@/` for `frontend/src/` imports: +### ESLint -```typescript -import { ContributorsTable } from '@/components/ContributorsTable' +```bash +cd frontend +npx eslint src/ ``` -### IDE Setup - -**Backend (IntelliJ IDEA):** -- Install plugins: **Lombok**, **Spring** -- Set Project SDK to Java 21 -- Enable annotation processing - -**Frontend (VS Code):** -- Install extensions: **ESLint**, **Prettier**, **TypeScript Language Features**, **GitLens** -- Create `.vscode/settings.json`: - -```json -{ - "editor.formatOnSave": true, - "editor.defaultFormatter": "esbenp.prettier-vscode", - "typescript.tsdk": "frontend/node_modules/typescript/lib", - "eslint.workingDirectories": ["frontend"] -} -``` +The project uses ESLint 9 with `eslint-plugin-react-hooks` and `typescript-eslint`. Fix all warnings before submitting a PR. --- -## Environment Variables - -### Never commit secrets +## Security Guidelines -Add your local environment file to `.gitignore`: +### Never Commit Secrets -```bash -echo ".env.local" >> .gitignore -``` +- GitHub tokens, LinkedIn credentials, and any API keys must **never** be committed to source control +- Use environment variables locally; use GitHub Secrets for CI/CD; use Kubernetes Secrets in production +- Add `.env` and `.env.local` to `.gitignore` if you use env files -### Required Variables +### Token Scopes -| Variable | Service | Description | -|----------|---------|-------------| -| `GITHUB_TOKENS` | Backend | Comma-separated GitHub PATs with `read:user` scope | -| `SPRING_REDIS_HOST` | Backend | Redis host (e.g., `localhost`) | -| `SPRING_REDIS_PORT` | Backend | Redis port (default: `6379`) | -| `SPRING_PROFILES_ACTIVE` | Backend | `backend-service` or `cache-updater` | -| `BACKEND_API_URL` | Frontend | Backend base URL for dev server proxy | +Only the minimum required GitHub token scopes should be used: -### GitHub Token Scopes +- `read:user` — read public user data +- `public_repo` — access public repository data -Minimum required scopes: -- `read:user` — contributor profile data -- `repo` — repository star counts +Do **not** request write, admin, or delete permissions. -> Multiple tokens are supported for higher throughput. The `GithubTokenRateManager` automatically selects the token with the most remaining quota. +### Input Validation ---- +- Backend: validate all request parameters before passing to services +- Frontend: the `useUrlState` hook validates URL parameters against `^[a-zA-Z0-9-]+$` — maintain this pattern for new filters +- Use `GitHubQueryBuilder`'s structured arguments for GitHub API queries — never string-interpolate user input into GraphQL -## Pull Request Guidelines +### Error Responses -### Before Opening a PR +- Use `GlobalExceptionHandler` and `ApiResponse.error()` — never expose raw stack traces in API responses +- Rate limit exceptions (`GithubRateLimitException`, `GithubTimeoutException`, etc.) should be handled gracefully -- [ ] Code compiles without errors (`mvn compile -DskipTests` / `npx webpack --mode production`) -- [ ] No secrets, tokens, or personal data in code or test fixtures -- [ ] ESLint passes for frontend changes (`npx eslint src/`) -- [ ] New environment variables are documented with empty defaults -- [ ] CORS origins are not expanded without justification -- [ ] Rate limiting behavior is preserved — do not bypass `GithubTokenRateManager` -- [ ] Log statements do not include token values or raw API responses +### Dependency Audits -### PR Description +Before merging dependency updates, run: -Include: -- **What** the change does -- **Why** it is needed -- **How** to test it locally -- Any related issues (e.g., `Closes #42`) +```bash +# Backend +cd backend +mvn dependency:resolve -### Review Process +# Frontend +cd frontend +npm audit +``` -- All PRs require at least one review before merge -- Maintainers may request changes — this is normal and part of the process -- Keep PRs focused — one logical change per PR is easier to review +Fix moderate and high-severity findings before merging. --- -## Security Vulnerabilities +## Reporting Issues + +- **Bugs and feature requests:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) +- **Security vulnerabilities:** Open a private security advisory via GitHub's **Security → Advisories** feature in the repository -**Do not open a public issue for security vulnerabilities.** +When filing a bug report, please include: -Please report security issues responsibly via [GitHub Security Advisories](https://github.com/flamingo-stack/major-league-github/security/advisories). This allows the maintainers to assess and patch the issue before public disclosure. +1. Steps to reproduce +2. Expected behavior +3. Actual behavior +4. Environment details (OS, Java version, Node.js version, browser) +5. Relevant logs (redact any tokens before pasting) --- -## Getting Help +## Additional Resources -- **Browse open issues:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) -- **Open a new issue:** [https://github.com/flamingo-stack/major-league-github/issues/new](https://github.com/flamingo-stack/major-league-github/issues/new) -- **Read the architecture docs:** [docs/development/architecture/README.md](./docs/development/architecture/README.md) -- **Full development guide:** [docs/development/setup/local-development.md](./docs/development/setup/local-development.md) +- [Documentation](./docs/README.md) — Full project documentation index +- [Architecture Overview](./docs/development/architecture/README.md) — System design and data flow +- [Local Development Guide](./docs/development/setup/local-development.md) — Detailed setup instructions +- [Security Guidelines](./docs/development/security/README.md) — Full security reference +- [Live Site](https://www.mlg.soccer) — See the project in production --- -
- Built with 💛 by the Flamingo team -
+Thank you for contributing to Major League GitHub! 🏆 diff --git a/README.md b/README.md index e45e633..1bc1c5f 100644 --- a/README.md +++ b/README.md @@ -7,110 +7,113 @@

- Live Site - License - Issues + License

# Major League GitHub -**[Major League GitHub](https://www.mlg.soccer)** is an open-source, sports-themed leaderboard that ranks GitHub contributors like professional soccer players. It maps open-source developers across the United States by programming language, geographic location, and proximity to MLS stadiums — combining GitHub GraphQL analytics with geospatial modeling to create a gamified developer leaderboard. +**[mlg.soccer](https://www.mlg.soccer)** — An independent, open-source side project that ranks GitHub contributors like professional soccer players. Inspired by Major League Soccer (MLS), it filters contributors by programming language, geographic location, and proximity to real MLS stadiums — turning open-source contribution data into a competitive, engaging leaderboard experience. -> **Live:** [https://www.mlg.soccer](https://www.mlg.soccer) · **Repository:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) +> Who are the top Java developers within 50 miles of a Chicago MLS stadium? Major League GitHub answers exactly that. --- ## Features -- **Language Filtering** — Filter contributors by any programming language (Java, Python, TypeScript, Go, and more) -- **Geographic Filtering** — Narrow results by city, state, or geographic region -- **MLS Stadium Proximity** — Rank contributors by distance to the nearest MLS stadium using Haversine distance -- **Contributor Scoring** — Transparent scoring formula: `commits × max(starsReceived, 1) × recencyMultiplier` -- **Real-Time Leaderboard** — GitHub GraphQL data refreshed on a schedule via the Cache Updater microservice -- **Shareable URLs** — Every filter combination is encoded in the URL — bookmark or share any leaderboard view -- **CSV Export** — Download any filtered leaderboard as a CSV file for hiring, analytics, or research -- **Hiring Section** — Highlights top contributors alongside associated job openings -- **Responsive UI** — Works across desktop and mobile with Material-UI components -- **Cache-First Architecture** — Redis-backed distributed cache with async background refresh to minimize API latency -- **Multi-Token GitHub Rate Management** — Distributes requests across multiple GitHub PATs for resilient throughput -- **SEO Build Optimization** — Custom Webpack plugins auto-generate `sitemap.xml`, `robots.txt`, and `favicon.ico` +- **Language Filtering** — Filter contributors by any programming language (Java, TypeScript, Python, and more) +- **Geographic Filtering** — Filter by city, state, or multi-state MLS region +- **MLS Team Proximity** — Find contributors near any Major League Soccer stadium using Haversine distance +- **Contributor Scoring** — Rank by a formula: `commits × max(stars, 1) × recencyMultiplier` +- **CSV Export** — Download ranked results as a CSV with social profile links +- **Hiring Mode** — Hiring managers can publish open roles and appear in contributor profiles +- **Distributed Cache** — Redis-backed caching protects GitHub API rate limits and serves fast results +- **URL-Driven State** — Every filter persists in the URL — results are shareable and bookmarkable --- ## Architecture -Major League GitHub is a distributed full-stack application split into two backend microservices and a React frontend: +Major League GitHub is a full-stack, microservice-based system: ```mermaid -flowchart TD - User["User Browser"] --> Frontend["React 19 Frontend"] - Frontend --> Backend["Backend Service (Port 8450)"] - Backend --> Redis[("Redis Cache")] +flowchart LR + User["User (Browser)"] --> Frontend["React 19 Frontend (Port 3000)"] + Frontend --> Backend["Backend Service (Spring Boot - Port 8450)"] + Backend --> Redis["Redis Cache"] Backend --> GitHub["GitHub GraphQL API"] - Backend --> LinkedIn["LinkedIn API (Hiring)"] - CacheUpdater["Cache Updater (Port 8451)"] --> Redis + Backend --> LinkedIn["LinkedIn API"] + CacheUpdater["Cache Updater (Port 8451)"] --> Backend CacheUpdater --> GitHub - GitHubActions["GitHub Actions CI/CD"] --> Docker["Docker Images"] - Docker --> GKE["Google Kubernetes Engine"] - GKE --> Backend - GKE --> CacheUpdater - GKE --> Redis ``` -### Backend Request Flow +### Microservices + +| Service | Port | Responsibility | +|---------|------|----------------| +| Backend Service | 8450 | REST API, contributor ranking engine | +| Cache Updater | 8451 | Scheduled cache pre-warming | + +### Contributor Ranking Engine ```mermaid -sequenceDiagram - participant Client as "Frontend" - participant Controller as "ContributorController" - participant Cache as "CacheServiceAbs" - participant Service as "GithubService" - participant Rate as "GithubTokenRateManager" - participant GitHub as "GitHub GraphQL API" - - Client->>Controller: GET /api/contributors/search - Controller->>Cache: isCacheReady()? - Cache-->>Controller: true - Controller->>Cache: getHttpResponse(filters, loader) - Cache->>Service: getTopContributorsIn(cities, language) - Service->>Rate: getBestAvailableClient() - Rate-->>Service: WebClient - Service->>GitHub: POST /graphql - GitHub-->>Service: JSON response - Service-->>Cache: List - Cache-->>Controller: Cached response - Controller-->>Client: ApiResponse> +flowchart TD + Request["Contributor Search Request"] --> CacheCheck["Cache Lookup"] + CacheCheck -->|"Hit"| Response["ApiResponse with Contributor List"] + CacheCheck -->|"Miss"| GithubFetch["GithubService"] + GithubFetch --> QueryBuilder["GitHubQueryBuilder (GraphQL)"] + QueryBuilder --> GitHubAPI["GitHub GraphQL API"] + GitHubAPI --> Parse["Parse and Map to Contributor"] + Parse --> Score["Apply Scoring Formula"] + Score --> Store["Store in Cache"] + Store --> Response +``` + +The scoring formula: + +```text +score = commits × max(starsReceived, 1) × recencyMultiplier +``` + +- **recencyMultiplier** ranges from `1.0` to `2.0`, rewarding contributors active in the past year + +### Deployment + +```mermaid +flowchart LR + GitHubRepo["GitHub Repository"] --> CI["GitHub Actions CI/CD"] + CI --> Docker["Docker Images"] + Docker --> GKE["Google Kubernetes Engine"] + GKE --> BackendPods["Backend + Cache Updater Pods"] + GKE --> RedisPod["Redis Pod"] + GKE --> FrontendService["Frontend Service"] ``` --- -## Tech Stack +## Technology Stack | Layer | Technology | |-------|-----------| -| Backend | Java 21 + Spring Boot 3.4 | -| Frontend | React 19 + TypeScript + Material-UI | -| State Management | URL-driven state via `useUrlState` hook | -| API Integration | Axios + React Query | -| Caching | Redis 7 (distributed) | -| External Data | GitHub GraphQL API | -| Hiring Data | LinkedIn API (optional) | -| Build | Webpack + custom SEO + favicon plugins | -| Deployment | Docker + Kubernetes (GKE) | -| CI/CD | GitHub Actions | +| **Backend** | Java 21, Spring Boot 3.4, Maven | +| **Frontend** | React 19, TypeScript, Material UI, TanStack React Query | +| **API Integration** | GitHub GraphQL API (multi-token rate management) | +| **Caching** | Redis (distributed), Disk (local dev fallback) | +| **Build** | Webpack 5 (production), Vite (dev server) | +| **Deployment** | Docker, Kubernetes (GKE), GitHub Actions CI/CD | --- ## Quick Start -Get the full stack running locally in about 5 minutes. +Get the app running locally in about 5 minutes. ### Prerequisites -- Java 21+, Maven 3.9+ +- Java 21+ +- Apache Maven 3.9+ - Node.js 18+, npm 9+ -- Docker (for Redis) -- A [GitHub Personal Access Token](https://github.com/settings/tokens) with `read:user` scope +- Redis 6+ +- A [GitHub Personal Access Token](https://github.com/settings/tokens) (scopes: `read:user`, `public_repo`) ### Run It @@ -120,125 +123,115 @@ git clone https://github.com/flamingo-stack/major-league-github.git cd major-league-github # 2. Start Redis -docker run -d -p 6379:6379 --name mlg-redis redis:7 - -# 3. Start the Backend Service (port 8450) -cd backend -GITHUB_TOKENS=your_github_pat \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pbackend-service +redis-server -# 4. (New terminal) Start the Cache Updater (port 8451) +# 3. Start the backend (new terminal) cd backend -GITHUB_TOKENS=your_github_pat \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pcache-updater +GITHUB_TOKENS=ghp_your_token_here mvn spring-boot:run -# 5. (New terminal) Start the Frontend Dev Server +# 4. Start the frontend (new terminal) cd frontend npm install -BACKEND_API_URL=http://localhost:8450 npx webpack serve +BACKEND_API_URL=http://localhost:8450 npm run dev ``` -Open [http://localhost:8450](http://localhost:8450) in your browser. The leaderboard will appear once the `PreCacheService` finishes its first warm-up pass (typically 30–90 seconds). +Open **[http://localhost:3000](http://localhost:3000)** in your browser. + +> **No Redis?** Use the disk cache instead — no Redis required: +> ```bash +> GITHUB_TOKENS=ghp_your_token_here CACHE_IMPLEMENTATION=disk mvn spring-boot:run +> ``` + +> **Multiple tokens?** Provide them comma-separated for higher API throughput: +> ```bash +> GITHUB_TOKENS=ghp_token1,ghp_token2,ghp_token3 mvn spring-boot:run +> ``` -### API Quick Test +### Verify the Backend ```bash -curl http://localhost:8450/api/contributors/search?languageId=java&maxResults=5 +curl http://localhost:8450/actuator/health +# {"status":"UP"} + +curl "http://localhost:8450/api/contributors/search?languageId=java&maxResults=5" ``` +--- + +## REST API + +The backend exposes a REST API on port 8450: + +| Endpoint | Description | +|----------|-------------| +| `GET /api/contributors/search` | Ranked contributors (filterable by language, city, state, region, team) | +| `GET /api/contributors/export` | Download results as CSV | +| `GET /api/autocomplete/cities` | City autocomplete | +| `GET /api/autocomplete/languages` | Language autocomplete | +| `GET /api/autocomplete/regions` | Region autocomplete | +| `GET /api/autocomplete/states` | State autocomplete | +| `GET /api/autocomplete/teams` | MLS team autocomplete | +| `GET /api/entities/cities/{id}` | Look up a city by ID | +| `GET /api/entities/languages/{id}` | Look up a language by ID | +| `GET /api/hiring/manager` | Hiring manager profile | +| `GET /api/hiring/jobs` | Active job openings | + +All endpoints return a consistent JSON envelope: + ```json { "status": "success", - "message": "Found 5 contributors matching the criteria", + "message": null, "data": [...] } ``` --- -## Project Structure +## Repository Structure ```text major-league-github/ -├── backend/ # Java 21 + Spring Boot 3.4 +├── backend/ # Java 21 + Spring Boot 3.4 (both microservices) │ └── src/main/java/cx/flamingo/analysis/ -│ ├── controller/ # REST API controllers (port 8450) -│ ├── service/ # Business logic + GitHub integration -│ ├── cache/ # Cache abstraction + Redis/disk implementations -│ ├── config/ # Spring configuration (CORS, Redis, scheduling) +│ ├── MajorLeagueGithubApplication.java +│ ├── cache/ # Cache abstraction (Redis, Disk, ReadOnly) +│ ├── config/ # Spring configuration (CORS, async, Redis) +│ ├── controller/ # REST controllers │ ├── graphql/ # GitHub GraphQL query builder -│ ├── model/ # Domain models (Contributor, City, Region, etc.) -│ └── rate/ # GitHub token rate management -├── frontend/ # React 19 + TypeScript (Webpack) -│ └── src/ -│ ├── components/ # UI components (ContributorsTable, FiltersPanel) -│ ├── hooks/ # useUrlState, useNearestRegion -│ ├── services/ # Axios API service layer -│ └── types/ # TypeScript API type definitions -└── docs/ # Full documentation -``` - ---- - -## Contributor Scoring - -The scoring formula is transparent and intentional: - -```text -Score = commits × max(starsReceived, 1) × recencyMultiplier +│ ├── model/ # Domain models (Contributor, City, Region…) +│ ├── rate/ # Multi-token GitHub rate management +│ └── service/ # Business logic +└── frontend/ # React 19 + TypeScript frontend + └── src/ + ├── components/ # UI components (table, filters, autocomplete) + ├── hooks/ # useNearestRegion, useUrlState + ├── services/ # Axios-based API layer + └── types/ # TypeScript contracts mirroring backend models ``` -| Component | Source | Effect | -|-----------|--------|--------| -| `commits` | GitHub contributions calendar | Rewards high activity volume | -| `starsReceived` | Stars on language-specific repos | Rewards community impact | -| `recencyMultiplier` | Activity freshness (1.0–2.0) | Rewards recent contributions | - --- -## REST API +## Documentation -| Endpoint | Description | -|----------|-------------| -| `GET /api/contributors/search` | Search and rank contributors by filters | -| `GET /api/contributors/export` | Download leaderboard as CSV | -| `GET /api/autocomplete/cities` | City autocomplete suggestions | -| `GET /api/autocomplete/languages` | Language autocomplete suggestions | -| `GET /api/autocomplete/regions` | Region autocomplete suggestions | -| `GET /api/autocomplete/states` | State autocomplete suggestions | -| `GET /api/autocomplete/teams` | MLS team autocomplete suggestions | -| `GET /api/entities/teams/{id}` | Look up an MLS team by ID | -| `GET /api/entities/regions/{id}` | Look up a region by ID | -| `GET /api/hiring/manager` | Hiring manager profile | -| `GET /api/hiring/jobs` | Active job openings | -| `GET /actuator/health` | Backend health check | +📚 See the [Documentation](./docs/README.md) for full guides covering setup, architecture, and development workflows. --- -## Documentation +## Contributing -📚 See the [Documentation](./docs/README.md) for comprehensive guides including getting started tutorials, local development setup, architecture deep-dives, and security guidelines. +Contributions are welcome! Please read [CONTRIBUTING.md](./CONTRIBUTING.md) before opening a pull request. -- [Introduction](./docs/getting-started/introduction.md) — What is Major League GitHub? -- [Prerequisites](./docs/getting-started/prerequisites.md) — Required tools and accounts -- [Quick Start](./docs/getting-started/quick-start.md) — Run the full stack in 5 minutes -- [First Steps](./docs/getting-started/first-steps.md) — Explore features after startup -- [Local Development](./docs/development/setup/local-development.md) — Full development workflow -- [Architecture Overview](./docs/development/architecture/README.md) — System design and module map +- [Open an Issue](https://github.com/flamingo-stack/major-league-github/issues) +- [Open a Pull Request](https://github.com/flamingo-stack/major-league-github/pulls) --- -## Contributing - -Contributions are welcome! Please read [CONTRIBUTING.md](./CONTRIBUTING.md) to understand the development workflow, code style, branching conventions, and PR process. +## Links -- 🐛 [Report a bug](https://github.com/flamingo-stack/major-league-github/issues) -- 💡 [Request a feature](https://github.com/flamingo-stack/major-league-github/issues) -- 🔒 [Report a security vulnerability](https://github.com/flamingo-stack/major-league-github/security/advisories) +- **Live Site:** [https://www.mlg.soccer](https://www.mlg.soccer) +- **Repository:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) +- **Issues:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) --- diff --git a/docs/README.md b/docs/README.md index f36f44d..07f5b96 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,6 @@ # Major League GitHub — Documentation -Welcome to the documentation for **[Major League GitHub](https://www.mlg.soccer)**, an open-source, sports-themed leaderboard that ranks GitHub contributors like professional soccer players — filtered by programming language, geographic location, and proximity to MLS stadiums. - -> **Live site:** [https://www.mlg.soccer](https://www.mlg.soccer) · **GitHub:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) +Welcome to the documentation for **[Major League GitHub](https://www.mlg.soccer)** — an open-source, sports-themed leaderboard that ranks GitHub contributors like professional soccer players, filtered by programming language, geographic location, and proximity to MLS stadiums. --- @@ -11,114 +9,88 @@ Welcome to the documentation for **[Major League GitHub](https://www.mlg.soccer) - [Getting Started](#-getting-started) - [Development](#-development) - [Reference Architecture](#-reference-architecture) -- [Diagrams](#-diagrams) +- [Architecture Diagrams](#-architecture-diagrams) - [Quick Links](#-quick-links) --- ## 🚀 Getting Started -New to Major League GitHub? Start here. +New to the project? Start here. -| Guide | Description | -|-------|-------------| -| [Introduction](./getting-started/introduction.md) | What Major League GitHub is, what it does, and who it's for | -| [Prerequisites](./getting-started/prerequisites.md) | Required tools, GitHub PATs, and environment variables | -| [Quick Start](./getting-started/quick-start.md) | Run the full stack locally in about 5 minutes | -| [First Steps](./getting-started/first-steps.md) | Explore filters, scoring, CSV export, and the REST API | +| Document | Description | +|----------|-------------| +| [Introduction](./getting-started/introduction.md) | What Major League GitHub is, who it's for, and how it works | +| [Prerequisites](./getting-started/prerequisites.md) | Required tools, accounts, and environment variables | +| [Quick Start](./getting-started/quick-start.md) | Get the app running locally in ~5 minutes | +| [First Steps](./getting-started/first-steps.md) | Explore the leaderboard, REST API, and configuration options | --- -## 🛠 Development +## 🛠️ Development -Guides for contributors and engineers working on the codebase. +Guides for contributors and developers working on the project. -| Guide | Description | -|-------|-------------| -| [Development Overview](./development/README.md) | Project structure, two backend profiles, and quick links | -| [Local Development](./development/setup/local-development.md) | Clone, build, run, debug, and inspect Redis locally | -| [Environment Setup](./development/setup/environment.md) | IDE configuration, editor extensions, linting, and path aliases | -| [Security](./development/security/README.md) | Secrets management, CORS, input validation, and vulnerability reporting | -| [Architecture Overview](./development/architecture/README.md) | System design, data flow, module map, and key design decisions | +| Document | Description | +|----------|-------------| +| [Development Overview](./development/README.md) | Documentation index, repository structure, and key dependencies | +| [Environment Setup](./development/setup/environment.md) | IDE recommendations, Lombok setup, ESLint, and editor extensions | +| [Local Development](./development/setup/local-development.md) | Clone, run, hot reload, debug configurations, and production builds | +| [Architecture Overview](./development/architecture/README.md) | System diagram, component breakdown, data flow, and design decisions | +| [Security Guidelines](./development/security/README.md) | Token management, secrets, CORS, input validation, and audit checklists | --- ## 📖 Reference Architecture -Detailed technical reference for every module in the system. - -### System Overview - -- [Architecture Overview](./reference/architecture/README.md) — Full system map and component index +Detailed technical documentation for every module in the codebase. ### Backend Modules -| Module | Description | -|--------|-------------| -| [Core Application](./reference/architecture/core-application/core-application.md) | Spring Boot bootstrap, `@EnableCaching`, `@EnableAsync` | -| [Controllers](./reference/architecture/controllers/controllers.md) | REST API layer — `/api/contributors`, `/api/autocomplete`, `/api/entities`, `/api/hiring` | -| [Service Layer](./reference/architecture/service-layer/service-layer.md) | GitHub integration, scoring engine, geographic filtering, cache warm-up | -| [Cache Services](./reference/architecture/cache-services/cache-services.md) | Redis + disk cache abstraction, read-only mode | -| [Configurations](./reference/architecture/configurations/configurations.md) | CORS, Redis, scheduling, async thread pool | -| [Rate Management](./reference/architecture/rate-management/rate-management.md) | Multi-token GitHub API rate limiting and token rotation | -| [GraphQL Components](./reference/architecture/graphql-components/graphql-components.md) | GitHub GraphQL query builder (fluent DSL) | -| [Model Entities](./reference/architecture/model-entities/model-entities.md) | Domain models: `Contributor`, `City`, `Region`, `State`, `SoccerTeam`, `Language` | - -### Backend Infrastructure Modules - -| Module | Description | -|--------|-------------| -| [Module 1](./reference/architecture/module_1/module_1.md) | Application bootstrap + cache abstraction (`CacheServiceAbs`) | -| [Module 2](./reference/architecture/module_2/module_2.md) | Redis implementation + async thread pool configuration | -| [Module 2 — Configuration Layer](./reference/architecture/module_2/configuration_layer.md) | Configuration layer detail | -| [Module 2 — Cache Layer](./reference/architecture/module_2/cache_layer.md) | Cache layer detail | -| [Module 3](./reference/architecture/module_3/module_3.md) | Infrastructure config: Redis, CORS, scheduling, JSON adapters | -| [Module 4](./reference/architecture/module_4/module_4.md) | REST controllers | -| [Module 5](./reference/architecture/module_5/module_5.md) | GitHub GraphQL query builder | -| [Module 6](./reference/architecture/module_6/module_6.md) | Domain models (Part 1) | -| [Module 7](./reference/architecture/module_7/module_7.md) | Domain models (Part 2) | -| [Module 8](./reference/architecture/module_8/module_8.md) | `GithubService`, scoring engine, `GithubTokenRateManager`, `CityService` | -| [Module 9](./reference/architecture/module_9/module_9.md) | `HiringService`, `LanguageService`, `PreCacheService`, `LinkedInService` | -| [Module 10](./reference/architecture/module_10/module_10.md) | `RegionService`, `StateService`, `SoccerTeamService`, `ReferencePopulationService` | +| Document | Description | +|----------|-------------| +| [Overview](./reference/architecture/README.md) | End-to-end architecture, system overview, and design principles | +| [Application Core](./reference/architecture/application-core/application-core.md) | Spring Boot bootstrap, `@EnableCaching`, `@EnableAsync` | +| [Backend Services](./reference/architecture/backend-services/backend-services.md) | Business logic: GitHub ranking, geographic modeling, hiring integration | +| [Cache Services](./reference/architecture/cache-services/cache-services.md) | Pluggable cache abstraction (Redis, Disk, ReadOnly) | +| [Configurations](./reference/architecture/configurations/configurations.md) | Spring beans, profiles, Redis, CORS, async thread pools | +| [Controllers](./reference/architecture/controllers/controllers.md) | REST endpoints: `/api/contributors`, `/api/autocomplete`, `/api/entities`, `/api/hiring` | +| [GraphQL Components](./reference/architecture/graphql-components/graphql-components.md) | Fluent GitHub GraphQL query builder | +| [Model Entities](./reference/architecture/model-entities/model-entities.md) | Domain models: `Contributor`, `City`, `Region`, `State`, `SoccerTeam` | +| [Rate Management](./reference/architecture/rate-management/rate-management.md) | Multi-token GitHub rate limit orchestration | ### Frontend Modules -| Module | Description | -|--------|-------------| -| [Frontend Components](./reference/architecture/frontend-components/frontend-components.md) | UI components: `ContributorsTable`, `FiltersPanel`, pagination | -| [Frontend Hooks](./reference/architecture/frontend-hooks/frontend-hooks.md) | Custom React hooks | -| [Frontend Services](./reference/architecture/frontend-services/frontend-services.md) | Axios API service layer | -| [Frontend Types](./reference/architecture/frontend-types/frontend-types.md) | TypeScript type definitions (`Contributor`, `City`, `ApiResponse`) | -| [Module 11](./reference/architecture/module_11/module_11.md) | Contributors table + UI contracts | -| [Module 12](./reference/architecture/module_12/module_12.md) | Pagination and mobile/desktop views | -| [Module 13](./reference/architecture/module_13/module_13.md) | URL state + geolocation hooks | -| [Module 13 — useUrlState](./reference/architecture/module_13/use_url_state.md) | URL ↔ filter state synchronization hook | -| [Module 13 — useNearestRegion](./reference/architecture/module_13/use_nearest_region.md) | Nearest MLS region geolocation hook | -| [Module 14](./reference/architecture/module_14/module_14.md) | API integration layer (Axios + `useUrlState`) | -| [Module 15](./reference/architecture/module_15/module_15.md) | Core API TypeScript types | -| [Module 16](./reference/architecture/module_16/module_16.md) | Enhanced types and models | -| [Module 17](./reference/architecture/module_17/module_17.md) | Hiring types (`HiringManagerProfile`, `JobOpening`) | -| [Module 18](./reference/architecture/module_18/module_18.md) | SEO Webpack plugin (`SeoFilesPlugin` → `sitemap.xml`, `robots.txt`) | -| [Webpack Plugins](./reference/architecture/webpack-plugins/webpack-plugins.md) | Custom Webpack plugins: SEO and favicon generation | +| Document | Description | +|----------|-------------| +| [Frontend Components](./reference/architecture/frontend-components/frontend-components.md) | UI components: leaderboard table, autocomplete, filters, pagination | +| [Frontend Hooks](./reference/architecture/frontend-hooks/frontend-hooks.md) | `useNearestRegion` (Haversine geolocation), `useUrlState` (URL-driven filters) | +| [Frontend Services](./reference/architecture/frontend-services/frontend-services.md) | Axios-based API service layer | +| [Frontend Types](./reference/architecture/frontend-types/frontend-types.md) | TypeScript contracts mirroring backend domain models | +| [Webpack Plugins](./reference/architecture/webpack-plugins/webpack-plugins.md) | Custom build plugins: `FaviconGeneratorPlugin`, `SeoFilesPlugin` | --- -## 📊 Diagrams +## 🗺️ Architecture Diagrams -Architecture diagrams are available as Mermaid (`.mmd`) files in the `docs/diagrams/architecture/` directory. +Visual Mermaid diagrams are available in the `docs/diagrams/architecture/` directory. Key diagrams include: -- `docs/diagrams/architecture/README.mmd` — System overview -- `docs/diagrams/architecture/service-layer.mmd` — Service layer dependency graph -- `docs/diagrams/architecture/core-application.mmd` — Core application startup flow -- `docs/diagrams/architecture/controllers.mmd` — Controller routing architecture -- `docs/diagrams/architecture/cache_layer.mmd` — Cache abstraction and flow -- `docs/diagrams/architecture/rate-management.mmd` — GitHub token rate management -- `docs/diagrams/architecture/frontend-components.mmd` — Frontend component hierarchy -- `docs/diagrams/architecture/use_url_state.mmd` — URL state hook data flow - -All `.mmd` files can be rendered with the [Mermaid CLI](https://github.com/mermaid-js/mermaid-cli) or viewed directly in GitHub. +- **System overview** — `docs/diagrams/architecture/README.mmd` +- **Backend services** — `docs/diagrams/architecture/backend-services.mmd` +- **Cache services** — `docs/diagrams/architecture/cache-services.mmd` +- **Rate management** — `docs/diagrams/architecture/rate-management.mmd` +- **Frontend components** — `docs/diagrams/architecture/frontend-components.mmd` +- **Frontend hooks** — `docs/diagrams/architecture/frontend-hooks.mmd` +- **Model entities** — `docs/diagrams/architecture/model-entities.mmd` +- **GraphQL components** — `docs/diagrams/architecture/graphql-components.mmd` +- **Controllers** — `docs/diagrams/architecture/controllers.mmd` +- **Configurations** — `docs/diagrams/architecture/configurations.mmd` +- **Webpack plugins** — `docs/diagrams/architecture/webpack-plugins.mmd` +- **Application core** — `docs/diagrams/architecture/application-core.mmd` +- **Frontend services** — `docs/diagrams/architecture/frontend-services.mmd` +- **Frontend types** — `docs/diagrams/architecture/frontend-types.mmd` --- @@ -130,11 +102,9 @@ All `.mmd` files can be rendered with the [Mermaid CLI](https://github.com/merma | Contributing Guide | [../CONTRIBUTING.md](../CONTRIBUTING.md) | | Live Site | [https://www.mlg.soccer](https://www.mlg.soccer) | | GitHub Repository | [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) | -| Open Issues | [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) | +| Issues | [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) | | Pull Requests | [https://github.com/flamingo-stack/major-league-github/pulls](https://github.com/flamingo-stack/major-league-github/pulls) | -| Releases | [https://github.com/flamingo-stack/major-league-github/releases](https://github.com/flamingo-stack/major-league-github/releases) | -| Security Advisories | [https://github.com/flamingo-stack/major-league-github/security/advisories](https://github.com/flamingo-stack/major-league-github/security/advisories) | --- -*Documentation generated by [🦩 Flamingo AI Technical Writer](https://flamingo.run)* +*Documentation generated by [🦩 Flamingo Code Documentation](https://flamingo.run)* diff --git a/docs/development/README.md b/docs/development/README.md index aa37916..7e5afad 100644 --- a/docs/development/README.md +++ b/docs/development/README.md @@ -1,67 +1,112 @@ # Development Documentation -Welcome to the Major League GitHub development guide. This section covers everything you need to contribute to or extend the project — from setting up your local environment to understanding the architecture and contributing code. +Welcome to the Major League GitHub developer documentation. This section covers everything you need to understand, run, and contribute to the project. --- -## Contents +## Overview -| Guide | Description | -|-------|-------------| -| [Environment Setup](setup/environment.md) | IDE configuration, editor extensions, and development tooling | -| [Local Development](setup/local-development.md) | Clone, build, run, and debug the full stack locally | -| [Architecture Overview](architecture/README.md) | High-level system design, data flow, and module breakdown | -| [Security](security/README.md) | Authentication patterns, secrets management, and vulnerability mitigations | -| [Testing](testing/README.md) | Test structure, running tests, and coverage guidelines | -| [Contributing Guidelines](contributing/guidelines.md) | Code style, branch naming, PR process, and review checklist | +Major League GitHub is a full-stack application consisting of: + +- A **Java 21 + Spring Boot 3.4** backend split into two microservices +- A **React 19 + TypeScript** frontend +- **Redis** for distributed caching +- **GitHub Actions** for CI/CD + +--- + +## Documentation Index + +### Setup + +| Document | Description | +|----------|-------------| +| [Environment Setup](./setup/environment.md) | IDE recommendations, tools, editor extensions | +| [Local Development](./setup/local-development.md) | Clone, run, hot reload, debug configuration | + +### Architecture + +| Document | Description | +|----------|-------------| +| [Architecture Overview](./architecture/README.md) | System diagram, core components, data flow | + +### Quality & Security + +| Document | Description | +|----------|-------------| +| [Security Guidelines](./security/README.md) | Auth patterns, secrets management, input validation | +| [Testing Overview](./testing/README.md) | Test structure, running tests, writing new tests | + +### Contribution + +| Document | Description | +|----------|-------------| +| [Contributing Guidelines](./contributing/guidelines.md) | Code style, branch naming, PR process, commit format | + +--- + +## Quick Navigation + +**Setting up for the first time?** +Start with [Environment Setup](./setup/environment.md), then follow [Local Development](./setup/local-development.md). + +**Understanding the system?** +Read the [Architecture Overview](./architecture/README.md). + +**Submitting a change?** +Review the [Contributing Guidelines](./contributing/guidelines.md) before opening a PR. + +**Thinking about security?** +Read the [Security Guidelines](./security/README.md) before working with tokens or configuration. --- -## Project Structure +## Repository Structure ```text major-league-github/ -├── backend/ # Java 21 + Spring Boot 3.4 backend +├── backend/ # Spring Boot backend (both microservices) │ └── src/main/java/cx/flamingo/analysis/ -│ ├── controller/ # REST API controllers -│ ├── service/ # Business logic services -│ ├── cache/ # Cache abstraction + implementations -│ ├── config/ # Spring configuration +│ ├── MajorLeagueGithubApplication.java +│ ├── cache/ # Cache abstraction (Redis, Disk) +│ ├── config/ # Spring configuration beans +│ ├── controller/ # REST controllers +│ ├── exception/ # Exception hierarchy │ ├── graphql/ # GitHub GraphQL query builder -│ ├── model/ # Domain models +│ ├── model/ # Domain models (Contributor, City, etc.) │ ├── rate/ # GitHub token rate management -│ └── exception/ # Exception handling -├── frontend/ # React 19 + TypeScript frontend -│ └── src/ -│ ├── components/ # UI components -│ ├── hooks/ # Custom React hooks -│ ├── services/ # API integration layer -│ ├── types/ # TypeScript type definitions -│ └── styles/ # Theme and color configuration -├── docs/ # Documentation -│ └── reference/ -│ └── architecture/ # Module-level reference docs -└── package.json # Root-level JS tooling (doc generation) +│ └── service/ # Business logic +└── frontend/ # React 19 + TypeScript frontend + ├── scripts/ # Build utility scripts + ├── src/ + │ ├── components/ # React UI components + │ ├── hooks/ # Custom React hooks + │ ├── services/ # Axios API services + │ ├── styles/ # Color mappings, themes + │ ├── types/ # TypeScript contracts + │ └── utils/ # Utility functions + ├── webpack-plugins/ # Custom Webpack plugins + └── webpack.config.js # Webpack 5 config ``` --- -## Two Backend Profiles - -The backend runs as **two separate services** from one codebase, activated by Maven profiles: +## Backend Microservices -| Service | Maven Profile | Port | Role | -|---------|--------------|------|------| -| Backend Service | `backend-service` (default) | 8450 | Serves the REST API | -| Cache Updater | `cache-updater` | 8451 | Runs scheduled cache refresh jobs | +| Service | Port | Maven Profile | Purpose | +|---------|------|---------------|---------| +| Backend Service | 8450 | `backend-service` (default) | REST API + ranking | +| Cache Updater | 8451 | `cache-updater` | Scheduled cache warming | -The active profile is set via `SPRING_PROFILES_ACTIVE` or the `-P` Maven flag. +Both services share the same codebase and pom.xml. The active profile determines which service starts. --- -## Quick Links +## Key External Dependencies -- **Repository:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) -- **Live site:** [https://www.mlg.soccer](https://www.mlg.soccer) -- **Issues:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) -- **Releases:** [https://github.com/flamingo-stack/major-league-github/releases](https://github.com/flamingo-stack/major-league-github/releases) +| System | Purpose | +|--------|---------| +| GitHub GraphQL API | Contributor data source | +| LinkedIn API | Job posting integration (optional) | +| Redis | Distributed cache (required in production) | +| Google Kubernetes Engine | Deployment target | diff --git a/docs/development/architecture/README.md b/docs/development/architecture/README.md index 5fa6db9..1cf273d 100644 --- a/docs/development/architecture/README.md +++ b/docs/development/architecture/README.md @@ -1,192 +1,233 @@ # Architecture Overview -Major League GitHub is a full-stack, distributed application organized into layered backend modules and a component-driven React frontend. This document provides a high-level map of how all the pieces fit together. +Major League GitHub is a full-stack, microservice-based system that transforms GitHub contribution data into a sports-themed leaderboard. This document provides the high-level architecture, component breakdown, data flow, and key design decisions. -> For deep-dives into individual modules, see the reference documentation in `docs/reference/architecture/`. +For detailed per-module documentation, see the [reference architecture](../../reference/architecture/README.md). --- -## System Architecture +## High-Level Architecture ```mermaid -flowchart TD - Browser["User Browser"] --> Frontend["React 19 Frontend (Webpack)"] - Frontend --> BackendService["Backend Service (Spring Boot, Port 8450)"] - BackendService --> Redis[("Redis Cache")] - BackendService --> GitHub["GitHub GraphQL API"] - BackendService --> LinkedIn["LinkedIn API"] - CacheUpdater["Cache Updater (Spring Boot, Port 8451)"] --> Redis +flowchart LR + User["User (Browser)"] --> Frontend["React 19 Frontend (Port 3000)"] + Frontend --> Backend["Backend Service (Spring Boot - Port 8450)"] + Backend --> Redis["Redis Cache"] + Backend --> GitHub["GitHub GraphQL API"] + Backend --> LinkedIn["LinkedIn API"] + CacheUpdater["Cache Updater (Port 8451)"] --> Backend CacheUpdater --> GitHub - GitHubActions["GitHub Actions CI/CD"] --> Docker["Docker Images"] - Docker --> GKE["Google Kubernetes Engine"] - GKE --> BackendService - GKE --> CacheUpdater - GKE --> Redis ``` +The system has three main runtime components: + +| Component | Port | Technology | Role | +|-----------|------|-----------|------| +| Frontend | 3000 | React 19 + TypeScript | Leaderboard UI | +| Backend Service | 8450 | Java 21 + Spring Boot 3.4 | REST API + ranking engine | +| Cache Updater | 8451 | Java 21 + Spring Boot 3.4 | Scheduled cache warming | + +Redis serves as the shared distributed cache between both backend services. + --- -## Core Components +## Backend Architecture + +The backend is a single Maven project that runs as two microservices via Spring profiles. -| Component | Technology | Role | -|-----------|-----------|------| -| **Backend Service** | Java 21 + Spring Boot 3.4 | Serves REST API on port 8450 | -| **Cache Updater** | Java 21 + Spring Boot 3.4 | Scheduled GitHub data refresh on port 8451 | -| **Redis** | Redis 7 | Distributed cache shared by both services | -| **React Frontend** | React 19 + TypeScript + Material-UI | Leaderboard UI | -| **GitHub GraphQL API** | External | Source of contributor data | -| **LinkedIn API** | External (optional) | Source of job postings | +```mermaid +flowchart TD + AppCore["Application Core"] + Controllers["REST Controllers"] + Services["Backend Services"] + CacheLayer["Cache Services"] + GraphQL["GraphQL Components"] + Rate["Rate Management"] + Models["Model Entities"] + Config["Configuration Layer"] + + AppCore --> Controllers + AppCore --> Services + AppCore --> CacheLayer + AppCore --> Config + + Controllers --> Services + Services --> GraphQL + Services --> Rate + Services --> Models + Services --> CacheLayer +``` + +### Backend Module Breakdown + +| Module | Path | Responsibility | +|--------|------|----------------| +| Application Core | `cx.flamingo.analysis` | Bootstrap, `@EnableCaching`, `@EnableAsync` | +| Controllers | `cx.flamingo.analysis.controller` | REST endpoints (`/api/contributors`, `/api/autocomplete`, etc.) | +| Backend Services | `cx.flamingo.analysis.service` | Business logic, GitHub data, scoring, hiring | +| Cache Services | `cx.flamingo.analysis.cache` | Pluggable cache (Redis, Disk, ReadOnly) | +| GraphQL Components | `cx.flamingo.analysis.graphql` | Fluent GitHub GraphQL query builder | +| Rate Management | `cx.flamingo.analysis.rate` | Multi-token GitHub rate limit orchestration | +| Model Entities | `cx.flamingo.analysis.model` | Domain models: `Contributor`, `City`, `Region`, etc. | +| Configurations | `cx.flamingo.analysis.config` | Spring beans, profiles, Redis, CORS, async pools | --- -## Backend Layer Architecture +## Frontend Architecture -The backend is organized into logical modules, each with a clearly defined responsibility: +The frontend is a React 19 + TypeScript SPA using Webpack 5 for production builds and Vite for development. ```mermaid flowchart TD - App["MajorLeagueGithubApplication"] --> Controllers["REST Controllers"] - Controllers --> ServiceLayer["Service Layer"] - ServiceLayer --> CacheAbs["CacheServiceAbs (Abstraction)"] - ServiceLayer --> GraphQL["GraphQL Query Builder"] - ServiceLayer --> RateManager["GitHub Token Rate Manager"] - CacheAbs --> Redis[("RedisCacheService")] - CacheAbs --> Disk["DiskCacheService (Dev)"] - CacheAbs --> ReadOnly["ReadOnlyCacheService"] - ServiceLayer --> Models["Domain Models (Contributor, City, Region, etc.)"] + Pages["React Pages"] --> Components["Frontend Components"] + Components --> Hooks["Custom React Hooks"] + Hooks --> Services["Frontend Services (Axios)"] + Services --> Backend["Backend REST API"] + Hooks --> Router["React Router (URL State)"] ``` -### Backend Module Map +### Frontend Module Breakdown -| Module | Contents | -|--------|---------| -| Module 1 | Application bootstrap + cache abstraction (`CacheServiceAbs`) | -| Module 2 | Redis implementation + async thread pool configuration | -| Module 3 | Infrastructure config: Redis, CORS, scheduling, JSON adapters | -| Module 4 | REST controllers (`/api/contributors`, `/api/autocomplete`, `/api/entities`, `/api/hiring`) | -| Module 5 | GitHub GraphQL query builder (fluent DSL) | -| Module 6–7 | Domain models: `Contributor`, `City`, `Region`, `State`, `SoccerTeam`, `Language` | -| Module 8 | `GithubService` (scoring engine) + `GithubTokenRateManager` + `CityService` | -| Module 9 | `HiringService`, `LanguageService`, `PreCacheService`, `LinkedInService` | -| Module 10 | `RegionService`, `StateService`, `SoccerTeamService`, `ReferencePopulationService` | +| Module | Path | Responsibility | +|--------|------|----------------| +| Components | `src/components/` | UI: filters, leaderboard table, autocomplete, tooltips | +| Hooks | `src/hooks/` | URL state management, nearest region geolocation | +| Services | `src/services/` | Axios-based API calls with typed responses | +| Types | `src/types/` | TypeScript contracts mirroring backend domain models | +| Styles | `src/styles/` | Theme configuration, color mappings | +| Webpack Plugins | `webpack-plugins/` | SEO files generator, favicon generator | --- -## GitHub Data Retrieval Flow +## Core Data Flow + +### Contributor Search Request ```mermaid sequenceDiagram - participant Client as "Frontend" - participant Controller as "ContributorController" - participant Cache as "CacheServiceAbs" - participant Service as "GithubService" - participant Rate as "GithubTokenRateManager" - participant Builder as "GitHubQueryBuilder" - participant GitHub as "GitHub GraphQL API" - - Client->>Controller: GET /api/contributors/search - Controller->>Cache: isCacheReady()? - Cache-->>Controller: true - Controller->>Cache: getHttpResponse(filters, loader) - Cache->>Service: getTopContributorsIn(cities, language) - Service->>Builder: build GraphQL query - Builder-->>Service: query string - Service->>Rate: getBestAvailableClient() - Rate-->>Service: WebClient - Service->>GitHub: POST /graphql - GitHub-->>Service: JSON response - Service-->>Cache: List - Cache-->>Controller: Cached response - Controller-->>Client: ApiResponse> + participant Browser + participant ReactApp as React App + participant Hook as useUrlState Hook + participant Service as Frontend Service + participant Controller as ContributorController + participant GithubSvc as GithubService + participant Cache as CacheServiceAbs + participant GitHubAPI as GitHub GraphQL API + + Browser->>ReactApp: User selects language/location filter + ReactApp->>Hook: Update URL state (cityId, languageId, etc.) + Hook->>Service: getContributors(params) + Service->>Controller: GET /api/contributors/search + Controller->>Cache: getHttpResponse() + Cache-->>Controller: Cache Hit (return cached list) + Controller-->>Service: ApiResponse with Contributor[] + Service-->>ReactApp: Contributor[] + ReactApp-->>Browser: Render leaderboard + + Note over Cache,GitHubAPI: On cache miss: + Cache->>GithubSvc: Execute supplier + GithubSvc->>GitHubAPI: GraphQL query (location + language) + GitHubAPI-->>GithubSvc: User data + GithubSvc->>GithubSvc: Score contributors + GithubSvc-->>Cache: Store result ``` --- ## Contributor Scoring Formula -The scoring engine in `GithubService` ranks developers using: +The ranking engine scores each contributor as: ```text -Score = commits × max(starsReceived, 1) × recencyMultiplier +score = commits × max(starsReceived, 1) × recencyMultiplier ``` -| Component | Source | Effect | -|-----------|--------|--------| -| `commits` | GitHub contributions calendar | Rewards high activity volume | -| `starsReceived` | Stars on language-specific repos | Rewards community impact | -| `recencyMultiplier` | Activity freshness (1.0–2.0) | Rewards recent contributions | +- **commits** — total commits across repositories +- **starsReceived** — total stars received (floored at 1 to prevent zero score) +- **recencyMultiplier** — `1.0` to `2.0`, based on activity in the past year --- -## Caching Strategy +## Cache Architecture -Major League GitHub uses a **cache-first** architecture to minimize GitHub API rate pressure and reduce response latency: +The cache layer uses a pluggable abstraction supporting three implementations: ```mermaid -flowchart TD - Request["Incoming Request"] --> CacheCheck["CacheServiceAbs.get()"] - CacheCheck --> Exists{"Entry Exists?"} - Exists -->|"No"| Fetch["Fetch From GitHub"] - Exists -->|"Yes"| Stale{"Is Stale?"} - Stale -->|"No"| Return["Return Cached Data"] - Stale -->|"Yes"| AsyncRefresh["Async Background Refresh"] - Fetch --> Store["Store In Cache"] - Store --> Return +flowchart LR + Services["Backend Services"] --> Abstract["CacheServiceAbs (Abstract)"] + Abstract --> Redis["RedisCacheService"] + Abstract --> Disk["DiskCacheService"] + Redis --> ReadOnly["ReadOnlyCacheService"] + + Redis --> RedisDB[("Redis")] + Disk --> FS[("File System")] ``` -| Cache Mode | Use Case | -|-----------|---------| -| `read-write` (default) | Normal production operation | -| `read-only` | Prevent writes during maintenance | -| Disk cache | Local development without Redis | +| Mode | Description | +|------|-------------| +| `read-write` | Normal operation — read from cache, write on miss | +| `read-only` | Safe mode — read from Redis, never write | +| `force-update` | Always bypass cache and fetch fresh | -Cache keys encode: city, language, and page number. Empty results are also cached to prevent repeated expensive API calls. +The active implementation is selected at startup via `cache.implementation` and `cache.mode` properties. --- -## Frontend Architecture +## Rate Management + +GitHub API rate limits are managed via `GithubTokenRateManager`: ```mermaid flowchart TD - BrowserRouter["BrowserRouter (react-router-dom)"] --> useUrlState["useUrlState Hook"] - useUrlState --> APIService["API Service (Axios)"] - APIService --> Backend["Backend REST API"] - APIService --> Types["TypeScript API Types"] - Types --> EnhancedTypes["Enhanced Models"] - EnhancedTypes --> ContributorsTable["ContributorsTable Component"] - ContributorsTable --> FiltersPanel["FiltersPanel"] - ContributorsTable --> Pagination["Pagination"] + Request["Outbound GitHub Request"] --> Evaluate["Evaluate All Tokens"] + Evaluate --> SecondaryCheck{"All Under Secondary Limit?"} + SecondaryCheck -->|"Yes"| WaitSecondary["Sleep Until Earliest Retry"] + SecondaryCheck -->|"No"| PrimaryCheck{"All Exhausted?"} + PrimaryCheck -->|"Yes"| WaitPrimary["Sleep Until Earliest Reset"] + PrimaryCheck -->|"No"| Select["Select Token with Highest Remaining Requests"] + WaitSecondary --> Select + WaitPrimary --> Select + Select --> Execute["Execute GitHub GraphQL API Call"] ``` -The frontend is driven by **URL state**. All filter parameters (language, city, state, region, team) are stored as URL query parameters via the `useUrlState` hook. This makes every leaderboard view fully shareable and bookmarkable. +Multiple tokens can be configured via `github.tokens`. The manager tracks: +- **Primary rate limits** (per-hour quota) +- **Secondary rate limits** (burst/abuse protection via `Retry-After` headers) + +--- + +## Geographic Modeling -### Key Frontend Modules +Contributors are filtered using a multi-level geographic model: -| Module | Contents | -|--------|---------| -| Module 11–12 | Contributors table, pagination, mobile/desktop views | -| Module 13 | `useUrlState` — URL ↔ filter state synchronization | -| Module 14 | API service layer (Axios) + `useUrlState` basic hook | -| Module 15 | Core API TypeScript types (`Contributor`, `City`, `ApiResponse`) | -| Module 16–17 | Enhanced types, hiring types (`HiringManagerProfile`, `JobOpening`) | -| Module 18 | SEO Webpack plugin (`SeoFilesPlugin` → `sitemap.xml`, `robots.txt`) | +```mermaid +flowchart TD + Region["Region (multi-state MLS area)"] --> State["State"] + State --> City["City"] + City --> SoccerTeam["Nearest MLS Team (Haversine distance)"] +``` + +Data is loaded from static CSV files at startup (`cities.csv`, `states.csv`, `regions.csv`, `teams.csv`). No database is required. + +The Haversine formula (Earth radius = 6371 km) is used to compute the nearest MLS stadium for each city. --- -## Deployment Architecture +## Deployment Model ```mermaid flowchart LR - GitHubActions["GitHub Actions CI/CD"] --> Docker["Docker Images"] - Docker --> GKE["Google Kubernetes Engine (GKE)"] - GKE --> BackendPod["Backend Pod (8450)"] - GKE --> CacheUpdaterPod["Cache Updater Pod (8451)"] + GitHubRepo["GitHub Repository"] --> CI["GitHub Actions CI/CD"] + CI --> Docker["Docker Images"] + Docker --> GKE["Google Kubernetes Engine"] + GKE --> BackendPods["Backend + Cache Updater Pods"] GKE --> RedisPod["Redis Pod"] - BackendPod --> RedisPod - CacheUpdaterPod --> RedisPod + GKE --> FrontendService["Frontend Service"] ``` -Both backend services are built from the same JAR but run in separate Kubernetes pods with different Spring profiles. CI/CD is managed through GitHub Actions, which builds Docker images and deploys to Google Kubernetes Engine (GKE). +- **Containerized:** All services run as Docker containers +- **Orchestrated:** Kubernetes (GKE) manages scaling and health +- **CI/CD:** GitHub Actions builds, tests, and deploys on push --- @@ -194,10 +235,10 @@ Both backend services are built from the same JAR but run in separate Kubernetes | Decision | Rationale | |----------|-----------| -| **Two Spring profiles, one JAR** | Simplifies build and deployment while enabling distinct runtime behaviors | -| **Cache-first with async refresh** | Prevents latency spikes from synchronous GitHub API calls | -| **Multi-token rate management** | Resilient throughput under GitHub's strict per-token rate limits | -| **CSV-based reference data** | Cities, states, regions, and teams load from CSVs at startup — no database required | -| **URL-driven frontend state** | Every filter combination is bookmarkable and shareable | -| **Haversine distance for MLS proximity** | Accurately calculates geographic distance to nearest stadium | -| **Custom Webpack plugins** | SEO and favicon assets generated at build time — zero runtime overhead | +| Two microservices from one codebase | Shared code, separated concerns; profile switching via Maven | +| Redis as distributed cache | Prevents redundant GitHub API calls across pods | +| Multi-token rate management | Maximizes GitHub API throughput without hitting limits | +| URL-driven frontend state | Filter state is shareable and bookmarkable without backend session | +| CSV data files (no database) | Eliminates database dependency for geographic reference data | +| Pluggable cache abstraction | Swap Redis ↔ Disk without changing business logic | +| Haversine proximity for teams | Accurate great-circle distance for stadium assignment | diff --git a/docs/development/security/README.md b/docs/development/security/README.md index 3ce9c68..ab49905 100644 --- a/docs/development/security/README.md +++ b/docs/development/security/README.md @@ -1,214 +1,210 @@ -# Security Best Practices +# Security Guidelines -This document describes the security patterns used in Major League GitHub and provides guidelines for keeping the application and its data safe during development and deployment. +This document covers security best practices for developing, configuring, and deploying Major League GitHub. Following these guidelines protects GitHub API credentials, user data, and the application's integrity. --- ## Authentication and Authorization -Major League GitHub does not implement user authentication for the public leaderboard — the API is intentionally read-only and publicly accessible. However, several integration points require credential management: - ### GitHub API Tokens -The backend authenticates to the GitHub GraphQL API using Personal Access Tokens (PATs). These are managed by the `GithubTokenRateManager` service. +Major League GitHub uses GitHub Personal Access Tokens (PATs) to authenticate with the GitHub GraphQL API. These tokens are the most security-sensitive credentials in the system. -**Token handling rules:** -- Tokens are read from the `GITHUB_TOKENS` environment variable at startup -- Each token is stored in memory as a `GithubToken` state object -- Tokens are never logged, exposed via API responses, or written to cache -- Multiple tokens are supported (comma-separated) for throughput resilience +**Required scopes (minimum):** +- `read:user` — read public user data +- `public_repo` — access public repository data -```text -GITHUB_TOKENS=ghp_token1,ghp_token2,ghp_token3 -``` +**Do not grant:** +- `repo` (private repository access — not needed) +- `write:*` (any write permission — not needed) +- `admin:*` (any admin permission — not needed) +- `delete_repo` or `gist` — not needed -**Required scopes** (minimum): -- `read:user` — to access contributor profile data -- `repo` — to access star counts on repositories +### Token Storage Rules -> **Principle of least privilege:** Only request the scopes your tokens actually need. Avoid `write:*` scopes entirely. +| Environment | Storage Method | +|-------------|----------------| +| Local development | Shell environment variable (`export GITHUB_TOKENS=...`) | +| CI/CD (GitHub Actions) | GitHub Secrets (never hardcoded in YAML) | +| Kubernetes (production) | Kubernetes Secrets mounted as environment variables | -### LinkedIn API (Optional) +**Never:** +- Commit tokens to source control +- Log token values (the `GithubTokenRateManager` logs token metadata, not raw token strings) +- Store tokens in `application.properties` committed to Git +- Embed tokens in Docker images -LinkedIn OAuth2 client credentials are used only if the hiring section is enabled. If not configured, the system falls back to static job entries — no error is thrown. +### LinkedIn API Credentials (Optional) -Credentials are stored exclusively in environment variables: +The LinkedIn integration uses OAuth 2.0 client credentials flow. If configured: -```text -LINKEDIN_CLIENT_ID=... -LINKEDIN_CLIENT_SECRET=... -LINKEDIN_ORGANIZATION_ID=... -``` +- Store `LINKEDIN_CLIENT_ID` and `LINKEDIN_CLIENT_SECRET` as Kubernetes Secrets or GitHub Actions Secrets +- Never hardcode credentials in source code or configuration files --- ## Secrets Management -### Development +### Local Development -In local development, set secrets as shell environment variables or in a `.env.local` file that is **never committed to Git**: +Use shell environment variables, never application config files: ```bash -# Add to .gitignore -echo ".env.local" >> .gitignore - -# Create local env file -cat > backend/.env.local << 'EOF' -GITHUB_TOKENS=ghp_yourDevToken -SPRING_REDIS_HOST=localhost -SPRING_REDIS_PORT=6379 -EOF +export GITHUB_TOKENS="ghp_your_token_here" +export LINKEDIN_CLIENT_ID="your_client_id" +export LINKEDIN_CLIENT_SECRET="your_client_secret" ``` -### Production (Kubernetes) - -In production on GKE, secrets should be stored as Kubernetes Secrets and injected as environment variables into pod containers — never hardcoded in Docker images, Kubernetes manifests committed to the repository, or application properties files. +Add sensitive variable names to `.gitignore` if you use `.env` files: -```bash -kubectl create secret generic mlg-secrets \ - --from-literal=GITHUB_TOKENS="ghp_token1,ghp_token2" \ - --from-literal=LINKEDIN_CLIENT_SECRET="..." +```text +.env +.env.local +.env.production ``` -Reference secrets in pod specs: +### GitHub Actions CI/CD + +Store secrets in **GitHub Repository Settings → Secrets and Variables → Actions**: + +- `GITHUB_TOKENS` +- `LINKEDIN_CLIENT_ID` +- `LINKEDIN_CLIENT_SECRET` + +Reference them in workflow YAML: ```yaml env: - - name: GITHUB_TOKENS - valueFrom: - secretKeyRef: - name: mlg-secrets - key: GITHUB_TOKENS + GITHUB_TOKENS: ${{ secrets.GITHUB_TOKENS }} ``` -### CI/CD (GitHub Actions) +**Do not** echo secrets in workflow steps or store them in workflow artifacts. -Store sensitive values as **GitHub Actions Secrets** in the repository settings. Never interpolate secrets directly into workflow YAML files — use the `secrets` context: +### Kubernetes (Production) -```yaml -- name: Deploy - env: - GITHUB_TOKENS: ${{ secrets.GITHUB_TOKENS }} +Create Kubernetes Secrets for sensitive values: + +```bash +kubectl create secret generic mlg-secrets \ + --from-literal=GITHUB_TOKENS="ghp_token1,ghp_token2" \ + --from-literal=LINKEDIN_CLIENT_SECRET="your_secret" ``` +Reference them in pod specs as environment variables. Avoid base64-encoding secrets manually and storing them in version-controlled YAML files. + +--- + +## Data Encryption + +### Data at Rest + +- **Redis:** All data cached in Redis is GitHub contributor data (public information). Redis should still be placed on a private network and not exposed publicly. +- **Disk cache:** JSON files on the filesystem contain public GitHub data. Ensure appropriate file system permissions. +- No personally identifiable information (PII) beyond public GitHub profiles is stored. + +### Data in Transit + +- All production traffic is served over **HTTPS** at `https://www.mlg.soccer` +- Backend-to-Redis communication should use a private network (within the Kubernetes cluster) +- Frontend-to-backend communication is over HTTPS in production + --- ## Input Validation and Sanitization -### Backend (Spring Boot) +### Backend -All request parameters accepted by the REST controllers are primitives or strings with well-defined domains: +The backend validates request parameters in controllers. Key practices: -- `cityId`, `regionId`, `stateId`, `teamId`, `languageId` — filtered against in-memory reference data (CSV-loaded IDs); if an ID doesn't match a known entity, a warning is logged and the request proceeds with default values -- `maxResults` — bounded by an integer with a default value; no upper limit is enforced in code, but pagination constraints limit result size -- No request body deserialization occurs on public endpoints; all input arrives as URL query parameters +**ID parameters** — Only alphanumeric IDs are accepted. The `useUrlState` frontend hook validates URL parameters with the regex `^[a-zA-Z0-9-]+$` before sending them to the API. -### Frontend (TypeScript) +**Query strings** — Autocomplete queries are passed to GitHub's search API after the backend assembles safe GraphQL queries using the `GitHubQueryBuilder`. The builder uses structured arguments rather than string interpolation, preventing injection. -URL parameter validation is enforced by the `useUrlState` hook before values are passed to the API service: +**No SQL** — The application does not use a SQL database. Geographic data comes from static CSV files loaded at startup, so SQL injection is not applicable. -```typescript -// Only alphanumeric characters and dashes are accepted -validate: (value: string) => /^[a-zA-Z0-9-]+$/.test(value) -``` +**Rate limit exceptions** — `GithubRateLimitException`, `GithubTimeoutException`, `GithubGeneralException`, and `GithubTooFastException` are handled by `GlobalExceptionHandler`, which returns structured `ApiError` responses without exposing internal stack traces. + +### Frontend -Parameters that fail validation are silently dropped and replaced with `null` (the default), preventing malformed values from reaching the backend. +**URL parameter validation** — The `useUrlState` hook validates all URL-driven filter values against `^[a-zA-Z0-9-]+$` before use. Invalid values fall back to defaults. + +**No user-generated content rendered as HTML** — All contributor data (names, locations) is rendered through React's JSX, which escapes HTML by default. --- ## CORS Configuration -CORS is configured in `WebConfig` and restricts API access to known origins: - -| Allowed Origin | Purpose | -|----------------|---------| -| `http://localhost:8450` | Local development | -| `http://localhost:3000` | Alternative local dev port | -| `https://www.mlg.soccer` | Production frontend | -| `http://www.mlg.soccer` | Production frontend (HTTP redirect) | +The backend's CORS policy (configured in `WebConfig`) restricts cross-origin requests to known origins: -Allowed HTTP methods: `GET`, `POST`, `PUT`, `DELETE`, `OPTIONS`. +```text +http://localhost:8450 +http://localhost:3000 +https://www.mlg.soccer +http://www.mlg.soccer +``` -> When adding a new deployment environment (e.g., a staging domain), update `WebConfig` to include the staging origin. +When deploying to a new environment, add its origin to the CORS allowlist rather than using wildcard (`*`) origins. Avoid `allowedOrigins("*")` in any environment that uses credentials. --- -## API Surface Security +## Common Security Vulnerabilities and Mitigations -The REST API is **read-only by design** — no authenticated write endpoints exist on the public-facing Backend Service: +| Vulnerability | Mitigation | +|---------------|-----------| +| **Token leakage** | Environment variables only; never commit to Git | +| **Rate limit exhaustion** | `GithubTokenRateManager` handles primary + secondary limits | +| **Denial of service via cache miss flood** | Redis cache with read-only mode and async refresh | +| **Injection via URL parameters** | Regex validation in `useUrlState`; structured GraphQL builder | +| **Exposed internal errors** | `GlobalExceptionHandler` returns `ApiError` without stack traces | +| **CORS bypass** | Explicit origin allowlist in `WebConfig` | +| **Sensitive data in logs** | Token values are not logged; rate metadata only | -- `GET /api/contributors/search` — read only -- `GET /api/contributors/export` — read only (triggers CSV download) -- `GET /api/autocomplete/*` — read only -- `GET /api/entities/*` — read only -- `GET /api/hiring/*` — read only +--- -The Cache Updater service (port 8451) does not expose a public HTTP API. Its scheduled jobs run internally and communicate only with Redis. +## Security Testing and Code Review Guidelines -**The `/api/` path is disallowed for search engine indexing** via the generated `robots.txt`: +### Before Committing -```text -Disallow: /api/ -``` +- [ ] No secrets, tokens, or passwords committed +- [ ] No wildcard CORS (`allowedOrigins("*")`) added +- [ ] No raw user input passed to external APIs without validation +- [ ] Exception handlers return clean `ApiError` responses, not raw stack traces ---- +### For Code Reviews -## Dependency Security +- Check that new environment variables are documented and not hardcoded +- Verify that any new endpoints validate their inputs +- Confirm that new external API integrations handle rate limits and timeouts +- Ensure new configuration properties have safe defaults -### Backend +### Dependency Security -- Use `mvn dependency:analyze` periodically to identify unused or missing dependencies -- Review the [GitHub Dependabot alerts](https://github.com/flamingo-stack/major-league-github/security/dependabot) for known CVEs in Maven dependencies -- The `spring-boot-starter-parent` version (`3.4.1`) manages most transitive dependency versions — keep the parent version up to date +Regularly audit dependencies for known CVEs: + +**Backend (Maven):** ```bash -# Check for dependency updates cd backend -mvn versions:display-dependency-updates +mvn dependency:resolve ``` -### Frontend +Consider using [OWASP Dependency-Check Maven Plugin](https://owasp.org/www-project-dependency-check/) for automated CVE scanning. -- Run `npm audit` regularly to scan for known vulnerabilities in npm packages -- Address `npm audit fix` suggestions promptly for high/critical severity issues +**Frontend (npm):** ```bash cd frontend npm audit -npm audit fix ``` ---- - -## Sensitive Data in Logs - -Spring Boot uses SLF4J + Logback. By default, the project logs: -- Incoming request parameters (city, language, state filters) — these are safe public values -- GitHub API rate limit status — safe to log -- Cache readiness state — safe to log - -**Never log:** -- `GITHUB_TOKENS` values -- LinkedIn client secrets -- Full GitHub API responses (they may contain private email addresses) - -If adding new log statements, apply this rule: log only IDs and counts, never raw token values or personal data. +Fix moderate and high-severity findings before merging. --- -## Security Testing Guidelines - -Before submitting a PR that touches authentication, configuration, or API layers, verify: - -- [ ] No secrets or tokens appear in code, config files, or test fixtures -- [ ] New environment variables are documented and have empty defaults -- [ ] CORS origins list is not expanded unnecessarily -- [ ] Any new URL parameters pass through the `useUrlState` validation pattern -- [ ] Rate limiting behavior is preserved (do not bypass `GithubTokenRateManager`) -- [ ] No new external API endpoints are called without error handling and fallback behavior +## Reporting Security Issues ---- +If you discover a security vulnerability in Major League GitHub, please report it via GitHub Issues: -## Reporting Security Issues +[https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) -If you discover a security vulnerability, please report it responsibly via [GitHub Security Advisories](https://github.com/flamingo-stack/major-league-github/security/advisories) rather than opening a public issue. +For sensitive disclosures, open a private security advisory via GitHub's **Security → Advisories** feature in the repository. diff --git a/docs/development/setup/environment.md b/docs/development/setup/environment.md index 7e93b0b..e77147c 100644 --- a/docs/development/setup/environment.md +++ b/docs/development/setup/environment.md @@ -1,194 +1,223 @@ # Development Environment Setup -This guide walks you through configuring your IDE, installing required tools, and setting up editor extensions for an optimal Major League GitHub development experience. +This guide covers IDE recommendations, required development tools, and editor extensions for working on Major League GitHub. --- -## Recommended IDE +## IDE Recommendations -### IntelliJ IDEA (Backend) +### Backend (Java / Spring Boot) -IntelliJ IDEA Community or Ultimate is the recommended IDE for Java backend development. +**IntelliJ IDEA** (recommended) -**Required plugins:** -- **Lombok** — enables annotation processing for `@Data`, `@Builder`, `@RequiredArgsConstructor`, etc. -- **Spring** — provides Spring Boot run configurations and bean navigation +IntelliJ IDEA provides the best Java 21 + Spring Boot 3.4 development experience: -**Setup steps:** +- Built-in Spring Boot run configurations +- Lombok annotation processing (required for model classes) +- Full Maven integration +- Integrated Redis monitoring via Database Tools -1. Open IntelliJ IDEA and choose **Open → select the `backend/` directory** (or the repo root) -2. IntelliJ auto-detects the Maven project from `backend/pom.xml` -3. Go to **Settings → Build, Execution, Deployment → Compiler → Annotation Processors** and enable annotation processing -4. Set the Project SDK to **Java 21** +**VS Code** (alternative) -```bash -# Verify your active JDK -java -version -# Requires: openjdk 21.x.x -``` +Use with the following extensions: +- **Extension Pack for Java** (Microsoft) +- **Spring Boot Extension Pack** (VMware) +- **Lombok Annotations Support for VS Code** -### VS Code (Frontend) +### Frontend (React / TypeScript) -VS Code is recommended for the React + TypeScript frontend. +**VS Code** (recommended) -**Required extensions:** -- **ESLint** (`dbaeumer.vscode-eslint`) — uses the project's `eslint.config.js` -- **Prettier** (`esbenp.prettier-vscode`) — code formatting -- **TypeScript and JavaScript Language Features** — built-in, provides IntelliSense -- **GitLens** (`eamodio.gitlens`) — enhanced Git tooling +The frontend is built with React 19 + TypeScript. Use VS Code with: -**Optional extensions:** -- **vscode-styled-components** — for MUI `sx` prop syntax highlighting -- **Error Lens** — inline error display -- **Import Cost** — shows bundle size impact of imports +- **ESLint** — enforces the project's ESLint config (`eslint.config.js`) +- **Prettier** — code formatting (configure to match project style) +- **TypeScript + JavaScript** (built-in) +- **ES7+ React/Redux/React-Native snippets** -**VS Code workspace settings** (create `.vscode/settings.json` in the repo root): +**IntelliJ IDEA / WebStorm** (alternative) -```json -{ - "editor.formatOnSave": true, - "editor.defaultFormatter": "esbenp.prettier-vscode", - "typescript.tsdk": "frontend/node_modules/typescript/lib", - "eslint.workingDirectories": ["frontend"] -} -``` +WebStorm provides excellent TypeScript and React support out of the box. --- ## Required Development Tools -| Tool | Install Command | Notes | -|------|----------------|-------| -| JDK 21 | [adoptium.net](https://adoptium.net) or `brew install temurin@21` | Set `JAVA_HOME` | -| Maven 3.9+ | `brew install maven` or [maven.apache.org](https://maven.apache.org) | | -| Node.js 18+ | [nodejs.org](https://nodejs.org) or `brew install node@18` | | -| npm 9+ | Bundled with Node.js | | -| Redis 7+ | `brew install redis` or Docker | | -| Docker | [docs.docker.com](https://docs.docker.com/get-docker/) | Optional but recommended | +### Java Toolchain ---- +Install Java 21. The recommended distributions: -## Environment Variables for Development +- **Temurin (Eclipse Adoptium)** — free, production-grade +- **Oracle JDK 21** — official release +- **Amazon Corretto 21** — AWS-maintained distribution -Create a file at `backend/.env.local` (or export in your shell profile) for backend development: +Verify after installation: ```bash -# Backend Service -export GITHUB_TOKENS="ghp_your_token_here" -export SPRING_REDIS_HOST="localhost" -export SPRING_REDIS_PORT="6379" -export SPRING_PROFILES_ACTIVE="backend-service" +java -version ``` -For the frontend dev server, pass variables inline or export them: +Expected output should show `openjdk version "21"` (or similar). + +### Maven + +Maven 3.9+ is required for building the backend. ```bash -# Frontend dev server -export BACKEND_API_URL="http://localhost:8450" -export NODE_ENV="development" -export PORT="8450" +mvn -version ``` -> **Never commit tokens or secrets.** The `GITHUB_TOKENS` variable is sensitive. Add `.env.local` to `.gitignore` and use environment-specific secret management in CI/CD. +> **Tip:** IntelliJ IDEA bundles a Maven version. You can use the bundled Maven for IDE-only builds, but install Maven system-wide for terminal builds. ---- +### Node.js and npm -## Path Aliases (Frontend) +Install Node.js 18 or later (LTS recommended). npm is bundled with Node.js. -The Webpack config defines two path aliases for cleaner imports: +```bash +node --version +npm --version +``` -| Alias | Resolves To | Example | -|-------|------------|---------| -| `@/` | `frontend/src/` | `import { Layout } from '@/components/Layout'` | -| `@flamingo/ui-kit` | `ui-kit/src/` | `import { Button } from '@flamingo/ui-kit'` | +Use [nvm](https://github.com/nvm-sh/nvm) to manage Node.js versions if you work on multiple projects. -These are configured in `frontend/webpack.config.js` and are available across all TypeScript source files in the frontend. +### Redis ---- +Redis 6 or later is required for the backend cache layer. -## Linting and Formatting +**macOS (Homebrew):** -The frontend uses ESLint configured via `frontend/eslint.config.js`. +```bash +brew install redis +brew services start redis +``` -Run the linter: +**Linux (apt):** ```bash -cd frontend -npx eslint src/ +sudo apt-get update +sudo apt-get install redis-server +sudo systemctl start redis +``` + +**Windows:** + +Use WSL2 with a Linux distribution and install Redis inside it. + +Verify: + +```bash +redis-cli ping ``` -The linter enforces: -- TypeScript type-safety rules -- React hooks exhaustive dependencies -- Import ordering +Expected: `PONG` --- -## Netty DNS (macOS Apple Silicon) +## Lombok Setup -The backend `pom.xml` includes a macOS-specific DNS resolver: +The backend uses **Lombok** for boilerplate reduction (`@Data`, `@Builder`, `@Slf4j`, etc.). Annotation processing must be enabled in your IDE. -```xml - - io.netty - netty-resolver-dns-native-macos - osx-aarch_64 - -``` +### IntelliJ IDEA + +1. Go to **Settings → Build, Execution, Deployment → Compiler → Annotation Processors** +2. Check **Enable annotation processing** +3. Install the **Lombok Plugin** from the marketplace if prompted + +### VS Code -This resolves a known Netty warning when running Spring WebFlux on Apple Silicon Macs. No action is required — it is already wired in. +Install the **Lombok Annotations Support for VS Code** extension. --- -## Redis in Development +## Environment Variables for Development -For local development, run Redis via Docker (no persistent volume needed): +Set these in your shell profile (`~/.bashrc`, `~/.zshrc`, or equivalent) or in your IDE run configuration: ```bash -docker run -d \ - --name mlg-redis \ - -p 6379:6379 \ - redis:7 -``` +# Required — at least one GitHub Personal Access Token +export GITHUB_TOKENS="ghp_your_token_here" -To stop and remove: +# Optional — use disk cache instead of Redis for simpler local dev +export CACHE_IMPLEMENTATION="disk" -```bash -docker stop mlg-redis && docker rm mlg-redis +# Optional — Redis connection (defaults to localhost:6379) +export SPRING_REDIS_HOST="localhost" +export SPRING_REDIS_PORT="6379" + +# Optional — Frontend API URL (for running frontend separately) +export BACKEND_API_URL="http://localhost:8450" ``` -Alternatively, if you installed Redis via Homebrew: +--- -```bash -brew services start redis -# Stop with: -brew services stop redis -``` +## IntelliJ IDEA Run Configuration (Backend) + +To run the backend from IntelliJ IDEA: + +1. Open the `backend/` directory as a Maven project +2. Find `MajorLeagueGithubApplication.java` +3. Right-click → **Run** +4. In the run configuration, add environment variables: + - `GITHUB_TOKENS` = your token(s) + - `CACHE_IMPLEMENTATION` = `disk` (optional, to skip Redis during dev) +5. The active profile defaults to `backend-service` (port 8450) --- -## Verify Your Setup +## Frontend Build System -Run this checklist to confirm everything is in place: +The frontend uses **Webpack 5** as its primary bundler with a Vite config also available: -```bash -# Java -java -version && javac -version +| Config | Use Case | +|--------|----------| +| `webpack.config.js` | Production builds, custom plugins (SEO, favicon) | +| `vite.config.js` | Development server with hot module replacement | -# Maven -mvn -version +The project includes two custom Webpack plugins: -# Node + npm -node -v && npm -v +- **FaviconGeneratorPlugin** — generates favicon assets at build time +- **SeoFilesPlugin** — generates sitemap and robots.txt at build time -# Redis -redis-cli ping +These run automatically during `npm run build`. -# Compile backend (no tests) -cd backend && mvn compile -DskipTests +--- + +## ESLint Configuration + +The project uses ESLint 9 with TypeScript support. The configuration is defined in `frontend/eslint.config.js`. + +Key rules enabled: +- `eslint-plugin-react-hooks` — enforces React Hooks rules +- `eslint-plugin-react-refresh` — warns about components incompatible with hot reload +- `typescript-eslint` — TypeScript-specific linting -# Install frontend deps -cd ../frontend && npm install && echo "Frontend deps OK" +Run lint checks: + +```bash +cd frontend +npx eslint src/ ``` -All commands should return without errors before you begin development. +--- + +## Recommended .editorconfig Settings + +If you create a `.editorconfig` at the repository root, use these settings to match the project's code style: + +```text +root = true + +[*] +indent_style = space +indent_size = 4 +end_of_line = lf +charset = utf-8 +trim_trailing_whitespace = true +insert_final_newline = true + +[*.{ts,tsx,js,jsx}] +indent_size = 2 + +[*.{yml,yaml}] +indent_size = 2 +``` diff --git a/docs/development/setup/local-development.md b/docs/development/setup/local-development.md index 083214d..40cd3d2 100644 --- a/docs/development/setup/local-development.md +++ b/docs/development/setup/local-development.md @@ -1,240 +1,249 @@ # Local Development Guide -This guide explains how to run the full Major League GitHub stack locally, including hot reload, debugging, and working with the two backend profiles. +This guide walks through cloning the project, running both the backend and frontend locally, configuring hot reload, and debugging. --- -## 1. Clone and Set Up +## Clone and Initial Setup ```bash git clone https://github.com/flamingo-stack/major-league-github.git cd major-league-github ``` -The project structure contains two main directories: +The repository contains two independent sub-projects: -```text -major-league-github/ -├── backend/ # Java 21 + Spring Boot 3.4 -└── frontend/ # React 19 + TypeScript (Webpack) -``` +- `backend/` — Java 21 + Spring Boot 3.4 (Maven project) +- `frontend/` — React 19 + TypeScript (npm project) --- -## 2. Start Redis +## Running the Backend Locally -Both backend services require Redis. Start it with Docker: +### 1. Ensure Redis Is Running ```bash -docker run -d \ - --name mlg-redis \ - -p 6379:6379 \ - redis:7 +redis-server ``` -Verify it is accepting connections: +Or with Homebrew on macOS: ```bash -redis-cli -h localhost -p 6379 ping -# Expected: PONG +brew services start redis ``` ---- - -## 3. Running the Backend Service - -The Backend Service exposes the REST API on port 8450. It is activated by the `backend-service` Maven profile (which is the default profile in the `pom.xml`). +### 2. Start the Backend Service ```bash cd backend -mvn spring-boot:run \ - -Pbackend-service \ - -Dspring-boot.run.jvmArguments="\ - -DGITHUB_TOKENS=ghp_yourTokenHere \ - -DSPRING_REDIS_HOST=localhost \ - -DSPRING_REDIS_PORT=6379" +GITHUB_TOKENS=ghp_your_token_here mvn spring-boot:run ``` -Or with exported environment variables: +The backend starts on **port 8450** using the `backend-service` profile (active by default). -```bash -export GITHUB_TOKENS="ghp_yourTokenHere" -export SPRING_REDIS_HOST="localhost" -export SPRING_REDIS_PORT="6379" +**With disk cache (skip Redis requirement):** +```bash cd backend -mvn spring-boot:run -Pbackend-service +GITHUB_TOKENS=ghp_your_token_here \ +CACHE_IMPLEMENTATION=disk \ +mvn spring-boot:run ``` -**Startup indicator:** - -```text -Started MajorLeagueGithubApplication in X.XXX seconds (JVM running for Y.YYY) -``` - -**Health check:** +### 3. Verify the Backend ```bash curl http://localhost:8450/actuator/health -# Expected: {"status":"UP"} ``` ---- +Expected: -## 4. Running the Cache Updater +```json +{"status":"UP"} +``` -The Cache Updater runs scheduled jobs that pre-warm Redis with GitHub contributor data. It is activated by the `cache-updater` Maven profile. +**Test the contributors endpoint:** ```bash -# In a new terminal -cd backend -GITHUB_TOKENS="ghp_yourTokenHere" \ -SPRING_REDIS_HOST="localhost" \ -SPRING_REDIS_PORT="6379" \ -mvn spring-boot:run -Pcache-updater +curl "http://localhost:8450/api/contributors/search?languageId=java&maxResults=5" ``` -> **Note:** The Cache Updater runs the `PreCacheService` on startup, which iterates all configured languages and triggers GitHub API calls to fill the cache. The Backend Service will return cached data once this completes (typically 30–90 seconds on first run, depending on rate limit availability). - --- -## 5. Running the Frontend Dev Server +## Running the Frontend Locally -The Webpack dev server proxies `/api` requests to the backend: +### 1. Install Dependencies ```bash cd frontend npm install +``` -# Start dev server with backend proxy -BACKEND_API_URL=http://localhost:8450 npx webpack serve +### 2. Start the Development Server + +```bash +BACKEND_API_URL=http://localhost:8450 npm run dev ``` -**Default dev server URL:** [http://localhost:8450](http://localhost:8450) +The frontend dev server starts on **port 3000** with hot module replacement (HMR) enabled via Vite. -The dev server enables: -- Source maps for debugging -- Automatic chunk splitting -- Hot module replacement (HMR) for fast iteration -- Proxy of `/api/*` requests to the backend service +Open: **[http://localhost:3000](http://localhost:3000)** -> **Port note:** The Webpack dev server runs on port `8450` by default (matching the backend port so the browser points at one address). You can change the `PORT` environment variable if needed. +> **Note:** The `BACKEND_API_URL` environment variable tells the frontend where to send API requests. Without it, the frontend defaults to `/` (same-origin), which only works when served by the backend itself. + +### 3. Frontend Dev Server Proxy + +During development, API calls from the browser (port 3000) to the backend (port 8450) require the `BACKEND_API_URL` variable. This is the simplest approach for local development. --- -## 6. Building for Production +## Hot Reload / Watch Mode -### Backend +### Frontend (Vite Dev Server) -```bash -cd backend -mvn clean package -DskipTests -# Output: backend/target/major-league-github-0.0.1-SNAPSHOT.jar -``` +The Vite dev server provides **fast Hot Module Replacement (HMR)**. Changes to `.tsx`, `.ts`, `.css`, and other source files are reflected in the browser instantly without a full page reload. -Run the packaged JAR: +No additional configuration is needed — HMR is enabled by default via `vite.config.js`. -```bash -java -jar backend/target/major-league-github-0.0.1-SNAPSHOT.jar \ - --spring.profiles.active=backend-service -``` +### Backend (Spring Boot DevTools) + +Spring Boot DevTools is not explicitly listed as a dependency in the current `pom.xml`. For backend hot-reload during development, the recommended approaches are: + +**Option A — Use IntelliJ IDEA's "Build Project Automatically":** +1. Enable **Build → Build Project Automatically** in IntelliJ IDEA +2. Enable **Advanced Settings → Allow auto-make to start even if developed application is currently running** +3. Spring Boot will detect class changes and restart automatically -### Frontend +**Option B — Restart manually:** +Stop the running `mvn spring-boot:run` process and restart it. Maven rebuilds and reloads the application. + +--- + +## Running the Cache Updater Service + +The Cache Updater is a separate Spring Boot service on port 8451. It pre-warms and refreshes the Redis cache on a schedule. ```bash -cd frontend -NODE_ENV=production \ -BACKEND_API_URL=https://www.mlg.soccer \ -OG_URL=https://www.mlg.soccer \ -BASE_URL=https://www.mlg.soccer \ -npx webpack --mode production -# Output: frontend/dist/ +cd backend +GITHUB_TOKENS=ghp_your_token_here \ +mvn spring-boot:run -Dspring-boot.run.profiles=cache-updater ``` -The production build runs two custom Webpack plugins automatically: -- **FaviconGeneratorPlugin** — converts `public/favicon.svg` to `favicon.ico` -- **SeoFilesPlugin** — generates `sitemap.xml` and `robots.txt` with the configured base URL +> For local development, the Cache Updater is typically not required. The backend service warms the cache on startup via `PreCacheService`. --- -## 7. Hot Reload +## Debug Configuration -### Frontend +### Debugging the Backend (IntelliJ IDEA) -The Webpack dev server provides hot module replacement. Any change to `.tsx` or `.ts` files is reflected immediately in the browser without a full page reload. +1. Open the backend as a Maven project in IntelliJ IDEA +2. Create a **Spring Boot run configuration**: + - Main class: `cx.flamingo.analysis.MajorLeagueGithubApplication` + - Environment variables: `GITHUB_TOKENS=ghp_your_token_here;CACHE_IMPLEMENTATION=disk` +3. Click the **Debug** button (instead of Run) +4. Set breakpoints in any service or controller class -### Backend +### Debugging the Backend (VS Code) -Spring Boot DevTools is not explicitly listed as a dependency. For backend changes during development, restart the Spring Boot process manually with: +Create a `launch.json` in the repository root: -```bash -mvn spring-boot:run -Pbackend-service +```json +{ + "version": "0.2.0", + "configurations": [ + { + "type": "java", + "name": "MajorLeagueGithubApplication", + "request": "launch", + "mainClass": "cx.flamingo.analysis.MajorLeagueGithubApplication", + "projectName": "major-league-github", + "env": { + "GITHUB_TOKENS": "ghp_your_token_here", + "CACHE_IMPLEMENTATION": "disk" + } + } + ] +} ``` -IntelliJ IDEA supports **Build → Recompile** (`Cmd+Shift+F9` on macOS) when the Spring Boot run configuration is active, which triggers a faster incremental rebuild. +### Debugging the Frontend (VS Code) + +Use the **VS Code JavaScript debugger** with the Vite dev server: + +1. Start the dev server: `npm run dev` +2. In VS Code, open **Run and Debug** → **Create a launch.json file** +3. Select **Chrome** or **Edge** +4. Set URL to `http://localhost:3000` +5. Launch and set breakpoints in `.tsx`/`.ts` files --- -## 8. Debug Configuration +## Building for Production -### Backend (IntelliJ IDEA) +### Backend (Fat JAR) -Create a Run Configuration in IntelliJ: +```bash +cd backend +mvn clean package -DskipTests +``` -- **Type:** Spring Boot -- **Main class:** `cx.flamingo.analysis.MajorLeagueGithubApplication` -- **Active profiles:** `backend-service` -- **Environment variables:** `GITHUB_TOKENS=...;SPRING_REDIS_HOST=localhost;SPRING_REDIS_PORT=6379` +The fat JAR is generated at: -Set breakpoints anywhere in the Spring Boot code and use **Debug** mode to step through requests. +```text +backend/target/major-league-github-0.0.1-SNAPSHOT.jar +``` -### Backend (Remote Debug via Maven) +Run it: ```bash -cd backend -mvnDebug spring-boot:run -Pbackend-service -# Listens on port 8000 for a remote debugger +GITHUB_TOKENS=ghp_your_token_here \ +java -jar backend/target/major-league-github-0.0.1-SNAPSHOT.jar ``` -Then attach from IntelliJ at `localhost:8000`. +### Frontend (Webpack Production Build) -### Frontend (Browser DevTools) +```bash +cd frontend +npm run build +``` + +Output is written to `frontend/dist/` (or the configured output directory). The production build includes: -Source maps are enabled in development mode. Open Chrome DevTools → **Sources** → navigate to `webpack://./src/` to set breakpoints in TypeScript. +- Minified and bundled assets +- Favicon assets (via `FaviconGeneratorPlugin`) +- SEO files — sitemap and robots.txt (via `SeoFilesPlugin`) --- -## 9. Redis Inspection +## Common Local Development Scenarios -Inspect cached data in Redis: +### Scenario: No Redis — Use Disk Cache ```bash -# Connect to Redis CLI -redis-cli -h localhost -p 6379 - -# List all cache keys -KEYS * +cd backend +GITHUB_TOKENS=ghp_your_token_here \ +CACHE_IMPLEMENTATION=disk \ +mvn spring-boot:run +``` -# Get a specific cached entry -GET "contributors:java:los-angeles:page-0" +### Scenario: Always fetch fresh data (bypass cache) -# Check if cache is populated -DBSIZE +```bash +cd backend +GITHUB_TOKENS=ghp_your_token_here \ +CACHE_MODE=force-update \ +mvn spring-boot:run ``` ---- +> **Warning:** This bypasses the cache and queries GitHub's API on every request. Use only for debugging and be aware of rate limits. -## 10. Multi-Service Local Architecture +### Scenario: Multiple GitHub tokens for higher throughput -```mermaid -flowchart LR - Browser["Browser :8450"] --> Webpack["Webpack Dev Server"] - Webpack --> Frontend["React App"] - Webpack -->|"/api/* proxy"| Backend["Backend Service :8450"] - Backend --> Redis["Redis :6379"] - CacheUpdater["Cache Updater :8451"] --> Redis - CacheUpdater --> GitHub["GitHub GraphQL API"] - Backend --> GitHub +```bash +cd backend +GITHUB_TOKENS=ghp_token1,ghp_token2,ghp_token3 \ +mvn spring-boot:run ``` -All four processes run concurrently during full-stack local development. +The `GithubTokenRateManager` automatically selects the token with the most remaining requests. diff --git a/docs/diagrams/architecture/README-2.mmd b/docs/diagrams/architecture/README-2.mmd index 2ccd60d..9408252 100644 --- a/docs/diagrams/architecture/README-2.mmd +++ b/docs/diagrams/architecture/README-2.mmd @@ -1,10 +1,19 @@ flowchart TD - App["MajorLeagueGithubApplication (Module 1)"] - App --> Controllers["Controllers (Module 4)"] - Controllers --> Services["Services (Module 8–10)"] - Services --> GraphQL["GraphQL Builder (Module 5)"] - Services --> RateManager["GitHub Rate Manager (Module 8)"] - Services --> Cache["Cache Abstraction (Module 1)"] - Cache --> RedisImpl["RedisCacheService (Module 2)"] - Cache --> DiskImpl["DiskCacheService (Module 1)"] - Services --> Models["Domain Models (Module 6–7)"] + AppCore["Application Core"] + Controllers["Controllers"] + Services["Backend Services"] + CacheLayer["Cache Services"] + GraphQL["GraphQL Components"] + Rate["Rate Management"] + Models["Model Entities"] + Config["Configurations"] + + AppCore --> Controllers + AppCore --> Services + AppCore --> CacheLayer + AppCore --> Config + Services --> GraphQL + Services --> Rate + Services --> Models + Controllers --> Services + Services --> CacheLayer diff --git a/docs/diagrams/architecture/README-3.mmd b/docs/diagrams/architecture/README-3.mmd index b0ab107..6607937 100644 --- a/docs/diagrams/architecture/README-3.mmd +++ b/docs/diagrams/architecture/README-3.mmd @@ -1,15 +1,6 @@ -sequenceDiagram - participant Controller - participant Service as GithubService - participant Rate as GithubTokenRateManager - participant Builder as GitHubQueryBuilder - participant GitHub - - Controller->>Service: getTopContributors(filters) - Service->>Builder: build GraphQL query - Builder-->>Service: query string - Service->>Rate: select optimal token - Rate-->>Service: WebClient - Service->>GitHub: Execute GraphQL request - GitHub-->>Service: JSON response - Service-->>Controller: Ranked Contributors +flowchart TD + Pages["React Pages"] --> Components["Frontend Components"] + Components --> Hooks["Frontend Hooks"] + Hooks --> Services["Frontend Services"] + Services --> Backend["Backend REST API"] + Services --> Types["Frontend Types"] diff --git a/docs/diagrams/architecture/README-4.mmd b/docs/diagrams/architecture/README-4.mmd index fb11d85..2dd01e5 100644 --- a/docs/diagrams/architecture/README-4.mmd +++ b/docs/diagrams/architecture/README-4.mmd @@ -1,11 +1,10 @@ flowchart TD - Router["React Router"] --> UrlState["useUrlState (Module 13)"] - UrlState --> ApiService["API Service (Module 14)"] - ApiService --> BackendAPI["Backend REST API"] - - BackendAPI --> Types["Core API Types (Module 15)"] - Types --> Enhanced["Enhanced Models (Module 16)"] - Enhanced --> Table["Contributors Table (Module 11–12)"] - - Table --> Autocomplete["BaseAutocomplete (Module 10)"] - Table --> Pagination["Pagination (Module 12)"] + Request["Contributor Search Request"] --> CacheCheck["CacheServiceAbs.getHttpResponse()"] + CacheCheck -->|Miss| GithubFetch["GithubService"] + GithubFetch --> QueryBuilder["GitHubQueryBuilder"] + QueryBuilder --> GitHubAPI["GitHub GraphQL API"] + GitHubAPI --> Parse["Parse & Map to Contributor"] + Parse --> Score["Apply Scoring Formula"] + Score --> Store["Store in Cache"] + Store --> Response["ApiResponse>"] + CacheCheck -->|Hit| Response diff --git a/docs/diagrams/architecture/README-5.mmd b/docs/diagrams/architecture/README-5.mmd index fc6728d..ed31681 100644 --- a/docs/diagrams/architecture/README-5.mmd +++ b/docs/diagrams/architecture/README-5.mmd @@ -1,9 +1,7 @@ -flowchart TD - Request["Incoming Request"] --> CacheCheck["CacheServiceAbs.get()"] - CacheCheck --> Exists{"Entry Exists?"} - Exists -->|"No"| Fetch["Fetch From GitHub"] - Exists -->|"Yes"| Stale{"Is Stale?"} - Stale -->|"No"| Return["Return Cached Data"] - Stale -->|"Yes"| AsyncRefresh["Async Background Refresh"] - Fetch --> Store["Store In Cache"] - Store --> Return +flowchart LR + GitHubRepo["GitHub Repository"] --> CI["GitHub Actions CI/CD"] + CI --> Docker["Docker Images"] + Docker --> GKE["Google Kubernetes Engine"] + GKE --> BackendPods["Backend + Cache Updater Pods"] + GKE --> RedisPod["Redis"] + GKE --> FrontendService["Frontend Service"] diff --git a/docs/diagrams/architecture/README-6.mmd b/docs/diagrams/architecture/README-6.mmd deleted file mode 100644 index 789232d..0000000 --- a/docs/diagrams/architecture/README-6.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart LR - GitHubActions["GitHub Actions CI/CD"] - GitHubActions --> Docker["Docker Images"] - Docker --> GKE["Google Kubernetes Engine"] - - GKE --> BackendPod["Backend Service (8450)"] - GKE --> CacheUpdaterPod["Cache Updater (8451)"] - GKE --> RedisPod["Redis"] - - BackendPod --> RedisPod - CacheUpdaterPod --> RedisPod diff --git a/docs/diagrams/architecture/README-7.mmd b/docs/diagrams/architecture/README-7.mmd deleted file mode 100644 index ea75338..0000000 --- a/docs/diagrams/architecture/README-7.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - Dev --> GitHubActions["GitHub Actions CI/CD"] - GitHubActions --> DockerImage["Docker Image"] - DockerImage --> Kubernetes["GKE Cluster"] - Kubernetes --> BackendPod - Kubernetes --> CacheUpdaterPod - BackendPod --> Redis diff --git a/docs/diagrams/architecture/README.mmd b/docs/diagrams/architecture/README.mmd index 19e69f6..9538b34 100644 --- a/docs/diagrams/architecture/README.mmd +++ b/docs/diagrams/architecture/README.mmd @@ -1,18 +1,8 @@ -flowchart TD - User["User Browser"] --> Frontend["React Frontend (Module 10–18)"] - Frontend --> ApiLayer["API Layer (Module 14)"] - ApiLayer --> Backend["Spring Boot Backend (Port 8450)"] - - Backend --> Controllers["REST Controllers (Module 4)"] - Controllers --> Services["Service Layer (Module 8–10)"] - Services --> CacheAbs["CacheServiceAbs (Module 1)"] - CacheAbs --> Redis["Redis (Module 2)"] - CacheAbs --> Disk["Disk Cache (Module 1)"] - - Services --> QueryBuilder["GraphQL Query Builder (Module 5)"] - QueryBuilder --> GitHubAPI["GitHub GraphQL API"] - - Services --> LinkedInService["LinkedIn Service (Module 9)"] - LinkedInService --> LinkedInAPI["LinkedIn API"] - - CacheUpdater["Cache Updater Service (Port 8451)"] --> Redis +flowchart LR + User["User (Browser)"] --> Frontend["React Frontend (Port 3000 / Prod)"] + Frontend --> Backend["Backend Service (Spring Boot - Port 8450)"] + Backend --> Cache["Redis Cache"] + Backend --> GitHub["GitHub GraphQL API"] + Backend --> LinkedIn["LinkedIn API"] + CacheUpdater["Cache Updater Service (Port 8451)"] --> Backend + CacheUpdater --> GitHub diff --git a/docs/diagrams/architecture/application-core-2.mmd b/docs/diagrams/architecture/application-core-2.mmd new file mode 100644 index 0000000..3a446b0 --- /dev/null +++ b/docs/diagrams/architecture/application-core-2.mmd @@ -0,0 +1,14 @@ +sequenceDiagram + participant JVM as JVM + participant Spring as SpringApplication + participant Context as ApplicationContext + participant Config as Configuration Beans + participant Services as Service Beans + participant Controllers as Controller Beans + + JVM->>Spring: main(args) + Spring->>Context: Create ApplicationContext + Context->>Config: Initialize configuration classes + Context->>Services: Instantiate service beans + Context->>Controllers: Instantiate controllers + Context-->>Spring: Application Ready diff --git a/docs/diagrams/architecture/application-core-3.mmd b/docs/diagrams/architecture/application-core-3.mmd new file mode 100644 index 0000000..37b3abc --- /dev/null +++ b/docs/diagrams/architecture/application-core-3.mmd @@ -0,0 +1,5 @@ +flowchart LR + Service["Backend Service"] -->|"@Cacheable"| CacheAbstraction["Spring Cache Abstraction"] + CacheAbstraction --> Redis["Redis Cache"] + CacheAbstraction --> Disk["Disk Cache"] + CacheAbstraction --> ReadOnly["Read Only Cache"] diff --git a/docs/diagrams/architecture/application-core-4.mmd b/docs/diagrams/architecture/application-core-4.mmd new file mode 100644 index 0000000..722194f --- /dev/null +++ b/docs/diagrams/architecture/application-core-4.mmd @@ -0,0 +1,4 @@ +flowchart TD + Controller["Controller"] --> Service["Service Method"] + Service -->|"@Async"| AsyncExecutor["Task Executor"] + AsyncExecutor --> BackgroundTask["Background Job"] diff --git a/docs/diagrams/architecture/application-core-5.mmd b/docs/diagrams/architecture/application-core-5.mmd new file mode 100644 index 0000000..536c374 --- /dev/null +++ b/docs/diagrams/architecture/application-core-5.mmd @@ -0,0 +1,10 @@ +flowchart TD + Client["Frontend Client"] --> Controller["Controller"] + Controller --> Service["Backend Service"] + Service --> CacheCheck["Cache Layer"] + CacheCheck -->|"Miss"| GitHub["GitHub GraphQL API"] + GitHub --> Service + Service --> Model["Model Entities"] + Service --> ApiResponse["ApiResponse"] + ApiResponse --> Controller + Controller --> Client diff --git a/docs/diagrams/architecture/application-core-6.mmd b/docs/diagrams/architecture/application-core-6.mmd new file mode 100644 index 0000000..35383a7 --- /dev/null +++ b/docs/diagrams/architecture/application-core-6.mmd @@ -0,0 +1,4 @@ +flowchart LR + Frontend["React Frontend"] --> Backend["Application Core (Spring Boot)"] + Backend --> GitHub["GitHub API"] + Backend --> Redis["Redis"] diff --git a/docs/diagrams/architecture/application-core.mmd b/docs/diagrams/architecture/application-core.mmd new file mode 100644 index 0000000..f5f7249 --- /dev/null +++ b/docs/diagrams/architecture/application-core.mmd @@ -0,0 +1,24 @@ +flowchart TD + AppCore["Application Core"] + + Controllers["Controllers"] + Services["Backend Services"] + Cache["Cache Services"] + Rate["Rate Management"] + Config["Configurations"] + Models["Model Entities"] + GraphQL["GraphQL Components"] + + AppCore --> Controllers + AppCore --> Services + AppCore --> Cache + AppCore --> Rate + AppCore --> Config + AppCore --> Models + AppCore --> GraphQL + + Controllers --> Services + Services --> Cache + Services --> Rate + Services --> GraphQL + Services --> Models diff --git a/docs/diagrams/architecture/backend-services-2.mmd b/docs/diagrams/architecture/backend-services-2.mmd new file mode 100644 index 0000000..7e67ca3 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-2.mmd @@ -0,0 +1,10 @@ +flowchart TD + Start["Request Contributors"] --> BuildQuery["Build GraphQL Query"] + BuildQuery --> Execute["Execute via WebClient"] + Execute --> RateCheck{"Rate Limited?"} + RateCheck -->|Yes| SwitchToken["Switch Token"] + RateCheck -->|No| Parse["Parse JSON Response"] + SwitchToken --> Execute + Parse --> Process["Process Users"] + Process --> Score["Calculate Score"] + Score --> Return["Return Ranked Contributors"] diff --git a/docs/diagrams/architecture/backend-services-3.mmd b/docs/diagrams/architecture/backend-services-3.mmd new file mode 100644 index 0000000..2fc4c68 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + Init["PostConstruct Init"] --> LoadRegions["Get All Regions"] + LoadRegions --> ResolveStates["Resolve States by Code"] + ResolveStates --> ResolveCities["Resolve Cities by Region"] + ResolveCities --> UpdateRegion["Update Region in RegionService"] diff --git a/docs/diagrams/architecture/backend-services-4.mmd b/docs/diagrams/architecture/backend-services-4.mmd new file mode 100644 index 0000000..d2a3b38 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-4.mmd @@ -0,0 +1,7 @@ +flowchart TD + Request["Get Hiring Profile"] --> CacheCheck{"In Cache?"} + CacheCheck -->|Yes| ReturnCached["Return Cached Profile"] + CacheCheck -->|No| FetchGitHub["Fetch via GithubService"] + FetchGitHub --> BuildProfile["Build HiringManagerProfile"] + BuildProfile --> StoreCache["Cache Result"] + StoreCache --> ReturnProfile["Return Profile"] diff --git a/docs/diagrams/architecture/backend-services-5.mmd b/docs/diagrams/architecture/backend-services-5.mmd new file mode 100644 index 0000000..97a0218 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-5.mmd @@ -0,0 +1,6 @@ +flowchart TD + Start["Get Company Job Postings"] --> Token["Request OAuth Token"] + Token --> FetchUpdates["Fetch Organization Updates"] + FetchUpdates --> FilterJobs["Filter jobPosting Content"] + FilterJobs --> MapJobs["Map to JobOpening Model"] + MapJobs --> Cache["Store in Cache"] diff --git a/docs/diagrams/architecture/backend-services-6.mmd b/docs/diagrams/architecture/backend-services-6.mmd new file mode 100644 index 0000000..e1b2aa0 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-6.mmd @@ -0,0 +1,6 @@ +flowchart TD + Startup["Application Startup"] --> LoopLanguages["Iterate Languages"] + LoopLanguages --> Trigger["Call ContributorController"] + Trigger --> GithubFetch["GithubService Fetch"] + GithubFetch --> CacheStore["Cache Responses"] + CacheStore --> Complete["Mark Cache Ready"] diff --git a/docs/diagrams/architecture/backend-services-7.mmd b/docs/diagrams/architecture/backend-services-7.mmd new file mode 100644 index 0000000..0464af7 --- /dev/null +++ b/docs/diagrams/architecture/backend-services-7.mmd @@ -0,0 +1,6 @@ +flowchart TD + Request["City Batch"] --> Async["CompletableFuture Execution"] + Async --> TokenManager["GithubTokenRateManager"] + TokenManager --> WebClient["WebClient Call"] + WebClient --> CacheLayer["CacheServiceAbs"] + CacheLayer --> Response["Parsed JSON"] diff --git a/docs/diagrams/architecture/backend-services-8.mmd b/docs/diagrams/architecture/backend-services-8.mmd new file mode 100644 index 0000000..9631f1c --- /dev/null +++ b/docs/diagrams/architecture/backend-services-8.mmd @@ -0,0 +1,6 @@ +flowchart LR + Frontend["Frontend Application"] --> Controllers["REST Controllers"] + Controllers --> Backend["Backend Services"] + Backend --> CacheLayer["Cache Services"] + Backend --> ExternalAPIs["External APIs"] + Backend --> StaticData["CSV Data"] diff --git a/docs/diagrams/architecture/backend-services.mmd b/docs/diagrams/architecture/backend-services.mmd new file mode 100644 index 0000000..dd29a3d --- /dev/null +++ b/docs/diagrams/architecture/backend-services.mmd @@ -0,0 +1,8 @@ +flowchart TD + Controller["Controllers"] --> Services["Backend Services"] + Services --> Cache["Cache Services"] + Services --> GraphQL["GraphQL Components"] + Services --> Rate["Rate Management"] + Services --> Models["Model Entities"] + Services --> ExternalGitHub["GitHub GraphQL API"] + Services --> ExternalLinkedIn["LinkedIn API"] diff --git a/docs/diagrams/architecture/cache-services-2.mmd b/docs/diagrams/architecture/cache-services-2.mmd index 9cd15bf..53b5799 100644 --- a/docs/diagrams/architecture/cache-services-2.mmd +++ b/docs/diagrams/architecture/cache-services-2.mmd @@ -1,9 +1,16 @@ -flowchart LR - Request["Incoming Request"] --> KeyGen["Key Generation"] - KeyGen --> CacheLookup["Cache Lookup"] - CacheLookup -->|"Hit"| ReturnCached["Return Cached Data"] - CacheLookup -->|"Miss"| Supplier["Execute Supplier"] - Supplier --> Store["Store in Cache"] - Store --> ReturnFresh["Return Fresh Data"] - - CacheLookup -->|"Stale"| AsyncRefresh["Async Refresh"] +flowchart TD + Request["Incoming Request"] --> CheckMode{"FORCE_UPDATE?"} + CheckMode -->|"No"| TryCache["Attempt Cache Read"] + CheckMode -->|"Yes"| Fetch + + TryCache --> Hit{"Cache Hit?"} + Hit -->|"Yes"| Stale{"Stale?"} + Hit -->|"No"| Fetch + + Stale -->|"Yes"| AsyncRefresh["Async Refresh"] + Stale -->|"No"| ReturnCached["Return Cached Value"] + + AsyncRefresh --> ReturnCached + + Fetch["Execute Supplier (HTTP/GitHub)"] --> Store["Put in Cache"] + Store --> ReturnFresh["Return Fresh Value"] diff --git a/docs/diagrams/architecture/cache-services-3.mmd b/docs/diagrams/architecture/cache-services-3.mmd index c900fd9..6c840b4 100644 --- a/docs/diagrams/architecture/cache-services-3.mmd +++ b/docs/diagrams/architecture/cache-services-3.mmd @@ -1,11 +1,14 @@ -sequenceDiagram - participant Client - participant Cache - participant Supplier +flowchart TD + Put["put(cachePath, key, value)"] --> Serialize["Serialize to JSON"] + Serialize --> WriteFile["Write .json"] - Client->>Cache: getHttpResponse() - Cache->>Cache: Check staleness - Cache-->>Client: Return stale value - Cache->>Supplier: Async refresh - Supplier-->>Cache: Fresh data - Cache->>Cache: Overwrite entry + Get["get(cachePath, key)"] --> Exists{"File Exists?"} + Exists -->|"No"| Miss["Cache Miss"] + Exists -->|"Yes"| Stale{"Stale?"} + + Stale -->|"Yes"| Delete["Delete File"] + Delete --> Miss + + Stale -->|"No"| Read["Read JSON"] + Read --> Deserialize["Gson.fromJson"] + Deserialize --> Hit["Return Value"] diff --git a/docs/diagrams/architecture/cache-services-4.mmd b/docs/diagrams/architecture/cache-services-4.mmd new file mode 100644 index 0000000..645c806 --- /dev/null +++ b/docs/diagrams/architecture/cache-services-4.mmd @@ -0,0 +1,8 @@ +flowchart LR + Put["put(cachePath, key)"] --> BuildKey["Build Redis Key"] + BuildKey --> StoreValue["SET key -> JSON"] + StoreValue --> StoreMeta["SET key:expiration -> timestamp"] + + Get["get(cachePath, key)"] --> Fetch["GET key"] + Fetch --> Deserialize["Gson.fromJson"] + Deserialize --> Return diff --git a/docs/diagrams/architecture/cache-services-5.mmd b/docs/diagrams/architecture/cache-services-5.mmd new file mode 100644 index 0000000..9c785e9 --- /dev/null +++ b/docs/diagrams/architecture/cache-services-5.mmd @@ -0,0 +1,7 @@ +flowchart TD + Request --> Mode{"Mode == FORCE_UPDATE?"} + Mode -->|"Yes"| Execute["Execute Supplier"] + Execute --> Store + Store --> Return + + Mode -->|"No"| Normal["Normal Cache Flow"] diff --git a/docs/diagrams/architecture/cache-services.mmd b/docs/diagrams/architecture/cache-services.mmd index 455bce4..c2dcf28 100644 --- a/docs/diagrams/architecture/cache-services.mmd +++ b/docs/diagrams/architecture/cache-services.mmd @@ -1,10 +1,11 @@ -flowchart TD - Controller["Controllers"] --> ServiceLayer["Service Layer"] - ServiceLayer --> CacheService["CacheServiceAbs"] +flowchart LR + Controllers["Controllers"] --> Services["Backend Services"] + Services --> CacheAbs["CacheServiceAbs (Abstract)"] - CacheService --> DiskCache["Disk Cache Service"] - CacheService --> RedisCache["Redis Cache Service"] - RedisCache --> ReadOnlyCache["Read Only Cache Service"] + CacheAbs --> Disk["DiskCacheService"] + CacheAbs --> Redis["RedisCacheService"] + Redis --> ReadOnly["ReadOnlyCacheService"] - DiskCache --> FileSystem[("File System")] - RedisCache --> Redis[("Redis")] + Disk --> FS[("File System")] + Redis --> RedisDB[("Redis")] + Services --> GitHub[("GitHub API")] diff --git a/docs/diagrams/architecture/cache_layer-2.mmd b/docs/diagrams/architecture/cache_layer-2.mmd deleted file mode 100644 index 3b666c8..0000000 --- a/docs/diagrams/architecture/cache_layer-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Service["Service"] --> CheckCache["Call get()"] - CheckCache --> RedisLookup["Redis GET key"] - RedisLookup --> Found{"Value Found?"} - Found -->|"Yes"| Deserialize["Gson fromJson()"] - Deserialize --> ReturnValue["Return Optional"] - Found -->|"No"| Empty["Return Optional.empty()"] diff --git a/docs/diagrams/architecture/cache_layer-3.mmd b/docs/diagrams/architecture/cache_layer-3.mmd deleted file mode 100644 index c6e85e9..0000000 --- a/docs/diagrams/architecture/cache_layer-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Service["Service"] --> PutCall["Call put()"] - PutCall --> Serialize["Gson toJson()"] - Serialize --> RedisSetValue["Redis SET key"] - Serialize --> RedisSetExpiration["Redis SET key:expiration"] diff --git a/docs/diagrams/architecture/cache_layer-4.mmd b/docs/diagrams/architecture/cache_layer-4.mmd deleted file mode 100644 index f2c08c6..0000000 --- a/docs/diagrams/architecture/cache_layer-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Caller["Service or Error Handler"] --> Invalidate["Call invalidate()"] - Invalidate --> DeleteValue["Redis DEL key"] - Invalidate --> DeleteExpiration["Redis DEL key:expiration"] diff --git a/docs/diagrams/architecture/cache_layer-5.mmd b/docs/diagrams/architecture/cache_layer-5.mmd deleted file mode 100644 index 4e5c57b..0000000 --- a/docs/diagrams/architecture/cache_layer-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Request["Incoming Request"] --> CacheLookup["Cache get()"] - CacheLookup --> TimestampCheck["Compare current time with insert timestamp"] - TimestampCheck --> Expired{"Expired?"} - Expired -->|"Yes"| Recompute["Fetch fresh data"] - Expired -->|"No"| ServeCached["Serve cached value"] diff --git a/docs/diagrams/architecture/cache_layer-6.mmd b/docs/diagrams/architecture/cache_layer-6.mmd deleted file mode 100644 index f164530..0000000 --- a/docs/diagrams/architecture/cache_layer-6.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - GitHubCache["github"] --> Key1["github:query1"] - HttpCache["http"] --> Key2["http:request1"] diff --git a/docs/diagrams/architecture/cache_layer.mmd b/docs/diagrams/architecture/cache_layer.mmd deleted file mode 100644 index d27f386..0000000 --- a/docs/diagrams/architecture/cache_layer.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Controller["Controller Layer"] --> Service["Service Layer"] - Service --> CacheAbs["CacheServiceAbs"] - CacheAbs --> RedisCache["RedisCacheService"] - RedisCache --> Redis["Redis Server"] diff --git a/docs/diagrams/architecture/configuration_layer-2.mmd b/docs/diagrams/architecture/configuration_layer-2.mmd deleted file mode 100644 index 8b14ea6..0000000 --- a/docs/diagrams/architecture/configuration_layer-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart LR - Property["github.api.concurrency"] --> CoreSize["Core Pool Size"] - CoreSize --> ExecutorLow["Low Priority Executor"] - CoreSize --> ExecutorHigh["High Priority Executor"] - - ExecutorLow --> QueueLow["Queue Capacity: 1000"] - ExecutorHigh --> QueueHigh["Queue Capacity: 1000"] - - ExecutorLow --> MaxLow["Max Pool Size: 100"] - ExecutorHigh --> MaxHigh["Max Pool Size: 100"] diff --git a/docs/diagrams/architecture/configuration_layer-3.mmd b/docs/diagrams/architecture/configuration_layer-3.mmd deleted file mode 100644 index c22767c..0000000 --- a/docs/diagrams/architecture/configuration_layer-3.mmd +++ /dev/null @@ -1,13 +0,0 @@ -flowchart TD - Start["Application Startup"] --> ReadProps["Read cache.implementation & cache.mode"] - ReadProps --> ParseMode["Parse CacheMode Enum"] - ReadProps --> ParseImpl["Parse CacheImplementation Enum"] - - ParseMode --> ApplyMode["Set Mode on All Cache Services"] - ParseImpl --> Decision{"READ_ONLY?"} - - Decision -->|"Yes"| ReturnRO["Return ReadOnlyCacheService"] - Decision -->|"No"| ImplChoice{"Implementation = REDIS?"} - - ImplChoice -->|"Yes"| ReturnRedis["Return RedisCacheService"] - ImplChoice -->|"No"| ReturnDisk["Return DiskCacheService"] diff --git a/docs/diagrams/architecture/configuration_layer-4.mmd b/docs/diagrams/architecture/configuration_layer-4.mmd deleted file mode 100644 index d883c1e..0000000 --- a/docs/diagrams/architecture/configuration_layer-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Service["Service Component"] -->|"@Autowired CacheServiceAbs"| CacheBean["Selected Cache Implementation"] - Service -->|"@Autowired contributorsAsyncExecutorLow"| ExecutorLow["Low Priority Executor"] - Service -->|"@Autowired contributorsAsyncExecutorHigh"| ExecutorHigh["High Priority Executor"] diff --git a/docs/diagrams/architecture/configuration_layer.mmd b/docs/diagrams/architecture/configuration_layer.mmd deleted file mode 100644 index 42ae55f..0000000 --- a/docs/diagrams/architecture/configuration_layer.mmd +++ /dev/null @@ -1,23 +0,0 @@ -flowchart TD - subgraph ConfigLayer["Configuration Layer"] - AsyncConfig["AsyncConfig"] - CacheConfig["CacheConfig"] - BackendServiceConfig["BackendServiceConfig"] - end - - subgraph CacheLayer["Cache Layer"] - RedisCacheService["RedisCacheService"] - DiskCacheService["DiskCacheService"] - ReadOnlyCacheService["ReadOnlyCacheService"] - CacheServiceAbs["CacheServiceAbs"] - end - - subgraph Services["Service Layer"] - GithubService["GithubService"] - ContributorServices["Contributor Services"] - end - - AsyncConfig -->|"ThreadPoolExecutor Beans"| Services - CacheConfig -->|"@Primary CacheServiceAbs"| Services - CacheConfig --> CacheLayer - BackendServiceConfig --> Services diff --git a/docs/diagrams/architecture/configurations-2.mmd b/docs/diagrams/architecture/configurations-2.mmd index 875de06..3f609a3 100644 --- a/docs/diagrams/architecture/configurations-2.mmd +++ b/docs/diagrams/architecture/configurations-2.mmd @@ -1,10 +1,6 @@ -flowchart TD - Start["Application Startup"] --> Mode["Read cache.mode"] - Mode --> Impl["Read cache.implementation"] - - Impl --> Decision{"Mode == read-only?"} - Decision -->|"Yes"| ReadOnly["Use ReadOnlyCacheService"] - Decision -->|"No"| ImplChoice{"Implementation"} - - ImplChoice -->|"redis"| RedisCache["Use RedisCacheService"] - ImplChoice -->|"disk"| DiskCache["Use DiskCacheService"] +flowchart LR + Controller["Controller"] --> Service["Backend Service"] + Service -->|"Submit Task"| LowPool["Low Priority Executor"] + Service -->|"Submit Task"| HighPool["High Priority Executor"] + LowPool --> GithubAPI["GitHub API"] + HighPool --> GithubAPI diff --git a/docs/diagrams/architecture/configurations-3.mmd b/docs/diagrams/architecture/configurations-3.mmd index ef53185..e357c95 100644 --- a/docs/diagrams/architecture/configurations-3.mmd +++ b/docs/diagrams/architecture/configurations-3.mmd @@ -1,5 +1,8 @@ -flowchart LR - Service["Service Layer"] --> Cache["RedisCacheService"] - Cache --> Template["RedisTemplate"] - Template --> Redis[("Redis")] - Cache --> Gson["Gson with Time Adapters"] +flowchart TD + Start["Application Startup"] --> ReadProps["Read cache.* Properties"] + ReadProps --> Mode{"Cache Mode?"} + Mode -->|"read-only"| ReadOnly["ReadOnlyCacheService"] + Mode -->|"read-write"| Impl{"Implementation?"} + Mode -->|"force-update"| Impl + Impl -->|"redis"| RedisImpl["RedisCacheService"] + Impl -->|"disk"| DiskImpl["DiskCacheService"] diff --git a/docs/diagrams/architecture/configurations-4.mmd b/docs/diagrams/architecture/configurations-4.mmd index 4efe397..00432af 100644 --- a/docs/diagrams/architecture/configurations-4.mmd +++ b/docs/diagrams/architecture/configurations-4.mmd @@ -1,4 +1,4 @@ -flowchart TD - Profile["cache-updater Profile"] --> Scheduling["EnableScheduling"] - Scheduling --> Jobs["Scheduled Cache Refresh Jobs"] - Jobs --> Cache["CacheServiceAbs"] +flowchart LR + RedisConfig["RedisConfig"] --> Factory["RedisConnectionFactory"] + Factory --> Template["RedisTemplate"] + Template --> CacheService["RedisCacheService"] diff --git a/docs/diagrams/architecture/configurations-5.mmd b/docs/diagrams/architecture/configurations-5.mmd new file mode 100644 index 0000000..5454832 --- /dev/null +++ b/docs/diagrams/architecture/configurations-5.mmd @@ -0,0 +1,4 @@ +flowchart LR + Browser["Frontend"] -->|"HTTP Request"| Backend["Backend Service"] + Backend -->|"CORS Validation"| WebConfig + WebConfig -->|"Allowed Origin"| Response["HTTP Response"] diff --git a/docs/diagrams/architecture/configurations-6.mmd b/docs/diagrams/architecture/configurations-6.mmd new file mode 100644 index 0000000..2658169 --- /dev/null +++ b/docs/diagrams/architecture/configurations-6.mmd @@ -0,0 +1,4 @@ +flowchart TD + Startup["Application Startup"] --> Profile{"Active Profile"} + Profile -->|"backend-service"| BackendMode["REST API Mode"] + Profile -->|"cache-updater"| UpdaterMode["Scheduled Refresh Mode"] diff --git a/docs/diagrams/architecture/configurations.mmd b/docs/diagrams/architecture/configurations.mmd index 989580d..f83efea 100644 --- a/docs/diagrams/architecture/configurations.mmd +++ b/docs/diagrams/architecture/configurations.mmd @@ -1,24 +1,26 @@ flowchart TD - App["MajorLeagueGithubApplication"] --> Config["Configurations Module"] + App["Spring Boot Application"] --> Config["Configurations Module"] subgraph infra["Infrastructure Beans"] Async["AsyncConfig"] Cache["CacheConfig"] Redis["RedisConfig"] Web["WebConfig"] - Backend["BackendServiceConfig"] - Updater["CacheUpdaterConfig"] + end + + subgraph profiles["Profile-Specific"] + BackendProfile["BackendServiceConfig"] + UpdaterProfile["CacheUpdaterConfig"] end Config --> Async Config --> Cache Config --> Redis Config --> Web - Config --> Backend - Config --> Updater + Config --> BackendProfile + Config --> UpdaterProfile - Cache --> CacheServices["Cache Services"] - Redis --> RedisServer[("Redis Server")] - Async --> Services["Service Layer"] - Web --> Controllers["Controllers"] - Updater --> Scheduler["Scheduled Jobs"] + Cache --> CacheServices["Cache Services Module"] + Async --> BackendServices["Backend Services Module"] + Web --> Controllers["Controllers Module"] + Redis --> CacheServices diff --git a/docs/diagrams/architecture/controllers-2.mmd b/docs/diagrams/architecture/controllers-2.mmd index 1340a74..087c173 100644 --- a/docs/diagrams/architecture/controllers-2.mmd +++ b/docs/diagrams/architecture/controllers-2.mmd @@ -1,5 +1,8 @@ -flowchart LR - Request["HTTP Request"] --> Controller["Autocomplete Controller"] - Controller --> Service["Domain Service"] - Service --> Entities["Model Entities"] - Entities --> Response["ApiResponse>"] +flowchart TD + Request["GET /api/autocomplete/*"] --> Controller["AutocompleteController"] + Controller --> CityServiceNode["CityService"] + Controller --> StateServiceNode["StateService"] + Controller --> RegionServiceNode["RegionService"] + Controller --> LanguageServiceNode["LanguageService"] + Controller --> SoccerTeamServiceNode["SoccerTeamService"] + Controller --> ResponseWrap["ApiResponse.success()"] diff --git a/docs/diagrams/architecture/controllers-3.mmd b/docs/diagrams/architecture/controllers-3.mmd index 51b5aa7..8e91671 100644 --- a/docs/diagrams/architecture/controllers-3.mmd +++ b/docs/diagrams/architecture/controllers-3.mmd @@ -1,11 +1,9 @@ flowchart TD - Request["Search Request"] --> CacheReady{"Cache Ready?"} - CacheReady -->|"No"| Error["Return ApiResponse.error"] - CacheReady -->|"Yes"| CacheLookup["Cache Service getHttpResponse()"] - - CacheLookup -->|"Hit"| ReturnCached["Return Cached Contributors"] - CacheLookup -->|"Miss"| FetchCities["GitHub Service getTargetCities()"] - FetchCities --> SelectLang["Resolve Language"] - SelectLang --> FetchContrib["GitHub Service getTopContributorsIn()"] - FetchContrib --> StoreCache["Store in Cache"] - StoreCache --> ReturnResponse["Return ApiResponse.success"] + Request["GET /search"] --> CacheCheck["CacheServiceAbs.isCacheReady()"] + CacheCheck -->|"not ready"| ErrorResp["ApiResponse.error()"] + CacheCheck -->|"ready"| CacheLookup["CacheServiceAbs.getHttpResponse()"] + CacheLookup --> GithubServiceNode["GithubService"] + GithubServiceNode --> CitiesNode["CityService.getTargetCities()"] + GithubServiceNode --> LanguageNode["LanguageService"] + GithubServiceNode --> Result["List"] + Result --> SuccessResp["ApiResponse.success()"] diff --git a/docs/diagrams/architecture/controllers-4.mmd b/docs/diagrams/architecture/controllers-4.mmd index 6688802..d1ea0af 100644 --- a/docs/diagrams/architecture/controllers-4.mmd +++ b/docs/diagrams/architecture/controllers-4.mmd @@ -1,6 +1,5 @@ -flowchart LR - Request["GET by ID"] --> Controller["Entity Controller"] - Controller --> Service["Domain Service"] - Service --> Found{"Entity Found?"} - Found -->|"No"| Error["ApiResponse.error"] - Found -->|"Yes"| Success["ApiResponse.success"] +flowchart TD + Request["GET /api/entities/{type}/{id}"] --> Controller["EntityController"] + Controller --> ServiceCall["*Service.getById()"] + ServiceCall -->|"null"| ErrorResp["ApiResponse.error()"] + ServiceCall -->|"found"| SuccessResp["ApiResponse.success()"] diff --git a/docs/diagrams/architecture/controllers-5.mmd b/docs/diagrams/architecture/controllers-5.mmd index 45f501d..479b6bc 100644 --- a/docs/diagrams/architecture/controllers-5.mmd +++ b/docs/diagrams/architecture/controllers-5.mmd @@ -1,5 +1,5 @@ -flowchart LR - Request["Hiring Request"] --> HiringCtrl["Hiring Controller"] - HiringCtrl --> HiringService["Hiring Service"] - HiringService --> JobOpening["JobOpening Model"] - JobOpening --> Response["JSON Response"] +flowchart TD + Request["GET /api/hiring/*"] --> HiringControllerNode["HiringController"] + HiringControllerNode --> HiringServiceNode["HiringService"] + HiringServiceNode --> ResponseMap["Map"] + ResponseMap --> Client["Frontend Client"] diff --git a/docs/diagrams/architecture/controllers.mmd b/docs/diagrams/architecture/controllers.mmd index aa3832d..970a8e8 100644 --- a/docs/diagrams/architecture/controllers.mmd +++ b/docs/diagrams/architecture/controllers.mmd @@ -1,37 +1,9 @@ flowchart TD - Client["Frontend (React App)"] -->|"HTTP REST"| Controllers["Controllers Module"] - - subgraph controllers_layer["Controllers"] - AutocompleteCtrl["Autocomplete Controller"] - ContributorCtrl["Contributor Controller"] - EntityCtrl["Entity Controller"] - HiringCtrl["Hiring Controller"] - end - - Controllers --> AutocompleteCtrl - Controllers --> ContributorCtrl - Controllers --> EntityCtrl - Controllers --> HiringCtrl - - AutocompleteCtrl -->|"delegates"| CityService["City Service"] - AutocompleteCtrl --> StateService["State Service"] - AutocompleteCtrl --> RegionService["Region Service"] - AutocompleteCtrl --> LanguageService["Language Service"] - AutocompleteCtrl --> SoccerTeamService["Soccer Team Service"] - - ContributorCtrl --> GithubService["GitHub Service"] - ContributorCtrl --> CacheService["Cache Service"] - ContributorCtrl --> CityService - ContributorCtrl --> LanguageService - - EntityCtrl --> CityService - EntityCtrl --> RegionService - EntityCtrl --> StateService - EntityCtrl --> LanguageService - EntityCtrl --> SoccerTeamService - - HiringCtrl --> HiringService["Hiring Service"] - - GithubService -->|"uses"| GraphQLLayer["GraphQL Components"] - GithubService -->|"rate limited by"| RateManager["GitHub Token Rate Manager"] - CacheService --> CacheImpl["Redis / Disk Cache"] + Client["Frontend Client"] -->|"HTTP Request"| ControllerLayer["Controllers"] + ControllerLayer -->|"delegates"| ServiceLayer["Backend Services"] + ServiceLayer -->|"reads/writes"| ModelLayer["Model Entities"] + ServiceLayer -->|"queries"| GraphQLLayer["GraphQL Components"] + ServiceLayer -->|"uses"| CacheLayer["Cache Services"] + ServiceLayer -->|"rate limits"| RateLayer["Rate Management"] + ControllerLayer -->|"wraps response"| ApiResponseNode["ApiResponse"] + ApiResponseNode --> Client diff --git a/docs/diagrams/architecture/core-application-2.mmd b/docs/diagrams/architecture/core-application-2.mmd deleted file mode 100644 index 40c0d95..0000000 --- a/docs/diagrams/architecture/core-application-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Client["Frontend Application"] -->|"HTTP Request"| Controller["Controller"] - Controller -->|"Invoke"| Service["Service Layer"] - Service -->|"Check Cache"| Cache["Cache Services"] - Service -->|"Build Query"| GraphQL["GraphQL Builder"] - GraphQL -->|"GitHub API Call"| GitHub["GitHub GraphQL API"] - GitHub -->|"Response"| Service - Service -->|"Map To"| Model["Model Entities"] - Service -->|"Return"| Controller - Controller -->|"JSON Response"| Client diff --git a/docs/diagrams/architecture/core-application-3.mmd b/docs/diagrams/architecture/core-application-3.mmd deleted file mode 100644 index 9ae3f2f..0000000 --- a/docs/diagrams/architecture/core-application-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Frontend["React Frontend"] -->|"REST API"| Backend["Core Application"] - Backend -->|"Cache"| Redis["Redis"] - Backend -->|"GraphQL"| GitHub["GitHub API"] - CacheUpdater["Cache Updater Service"] -->|"Warm Cache"| Redis diff --git a/docs/diagrams/architecture/core-application-4.mmd b/docs/diagrams/architecture/core-application-4.mmd deleted file mode 100644 index e4cfadd..0000000 --- a/docs/diagrams/architecture/core-application-4.mmd +++ /dev/null @@ -1,14 +0,0 @@ -sequenceDiagram - participant JVM as JVM - participant Spring as Spring Boot - participant Core as Core Application - participant Context as Application Context - - JVM->>Spring: Launch main() - Spring->>Core: Initialize - Core->>Context: Component Scan - Context->>Context: Register Beans - Context->>Context: Apply Configurations - Spring->>Spring: Enable Caching - Spring->>Spring: Enable Async - Spring-->>JVM: Application Ready diff --git a/docs/diagrams/architecture/core-application.mmd b/docs/diagrams/architecture/core-application.mmd deleted file mode 100644 index 37b3501..0000000 --- a/docs/diagrams/architecture/core-application.mmd +++ /dev/null @@ -1,18 +0,0 @@ -flowchart TD - CoreApp["Core Application"] - - Controllers["Controllers"] - Services["Service Layer"] - Cache["Cache Services"] - Config["Configurations"] - GraphQL["GraphQL Components"] - RateMgmt["Rate Management"] - Models["Model Entities"] - - CoreApp -->|"Component Scan"| Controllers - CoreApp -->|"Component Scan"| Services - CoreApp -->|"EnableCaching"| Cache - CoreApp -->|"AutoConfig"| Config - CoreApp -->|"GitHub Integration"| GraphQL - CoreApp -->|"Rate Limiting"| RateMgmt - CoreApp -->|"Domain Models"| Models diff --git a/docs/diagrams/architecture/frontend-components-2.mmd b/docs/diagrams/architecture/frontend-components-2.mmd index 73d3923..292f2b0 100644 --- a/docs/diagrams/architecture/frontend-components-2.mmd +++ b/docs/diagrams/architecture/frontend-components-2.mmd @@ -1,8 +1,6 @@ flowchart TD - Input["User Types"] --> InputChange["onInputChange"] - InputChange --> ParentState["Parent State Update"] - ParentState --> OptionsUpdate["Options Prop Updated"] - OptionsUpdate --> AutocompleteRender["Autocomplete Renders Options"] - - Select["User Selects Option"] --> OnChange["onChange Handler"] - OnChange --> ParentSelection["Parent Updates Selected Value"] + Input["User Types"] --> QueryState["inputValue Updated"] + QueryState --> Options["Options Provided"] + Options --> Select["User Selects Option"] + Select --> OnChange["onChange(value)"] + OnChange --> Sync["Input Normalized on Blur"] diff --git a/docs/diagrams/architecture/frontend-components-3.mmd b/docs/diagrams/architecture/frontend-components-3.mmd index 39cab27..6ed6dc8 100644 --- a/docs/diagrams/architecture/frontend-components-3.mmd +++ b/docs/diagrams/architecture/frontend-components-3.mmd @@ -1,7 +1,6 @@ flowchart TD - UserInput["User Types Language"] --> QueryKey["Query Key: ['languages', inputValue]"] - QueryKey --> ReactQuery["useQuery()"] + UserInput["User Types Language"] --> ReactQuery["useQuery"] ReactQuery --> ApiCall["autocompleteLanguages(inputValue)"] - ApiCall --> Backend["Backend Controller"] - Backend --> ApiResponse["Language[]"] - ApiResponse --> BaseAuto["BaseAutocomplete"] + ApiCall --> Backend["Backend Language Endpoint"] + Backend --> Response["Language[]"] + Response --> Autocomplete["BaseAutocomplete"] diff --git a/docs/diagrams/architecture/frontend-components-4.mmd b/docs/diagrams/architecture/frontend-components-4.mmd index 0852d92..8e02a12 100644 --- a/docs/diagrams/architecture/frontend-components-4.mmd +++ b/docs/diagrams/architecture/frontend-components-4.mmd @@ -1,6 +1,6 @@ -flowchart LR - ApiContributor["API Contributor"] --> TableProps["ContributorsTableProps"] - TableProps --> InfoProps["ContributorInfoProps"] - TableProps --> LocationProps["LocationInfoProps"] - TableProps --> StatsProps["StatsDisplayProps"] - TableProps --> TooltipProps["ContributorTooltipProps"] +flowchart TD + Contributor["Contributor API Model"] --> Info["ContributorInfoProps"] + Contributor --> Location["LocationInfoProps"] + Contributor --> Stats["StatsDisplayProps"] + Contributor --> Tooltip["ContributorTooltipProps"] + Contributor --> LocationTooltip["LocationTooltipProps"] diff --git a/docs/diagrams/architecture/frontend-components-5.mmd b/docs/diagrams/architecture/frontend-components-5.mmd index ce84fce..f75faca 100644 --- a/docs/diagrams/architecture/frontend-components-5.mmd +++ b/docs/diagrams/architecture/frontend-components-5.mmd @@ -1,6 +1,4 @@ -flowchart TD - Start["Render Pagination"] --> CheckPages{"totalPages <= 1?"} - CheckPages -->|Yes| Hide["Return null"] - CheckPages -->|No| ShowControls["Render Previous / Next"] - ShowControls --> PrevClick["onPageChange(currentPage - 1)"] - ShowControls --> NextClick["onPageChange(currentPage + 1)"] +flowchart LR + Prev["Previous Button"] --> PageState["currentPage"] + Next["Next Button"] --> PageState + PageState --> Render["Re-render with new page"] diff --git a/docs/diagrams/architecture/frontend-components-6.mmd b/docs/diagrams/architecture/frontend-components-6.mmd index a541c1d..2d3478a 100644 --- a/docs/diagrams/architecture/frontend-components-6.mmd +++ b/docs/diagrams/architecture/frontend-components-6.mmd @@ -1,16 +1,7 @@ -sequenceDiagram - participant User - participant UI as "LanguageAutocomplete" - participant Query as "React Query" - participant API as "Frontend API Service" - participant Backend - - User->>UI: Type "Java" - UI->>Query: Trigger query with inputValue - Query->>API: autocompleteLanguages("Java") - API->>Backend: HTTP Request - Backend->>API: Return Language[] - API->>Query: Resolve Promise - Query->>UI: Provide options - User->>UI: Select Language - UI->>User: Filter applied in table +flowchart TD + Page["Leaderboard Page"] --> LanguageAuto["LanguageAutocomplete"] + LanguageAuto --> BaseAuto["BaseAutocomplete"] + Page --> Table["Contributors Table"] + Page --> PaginationComp["Pagination"] + LanguageAuto --> Services["Frontend Services"] + Table --> Types["Frontend Types"] diff --git a/docs/diagrams/architecture/frontend-components.mmd b/docs/diagrams/architecture/frontend-components.mmd index 55f2451..73ebe0e 100644 --- a/docs/diagrams/architecture/frontend-components.mmd +++ b/docs/diagrams/architecture/frontend-components.mmd @@ -1,10 +1,6 @@ flowchart TD - User["User Interaction"] --> Page["Page Component"] - Page --> DomainComponent["Domain Component
LanguageAutocomplete"] - DomainComponent --> BaseComponent["BaseAutocomplete"] - Page --> TableTypes["ContributorsTable Types"] - Page --> PaginationComp["Pagination"] - - DomainComponent --> ReactQuery["React Query"] - ReactQuery --> ApiService["Frontend Services"] - ApiService --> Backend["Backend API"] + User["User"] --> UI["React Pages"] + UI --> Components["Frontend Components"] + Components --> Hooks["Frontend Hooks"] + Components --> Services["Frontend Services"] + Services --> Backend["Backend API"] diff --git a/docs/diagrams/architecture/frontend-hooks-2.mmd b/docs/diagrams/architecture/frontend-hooks-2.mmd index 45e2f42..958780b 100644 --- a/docs/diagrams/architecture/frontend-hooks-2.mmd +++ b/docs/diagrams/architecture/frontend-hooks-2.mmd @@ -1,7 +1,10 @@ flowchart TD - Start["Component Mount"] --> ReadParams["Read searchParams"] - ReadParams --> Parse["parseUrlValue()"] - Parse --> Validate["validateValue()"] - Validate --> BuildState["Construct UrlState"] - BuildState --> Memoize["useMemo"] - Memoize --> ReturnState["Return Hook API"] + Start["Hook Initialized"] --> CheckRegions{"Regions Provided?"} + CheckRegions -->|"No"| EndA["Return null"] + CheckRegions -->|"Yes"| CheckGeo{"Geolocation Supported?"} + CheckGeo -->|"No"| ErrorA["Set Error: Not Supported"] + CheckGeo -->|"Yes"| GetPosition["navigator.geolocation.getCurrentPosition()"] + GetPosition --> Calc["Calculate Distance to Each Region"] + Calc --> FindMin["Find Minimum Distance"] + FindMin --> SetRegion["Set nearestRegion"] + SetRegion --> EndB["Return { nearestRegion, error }"] diff --git a/docs/diagrams/architecture/frontend-hooks-3.mmd b/docs/diagrams/architecture/frontend-hooks-3.mmd index 00fe121..4089a35 100644 --- a/docs/diagrams/architecture/frontend-hooks-3.mmd +++ b/docs/diagrams/architecture/frontend-hooks-3.mmd @@ -1,9 +1,5 @@ flowchart TD - UpdateCall["updateUrlState(newState)"] --> Compare["Compare with current params"] - Compare --> HasChanges{"Changes?"} - HasChanges -->|"No"| Exit["Skip Update"] - HasChanges -->|"Yes"| DebounceCheck{"Debounce?"} - DebounceCheck -->|"Immediate"| Apply["setSearchParams()"] - DebounceCheck -->|"Delayed"| Timeout["setTimeout()"] - Timeout --> Apply - Apply --> EndNode["URL Updated"] + URL["URLSearchParams"] --> Parse["parseUrlValue()"] + Parse --> Validate{"Valid?"} + Validate -->|"Yes"| State["Populate UrlState"] + Validate -->|"No"| Default["Use defaultValue"] diff --git a/docs/diagrams/architecture/frontend-hooks-4.mmd b/docs/diagrams/architecture/frontend-hooks-4.mmd index 197bf0a..08c2cb9 100644 --- a/docs/diagrams/architecture/frontend-hooks-4.mmd +++ b/docs/diagrams/architecture/frontend-hooks-4.mmd @@ -1,9 +1,10 @@ -flowchart TD - Start["Regions Provided"] --> GeoCheck{"Geolocation Supported?"} - GeoCheck -->|"No"| ErrorNode["Set Error"] - GeoCheck -->|"Yes"| GetPos["getCurrentPosition()"] - GetPos --> Loop["Iterate Regions"] - Loop --> Compute["getDistance()"] - Compute --> Compare["Track Minimum Distance"] - Compare --> Select["Select Nearest Region"] - Select --> ReturnNode["Return nearestRegion"] +sequenceDiagram + participant UI + participant Hook as "useUrlState" + participant Router as "React Router" + + UI->>Hook: updateUrlState(newState) + Hook->>Hook: debounce if configured + Hook->>Router: setSearchParams() + Router->>Hook: searchParams updated + Hook->>UI: new urlState diff --git a/docs/diagrams/architecture/frontend-hooks-5.mmd b/docs/diagrams/architecture/frontend-hooks-5.mmd index fb7c45f..1c4161e 100644 --- a/docs/diagrams/architecture/frontend-hooks-5.mmd +++ b/docs/diagrams/architecture/frontend-hooks-5.mmd @@ -1,11 +1,4 @@ -sequenceDiagram - participant User - participant Component as "Filter Component" - participant Hook as "useUrlState" - participant Service as "API Service" - - User->>Component: Select language - Component->>Hook: updateUrlState({ languageId }) - Hook->>Component: Updated urlState - Component->>Service: Fetch contributors with filters - Service->>Component: Return filtered data +flowchart LR + URL["URL Params"] --> State["urlState Object"] + State --> Update["updateUrlState()"] + Update --> URL diff --git a/docs/diagrams/architecture/frontend-hooks-6.mmd b/docs/diagrams/architecture/frontend-hooks-6.mmd new file mode 100644 index 0000000..f362860 --- /dev/null +++ b/docs/diagrams/architecture/frontend-hooks-6.mmd @@ -0,0 +1,6 @@ +flowchart TD + Hooks["Frontend Hooks"] --> Components["Frontend Components"] + Components --> Services["Frontend Services"] + Services --> Backend["Backend Services"] + + Hooks --> Types["Frontend Types"] diff --git a/docs/diagrams/architecture/frontend-hooks.mmd b/docs/diagrams/architecture/frontend-hooks.mmd index 64df2b3..099adfc 100644 --- a/docs/diagrams/architecture/frontend-hooks.mmd +++ b/docs/diagrams/architecture/frontend-hooks.mmd @@ -1,11 +1,6 @@ flowchart TD - Browser["Browser Environment"] -->|"query params"| Router["React Router"] - Browser -->|"Geolocation API"| NearestRegionHook["useNearestRegion Hook"] - - Router --> UrlStateHook["useUrlState Hook"] - - UrlStateHook --> Components["Frontend Components"] - NearestRegionHook --> Components - - Components --> Services["Frontend Services"] - Services --> Backend["Backend Service API"] + UI["Frontend Components"] -->|"uses"| Hooks["Frontend Hooks"] + Hooks -->|"reads/writes"| Router["React Router Search Params"] + Hooks -->|"consumes"| Types["Frontend Types"] + Hooks -->|"drives filters"| Services["Frontend Services"] + Services -->|"calls"| Backend["Backend API"] diff --git a/docs/diagrams/architecture/frontend-services-2.mmd b/docs/diagrams/architecture/frontend-services-2.mmd index ce3f377..0e68ace 100644 --- a/docs/diagrams/architecture/frontend-services-2.mmd +++ b/docs/diagrams/architecture/frontend-services-2.mmd @@ -1,12 +1,11 @@ sequenceDiagram - participant UI as React Component - participant Hook as Custom Hook + participant UI as React UI participant Service as Frontend Services - participant Backend as Backend Controller + participant API as Backend API - UI->>Hook: Trigger search - Hook->>Service: getContributors(params) - Service->>Backend: GET /api/contributors/search - Backend-->>Service: ApiResponse - Service-->>Hook: Contributor[] - Hook-->>UI: Render table + UI->>Service: getContributors(filters) + Service->>Service: Build URLSearchParams + Service->>API: GET /api/contributors/search + API-->>Service: ApiResponse + Service->>Service: Validate status === "success" + Service-->>UI: Contributor[] diff --git a/docs/diagrams/architecture/frontend-services-3.mmd b/docs/diagrams/architecture/frontend-services-3.mmd index d41e106..af1950b 100644 --- a/docs/diagrams/architecture/frontend-services-3.mmd +++ b/docs/diagrams/architecture/frontend-services-3.mmd @@ -1,5 +1,6 @@ flowchart TD - BuildParams["Build Query Parameters"] --> CreateLink["Create Hidden Anchor Element"] - CreateLink --> SetHref["Set export URL"] - SetHref --> ClickLink["Trigger click()"] - ClickLink --> Download["Browser Downloads CSV"] + A["User Clicks Export"] --> B["Build Query Parameters"] + B --> C["Create Hidden Anchor Element"] + C --> D["Set href to Export Endpoint"] + D --> E["Trigger click()"] + E --> F["Browser Downloads contributors.csv"] diff --git a/docs/diagrams/architecture/frontend-services-4.mmd b/docs/diagrams/architecture/frontend-services-4.mmd index 81340a3..b555c97 100644 --- a/docs/diagrams/architecture/frontend-services-4.mmd +++ b/docs/diagrams/architecture/frontend-services-4.mmd @@ -1,5 +1,6 @@ flowchart LR - HiringPage["Hiring Page"] --> HiringService["Frontend Services"] - HiringService --> BackendHiring["/api/hiring/* Endpoints"] - BackendHiring --> HiringService - HiringService --> HiringPage + QueryInput["User Types"] --> Debounce["Debounced Hook"] + Debounce --> ServiceCall["autocompleteX()"] + ServiceCall --> BackendCall["GET /api/autocomplete/*"] + BackendCall --> Response["ApiResponse"] + Response --> FilteredList["Return T[]"] diff --git a/docs/diagrams/architecture/frontend-services-5.mmd b/docs/diagrams/architecture/frontend-services-5.mmd index 502aa2d..23b566a 100644 --- a/docs/diagrams/architecture/frontend-services-5.mmd +++ b/docs/diagrams/architecture/frontend-services-5.mmd @@ -1,5 +1,6 @@ flowchart TD - Services["Frontend Services"] --> Types["Frontend Types"] - Services --> Hooks["Frontend Hooks"] - Services --> Components["Frontend Components"] - Services --> BackendControllers["Backend Controllers"] + A["getEntityById(id)"] --> B["GET /api/entities/{type}/{id}"] + B --> C["ApiResponse"] + C --> D{"status success?"} + D -->|"Yes"| E["Return data"] + D -->|"No"| F["Throw Error"] diff --git a/docs/diagrams/architecture/frontend-services-6.mmd b/docs/diagrams/architecture/frontend-services-6.mmd new file mode 100644 index 0000000..9873b9d --- /dev/null +++ b/docs/diagrams/architecture/frontend-services-6.mmd @@ -0,0 +1,5 @@ +flowchart LR + Services["Frontend Services"] --> ApiResponseType["ApiResponse"] + Services --> ContributorType["Contributor"] + Services --> GeoTypes["City / Region / State"] + Services --> HiringTypes["HiringManagerProfile / JobOpening"] diff --git a/docs/diagrams/architecture/frontend-services.mmd b/docs/diagrams/architecture/frontend-services.mmd index d8bcd84..839dfe5 100644 --- a/docs/diagrams/architecture/frontend-services.mmd +++ b/docs/diagrams/architecture/frontend-services.mmd @@ -1,19 +1,5 @@ flowchart LR - subgraph UI["Frontend UI Layer"] - Components["React Components"] - Hooks["Custom Hooks"] - end - - subgraph Services["Frontend Services"] - ApiModule["api.ts"] - AxiosConfig["Axios Configuration"] - end - - subgraph Backend["Backend Service"] - Controllers["REST Controllers"] - end - - Components -->|"calls"| Hooks - Hooks -->|"invokes"| ApiModule - ApiModule -->|"uses"| AxiosConfig - ApiModule -->|"HTTP GET /api/..."| Controllers + UI["React Components"] --> Hooks["Custom Hooks"] + Hooks --> Services["Frontend Services"] + Services --> Axios["Axios HTTP Client"] + Axios --> Backend["Spring Boot Backend API"] diff --git a/docs/diagrams/architecture/frontend-types-2.mmd b/docs/diagrams/architecture/frontend-types-2.mmd index 294d5d0..cc8faef 100644 --- a/docs/diagrams/architecture/frontend-types-2.mmd +++ b/docs/diagrams/architecture/frontend-types-2.mmd @@ -1,8 +1,17 @@ -flowchart TD - Root["Frontend Types"] +flowchart LR + ApiTypes["api.ts"] --> ContributorType["Contributor (API)"] + ApiTypes --> CityType["City"] + ApiTypes --> RegionType["Region"] + ApiTypes --> StateType["State"] + ApiTypes --> TeamType["SoccerTeam"] + ApiTypes --> LanguageType["Language"] - Root --> Api["API Types"] - Root --> Contributor["Contributor Projection Types"] - Root --> Enhanced["Enhanced Relational Types"] - Root --> Hiring["Hiring Domain Types"] - Root --> HiringIndex["Hiring Re-exports (Index)"] + ContributorDomain["contributor.ts"] --> ContributorUI["Contributor (UI)"] + + EnhancedTypes["enhanced.ts"] --> EnhancedCity["EnhancedCity"] + EnhancedTypes --> EnhancedRegion["EnhancedRegion"] + EnhancedTypes --> EnhancedState["EnhancedState"] + + HiringTypes["hiring.ts"] --> HiringProfile["HiringManagerProfile"] + HiringTypes --> JobOpeningType["JobOpening"] + HiringTypes --> SocialLinkType["SocialLink"] diff --git a/docs/diagrams/architecture/frontend-types-3.mmd b/docs/diagrams/architecture/frontend-types-3.mmd index 225cb1e..f41104c 100644 --- a/docs/diagrams/architecture/frontend-types-3.mmd +++ b/docs/diagrams/architecture/frontend-types-3.mmd @@ -1,7 +1,3 @@ flowchart TD - Contributor["Contributor (API)"] - - Contributor -->|"belongs to"| City["City"] - Contributor -->|"nearest"| Team["SoccerTeam"] - Contributor -->|"contains"| Stats["githubStats"] - Contributor -->|"contains"| Social["SocialLink[]"] + ApiContributor["Contributor (API)"] --> Transform["Transform / Map"] + Transform --> UIContributor["Contributor (UI)"] diff --git a/docs/diagrams/architecture/frontend-types-4.mmd b/docs/diagrams/architecture/frontend-types-4.mmd index b82d4cd..1c0c46f 100644 --- a/docs/diagrams/architecture/frontend-types-4.mmd +++ b/docs/diagrams/architecture/frontend-types-4.mmd @@ -1,2 +1,4 @@ flowchart LR - ApiContributor["Contributor (API)"] -->|"transform"| UiContributor["Contributor (UI)"] + RawRegion["Region (IDs)"] --> EnhanceRegion["EnhancedRegion (Objects)"] + RawState["State (IDs)"] --> EnhanceState["EnhancedState (Objects)"] + RawCity["City (Optional refs)"] --> EnhanceCity["EnhancedCity (Resolved refs)"] diff --git a/docs/diagrams/architecture/frontend-types-5.mmd b/docs/diagrams/architecture/frontend-types-5.mmd index 959a5b8..fb86753 100644 --- a/docs/diagrams/architecture/frontend-types-5.mmd +++ b/docs/diagrams/architecture/frontend-types-5.mmd @@ -1,4 +1,4 @@ flowchart TD - Region["EnhancedRegion"] --> State["EnhancedState"] - State --> City["EnhancedCity"] - City --> Team["SoccerTeam"] + HiringManager["HiringManagerProfile"] --> Stats["GitHub Stats"] + HiringManager --> Links["Social Links"] + HiringManager --> Activity["Last Active"] diff --git a/docs/diagrams/architecture/frontend-types-6.mmd b/docs/diagrams/architecture/frontend-types-6.mmd index 45cb530..5fc0694 100644 --- a/docs/diagrams/architecture/frontend-types-6.mmd +++ b/docs/diagrams/architecture/frontend-types-6.mmd @@ -1,4 +1,7 @@ flowchart TD - HiringManager["HiringManagerProfile"] --> SocialLinks["SocialLink[]"] - HiringManager --> Stats["githubStats"] - HiringManager --> Jobs["JobOpening"] + API["Backend API"] --> ApiModels["API Types"] + ApiModels --> Services["Service Layer"] + Services --> UIModels["UI Contributor Type"] + Services --> EnhancedModels["Enhanced Geographic Types"] + UIModels --> Components["Leaderboard Components"] + EnhancedModels --> Filters["Filtering & Proximity Logic"] diff --git a/docs/diagrams/architecture/frontend-types-7.mmd b/docs/diagrams/architecture/frontend-types-7.mmd deleted file mode 100644 index 90ce54a..0000000 --- a/docs/diagrams/architecture/frontend-types-7.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Backend["Backend Services"] --> ApiResponse["ApiResponse"] - ApiResponse --> ApiModels["API Domain Models"] - ApiModels --> Transform["Transformation Layer"] - Transform --> UiModels["UI Contributor Type"] - ApiModels --> EnhancedModels["Enhanced Geographic Types"] - UiModels --> Components["Leaderboard UI"] - EnhancedModels --> Hooks["Geolocation Hooks"] diff --git a/docs/diagrams/architecture/frontend-types.mmd b/docs/diagrams/architecture/frontend-types.mmd index 36205b1..1ccef69 100644 --- a/docs/diagrams/architecture/frontend-types.mmd +++ b/docs/diagrams/architecture/frontend-types.mmd @@ -1,5 +1,7 @@ -flowchart LR - Backend["Spring Boot Backend"] -->|"JSON over HTTP"| ApiLayer["Frontend API Services"] - ApiLayer -->|"Typed responses"| Types["Frontend Types"] - Types -->|"Strongly typed models"| Components["React Components"] - Types -->|"Shared interfaces"| Hooks["Custom Hooks"] +flowchart TD + Backend["Backend REST API"] -->|"JSON"| ApiResponseType["ApiResponse"] + ApiResponseType --> ApiModels["API Domain Models"] + ApiModels --> Services["Frontend Services"] + Services --> Hooks["Custom Hooks"] + Hooks --> EnhancedModels["Enhanced Models"] + EnhancedModels --> Components["React Components"] diff --git a/docs/diagrams/architecture/graphql-components-2.mmd b/docs/diagrams/architecture/graphql-components-2.mmd index b7d6036..1ffed50 100644 --- a/docs/diagrams/architecture/graphql-components-2.mmd +++ b/docs/diagrams/architecture/graphql-components-2.mmd @@ -1,6 +1,7 @@ flowchart TD - GitHubQueryBuilder --> SearchFieldInner["SearchField (Inner Class)"] - SearchFieldInner --> DefaultFields["Default User Fields"] - DefaultFields --> Contributions["Contribution Data"] - DefaultFields --> Repositories["Repositories & Stars"] - DefaultFields --> SocialAccounts["Social Accounts"] + Root["Field: search"] --> Args["Arguments"] + Root --> Nodes["Subfields"] + Nodes --> User["... on User"] + User --> Login["login"] + User --> Location["location"] + User --> Repositories["repositories"] diff --git a/docs/diagrams/architecture/graphql-components-3.mmd b/docs/diagrams/architecture/graphql-components-3.mmd index 8e5cd95..188f816 100644 --- a/docs/diagrams/architecture/graphql-components-3.mmd +++ b/docs/diagrams/architecture/graphql-components-3.mmd @@ -1,8 +1,12 @@ -flowchart TD - Start["serialize(fields)"] --> OpenQuery["append 'query {'"] - OpenQuery --> Iterate["iterate fields"] - Iterate --> SerializeField["serializeField()"] - SerializeField --> SerializeArgs["serializeArguments()"] - SerializeField --> SerializeChildren["process subfields"] - SerializeChildren --> CloseBlock["append '}'"] - CloseBlock --> End["return string"] +sequenceDiagram + participant Service as Backend Service + participant Builder as GitHubQueryBuilder + participant API as GitHub GraphQL API + + Service->>Builder: searchUsers(size) + Service->>Builder: location(city) + Service->>Builder: language(lang) + Service->>Builder: cursor(after) + Service->>Builder: build() + Builder-->>Service: query string + Service->>API: Execute GraphQL query diff --git a/docs/diagrams/architecture/graphql-components-4.mmd b/docs/diagrams/architecture/graphql-components-4.mmd index 08eea08..6a331e4 100644 --- a/docs/diagrams/architecture/graphql-components-4.mmd +++ b/docs/diagrams/architecture/graphql-components-4.mmd @@ -1,7 +1,6 @@ -flowchart LR - Controller["Controller Layer"] --> Service["GithubService"] - Service --> Builder["GitHubQueryBuilder"] - Builder --> Query["GraphQL Query String"] - Query --> GitHubAPI["GitHub GraphQL API"] - GitHubAPI --> Response["JSON Response"] - Response --> Service +flowchart TD + Fields["List of Field"] --> SerializeFields["serializeFields()"] + SerializeFields --> SerializeField["serializeField()"] + SerializeField --> SerializeArgs["serializeArguments()"] + SerializeField --> Recurse["Serialize Subfields"] + Recurse --> SerializeField diff --git a/docs/diagrams/architecture/graphql-components-5.mmd b/docs/diagrams/architecture/graphql-components-5.mmd new file mode 100644 index 0000000..7c92623 --- /dev/null +++ b/docs/diagrams/architecture/graphql-components-5.mmd @@ -0,0 +1,7 @@ +flowchart LR + Controller["Controller"] --> Service["GithubService"] + Service --> Builder["GitHubQueryBuilder"] + Builder --> Query["GraphQL Query String"] + Query --> GitHubAPI["GitHub API"] + GitHubAPI --> Response["JSON Response"] + Response --> Model["Model Entities"] diff --git a/docs/diagrams/architecture/graphql-components.mmd b/docs/diagrams/architecture/graphql-components.mmd index c5be731..69b97cb 100644 --- a/docs/diagrams/architecture/graphql-components.mmd +++ b/docs/diagrams/architecture/graphql-components.mmd @@ -1,7 +1,8 @@ flowchart TD - ServiceLayer["Service Layer"] -->|"builds query"| GitHubQueryBuilder["GitHubQueryBuilder"] - GitHubQueryBuilder -->|"uses"| SearchFieldBuilder["SearchField (Builder)"] - GitHubQueryBuilder -->|"composes"| InnerField["Field (Inner Class)"] - QuerySerializer["QuerySerializer"] -->|"serializes"| FieldModel["Field (Model)"] - GitHubQueryBuilder -->|"produces"| QueryString["GraphQL Query String"] - QuerySerializer -->|"produces"| QueryString + BackendService["Backend Services"] -->|"build query"| GitHubQueryBuilder["GitHubQueryBuilder"] + GitHubQueryBuilder -->|"composes"| SearchFieldBuilder["SearchField (Builder)"] + GitHubQueryBuilder -->|"returns string"| GraphQLQuery["GraphQL Query String"] + FieldCore["Field (Core Model)"] --> QuerySerializer["QuerySerializer"] + QuerySerializer --> SerializedQuery["Formatted GraphQL Query"] + GraphQLQuery --> GitHubAPI["GitHub GraphQL API"] + SerializedQuery --> GitHubAPI diff --git a/docs/diagrams/architecture/model-entities-2.mmd b/docs/diagrams/architecture/model-entities-2.mmd index 33c59e2..0c201d9 100644 --- a/docs/diagrams/architecture/model-entities-2.mmd +++ b/docs/diagrams/architecture/model-entities-2.mmd @@ -1,4 +1,9 @@ -flowchart TD - Region["Region"] -->|"contains"| State["State"] - State -->|"contains"| City["City"] - City -->|"near"| SoccerTeam["SoccerTeam"] +sequenceDiagram + participant Client + participant Controller + participant Service + + Client->>Controller: HTTP Request + Controller->>Service: Execute logic + Service->>Controller: Domain Model + Controller->>Client: ApiResponse diff --git a/docs/diagrams/architecture/model-entities-3.mmd b/docs/diagrams/architecture/model-entities-3.mmd index dbad8c8..ba4c22c 100644 --- a/docs/diagrams/architecture/model-entities-3.mmd +++ b/docs/diagrams/architecture/model-entities-3.mmd @@ -1,9 +1,4 @@ -flowchart LR - Contributor["Contributor"] -->|"located in"| City["City"] - City -->|"belongs to"| State["State"] - State -->|"part of"| Region["Region"] - City -->|"nearest"| SoccerTeam["SoccerTeam"] - Contributor -->|"links"| SocialLink["SocialLink"] - Contributor -->|"stats"| Stats["GitHub Stats"] - HiringProfile["HiringManagerProfile"] -->|"links"| SocialLink - HiringProfile -->|"stats"| Stats +flowchart TD + Contributor["Contributor"] --> RoleCheck{{"Role?"}} + RoleCheck -->|"CONTRIBUTOR"| BuildMap["Build stats map from fields"] + RoleCheck -->|"HIRING_MANAGER"| UseStored["Use githubStats field"] diff --git a/docs/diagrams/architecture/model-entities-4.mmd b/docs/diagrams/architecture/model-entities-4.mmd new file mode 100644 index 0000000..651ac4c --- /dev/null +++ b/docs/diagrams/architecture/model-entities-4.mmd @@ -0,0 +1,4 @@ +flowchart TD + Region["Region"] --> State["State"] + State --> City["City"] + City --> Contributor["Contributor"] diff --git a/docs/diagrams/architecture/model-entities-5.mmd b/docs/diagrams/architecture/model-entities-5.mmd new file mode 100644 index 0000000..06590fc --- /dev/null +++ b/docs/diagrams/architecture/model-entities-5.mmd @@ -0,0 +1,4 @@ +flowchart TD + Contributor["Contributor"] --> City["City"] + City --> TeamLookup["Find nearest team"] + TeamLookup --> SoccerTeam["SoccerTeam"] diff --git a/docs/diagrams/architecture/model-entities-6.mmd b/docs/diagrams/architecture/model-entities-6.mmd new file mode 100644 index 0000000..1a7f023 --- /dev/null +++ b/docs/diagrams/architecture/model-entities-6.mmd @@ -0,0 +1,7 @@ +flowchart LR + GitHubService["GitHub Service"] --> Contributor + CityService["City Service"] --> City + RegionService["Region Service"] --> Region + SoccerTeamService["Soccer Team Service"] --> SoccerTeam + HiringService["Hiring Service"] --> HiringManagerProfile + Controllers["Controllers"] --> ApiResponse diff --git a/docs/diagrams/architecture/model-entities.mmd b/docs/diagrams/architecture/model-entities.mmd index e10b7c1..2197084 100644 --- a/docs/diagrams/architecture/model-entities.mmd +++ b/docs/diagrams/architecture/model-entities.mmd @@ -1,10 +1,7 @@ flowchart TD - Controllers["REST Controllers"] -->|"return"| ApiResponse["ApiResponse"] - Controllers -->|"use"| Services["Service Layer"] - Services -->|"construct"| Contributor["Contributor"] - Services -->|"construct"| Geography["City / State / Region"] - Services -->|"construct"| SoccerTeam["SoccerTeam"] - Services -->|"construct"| Language["Language"] - Services -->|"construct"| Hiring["HiringManagerProfile / JobOpening"] - Contributor -->|"references"| Geography - Geography -->|"links to"| SoccerTeam + Controllers["Controllers"] -->|"return ApiResponse"| ApiResponse["ApiResponse"] + Controllers --> Services["Backend Services"] + Services --> Models["Model Entities"] + Services --> Cache["Cache Services"] + Cache --> Models + Models --> Frontend["Frontend (TypeScript Types)"] diff --git a/docs/diagrams/architecture/rate-management-2.mmd b/docs/diagrams/architecture/rate-management-2.mmd index 7c99eb5..6d0c32c 100644 --- a/docs/diagrams/architecture/rate-management-2.mmd +++ b/docs/diagrams/architecture/rate-management-2.mmd @@ -1,5 +1,10 @@ -flowchart TD - CheckSecondary["isUnderSecondaryLimit()"] --> RetryCheck{"retryAfterSeconds and lastSecondaryLimitHit set?"} - RetryCheck -->|"No"| NotLimited["Return false"] - RetryCheck -->|"Yes"| TimeCheck["elapsedSeconds < retryAfterSeconds"] - TimeCheck --> Result["Return true or false"] +sequenceDiagram + participant App as Application + participant Manager as GithubTokenRateManager + participant GitHub as GitHub API + + App->>Manager: init() + Manager->>Manager: Build WebClient per token + Manager->>GitHub: GET /rate_limit (per token) + GitHub-->>Manager: Rate headers + Manager->>Manager: updateTokenRateLimits() diff --git a/docs/diagrams/architecture/rate-management-3.mmd b/docs/diagrams/architecture/rate-management-3.mmd index a1b0324..5e609d7 100644 --- a/docs/diagrams/architecture/rate-management-3.mmd +++ b/docs/diagrams/architecture/rate-management-3.mmd @@ -1,11 +1,10 @@ -sequenceDiagram - participant Spring - participant Manager as "GithubTokenRateManager" - participant GitHub - - Spring->>Manager: PostConstruct init() - Manager->>Manager: Create WebClient per token - Spring->>Manager: initializeRateLimits() - Manager->>GitHub: GET /rate_limit per token - GitHub-->>Manager: Rate headers - Manager->>Manager: updateTokenRateLimits() +flowchart TD + Start["Request Client"] --> Evaluate["Evaluate All Tokens"] + Evaluate --> SecondaryCheck{"All Under Secondary?"} + SecondaryCheck -->|"Yes"| WaitSecondary["Sleep Until Earliest Secondary Reset"] + SecondaryCheck -->|"No"| PrimaryCheck{"All Exhausted?"} + PrimaryCheck -->|"Yes"| WaitPrimary["Sleep Until Earliest Reset"] + PrimaryCheck -->|"No"| Select["Select Highest Remaining Token"] + WaitSecondary --> Select + WaitPrimary --> Select + Select --> End["Return WebClient + GithubToken"] diff --git a/docs/diagrams/architecture/rate-management-4.mmd b/docs/diagrams/architecture/rate-management-4.mmd deleted file mode 100644 index 67921c2..0000000 --- a/docs/diagrams/architecture/rate-management-4.mmd +++ /dev/null @@ -1,15 +0,0 @@ -flowchart TD - Start["Request Client"] --> Evaluate["Iterate All Tokens"] - - Evaluate --> SecondaryCheck{"Under Secondary Limit?"} - SecondaryCheck -->|"Yes"| SkipSecondary["Track earliest secondary reset"] - SecondaryCheck -->|"No"| PrimaryCheck{"Has rate info?"} - - PrimaryCheck -->|"No"| SkipToken["Skip token"] - PrimaryCheck -->|"Yes"| Compare["Compare remaining and reset time"] - - Compare --> Select["Track best token"] - Select --> Exhausted{"All exhausted?"} - - Exhausted -->|"Yes"| WaitPrimary["Sleep until earliest reset"] - Exhausted -->|"No"| ReturnBest["Return best client"] diff --git a/docs/diagrams/architecture/rate-management-5.mmd b/docs/diagrams/architecture/rate-management-5.mmd deleted file mode 100644 index 8600e27..0000000 --- a/docs/diagrams/architecture/rate-management-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Primary["Primary Limit"] -->|"Hard quota"| ResetTime["Reset Timestamp"] - Secondary["Secondary Limit"] -->|"Burst protection"| RetryAfter["Retry-After seconds"] diff --git a/docs/diagrams/architecture/rate-management-6.mmd b/docs/diagrams/architecture/rate-management-6.mmd deleted file mode 100644 index 5c1acec..0000000 --- a/docs/diagrams/architecture/rate-management-6.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Response["GitHub Response"] --> Headers["Extract Headers"] - Headers --> UpdatePrimary["Update primary fields"] - Headers --> UpdateSecondary["Update Retry-After"] - UpdatePrimary --> Store["Mutate GithubToken"] - UpdateSecondary --> Store diff --git a/docs/diagrams/architecture/rate-management.mmd b/docs/diagrams/architecture/rate-management.mmd index 08a8bb1..f81dcde 100644 --- a/docs/diagrams/architecture/rate-management.mmd +++ b/docs/diagrams/architecture/rate-management.mmd @@ -1,7 +1,6 @@ -flowchart TD - Controllers["Controllers"] --> Services["Service Layer"] - Services --> RateManager["GithubTokenRateManager"] - RateManager --> WebClient["WebClient per Token"] - WebClient --> GitHubAPI["GitHub API"] - - RateManager --> TokenState["GithubToken State"] +flowchart LR + Service["Backend Service (e.g. GithubService)"] -->|"requests WebClient"| RateManager["GithubTokenRateManager"] + RateManager -->|"selects best token"| Token["GithubToken"] + RateManager -->|"uses WebClient"| GitHubAPI["GitHub API"] + GitHubAPI -->|"response headers"| RateManager + RateManager -->|"updateTokenRateLimits()"| Token diff --git a/docs/diagrams/architecture/service-layer-2.mmd b/docs/diagrams/architecture/service-layer-2.mmd deleted file mode 100644 index 122dabf..0000000 --- a/docs/diagrams/architecture/service-layer-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - CitiesCSV["cities.csv"] --> CityService - CityService --> StateService - CityService --> SoccerTeamService - CityService --> CityModel["City Model"] diff --git a/docs/diagrams/architecture/service-layer-3.mmd b/docs/diagrams/architecture/service-layer-3.mmd deleted file mode 100644 index 32f0f98..0000000 --- a/docs/diagrams/architecture/service-layer-3.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart TD - City["City"] -->|"latitude/longitude"| DistanceCalc["Distance Calculation"] - DistanceCalc --> Team["Nearest Soccer Team"] diff --git a/docs/diagrams/architecture/service-layer-4.mmd b/docs/diagrams/architecture/service-layer-4.mmd deleted file mode 100644 index 86fbb7c..0000000 --- a/docs/diagrams/architecture/service-layer-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - RegionService --> ReferencePopulationService - StateService --> ReferencePopulationService - CityService --> ReferencePopulationService - ReferencePopulationService --> UpdatedRegions["Enriched Regions"] diff --git a/docs/diagrams/architecture/service-layer-5.mmd b/docs/diagrams/architecture/service-layer-5.mmd deleted file mode 100644 index 09d3fa0..0000000 --- a/docs/diagrams/architecture/service-layer-5.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Request["Contributor Request"] --> TargetCities["Resolve Target Cities"] - TargetCities --> Batch["Batch by Concurrency"] - Batch --> AsyncCalls["Async GitHub Calls"] - AsyncCalls --> ProcessUsers["Process & Score Users"] - ProcessUsers --> Merge["Merge & Deduplicate"] - Merge --> Sorted["Sort by Score"] diff --git a/docs/diagrams/architecture/service-layer-6.mmd b/docs/diagrams/architecture/service-layer-6.mmd deleted file mode 100644 index 7edac62..0000000 --- a/docs/diagrams/architecture/service-layer-6.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Filters["Filters Provided"] --> CityFilter - Filters --> StateFilter - Filters --> RegionFilter - Filters --> TeamFilter - CityFilter --> Intersect - StateFilter --> Intersect - RegionFilter --> Intersect - TeamFilter --> Intersect - Intersect --> FinalCities["Final Target Cities"] diff --git a/docs/diagrams/architecture/service-layer-7.mmd b/docs/diagrams/architecture/service-layer-7.mmd deleted file mode 100644 index 4b01c12..0000000 --- a/docs/diagrams/architecture/service-layer-7.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - CacheCheck["Check Cache"] -->|"miss"| GithubProfile["Fetch GitHub Profile"] - GithubProfile --> BuildProfile["Build HiringManagerProfile"] - BuildProfile --> StoreCache["Store in Cache"] - StoreCache --> ReturnProfile["Return Response"] diff --git a/docs/diagrams/architecture/service-layer-8.mmd b/docs/diagrams/architecture/service-layer-8.mmd deleted file mode 100644 index 44160bf..0000000 --- a/docs/diagrams/architecture/service-layer-8.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> PreCacheService - PreCacheService --> ForEachLanguage["Iterate Languages"] - ForEachLanguage --> ContributorController - ContributorController --> GithubService - GithubService --> CacheFilled["Cache Filled"] diff --git a/docs/diagrams/architecture/service-layer-9.mmd b/docs/diagrams/architecture/service-layer-9.mmd deleted file mode 100644 index 3dfbfa2..0000000 --- a/docs/diagrams/architecture/service-layer-9.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart LR - GithubService --> CityService - GithubService --> LanguageService - GithubService --> SoccerTeamService - GithubService --> CacheService - GithubService --> RateManager - - CityService --> StateService - CityService --> SoccerTeamService - - RegionService --> StateService - RegionService --> CityService - - ReferencePopulationService --> RegionService - ReferencePopulationService --> StateService - ReferencePopulationService --> CityService - - HiringService --> GithubService - HiringService --> LinkedInService - HiringService --> CacheService diff --git a/docs/diagrams/architecture/service-layer.mmd b/docs/diagrams/architecture/service-layer.mmd deleted file mode 100644 index 7c4ad09..0000000 --- a/docs/diagrams/architecture/service-layer.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Controller["Controllers"] --> ServiceLayer["Service Layer"] - ServiceLayer --> Cache["Cache Services"] - ServiceLayer --> Rate["Rate Management"] - ServiceLayer --> GraphQL["GraphQL Components"] - ServiceLayer --> Models["Model Entities"] - ServiceLayer --> External["External APIs
GitHub & LinkedIn"] diff --git a/docs/diagrams/architecture/use_nearest_region-2.mmd b/docs/diagrams/architecture/use_nearest_region-2.mmd deleted file mode 100644 index d6b8621..0000000 --- a/docs/diagrams/architecture/use_nearest_region-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Start["User Coordinates"] --> Iterate["Iterate Regions"] - Iterate --> Validate["Validate Geo Coordinates"] - Validate --> Calc["Compute Haversine Distance"] - Calc --> Compare["Compare With Minimum Distance"] - Compare --> Update["Update Nearest Region"] - Update --> EndNode["Return Nearest Region"] diff --git a/docs/diagrams/architecture/use_nearest_region-3.mmd b/docs/diagrams/architecture/use_nearest_region-3.mmd deleted file mode 100644 index 29fe7ff..0000000 --- a/docs/diagrams/architecture/use_nearest_region-3.mmd +++ /dev/null @@ -1,10 +0,0 @@ -sequenceDiagram - participant Component - participant Hook as UseNearestRegion - participant Geo as BrowserGeolocation - - Component->>Hook: Call useNearestRegion(regions) - Hook->>Geo: getCurrentPosition() - Geo-->>Hook: Position or Error - Hook->>Hook: Compute distances - Hook-->>Component: nearestRegion and error diff --git a/docs/diagrams/architecture/use_nearest_region-4.mmd b/docs/diagrams/architecture/use_nearest_region-4.mmd deleted file mode 100644 index cbb420c..0000000 --- a/docs/diagrams/architecture/use_nearest_region-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - RegionList["Region Array"] --> Filter["Skip Missing Coordinates"] - Filter --> Distance["Calculate Distance"] - Distance --> Select["Select Minimum"] diff --git a/docs/diagrams/architecture/use_nearest_region.mmd b/docs/diagrams/architecture/use_nearest_region.mmd deleted file mode 100644 index 8b655ca..0000000 --- a/docs/diagrams/architecture/use_nearest_region.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Browser["Browser Geolocation API"] --> Hook["Use Nearest Region Hook"] - Hook --> RegionType["Region Type"] - Hook --> UI["UI Components"] - UI --> UrlState["Use Url State Hook"] diff --git a/docs/diagrams/architecture/use_url_state-2.mmd b/docs/diagrams/architecture/use_url_state-2.mmd deleted file mode 100644 index f7e9e74..0000000 --- a/docs/diagrams/architecture/use_url_state-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Start["URL Search Params Changed"] --> Loop["Iterate URL_PARAMS"] - Loop --> GetValue["Get Param Value"] - GetValue --> Transform["Apply Transform If Present"] - Transform --> Validate["Run validate()"] - Validate -->|"Valid"| Assign["Assign To UrlState"] - Validate -->|"Invalid"| Default["Use Default Value"] - Assign --> Continue["Next Param"] - Default --> Continue - Continue --> End["Return UrlState"] diff --git a/docs/diagrams/architecture/use_url_state-3.mmd b/docs/diagrams/architecture/use_url_state-3.mmd deleted file mode 100644 index 7e2b5f1..0000000 --- a/docs/diagrams/architecture/use_url_state-3.mmd +++ /dev/null @@ -1,12 +0,0 @@ -flowchart TD - Trigger["updateUrlState Called"] --> Clear["Clear Existing Timeout"] - Clear --> Decide["Immediate Or Debounced?"] - Decide -->|"Immediate"| Execute["Execute Update"] - Decide -->|"Debounced"| Schedule["setTimeout(updateFn)"] - Schedule --> Execute - - Execute --> Clone["Clone Current SearchParams"] - Clone --> Compare["Compare Each New Value"] - Compare -->|"Changed"| Modify["Set Or Delete Param"] - Compare -->|"No Change"| Skip["Skip"] - Modify --> Replace["setSearchParams replace=true"] diff --git a/docs/diagrams/architecture/use_url_state-4.mmd b/docs/diagrams/architecture/use_url_state-4.mmd deleted file mode 100644 index 63f6548..0000000 --- a/docs/diagrams/architecture/use_url_state-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - NewState["New UrlState"] --> HasPrevious["Previous Exists?"] - HasPrevious -->|"No"| Store["Store As Previous"] - HasPrevious -->|"Yes"| Compare["Compare Each Field"] - Compare -->|"Any Difference"| True["Return True"] - Compare -->|"No Difference"| False["Return False"] - True --> UpdatePrev["Update Previous Ref"] - False --> UpdatePrev diff --git a/docs/diagrams/architecture/use_url_state-5.mmd b/docs/diagrams/architecture/use_url_state-5.mmd deleted file mode 100644 index 183cbe0..0000000 --- a/docs/diagrams/architecture/use_url_state-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Autocomplete["Language Or Region Autocomplete"] --> Hook["useUrlState"] - Pagination["Pagination Component"] --> Hook - Hook --> ApiLayer["API Service Layer"] - ApiLayer --> Backend["Backend Service"] diff --git a/docs/diagrams/architecture/use_url_state.mmd b/docs/diagrams/architecture/use_url_state.mmd deleted file mode 100644 index a0c2bb2..0000000 --- a/docs/diagrams/architecture/use_url_state.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Router["React Router useSearchParams"] --> Hook["useUrlState Hook"] - Hook --> Parser["parseUrlValue()"] - Hook --> Updater["updateUrlState()"] - Hook --> Reset["resetUrlState()"] - Hook --> ChangeDetector["hasStateChanged"] - - Parser --> Config["URL_PARAMS Config"] - Updater --> Config - Hook --> Components["Filter Components"] diff --git a/docs/diagrams/architecture/webpack-plugins-2.mmd b/docs/diagrams/architecture/webpack-plugins-2.mmd index 9329d5d..5dd6c2f 100644 --- a/docs/diagrams/architecture/webpack-plugins-2.mmd +++ b/docs/diagrams/architecture/webpack-plugins-2.mmd @@ -1,8 +1,18 @@ -flowchart LR - Compiler["Webpack Compiler"] --> BeforeRun["beforeRun Hook"] - Compiler --> WatchRun["watchRun Hook"] - Compiler --> Emit["emit Hook"] - - BeforeRun --> FaviconPlugin["FaviconGeneratorPlugin"] - WatchRun --> FaviconPlugin - Emit --> SeoPlugin["SeoFilesPlugin"] +flowchart TD + Start["Webpack Build Starts"] --> Hook1["beforeRun Hook"] + Start --> Hook2["watchRun Hook"] + + Hook1 --> Generate["generateFavicon()"] + Hook2 --> Generate + + Generate --> CheckSVG{"SVG Exists?"} + CheckSVG -->|"No"| Warn["Log Warning"] + CheckSVG -->|"Yes"| CheckTime{"ICO Newer?"} + + CheckTime -->|"Yes"| Skip["Skip Generation"] + CheckTime -->|"No"| Convert["Convert SVG → PNG → ICO"] + + Convert --> Save["Write favicon.ico"] + Save --> End["Continue Build"] + Skip --> End + Warn --> End diff --git a/docs/diagrams/architecture/webpack-plugins-3.mmd b/docs/diagrams/architecture/webpack-plugins-3.mmd index 35d6434..1c2ce63 100644 --- a/docs/diagrams/architecture/webpack-plugins-3.mmd +++ b/docs/diagrams/architecture/webpack-plugins-3.mmd @@ -1,10 +1,11 @@ flowchart TD - Start["Plugin Triggered"] --> CheckSVG["Check if SVG Exists"] - CheckSVG -->|"No"| Warn["Log Warning"] - CheckSVG -->|"Yes"| CheckICO["Check if ICO Exists"] - CheckICO --> CompareTime["Compare Modification Time"] - CompareTime -->|"ICO Newer"| Skip["Skip Generation"] - CompareTime -->|"SVG Newer"| Convert["Convert SVG to PNG via sharp"] - Convert --> ToICO["Convert PNG to ICO via to-ico"] - ToICO --> Save["Write favicon.ico to Disk"] - Save --> End["Done"] + Build["Webpack Emit Phase"] --> EmitHook["emit Hook"] + EmitHook --> Date["Compute Current Date"] + Date --> SitemapGen["Generate sitemap.xml"] + Date --> RobotsGen["Generate robots.txt"] + + SitemapGen --> Inject1["Add to compilation.assets"] + RobotsGen --> Inject2["Add to compilation.assets"] + + Inject1 --> Output["Build Output Directory"] + Inject2 --> Output diff --git a/docs/diagrams/architecture/webpack-plugins-4.mmd b/docs/diagrams/architecture/webpack-plugins-4.mmd index a278f59..c6b12a0 100644 --- a/docs/diagrams/architecture/webpack-plugins-4.mmd +++ b/docs/diagrams/architecture/webpack-plugins-4.mmd @@ -1,8 +1,14 @@ -flowchart TD - EmitStart["emit Hook Triggered"] --> DateGen["Generate Current Date"] - DateGen --> Sitemap["Build sitemap.xml Content"] - DateGen --> Robots["Build robots.txt Content"] - Sitemap --> Inject1["Add sitemap.xml to compilation.assets"] - Robots --> Inject2["Add robots.txt to compilation.assets"] - Inject1 --> Done["Assets Ready for Output"] - Inject2 --> Done +flowchart LR + subgraph BuildTime["Build Time"] + WP["Webpack"] --> FP["FaviconGeneratorPlugin"] + WP --> SP["SeoFilesPlugin"] + end + + subgraph Runtime["Browser Runtime"] + React["React Application"] + API["Backend API"] + end + + FP --> Assets["Static Assets"] + SP --> Assets + Assets --> React diff --git a/docs/diagrams/architecture/webpack-plugins-5.mmd b/docs/diagrams/architecture/webpack-plugins-5.mmd deleted file mode 100644 index 7cae3b0..0000000 --- a/docs/diagrams/architecture/webpack-plugins-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Runtime["Frontend Application Code"] --> Browser["Browser Runtime"] - BuildTime["Webpack Plugins"] --> Output["Static Build Artifacts"] - - Runtime -.->|"No Direct Dependency"| BuildTime diff --git a/docs/diagrams/architecture/webpack-plugins.mmd b/docs/diagrams/architecture/webpack-plugins.mmd index b663b01..63a00ad 100644 --- a/docs/diagrams/architecture/webpack-plugins.mmd +++ b/docs/diagrams/architecture/webpack-plugins.mmd @@ -1,10 +1,19 @@ flowchart TD Dev["Developer Runs Build"] --> Webpack["Webpack Compiler"] - Webpack -->|"beforeRun / watchRun"| FaviconPlugin["FaviconGeneratorPlugin"] - Webpack -->|"emit"| SeoPlugin["SeoFilesPlugin"] - FaviconPlugin --> FileSystem["File System"] - SeoPlugin --> Assets["Compilation Assets"] + subgraph plugins["Webpack Plugins Module"] + direction TB + FaviconPlugin["FaviconGeneratorPlugin"] + SeoPlugin["SeoFilesPlugin"] + end - Assets --> Output["Build Output Directory"] - FileSystem --> Output + Webpack -->|"beforeRun / watchRun"| FaviconPlugin + Webpack -->|"emit"| SeoPlugin + + FaviconPlugin -->|"Generates"| IcoFile["favicon.ico"] + SeoPlugin -->|"Injects"| Sitemap["sitemap.xml"] + SeoPlugin -->|"Injects"| Robots["robots.txt"] + + IcoFile --> Output["Build Output Directory"] + Sitemap --> Output + Robots --> Output diff --git a/docs/getting-started/first-steps.md b/docs/getting-started/first-steps.md index 0606f63..d5f596f 100644 --- a/docs/getting-started/first-steps.md +++ b/docs/getting-started/first-steps.md @@ -1,129 +1,172 @@ # First Steps -After starting Major League GitHub locally, here are the five things to do first to orient yourself and explore the platform. +After completing the quick start and getting the app running, here are the first things to explore and configure. --- -## 1. Explore the Leaderboard Filters +## 1. Explore the Leaderboard -The filter panel at the top of the page lets you slice the leaderboard across several dimensions simultaneously: +Open **[http://localhost:3000](http://localhost:3000)** and try the following: -| Filter | Description | -|--------|-------------| -| **Language** | Programming language (Java, Python, TypeScript, Go, etc.) | -| **City** | Filter contributors near a specific U.S. city | -| **State** | Filter by U.S. state | -| **Region** | Filter by geographic region (e.g., Pacific Northwest) | -| **MLS Team** | Filter contributors nearest to an MLS stadium | +### Filter by Programming Language -Try combining a language (e.g., `Python`) with a state (e.g., `California`) to see how the leaderboard responds. Notice that the URL updates as you apply filters — the full filter state is encoded in the query parameters. +Use the **Language** autocomplete to select a language (e.g., Java, TypeScript, Python). The leaderboard updates automatically using the TanStack React Query caching layer — no page reload needed. ---- +### Filter by Location -## 2. Share a Leaderboard View +Combine geographic filters: -Every leaderboard configuration is fully shareable via URL. For example: +- **City** — drill down to a specific city +- **State** — view contributors across an entire U.S. state +- **Region** — multi-state MLS regions (e.g., Pacific, Southeast) +- **MLS Team** — contributors near a specific stadium -```text -https://www.mlg.soccer/?languageId=python&stateId=california -``` +> Filters are URL-driven. Every filter change updates the browser URL, making results shareable and bookmarkable. -All filter parameters are managed by the `useUrlState` hook in the frontend, which keeps the browser URL in sync with the current view. This means: +### Try the "Near Me" Feature -- Bookmarking a URL preserves your filters -- Sharing a link with a colleague shows them the exact same leaderboard -- Refreshing the page maintains your current filter selections +The `useNearestRegion` hook uses your browser's Geolocation API to automatically suggest the closest MLS region. Click **Use My Location** if prompted by the browser. --- -## 3. Understand Contributor Scoring +## 2. Explore the REST API -Each contributor card displays a score. The scoring formula is: +With the backend running on port 8450, open a browser or use curl to explore the API directly: -```text -Score = commits × max(starsReceived, 1) × recencyMultiplier +**Get ranked contributors (default language, no location filter):** + +```bash +curl "http://localhost:8450/api/contributors/search" ``` -Where: -- **commits** = total contributions to repositories in the selected language -- **starsReceived** = total stars on repositories in that language -- **recencyMultiplier** = 1.0–2.0 based on activity within the past year +**Filter by language and state:** -Higher scores indicate developers who are prolific, have impactful projects, and have been recently active. This formula deliberately rewards both volume and impact. +```bash +curl "http://localhost:8450/api/contributors/search?languageId=java&stateId=ca" +``` ---- +**Autocomplete cities:** + +```bash +curl "http://localhost:8450/api/autocomplete/cities?query=San" +``` + +**Autocomplete languages:** -## 4. Export Results to CSV +```bash +curl "http://localhost:8450/api/autocomplete/languages?query=ty" +``` -Any filtered leaderboard view can be exported as a CSV file. The export button triggers a browser download of a file named `contributors.csv` containing: +**Get a specific city by ID:** -- GitHub username, display name, and profile URL -- Location (city, state, region) -- Nearest MLS team -- Score breakdown (commits, stars, recency) -- Social links (GitHub, Twitter, Mastodon, Bluesky, website, email) +```bash +curl "http://localhost:8450/api/entities/cities/1" +``` -To export, click the **Export CSV** button in the UI, or make a direct request: +**Export as CSV:** ```bash -curl "http://localhost:8450/api/contributors/export?languageId=java&maxResults=15" \ +curl "http://localhost:8450/api/contributors/export?languageId=java" \ -o contributors.csv ``` +The API always returns a consistent JSON envelope: + +```json +{ + "status": "success", + "message": null, + "data": [...] +} +``` + +--- + +## 3. Understand the Scoring Formula + +The ranking formula is: + +```text +score = commits × max(starsReceived, 1) × recencyMultiplier +``` + +- **recencyMultiplier** ranges from `1.0` to `2.0` +- Contributors with activity in the past year receive a higher multiplier +- Stars floored at `1` to prevent zero-scores for active contributors with few-starred repos + +This means a contributor with many commits and recent activity will outrank someone with high stars but old activity. + --- -## 5. Check the REST API Directly +## 4. Configure the Cache + +By default, the backend uses **Redis** in `read-write` mode. You can switch to disk-based caching for simpler local development: + +**Disk cache (no Redis required):** -The backend exposes a clean REST API you can explore directly. Key endpoints: +```bash +GITHUB_TOKENS=your_token \ +CACHE_IMPLEMENTATION=disk \ +mvn spring-boot:run -f backend/pom.xml +``` + +**Force-update mode (bypasses cache, always fetches fresh):** ```bash -# Get top Java contributors in California -curl "http://localhost:8450/api/contributors/search?languageId=java&stateId=california" +GITHUB_TOKENS=your_token \ +CACHE_MODE=force-update \ +mvn spring-boot:run -f backend/pom.xml +``` -# Autocomplete cities starting with "San" -curl "http://localhost:8450/api/autocomplete/cities?query=San" +> **Warning:** `force-update` mode makes a live GitHub API call on every request. Use sparingly to avoid hitting rate limits. + +--- -# Autocomplete available programming languages -curl "http://localhost:8450/api/autocomplete/languages?query=py" +## 5. Check Application Health -# Look up a specific MLS team by ID -curl "http://localhost:8450/api/entities/teams/la-galaxy" +Spring Boot Actuator is included. Check application health: -# Look up a specific region by ID -curl "http://localhost:8450/api/entities/regions/west-coast" +```bash +curl "http://localhost:8450/actuator/health" ``` -All responses follow the standardized `ApiResponse` wrapper: +Expected response: ```json { - "status": "success", - "message": "...", - "data": ... + "status": "UP" } ``` --- -## Where to Get Help +## Key Configuration Reference -- **Open an issue:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) -- **Browse open PRs:** [https://github.com/flamingo-stack/major-league-github/pulls](https://github.com/flamingo-stack/major-league-github/pulls) -- **Read the architecture docs** in the `docs/reference/architecture/` folder of this repository for deep-dives into each module -- **Spring Boot Actuator** — the backend exposes `/actuator/health` for service health checks at [http://localhost:8450/actuator/health](http://localhost:8450/actuator/health) +| Property | Default | Description | +|----------|---------|-------------| +| `cache.implementation` | `redis` | Cache backend: `redis` or `disk` | +| `cache.mode` | `read-write` | Cache mode: `read-write`, `read-only`, `force-update` | +| `spring.redis.host` | `localhost` | Redis host | +| `spring.redis.port` | `6379` | Redis port | +| `github.tokens` | — | Comma-separated GitHub PATs | +| `github.api.concurrency` | varies | Concurrent GitHub API calls | --- -## Common Early Questions +## Where to Get Help -**Q: The leaderboard is empty after starting. What's wrong?** -A: The `PreCacheService` warms the Redis cache on startup by iterating all languages. Wait 30–90 seconds and refresh. The `/actuator/health` endpoint confirms the backend is running. +- **GitHub Issues:** [https://github.com/flamingo-stack/major-league-github/issues](https://github.com/flamingo-stack/major-league-github/issues) +- **GitHub Discussions / PRs:** [https://github.com/flamingo-stack/major-league-github/pulls](https://github.com/flamingo-stack/major-league-github/pulls) +- **Live Site:** [https://www.mlg.soccer](https://www.mlg.soccer) -**Q: Can I add more GitHub tokens?** -A: Yes. Set `GITHUB_TOKENS` to a comma-separated list. The `GithubTokenRateManager` automatically distributes requests across tokens and selects the one with the most remaining quota. +--- -**Q: Where is geographic data stored?** -A: Cities, states, regions, and teams are loaded from CSV files in `backend/src/main/resources/data/` at startup. No database migrations are needed. +## Common First-Run Issues -**Q: How do I change the default language?** -A: The default language is Java (configured in `LanguageService`). Change `languageId` in the URL to switch — or modify the `LanguageService` default for your own deployment. +| Issue | Likely Cause | Fix | +|-------|-------------|-----| +| Backend won't start | Redis not running | Run `redis-server` first | +| Empty leaderboard | Cache still warming | Wait 30–60 seconds after startup | +| `Rate limit exceeded` error | GitHub token missing or exhausted | Check `GITHUB_TOKENS` env var | +| CORS error in browser | Frontend/backend URL mismatch | Set `BACKEND_API_URL=http://localhost:8450` | +| Frontend 404 on refresh | Dev server not configured for SPA routing | Use the frontend dev server, not a static server | diff --git a/docs/getting-started/introduction.md b/docs/getting-started/introduction.md index 7ec367a..315cf44 100644 --- a/docs/getting-started/introduction.md +++ b/docs/getting-started/introduction.md @@ -1,26 +1,18 @@ # Introduction to Major League GitHub -**Major League GitHub** ([mlg.soccer](https://www.mlg.soccer)) is an open-source, sports-themed leaderboard that ranks GitHub contributors like professional soccer players. It maps open-source developers across the United States using programming language preferences, geographic location, and proximity to MLS stadiums — combining GitHub analytics with geospatial modeling to create a uniquely gamified developer leaderboard. +**Major League GitHub** ([mlg.soccer](https://www.mlg.soccer)) is an open-source, sports-themed leaderboard that ranks GitHub contributors like professional soccer players. Inspired by Major League Soccer (MLS), it filters contributors by programming language, geographic location, and proximity to real MLS stadiums — turning open-source contribution data into a competitive, engaging leaderboard experience. -> **Repository:** [https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github) +> **This is an independent open-source side project.** It is not affiliated with any commercial platform. --- -## What Is It? +## Elevator Pitch -Major League GitHub turns GitHub contributor statistics into a leaderboard experience inspired by Major League Soccer (MLS). Just as soccer rankings reward goals, assists, and appearances, MLG ranks developers using: +GitHub has millions of contributors. Major League GitHub answers the question: -- **Commits** — total contributions made -- **Stars received** — community impact of their repositories -- **Recency multiplier** — how recently they have been active +> *"Who are the top Java developers within 50 miles of a Chicago MLS stadium?"* -The formula is intentionally transparent: - -```text -Score = commits × max(starsReceived, 1) × recencyMultiplier -``` - -Where `recencyMultiplier` ranges from 1.0 to 2.0 based on activity within the past year. +It pulls real-time data from GitHub's GraphQL API, applies a scoring formula based on commits and repository stars, and presents results in a clean, filterable leaderboard — all filtered by language, city, state, region, and nearest MLS team. --- @@ -28,66 +20,108 @@ Where `recencyMultiplier` ranges from 1.0 to 2.0 based on activity within the pa | Feature | Description | |---------|-------------| -| **Language Filtering** | Filter contributors by any programming language (Java, Python, TypeScript, etc.) | -| **Geographic Filtering** | Narrow results by city, state, or region | -| **MLS Stadium Proximity** | Rank contributors by their distance to the nearest MLS stadium | -| **Real-Time Leaderboard** | GitHub GraphQL data refreshed on a schedule via the Cache Updater service | -| **Shareable URLs** | All filters are reflected in the URL — bookmark or share any leaderboard view | -| **CSV Export** | Download any filtered leaderboard result as a CSV file | -| **Hiring Section** | Highlights top developers and associated job openings | -| **Responsive UI** | Works across desktop and mobile with Material-UI components | +| **Language Filtering** | Filter contributors by any programming language (Java, TypeScript, Python, etc.) | +| **Geographic Filtering** | Filter by city, state, or multi-state region | +| **MLS Team Proximity** | Find contributors near any Major League Soccer stadium | +| **Contributor Scoring** | Rank by a formula: `commits × max(stars, 1) × recency multiplier` | +| **CSV Export** | Download ranked results as a CSV with social links | +| **Hiring Mode** | Hiring managers can publish open roles and appear in contributor profiles | +| **Distributed Cache** | Redis-backed caching protects GitHub API rate limits | +| **URL-Driven State** | Filters persist in the URL — shareable and bookmarkable | --- -## Target Audience - -Major League GitHub is designed for: +## Who Is This For? -- **Developers** who want to see how they rank among regional peers for a given language -- **Hiring managers** looking to discover talented open-source contributors near their offices -- **Open-source enthusiasts** who enjoy gamified community analytics -- **Engineers** interested in how to build a full-stack, production-grade application with Spring Boot, React, Redis, and Kubernetes +- **Developers** curious about where the best contributors in their city or language are +- **Hiring managers** looking to find top open-source contributors near their offices +- **Open-source enthusiasts** who want to explore contribution patterns by geography +- **Contributors** who want to see how they rank among their peers --- ## System Overview +Major League GitHub is a full-stack, microservice-based system: + ```mermaid -flowchart TD - User["User Browser"] --> Frontend["React + TypeScript Frontend"] - Frontend --> BackendAPI["Backend Service (Port 8450)"] - BackendAPI --> Redis["Redis Cache"] - BackendAPI --> GitHub["GitHub GraphQL API"] - CacheUpdater["Cache Updater (Port 8451)"] --> Redis +flowchart LR + User["User (Browser)"] --> Frontend["React 19 Frontend"] + Frontend --> Backend["Backend Service (Port 8450)"] + Backend --> Redis["Redis Cache"] + Backend --> GitHub["GitHub GraphQL API"] + Backend --> LinkedIn["LinkedIn API"] + CacheUpdater["Cache Updater (Port 8451)"] --> Backend CacheUpdater --> GitHub - BackendAPI --> LinkedIn["LinkedIn API (Hiring)"] ``` -The system is split into two backend microservices: +### Technology Stack + +**Backend** — Java 21 + Spring Boot 3.4 +- Two microservices: Backend Service (port 8450) and Cache Updater (port 8451) +- GitHub GraphQL API integration with multi-token rate management +- Redis for distributed caching +- Apache Commons CSV for export +- Lombok for clean model definitions + +**Frontend** — React 19 + TypeScript +- Material UI (MUI) component library +- TanStack React Query for server-state management +- React Router for URL-driven filter state +- Axios for HTTP communication +- Webpack 5 build system with custom SEO and favicon plugins + +**Infrastructure** — Docker + Kubernetes (GKE) +- Google Kubernetes Engine deployment +- GitHub Actions CI/CD pipeline + +--- + +## Contributor Ranking Algorithm + +The scoring formula at the heart of the leaderboard is: + +```text +score = commits × max(starsReceived, 1) × recencyMultiplier +``` + +- **Commits** — total commit count across repositories +- **starsReceived** — total stars across repositories (minimum of 1 to avoid zero scores) +- **recencyMultiplier** — ranges from 1.0 to 2.0, rewarding contributors active in the past year -- **Backend Service (port 8450)** — serves the public REST API consumed by the React frontend -- **Cache Updater (port 8451)** — runs scheduled jobs that keep GitHub contributor data fresh in Redis +--- + +## Repository + +The source code is available at: -Both services are built from the same Spring Boot codebase, activated via Maven profiles. +**[https://github.com/flamingo-stack/major-league-github](https://github.com/flamingo-stack/major-league-github)** --- -## Tech Stack at a Glance +## Project Structure -| Layer | Technology | -|-------|-----------| -| Backend | Java 21 + Spring Boot 3.4 | -| Frontend | React 19 + TypeScript + Material-UI | -| Caching | Redis (distributed) | -| External Data | GitHub GraphQL API | -| Build | Webpack (custom plugins for SEO + favicon) | -| Deployment | Docker + Kubernetes (GKE) | -| CI/CD | GitHub Actions | +```text +major-league-github/ +├── backend/ # Java 21 + Spring Boot 3.4 backend +│ └── src/main/java/cx/flamingo/analysis/ +│ ├── controller/ # REST API endpoints +│ ├── service/ # Business logic +│ ├── cache/ # Redis/Disk caching +│ ├── graphql/ # GitHub GraphQL query builder +│ ├── model/ # Domain models +│ ├── rate/ # GitHub token rate management +│ └── config/ # Spring Boot configuration +└── frontend/ # React 19 + TypeScript frontend + └── src/ + ├── components/ # UI components + ├── hooks/ # React hooks + ├── services/ # API service layer + └── types/ # TypeScript type contracts +``` --- -## How to Get Started +## Live Site -- **Install prerequisites** — see the [Prerequisites](prerequisites.md) guide -- **Run it in 5 minutes** — see the [Quick Start](quick-start.md) guide -- **First things to do** — see the [First Steps](first-steps.md) guide +The application runs live at **[https://www.mlg.soccer](https://www.mlg.soccer)**. diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index 499586c..6308403 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -1,6 +1,6 @@ # Prerequisites -Before running Major League GitHub locally, make sure you have the following tools and accounts in place. +Before setting up Major League GitHub locally, ensure you have the required tools, accounts, and environment variables in place. --- @@ -8,126 +8,129 @@ Before running Major League GitHub locally, make sure you have the following too | Tool | Minimum Version | Purpose | |------|----------------|---------| -| Java (JDK) | 21 | Backend runtime (Spring Boot 3.4 requires Java 17+; project targets Java 21) | -| Maven | 3.9+ | Backend build tool | -| Node.js | 18+ | Frontend build toolchain (Webpack, npm) | -| npm | 9+ | Frontend package manager | -| Redis | 7+ | Distributed cache (required for production mode) | -| Docker | 24+ | Containerized local Redis or full deployment | -| Git | 2.40+ | Source control | +| **Java JDK** | 21 | Backend runtime (Spring Boot) | +| **Apache Maven** | 3.9+ | Backend build and dependency management | +| **Node.js** | 18+ | Frontend build toolchain | +| **npm** | 9+ | Frontend package manager | +| **Redis** | 6+ | Distributed cache (required for backend) | +| **Git** | 2.x | Source control | --- -## Verification Commands - -Run these commands to confirm your environment is ready: - -```bash -# Java 21 -java -version -# Expected: openjdk 21.x.x ... - -# Maven -mvn -version -# Expected: Apache Maven 3.9.x ... - -# Node.js -node --version -# Expected: v18.x.x or higher +## System Requirements -# npm -npm --version -# Expected: 9.x.x or higher +| Resource | Recommended | +|----------|-------------| +| **RAM** | 4 GB minimum, 8 GB recommended | +| **Disk** | 2 GB free (for Maven + npm dependencies) | +| **OS** | macOS, Linux, or Windows (WSL2 recommended) | +| **CPU** | Any modern x86-64 or ARM64 (Apple Silicon supported) | -# Redis (if running locally) -redis-cli ping -# Expected: PONG +> **Apple Silicon (M1/M2/M3) Note:** The pom.xml includes the `netty-resolver-dns-native-macos` dependency with the `osx-aarch_64` classifier, which is required for macOS ARM64 DNS resolution. No special configuration needed beyond the standard Maven build. -# Docker -docker --version -# Expected: Docker version 24.x.x ... +--- -# Git -git --version -# Expected: git version 2.x.x -``` +## Account Requirements ---- +### GitHub Personal Access Token (Required) -## GitHub API Access +The backend queries the GitHub GraphQL API on your behalf. You need at least one GitHub Personal Access Token. -The backend calls the GitHub GraphQL API. You will need one or more **GitHub Personal Access Tokens (PATs)** with at minimum `read:user` scope. +1. Go to [GitHub Settings → Developer Settings → Personal Access Tokens](https://github.com/settings/tokens) +2. Create a **Classic** or **Fine-grained** token +3. Required scopes: `read:user`, `public_repo` -### Creating a GitHub PAT +> **Multiple tokens:** For production use, the rate manager supports multiple tokens (`github.tokens`). More tokens allow higher API concurrency and better rate limit resilience. -1. Go to [https://github.com/settings/tokens](https://github.com/settings/tokens) -2. Click **Generate new token (classic)** -3. Select the scopes: - - `read:user` - - `repo` (if you want repository star counts) -4. Copy the generated token +### LinkedIn API Credentials (Optional) -> **Multi-token support:** The backend supports multiple tokens for increased throughput. Configure them as a comma-separated list in the `GITHUB_TOKENS` environment variable. The `GithubTokenRateManager` automatically selects the optimal token per request. +The hiring section integrates with the LinkedIn API to pull job postings. This is optional for development; the system falls back to predefined remote roles if the API is unavailable. --- ## Environment Variables -The following environment variables are required to run the backend. Set them in your shell, a `.env` file, or Kubernetes secrets depending on your deployment method. +The following environment variables are required or optional depending on your setup: -### Backend Service +### Backend Environment Variables | Variable | Required | Description | |----------|----------|-------------| -| `GITHUB_TOKENS` | Yes | Comma-separated GitHub Personal Access Tokens | -| `SPRING_REDIS_HOST` | Yes | Redis host (e.g., `localhost`) | -| `SPRING_REDIS_PORT` | Yes | Redis port (default: `6379`) | -| `SPRING_PROFILES_ACTIVE` | Yes | Profile to activate: `backend-service` or `cache-updater` | +| `GITHUB_TOKENS` | **Yes** | Comma-separated GitHub Personal Access Tokens | +| `SPRING_REDIS_HOST` | No | Redis host (default: `localhost`) | +| `SPRING_REDIS_PORT` | No | Redis port (default: `6379`) | +| `CACHE_IMPLEMENTATION` | No | Cache backend: `redis` or `disk` (default: `redis`) | +| `CACHE_MODE` | No | Cache mode: `read-write`, `read-only`, `force-update` (default: `read-write`) | +| `GITHUB_API_CONCURRENCY` | No | Number of concurrent GitHub API requests (default varies by profile) | -### Frontend Development +### Frontend Environment Variables -| Variable | Default | Description | -|----------|---------|-------------| -| `BACKEND_API_URL` | `https://www.mlg.soccer` | Backend API base URL for Webpack dev server proxy | -| `PORT` | `8450` | Webpack dev server port | -| `NODE_ENV` | `development` | Build mode | +| Variable | Required | Description | +|----------|----------|-------------| +| `BACKEND_API_URL` | No | Backend API base URL (default: `/`, same-origin) | + +> Set `BACKEND_API_URL` to `http://localhost:8450` when running the frontend separately from the backend during local development. --- -## System Requirements +## Verification Commands -| Resource | Recommended | -|----------|-------------| -| RAM | 4 GB+ (8 GB for running all services concurrently) | -| CPU | 2+ cores | -| Disk | 2 GB free (Maven + npm dependency caches) | -| OS | macOS, Linux, or Windows (WSL2 recommended on Windows) | +Use these commands to verify your environment is ready: ---- +**Check Java version:** +```bash +java -version +``` +Expected output should show Java 21 or higher. + +**Check Maven version:** +```bash +mvn -version +``` + +**Check Node.js version:** +```bash +node --version +``` + +**Check npm version:** +```bash +npm --version +``` -## macOS Note +**Check Redis connectivity:** +```bash +redis-cli ping +``` +Expected output: `PONG` -The backend's `pom.xml` includes a Netty DNS resolver for macOS (`netty-resolver-dns-native-macos` for `osx-aarch_64`). If you are on an Apple Silicon Mac, this dependency is already bundled and no additional configuration is required. +**Check Git version:** +```bash +git --version +``` --- -## Optional: LinkedIn API +## Spring Boot Maven Profiles -The hiring section fetches job postings via the LinkedIn API. This is entirely optional. If LinkedIn credentials are not configured, the system falls back to static default job entries. +The backend uses two Maven profiles that determine which microservice starts: -| Variable | Description | -|----------|-------------| -| `LINKEDIN_CLIENT_ID` | LinkedIn OAuth2 Client ID | -| `LINKEDIN_CLIENT_SECRET` | LinkedIn OAuth2 Client Secret | -| `LINKEDIN_ORGANIZATION_ID` | LinkedIn Organization ID for job postings | +| Profile | Port | Purpose | +|---------|------|---------| +| `backend-service` | 8450 | REST API (active by default) | +| `cache-updater` | 8451 | Scheduled cache warming | + +The `backend-service` profile is active by default. You do not need to set anything extra to run the API server. --- -## Repository +## CORS Allowed Origins -Clone the repository to get started: +The backend is pre-configured to allow CORS from: -```bash -git clone https://github.com/flamingo-stack/major-league-github.git -cd major-league-github -``` +- `http://localhost:3000` (local frontend dev server) +- `http://localhost:8450` (local backend) +- `https://www.mlg.soccer` (production) +- `http://www.mlg.soccer` (production HTTP) + +No additional CORS configuration is required for standard local development. diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md index ac7c0e8..f4c9cb1 100644 --- a/docs/getting-started/quick-start.md +++ b/docs/getting-started/quick-start.md @@ -11,34 +11,24 @@ Get Major League GitHub running locally in about 5 minutes. git clone https://github.com/flamingo-stack/major-league-github.git cd major-league-github -# 2. Start Redis (Docker) -docker run -d -p 6379:6379 --name mlg-redis redis:7 +# 2. Start Redis +redis-server -# 3. Start the Backend Service +# 3. Start the backend (new terminal) cd backend -GITHUB_TOKENS=your_github_pat \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pbackend-service +GITHUB_TOKENS=your_github_token_here mvn spring-boot:run -# 4. (New terminal) Start the Cache Updater -cd backend -GITHUB_TOKENS=your_github_pat \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pcache-updater - -# 5. (New terminal) Start the Frontend +# 4. Start the frontend (new terminal) cd frontend npm install -BACKEND_API_URL=http://localhost:8450 npx webpack serve +BACKEND_API_URL=http://localhost:8450 npm run dev ``` -Open your browser at [http://localhost:8450](http://localhost:8450). +Open your browser at **[http://localhost:3000](http://localhost:3000)**. --- -## Step-by-Step +## Step-by-Step Setup ### Step 1 — Clone the Repository @@ -49,118 +39,115 @@ cd major-league-github ### Step 2 — Start Redis -The backend requires Redis for distributed caching. The quickest way is Docker: +Redis must be running before the backend starts. If you installed Redis via Homebrew (macOS): ```bash -docker run -d \ - -p 6379:6379 \ - --name mlg-redis \ - redis:7 +brew services start redis ``` -Verify Redis is running: +Or start it directly: ```bash -docker logs mlg-redis -# Expected: Ready to accept connections +redis-server ``` -### Step 3 — Configure GitHub Tokens - -Export your GitHub Personal Access Token as an environment variable: +Verify Redis is running: ```bash -export GITHUB_TOKENS="ghp_yourTokenHere" +redis-cli ping ``` -> For higher throughput, supply multiple tokens separated by commas: -> `export GITHUB_TOKENS="ghp_token1,ghp_token2,ghp_token3"` +Expected output: -### Step 4 — Start the Backend Service +```text +PONG +``` -The Backend Service serves the REST API on port 8450: +### Step 3 — Start the Backend + +The backend is a Spring Boot 3.4 application built with Maven. The `backend-service` profile runs the REST API on port 8450. ```bash cd backend -GITHUB_TOKENS=$GITHUB_TOKENS \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pbackend-service +GITHUB_TOKENS=ghp_your_token_here mvn spring-boot:run ``` -Wait for this log line before proceeding: +> Replace `ghp_your_token_here` with a real GitHub Personal Access Token. See the Prerequisites guide for token creation instructions. + +You should see Spring Boot startup output followed by: ```text -Started MajorLeagueGithubApplication in X.XXX seconds +Started MajorLeagueGithubApplication ``` -### Step 5 — Start the Cache Updater (Optional but Recommended) +The cache will begin warming up immediately. The first load may take a moment as GitHub data is fetched and cached. -The Cache Updater populates Redis with fresh GitHub data in the background: +**To use the disk cache instead of Redis (simpler for dev):** ```bash -# In a new terminal cd backend -GITHUB_TOKENS=$GITHUB_TOKENS \ -SPRING_REDIS_HOST=localhost \ -SPRING_REDIS_PORT=6379 \ -mvn spring-boot:run -Pcache-updater +GITHUB_TOKENS=ghp_your_token_here \ +CACHE_IMPLEMENTATION=disk \ +mvn spring-boot:run ``` -> **Note:** The Cache Updater runs scheduled refresh jobs. On first startup the `PreCacheService` iterates all configured languages and pre-warms the Redis cache. The frontend will show an empty leaderboard until the cache is ready. - -### Step 6 — Start the Frontend Dev Server +### Step 4 — Install Frontend Dependencies ```bash -# In a new terminal cd frontend npm install -BACKEND_API_URL=http://localhost:8450 npx webpack serve ``` -The dev server proxies all `/api` requests to the backend. +### Step 5 — Start the Frontend Dev Server ---- +```bash +BACKEND_API_URL=http://localhost:8450 npm run dev +``` -## Expected Result +The frontend dev server starts on port 3000. -Open [http://localhost:8450](http://localhost:8450) in your browser. +### Step 6 — Open the App -You should see: -- The Major League GitHub leaderboard loading -- Filter controls for language, city, state, region, and MLS team -- Contributor cards rendering as the cache warms up +Navigate to: -If the leaderboard shows "Cache is still being populated", wait 30–60 seconds for the `PreCacheService` to finish its first pass. +```text +http://localhost:3000 +``` + +You should see the Major League GitHub leaderboard. Use the filter panel to select a programming language and geographic location. --- -## API Quick Test +## Expected Results -Confirm the backend is responding: +Once the app is running, you should see: -```bash -curl http://localhost:8450/api/contributors/search?languageId=java&maxResults=5 -``` +- A leaderboard displaying ranked GitHub contributors +- Filter dropdowns for language, city, state, region, and MLS team +- A scoring display showing commits, stars, and the calculated score +- A CSV export button to download results + +--- -Expected response shape: +## Multiple GitHub Tokens (Optional) -```json -{ - "status": "success", - "message": "Found 5 contributors matching the criteria", - "data": [...] -} +To increase rate limits and support higher API concurrency, provide multiple tokens separated by commas: + +```bash +GITHUB_TOKENS=ghp_token1,ghp_token2,ghp_token3 mvn spring-boot:run ``` +The `GithubTokenRateManager` will automatically distribute requests across all tokens and rotate intelligently based on remaining rate limits. + --- -## What Happens Next +## Running the Cache Updater (Optional) -After the cache is warm, the leaderboard becomes fully responsive. Try: +The Cache Updater is a second microservice that pre-warms and refreshes cached data on a schedule. Run it on port 8451: -- Selecting a different programming language from the filter panel -- Filtering by a U.S. state or MLS team -- Copying the URL — all filters are encoded as query parameters for sharing +```bash +cd backend +GITHUB_TOKENS=ghp_your_token_here mvn spring-boot:run -Dspring-boot.run.profiles=cache-updater +``` -For a guided tour of features, see the [First Steps](first-steps.md) guide. +For most local development scenarios, the Cache Updater is not required. The backend service warms the cache on startup automatically. diff --git a/docs/reference/architecture/README.md b/docs/reference/architecture/README.md index 2235d45..0cc65ae 100644 --- a/docs/reference/architecture/README.md +++ b/docs/reference/architecture/README.md @@ -1,260 +1,365 @@ # Major League GitHub **Repository:** https://github.com/flamingo-stack/major-league-github +**Live Site:** https://www.mlg.soccer -Major League GitHub (https://www.mlg.soccer) is an open-source, standalone sports-themed leaderboard that ranks GitHub contributors like professional soccer players. Contributors are filtered and ranked by: +Major League GitHub is an independent, open-source side project that ranks GitHub contributors like professional soccer players. It combines GitHub GraphQL data, geographic modeling, and MLS stadium proximity to create a sports-style leaderboard filtered by: - Programming language -- Geographic location (city, state, region) -- Proximity to MLS stadiums +- City, state, and region +- Nearest MLS team +- Hiring status -The system combines GitHub GraphQL analytics, geospatial modeling, Redis-backed caching, and a React frontend to create a real-time, location-aware developer leaderboard. +The platform consists of: + +- A **Java 21 + Spring Boot 3.4 backend** (two microservices) +- A **React 19 + TypeScript frontend** +- **Redis** for distributed caching +- **Docker + Kubernetes (GKE)** for deployment +- **GitHub Actions CI/CD** --- -# Purpose of the Repository +# 1. End-to-End Architecture + +Major League GitHub is designed as a layered, modular, microservice-based system. + +```mermaid +flowchart LR + User["User (Browser)"] --> Frontend["React Frontend (Port 3000 / Prod)"] + Frontend --> Backend["Backend Service (Spring Boot - Port 8450)"] + Backend --> Cache["Redis Cache"] + Backend --> GitHub["GitHub GraphQL API"] + Backend --> LinkedIn["LinkedIn API"] + CacheUpdater["Cache Updater Service (Port 8451)"] --> Backend + CacheUpdater --> GitHub +``` -The goal of `major-league-github` is to: +### High-Level Flow -1. **Analyze GitHub contributors** via the GitHub GraphQL API. -2. **Rank contributors** using a weighted scoring formula (commits × stars × recency multiplier). -3. **Map contributors geographically** to cities, states, regions, and MLS teams. -4. **Gamify open-source activity** by treating contributors like athletes on a sports leaderboard. -5. **Expose hiring workflows** that highlight top developers and associated job openings. -6. **Deliver a full-stack, production-ready architecture** using Spring Boot, Redis, React, and Kubernetes. +1. User selects filters (language, city, region, team). +2. Frontend calls backend REST endpoints. +3. Backend: + - Checks cache (Redis or disk). + - Uses GitHub GraphQL API to fetch contributor data. + - Applies scoring algorithm. + - Returns ranked contributors. +4. Cache Updater service pre-warms and refreshes cache asynchronously. --- -# High-Level Architecture +# 2. Backend Architecture (Spring Boot) + +The backend is modular and cleanly layered. -Major League GitHub is a distributed full-stack application consisting of: +```mermaid +flowchart TD + AppCore["Application Core"] + Controllers["Controllers"] + Services["Backend Services"] + CacheLayer["Cache Services"] + GraphQL["GraphQL Components"] + Rate["Rate Management"] + Models["Model Entities"] + Config["Configurations"] + + AppCore --> Controllers + AppCore --> Services + AppCore --> CacheLayer + AppCore --> Config + Services --> GraphQL + Services --> Rate + Services --> Models + Controllers --> Services + Services --> CacheLayer +``` -- **Backend Service (Port 8450)** – Public REST API -- **Cache Updater Service (Port 8451)** – Scheduled refresh jobs -- **Redis** – Distributed cache -- **React + TypeScript Frontend** – UI layer -- **GitHub GraphQL API** – External data source -- **LinkedIn API** – Hiring/job data -- **Docker + Kubernetes (GKE)** – Deployment -- **GitHub Actions** – CI/CD +### Backend Microservices + +| Service | Port | Responsibility | +|----------|------|----------------| +| Backend Service | 8450 | REST API, ranking logic | +| Cache Updater | 8451 | Scheduled cache warming | --- -## End-to-End System Architecture +# 3. Frontend Architecture (React + TypeScript) + +The frontend is fully typed and layered. ```mermaid flowchart TD - User["User Browser"] --> Frontend["React Frontend (Module 10–18)"] - Frontend --> ApiLayer["API Layer (Module 14)"] - ApiLayer --> Backend["Spring Boot Backend (Port 8450)"] + Pages["React Pages"] --> Components["Frontend Components"] + Components --> Hooks["Frontend Hooks"] + Hooks --> Services["Frontend Services"] + Services --> Backend["Backend REST API"] + Services --> Types["Frontend Types"] +``` - Backend --> Controllers["REST Controllers (Module 4)"] - Controllers --> Services["Service Layer (Module 8–10)"] - Services --> CacheAbs["CacheServiceAbs (Module 1)"] - CacheAbs --> Redis["Redis (Module 2)"] - CacheAbs --> Disk["Disk Cache (Module 1)"] +### Frontend Stack - Services --> QueryBuilder["GraphQL Query Builder (Module 5)"] - QueryBuilder --> GitHubAPI["GitHub GraphQL API"] +- React 19 +- TypeScript +- Material UI +- React Query +- Custom Webpack plugins (SEO + favicon) - Services --> LinkedInService["LinkedIn Service (Module 9)"] - LinkedInService --> LinkedInAPI["LinkedIn API"] +--- - CacheUpdater["Cache Updater Service (Port 8451)"] --> Redis -``` +# 4. Repository Structure + +## Backend Modules + +### 1. Application Core +**Path:** `backend/src/main/java/cx/flamingo/analysis` + +Bootstraps Spring Boot, enables caching and async execution. + +Documentation: +- `application-core/application-core.md` --- -# Backend Architecture +### 2. Cache Services +**Path:** `backend/src/main/java/cx/flamingo/analysis/cache` -The backend is built using **Java 21 + Spring Boot 3.4** and organized into layered modules. +Pluggable caching abstraction with: -## Backend Layered Architecture +- `RedisCacheService` +- `DiskCacheService` +- `ReadOnlyCacheService` +- `CacheServiceAbs` -```mermaid -flowchart TD - App["MajorLeagueGithubApplication (Module 1)"] - App --> Controllers["Controllers (Module 4)"] - Controllers --> Services["Services (Module 8–10)"] - Services --> GraphQL["GraphQL Builder (Module 5)"] - Services --> RateManager["GitHub Rate Manager (Module 8)"] - Services --> Cache["Cache Abstraction (Module 1)"] - Cache --> RedisImpl["RedisCacheService (Module 2)"] - Cache --> DiskImpl["DiskCacheService (Module 1)"] - Services --> Models["Domain Models (Module 6–7)"] -``` +Documentation: +- `cache-services/cache-services.md` --- -## GitHub Data Retrieval Flow +### 3. Configurations +**Path:** `backend/src/main/java/cx/flamingo/analysis/config` -```mermaid -sequenceDiagram - participant Controller - participant Service as GithubService - participant Rate as GithubTokenRateManager - participant Builder as GitHubQueryBuilder - participant GitHub - - Controller->>Service: getTopContributors(filters) - Service->>Builder: build GraphQL query - Builder-->>Service: query string - Service->>Rate: select optimal token - Rate-->>Service: WebClient - Service->>GitHub: Execute GraphQL request - GitHub-->>Service: JSON response - Service-->>Controller: Ranked Contributors -``` +Centralizes: + +- Async thread pools +- Cache selection strategy +- Redis configuration +- CORS setup +- Profile switching (backend-service vs cache-updater) + +Documentation: +- `configurations/configurations.md` --- -## Contributor Scoring Formula +### 4. Controllers +**Path:** `backend/src/main/java/cx/flamingo/analysis/controller` -```text -Score = commits × max(starsReceived, 1) × recencyMultiplier -``` +REST endpoints: -Where: +- `/api/contributors` +- `/api/autocomplete` +- `/api/entities` +- `/api/hiring` -- `commits` = total contributions -- `starsReceived` = stars on repositories in selected language -- `recencyMultiplier` ∈ [1.0, 2.0] based on activity freshness +Documentation: +- `controllers/controllers.md` -This rewards: +--- + +### 5. Backend Services +**Path:** `backend/src/main/java/cx/flamingo/analysis/service` + +Core business logic: -- High commit volume -- High-impact repositories -- Recent contribution activity +- `GithubService` (ranking + scoring) +- `CityService` +- `RegionService` +- `StateService` +- `SoccerTeamService` +- `LanguageService` +- `HiringService` +- `PreCacheService` + +Documentation: +- `backend-services/backend-services.md` --- -# Frontend Architecture +### 6. GraphQL Components +**Path:** `backend/src/main/java/cx/flamingo/analysis/graphql` -The frontend is built using: +Fluent GitHub query builder: -- **React 19** -- **TypeScript** -- **Material-UI** -- **React Query** -- **Custom Webpack Plugins** +- `GitHubQueryBuilder` +- `Field` +- `QuerySerializer` -## Frontend Architecture Overview +Documentation: +- `graphql-components/graphql-components.md` -```mermaid -flowchart TD - Router["React Router"] --> UrlState["useUrlState (Module 13)"] - UrlState --> ApiService["API Service (Module 14)"] - ApiService --> BackendAPI["Backend REST API"] +--- - BackendAPI --> Types["Core API Types (Module 15)"] - Types --> Enhanced["Enhanced Models (Module 16)"] - Enhanced --> Table["Contributors Table (Module 11–12)"] +### 7. Model Entities +**Path:** `backend/src/main/java/cx/flamingo/analysis/model` - Table --> Autocomplete["BaseAutocomplete (Module 10)"] - Table --> Pagination["Pagination (Module 12)"] -``` +Domain models: + +- `Contributor` +- `City` +- `Region` +- `State` +- `SoccerTeam` +- `Language` +- `ApiResponse` +- Hiring models + +Documentation: +- `model-entities/model-entities.md` --- -# Caching & Performance Model +### 8. Rate Management +**Path:** `backend/src/main/java/cx/flamingo/analysis/rate` -Major League GitHub aggressively caches data to: +GitHub token orchestration: -- Minimize GitHub API rate pressure -- Reduce latency -- Enable scalable horizontal deployments +- Multi-token pooling +- Primary + secondary rate limit handling +- Intelligent wait and retry logic -## Cache Flow +Documentation: +- `rate-management/rate-management.md` -```mermaid -flowchart TD - Request["Incoming Request"] --> CacheCheck["CacheServiceAbs.get()"] - CacheCheck --> Exists{"Entry Exists?"} - Exists -->|"No"| Fetch["Fetch From GitHub"] - Exists -->|"Yes"| Stale{"Is Stale?"} - Stale -->|"No"| Return["Return Cached Data"] - Stale -->|"Yes"| AsyncRefresh["Async Background Refresh"] - Fetch --> Store["Store In Cache"] - Store --> Return -``` +--- -Supports: +## Frontend Modules -- Disk cache (local/dev) -- Redis distributed cache (production) -- Read-only Redis mode -- Scheduled pre-warming (Module 9) +### 1. Frontend Components +**Path:** `frontend/src/components` + +- `BaseAutocomplete` +- `LanguageAutocomplete` +- `ContributorsTable` +- `Pagination` + +Documentation: +- `frontend-components/frontend-components.md` --- -# Repository Structure Overview +### 2. Frontend Hooks +**Path:** `frontend/src/hooks` -The project is modularized into 18 logical modules: +- `useNearestRegion` (Haversine proximity) +- `useUrlState` (validated URL-driven filtering) -## Backend Core +Documentation: +- `frontend-hooks/frontend-hooks.md` -- **Module 1** – Application bootstrap + cache abstraction -- **Module 2** – Redis + async configuration -- **Module 3** – Infrastructure config (Redis, CORS, scheduling) -- **Module 4** – REST controllers -- **Module 5** – GitHub GraphQL builder -- **Module 6–7** – Domain models -- **Module 8** – GitHub service + rate limiting + scoring -- **Module 9** – Hiring, language, pre-cache services -- **Module 10** – Region, state, soccer team services +--- -## Frontend Core +### 3. Frontend Services +**Path:** `frontend/src/services` -- **Module 11–12** – Contributors table + UI contracts -- **Module 13** – URL state + geolocation hooks -- **Module 14** – API integration layer -- **Module 15** – Core frontend types -- **Module 16–17** – Enhanced + hiring types -- **Module 18** – SEO Webpack plugin +Centralized Axios-based API layer. + +Documentation: +- `frontend-services/frontend-services.md` --- -# Deployment Architecture +### 4. Frontend Types +**Path:** `frontend/src/types` + +Type contracts mirroring backend domain models. + +Documentation: +- `frontend-types/frontend-types.md` + +--- + +### 5. Webpack Plugins +**Path:** `frontend/webpack-plugins` + +Custom build-time plugins: + +- `FaviconGeneratorPlugin` +- `SeoFilesPlugin` + +Documentation: +- `webpack-plugins/webpack-plugins.md` + +--- + +# 5. Core Contributor Ranking Flow + +The heart of the system is the GitHub ranking engine. + +```mermaid +flowchart TD + Request["Contributor Search Request"] --> CacheCheck["CacheServiceAbs.getHttpResponse()"] + CacheCheck -->|Miss| GithubFetch["GithubService"] + GithubFetch --> QueryBuilder["GitHubQueryBuilder"] + QueryBuilder --> GitHubAPI["GitHub GraphQL API"] + GitHubAPI --> Parse["Parse & Map to Contributor"] + Parse --> Score["Apply Scoring Formula"] + Score --> Store["Store in Cache"] + Store --> Response["ApiResponse>"] + CacheCheck -->|Hit| Response +``` + +### Scoring Formula + +```text +score = commits × max(starsReceived, 1) × recencyMultiplier +``` + +Recency multiplier rewards contributors active within the past year. + +--- + +# 6. Deployment Model ```mermaid flowchart LR - GitHubActions["GitHub Actions CI/CD"] - GitHubActions --> Docker["Docker Images"] + GitHubRepo["GitHub Repository"] --> CI["GitHub Actions CI/CD"] + CI --> Docker["Docker Images"] Docker --> GKE["Google Kubernetes Engine"] - - GKE --> BackendPod["Backend Service (8450)"] - GKE --> CacheUpdaterPod["Cache Updater (8451)"] + GKE --> BackendPods["Backend + Cache Updater Pods"] GKE --> RedisPod["Redis"] - - BackendPod --> RedisPod - CacheUpdaterPod --> RedisPod + GKE --> FrontendService["Frontend Service"] ``` +- Containerized services +- Horizontally scalable API nodes +- Independent scaling of cache updater +- Redis as shared distributed cache + --- -# Core Design Principles +# 7. Design Principles -- **Separation of concerns** – Controllers, services, caching, and models are isolated. -- **Strong typing end-to-end** – Java DTOs ↔ TypeScript interfaces. -- **Cache-first architecture** – Async refresh prevents latency spikes. -- **Multi-token GitHub rate management** – Resilient API usage. -- **Geospatial gamification** – Haversine distance for stadium proximity. -- **URL-driven state** – Fully shareable leaderboard filters. -- **Build-time optimization** – SEO and favicon plugins via Webpack. +- **Modular Backend Architecture** +- **Strong Type Contracts (Backend + Frontend)** +- **Distributed Cache Abstraction** +- **Multi-Token GitHub Rate Management** +- **URL-Driven Frontend State** +- **Geographic + MLS-Based Segmentation** +- **Build-Time SEO Automation** --- # Summary -`major-league-github` is a full-stack, production-grade analytics platform that transforms GitHub contributor data into a sports-style leaderboard experience. +Major League GitHub is a full-stack, microservice-based platform that transforms GitHub contribution data into a sports-themed leaderboard experience. It combines: -- Advanced GitHub GraphQL query generation -- Distributed caching and rate-limit management -- Geospatial modeling and MLS-themed gamification -- Strongly typed frontend architecture -- Automated SEO and asset generation -- Kubernetes-native deployment +- GitHub GraphQL data +- Intelligent rate-limit orchestration +- Distributed caching +- Geographic modeling +- MLS stadium proximity logic +- React-driven interactive filtering -The result is a scalable, performant, and highly modular system that ranks open-source developers like professional athletes — filtered by language, geography, and stadium proximity. \ No newline at end of file +The repository is structured into clearly separated backend and frontend modules, each documented independently, making it scalable, testable, and production-ready. \ No newline at end of file diff --git a/docs/reference/architecture/application-core/application-core.md b/docs/reference/architecture/application-core/application-core.md new file mode 100644 index 0000000..ac98992 --- /dev/null +++ b/docs/reference/architecture/application-core/application-core.md @@ -0,0 +1,246 @@ +# Application Core + +The **Application Core** module is the entry point and central bootstrap layer of the Major League GitHub backend system. It initializes the Spring Boot runtime, activates cross-cutting infrastructure features such as caching and asynchronous execution, and wires together all backend submodules including controllers, services, caching, rate management, and configuration components. + +At the heart of this module is the `MajorLeagueGithubApplication` class, which defines the application boundary and enables foundational Spring capabilities used across the platform. + +--- + +## 1. Purpose and Responsibilities + +The Application Core module is responsible for: + +- Bootstrapping the Spring Boot application context +- Enabling distributed caching across backend services +- Enabling asynchronous task execution +- Registering and scanning all submodules +- Acting as the composition root for the backend microservice + +It does **not** contain business logic. Instead, it orchestrates and activates the functional modules listed below. + +--- + +## 2. Core Component + +### MajorLeagueGithubApplication + +Located at: + +```text +backend/src/main/java/cx/flamingo/analysis/MajorLeagueGithubApplication.java +``` + +Key annotations: + +```java +@SpringBootApplication +@EnableCaching +@EnableAsync +public class MajorLeagueGithubApplication { + public static void main(String[] args) { + SpringApplication.run(MajorLeagueGithubApplication.class, args); + } +} +``` + +### Annotation Breakdown + +- `@SpringBootApplication` + - Enables component scanning + - Activates auto-configuration + - Registers configuration classes + +- `@EnableCaching` + - Activates Spring’s cache abstraction + - Integrates with implementations from the [Cache Services](cache-services/cache-services.md) module + +- `@EnableAsync` + - Enables `@Async` execution + - Works with thread pool definitions in [Configurations](configurations/configurations.md) + +--- + +## 3. High-Level Architecture + +The Application Core sits at the center of the backend service and wires together all major modules. + +```mermaid +flowchart TD + AppCore["Application Core"] + + Controllers["Controllers"] + Services["Backend Services"] + Cache["Cache Services"] + Rate["Rate Management"] + Config["Configurations"] + Models["Model Entities"] + GraphQL["GraphQL Components"] + + AppCore --> Controllers + AppCore --> Services + AppCore --> Cache + AppCore --> Rate + AppCore --> Config + AppCore --> Models + AppCore --> GraphQL + + Controllers --> Services + Services --> Cache + Services --> Rate + Services --> GraphQL + Services --> Models +``` + +--- + +## 4. Module Integration Map + +The Application Core composes the following backend modules: + +- [Cache Services](cache-services/cache-services.md) +- [Configurations](configurations/configurations.md) +- [Controllers](controllers/controllers.md) +- [GraphQL Components](graphql-components/graphql-components.md) +- [Model Entities](model-entities/model-entities.md) +- [Rate Management](rate-management/rate-management.md) +- [Backend Services](backend-services/backend-services.md) + +It also interacts indirectly with frontend modules via REST APIs exposed by the Controllers layer: + +- [Frontend Components](frontend-components/frontend-components.md) +- [Frontend Hooks](frontend-hooks/frontend-hooks.md) +- [Frontend Services](frontend-services/frontend-services.md) +- [Frontend Types](frontend-types/frontend-types.md) + +--- + +## 5. Application Startup Flow + +The startup lifecycle is managed by Spring Boot and follows this sequence: + +```mermaid +sequenceDiagram + participant JVM as JVM + participant Spring as SpringApplication + participant Context as ApplicationContext + participant Config as Configuration Beans + participant Services as Service Beans + participant Controllers as Controller Beans + + JVM->>Spring: main(args) + Spring->>Context: Create ApplicationContext + Context->>Config: Initialize configuration classes + Context->>Services: Instantiate service beans + Context->>Controllers: Instantiate controllers + Context-->>Spring: Application Ready +``` + +--- + +## 6. Caching Enablement + +Because `@EnableCaching` is declared at the Application Core level, all beans across the system can leverage Spring’s caching abstraction. + +```mermaid +flowchart LR + Service["Backend Service"] -->|"@Cacheable"| CacheAbstraction["Spring Cache Abstraction"] + CacheAbstraction --> Redis["Redis Cache"] + CacheAbstraction --> Disk["Disk Cache"] + CacheAbstraction --> ReadOnly["Read Only Cache"] +``` + +Concrete implementations are defined in the [Cache Services](cache-services/cache-services.md) module. + +--- + +## 7. Asynchronous Execution Model + +With `@EnableAsync`, services can execute long-running tasks without blocking request threads. + +```mermaid +flowchart TD + Controller["Controller"] --> Service["Service Method"] + Service -->|"@Async"| AsyncExecutor["Task Executor"] + AsyncExecutor --> BackgroundTask["Background Job"] +``` + +Thread pool configuration is provided by the [Configurations](configurations/configurations.md) module. + +--- + +## 8. Backend Request Flow + +A typical REST request flows through the system as follows: + +```mermaid +flowchart TD + Client["Frontend Client"] --> Controller["Controller"] + Controller --> Service["Backend Service"] + Service --> CacheCheck["Cache Layer"] + CacheCheck -->|"Miss"| GitHub["GitHub GraphQL API"] + GitHub --> Service + Service --> Model["Model Entities"] + Service --> ApiResponse["ApiResponse"] + ApiResponse --> Controller + Controller --> Client +``` + +Modules involved: + +- Controller logic: [Controllers](controllers/controllers.md) +- Business logic: [Backend Services](backend-services/backend-services.md) +- Data modeling: [Model Entities](model-entities/model-entities.md) +- GitHub integration: [GraphQL Components](graphql-components/graphql-components.md) +- Rate limiting: [Rate Management](rate-management/rate-management.md) + +--- + +## 9. Relationship to Cache Updater Service + +The repository also includes a cache updater microservice. While deployed separately, both services share configuration and caching concepts defined under: + +- [Cache Services](cache-services/cache-services.md) +- [Configurations](configurations/configurations.md) + +The Application Core described here represents the **Backend Service runtime (port 8450)**. + +--- + +## 10. Architectural Role in the System + +In the broader Major League GitHub architecture: + +- The Application Core initializes the backend microservice. +- Controllers expose REST endpoints. +- Services orchestrate GitHub data retrieval and enrichment. +- Caching and rate management protect API quotas. +- Frontend modules consume the exposed APIs. + +```mermaid +flowchart LR + Frontend["React Frontend"] --> Backend["Application Core (Spring Boot)"] + Backend --> GitHub["GitHub API"] + Backend --> Redis["Redis"] +``` + +--- + +## 11. Key Design Principles + +- **Composition Root Pattern** – All infrastructure wiring begins here. +- **Separation of Concerns** – No business logic inside the bootstrap class. +- **Annotation-Driven Infrastructure** – Caching and async behavior are declarative. +- **Modular Backend Architecture** – Functional modules remain isolated and testable. + +--- + +## Summary + +The **Application Core** module is the foundational bootstrap layer of the Major League GitHub backend. While small in code footprint, it activates the entire application ecosystem: + +- Spring Boot auto-configuration +- Distributed caching +- Asynchronous execution +- Dependency injection and module composition + +All backend functionality ultimately depends on the initialization performed by this module. \ No newline at end of file diff --git a/docs/reference/architecture/backend-services/backend-services.md b/docs/reference/architecture/backend-services/backend-services.md new file mode 100644 index 0000000..2493266 --- /dev/null +++ b/docs/reference/architecture/backend-services/backend-services.md @@ -0,0 +1,321 @@ +# Backend Services + +## Overview + +The **Backend Services** module contains the core business logic of Major League GitHub. It orchestrates data loading, GitHub and LinkedIn API integrations, caching, geographic modeling, scoring algorithms, and pre-computation workflows. + +This module sits between the Controllers layer and foundational infrastructure such as: + +- Cache Services +- GraphQL Components +- Rate Management +- Model Entities + +It is responsible for transforming raw external API data and static datasets into structured domain models such as `Contributor`, `City`, `Region`, `State`, `SoccerTeam`, and `JobOpening`. + +--- + +## Architectural Role + +The Backend Services module acts as the domain orchestration layer. + +```mermaid +flowchart TD + Controller["Controllers"] --> Services["Backend Services"] + Services --> Cache["Cache Services"] + Services --> GraphQL["GraphQL Components"] + Services --> Rate["Rate Management"] + Services --> Models["Model Entities"] + Services --> ExternalGitHub["GitHub GraphQL API"] + Services --> ExternalLinkedIn["LinkedIn API"] +``` + +### Responsibilities + +- Domain aggregation (City, Region, State relationships) +- GitHub GraphQL query orchestration +- Rate limit handling and token switching +- Contributor scoring logic +- Hiring manager profile composition +- LinkedIn job integration +- Cache warm-up and pre-computation + +--- + +# Service Components + +## 1. GithubService + +**Primary responsibility:** Retrieve, score, and aggregate GitHub contributor data. + +### Key Capabilities + +- Builds GraphQL queries using `GitHubQueryBuilder` +- Executes requests using `WebClient` +- Manages concurrency via separate executors +- Handles: + - Rate limits + - Timeouts + - Token rotation + - Retry strategies +- Deduplicates contributors across cities +- Calculates ranking score + +### Scoring Formula + +```text +score = commits × max(starsReceived, 1) × recencyMultiplier +``` + +Recency multiplier ranges from 1.0 to 2.0 depending on activity within the past year. + +### Contributor Retrieval Flow + +```mermaid +flowchart TD + Start["Request Contributors"] --> BuildQuery["Build GraphQL Query"] + BuildQuery --> Execute["Execute via WebClient"] + Execute --> RateCheck{"Rate Limited?"} + RateCheck -->|Yes| SwitchToken["Switch Token"] + RateCheck -->|No| Parse["Parse JSON Response"] + SwitchToken --> Execute + Parse --> Process["Process Users"] + Process --> Score["Calculate Score"] + Score --> Return["Return Ranked Contributors"] +``` + +--- + +## 2. CityService + +**Responsibility:** Load and manage city metadata. + +- Loads `cities.csv` on startup +- Associates cities with: + - States + - Regions + - Nearest Soccer Team +- Provides filtering by: + - State + - Region + - Team + +Cities are sorted by population when autocompleting. + +--- + +## 3. StateService + +**Responsibility:** Manage U.S. state metadata. + +- Loads `states.csv` +- Calculates total population dynamically from cities +- Supports filtering by region and city + +Uses lazy injection to avoid circular dependency with `CityService`. + +--- + +## 4. RegionService + +**Responsibility:** Geographic region modeling. + +- Loads `regions.csv` +- Associates: + - State IDs + - Cities (resolved later) +- Sorts by total regional population + +Works closely with `ReferencePopulationService`. + +--- + +## 5. ReferencePopulationService + +**Responsibility:** Populate bidirectional entity references after initialization. + +This service ensures: + +- Regions contain fully populated `State` objects +- Regions contain full `City` objects +- States reference regions + +### Population Flow + +```mermaid +flowchart TD + Init["PostConstruct Init"] --> LoadRegions["Get All Regions"] + LoadRegions --> ResolveStates["Resolve States by Code"] + ResolveStates --> ResolveCities["Resolve Cities by Region"] + ResolveCities --> UpdateRegion["Update Region in RegionService"] +``` + +--- + +## 6. SoccerTeamService + +**Responsibility:** MLS team modeling and proximity calculations. + +- Loads `teams.csv` +- Computes nearest team using Haversine distance +- Enables geographic-based filtering + +Distance formula uses Earth radius = 6371 km. + +--- + +## 7. LanguageService + +**Responsibility:** Programming language metadata. + +- Loads `languages.csv` +- Provides autocomplete +- Defines default language (Java) + +Used heavily by `GithubService` and controllers. + +--- + +## 8. HiringService + +**Responsibility:** Hiring manager profile aggregation and job listings. + +### Profile Flow + +```mermaid +flowchart TD + Request["Get Hiring Profile"] --> CacheCheck{"In Cache?"} + CacheCheck -->|Yes| ReturnCached["Return Cached Profile"] + CacheCheck -->|No| FetchGitHub["Fetch via GithubService"] + FetchGitHub --> BuildProfile["Build HiringManagerProfile"] + BuildProfile --> StoreCache["Cache Result"] + StoreCache --> ReturnProfile["Return Profile"] +``` + +### Job Openings + +- Retrieves jobs via `LinkedInService` +- Falls back to predefined remote roles if API fails +- Cached with configurable refresh interval + +--- + +## 9. LinkedInService + +**Responsibility:** LinkedIn job posting integration. + +### Flow + +```mermaid +flowchart TD + Start["Get Company Job Postings"] --> Token["Request OAuth Token"] + Token --> FetchUpdates["Fetch Organization Updates"] + FetchUpdates --> FilterJobs["Filter jobPosting Content"] + FilterJobs --> MapJobs["Map to JobOpening Model"] + MapJobs --> Cache["Store in Cache"] +``` + +Features: + +- OAuth client credentials flow +- 10-second timeout protection +- Caching layer abstraction + +--- + +## 10. PreCacheService + +**Responsibility:** Warm up cache on application startup. + +- Runs scheduled task immediately on startup +- Iterates through all languages +- Triggers contributor retrieval +- Marks cache as ready when finished + +### Cache Warm-Up Flow + +```mermaid +flowchart TD + Startup["Application Startup"] --> LoopLanguages["Iterate Languages"] + LoopLanguages --> Trigger["Call ContributorController"] + Trigger --> GithubFetch["GithubService Fetch"] + GithubFetch --> CacheStore["Cache Responses"] + CacheStore --> Complete["Mark Cache Ready"] +``` + +--- + +# Concurrency & Rate Management + +`GithubService` uses: + +- Configurable concurrency (`github.api.concurrency`) +- Two executors: + - High priority + - Low priority +- `GithubTokenRateManager` for token rotation + +```mermaid +flowchart TD + Request["City Batch"] --> Async["CompletableFuture Execution"] + Async --> TokenManager["GithubTokenRateManager"] + TokenManager --> WebClient["WebClient Call"] + WebClient --> CacheLayer["CacheServiceAbs"] + CacheLayer --> Response["Parsed JSON"] +``` + +--- + +# Data Initialization Strategy + +Most services load static CSV data during `@PostConstruct`: + +- Cities +- States +- Regions +- Soccer Teams +- Languages + +This ensures: + +- No database dependency +- Predictable in-memory dataset +- Fast lookups + +--- + +# How Backend Services Fit the System + +```mermaid +flowchart LR + Frontend["Frontend Application"] --> Controllers["REST Controllers"] + Controllers --> Backend["Backend Services"] + Backend --> CacheLayer["Cache Services"] + Backend --> ExternalAPIs["External APIs"] + Backend --> StaticData["CSV Data"] +``` + +Backend Services provide: + +- Ranking engine +- Geographic intelligence +- Hiring integration +- Caching orchestration +- Rate-limit resilience + +They form the computational heart of the application. + +--- + +# Summary + +The **Backend Services** module: + +- Aggregates geographic and contributor data +- Communicates with GitHub and LinkedIn APIs +- Applies ranking and scoring algorithms +- Maintains cache consistency +- Warms data on startup +- Resolves complex entity relationships + +It is the central domain engine powering contributor rankings and hiring visibility in Major League GitHub. \ No newline at end of file diff --git a/docs/reference/architecture/cache-services/cache-services.md b/docs/reference/architecture/cache-services/cache-services.md index 2bc445c..5405162 100644 --- a/docs/reference/architecture/cache-services/cache-services.md +++ b/docs/reference/architecture/cache-services/cache-services.md @@ -1,293 +1,357 @@ # Cache Services -The **Cache Services** module provides a unified, extensible caching abstraction for the Major League GitHub backend. It centralizes cache key generation, staleness detection, refresh strategies, and storage implementations (Disk, Redis, and Read-Only Redis). +The **Cache Services** module provides a pluggable, environment-aware caching abstraction for the Major League GitHub backend. It is responsible for: -This module plays a critical role in: +- Reducing load on the GitHub GraphQL API +- Minimizing repeated HTTP computations for contributor queries +- Supporting multiple cache backends (Disk and Redis) +- Enabling read-only and force-update operational modes +- Managing cache freshness and asynchronous refresh -- Reducing GitHub API calls -- Improving HTTP response times -- Supporting pre-warmed production caches -- Enabling environment-specific cache modes (read-write, force update, read-only) - -Cache Services is primarily consumed by the Service Layer (e.g., `GithubService`, `PreCacheService`) and indirectly supports Controllers and frontend requests. +This module sits between the **Backend Services** layer and external systems (GitHub API, Redis, filesystem), acting as a performance and resilience layer. --- -## Architectural Overview +## 1. Architectural Overview + +At a high level, Cache Services defines a common abstraction (`CacheServiceAbs`) and multiple concrete implementations: + +- `DiskCacheService` – File-based caching +- `RedisCacheService` – Distributed Redis-based caching +- `ReadOnlyCacheService` – Redis-backed, read-only variant ```mermaid -flowchart TD - Controller["Controllers"] --> ServiceLayer["Service Layer"] - ServiceLayer --> CacheService["CacheServiceAbs"] +flowchart LR + Controllers["Controllers"] --> Services["Backend Services"] + Services --> CacheAbs["CacheServiceAbs (Abstract)"] - CacheService --> DiskCache["Disk Cache Service"] - CacheService --> RedisCache["Redis Cache Service"] - RedisCache --> ReadOnlyCache["Read Only Cache Service"] + CacheAbs --> Disk["DiskCacheService"] + CacheAbs --> Redis["RedisCacheService"] + Redis --> ReadOnly["ReadOnlyCacheService"] - DiskCache --> FileSystem[("File System")] - RedisCache --> Redis[("Redis")] + Disk --> FS[("File System")] + Redis --> RedisDB[("Redis")] + Services --> GitHub[("GitHub API")] ``` -### Design Principles +### Key Responsibilities -1. **Abstraction First** – `CacheServiceAbs` defines the contract and refresh semantics. -2. **Storage Agnostic** – Implementations can store data in disk or Redis. -3. **Asynchronous Refresh** – Stale entries are refreshed in the background. -4. **Environment-Aware** – Read-only mode prevents accidental cache mutation. -5. **Key Normalization** – Deterministic composite keys ensure consistent cache hits. +| Layer | Responsibility | +|-------|----------------| +| CacheServiceAbs | Core caching workflow, key generation, staleness logic | +| DiskCacheService | Persistent JSON file-based storage | +| RedisCacheService | Distributed cache using Redis | +| ReadOnlyCacheService | Safe read-only cache access (e.g., web profile) | --- -## Core Abstraction: CacheServiceAbs - -The backbone of the module is: +## 2. Core Abstraction: CacheServiceAbs -- `CacheServiceAbs` -- `CachedResponse` +**Core Component:** +`major-league-github.backend.src.main.java.cx.flamingo.analysis.cache.CacheServiceAbs.CacheServiceAbs` This abstract class defines: -- Cache read/write contract -- Staleness detection - Cache key generation -- GitHub-specific caching strategy -- HTTP query result caching -- Async refresh handling -- Cache readiness state -- Cache mode (READ_WRITE, FORCE_UPDATE) +- Staleness detection +- Async refresh workflow +- GitHub API–specific caching +- HTTP response caching for contributor queries +- Cache readiness checks +- Cache mode switching (READ_WRITE vs FORCE_UPDATE) -### High-Level Responsibilities +### 2.1 Caching Workflow ```mermaid -flowchart LR - Request["Incoming Request"] --> KeyGen["Key Generation"] - KeyGen --> CacheLookup["Cache Lookup"] - CacheLookup -->|"Hit"| ReturnCached["Return Cached Data"] - CacheLookup -->|"Miss"| Supplier["Execute Supplier"] - Supplier --> Store["Store in Cache"] - Store --> ReturnFresh["Return Fresh Data"] - - CacheLookup -->|"Stale"| AsyncRefresh["Async Refresh"] -``` +flowchart TD + Request["Incoming Request"] --> CheckMode{"FORCE_UPDATE?"} + CheckMode -->|"No"| TryCache["Attempt Cache Read"] + CheckMode -->|"Yes"| Fetch -### Key Features + TryCache --> Hit{"Cache Hit?"} + Hit -->|"Yes"| Stale{"Stale?"} + Hit -->|"No"| Fetch -#### 1. Deterministic Cache Key Generation + Stale -->|"Yes"| AsyncRefresh["Async Refresh"] + Stale -->|"No"| ReturnCached["Return Cached Value"] -Composite keys are built from filtering dimensions: + AsyncRefresh --> ReturnCached -- City -- Region -- State -- Team -- Language -- Page number or max results + Fetch["Execute Supplier (HTTP/GitHub)"] --> Store["Put in Cache"] + Store --> ReturnFresh["Return Fresh Value"] +``` -Each implementation defines its own delimiter: +### 2.2 Key Concepts -- Disk: `/` -- Redis: `:` +#### 1. Supplier-Based Execution +Cache retrieval methods accept a `Supplier`: -#### 2. Staleness Detection +- If cache hit → return cached value +- If cache miss → execute supplier +- Store result in cache +- Return fresh data -Cache entries are evaluated against configurable intervals: +This pattern ensures backend services remain cache-agnostic. -- `github.cache.refresh.interval` -- `http.cache.refresh.interval` -- `cache.expiration.ms` +#### 2. Staleness Detection +Each entry stores an insertion timestamp (backend-dependent). +Staleness is determined by comparing: + +- `System.currentTimeMillis()` +- Insert timestamp +- Configured refresh interval If stale: +- Return current cached value +- Trigger asynchronous refresh -- Cached value is returned immediately -- Refresh is executed asynchronously -- New value replaces old entry only after successful retrieval +This enables **non-blocking refresh** behavior. -This ensures zero-downtime cache refresh. +#### 3. Specialized Cache Methods -#### 3. Async Refresh +Two primary entry points: -```mermaid -sequenceDiagram - participant Client - participant Cache - participant Supplier - - Client->>Cache: getHttpResponse() - Cache->>Cache: Check staleness - Cache-->>Client: Return stale value - Cache->>Supplier: Async refresh - Supplier-->>Cache: Fresh data - Cache->>Cache: Overwrite entry -``` +- `getGitHubApiResponse(...)` +- `getHttpResponse(...)` + +They differ in: + +| Method | Cached Data | Key Composition | +|--------|------------|----------------| +| GitHub API | Raw `JsonObject` | City + Language + Page | +| HTTP Response | `List` | City + Region + State + Team + Language + MaxResults | + +--- -#### 4. Cache Readiness Flag +## 3. DiskCacheService -The cache readiness mechanism allows environments to: +**Core Component:** +`major-league-github.backend.src.main.java.cx.flamingo.analysis.cache.impl.DiskCacheService.DiskCacheService` -- Delay traffic until cache warm-up completes -- Ensure pre-cached Redis data is available +Provides filesystem-based caching using JSON files. -A special key path is used internally to track readiness. +### 3.1 Storage Model -#### 5. Cache Modes +- Each cache entry → `/.json` +- Insert time derived from file last-modified timestamp +- Directories auto-created at startup -- **READ_WRITE** – Default behavior -- **FORCE_UPDATE** – Always bypass cache +```mermaid +flowchart TD + Put["put(cachePath, key, value)"] --> Serialize["Serialize to JSON"] + Serialize --> WriteFile["Write .json"] + + Get["get(cachePath, key)"] --> Exists{"File Exists?"} + Exists -->|"No"| Miss["Cache Miss"] + Exists -->|"Yes"| Stale{"Stale?"} + + Stale -->|"Yes"| Delete["Delete File"] + Delete --> Miss -Mode is controlled via `CacheConfig.CacheMode`. + Stale -->|"No"| Read["Read JSON"] + Read --> Deserialize["Gson.fromJson"] + Deserialize --> Hit["Return Value"] +``` + +### 3.2 Characteristics + +✅ Simple and transparent +✅ Good for local development +✅ Persistent across restarts +❌ Not distributed +❌ Slower than Redis under load --- -## Storage Implementations +## 4. RedisCacheService -The module includes three concrete implementations: +**Core Component:** +`major-league-github.backend.src.main.java.cx.flamingo.analysis.cache.impl.RedisCacheService.RedisCacheService` -### 1. Disk Cache Service +Provides distributed caching using Redis. -Documentation: [Disk Cache Service](cache-services/disk_cache_service/disk_cache_service.md) +### 4.1 Key Structure -- Stores cache entries as JSON files -- Uses file modification timestamp for staleness -- Deletes corrupted or stale files -- Ideal for local development +Keys are structured as: -### 2. Redis Cache Service +``` +: +``` -Documentation: [Redis Cache Service](cache-services/redis_cache_service/redis_cache_service.md) +An additional expiration key is stored: -- Stores serialized JSON values in Redis -- Uses a separate expiration metadata key -- Suitable for distributed deployments -- Enables Kubernetes scaling +``` +::expiration +``` -### 3. Read Only Cache Service +The expiration key stores a serialized `Expiration` object containing: -Documentation: [Read Only Cache Service](cache-services/read_only_cache_service/read_only_cache_service.md) +- `timestamp` -- Extends Redis Cache Service -- Disables writes and invalidation -- Always returns cached values if present -- Used for production web profile with pre-warmed cache +### 4.2 Storage Workflow ---- +```mermaid +flowchart LR + Put["put(cachePath, key)"] --> BuildKey["Build Redis Key"] + BuildKey --> StoreValue["SET key -> JSON"] + StoreValue --> StoreMeta["SET key:expiration -> timestamp"] + + Get["get(cachePath, key)"] --> Fetch["GET key"] + Fetch --> Deserialize["Gson.fromJson"] + Deserialize --> Return +``` -## Integration with Other Modules +### 4.3 Characteristics -### Service Layer +✅ Distributed and scalable +✅ Fast read/write +✅ Suitable for production +❌ Requires external Redis instance -Cache Services is primarily consumed by: +--- + +## 5. ReadOnlyCacheService -- `GithubService` -- `PreCacheService` -- `LanguageService` -- `RegionService` +**Core Component:** +`major-league-github.backend.src.main.java.cx.flamingo.analysis.cache.impl.ReadOnlyCacheService.ReadOnlyCacheService` -These services pass supplier functions that: +Extends `RedisCacheService` but disables all write operations. -- Call GitHub GraphQL -- Aggregate contributor results -- Transform API responses +### 5.1 Behavior Changes -### Rate Management +| Operation | Behavior | +|-----------|----------| +| `put()` | Ignored | +| `invalidate()` | Ignored | +| `get()` | Returns value regardless of refresh interval | -Caching significantly reduces pressure on: +### 5.2 Use Case -- `GithubTokenRateManager` +Designed for: + +- Web profile deployments +- Read-only production replicas +- Environments where cache mutation is restricted This ensures: -- Lower GitHub rate consumption -- Fewer token rotations -- More predictable API behavior +- No accidental writes +- No cache invalidation +- Safe consumption of pre-populated cache -### Controllers +--- -Controllers indirectly benefit via: +## 6. Cache Modes -- `ContributorController` -- `AutocompleteController` +Cache mode is defined via `CacheConfig.CacheMode`. -Since they rely on cached service results. +Supported modes: ---- +- `READ_WRITE` (default) +- `FORCE_UPDATE` -## Cache Key Strategy +### FORCE_UPDATE Mode -### HTTP Query Cache Key Structure +If enabled: -```text -cityId/regionId/stateId/teamId/languageId/maxResults -``` +- Cache is bypassed +- Supplier always executes +- Cache entry is replaced -### GitHub API Cache Key Structure +```mermaid +flowchart TD + Request --> Mode{"Mode == FORCE_UPDATE?"} + Mode -->|"Yes"| Execute["Execute Supplier"] + Execute --> Store + Store --> Return -```text -/delimiter/cityId/language/page_X + Mode -->|"No"| Normal["Normal Cache Flow"] ``` -The delimiter differs by implementation. +This is useful for: -This design guarantees: - -- Stable key generation -- Environment portability -- Cross-instance consistency +- Manual cache refresh +- Batch pre-caching jobs +- Operational recovery --- -## Environment Profiles +## 7. Cache Readiness Flag + +Cache Services supports a readiness mechanism: + +- Path: `cache_is_ready` +- Key: `cache_is_ready` + +If `cache.should.be.ready=true`: -| Environment | Implementation | Purpose | -|-------------|---------------|----------| -| Local Dev | Disk Cache | Easy inspection | -| Kubernetes | Redis Cache | Distributed caching | -| Web Profile | Read Only Redis | Pre-warmed production cache | +- Application checks readiness before serving traffic +- Useful for pre-warmed production environments --- -## Failure Handling Strategy +## 8. Interaction with Other Modules -Cache Services is defensive by design: +### Backend Services -- Corrupted entries are deleted -- Deserialization failures invalidate entries -- Missing insert times default to stale -- Supplier exceptions do not crash request flow +Backend Services use Cache Services indirectly via supplier-based calls: -If cache lookup fails, the system gracefully falls back to supplier execution. +- `GithubService` → uses GitHub API cache +- Contributor queries → use HTTP response cache + +Cache Services ensures that business logic remains clean and does not directly depend on Redis or filesystem logic. + +### Configurations + +Configuration values influence behavior: + +- `github.cache.refresh.interval` +- `http.cache.refresh.interval` +- `cache.expiration.ms` +- `github.cache.path` +- `http.cache.path` + +These are injected via Spring `@Value` properties. --- -## Extension Points +## 9. Design Principles + +### 1. Backend-Agnostic Abstraction +All cache consumers depend only on `CacheServiceAbs`. + +### 2. Non-Blocking Refresh +Stale entries are refreshed asynchronously while serving existing data. + +### 3. Environment Flexibility -To add a new cache implementation: +| Environment | Implementation | +|------------|---------------| +| Local Development | DiskCacheService | +| Production | RedisCacheService | +| Web Profile (Read-Only) | ReadOnlyCacheService | -1. Extend `CacheServiceAbs` -2. Implement: - - `get()` - - `put()` - - `invalidate()` - - `getInsertTime()` - - `getHttpCachePath()` - - `getGithubCachePath()` -3. Register as a Spring `@Service` +### 4. Graceful Degradation -This enables alternative storage backends such as: +If: +- Deserialization fails +- File is corrupted +- Redis entry malformed -- In-memory caches -- Cloud storage buckets -- Hybrid layered caches +The system: +- Invalidates entry +- Falls back to supplier --- -## Summary +## 10. Summary -The **Cache Services** module is a foundational backend component that: +The **Cache Services** module is a foundational performance layer in the backend architecture. It: -- Shields the system from excessive GitHub API calls -- Provides consistent cache semantics across storage types -- Supports async refresh and stale-while-revalidate patterns -- Enables production-grade distributed caching with Redis -- Allows read-only deployment models +- Abstracts cache storage behind a unified interface +- Supports both local and distributed backends +- Enables asynchronous refresh for stale entries +- Provides operational modes for force update and read-only execution +- Maintains clean separation between business logic and infrastructure concerns -It acts as a performance accelerator, rate-limit shield, and reliability enhancer for the entire Major League GitHub platform. +It plays a critical role in ensuring the scalability and responsiveness of Major League GitHub, especially under heavy GitHub API usage and complex contributor filtering queries. diff --git a/docs/reference/architecture/configurations/configurations.md b/docs/reference/architecture/configurations/configurations.md index 77769d9..170c494 100644 --- a/docs/reference/architecture/configurations/configurations.md +++ b/docs/reference/architecture/configurations/configurations.md @@ -1,374 +1,330 @@ # Configurations -The **Configurations** module centralizes all Spring Boot configuration for the Major League GitHub backend. It defines infrastructure beans, environment profiles, caching strategy selection, Redis connectivity, JSON serialization behavior, asynchronous execution, scheduling, and CORS rules. +The **Configurations** module centralizes all Spring Boot configuration for the Major League GitHub backend. It defines infrastructure beans, environment profiles, cache selection strategies, asynchronous execution pools, Redis integration, JSON serialization behavior, and web-level cross-origin policies. -This module acts as the foundational wiring layer between: +This module acts as the wiring layer between: -- The Core Application bootstrap -- The Service Layer -- The Cache Services -- The Controllers -- The Cache Updater microservice +- [Application Core](../application-core/application-core.md) +- [Cache Services](../cache-services/cache-services.md) +- [Backend Services](../backend-services/backend-services.md) +- [Controllers](../controllers/controllers.md) +- [Rate Management](../rate-management/rate-management.md) -Rather than containing business logic, the Configurations module defines how components are instantiated, connected, and tuned for different runtime environments. +Rather than implementing business logic, the Configurations module ensures that all other modules are correctly initialized, connected, and parameterized based on environment and runtime properties. --- ## Architectural Overview -The Configurations module provides Spring beans that are consumed across the backend microservices. - ```mermaid flowchart TD - App["MajorLeagueGithubApplication"] --> Config["Configurations Module"] + App["Spring Boot Application"] --> Config["Configurations Module"] subgraph infra["Infrastructure Beans"] Async["AsyncConfig"] Cache["CacheConfig"] Redis["RedisConfig"] Web["WebConfig"] - Backend["BackendServiceConfig"] - Updater["CacheUpdaterConfig"] + end + + subgraph profiles["Profile-Specific"] + BackendProfile["BackendServiceConfig"] + UpdaterProfile["CacheUpdaterConfig"] end Config --> Async Config --> Cache Config --> Redis Config --> Web - Config --> Backend - Config --> Updater - - Cache --> CacheServices["Cache Services"] - Redis --> RedisServer[("Redis Server")] - Async --> Services["Service Layer"] - Web --> Controllers["Controllers"] - Updater --> Scheduler["Scheduled Jobs"] -``` + Config --> BackendProfile + Config --> UpdaterProfile -### Responsibilities - -The module is responsible for: - -- Selecting and configuring cache implementations -- Managing Redis connectivity and serialization -- Defining asynchronous thread pools for GitHub API concurrency -- Enabling profile-specific configuration for backend and cache-updater services -- Configuring CORS rules for frontend integration -- Providing JSON adapters for time-based objects - ---- - -## Configuration Classes + Cache --> CacheServices["Cache Services Module"] + Async --> BackendServices["Backend Services Module"] + Web --> Controllers["Controllers Module"] + Redis --> CacheServices +``` -The Configurations module consists of the following components: +The Configurations module: -| Class | Responsibility | -|--------|----------------| -| AsyncConfig | Thread pools for concurrent GitHub API calls | -| CacheConfig | Cache mode and implementation selection | -| CacheUpdaterConfig | Scheduling and profile config for cache updater service | -| BackendServiceConfig | Backend-specific profile configuration | -| RedisConfig | Redis connection, template, and Gson configuration | -| WebConfig | CORS and web layer configuration | -| LocalDateTimeAdapter | Gson adapter for LocalDateTime | -| InstantTypeAdapter | Gson adapter for Instant | +- Defines bean lifecycles +- Selects cache implementation dynamically +- Provides thread pools for concurrent GitHub API calls +- Enables scheduling for the cache updater service +- Configures Redis connectivity and JSON serialization +- Establishes global CORS policy --- -# Async Configuration - -## AsyncConfig - -The AsyncConfig class defines two thread pools used for GitHub contributor data processing. +# Core Configuration Areas -### Thread Pools +## 1. Asynchronous Execution -Two executors are defined: +**Component:** `AsyncConfig` -- contributorsAsyncExecutorLow -- contributorsAsyncExecutorHigh +The system performs heavy GitHub API queries and contributor aggregation. To prevent blocking HTTP request threads, the Configurations module provides two dedicated thread pools: -Both are instances of ThreadPoolTaskExecutor and are exposed as ThreadPoolExecutor beans. +- `contributorsAsyncExecutorLow` +- `contributorsAsyncExecutorHigh` -### Key Properties +### Thread Pool Characteristics -| Property | Value Source | -|-----------|-------------| -| Core pool size | github.api.concurrency property (default 10) | -| Max pool size | 100 | -| Queue capacity | 1000 | -| Await termination | 60 seconds | +- Core pool size configurable via property `github.api.concurrency` +- Maximum pool size: 100 +- Queue capacity: 1000 +- Graceful shutdown with timeout handling +- Custom thread name prefixes -The concurrency level is externally configurable using the property: +### Execution Model -```text -github.api.concurrency=10 +```mermaid +flowchart LR + Controller["Controller"] --> Service["Backend Service"] + Service -->|"Submit Task"| LowPool["Low Priority Executor"] + Service -->|"Submit Task"| HighPool["High Priority Executor"] + LowPool --> GithubAPI["GitHub API"] + HighPool --> GithubAPI ``` -### Graceful Shutdown - -The class implements a @PreDestroy lifecycle hook to: +The executors ensure: -- Shut down executors gracefully -- Wait up to 10 seconds for termination -- Force shutdown if necessary -- Preserve interruption status +- Parallel contributor fetching +- Controlled GitHub API concurrency +- Graceful shutdown during service termination -This ensures safe shutdown during Kubernetes pod termination or service restarts. +Shutdown logic is handled using `@PreDestroy`, guaranteeing proper cleanup of threads. --- -# Cache Configuration +## 2. Cache Strategy Configuration -## CacheConfig +**Component:** `CacheConfig` -The CacheConfig class dynamically selects the active cache implementation and cache mode at runtime. +This configuration determines which cache implementation is used at runtime and in what mode. -### Cache Modes +### Supported Cache Modes -```text -read-only -read-write -force-update -``` +- `read-only` +- `read-write` +- `force-update` -| Mode | Behavior | -|------|----------| -| read-only | No writes allowed; serves existing cache only | -| read-write | Normal cache behavior | -| force-update | Forces refresh behavior (used by updater) | +### Supported Implementations -### Cache Implementations +- `redis` +- `disk` -```text -redis -disk -``` +These map directly to the implementations in the [Cache Services](../cache-services/cache-services.md) module: -### Runtime Selection Logic +- `RedisCacheService` +- `DiskCacheService` +- `ReadOnlyCacheService` + +### Selection Flow ```mermaid flowchart TD - Start["Application Startup"] --> Mode["Read cache.mode"] - Mode --> Impl["Read cache.implementation"] - - Impl --> Decision{"Mode == read-only?"} - Decision -->|"Yes"| ReadOnly["Use ReadOnlyCacheService"] - Decision -->|"No"| ImplChoice{"Implementation"} - - ImplChoice -->|"redis"| RedisCache["Use RedisCacheService"] - ImplChoice -->|"disk"| DiskCache["Use DiskCacheService"] + Start["Application Startup"] --> ReadProps["Read cache.* Properties"] + ReadProps --> Mode{"Cache Mode?"} + Mode -->|"read-only"| ReadOnly["ReadOnlyCacheService"] + Mode -->|"read-write"| Impl{"Implementation?"} + Mode -->|"force-update"| Impl + Impl -->|"redis"| RedisImpl["RedisCacheService"] + Impl -->|"disk"| DiskImpl["DiskCacheService"] ``` -The selected implementation is exposed as the primary CacheServiceAbs bean and injected throughout the Service Layer. - -This design allows: +### Primary Bean -- Local development with disk cache -- Production deployment with Redis -- Safe read-only mode during maintenance -- Forced refresh behavior for cache-updater jobs - ---- +The `cacheService()` method is marked `@Primary`, ensuring: -# Redis Configuration +- Only one active `CacheServiceAbs` implementation is injected +- Backend services do not need to know which cache backend is active +- Switching between Redis and Disk requires only property changes -## RedisConfig +This design isolates infrastructure concerns from business logic. -The RedisConfig class defines Redis connectivity and JSON serialization. - -### Connection Factory - -Uses: - -- RedisStandaloneConfiguration -- LettuceConnectionFactory +--- -Configured via properties: +## 3. Redis Integration & Serialization -```text -spring.redis.host=localhost -spring.redis.port=6379 -``` +**Component:** `RedisConfig` -### RedisTemplate +Provides: -Configured with: +- `RedisConnectionFactory` +- `RedisTemplate` +- Customized `Gson` bean -- StringRedisSerializer for keys -- StringRedisSerializer for values -- Transaction support enabled +### Redis Connection -The template is used by RedisCacheService for storing serialized JSON objects. +Configuration is property-driven: -### Gson Configuration +- `spring.redis.host` +- `spring.redis.port` -A custom Gson bean is defined with: +Uses: -- LocalDateTimeAdapter -- InstantTypeAdapter +- `RedisStandaloneConfiguration` +- `LettuceConnectionFactory` -This ensures consistent serialization of time-based values stored in Redis. +### RedisTemplate Setup ```mermaid flowchart LR - Service["Service Layer"] --> Cache["RedisCacheService"] - Cache --> Template["RedisTemplate"] - Template --> Redis[("Redis")] - Cache --> Gson["Gson with Time Adapters"] + RedisConfig["RedisConfig"] --> Factory["RedisConnectionFactory"] + Factory --> Template["RedisTemplate"] + Template --> CacheService["RedisCacheService"] ``` ---- - -# JSON Time Adapters - -## LocalDateTimeAdapter - -Implements: - -- JsonSerializer -- JsonDeserializer +- String serializers for keys and values +- Transaction support enabled -Uses ISO_LOCAL_DATE_TIME format. +### JSON Time Handling -Ensures LocalDateTime values are consistently serialized and parsed. +To ensure consistent serialization of temporal values stored in cache: -## InstantTypeAdapter +- `LocalDateTimeAdapter` +- `InstantTypeAdapter` -Custom Gson TypeAdapter for Instant. +These adapters: -- Writes Instant as ISO-8601 string -- Parses string back into Instant -- Handles null values safely +- Serialize to ISO-8601 strings +- Ensure lossless deserialization +- Avoid timestamp timezone inconsistencies -These adapters prevent timezone inconsistencies and serialization failures when caching or returning API responses. +The `Gson` bean becomes the shared serializer across cache and services. --- -# Web Configuration +## 4. Web & CORS Configuration -## WebConfig +**Component:** `WebConfig` -Implements WebMvcConfigurer to configure CORS rules. +Implements `WebMvcConfigurer` to define global CORS policy. ### Allowed Origins -Development: - -- http://localhost:8450 -- http://localhost:3000 +- `http://localhost:8450` +- `http://localhost:3000` +- `https://www.mlg.soccer` +- `http://www.mlg.soccer` -Production: +### Enabled For -- https://www.mlg.soccer -- http://www.mlg.soccer +- All endpoints (`/**`) +- All standard HTTP methods +- Credentials allowed +- 1-hour preflight cache -### Allowed Methods - -- GET -- POST -- PUT -- DELETE -- OPTIONS - -### Additional Settings +```mermaid +flowchart LR + Browser["Frontend"] -->|"HTTP Request"| Backend["Backend Service"] + Backend -->|"CORS Validation"| WebConfig + WebConfig -->|"Allowed Origin"| Response["HTTP Response"] +``` -- Allow credentials: true -- Exposed headers: Access-Control-Allow-Origin -- Max age: 3600 seconds +This configuration allows: -This configuration allows the React frontend to safely interact with the backend API in both development and production environments. +- Local development (React frontend on port 3000) +- Production deployment behind ingress +- Cross-origin API calls with cookies/credentials --- -# Profile-Based Configuration +## 5. Profile-Based Bootstrapping -## BackendServiceConfig +The backend runs in two distinct service modes: -Activated under the profile: +### Backend Service Profile -```text -backend-service -``` +**Component:** `BackendServiceConfig` -Enables: +- Activated via profile `backend-service` +- Enables Spring MVC (`@EnableWebMvc`) +- Hosts REST controllers and API endpoints -- Spring MVC -- Backend-specific configuration extensions +### Cache Updater Profile -This profile is used when running the main API service. +**Component:** `CacheUpdaterConfig` ---- +- Activated via profile `cache-updater` +- Enables scheduling (`@EnableScheduling`) +- Extends Web MVC auto configuration +- Drives background refresh and pre-cache jobs -## CacheUpdaterConfig +```mermaid +flowchart TD + Startup["Application Startup"] --> Profile{"Active Profile"} + Profile -->|"backend-service"| BackendMode["REST API Mode"] + Profile -->|"cache-updater"| UpdaterMode["Scheduled Refresh Mode"] +``` -Activated under the profile: +This separation allows: -```text -cache-updater -``` +- Horizontal scaling of API nodes +- Independent scaling of background cache updater +- Cleaner infrastructure boundaries in Kubernetes -Enables: +--- -- Scheduling support via @EnableScheduling +# Cross-Module Relationships -This configuration powers the cache updater microservice, which periodically refreshes GitHub data. +The Configurations module connects directly to: -```mermaid -flowchart TD - Profile["cache-updater Profile"] --> Scheduling["EnableScheduling"] - Scheduling --> Jobs["Scheduled Cache Refresh Jobs"] - Jobs --> Cache["CacheServiceAbs"] -``` +- [Cache Services](../cache-services/cache-services.md) – selects and configures cache backend +- [Backend Services](../backend-services/backend-services.md) – provides executors and cache beans +- [Controllers](../controllers/controllers.md) – enables web layer & CORS +- [Rate Management](../rate-management/rate-management.md) – indirectly influences GitHub API concurrency +- [Application Core](../application-core/application-core.md) – loaded at application startup + +It does **not** contain business rules. Instead, it defines the runtime environment that allows the rest of the system to function predictably across development, staging, and production. --- -# Integration with Other Modules +# Design Principles -The Configurations module integrates closely with: +## Environment-Driven Behavior -- Core Application for application bootstrap -- Cache Services for implementation selection -- Service Layer for async execution and caching -- Controllers for CORS and web configuration -- Rate Management for concurrency tuning +All major behavior changes are controlled via properties: -It ensures both backend-service and cache-updater microservices can share infrastructure while enabling profile-specific behavior. +- Cache implementation +- Cache mode +- GitHub API concurrency +- Redis host/port +- Active profile ---- +This enables deployment flexibility without code modification. -# Deployment Considerations +## Infrastructure Isolation -## Environment Variables and Properties +- Business services depend only on abstractions (`CacheServiceAbs`) +- Cache implementation is selected centrally +- Async execution is abstracted behind named executors -Key configurable properties include: +## Safe Shutdown -```text -github.api.concurrency=10 -cache.implementation=redis -cache.mode=read-write -spring.redis.host=localhost -spring.redis.port=6379 -spring.profiles.active=backend-service -``` +- Thread pools are explicitly shut down +- Executors wait for task completion +- Forced termination fallback exists -Proper tuning of these values is critical for: +This prevents: -- Managing GitHub API rate limits -- Optimizing Redis performance -- Preventing thread exhaustion -- Supporting horizontal scaling in Kubernetes +- Orphaned threads +- Partial writes +- Corrupted cache states --- -# Design Principles +# Summary + +The **Configurations** module is the infrastructure backbone of the Major League GitHub backend. -The Configurations module follows these principles: +It provides: -1. Separation of infrastructure from business logic -2. Environment-driven behavior via profiles -3. Externalized configuration via properties -4. Pluggable cache implementations -5. Safe concurrency and graceful shutdown +- Dynamic cache selection +- Redis connectivity +- Controlled asynchronous execution +- JSON time serialization +- CORS configuration +- Profile-driven runtime behavior -By centralizing infrastructure configuration, the backend remains modular, flexible, and production-ready for both API serving and background cache update workloads. +By separating infrastructure configuration from business logic, the system remains modular, environment-aware, and production-ready. \ No newline at end of file diff --git a/docs/reference/architecture/controllers/controllers.md b/docs/reference/architecture/controllers/controllers.md index 17306ff..ede154d 100644 --- a/docs/reference/architecture/controllers/controllers.md +++ b/docs/reference/architecture/controllers/controllers.md @@ -1,322 +1,293 @@ # Controllers -The **Controllers** module is the HTTP entry point of the Major League GitHub backend service. It exposes RESTful APIs that power the React frontend, handling contributor search, autocomplete filters, entity lookups, and hiring-related endpoints. +The **Controllers** module exposes the public REST API for the Major League GitHub backend service. It acts as the entry point for all HTTP requests coming from the React frontend and external clients. -Built on Spring Boot 3.4, the Controllers module follows a clean layered architecture: +Controllers are responsible for: -- Controllers handle HTTP requests and responses -- Services encapsulate business logic -- Models represent domain entities -- Cache and rate management layers optimize GitHub API usage +- Mapping HTTP routes to application use cases +- Validating and parsing request parameters +- Delegating business logic to services +- Formatting responses using `ApiResponse` or HTTP entities +- Handling basic error and cache readiness scenarios -This module sits at the boundary between the frontend and the backend service layer. +This module sits at the boundary between the web layer (Spring MVC) and the business logic layer implemented in the [Backend Services](../backend-services/backend-services.md) module. --- ## Architectural Overview -The Controllers module orchestrates requests across the Service Layer, Cache Services, and Model Entities. +At a high level, the Controllers module follows a classic Spring Boot layered architecture: ```mermaid flowchart TD - Client["Frontend (React App)"] -->|"HTTP REST"| Controllers["Controllers Module"] - - subgraph controllers_layer["Controllers"] - AutocompleteCtrl["Autocomplete Controller"] - ContributorCtrl["Contributor Controller"] - EntityCtrl["Entity Controller"] - HiringCtrl["Hiring Controller"] - end - - Controllers --> AutocompleteCtrl - Controllers --> ContributorCtrl - Controllers --> EntityCtrl - Controllers --> HiringCtrl - - AutocompleteCtrl -->|"delegates"| CityService["City Service"] - AutocompleteCtrl --> StateService["State Service"] - AutocompleteCtrl --> RegionService["Region Service"] - AutocompleteCtrl --> LanguageService["Language Service"] - AutocompleteCtrl --> SoccerTeamService["Soccer Team Service"] - - ContributorCtrl --> GithubService["GitHub Service"] - ContributorCtrl --> CacheService["Cache Service"] - ContributorCtrl --> CityService - ContributorCtrl --> LanguageService - - EntityCtrl --> CityService - EntityCtrl --> RegionService - EntityCtrl --> StateService - EntityCtrl --> LanguageService - EntityCtrl --> SoccerTeamService - - HiringCtrl --> HiringService["Hiring Service"] - - GithubService -->|"uses"| GraphQLLayer["GraphQL Components"] - GithubService -->|"rate limited by"| RateManager["GitHub Token Rate Manager"] - CacheService --> CacheImpl["Redis / Disk Cache"] + Client["Frontend Client"] -->|"HTTP Request"| ControllerLayer["Controllers"] + ControllerLayer -->|"delegates"| ServiceLayer["Backend Services"] + ServiceLayer -->|"reads/writes"| ModelLayer["Model Entities"] + ServiceLayer -->|"queries"| GraphQLLayer["GraphQL Components"] + ServiceLayer -->|"uses"| CacheLayer["Cache Services"] + ServiceLayer -->|"rate limits"| RateLayer["Rate Management"] + ControllerLayer -->|"wraps response"| ApiResponseNode["ApiResponse"] + ApiResponseNode --> Client ``` -### Key Responsibilities +### Key Relationships -1. **Request validation and parameter parsing** -2. **Delegation to appropriate services** -3. **Response wrapping using `ApiResponse`** -4. **Cache-aware contributor search** -5. **CSV export generation for leaderboard data** +- **Controllers → Backend Services**: All business logic is delegated to services. +- **Controllers → Cache Services**: Contributor-related endpoints use caching abstractions. +- **Controllers → Model Entities**: Domain models (City, Region, Contributor, etc.) are serialized into JSON. +- **Controllers → ApiResponse**: Most endpoints return a standardized response wrapper. --- -## Controller Breakdown +## REST Endpoint Structure -The Controllers module consists of four primary REST controllers: +The module defines four main REST controllers: -- **Autocomplete Controller** – Filter suggestions for UI dropdowns -- **Contributor Controller** – Contributor search and CSV export -- **Entity Controller** – Lookup of single domain entities by ID -- **Hiring Controller** – Hiring manager profile and job openings +1. **AutocompleteController** – Autocomplete endpoints for filters. +2. **ContributorController** – Contributor search and export functionality. +3. **EntityController** – Direct entity lookup by ID. +4. **HiringController** – Hiring manager and job openings endpoints. -Each controller is described below. +All routes are prefixed with `/api` and organized by domain responsibility. --- -# Autocomplete Controller +# Controller Components -**Base Path:** `/api/autocomplete` - -Provides fast lookup endpoints used by frontend autocomplete components. +## 1. AutocompleteController -### Endpoints +**Base Path:** `/api/autocomplete` -| Endpoint | Description | -|-----------|-------------| -| `GET /cities` | Autocomplete cities with optional region/state filters | -| `GET /regions` | Autocomplete regions | -| `GET /states` | Autocomplete states | -| `GET /languages` | Autocomplete programming languages | -| `GET /teams` | Autocomplete soccer teams | +Provides autocomplete suggestions for filterable entities such as cities, regions, states, languages, and soccer teams. -### Behavior +### Supported Endpoints -- All endpoints: - - Accept optional `query` parameter - - Accept filtering IDs where relevant - - Support configurable `maxResults` (default: 50) - - Return `ApiResponse>` -- Logging captures all query combinations for observability. +- `GET /cities` +- `GET /regions` +- `GET /states` +- `GET /languages` +- `GET /teams` -### Data Flow +### Request Flow ```mermaid -flowchart LR - Request["HTTP Request"] --> Controller["Autocomplete Controller"] - Controller --> Service["Domain Service"] - Service --> Entities["Model Entities"] - Entities --> Response["ApiResponse>"] +flowchart TD + Request["GET /api/autocomplete/*"] --> Controller["AutocompleteController"] + Controller --> CityServiceNode["CityService"] + Controller --> StateServiceNode["StateService"] + Controller --> RegionServiceNode["RegionService"] + Controller --> LanguageServiceNode["LanguageService"] + Controller --> SoccerTeamServiceNode["SoccerTeamService"] + Controller --> ResponseWrap["ApiResponse.success()"] ``` -### Dependencies +### Characteristics -- City Service -- State Service -- Region Service -- Language Service -- Soccer Team Service +- All parameters are optional except where default values are provided. +- Supports contextual filtering (e.g., cities by region or state). +- Default `maxResults` is 50. +- Returns a standardized `ApiResponse>`. + +### Dependencies -These services operate on domain models such as `City`, `State`, `Region`, `Language`, and `SoccerTeam`. +- [Backend Services](../backend-services/backend-services.md) +- [Model Entities](../model-entities/model-entities.md) --- -# Contributor Controller +## 2. ContributorController **Base Path:** `/api/contributors` -This controller powers the core leaderboard functionality of Major League GitHub. +This controller drives the core leaderboard functionality of the platform. -## 1. Search Endpoint +### Endpoints + +- `GET /search` – Retrieve ranked contributors. +- `GET /export` – Export contributor results as CSV. + +--- -**Endpoint:** `GET /search` +### 2.1 Search Endpoint -### Parameters +**Route:** `GET /api/contributors/search` -- `cityId` -- `regionId` -- `stateId` -- `teamId` -- `languageId` -- `maxResults` (default: 15) -- `priority` (GitHub API priority level) +#### Responsibilities -### Execution Flow +- Validate cache readiness +- Resolve geographic filters +- Determine selected programming language +- Delegate contributor ranking to `GithubService` +- Use `CacheServiceAbs` for caching and HTTP response reuse + +#### Flow Diagram ```mermaid flowchart TD - Request["Search Request"] --> CacheReady{"Cache Ready?"} - CacheReady -->|"No"| Error["Return ApiResponse.error"] - CacheReady -->|"Yes"| CacheLookup["Cache Service getHttpResponse()"] - - CacheLookup -->|"Hit"| ReturnCached["Return Cached Contributors"] - CacheLookup -->|"Miss"| FetchCities["GitHub Service getTargetCities()"] - FetchCities --> SelectLang["Resolve Language"] - SelectLang --> FetchContrib["GitHub Service getTopContributorsIn()"] - FetchContrib --> StoreCache["Store in Cache"] - StoreCache --> ReturnResponse["Return ApiResponse.success"] + Request["GET /search"] --> CacheCheck["CacheServiceAbs.isCacheReady()"] + CacheCheck -->|"not ready"| ErrorResp["ApiResponse.error()"] + CacheCheck -->|"ready"| CacheLookup["CacheServiceAbs.getHttpResponse()"] + CacheLookup --> GithubServiceNode["GithubService"] + GithubServiceNode --> CitiesNode["CityService.getTargetCities()"] + GithubServiceNode --> LanguageNode["LanguageService"] + GithubServiceNode --> Result["List"] + Result --> SuccessResp["ApiResponse.success()"] ``` -### Key Concepts +#### Key Features + +- Multi-dimensional filtering (city, region, state, team, language) +- Default result limit: 15 +- Configurable GitHub API priority +- Cache-backed response generation -- **Cache Guard:** If the cache is still populating, search is blocked. -- **Priority-based GitHub Calls:** Uses `GithubApiPriority` to manage API rate usage. -- **Language Fallback:** Defaults to configured language if invalid ID provided. -- **Optional-Based Flow:** Uses `Optional` from cache service to handle failures gracefully. +#### Cross-Module Dependencies -## 2. CSV Export Endpoint +- [Backend Services](../backend-services/backend-services.md) +- [Cache Services](../cache-services/cache-services.md) +- [Rate Management](../rate-management/rate-management.md) +- [GraphQL Components](../graphql-components/graphql-components.md) + +--- -**Endpoint:** `GET /export` +### 2.2 Export Endpoint -Generates a downloadable CSV leaderboard file. +**Route:** `GET /api/contributors/export` -### Features +Generates a CSV export of contributor results. + +#### Additional Responsibilities - Builds CSV via Apache Commons CSV -- Dynamically constructs filename: +- Extracts contributor social links +- Dynamically constructs filename +- Returns `ResponseEntity` with `text/csv` content type + +#### CSV Columns ```text -mlg-contributors-{language}-{location}-{yyyy-MM-dd}.csv +Rank, First Name, Last Name, City, State, MLG URL, GitHub URL, Email, Twitter, LinkedIn ``` -- Extracts: - - First and last name - - City and state - - MLG URL - - GitHub, email, Twitter, LinkedIn +#### Notable Behaviors -### Response Type +- Automatically falls back to default language if invalid ID provided. +- Generates MLG deep-link URLs using base domain. +- Dynamically builds filename: -- `ResponseEntity` -- `Content-Disposition: attachment` -- `Content-Type: text/csv` +```text +mlg-contributors-{language}-{location}-{date}.csv +``` -This endpoint enables data portability for hiring managers, recruiters, or analytics. +This endpoint combines service orchestration, caching, transformation, and response streaming logic. --- -# Entity Controller +## 3. EntityController **Base Path:** `/api/entities` -Provides lookup-by-ID endpoints for domain entities. +Provides direct lookup of individual domain entities by ID. ### Endpoints -| Endpoint | Returns | -|-----------|----------| -| `/cities/{id}` | City | -| `/regions/{id}` | Region | -| `/states/{id}` | State | -| `/languages/{id}` | Language | -| `/teams/{id}` | SoccerTeam | +- `GET /cities/{id}` +- `GET /regions/{id}` +- `GET /states/{id}` +- `GET /languages/{id}` +- `GET /teams/{id}` -### Pattern +### Flow ```mermaid -flowchart LR - Request["GET by ID"] --> Controller["Entity Controller"] - Controller --> Service["Domain Service"] - Service --> Found{"Entity Found?"} - Found -->|"No"| Error["ApiResponse.error"] - Found -->|"Yes"| Success["ApiResponse.success"] +flowchart TD + Request["GET /api/entities/{type}/{id}"] --> Controller["EntityController"] + Controller --> ServiceCall["*Service.getById()"] + ServiceCall -->|"null"| ErrorResp["ApiResponse.error()"] + ServiceCall -->|"found"| SuccessResp["ApiResponse.success()"] ``` ### Characteristics -- Returns standardized `ApiResponse` -- Logs warnings for missing IDs -- Ensures consistent frontend error handling +- Thin pass-through endpoints. +- Standardized error logging. +- Uniform success/error response formatting. + +This controller acts as a lightweight read-only gateway to core domain entities. --- -# Hiring Controller +## 4. HiringController **Base Path:** `/api/hiring` -Exposes hiring-related data used in the platform’s hiring feature. +Supports hiring-related features for the platform. ### Endpoints -| Endpoint | Description | -|-----------|-------------| -| `/manager` | Hiring manager profile | -| `/jobs` | Active job openings | +- `GET /manager` – Hiring manager profile. +- `GET /jobs` – Active job openings. -### Design Notes - -- Delegates to Hiring Service -- Returns lightweight `Map` responses -- `jobs` endpoint wraps results in: - - `status` - - `message` - - `data` +### Flow ```mermaid -flowchart LR - Request["Hiring Request"] --> HiringCtrl["Hiring Controller"] - HiringCtrl --> HiringService["Hiring Service"] - HiringService --> JobOpening["JobOpening Model"] - JobOpening --> Response["JSON Response"] +flowchart TD + Request["GET /api/hiring/*"] --> HiringControllerNode["HiringController"] + HiringControllerNode --> HiringServiceNode["HiringService"] + HiringServiceNode --> ResponseMap["Map"] + ResponseMap --> Client["Frontend Client"] ``` ---- - -## Response Standardization +### Characteristics -Most endpoints return: +- Delegates to `HiringService`. +- Returns raw map-based JSON rather than `ApiResponse`. +- Designed for lightweight content retrieval. -```text -ApiResponse - - status - - message - - data -``` +### Dependency -This ensures consistent frontend parsing and type alignment with TypeScript definitions in the frontend. +- [Backend Services](../backend-services/backend-services.md) --- -## Cross-Module Interaction Summary +# Error Handling and Response Strategy + +Most controllers rely on the `ApiResponse` wrapper defined in the Model Entities module. This provides: -The Controllers module depends heavily on: +- `success(data, message)` +- `error(message)` -- Service Layer (business logic) -- Cache Services (performance optimization) -- GraphQL Components (GitHub query construction) -- Rate Management (token throttling) -- Model Entities (domain objects) +Benefits: -It does **not** contain business logic — it acts strictly as a routing and orchestration layer. +- Consistent JSON structure +- Predictable frontend parsing +- Clear success/error semantics + +Contributor export is an exception, returning `ResponseEntity` for file download semantics. --- -## Design Principles +# Design Principles + +The Controllers module adheres to the following principles: -- ✅ Thin controllers -- ✅ Constructor injection (except legacy Entity Controller) -- ✅ Clear separation of concerns -- ✅ Cache-first contributor search -- ✅ Graceful error handling -- ✅ CSV export support -- ✅ Structured logging +1. **Thin Controllers** – Business logic is delegated to services. +2. **Separation of Concerns** – Controllers only orchestrate. +3. **Standardized Responses** – Uniform API contract. +4. **Cache Awareness** – Contributor endpoints respect cache readiness. +5. **Frontend-Oriented API Design** – Endpoints structured around UI filters and leaderboard needs. --- -## Summary +# Position Within the System -The **Controllers** module is the API façade of the Major League GitHub backend. It: +Within the backend architecture: -- Powers leaderboard search -- Provides dynamic filtering -- Exposes structured domain entities -- Enables hiring visibility -- Optimizes performance via caching +- The application entry point is defined in the Application Core module. +- Controllers define the HTTP boundary. +- Services implement domain logic. +- Cache and Rate Management protect GitHub API usage. +- GraphQL Components build GitHub queries. +- Model Entities define serializable domain objects. -By cleanly delegating to the Service Layer and Cache Services, the Controllers module ensures scalability, maintainability, and frontend compatibility while preserving separation of concerns within the system architecture. +Together, the Controllers module exposes the public API that powers the Major League GitHub leaderboard and hiring features. \ No newline at end of file diff --git a/docs/reference/architecture/core-application/core-application.md b/docs/reference/architecture/core-application/core-application.md deleted file mode 100644 index aad3861..0000000 --- a/docs/reference/architecture/core-application/core-application.md +++ /dev/null @@ -1,271 +0,0 @@ -# Core Application - -The **Core Application** module is the bootstrap and runtime entry point of the Major League GitHub backend service. It initializes the Spring Boot environment, enables cross-cutting infrastructure capabilities such as caching and asynchronous execution, and wires together all backend modules including controllers, services, caching layers, GraphQL integrations, and rate management. - -At its center is the `MajorLeagueGithubApplication` class, which starts the Spring container and activates key framework features required by the rest of the system. - ---- - -## 1. Purpose and Responsibilities - -The Core Application module is responsible for: - -- Bootstrapping the Spring Boot runtime -- Enabling distributed caching across services -- Enabling asynchronous task execution -- Activating component scanning and auto-configuration -- Serving as the root wiring layer for all backend modules - -It does **not** contain business logic. Instead, it orchestrates and activates the modules that implement the domain logic of Major League GitHub. - ---- - -## 2. Core Component - -### MajorLeagueGithubApplication - -Located in: - -```text -backend/src/main/java/cx/flamingo/analysis/MajorLeagueGithubApplication.java -``` - -### Annotations Overview - -```java -@SpringBootApplication -@EnableCaching -@EnableAsync -public class MajorLeagueGithubApplication { - public static void main(String[] args) { - SpringApplication.run(MajorLeagueGithubApplication.class, args); - } -} -``` - -### Annotation Responsibilities - -| Annotation | Responsibility | -|------------|---------------| -| `@SpringBootApplication` | Enables auto-configuration, component scanning, and configuration support | -| `@EnableCaching` | Activates Spring caching abstraction used by cache services | -| `@EnableAsync` | Enables asynchronous method execution via `@Async` | - -These annotations are foundational for modules such as: - -- [Cache Services](../cache-services/cache-services.md) -- [Service Layer](../service-layer/service-layer.md) -- [Configurations](../configurations/configurations.md) - ---- - -## 3. High-Level System Architecture - -The Core Application sits at the center of the backend microservice and activates all domain modules. - -```mermaid -flowchart TD - CoreApp["Core Application"] - - Controllers["Controllers"] - Services["Service Layer"] - Cache["Cache Services"] - Config["Configurations"] - GraphQL["GraphQL Components"] - RateMgmt["Rate Management"] - Models["Model Entities"] - - CoreApp -->|"Component Scan"| Controllers - CoreApp -->|"Component Scan"| Services - CoreApp -->|"EnableCaching"| Cache - CoreApp -->|"AutoConfig"| Config - CoreApp -->|"GitHub Integration"| GraphQL - CoreApp -->|"Rate Limiting"| RateMgmt - CoreApp -->|"Domain Models"| Models -``` - -### Key Relationships - -- **Controllers** expose REST endpoints. -- **Service Layer** implements business logic. -- **Cache Services** optimize GitHub API access. -- **GraphQL Components** build and serialize GitHub queries. -- **Rate Management** protects against GitHub API limits. -- **Model Entities** define backend domain objects. - ---- - -## 4. Backend Request Lifecycle - -The Core Application enables the full request-processing pipeline. - -```mermaid -flowchart TD - Client["Frontend Application"] -->|"HTTP Request"| Controller["Controller"] - Controller -->|"Invoke"| Service["Service Layer"] - Service -->|"Check Cache"| Cache["Cache Services"] - Service -->|"Build Query"| GraphQL["GraphQL Builder"] - GraphQL -->|"GitHub API Call"| GitHub["GitHub GraphQL API"] - GitHub -->|"Response"| Service - Service -->|"Map To"| Model["Model Entities"] - Service -->|"Return"| Controller - Controller -->|"JSON Response"| Client -``` - -The Core Application ensures: - -- All components are discovered via component scanning. -- Caching interceptors are active. -- Async execution is available where configured. -- Configuration classes are applied at startup. - ---- - -## 5. Caching Enablement - -The `@EnableCaching` annotation activates the Spring caching abstraction used by implementations such as: - -- Redis-based caching -- Disk-based caching -- Read-only cache layers - -See: [Cache Services](../cache-services/cache-services.md) - -Without this annotation: - -- Cache annotations in services would be ignored -- Rate-limited GitHub calls would increase -- Performance would degrade significantly - ---- - -## 6. Asynchronous Execution - -The `@EnableAsync` annotation allows services to execute non-blocking operations using `@Async`. - -Typical async use cases include: - -- Background pre-caching -- Token rotation handling -- External service enrichment (e.g., LinkedIn) - -See: [Service Layer](../service-layer/service-layer.md) and [Configurations](../configurations/configurations.md) - ---- - -## 7. Integration With Other Backend Modules - -The Core Application wires together the backend microservice modules: - -| Module | Role in System | -|--------|----------------| -| [Controllers](../controllers/controllers.md) | REST API layer | -| [Service Layer](../service-layer/service-layer.md) | Business logic and orchestration | -| [Cache Services](../cache-services/cache-services.md) | Performance and data reuse | -| [GraphQL Components](../graphql-components/graphql-components.md) | GitHub query construction | -| [Rate Management](../rate-management/rate-management.md) | GitHub API token rotation and rate limiting | -| [Model Entities](../model-entities/model-entities.md) | Domain representation | -| [Configurations](../configurations/configurations.md) | Infrastructure setup | - -The Core Application is the entry boundary that binds them into a cohesive runtime. - ---- - -## 8. Position Within Overall System - -Major League GitHub consists of: - -- A Backend Service (this Spring Boot application) -- A Cache Updater microservice -- A React frontend -- Redis for distributed caching - -Within that architecture, the Core Application: - -- Boots the backend service (default port configured externally) -- Exposes REST APIs consumed by the frontend -- Coordinates GitHub data retrieval -- Manages caching and rate control - -```mermaid -flowchart LR - Frontend["React Frontend"] -->|"REST API"| Backend["Core Application"] - Backend -->|"Cache"| Redis["Redis"] - Backend -->|"GraphQL"| GitHub["GitHub API"] - CacheUpdater["Cache Updater Service"] -->|"Warm Cache"| Redis -``` - ---- - -## 9. Startup Flow - -When the application starts: - -```mermaid -sequenceDiagram - participant JVM as JVM - participant Spring as Spring Boot - participant Core as Core Application - participant Context as Application Context - - JVM->>Spring: Launch main() - Spring->>Core: Initialize - Core->>Context: Component Scan - Context->>Context: Register Beans - Context->>Context: Apply Configurations - Spring->>Spring: Enable Caching - Spring->>Spring: Enable Async - Spring-->>JVM: Application Ready -``` - -Key phases: - -1. JVM starts the `main` method. -2. Spring Boot initializes auto-configuration. -3. Component scanning discovers controllers, services, and configs. -4. Caching and async capabilities are activated. -5. The application becomes ready to serve HTTP requests. - ---- - -## 10. Design Characteristics - -### Lightweight Entry Point -The Core Application contains minimal logic. This ensures: - -- Clear separation of concerns -- Easy testing of individual modules -- Simplified maintainability - -### Annotation-Driven Infrastructure -Infrastructure behavior is declarative: - -- Caching via `@EnableCaching` -- Async via `@EnableAsync` -- Bean discovery via `@SpringBootApplication` - -### Microservice-Oriented -Although simple in code, this module forms the root of a microservice that: - -- Integrates external APIs (GitHub) -- Uses distributed caching (Redis) -- Supports CI/CD and containerized deployment - ---- - -## 11. Summary - -The **Core Application** module is the runtime foundation of the Major League GitHub backend. It: - -- Boots the Spring ecosystem -- Activates caching and asynchronous execution -- Connects controllers, services, caching, GraphQL builders, and rate managers -- Enables the full contributor leaderboard and filtering experience - -While small in code size, it is structurally critical — without it, none of the backend modules would be instantiated or operational. - -For deeper understanding of domain logic, continue with: - -- [Service Layer](../service-layer/service-layer.md) -- [Controllers](../controllers/controllers.md) -- [Cache Services](../cache-services/cache-services.md) diff --git a/docs/reference/architecture/frontend-components/frontend-components.md b/docs/reference/architecture/frontend-components/frontend-components.md index 566e655..a77bb0b 100644 --- a/docs/reference/architecture/frontend-components/frontend-components.md +++ b/docs/reference/architecture/frontend-components/frontend-components.md @@ -1,45 +1,40 @@ # Frontend Components -The **Frontend Components** module contains the reusable UI building blocks for the Major League GitHub React application. These components are responsible for rendering interactive controls, contributor data visualizations, autocomplete inputs, and lightweight pagination. +The **Frontend Components** module contains the reusable UI building blocks used by the Major League GitHub React application. It provides typed, composable, and API-integrated components that power search, filtering, leaderboard display, and navigation interactions in the frontend. -Built with **React 19**, **TypeScript**, and **Material-UI (MUI)**, this module emphasizes: +This module is focused purely on presentation and user interaction. It relies on: -- Type-safe component contracts -- Reusable generic UI primitives -- Clear separation between presentation and data fetching -- Compatibility with React Query for asynchronous data +- Frontend Services for API communication +- Frontend Types for shared domain models +- React Query for server state management +- Material UI for design system components -This module sits at the presentation layer of the frontend architecture and interacts primarily with: - -- Frontend Hooks (state + URL synchronization) -- Frontend Services (API communication) -- Frontend Types (shared domain contracts) +Together, these components render contributor rankings, language filters, and lightweight pagination behavior across the application. --- ## Architectural Overview -The Frontend Components module follows a layered UI composition pattern: - -- **Generic UI primitives** (e.g., BaseAutocomplete) -- **Domain-specific components** (e.g., LanguageAutocomplete) -- **Typed table contracts** (ContributorsTable types) -- **Lightweight utility UI elements** (Pagination) +The Frontend Components module sits between the UI layer and the frontend service layer. ```mermaid flowchart TD - User["User Interaction"] --> Page["Page Component"] - Page --> DomainComponent["Domain Component
LanguageAutocomplete"] - DomainComponent --> BaseComponent["BaseAutocomplete"] - Page --> TableTypes["ContributorsTable Types"] - Page --> PaginationComp["Pagination"] - - DomainComponent --> ReactQuery["React Query"] - ReactQuery --> ApiService["Frontend Services"] - ApiService --> Backend["Backend API"] + User["User"] --> UI["React Pages"] + UI --> Components["Frontend Components"] + Components --> Hooks["Frontend Hooks"] + Components --> Services["Frontend Services"] + Services --> Backend["Backend API"] ``` -The diagram shows how reusable base components support domain-specific components, which then integrate with data services and the backend. +### Responsibilities + +- Provide reusable, generic UI primitives +- Encapsulate Material UI configuration +- Integrate with React Query for data-driven components +- Define component-level TypeScript contracts +- Render domain models (Contributor, Language, State, SoccerTeam) + +The module is intentionally thin in business logic. All data processing and aggregation are delegated to backend services and frontend hooks. --- @@ -47,335 +42,207 @@ The diagram shows how reusable base components support domain-specific component ## 1. BaseAutocomplete -**File:** `frontend/src/components/BaseAutocomplete.tsx` - -### Purpose - -`BaseAutocomplete` is a fully generic, reusable wrapper around MUI's `Autocomplete` component. It abstracts common behaviors such as: +**Core Types:** +- `BaseEntity` +- `BaseAutocompleteProps` -- Free text entry (`freeSolo` mode) -- Controlled input and selected value -- Icon rendering for selected items -- Custom option rendering -- Clear/reset handling +The `BaseAutocomplete` component is a generic, reusable wrapper around Material UI's `Autocomplete` component. It standardizes: -It enables strong typing through the `BaseEntity` interface and generic constraints. - ---- +- Controlled input behavior +- Icon rendering +- Free-text support (`freeSolo` mode) +- Option rendering +- Blur normalization -### BaseEntity Interface +### Generic Entity Model -```typescript -export interface BaseEntity { - id: string; - name?: string; - displayName?: string; - iconUrl?: string; - logoUrl?: string; -} +```text +BaseEntity + ├── id + ├── name? + ├── displayName? + ├── iconUrl? + └── logoUrl? ``` -This ensures every selectable entity: +Any domain object used in autocomplete (Language, City, State, etc.) must satisfy the `BaseEntity` contract. -- Has a stable `id` -- May include display labels -- May optionally include an icon or logo - -This abstraction allows the component to support: - -- Languages -- Regions -- States -- Soccer teams -- Any future entity with minimal additional configuration - ---- - -### BaseAutocompleteProps - -The component is fully controlled: - -- `value`: Selected entity -- `onChange`: Selection handler -- `inputValue`: Current input string -- `onInputChange`: Input change handler -- `options`: Available suggestions -- `getOptionLabel`: Label resolver -- `renderIcon`: Optional icon extractor -- `renderOptionContent`: Custom row rendering override - -This design prevents hidden state and keeps business logic outside the component. - ---- - -### Rendering Flow +### Behavior Flow ```mermaid flowchart TD - Input["User Types"] --> InputChange["onInputChange"] - InputChange --> ParentState["Parent State Update"] - ParentState --> OptionsUpdate["Options Prop Updated"] - OptionsUpdate --> AutocompleteRender["Autocomplete Renders Options"] - - Select["User Selects Option"] --> OnChange["onChange Handler"] - OnChange --> ParentSelection["Parent Updates Selected Value"] + Input["User Types"] --> QueryState["inputValue Updated"] + QueryState --> Options["Options Provided"] + Options --> Select["User Selects Option"] + Select --> OnChange["onChange(value)"] + OnChange --> Sync["Input Normalized on Blur"] ``` -Key behaviors: +### Key Design Decisions -- Clearing input resets both `inputValue` and `value` -- Blur restores the selected label -- Icon is rendered using `iconUrl` or `logoUrl` +- **Free Solo Mode**: Allows raw typing even if no option is selected +- **Icon Rendering Strategy**: Automatically displays `iconUrl` or `logoUrl` +- **Extensible Rendering**: Optional `renderOptionContent` for custom row rendering +- **Strict Generics**: Ensures type safety across autocomplete usages -This makes `BaseAutocomplete` the foundation for all autocomplete-style inputs. +This component acts as the foundation for all search dropdowns in the application. --- ## 2. LanguageAutocomplete -**File:** `frontend/src/components/LanguageAutocomplete.tsx` - -### Purpose - -`LanguageAutocomplete` is a domain-specific wrapper around `BaseAutocomplete` configured for programming languages. - -It connects UI interaction with remote API calls using React Query. +**Core Type:** +- `LanguageAutocompleteProps` ---- +`LanguageAutocomplete` is a specialization of `BaseAutocomplete` for GitHub programming languages. -### Responsibilities +It integrates directly with: -- Fetch language suggestions dynamically -- Debounce via query key behavior -- Bind API results into `BaseAutocomplete` -- Provide correct label resolution (`displayName`) +- React Query (`useQuery`) +- `autocompleteLanguages` API function +- Shared `Language` type from frontend types ---- - -### Data Flow +### Data Fetching Flow ```mermaid flowchart TD - UserInput["User Types Language"] --> QueryKey["Query Key: ['languages', inputValue]"] - QueryKey --> ReactQuery["useQuery()"] + UserInput["User Types Language"] --> ReactQuery["useQuery"] ReactQuery --> ApiCall["autocompleteLanguages(inputValue)"] - ApiCall --> Backend["Backend Controller"] - Backend --> ApiResponse["Language[]"] - ApiResponse --> BaseAuto["BaseAutocomplete"] + ApiCall --> Backend["Backend Language Endpoint"] + Backend --> Response["Language[]"] + Response --> Autocomplete["BaseAutocomplete"] ``` -Important details: - -- `staleTime: 0` ensures fresh suggestions -- `signal` enables request cancellation -- Strong typing via `Language` interface from shared API types +### Important Characteristics -This component demonstrates the architectural pattern: +- Query key is derived from `inputValue` +- No stale caching (`staleTime: 0`) +- Fully controlled input state +- Uses `displayName` as the option label -> Generic UI primitive + Domain binding + Data fetching hook +This pattern ensures responsive, server-driven autocomplete behavior while preserving type safety. --- -## 3. ContributorsTable Types - -**File:** `frontend/src/components/ContributorsTable/types.ts` - -### Purpose - -This file defines the TypeScript contracts for contributor table rendering. - -It separates: - -- Visual structure -- Tooltip data contracts -- Location display logic -- Statistics display - ---- - -### Domain Models Included +## 3. Contributors Table Types +**Core Types:** +- `ContributorsTableProps` +- `ContributorInfoProps` +- `LocationInfoProps` +- `StatsDisplayProps` +- `LocationTooltipProps` +- `ContributorTooltipProps` - `SoccerTeam` - `State` -These extend backend data models with UI-focused fields such as: - -- `logoUrl` -- `displayName` -- Geographic metadata - ---- - -### Table Props - -```typescript -export interface ContributorsTableProps { - contributors: Contributor[]; - isLoading: boolean; - error: Error | null; -} -``` +This file defines all UI-level type contracts used by the contributors leaderboard table. -This ensures the table component: +### Domain Alignment -- Supports loading states -- Displays error states -- Is fully controlled by parent container +The table relies on the shared `Contributor` API type but extends UI-specific display requirements such as: ---- +- Tooltip metadata +- Location formatting +- Team display information +- State and regional grouping -### Type Relationships +### Table Data Model ```mermaid -flowchart LR - ApiContributor["API Contributor"] --> TableProps["ContributorsTableProps"] - TableProps --> InfoProps["ContributorInfoProps"] - TableProps --> LocationProps["LocationInfoProps"] - TableProps --> StatsProps["StatsDisplayProps"] - TableProps --> TooltipProps["ContributorTooltipProps"] +flowchart TD + Contributor["Contributor API Model"] --> Info["ContributorInfoProps"] + Contributor --> Location["LocationInfoProps"] + Contributor --> Stats["StatsDisplayProps"] + Contributor --> Tooltip["ContributorTooltipProps"] + Contributor --> LocationTooltip["LocationTooltipProps"] ``` -This layered typing approach enables: - -- Clear separation of responsibilities -- Strong typing for subcomponents -- Safer refactoring +### SoccerTeam and State Models ---- - -## 4. Pagination +These interfaces represent enriched geographic and league information used to: -**File:** `frontend/src/components/pagination.tsx` +- Display MLS team branding +- Show state and regional affiliation +- Associate contributors with nearest stadiums -### Purpose - -`Pagination` is a minimal stub implementation designed specifically for Major League GitHub. - -Unlike complex enterprise pagination systems, this implementation: - -- Only supports simple previous/next navigation -- Hides itself when `totalPages <= 1` -- Delegates state control to parent +They are UI-level representations and may include display-specific properties such as `iconUrl` and `logoUrl`. --- -### Props Contract +## 4. Pagination -```typescript -interface PaginationProps { - currentPage: number; - totalPages: number; - onPageChange?: (page: number) => void; -} -``` +**Core Type:** +- `PaginationProps` ---- +The `Pagination` component is intentionally minimal. Major League GitHub does not require complex cursor-based or infinite scrolling patterns. -### Behavioral Logic +### Behavior ```mermaid -flowchart TD - Start["Render Pagination"] --> CheckPages{"totalPages <= 1?"} - CheckPages -->|Yes| Hide["Return null"] - CheckPages -->|No| ShowControls["Render Previous / Next"] - ShowControls --> PrevClick["onPageChange(currentPage - 1)"] - ShowControls --> NextClick["onPageChange(currentPage + 1)"] +flowchart LR + Prev["Previous Button"] --> PageState["currentPage"] + Next["Next Button"] --> PageState + PageState --> Render["Re-render with new page"] ``` -Design considerations: +### Design Characteristics -- No internal state -- Fully controlled component -- Minimal UI complexity +- Stateless aside from props +- Optional `onPageChange` +- Disabled boundaries at first and last page +- Tailwind-based lightweight styling -This aligns with the lightweight UX needs of the project. +It exists primarily to satisfy persistent pagination imports while keeping UI logic simple. --- -# Design Principles of Frontend Components - -## 1. Strong Typing Everywhere +# Component Interaction Model -All components rely heavily on: +The following diagram shows how the primary frontend components collaborate: -- Generic constraints -- Shared API type contracts -- Explicit prop interfaces - -This prevents UI/data mismatches and improves maintainability. - ---- - -## 2. Controlled Components Only - -Every major component follows controlled patterns: - -- Value passed from parent -- Changes emitted via callback -- No hidden state - -This simplifies debugging and URL synchronization. - ---- - -## 3. Separation of Concerns - -- UI logic in components -- Data fetching in hooks or React Query -- API calls in services -- Domain contracts in shared types - -This results in clean boundaries and high testability. +```mermaid +flowchart TD + Page["Leaderboard Page"] --> LanguageAuto["LanguageAutocomplete"] + LanguageAuto --> BaseAuto["BaseAutocomplete"] + Page --> Table["Contributors Table"] + Page --> PaginationComp["Pagination"] + LanguageAuto --> Services["Frontend Services"] + Table --> Types["Frontend Types"] +``` --- -# End-to-End UI Interaction Example +# Design Principles -Below is a simplified interaction sequence for filtering contributors by language. +## 1. Strong Typing Everywhere -```mermaid -sequenceDiagram - participant User - participant UI as "LanguageAutocomplete" - participant Query as "React Query" - participant API as "Frontend API Service" - participant Backend - - User->>UI: Type "Java" - UI->>Query: Trigger query with inputValue - Query->>API: autocompleteLanguages("Java") - API->>Backend: HTTP Request - Backend->>API: Return Language[] - API->>Query: Resolve Promise - Query->>UI: Provide options - User->>UI: Select Language - UI->>User: Filter applied in table -``` +All components are built around explicit TypeScript interfaces. Domain objects from the API are reused and extended rather than duplicated. ---- +## 2. Clear Separation of Concerns -# How This Module Fits into the System +- Components handle rendering and interaction +- Hooks manage derived state and URL synchronization +- Services handle network communication +- Backend handles business logic -The Frontend Components module represents the **presentation layer** of the frontend architecture. +## 3. Reusability via Generics -It: +`BaseAutocomplete` demonstrates the module's pattern: define a generic abstraction once, specialize it per domain. -- Consumes typed API data -- Renders domain-specific UI -- Delegates state and data to higher layers -- Remains framework-consistent with MUI +## 4. Minimal UI Logic -By combining generic UI primitives with strongly typed domain wrappers, this module enables Major League GitHub to maintain a clean, scalable, and maintainable React codebase. +The module avoids embedding business logic. All ranking, geographic filtering, and rate-limited GitHub querying are handled outside this layer. --- -# Summary +# How Frontend Components Fit into the System -The **Frontend Components** module provides: +In the overall system architecture: -- A generic and reusable Autocomplete abstraction -- Domain-specific language filtering UI -- Strongly typed contributor table contracts -- Lightweight pagination controls +- The backend provides contributor rankings, geographic data, and language metadata. +- Frontend services call backend endpoints. +- Frontend hooks synchronize URL state and derived logic. +- Frontend Components render structured, interactive UI elements. -Together, these components deliver the interactive experience that powers the Major League GitHub leaderboard interface while preserving architectural clarity and type safety. \ No newline at end of file +This module represents the visual and interactive layer of Major League GitHub, transforming structured API data into a sports-inspired leaderboard experience. \ No newline at end of file diff --git a/docs/reference/architecture/frontend-hooks/frontend-hooks.md b/docs/reference/architecture/frontend-hooks/frontend-hooks.md index 8a2bfab..531694c 100644 --- a/docs/reference/architecture/frontend-hooks/frontend-hooks.md +++ b/docs/reference/architecture/frontend-hooks/frontend-hooks.md @@ -1,267 +1,303 @@ # Frontend Hooks -The **Frontend Hooks** module encapsulates reusable React hooks that manage browser-driven state and location-aware behavior in the Major League GitHub frontend. These hooks act as the bridge between: +The **Frontend Hooks** module encapsulates reusable React hooks that manage client-side state derived from the browser environment and URL. These hooks provide: -- The browser environment (URL, geolocation API) -- React Router state -- UI components in the Frontend Components module -- API-driven filtering logic in the Frontend Services module +- Geolocation-based region detection +- URL-driven filter state management +- Validation and synchronization between UI state and query parameters -By centralizing URL synchronization and geolocation logic, this module ensures consistent filtering, shareable URLs, and location-based personalization across the application. +This module acts as a bridge between: ---- - -## Module Responsibilities - -The Frontend Hooks module provides: +- **Frontend Components** (UI layer) +- **Frontend Services** (API requests) +- **Frontend Types** (shared TypeScript models) -1. **URL State Management** – Synchronizes filter state (city, region, language, team, state) with query parameters. -2. **Validation & Debouncing** – Ensures URL parameters are valid and updates are optimized. -3. **Geolocation-Based Region Detection** – Determines the nearest region using the Haversine formula. -4. **Derived State Utilities** – Exposes helper signals such as `hasStateChanged` and `isStateEmpty`. +By centralizing cross-cutting concerns (URL parsing, validation, geolocation), the Frontend Hooks module keeps components clean, declarative, and predictable. --- -## High-Level Architecture +## Architectural Overview ```mermaid flowchart TD - Browser["Browser Environment"] -->|"query params"| Router["React Router"] - Browser -->|"Geolocation API"| NearestRegionHook["useNearestRegion Hook"] + UI["Frontend Components"] -->|"uses"| Hooks["Frontend Hooks"] + Hooks -->|"reads/writes"| Router["React Router Search Params"] + Hooks -->|"consumes"| Types["Frontend Types"] + Hooks -->|"drives filters"| Services["Frontend Services"] + Services -->|"calls"| Backend["Backend API"] +``` - Router --> UrlStateHook["useUrlState Hook"] +### Responsibilities - UrlStateHook --> Components["Frontend Components"] - NearestRegionHook --> Components +| Hook | Responsibility | +|------|---------------| +| `useNearestRegion` | Determines the closest MLS region based on browser geolocation | +| `useUrlState` | Synchronizes filter state with URL query parameters | - Components --> Services["Frontend Services"] - Services --> Backend["Backend Service API"] -``` +--- -### Explanation +# 1. useNearestRegion -- **React Router** provides access to query parameters. -- **useUrlState** parses, validates, and updates URL parameters. -- **useNearestRegion** interacts with the browser geolocation API. -- **Frontend Components** consume derived state to render filters and tables. -- **Frontend Services** use the URL-derived state to fetch filtered contributor data. +**Source:** `frontend/src/hooks/useNearestRegion.ts` +**Core Component:** `Coordinates` ---- +## Purpose + +`useNearestRegion` determines the closest region to the user using the browser's Geolocation API and the Haversine formula. It enhances UX by automatically suggesting or pre-selecting the geographically nearest MLS region. -## Core Hooks Overview +## Key Concepts -### 1. useUrlState (Advanced Implementation) +### Coordinates Interface -**File:** `frontend/src/hooks/useUrlState.ts` +```typescript +interface Coordinates { + latitude: number; + longitude: number; +} +``` -This is the primary URL state management hook. It provides: +Represents a geographic point in decimal degrees. -- Strongly typed `UrlState` -- Centralized parameter configuration (`URL_PARAMS`) -- Validation via regex rules -- Transformation support -- Debounced updates -- Error handling via `UrlStateError` -- Change detection (`hasStateChanged`) -- Reset capability +### Haversine Distance Formula -#### URL State Model +The hook uses the Haversine formula to compute great-circle distance between two geographic coordinates: ```text -UrlState -├── selectedCityId -├── selectedRegionId -├── stateId -├── languageId -└── teamId +Distance = 2R * arcsin( + sqrt( + sin²((Δlat)/2) + + cos(lat1) * cos(lat2) * sin²((Δlon)/2) + ) +) ``` -Each property maps to a query parameter: +Where: +- `R` = Earth radius (6371 km) +- `Δlat`, `Δlon` = differences in radians -| State Field | Query Param | -|--------------------|------------| -| selectedCityId | cityId | -| selectedRegionId | regionId | -| stateId | stateId | -| languageId | languageId | -| teamId | teamId | +--- -#### Internal Processing Flow +## Execution Flow ```mermaid flowchart TD - Start["Component Mount"] --> ReadParams["Read searchParams"] - ReadParams --> Parse["parseUrlValue()"] - Parse --> Validate["validateValue()"] - Validate --> BuildState["Construct UrlState"] - BuildState --> Memoize["useMemo"] - Memoize --> ReturnState["Return Hook API"] + Start["Hook Initialized"] --> CheckRegions{"Regions Provided?"} + CheckRegions -->|"No"| EndA["Return null"] + CheckRegions -->|"Yes"| CheckGeo{"Geolocation Supported?"} + CheckGeo -->|"No"| ErrorA["Set Error: Not Supported"] + CheckGeo -->|"Yes"| GetPosition["navigator.geolocation.getCurrentPosition()"] + GetPosition --> Calc["Calculate Distance to Each Region"] + Calc --> FindMin["Find Minimum Distance"] + FindMin --> SetRegion["Set nearestRegion"] + SetRegion --> EndB["Return { nearestRegion, error }"] ``` -#### Update Flow with Debouncing +--- + +## Return Value -```mermaid -flowchart TD - UpdateCall["updateUrlState(newState)"] --> Compare["Compare with current params"] - Compare --> HasChanges{"Changes?"} - HasChanges -->|"No"| Exit["Skip Update"] - HasChanges -->|"Yes"| DebounceCheck{"Debounce?"} - DebounceCheck -->|"Immediate"| Apply["setSearchParams()"] - DebounceCheck -->|"Delayed"| Timeout["setTimeout()"] - Timeout --> Apply - Apply --> EndNode["URL Updated"] +```typescript +{ + nearestRegion: Region | null, + error: string | null +} ``` -#### Key Design Decisions +## Integration Points -- **Central Param Registry:** All query parameter behavior is defined in `URL_PARAMS`. -- **Regex Validation:** Prevents malformed IDs from entering application state. -- **Safe Parsing:** Invalid values fall back to defaults. -- **Replace Mode Updates:** Avoids polluting browser history. -- **Derived Flags:** `hasStateChanged` enables optimized re-fetching. +- Consumes `Region` from **Frontend Types** +- Typically used inside location-aware pages +- Can prefill region filters managed by `useUrlState` --- -### 2. useUrlState (Lightweight Variant) +# 2. useUrlState (Validated & Debounced Version) -**File:** `frontend/src/hooks/useUrlState/index.ts` +**Source:** `frontend/src/hooks/useUrlState.ts` -This simplified version: +## Purpose -- Directly maps query params to state -- Provides basic update capability -- Does not include validation or debouncing +`useUrlState` provides a structured and validated interface for synchronizing UI filter state with URL query parameters. -It is suitable for simpler routing scenarios but lacks the advanced protections of the main implementation. +It ensures: ---- +- Strong validation of URL parameters +- Controlled debounced updates +- Graceful fallback to defaults +- Change detection -### 3. useNearestRegion +--- -**File:** `frontend/src/hooks/useNearestRegion.ts` +## URL State Model -This hook determines the nearest `Region` based on user geolocation. +```typescript +export interface UrlState { + selectedCityId: string | null; + selectedRegionId: string | null; + stateId: string | null; + languageId: string | null; + teamId: string | null; +} +``` -#### Responsibilities +Each property maps to a URL parameter: -- Access browser geolocation API -- Compute distances using the Haversine formula -- Select the nearest region with valid coordinates -- Return `{ nearestRegion, error }` +| State Key | URL Param | +|-----------|-----------| +| selectedCityId | cityId | +| selectedRegionId | regionId | +| stateId | stateId | +| languageId | languageId | +| teamId | teamId | -#### Distance Calculation +--- -The hook uses the Haversine formula to compute spherical distance between two latitude/longitude pairs. +## Validation & Parsing Pipeline ```mermaid flowchart TD - Start["Regions Provided"] --> GeoCheck{"Geolocation Supported?"} - GeoCheck -->|"No"| ErrorNode["Set Error"] - GeoCheck -->|"Yes"| GetPos["getCurrentPosition()"] - GetPos --> Loop["Iterate Regions"] - Loop --> Compute["getDistance()"] - Compute --> Compare["Track Minimum Distance"] - Compare --> Select["Select Nearest Region"] - Select --> ReturnNode["Return nearestRegion"] -``` - -#### Coordinates Interface - -```text -Coordinates -├── latitude: number -└── longitude: number + URL["URLSearchParams"] --> Parse["parseUrlValue()"] + Parse --> Validate{"Valid?"} + Validate -->|"Yes"| State["Populate UrlState"] + Validate -->|"No"| Default["Use defaultValue"] ``` -#### Edge Case Handling +### Validation Rules -- Geolocation unsupported -- Permission denied -- No regions with valid coordinates -- Empty region list +- Regex validation: `^[a-zA-Z0-9-]+$` +- Optional transform step +- Default fallback on failure +- Custom error hook support via `onError` --- -## Interaction with Other Modules +## Debounced Updates -The Frontend Hooks module does not operate in isolation. It integrates with the following modules: +To prevent excessive URL updates (e.g., typing filters): -- **Frontend Components** – Components such as tables, filters, and autocompletes consume `urlState` and `nearestRegion`. -- **Frontend Services** – Query parameters derived from `urlState` are passed to API request builders. -- **Frontend Types** – Uses strongly typed models such as `Region` and API response types. - -Typical interaction flow: +- Immediate updates for input clearing +- Optional `debounceMs` for delayed synchronization +- Automatic cleanup on unmount ```mermaid sequenceDiagram - participant User - participant Component as "Filter Component" + participant UI participant Hook as "useUrlState" - participant Service as "API Service" + participant Router as "React Router" + + UI->>Hook: updateUrlState(newState) + Hook->>Hook: debounce if configured + Hook->>Router: setSearchParams() + Router->>Hook: searchParams updated + Hook->>UI: new urlState +``` + +--- + +## Returned API - User->>Component: Select language - Component->>Hook: updateUrlState({ languageId }) - Hook->>Component: Updated urlState - Component->>Service: Fetch contributors with filters - Service->>Component: Return filtered data +```typescript +{ + urlState, + updateUrlState, + resetUrlState, + hasStateChanged, + isStateEmpty +} ``` +### Key Features + +- `updateUrlState(partialState)` — partial updates +- `resetUrlState()` — clears all filters +- `hasStateChanged` — shallow comparison with previous state +- `isStateEmpty` — convenience flag + --- -## Error Handling Strategy +# 3. useUrlState (Lightweight Index Version) + +**Source:** `frontend/src/hooks/useUrlState/index.ts` -### URL Validation Errors +This version provides a simplified URL synchronization mechanism without: -- Invalid values trigger `UrlStateError` -- Fallback to default values -- Optional `onError` callback allows centralized logging +- Validation +- Debouncing +- Change tracking -### Geolocation Errors +## Characteristics + +- Direct mapping between URL params and state +- Immediate updates +- Minimal abstraction + +```mermaid +flowchart LR + URL["URL Params"] --> State["urlState Object"] + State --> Update["updateUrlState()"] + Update --> URL +``` -- Browser not supported -- Permission denied -- Retrieval failure +This lightweight version may be used for: -All errors are surfaced as string messages, allowing components to render appropriate UI feedback. +- Simpler pages +- Legacy compatibility +- Controlled environments --- -## Performance Considerations +# Cross-Module Integration + +```mermaid +flowchart TD + Hooks["Frontend Hooks"] --> Components["Frontend Components"] + Components --> Services["Frontend Services"] + Services --> Backend["Backend Services"] + + Hooks --> Types["Frontend Types"] +``` + +### Relationships -- **useMemo** prevents unnecessary recomputation of parsed state. -- **useCallback** ensures stable function references. -- **Debouncing** reduces excessive URL updates. -- **Change detection** avoids redundant fetch operations. +- **Frontend Components** depend on these hooks for filter and location state +- **Frontend Services** consume values from `urlState` to build API requests +- **Frontend Types** provide shared models (`Region`, `Contributor`, etc.) --- -## Extending the Module +# Design Principles + +## 1. Separation of Concerns -To add a new URL parameter: +Components focus on rendering. +Hooks manage state logic and browser APIs. -1. Extend the `UrlState` interface. -2. Add a new entry in `URL_PARAMS`. -3. Define validation and default behavior. -4. Ensure consuming components use the new state key. +## 2. Deterministic URL State -To enhance geolocation logic: +The URL is the single source of truth for filter state. +This enables: -- Add radius thresholds. -- Introduce fallback region logic. -- Cache last-known region in local storage. +- Shareable links +- Deep linking +- Back/forward navigation compatibility + +## 3. Progressive Enhancement + +- Geolocation is optional +- Validation failures degrade gracefully +- Browser support is checked explicitly --- -## Summary +# Summary -The **Frontend Hooks** module provides the state synchronization and location intelligence that powers the filtering experience of Major League GitHub. +The **Frontend Hooks** module provides the foundational state logic that powers the interactive behavior of Major League GitHub's frontend. -It ensures: +It enables: -- Shareable and bookmarkable filter states -- Validated and controlled query parameters -- Optimized URL updates -- Location-aware region selection -- Clear separation between UI, routing, and API layers +- Location-aware experiences +- Clean URL-driven filtering +- Predictable navigation state +- Improved user experience through validation and debouncing -By isolating browser-specific logic inside reusable hooks, the application remains modular, testable, and scalable. \ No newline at end of file +By abstracting browser APIs and URL synchronization into dedicated hooks, the module ensures maintainability, composability, and scalability across the frontend application. diff --git a/docs/reference/architecture/frontend-services/frontend-services.md b/docs/reference/architecture/frontend-services/frontend-services.md index 39d071a..dae0bf6 100644 --- a/docs/reference/architecture/frontend-services/frontend-services.md +++ b/docs/reference/architecture/frontend-services/frontend-services.md @@ -1,93 +1,69 @@ # Frontend Services -The **Frontend Services** module acts as the API integration layer of the React + TypeScript frontend. It provides a typed, centralized abstraction over all HTTP communication with the Backend Service, encapsulating request construction, query parameter handling, response validation, file downloads, and error propagation. +The **Frontend Services** module acts as the HTTP communication layer between the React frontend and the Spring Boot backend of Major League GitHub. It centralizes all API calls, enforces consistent response handling, and provides strongly typed interfaces for data exchange. -By isolating network logic in one place, the Frontend Services module keeps UI components declarative and focused on presentation while ensuring consistent communication patterns across the application. +This module is responsible for: ---- - -## 1. Purpose and Responsibilities - -The Frontend Services module is responsible for: +- Configuring the HTTP client (Axios) and base backend URL +- Fetching and exporting contributor rankings +- Powering autocomplete search across geographic and language filters +- Fetching entity details by ID +- Exposing hiring-related endpoints +- Ensuring consistent `ApiResponse` validation and error handling -- Configuring the Axios HTTP client -- Managing the backend base URL via environment configuration -- Defining strongly typed API parameter contracts (e.g., `GetContributorsParams`) -- Wrapping backend endpoints in reusable service functions -- Validating `ApiResponse` envelopes -- Supporting request cancellation via `AbortSignal` -- Triggering file downloads (CSV export) - -It interacts closely with: - -- [Frontend Types](../frontend-types/frontend-types.md) for shared API and domain models -- [Frontend Hooks](../frontend-hooks/frontend-hooks.md) for state synchronization and request lifecycle handling -- [Frontend Components](../frontend-components/frontend-components.md) which consume these services -- Backend Controllers exposed by the Backend Service +By isolating API logic from UI components and hooks, the Frontend Services module keeps presentation concerns separate from networking and data orchestration. --- -## 2. High-Level Architecture +## 1. Architectural Role + +The Frontend Services module sits between React components/hooks and the backend REST API. ```mermaid flowchart LR - subgraph UI["Frontend UI Layer"] - Components["React Components"] - Hooks["Custom Hooks"] - end - - subgraph Services["Frontend Services"] - ApiModule["api.ts"] - AxiosConfig["Axios Configuration"] - end - - subgraph Backend["Backend Service"] - Controllers["REST Controllers"] - end - - Components -->|"calls"| Hooks - Hooks -->|"invokes"| ApiModule - ApiModule -->|"uses"| AxiosConfig - ApiModule -->|"HTTP GET /api/..."| Controllers + UI["React Components"] --> Hooks["Custom Hooks"] + Hooks --> Services["Frontend Services"] + Services --> Axios["Axios HTTP Client"] + Axios --> Backend["Spring Boot Backend API"] ``` -### Key Observations +### Responsibilities by Layer -- UI never calls `axios` directly. -- All API requests pass through `api.ts`. -- Backend responses are wrapped in a typed `ApiResponse` envelope. -- Errors are normalized by checking `response.data.status`. +- **React Components**: Render tables, filters, and views. +- **Custom Hooks**: Manage URL state, filtering logic, and lifecycle behavior. +- **Frontend Services**: Perform HTTP requests and validate API responses. +- **Backend API**: Provides REST endpoints for contributors, autocomplete, entities, and hiring data. ---- +The module ensures UI layers never directly construct raw URLs or handle response envelope parsing. -## 3. Axios Configuration +--- -### Base URL Resolution +## 2. Axios Configuration and Environment Integration -The backend URL is resolved from the environment: +At initialization, Axios is configured with a base URL: ```typescript const BACKEND_API_URL = process.env.BACKEND_API_URL || '/'; axios.defaults.baseURL = BACKEND_API_URL; ``` -This enables: +### Key Characteristics -- Local development against `http://localhost:8450` -- Production deployments with environment-specific backend routing -- Reverse-proxy setups where `/api` is forwarded to the backend +- Uses `process.env.BACKEND_API_URL` for environment-based backend routing. +- Defaults to `/` for same-origin deployments. +- Applies globally via `axios.defaults.baseURL`. -### Global Behavior +This design supports: -- All requests inherit the configured `baseURL` -- Requests return typed `ApiResponse` -- Non-success responses throw a JavaScript `Error` +- Local development against a remote backend +- Reverse-proxy deployments +- Containerized or Kubernetes-based environments --- -## 4. Core Interface: GetContributorsParams +## 3. Core Interface: GetContributorsParams -The `GetContributorsParams` interface defines filtering criteria for contributor search: +The `GetContributorsParams` interface defines filter inputs for contributor search. ```typescript interface GetContributorsParams { @@ -101,86 +77,73 @@ interface GetContributorsParams { } ``` -### Design Characteristics +### Design Principles -- All filters are optional -- Supports request cancellation (`AbortSignal`) -- Encapsulates search query construction logic +- All filters are optional to support flexible combinations. +- `maxResults` defaults to 15. +- `signal` enables request cancellation (important for rapid filter changes). -This interface is consumed by both search and export functionality. +This interface enforces strong typing at compile time and prevents malformed queries. --- -## 5. Contributor Search Flow - -### Service Function - -```typescript -getContributors(params: GetContributorsParams): Promise -``` +## 4. Contributor Retrieval Flow -### Request Lifecycle +The `getContributors` function builds query parameters dynamically and validates the backend response envelope. ```mermaid sequenceDiagram - participant UI as React Component - participant Hook as Custom Hook + participant UI as React UI participant Service as Frontend Services - participant Backend as Backend Controller - - UI->>Hook: Trigger search - Hook->>Service: getContributors(params) - Service->>Backend: GET /api/contributors/search - Backend-->>Service: ApiResponse - Service-->>Hook: Contributor[] - Hook-->>UI: Render table + participant API as Backend API + + UI->>Service: getContributors(filters) + Service->>Service: Build URLSearchParams + Service->>API: GET /api/contributors/search + API-->>Service: ApiResponse + Service->>Service: Validate status === "success" + Service-->>UI: Contributor[] ``` ### Important Behaviors -- Query parameters built via `URLSearchParams` -- `maxResults` defaults to 15 -- Response envelope validated (`status === 'success'`) -- Throws error if backend reports failure +1. Dynamically constructs `URLSearchParams`. +2. Sends `GET /api/contributors/search`. +3. Expects `ApiResponse`. +4. Throws an error if `status !== "success"`. +5. Returns only the `data` field to the caller. -The returned `Contributor` type originates from [Frontend Types](../frontend-types/frontend-types.md). +This prevents UI components from needing to understand the response wrapper format. --- -## 6. CSV Export Flow +## 5. CSV Export Flow -### Service Function - -```typescript -downloadContributors(params: Omit) -``` - -### Behavior - -Instead of using Axios for file streaming, this function: - -1. Constructs query parameters -2. Creates a temporary anchor element -3. Sets `href` to `/api/contributors/export` -4. Programmatically triggers download +The `downloadContributors` function triggers a CSV export. ```mermaid flowchart TD - BuildParams["Build Query Parameters"] --> CreateLink["Create Hidden Anchor Element"] - CreateLink --> SetHref["Set export URL"] - SetHref --> ClickLink["Trigger click()"] - ClickLink --> Download["Browser Downloads CSV"] + A["User Clicks Export"] --> B["Build Query Parameters"] + B --> C["Create Hidden Anchor Element"] + C --> D["Set href to Export Endpoint"] + D --> E["Trigger click()"] + E --> F["Browser Downloads contributors.csv"] ``` -This avoids CORS or blob handling complexity and delegates file handling to the browser. +### Characteristics ---- +- Uses `/api/contributors/export`. +- Constructs a hidden `` element. +- Avoids using Axios for file streaming. +- Delegates download handling to the browser. + +This approach simplifies file handling and avoids Blob management complexity. -## 7. Autocomplete Services +--- -Autocomplete endpoints support dynamic filtering in UI components such as dropdowns. +## 6. Autocomplete Endpoints -### Supported Autocomplete Domains +Autocomplete endpoints support dynamic filtering across multiple dimensions: - Regions - States @@ -188,27 +151,28 @@ Autocomplete endpoints support dynamic filtering in UI components such as dropdo - Languages - Soccer Teams -Each function: +### Autocomplete Request Pattern -- Accepts a `query` string -- Accepts optional filtering context (e.g., `stateId`, `regionId`) -- Supports `AbortSignal` for debounced cancellation -- Returns typed arrays (`Region[]`, `State[]`, etc.) +All autocomplete functions: -### Example Pattern +- Call `/api/autocomplete/...` +- Accept a `query` string +- Optionally accept related entity filters +- Support `AbortSignal` +- Validate `ApiResponse` -```typescript -export const autocompleteRegions = async ( - query: string, - stateId?: string, - cityIds?: string[], - signal?: AbortSignal -): Promise +```mermaid +flowchart LR + QueryInput["User Types"] --> Debounce["Debounced Hook"] + Debounce --> ServiceCall["autocompleteX()"] + ServiceCall --> BackendCall["GET /api/autocomplete/*"] + BackendCall --> Response["ApiResponse"] + Response --> FilteredList["Return T[]"] ``` -### Parameter Serialization +### Special Case: Array Parameter Serialization -For array parameters such as `cityIds`, Axios is configured with: +For endpoints accepting `cityIds`, Axios is configured with: ```typescript paramsSerializer: { @@ -216,16 +180,13 @@ paramsSerializer: { } ``` -This ensures: - -- `cityIds=1&cityIds=2` -- Not `cityIds[]=1&cityIds[]=2` +This prevents `[]` suffixes in query strings and ensures backend compatibility. --- -## 8. Entity Lookup Services +## 7. Entity Retrieval by ID -These services retrieve individual entities by ID: +The module exposes entity-specific retrieval functions: - `getRegionById` - `getStateById` @@ -233,49 +194,41 @@ These services retrieve individual entities by ID: - `getLanguageById` - `getTeamById` -### Pattern +All follow the same structure: -```typescript -axios.get>(`/api/entities/.../${id}`) +```mermaid +flowchart TD + A["getEntityById(id)"] --> B["GET /api/entities/{type}/{id}"] + B --> C["ApiResponse"] + C --> D{"status success?"} + D -->|"Yes"| E["Return data"] + D -->|"No"| F["Throw Error"] ``` -### Responsibilities - -- Enforce consistent response envelope validation -- Return strongly typed domain objects -- Shield UI from endpoint structure changes +This uniform pattern improves predictability and simplifies testing. --- -## 9. Hiring Services +## 8. Hiring Endpoints -The hiring endpoints expose: +The module also integrates hiring-related features: -- `getHiringManagerProfile()` -- `getJobOpenings()` +- `getHiringManagerProfile()` → `/api/hiring/manager` +- `getJobOpenings()` → `/api/hiring/jobs` -These return: +Both: -- `HiringManagerProfile` -- `JobOpening[]` - -Types originate from [Frontend Types](../frontend-types/frontend-types.md). - -### Hiring Data Flow +- Expect `ApiResponse`. +- Enforce strict success validation. +- Return typed domain objects. -```mermaid -flowchart LR - HiringPage["Hiring Page"] --> HiringService["Frontend Services"] - HiringService --> BackendHiring["/api/hiring/* Endpoints"] - BackendHiring --> HiringService - HiringService --> HiringPage -``` +These endpoints power hiring pages and profile displays in the frontend. --- -## 10. Error Handling Strategy +## 9. Error Handling Strategy -All service methods follow a consistent pattern: +Every request validates: ```typescript if (response.data.status !== 'success') { @@ -283,69 +236,71 @@ if (response.data.status !== 'success') { } ``` -### Benefits +### Implications + +- Centralized validation logic. +- UI receives either typed data or a thrown error. +- No partial or malformed payloads propagate upward. -- Centralized error normalization -- UI components can rely on promise rejection -- Compatible with React error boundaries -- Clean integration with async hooks +This enforces a clean contract between frontend and backend. --- -## 11. Dependency Relationships +## 10. Data Type Integration + +The module relies on strongly typed interfaces: + +- `ApiResponse` +- `Contributor` +- `City` +- `Region` +- `State` +- `Language` +- `SoccerTeam` +- `HiringManagerProfile` +- `JobOpening` ```mermaid -flowchart TD - Services["Frontend Services"] --> Types["Frontend Types"] - Services --> Hooks["Frontend Hooks"] - Services --> Components["Frontend Components"] - Services --> BackendControllers["Backend Controllers"] +flowchart LR + Services["Frontend Services"] --> ApiResponseType["ApiResponse"] + Services --> ContributorType["Contributor"] + Services --> GeoTypes["City / Region / State"] + Services --> HiringTypes["HiringManagerProfile / JobOpening"] ``` -### Module Responsibilities Separation - -| Module | Responsibility | -|--------|----------------| -| Frontend Services | API communication layer | -| Frontend Types | Shared domain and API typing | -| Frontend Hooks | URL state + lifecycle coordination | -| Frontend Components | Presentation and user interaction | +This guarantees compile-time safety and consistency with backend contracts. --- -## 12. Design Principles +## 11. Design Patterns Used -### 1. Single Source of Network Truth -All HTTP calls live in one file (`api.ts`). +### 1. Service Layer Abstraction +All HTTP logic is centralized in one module. -### 2. Strong Typing -Every request returns a typed domain model. +### 2. Response Envelope Validation +Backend responses are always unwrapped before returning to the UI. -### 3. Envelope Validation -No component must manually check `status`. +### 3. Optional Filter Composition +Query parameters are dynamically composed only when provided. -### 4. Separation of Concerns -UI logic is decoupled from backend communication. +### 4. Request Cancellation Support +`AbortSignal` enables cancellation for: -### 5. Abortable Requests -Autocomplete and search support cancellation to prevent race conditions. +- Autocomplete queries +- Rapid filter switching +- Component unmount safety --- -## 13. How This Module Fits Into the System - -The Frontend Services module forms the boundary between: - -- The React application -- The Spring Boot Backend Service - -It translates UI interactions into REST calls and transforms backend envelopes into usable domain models. +## 12. Summary -Without this module: +The Frontend Services module is the networking backbone of the Major League GitHub frontend. It: -- Components would duplicate HTTP logic -- Error handling would be inconsistent -- Type safety would degrade -- Backend endpoint changes would require widespread refactoring +- Encapsulates all REST communication +- Normalizes backend responses +- Provides strong typing guarantees +- Enables flexible filtering and search +- Supports CSV exports +- Maintains separation between UI and transport logic -By centralizing network communication, the Frontend Services module ensures scalability, maintainability, and clarity across the Major League GitHub frontend architecture. +By isolating HTTP concerns in a dedicated service layer, the application remains modular, maintainable, and scalable as new backend endpoints are introduced. \ No newline at end of file diff --git a/docs/reference/architecture/frontend-types/frontend-types.md b/docs/reference/architecture/frontend-types/frontend-types.md index fcedc50..f7ec871 100644 --- a/docs/reference/architecture/frontend-types/frontend-types.md +++ b/docs/reference/architecture/frontend-types/frontend-types.md @@ -1,67 +1,75 @@ # Frontend Types -The **Frontend Types** module defines the TypeScript domain model for the Major League GitHub React application. It acts as the contract layer between the backend API and the frontend UI, ensuring strong typing, predictable data flow, and alignment with backend entities. +The **Frontend Types** module defines the TypeScript interfaces that model API responses, domain entities, enriched view models, and hiring-related data structures used throughout the React frontend. -This module provides: +It acts as the **type contract layer** between: -- API response and domain entity types -- UI-focused contributor projections -- Enhanced relational types for graph-like data modeling -- Hiring-related profile and job types -- Shared generic wrappers (e.g., `ApiResponse`) +- The Spring Boot backend APIs +- Frontend services and hooks +- UI components +- Derived or enhanced client-side data models -By centralizing these definitions, the application maintains type safety across components, hooks, and services while reflecting backend models defined in the service layer. +By centralizing these interfaces, the module ensures strong compile-time guarantees, consistent data handling, and a clear separation between raw API payloads and UI-specific representations. --- -## Architectural Role +## 1. Architectural Role -The Frontend Types module sits between API services and UI components. +The Frontend Types module sits between the API layer and UI components, defining the canonical shape of data flowing through the frontend. ```mermaid -flowchart LR - Backend["Spring Boot Backend"] -->|"JSON over HTTP"| ApiLayer["Frontend API Services"] - ApiLayer -->|"Typed responses"| Types["Frontend Types"] - Types -->|"Strongly typed models"| Components["React Components"] - Types -->|"Shared interfaces"| Hooks["Custom Hooks"] +flowchart TD + Backend["Backend REST API"] -->|"JSON"| ApiResponseType["ApiResponse"] + ApiResponseType --> ApiModels["API Domain Models"] + ApiModels --> Services["Frontend Services"] + Services --> Hooks["Custom Hooks"] + Hooks --> EnhancedModels["Enhanced Models"] + EnhancedModels --> Components["React Components"] ``` -### Responsibilities +### Key Responsibilities -1. **Define API contracts** matching backend responses -2. **Normalize data structures** for UI consumption -3. **Provide enhanced graph relationships** for location and region modeling -4. **Separate raw API models from UI-optimized projections** -5. **Ensure hiring and contributor domains remain consistent** +- Define strongly typed API response wrappers +- Model domain entities such as contributors, cities, regions, and teams +- Provide enriched client-side variants of core entities +- Model hiring-related data for hiring manager views +- Maintain consistent naming and shape alignment with backend models --- -# Module Structure Overview +## 2. Module Structure Overview -The module is organized into five logical type groups: +The Frontend Types module is organized into four primary type groups: ```mermaid -flowchart TD - Root["Frontend Types"] - - Root --> Api["API Types"] - Root --> Contributor["Contributor Projection Types"] - Root --> Enhanced["Enhanced Relational Types"] - Root --> Hiring["Hiring Domain Types"] - Root --> HiringIndex["Hiring Re-exports (Index)"] +flowchart LR + ApiTypes["api.ts"] --> ContributorType["Contributor (API)"] + ApiTypes --> CityType["City"] + ApiTypes --> RegionType["Region"] + ApiTypes --> StateType["State"] + ApiTypes --> TeamType["SoccerTeam"] + ApiTypes --> LanguageType["Language"] + + ContributorDomain["contributor.ts"] --> ContributorUI["Contributor (UI)"] + + EnhancedTypes["enhanced.ts"] --> EnhancedCity["EnhancedCity"] + EnhancedTypes --> EnhancedRegion["EnhancedRegion"] + EnhancedTypes --> EnhancedState["EnhancedState"] + + HiringTypes["hiring.ts"] --> HiringProfile["HiringManagerProfile"] + HiringTypes --> JobOpeningType["JobOpening"] + HiringTypes --> SocialLinkType["SocialLink"] ``` ---- - -# 1. API Types +Each group serves a different layer of abstraction in the frontend. -**File:** `frontend/src/types/api.ts` +--- -These interfaces represent the canonical backend contract. +# 3. API Domain Models (`api.ts`) -## ApiResponse +These interfaces represent **raw backend payloads** and mirror backend model entities. -Generic wrapper used for all backend responses. +## 3.1 ApiResponse ```typescript export interface ApiResponse { @@ -72,285 +80,305 @@ export interface ApiResponse { ``` ### Purpose -- Standardizes backend responses -- Enables consistent error and success handling -- Provides strong typing for generic payloads ---- +- Generic wrapper for all backend responses +- Standardizes error and success messaging +- Enables strongly typed API service calls -## Geographic Domain Models +Example usage: -These types represent the geographic hierarchy used for leaderboard filtering and proximity calculations. +```typescript +const response: ApiResponse = await fetchContributors(); +``` + +--- + +## 3.2 Core Geographic Models ### City -- Linked to a state + +Represents a physical city with optional relational references. + +Key characteristics: + - Contains geographic coordinates -- References nearest soccer team -- May include resolved `state` and `nearestTeam` +- References related `State` and `SoccerTeam` +- May include embedded reference objects -### Region -- Aggregates multiple states -- Contains central geo coordinates -- Supports UI grouping by region +Notable design detail: -### State -- Belongs to one or more regions -- Contains metadata (icon, displayName) +```typescript +state?: State; +nearestTeam?: SoccerTeam; +``` -### SoccerTeam -- MLS-style team metadata -- Used for proximity-based ranking -- Includes stadium and coaching information +These fields allow the backend to optionally embed related entities to reduce round trips. --- -## Language - -Represents a programming language filter. +### Region -- `id` -- `displayName` -- `iconUrl` +Represents a broader grouping of states. -Used in leaderboard filtering and autocomplete components. +```typescript +geo: { + latitude: number; + longitude: number; +}; +``` ---- +Design highlights: -## Contributor (API Version) +- Uses `stateIds` for lightweight references +- May optionally embed full `State[]` objects -The API-level Contributor is a fully hydrated backend representation. +--- -### Key Characteristics +### State -- Contains relational objects (`city`, `nearestTeam`) -- Includes detailed GitHub metrics -- Contains flattened metric duplicates for convenience -- Distinguishes contributor types: - - `CONTRIBUTOR` - - `HIRING_MANAGER` +Represents a U.S. state or geographic region. -```mermaid -flowchart TD - Contributor["Contributor (API)"] +Important properties: - Contributor -->|"belongs to"| City["City"] - Contributor -->|"nearest"| Team["SoccerTeam"] - Contributor -->|"contains"| Stats["githubStats"] - Contributor -->|"contains"| Social["SocialLink[]"] -``` +- `code` (e.g., CA, TX) +- `displayName` +- `iconUrl` +- `regionIds` linking to parent regions --- -# 2. Contributor Projection Types +## 3.3 Language -**File:** `frontend/src/types/contributor.ts` +Represents a programming language used for leaderboard filtering. -This file defines a UI-focused projection of Contributor. +Includes: -## Contributor (UI Version) +- Identifier +- Display name +- Icon URL -This type: +--- -- Flattens key metrics -- Uses `latestCommitDate` instead of raw timestamp -- Adds UI-specific fields like `location` -- Keeps `city` and `nearestTeam` references +## 3.4 SoccerTeam -### Why Separate from API Contributor? +Represents an MLS team used for geographic proximity ranking. -The API version reflects backend structure. The UI version: +Key fields: -- Matches table rendering needs -- Avoids redundant nested structures -- Enables transformation without mutating raw API data +- Geographic coordinates +- Stadium information +- League metadata +- Branding URLs -```mermaid -flowchart LR - ApiContributor["Contributor (API)"] -->|"transform"| UiContributor["Contributor (UI)"] -``` +This model is heavily used for: -This separation improves maintainability and allows independent backend evolution. +- Proximity calculations +- Leaderboard grouping +- Visual identity in UI components --- -# 3. Enhanced Relational Types +## 3.5 Contributor (API Model) -**File:** `frontend/src/types/enhanced.ts` +The most central domain entity in the application. -These types enrich geographic models with full object references and set-based relationships. +```typescript +export interface Contributor { + id: string; + login: string; + name: string | null; + type: 'CONTRIBUTOR' | 'HIRING_MANAGER'; + city: City; + nearestTeam: SoccerTeam | null; + githubStats: { + score: number; + totalCommits: number; + starsGiven: number; + starsReceived: number; + forksReceived: number; + forksGiven: number; + javaRepos: number; + }; +} +``` -## EnhancedCity +### Architectural Characteristics -Extends City while: +- Contains both flattened metrics (e.g., `score`) and nested `githubStats` +- Embeds relational objects (`city`, `nearestTeam`) +- Includes hiring-related metadata via `type` and `socialLinks` -- Ensuring `state` and `nearestTeam` are resolved -- Replacing optional references with nullable concrete ones +This model powers: -## EnhancedRegion +- Leaderboards +- Contributor profile views +- Hiring manager displays -Extends Region while: +--- -- Using `Set` instead of array -- Adding `cities: Set` +# 4. UI-Level Contributor Model (`contributor.ts`) -## EnhancedState +This `Contributor` interface represents a **frontend-optimized version** of contributor data. -Extends State while: +Key differences from API model: -- Adding `regions: Set` -- Adding `cities: Set` +- Focused on display-ready metrics +- Uses `latestCommitDate` (string) instead of timestamp +- Includes `location` string +- Removes deeply nested structures ```mermaid flowchart TD - Region["EnhancedRegion"] --> State["EnhancedState"] - State --> City["EnhancedCity"] - City --> Team["SoccerTeam"] + ApiContributor["Contributor (API)"] --> Transform["Transform / Map"] + Transform --> UIContributor["Contributor (UI)"] ``` -### Purpose of Enhanced Types - -These types are used when: +### Purpose -- Building in-memory geographic graphs -- Computing nearest regions -- Supporting advanced filtering -- Avoiding repeated lookups +- Simplify table rendering +- Provide display-ready fields +- Reduce UI transformation logic inside components -Using `Set` ensures uniqueness and improves traversal logic. +This separation prevents UI components from depending directly on backend data shape. --- -# 4. Hiring Domain Types +# 5. Enhanced Models (`enhanced.ts`) -**File:** `frontend/src/types/hiring.ts` +Enhanced models introduce **client-side relational enrichment** using `Set` collections. -Represents hiring managers and job-related information. +## 5.1 EnhancedCity -## SocialLink +```typescript +export interface EnhancedCity extends Omit { + state: State | null; + nearestTeam: SoccerTeam | null; +} +``` -Supports multiple platforms: +Replaces optional references with explicitly resolved objects. -- linkedin -- twitter / x -- github -- facebook -- instagram -- mastodon -- bluesky -- email -- website +--- -## JobOpening +## 5.2 EnhancedRegion -Represents: +```typescript +states: Set; +cities: Set; +``` -- Title -- Location -- External URL +Transforms: -## HiringManagerProfile +- `stateIds` → `Set` +- Adds city relationships -Contains: +--- -- Personal metadata -- Social links -- GitHub statistics -- Activity timestamp +## 5.3 EnhancedState + +Extends `State` by adding: + +- `regions: Set` +- `cities: Set` ```mermaid -flowchart TD - HiringManager["HiringManagerProfile"] --> SocialLinks["SocialLink[]"] - HiringManager --> Stats["githubStats"] - HiringManager --> Jobs["JobOpening"] +flowchart LR + RawRegion["Region (IDs)"] --> EnhanceRegion["EnhancedRegion (Objects)"] + RawState["State (IDs)"] --> EnhanceState["EnhancedState (Objects)"] + RawCity["City (Optional refs)"] --> EnhanceCity["EnhancedCity (Resolved refs)"] ``` -This domain integrates contributor ranking with recruiting capabilities. +### Design Motivation + +- Avoid repeated lookups +- Enable fast graph traversal +- Support proximity and filtering logic --- -# 5. Hiring Index Types +# 6. Hiring Models (`hiring.ts`) -**File:** `frontend/src/types/hiring/index.ts` +The hiring types support the hiring manager feature. -This file provides simplified exports of hiring-related types. +## 6.1 SocialLink -Differences from the full hiring types: +```typescript +platform: 'linkedin' | 'twitter' | 'x' | 'github' | 'facebook' | 'instagram' | 'mastodon' | 'bluesky' | 'email' | 'website'; +``` -- Restricted social platforms -- Reduced GitHub metrics -- Lighter-weight profile shape +Provides typed platform validation. -### Purpose +--- + +## 6.2 JobOpening -- Support lightweight imports -- Reduce bundle coupling -- Enable controlled exposure of hiring interfaces +Represents open roles associated with a hiring manager. --- -# Data Flow Summary +## 6.3 HiringManagerProfile + +Includes: + +- Public profile information +- Social links +- Extended GitHub metrics +- Activity timestamp ```mermaid flowchart TD - Backend["Backend Services"] --> ApiResponse["ApiResponse"] - ApiResponse --> ApiModels["API Domain Models"] - ApiModels --> Transform["Transformation Layer"] - Transform --> UiModels["UI Contributor Type"] - ApiModels --> EnhancedModels["Enhanced Geographic Types"] - UiModels --> Components["Leaderboard UI"] - EnhancedModels --> Hooks["Geolocation Hooks"] + HiringManager["HiringManagerProfile"] --> Stats["GitHub Stats"] + HiringManager --> Links["Social Links"] + HiringManager --> Activity["Last Active"] ``` --- -# Design Principles - -## 1. Strong Contract Alignment - -Frontend API types closely mirror backend entities to avoid serialization mismatches. +# 7. Data Flow Summary -## 2. Separation of Concerns +The following diagram summarizes how types evolve across the frontend: -- API types represent backend truth -- UI types represent presentation needs -- Enhanced types represent relational graph logic +```mermaid +flowchart TD + API["Backend API"] --> ApiModels["API Types"] + ApiModels --> Services["Service Layer"] + Services --> UIModels["UI Contributor Type"] + Services --> EnhancedModels["Enhanced Geographic Types"] + UIModels --> Components["Leaderboard Components"] + EnhancedModels --> Filters["Filtering & Proximity Logic"] +``` -## 3. Immutability Friendly +--- -Interfaces encourage pure transformation functions rather than mutation. +# 8. Design Principles -## 4. Domain-Driven Structure +## 8.1 Clear Separation of Concerns -Types reflect real-world concepts: +- API types mirror backend contracts +- UI types optimize rendering +- Enhanced types optimize relational traversal -- Geographic hierarchy -- Soccer team proximity -- Contributor scoring -- Hiring workflows +## 8.2 Strong Type Safety ---- +- Generic API response wrapping +- Discriminated unions (`CONTRIBUTOR | HIRING_MANAGER`) +- Strict platform enums for social links -# When to Use Each Type +## 8.3 Extensibility -| Scenario | Recommended Type | -|----------|-----------------| -| Raw API response | `ApiResponse` | -| Leaderboard row | UI `Contributor` | -| Data normalization | API `Contributor` | -| Geographic graph building | `EnhancedRegion`, `EnhancedState`, `EnhancedCity` | -| Hiring profile display | `HiringManagerProfile` | +- `Omit<>` usage allows safe overrides +- `Set<>` enables efficient graph modeling +- Optional embedding supports backend flexibility --- -# Conclusion - -The **Frontend Types** module is the foundation of type safety across the Major League GitHub frontend. +# 9. Conclusion -It: +The **Frontend Types** module provides the structural backbone of the frontend application. It: -- Bridges backend contracts and UI rendering -- Models complex geographic and contributor relationships -- Supports hiring workflows -- Enables scalable, maintainable frontend development +- Defines the contract with the backend +- Enables safe transformations into UI-ready models +- Supports geographic enrichment logic +- Powers contributor leaderboard and hiring features -By clearly separating API contracts, UI projections, and enhanced relational models, the application maintains both flexibility and structural integrity as features evolve. \ No newline at end of file +Without this module, data transformations would be scattered across components and services. Instead, Frontend Types centralizes domain modeling, improving maintainability, clarity, and long-term scalability of the application. diff --git a/docs/reference/architecture/graphql-components/graphql-components.md b/docs/reference/architecture/graphql-components/graphql-components.md index 1186c09..3c678dd 100644 --- a/docs/reference/architecture/graphql-components/graphql-components.md +++ b/docs/reference/architecture/graphql-components/graphql-components.md @@ -2,29 +2,28 @@ ## Overview -The **Graphql Components** module is responsible for programmatically constructing and serializing GraphQL queries used to retrieve contributor and repository data from the GitHub GraphQL API. +The **Graphql Components** module is responsible for programmatically constructing and serializing GitHub GraphQL queries used by the backend services. Instead of relying on hard-coded query strings, this module provides a fluent, type-safe builder API that dynamically assembles complex GraphQL queries for searching users, retrieving repository statistics, and collecting contribution metrics. -Instead of relying on static query strings, this module provides a fluent, object-oriented query builder that: - -- Dynamically builds complex GraphQL queries -- Supports nested fields and inline fragments -- Adds filters such as location and language -- Applies sorting and pagination (cursor-based) -- Serializes queries into valid GraphQL syntax - -This module is primarily consumed by the Service Layer (notably `GithubService`) to fetch contributor data that powers the Major League GitHub leaderboard. +It acts as the query composition layer between the **Backend Services** (notably `GithubService`) and the external GitHub GraphQL API. --- -## Responsibilities +## Purpose and Responsibilities -The Graphql Components module provides three major capabilities: +The Graphql Components module provides: -1. **Structured Field Modeling** – Represent GraphQL fields as hierarchical objects -2. **GitHub-Specific Query Construction** – Build search queries tailored for GitHub users -3. **Query Serialization** – Convert field trees into valid GraphQL query strings +- A fluent query builder for GitHub user search +- Structured GraphQL field composition with nesting support +- Automatic query filter construction (location, language, sorting) +- Pagination support via cursors +- Query serialization into executable GraphQL strings -Together, these capabilities allow dynamic and extensible query generation without manual string concatenation. +This abstraction improves: + +- Maintainability (no scattered query strings) +- Reusability (common query structure reused across services) +- Extensibility (easy to add new fields or filters) +- Readability (clear tree-based structure of GraphQL fields) --- @@ -32,294 +31,238 @@ Together, these capabilities allow dynamic and extensible query generation witho ```mermaid flowchart TD - ServiceLayer["Service Layer"] -->|"builds query"| GitHubQueryBuilder["GitHubQueryBuilder"] - GitHubQueryBuilder -->|"uses"| SearchFieldBuilder["SearchField (Builder)"] - GitHubQueryBuilder -->|"composes"| InnerField["Field (Inner Class)"] - QuerySerializer["QuerySerializer"] -->|"serializes"| FieldModel["Field (Model)"] - GitHubQueryBuilder -->|"produces"| QueryString["GraphQL Query String"] - QuerySerializer -->|"produces"| QueryString + BackendService["Backend Services"] -->|"build query"| GitHubQueryBuilder["GitHubQueryBuilder"] + GitHubQueryBuilder -->|"composes"| SearchFieldBuilder["SearchField (Builder)"] + GitHubQueryBuilder -->|"returns string"| GraphQLQuery["GraphQL Query String"] + FieldCore["Field (Core Model)"] --> QuerySerializer["QuerySerializer"] + QuerySerializer --> SerializedQuery["Formatted GraphQL Query"] + GraphQLQuery --> GitHubAPI["GitHub GraphQL API"] + SerializedQuery --> GitHubAPI ``` -The module contains two complementary query-building approaches: +The module contains two parallel mechanisms for building GraphQL queries: -- A **string-based fluent builder** optimized for GitHub search queries -- A **tree-based field model** with structured serialization +1. **GitHubQueryBuilder + nested Field classes** (primary implementation) +2. **Field + QuerySerializer** (generic GraphQL builder and serializer) --- ## Core Components -### 1. Field (Tree Model) +### 1. Field (Core Graph Model) **Class:** `cx.flamingo.analysis.graphql.Field` -This class represents a GraphQL field as a node in a hierarchical tree structure. - -### Key Features - -- Maintains: - - Field name - - Arguments (ordered via `LinkedHashMap`) - - Subfields - - Parent reference (for fluent traversal) -- Supports fluent nesting and sibling navigation - -### Example Usage Pattern - -```java -Field root = new Field("search") - .addArg("type", "USER") - .addArg("first", 25); - -Field nodes = root.addField("nodes"); -nodes.addField("login"); -nodes.addField("location"); -``` - -### Design Characteristics - -- Preserves argument order -- Supports deep nesting -- Enables inline fragment usage (e.g., `... on User`) -- Parent tracking allows returning to upper levels in the tree - -This structure is later serialized by `QuerySerializer`. +This class represents a generic GraphQL field node. It models: ---- - -### 2. SearchField (Tree Extension) - -**Class:** `cx.flamingo.analysis.graphql.SearchField` +- Field name +- Arguments (`Map`) +- Nested subfields +- Parent-child relationships -This class extends the tree-based `Field` model and specializes it for GitHub search queries. +### Key Capabilities -### Key Responsibilities +- `addArg(key, value)` – attach GraphQL arguments +- `addField(name)` – add nested subfields +- `nest(name)` – descend into nested structure +- `add(name)` – return to parent level -- Appends query fragments (e.g., location, language) -- Maintains a combined `query` argument -- Escapes values properly +This implementation provides a tree-based representation of GraphQL queries. -### Example +#### Structural Representation -```java -SearchField search = new SearchField("search") - .addArg("type", "USER") - .appendQuery("location:\"Texas\"") - .appendQuery("language:Java"); +```mermaid +flowchart TD + Root["Field: search"] --> Args["Arguments"] + Root --> Nodes["Subfields"] + Nodes --> User["... on User"] + User --> Login["login"] + User --> Location["location"] + User --> Repositories["repositories"] ``` -This approach is useful when building structured search filters dynamically. - --- -### 3. GitHubQueryBuilder (Fluent Query Builder) +### 2. GitHubQueryBuilder **Class:** `cx.flamingo.analysis.graphql.GitHubQueryBuilder` -This is the primary entry point used by the Service Layer to construct GitHub-specific queries. +This is the primary high-level builder used by backend services to construct GitHub-specific queries. + +It encapsulates: -Unlike the generic `Field` model, this builder is optimized specifically for GitHub user search. +- Search configuration +- Sorting logic +- Filtering logic +- Pagination support +- Default field selection -### Internal Structure +### Typical Usage Flow ```mermaid -flowchart TD - GitHubQueryBuilder --> SearchFieldInner["SearchField (Inner Class)"] - SearchFieldInner --> DefaultFields["Default User Fields"] - DefaultFields --> Contributions["Contribution Data"] - DefaultFields --> Repositories["Repositories & Stars"] - DefaultFields --> SocialAccounts["Social Accounts"] +sequenceDiagram + participant Service as Backend Service + participant Builder as GitHubQueryBuilder + participant API as GitHub GraphQL API + + Service->>Builder: searchUsers(size) + Service->>Builder: location(city) + Service->>Builder: language(lang) + Service->>Builder: cursor(after) + Service->>Builder: build() + Builder-->>Service: query string + Service->>API: Execute GraphQL query ``` -### Default Query Structure - -When instantiated, `SearchField` (inner class) automatically configures: +### Search Configuration -- `userCount` -- `pageInfo { hasNextPage, endCursor }` -- `nodes { ... on User { ... } }` +The builder supports: -For each user: - -- Identity fields (login, name, avatar, etc.) -- Social accounts -- Contributions collection and calendar -- Starred repositories -- Repository metadata (stars, forks, primary language) - -This ensures consistent data retrieval across leaderboard requests. +- `searchUsers(int size)` – defines search type and sorting +- `location(String location)` – filters by user location +- `language(String language)` – filters by programming language +- `cursor(String cursor)` – pagination support +- `build()` – generates final query string --- -### Fluent API +### 3. GitHubQueryBuilder.SearchField (Inner Class) -Example usage: +This is a specialized search node that: -```java -String query = new GitHubQueryBuilder() - .searchUsers(25) - .location("Texas") - .language("Java") - .cursor("abc123") - .build(); -``` +- Defines default query structure +- Adds sorting rules +- Appends dynamic search filters +- Escapes query strings safely -This produces: +#### Default Query Structure -```text -query { search(type: USER, first: 25, query: "location:\"Texas\" language:Java sort:repositories-desc sort:stars-desc sort:followers-desc") { ... } } -``` +The builder automatically includes: -### Key Builder Methods +- `userCount` +- `pageInfo` (pagination metadata) +- `nodes` with `... on User` fragment +- Social accounts +- Contribution statistics +- Repository metadata +- Primary language information -- `searchUsers(int size)` – Sets search type and default sorting -- `location(String location)` – Adds location filter -- `language(String language)` – Adds language filter -- `cursor(String cursor)` – Enables pagination -- `build()` – Returns final GraphQL query string +This ensures consistent response payloads across the system. --- -### 4. GitHubQueryBuilder.Field (Inner Class) - -This inner class provides a lightweight string-based representation of fields. - -Features: - -- Field aliasing -- Argument concatenation -- Recursive `build()` method -- Efficient string assembly +### 4. QuerySerializer -This version is optimized for performance and compact query generation. - ---- +**Class:** `cx.flamingo.analysis.graphql.QuerySerializer` -### 5. GitHubQueryBuilder.SearchField (Inner Class) +This component serializes a list of `Field` objects into a properly formatted GraphQL query string with indentation. -This inner class extends the inner `Field` class and provides: +### Responsibilities -- GitHub search filter aggregation -- Sort configuration -- Query argument rewriting -- Automatic escaping +- Traverse the field tree recursively +- Serialize arguments +- Maintain indentation levels +- Handle inline fragments (e.g., `... on User`) -It maintains a `queryFilters` string that consolidates: +#### Serialization Flow -- Location filters -- Language filters -- Sort directives +```mermaid +flowchart TD + Fields["List of Field"] --> SerializeFields["serializeFields()"] + SerializeFields --> SerializeField["serializeField()"] + SerializeField --> SerializeArgs["serializeArguments()"] + SerializeField --> Recurse["Serialize Subfields"] + Recurse --> SerializeField +``` -Each update reconstructs the `query` argument safely. +This serializer is independent of GitHub-specific logic and can be reused for other GraphQL queries. --- -### 6. QuerySerializer +### 5. SearchField (Standalone Class) -**Class:** `cx.flamingo.analysis.graphql.QuerySerializer` - -This class converts a list of tree-based `Field` objects into a formatted GraphQL query string. - -### Responsibilities - -- Adds indentation for readability -- Serializes arguments -- Recursively processes subfields -- Handles inline fragments (e.g., `... on User`) +**Class:** `cx.flamingo.analysis.graphql.SearchField` -### Serialization Flow +This class extends the core `Field` class and provides: -```mermaid -flowchart TD - Start["serialize(fields)"] --> OpenQuery["append 'query {'"] - OpenQuery --> Iterate["iterate fields"] - Iterate --> SerializeField["serializeField()"] - SerializeField --> SerializeArgs["serializeArguments()"] - SerializeField --> SerializeChildren["process subfields"] - SerializeChildren --> CloseBlock["append '}'"] - CloseBlock --> End["return string"] -``` +- A query string accumulator +- Fluent `appendQuery()` method +- Automatic updating of the `query` argument -The serializer is useful when a fully structured query tree is built using the standalone `Field` model. +It is a lighter alternative to the inner `SearchField` used in `GitHubQueryBuilder`. --- -## Data Flow Within the System +## End-to-End Query Lifecycle ```mermaid flowchart LR - Controller["Controller Layer"] --> Service["GithubService"] + Controller["Controller"] --> Service["GithubService"] Service --> Builder["GitHubQueryBuilder"] Builder --> Query["GraphQL Query String"] - Query --> GitHubAPI["GitHub GraphQL API"] + Query --> GitHubAPI["GitHub API"] GitHubAPI --> Response["JSON Response"] - Response --> Service + Response --> Model["Model Entities"] ``` -1. Controller triggers contributor retrieval -2. Service constructs a query via `GitHubQueryBuilder` -3. Query is sent to GitHub GraphQL API -4. Response is mapped into model entities -5. Data flows back to the frontend +1. A controller requests contributor data. +2. The service layer constructs a query using Graphql Components. +3. The query is sent to GitHub's GraphQL API. +4. The response is mapped into model entities. --- -## Design Decisions +## Design Characteristics -### 1. Programmatic Query Construction +### Fluent API Design -Avoids brittle string templates and enables: +The builder pattern allows readable chained calls such as: -- Dynamic filtering -- Safe nesting -- Reusable logic +```text +searchUsers(50) + .location("Austin") + .language("Java") + .cursor("abc123") + .build() +``` -### 2. Separation of Concerns +### Tree-Based Query Modeling -- `Field` → generic tree modeling -- `SearchField` → search-specific logic -- `GitHubQueryBuilder` → GitHub-specific orchestration -- `QuerySerializer` → formatting and output +GraphQL’s hierarchical structure is mirrored directly in Java objects. -### 3. GitHub-Optimized Defaults +### Separation of Concerns -The builder preconfigures common fields required for leaderboard ranking: +- Query construction logic is isolated from business logic +- Serialization is separated from query composition +- Backend services remain unaware of raw query syntax -- Contributions -- Stars -- Repository metadata -- Social accounts +### Extensibility -This guarantees consistent backend responses. +To add new GitHub fields: ---- +1. Modify default structure inside `SearchField` +2. Add new filters or sorting rules +3. Extend `Field` logic if needed -## Extending the Module +No controller or service changes are required unless new filtering parameters are exposed. -To extend functionality: +--- -- Add additional filters in `SearchField` -- Add new nested fields in `setupDefaultFields()` -- Introduce reusable field fragments using the tree-based `Field` model -- Extend sorting logic within `addSort()` +## Why This Module Matters -When modifying filters, ensure: +The Graphql Components module is a foundational infrastructure layer that enables: -- Proper escaping of quotes -- No duplication of the `query` argument -- Compatibility with GitHub GraphQL schema +- Accurate GitHub contributor ranking +- Advanced filtering (language, region, location) +- Rich contributor profiles (repos, stars, contributions) +- Pagination support for leaderboard views + +By abstracting away GraphQL complexity, it keeps backend services clean while ensuring powerful and flexible GitHub data retrieval. --- ## Summary -The **Graphql Components** module provides a structured, extensible, and GitHub-optimized way to build GraphQL queries for contributor discovery and ranking. - -It acts as the backbone of data retrieval in Major League GitHub by: - -- Constructing dynamic search queries -- Supporting pagination and filtering -- Fetching comprehensive contributor statistics -- Ensuring consistent data shape for downstream services +The **Graphql Components** module provides a structured, fluent, and extensible way to construct GitHub GraphQL queries. It centralizes query logic, enforces consistency, and integrates seamlessly with backend services and model entities. -Without this module, the leaderboard’s data pipeline would rely on fragile string concatenation and duplicated query logic. Instead, Graphql Components centralizes and standardizes query construction across the backend. \ No newline at end of file +It is the backbone of all GitHub data retrieval in the system. \ No newline at end of file diff --git a/docs/reference/architecture/model-entities/model-entities.md b/docs/reference/architecture/model-entities/model-entities.md index 0a21f7f..d9faffe 100644 --- a/docs/reference/architecture/model-entities/model-entities.md +++ b/docs/reference/architecture/model-entities/model-entities.md @@ -1,175 +1,173 @@ # Model Entities -The **Model Entities** module defines the core domain model for the Major League GitHub backend. It contains the immutable and mutable data structures that represent contributors, hiring managers, geographic hierarchies, soccer teams, programming languages, job openings, and standardized API responses. +The **Model Entities** module defines the core domain objects used by the Major League GitHub backend service. These entities represent contributors, geographic structures, hiring profiles, soccer teams, and standardized API responses. -This module is the foundation of the backend architecture. All higher-level layers—controllers, services, caching, and GraphQL integrations—operate on these entities to produce API responses consumed by the frontend. +This module acts as the **central data contract layer** between: ---- - -## 1. Purpose and Responsibilities +- Controllers (REST endpoints) +- Backend services (business logic) +- Cache services (Redis/Disk) +- Frontend API consumers -The Model Entities module is responsible for: +All higher-level modules depend on the data structures defined here. -- Defining domain objects shared across the backend -- Modeling relationships between geography, contributors, and soccer teams -- Representing GitHub statistics and hiring metadata -- Providing a consistent API response wrapper -- Enabling serialization/deserialization via Jackson -- Supporting builder-style object creation via Lombok +--- -These entities are intentionally lightweight and primarily serve as data carriers (POJOs) with minimal business logic. +## 1. Architectural Role ---- +The Model Entities module provides: -## 2. High-Level Architecture +- ✅ Domain models for contributors and hiring managers +- ✅ Geographic hierarchy (Region → State → City) +- ✅ Soccer team metadata for proximity-based ranking +- ✅ Standardized API response wrapper +- ✅ Shared objects reused across backend and frontend type systems -The Model Entities module sits at the core of the backend service layer. +### High-Level Architecture Context ```mermaid flowchart TD - Controllers["REST Controllers"] -->|"return"| ApiResponse["ApiResponse"] - Controllers -->|"use"| Services["Service Layer"] - Services -->|"construct"| Contributor["Contributor"] - Services -->|"construct"| Geography["City / State / Region"] - Services -->|"construct"| SoccerTeam["SoccerTeam"] - Services -->|"construct"| Language["Language"] - Services -->|"construct"| Hiring["HiringManagerProfile / JobOpening"] - Contributor -->|"references"| Geography - Geography -->|"links to"| SoccerTeam + Controllers["Controllers"] -->|"return ApiResponse"| ApiResponse["ApiResponse"] + Controllers --> Services["Backend Services"] + Services --> Models["Model Entities"] + Services --> Cache["Cache Services"] + Cache --> Models + Models --> Frontend["Frontend (TypeScript Types)"] ``` -### Key Design Characteristics +The Model Entities module is a **pure data layer**: -- **Separation of concerns**: Entities contain no service or persistence logic. -- **Bidirectional enrichment**: Many entities include both ID references and optional embedded reference objects. -- **Serialization control**: `@JsonInclude(JsonInclude.Include.NON_NULL)` prevents unnecessary payload bloat. -- **Builder pattern**: All mutable entities use Lombok `@Builder` for safe and readable construction. +- No persistence logic +- No HTTP logic +- No infrastructure logic +- Only structured, serializable domain objects --- -## 3. Core Entity Groups +# 2. Core Entity Groups -The Model Entities module can be logically divided into the following groups: +The module can be logically divided into the following groups: 1. API Wrapper 2. Contributor & Hiring Domain 3. Geographic Hierarchy 4. Soccer Team Domain 5. Language Domain -6. Social & Job Metadata - -Each group is described below. --- -# 4. API Wrapper +# 3. API Wrapper -## ApiResponse +## ApiResponse **Class:** `ApiResponse` -**Purpose:** Standardized API response envelope used by controllers. + +A generic response wrapper used by all REST endpoints. ### Structure -```text -ApiResponse - ├─ status : "success" | "error" - ├─ message : optional message - └─ data : generic payload +```java +public class ApiResponse { + private String status; + private String message; + private T data; +} ``` -### Static Factory Methods +### Factory Methods + +- `success(data)` +- `success(data, message)` +- `error(message)` + +### Purpose -- `success(T data)` -- `success(T data, String message)` -- `error(String message)` +- Standardizes all API responses +- Simplifies frontend error handling +- Enforces consistent JSON shape -### Why This Matters +### Response Flow -- Ensures consistent JSON response structure -- Simplifies frontend parsing -- Centralizes success/error semantics -- Enables strong typing with generics +```mermaid +sequenceDiagram + participant Client + participant Controller + participant Service + + Client->>Controller: HTTP Request + Controller->>Service: Execute logic + Service->>Controller: Domain Model + Controller->>Client: ApiResponse +``` --- -# 5. Contributor & Hiring Domain +# 4. Contributor & Hiring Domain -## 5.1 Contributor +This domain models GitHub contributors and hiring managers. + +## 4.1 Contributor **Class:** `Contributor` -This is the most central entity in the system. It models both: +Represents either: -- GitHub contributors -- Hiring managers +- A ranked GitHub contributor +- A hiring manager profile -### Role Enum +### Key Design Feature -```text -Role - ├─ CONTRIBUTOR - └─ HIRING_MANAGER -``` +The `Role` enum differentiates between: -### Core Identity Fields +- `CONTRIBUTOR` +- `HIRING_MANAGER` + +### Core Fields + +Common fields: - `login` - `name` - `avatarUrl` -- `url` -- `email` - `role` (job title) - `bio` -- `type` (Role enum) - -### Location & Team Association - +- `socialLinks` - `cityId` - `nearestTeamId` -- `city` (optional reference) -- `nearestTeam` (optional reference) - -### GitHub Statistics - -There are two storage patterns: - -#### A. Contributor (CONTRIBUTOR role) +- `lastActive` -Individual numeric fields: +Statistics: +- `score` - `totalCommits` -- `javaRepos` - `starsReceived` - `forksReceived` - `starsGiven` - `forksGiven` -- `score` - -#### B. Hiring Manager (HIRING_MANAGER role) - -- `githubStats` (Map) - -### Unified Stats Access - -`getGithubStats()` normalizes both representations: +- `javaRepos` -- If role is `CONTRIBUTOR`, it dynamically converts individual fields into a map -- If role is `HIRING_MANAGER`, it returns the stored map +### Dynamic Stats Mapping -This ensures API consumers always receive consistent stat structures. +For contributors, `getGithubStats()` converts individual numeric fields into a map structure for uniform serialization. -### Activity Tracking +```mermaid +flowchart TD + Contributor["Contributor"] --> RoleCheck{{"Role?"}} + RoleCheck -->|"CONTRIBUTOR"| BuildMap["Build stats map from fields"] + RoleCheck -->|"HIRING_MANAGER"| UseStored["Use githubStats field"] +``` -- `lastActive : Instant` +This ensures frontend consumers always receive a consistent stats structure. --- -## 5.2 HiringManagerProfile +## 4.2 HiringManagerProfile -A specialized profile structure for hiring managers. +**Class:** `HiringManagerProfile` -### Fields +A simplified representation of a hiring manager. + +Contains: - `name` - `avatarUrl` @@ -179,83 +177,96 @@ A specialized profile structure for hiring managers. - `githubStats` - `lastActive` -This is a simplified representation used in hiring-specific flows. +This object is optimized for hiring-specific views rather than leaderboard ranking. --- -## 5.3 JobOpening +## 4.3 JobOpening + +**Class:** `JobOpening` -Represents a job posting linked to a hiring manager. +Represents a job listing attached to hiring profiles. -### Fields +Fields: - `id` - `title` - `location` - `url` -Designed to be lightweight and embeddable within hiring workflows. +This entity supports hiring-focused features in the application. --- -## 5.4 SocialLink +## 4.4 SocialLink -Represents external platform links. +**Class:** `SocialLink` -### Fields +Encapsulates external profile links. + +Fields: - `platform` - `url` -Used in: - -- `Contributor` -- `HiringManagerProfile` +Used by both Contributor and HiringManagerProfile. --- -# 6. Geographic Hierarchy +# 5. Geographic Hierarchy -The geographic model is hierarchical and relational. +The application ranks contributors geographically and by proximity to MLS stadiums. + +This module models a strict geographic hierarchy: ```mermaid flowchart TD - Region["Region"] -->|"contains"| State["State"] - State -->|"contains"| City["City"] - City -->|"near"| SoccerTeam["SoccerTeam"] + Region["Region"] --> State["State"] + State --> City["City"] + City --> Contributor["Contributor"] ``` -## 6.1 Region +--- + +## 5.1 Region + +**Class:** `Region` -**Immutable Value Object** using Lombok `@Value`. +Immutable (`@Value`) object representing a geographic region. -### Fields +Fields: - `id` -- `name` (internal identifier) -- `displayName` (human-readable) -- `geo : GeoCoordinates` +- `name` (internal slug) +- `displayName` +- `GeoCoordinates geo` - `stateIds` -- `states` (optional reference set) -- `cities` (optional reference set) -### Nested Class: GeoCoordinates +Reference objects: + +- `Set states` +- `Set cities` -```text -GeoCoordinates - ├─ latitude - └─ longitude +### GeoCoordinates (Nested Class) + +```java +public static class GeoCoordinates { + double latitude; + double longitude; +} ``` -Represents the geographic center of a region. +Used for geographic center calculations and proximity logic. --- -## 6.2 State +## 5.2 State + +**Class:** `State` Represents a U.S. state. -### Fields +Fields: - `id` - `name` @@ -263,18 +274,21 @@ Represents a U.S. state. - `displayName` - `iconUrl` - `regionIds` -- `regions` (optional reference) -- `cities` (optional reference) -Uses `@JsonInclude(NON_NULL)` to prevent null reference collections from appearing in API responses. +Reference objects: + +- `regions` +- `cities` --- -## 6.3 City +## 5.3 City -Represents a city within a state. +**Class:** `City` -### Fields +Represents a city tied to contributor location. + +Fields: - `id` - `name` @@ -285,26 +299,25 @@ Represents a city within a state. - `regionIds` - `nearestTeamId` -### Reference Objects +Reference objects: - `state` - `regions` - `nearestTeam` -This dual design (IDs + references) allows: - -- Lightweight responses when only IDs are needed -- Fully enriched responses when deep object graphs are required +Cities are central to proximity-based ranking logic. --- -# 7. Soccer Team Domain +# 6. Soccer Team Domain ## SoccerTeam -Represents a professional soccer team used to gamify contributor rankings. +**Class:** `SoccerTeam` -### Fields +Represents a professional soccer team used for geographic comparison. + +Key fields: - `id` - `name` @@ -312,119 +325,111 @@ Represents a professional soccer team used to gamify contributor rankings. - `state` - `latitude` - `longitude` -- `league` - `stadium` - `stadiumCapacity` -- `joinedYear` +- `league` - `headCoach` - `teamUrl` -- `wikipediaUrl` - `logoUrl` -### Role in the System +### Proximity Flow -- Used to associate contributors with their nearest MLS team -- Supports geographic ranking views -- Enables stadium proximity filtering +```mermaid +flowchart TD + Contributor["Contributor"] --> City["City"] + City --> TeamLookup["Find nearest team"] + TeamLookup --> SoccerTeam["SoccerTeam"] +``` + +This supports MLS-style leaderboard segmentation. --- -# 8. Language Domain +# 7. Language Domain ## Language -Represents a programming language used for filtering and ranking. +**Class:** `Language` + +Represents programming languages used for leaderboard filtering. -### Fields +Fields: - `id` - `name` - `displayName` - `iconUrl` -Used in: +Languages are used in: -- Autocomplete flows -- Filtering contributor leaderboards -- Frontend language badges +- Contributor ranking filters +- Autocomplete features +- UI filtering --- -# 9. Entity Relationship Overview +# 8. Cross-Module Interaction Summary -Below is a consolidated view of relationships across the domain model: +The Model Entities module integrates across the system as follows: ```mermaid flowchart LR - Contributor["Contributor"] -->|"located in"| City["City"] - City -->|"belongs to"| State["State"] - State -->|"part of"| Region["Region"] - City -->|"nearest"| SoccerTeam["SoccerTeam"] - Contributor -->|"links"| SocialLink["SocialLink"] - Contributor -->|"stats"| Stats["GitHub Stats"] - HiringProfile["HiringManagerProfile"] -->|"links"| SocialLink - HiringProfile -->|"stats"| Stats + GitHubService["GitHub Service"] --> Contributor + CityService["City Service"] --> City + RegionService["Region Service"] --> Region + SoccerTeamService["Soccer Team Service"] --> SoccerTeam + HiringService["Hiring Service"] --> HiringManagerProfile + Controllers["Controllers"] --> ApiResponse ``` ---- - -# 10. Serialization & Design Decisions +### Key Observations -## 10.1 Lombok Usage +- Services construct these entities +- Controllers wrap them in `ApiResponse` +- Cache services serialize and store them +- Frontend mirrors them with TypeScript types -The module relies heavily on: - -- `@Data` -- `@Builder` -- `@NoArgsConstructor` -- `@AllArgsConstructor` -- `@Value` +--- -This minimizes boilerplate and keeps entities readable. +# 9. Design Characteristics -## 10.2 JSON Behavior +### ✅ Immutability Where Needed -- `@JsonInclude(JsonInclude.Include.NON_NULL)` avoids null fields in API responses. -- Nested reference objects are optional and populated only when needed. +- `Region` uses `@Value` -## 10.3 Immutability vs Mutability +### ✅ Builder Pattern -- `Region` is immutable (`@Value`) -- Most other entities are mutable via Lombok-generated setters +- Most models use `@Builder` +- Improves readability in service layer construction -This hybrid approach balances safety and flexibility. +### ✅ JSON Optimization ---- +- `@JsonInclude(JsonInclude.Include.NON_NULL)` reduces payload size -# 11. How This Module Fits Into the System +### ✅ Reference Expansion Pattern -Within the backend architecture: +Entities include both: -- **Controllers** return `ApiResponse` wrapping model entities. -- **Services** construct and enrich entities. -- **Cache layer** stores serialized entities. -- **GraphQL components** transform remote GitHub data into these entities. -- **Frontend** consumes serialized versions of these models. +- ID references (`stateId`, `regionIds`, `nearestTeamId`) +- Fully resolved reference objects (`state`, `regions`, `nearestTeam`) -The Model Entities module therefore acts as: +This enables: -- The canonical domain contract of the backend -- The shared language between services and controllers -- The schema foundation for frontend integration +- Lightweight responses (IDs only) +- Fully hydrated responses (expanded objects) --- -# 12. Summary +# 10. Summary -The **Model Entities** module defines the structural backbone of Major League GitHub. +The **Model Entities** module forms the backbone of the Major League GitHub domain model. -It models: +It provides: -- Contributors and hiring managers -- Geographic hierarchies -- Soccer teams and proximity relationships -- Programming languages -- Job openings and social links +- A unified contributor representation +- Geographic hierarchy modeling +- Soccer team metadata +- Hiring ecosystem structures - Standardized API responses -By centralizing domain definitions in a clean, well-structured module, the system maintains consistency across services, caching, APIs, and frontend integration. +Every service, controller, and frontend feature ultimately depends on these data structures. As such, this module defines the authoritative shape of the system’s data contracts. \ No newline at end of file diff --git a/docs/reference/architecture/rate-management/rate-management.md b/docs/reference/architecture/rate-management/rate-management.md index 77e5d21..b2f64ed 100644 --- a/docs/reference/architecture/rate-management/rate-management.md +++ b/docs/reference/architecture/rate-management/rate-management.md @@ -2,330 +2,256 @@ ## Overview -The **Rate Management** module is responsible for handling GitHub API rate limits across multiple access tokens in the Major League GitHub backend. Since the platform aggregates contributor and repository data from the GitHub GraphQL and REST APIs, it must operate within strict primary and secondary rate limits imposed by GitHub. +The **Rate Management** module is responsible for orchestrating and optimizing all outbound GitHub API calls in the Major League GitHub backend. Because the application relies heavily on GitHub’s REST and GraphQL APIs for contributor discovery, statistics, and search, respecting GitHub’s primary and secondary rate limits is critical for reliability. -This module ensures: +This module: -- Safe and efficient use of multiple GitHub tokens -- Automatic detection of primary and secondary rate limits -- Intelligent token selection based on remaining capacity -- Blocking and retry behavior when all tokens are exhausted -- Centralized rate state tracking for all outbound GitHub API calls +- Manages multiple GitHub access tokens +- Tracks primary rate limits (request quotas per hour) +- Detects and handles secondary rate limits (abuse protection) +- Selects the optimal token for each outgoing API call +- Blocks and waits intelligently when all tokens are temporarily exhausted -At its core, the module consists of: - -- `GithubToken` – A stateful model representing a single token and its rate metadata -- `GithubTokenRateManager` – A Spring-managed service responsible for token lifecycle, selection, and synchronization - ---- - -## Architectural Context - -The Rate Management module sits between the Service Layer (e.g., GitHub integration services) and the external GitHub API. - -```mermaid -flowchart TD - Controllers["Controllers"] --> Services["Service Layer"] - Services --> RateManager["GithubTokenRateManager"] - RateManager --> WebClient["WebClient per Token"] - WebClient --> GitHubAPI["GitHub API"] - - RateManager --> TokenState["GithubToken State"] -``` - -### Flow Summary - -1. A service (e.g., GitHub data aggregation) needs to call GitHub. -2. It requests the best available `WebClient` from `GithubTokenRateManager`. -3. The manager selects the optimal token based on remaining quota and reset time. -4. After the call, response headers are used to update the token's rate metadata. -5. If limits are exceeded, the manager enforces wait logic before allowing further calls. +The Rate Management module is primarily used by backend services such as `GithubService`, ensuring that all GitHub API interactions are rate-aware and resilient. --- ## Core Components -### 1. GithubToken - -**Class:** `cx.flamingo.analysis.rate.GithubToken` - -`GithubToken` is a state container representing the real-time rate status of a single GitHub access token. +The module consists of two core classes: -#### Primary Rate Limit Fields +1. **GithubToken** +2. **GithubTokenRateManager** -| Field | Description | -|--------|------------| -| `token` | Raw GitHub access token value | -| `remainingRequests` | Remaining calls from `X-RateLimit-Remaining` | -| `resetTime` | UNIX timestamp from `X-RateLimit-Reset` | -| `rateLimit` | Total limit from `X-RateLimit-Limit` | -| `usedRequests` | Used quota from `X-RateLimit-Used` | +### GithubToken -#### Secondary Rate Limit Fields +`GithubToken` is a data model representing a single GitHub API token and its associated rate limit state. -| Field | Description | -|--------|------------| -| `retryAfterSeconds` | `Retry-After` header value | -| `lastSecondaryLimitHit` | Timestamp when secondary limit occurred | +**Responsibilities:** -#### Key Behaviors +- Store primary rate limit metadata: + - Remaining requests + - Total limit + - Reset time (Unix timestamp) + - Used requests +- Track secondary rate limit state: + - Retry-After duration + - Timestamp of last secondary limit hit +- Provide helper methods for: + - Checking availability + - Computing seconds until reset + - Determining if token is under secondary rate restriction -```mermaid -flowchart TD - CheckSecondary["isUnderSecondaryLimit()"] --> RetryCheck{"retryAfterSeconds and lastSecondaryLimitHit set?"} - RetryCheck -->|"No"| NotLimited["Return false"] - RetryCheck -->|"Yes"| TimeCheck["elapsedSeconds < retryAfterSeconds"] - TimeCheck --> Result["Return true or false"] -``` - -- **`isUnderSecondaryLimit()`** - - Determines if a token is temporarily blocked by GitHub secondary rate limiting. - - Uses wall-clock time and `Retry-After` value. +#### Primary vs Secondary Limits -- **`hasRemainingRequests()`** - - Ensures the token has positive remaining quota. - - Automatically excludes tokens under secondary limit. +- **Primary limit**: Standard per-hour quota (e.g., 5000 requests/hour per token). +- **Secondary limit**: GitHub’s abuse detection mechanism, triggered by high-frequency or burst traffic. Enforced via `Retry-After` headers. -- **`getSecondsUntilReset()`** - - Calculates time until primary rate reset. - - Returns 0 if already reset or not initialized. +Key logic: -This class is intentionally lightweight and mutable so that it can be updated dynamically after each GitHub API response. +- `hasRemainingRequests()` → true only if: + - Remaining > 0 + - Not under secondary rate limit +- `isUnderSecondaryLimit()` → checks whether the retry window has elapsed. --- -### 2. GithubTokenRateManager +### GithubTokenRateManager -**Class:** `cx.flamingo.analysis.rate.GithubTokenRateManager` +`GithubTokenRateManager` is a Spring `@Service` responsible for: -This is a Spring `@Service` responsible for: +- Initializing token-aware `WebClient` instances +- Fetching and updating rate limit metadata +- Selecting the best available token for outgoing requests +- Blocking and retrying when necessary -- Initializing token clients -- Fetching initial rate limit state -- Selecting the optimal token per request -- Handling exhaustion and wait logic -- Updating token state from response headers +It acts as the central gateway for all GitHub API calls. --- -## Initialization Lifecycle +## Architecture Overview + +The Rate Management module sits between backend services and GitHub’s API. ```mermaid -sequenceDiagram - participant Spring - participant Manager as "GithubTokenRateManager" - participant GitHub - - Spring->>Manager: PostConstruct init() - Manager->>Manager: Create WebClient per token - Spring->>Manager: initializeRateLimits() - Manager->>GitHub: GET /rate_limit per token - GitHub-->>Manager: Rate headers - Manager->>Manager: updateTokenRateLimits() +flowchart LR + Service["Backend Service (e.g. GithubService)"] -->|"requests WebClient"| RateManager["GithubTokenRateManager"] + RateManager -->|"selects best token"| Token["GithubToken"] + RateManager -->|"uses WebClient"| GitHubAPI["GitHub API"] + GitHubAPI -->|"response headers"| RateManager + RateManager -->|"updateTokenRateLimits()"| Token ``` -### Token Map Structure - -Internally the manager maintains: - -- `HashMap> tokenMap` - -Each configured token maps to: - -- A `GithubToken` object (state) -- A dedicated `WebClient` configured with: - - Base GitHub API URL - - `Authorization: Bearer ` header - - Increased buffer size (1MB) - -This ensures: +### Flow Summary -- Isolation between tokens -- Clean state tracking per token -- Stateless consumers (services do not manage tokens directly) +1. A backend service requests a GitHub client. +2. The Rate Manager selects the best token. +3. The API call is executed. +4. Response headers are parsed. +5. Token state is updated. --- -## Intelligent Token Selection Algorithm - -The heart of the module is: +## Token Initialization Lifecycle -```text -getBestAvailableClient() -``` +At application startup: -### Selection Strategy +1. Tokens are injected via configuration (`github.tokens`). +2. For each token: + - A `GithubToken` object is created. + - A dedicated `WebClient` is built with: + - Base URL + - Authorization header + - Increased memory buffer +3. `initializeRateLimits()` calls GitHub’s `/rate_limit` endpoint. +4. Each token’s metadata is populated. ```mermaid -flowchart TD - Start["Request Client"] --> Evaluate["Iterate All Tokens"] - - Evaluate --> SecondaryCheck{"Under Secondary Limit?"} - SecondaryCheck -->|"Yes"| SkipSecondary["Track earliest secondary reset"] - SecondaryCheck -->|"No"| PrimaryCheck{"Has rate info?"} - - PrimaryCheck -->|"No"| SkipToken["Skip token"] - PrimaryCheck -->|"Yes"| Compare["Compare remaining and reset time"] - - Compare --> Select["Track best token"] - Select --> Exhausted{"All exhausted?"} +sequenceDiagram + participant App as Application + participant Manager as GithubTokenRateManager + participant GitHub as GitHub API - Exhausted -->|"Yes"| WaitPrimary["Sleep until earliest reset"] - Exhausted -->|"No"| ReturnBest["Return best client"] + App->>Manager: init() + Manager->>Manager: Build WebClient per token + Manager->>GitHub: GET /rate_limit (per token) + GitHub-->>Manager: Rate headers + Manager->>Manager: updateTokenRateLimits() ``` -### Decision Rules - -1. **Skip tokens under secondary limit** - - Respect `Retry-After` duration. - - Track earliest secondary reset. - -2. **Skip tokens without rate info** - - Ensures safe selection. - -3. **Prefer token with:** - - Highest `remainingRequests` - - If tie → Latest `resetTime` +--- -4. **If all tokens are exhausted:** - - Sleep until earliest primary reset - - Reinitialize limits - - Retry selection recursively +## Token Selection Strategy -5. **If all tokens are under secondary limit:** - - Sleep until earliest secondary reset - - Retry selection +The `getBestAvailableClient()` method implements intelligent selection logic. -This guarantees that the system: +### Selection Criteria -- Maximizes throughput -- Avoids unnecessary failures -- Self-recovers from temporary rate exhaustion +For each token: ---- +1. Skip tokens under secondary rate limit. +2. Skip tokens without rate metadata. +3. Prefer tokens with: + - Highest remaining requests + - Latest reset time (if tie) -## Primary vs Secondary Rate Limits +### Exhaustion Handling -### Primary Rate Limit +If all tokens are: -- Controlled via headers: - - `X-RateLimit-Remaining` - - `X-RateLimit-Reset` - - `X-RateLimit-Limit` - - `X-RateLimit-Used` -- Reset occurs at a fixed UNIX timestamp. -- Hard quota enforcement. +- **Under secondary limit** → Wait until earliest retry window expires. +- **Primary exhausted (remaining = 0)** → Wait until earliest reset timestamp. -### Secondary Rate Limit +This ensures: -- Triggered by abuse detection or burst traffic. -- Signaled via `Retry-After` header. -- No official remaining counter. -- Temporarily blocks the token. +- No unnecessary failures +- Full utilization of all configured tokens +- Graceful degradation under heavy load ```mermaid -flowchart LR - Primary["Primary Limit"] -->|"Hard quota"| ResetTime["Reset Timestamp"] - Secondary["Secondary Limit"] -->|"Burst protection"| RetryAfter["Retry-After seconds"] +flowchart TD + Start["Request Client"] --> Evaluate["Evaluate All Tokens"] + Evaluate --> SecondaryCheck{"All Under Secondary?"} + SecondaryCheck -->|"Yes"| WaitSecondary["Sleep Until Earliest Secondary Reset"] + SecondaryCheck -->|"No"| PrimaryCheck{"All Exhausted?"} + PrimaryCheck -->|"Yes"| WaitPrimary["Sleep Until Earliest Reset"] + PrimaryCheck -->|"No"| Select["Select Highest Remaining Token"] + WaitSecondary --> Select + WaitPrimary --> Select + Select --> End["Return WebClient + GithubToken"] ``` -The Rate Management module handles both transparently. - --- -## Rate Limit Update Flow +## Rate Limit Updates -After each GitHub API call, response headers should be passed into: +After each GitHub response, the manager extracts headers such as: -```text -updateTokenRateLimits(GithubToken token, Map> headers) -``` +- `X-RateLimit-Remaining` +- `X-RateLimit-Reset` +- `X-RateLimit-Limit` +- `X-RateLimit-Used` +- `Retry-After` -### Header Processing +These are parsed and injected into the associated `GithubToken`. -- Parses primary headers -- Parses `Retry-After` -- Updates token state -- Logs structured debug information +### Primary Limit Handling -```mermaid -flowchart TD - Response["GitHub Response"] --> Headers["Extract Headers"] - Headers --> UpdatePrimary["Update primary fields"] - Headers --> UpdateSecondary["Update Retry-After"] - UpdatePrimary --> Store["Mutate GithubToken"] - UpdateSecondary --> Store -``` +- Remaining requests updated +- Reset timestamp updated +- Used requests tracked -This design ensures state accuracy without requiring persistent storage. +### Secondary Limit Handling ---- +When `Retry-After` is present: -## Concurrency and Synchronization +- `retryAfterSeconds` is stored +- `lastSecondaryLimitHit` timestamp is recorded +- Token becomes temporarily unavailable + +--- -### Thread Safety Considerations +## Concurrency and Thread Safety -- `initializeRateLimits()` is `synchronized` -- Token selection is deterministic and based on in-memory state -- Blocking logic uses `Thread.sleep()` when required +- `initializeRateLimits()` is `synchronized` to prevent double initialization. +- Token selection is deterministic and read-based. +- Blocking uses `Thread.sleep()` when waiting for reset windows. -Because this service is a singleton Spring bean, it acts as a centralized rate coordination point across all backend threads. +While simple, this design ensures predictable behavior without requiring distributed coordination. --- ## Configuration Dependencies -The module relies on Spring configuration properties: +The module relies on configuration values injected via Spring: -- `github.tokens` – List of GitHub access tokens -- `github.api.url.rate_limit` – Endpoint for rate limit inspection -- `github.api.url` – Base GitHub API URL +- `github.tokens` → List of personal access tokens +- `github.api.url` → Base GitHub API URL +- `github.api.url.rate_limit` → Rate limit endpoint -These values are injected via `@Value` annotations. +These are typically defined in the application configuration and loaded via the Configuration module. --- -## Design Characteristics +## Interaction with Other Modules -### Strengths +The Rate Management module integrates with: -- Multi-token load balancing -- Automatic backoff handling -- Primary + secondary limit awareness -- Minimal external dependencies -- Centralized coordination model +- **Backend Services** → Especially `GithubService`, which delegates API calls. +- **GraphQL Components** → When building and executing GitHub queries. +- **Cache Services** → Rate management ensures cache refreshes do not overwhelm GitHub. -### Trade-offs +It does not expose HTTP endpoints directly. Instead, it acts as an internal infrastructure service. -- Blocking wait using `Thread.sleep()` -- In-memory rate tracking (not distributed) -- Assumes single-instance coordination +--- -For horizontally scaled deployments, a distributed coordination strategy (e.g., Redis-backed rate tracking) could extend this design. +## Design Strengths ---- +- ✅ Multi-token load balancing +- ✅ Intelligent wait-and-retry behavior +- ✅ Secondary rate limit awareness +- ✅ Transparent integration with WebClient +- ✅ Centralized rate logic -## How It Fits Into the Overall System +--- -Within the Major League GitHub backend architecture: +## Potential Extension Points -- Controllers expose REST endpoints. -- Services orchestrate business logic and GitHub queries. -- The Rate Management module ensures safe GitHub API usage. -- Cache services reduce redundant calls. -- Model entities represent domain objects returned to the frontend. +The module could be enhanced with: -The Rate Management module acts as a protective boundary between internal services and GitHub, preventing quota exhaustion and service instability. +- Non-blocking wait strategies (reactive delay instead of `Thread.sleep()`) +- Metrics export (Prometheus/Grafana integration) +- Distributed token coordination (Redis-backed state) +- Adaptive backoff strategies --- ## Summary -The **Rate Management** module provides a robust, centralized mechanism for managing GitHub API limits across multiple tokens. It: +The **Rate Management** module ensures that Major League GitHub can scale GitHub API interactions safely and efficiently. By combining: -- Tracks real-time rate state per token -- Selects the most optimal token dynamically -- Handles both primary and secondary rate limits -- Self-recovers through intelligent waiting and reinitialization +- Multi-token pooling +- Primary and secondary rate awareness +- Intelligent selection and wait strategies -Without this module, the application would risk frequent API failures, degraded performance, and quota exhaustion. It is a foundational reliability component of the backend system. \ No newline at end of file +it transforms GitHub’s strict rate limits into a manageable, resilient infrastructure layer for the entire backend system. diff --git a/docs/reference/architecture/service-layer/service-layer.md b/docs/reference/architecture/service-layer/service-layer.md deleted file mode 100644 index 8a2af41..0000000 --- a/docs/reference/architecture/service-layer/service-layer.md +++ /dev/null @@ -1,390 +0,0 @@ -# Service Layer - -## Overview - -The **Service Layer** is the core business logic module of the Major League GitHub backend. It orchestrates data loading, GitHub and LinkedIn integrations, scoring logic, geographic filtering, and caching coordination. - -This layer sits between the Controllers and the underlying infrastructure modules (cache services, rate management, GraphQL components, and model entities). It transforms raw external API responses and static CSV data into rich domain models such as `Contributor`, `City`, `Region`, and `SoccerTeam`. - -At a high level, the Service Layer is responsible for: - -- Loading and managing reference data (cities, states, regions, languages, teams) -- Fetching and ranking GitHub contributors using GraphQL -- Managing GitHub API rate limits and concurrency -- Enriching contributors with geography and soccer team proximity -- Integrating LinkedIn job postings for hiring features -- Pre-warming and maintaining cache readiness - ---- - -## Architectural Position - -```mermaid -flowchart TD - Controller["Controllers"] --> ServiceLayer["Service Layer"] - ServiceLayer --> Cache["Cache Services"] - ServiceLayer --> Rate["Rate Management"] - ServiceLayer --> GraphQL["GraphQL Components"] - ServiceLayer --> Models["Model Entities"] - ServiceLayer --> External["External APIs
GitHub & LinkedIn"] -``` - -The Service Layer: - -- Receives filtered requests from Controllers. -- Coordinates with Cache Services to reduce API load. -- Uses Rate Management to safely consume GitHub tokens. -- Builds queries via GraphQL components. -- Produces enriched domain entities used by the frontend. - ---- - -# Core Service Responsibilities - -## 1. Geographic & Reference Data Services - -These services load static reference data from CSV files at startup and provide filtering, enrichment, and cross-linking logic. - -### CityService - -**Purpose:** -- Loads cities from `data/cities.csv` -- Associates cities with regions and states -- Computes nearest soccer team using geographic distance - -**Key Features:** -- Autocomplete by name, region, and state -- Sorting by population -- Lazy state population -- Filtering by nearest team - -```mermaid -flowchart LR - CitiesCSV["cities.csv"] --> CityService - CityService --> StateService - CityService --> SoccerTeamService - CityService --> CityModel["City Model"] -``` - -Cities are enriched with: -- State reference (via StateService) -- Nearest soccer team ID (via SoccerTeamService) - ---- - -### StateService - -**Purpose:** -- Loads states from `data/states.csv` -- Supports region-based filtering -- Calculates total population dynamically from cities - -Population is derived dynamically from CityService rather than persisted. - ---- - -### RegionService - -**Purpose:** -- Loads regions from `data/regions.csv` -- Links regions to states and cities -- Supports filtering by state and city -- Sorts by total regional population - -Regions initially load with ID references and are later enriched by ReferencePopulationService. - ---- - -### SoccerTeamService - -**Purpose:** -- Loads teams from `data/teams.csv` -- Calculates nearest team using Haversine distance -- Supports autocomplete by name, city, or state - -```mermaid -flowchart TD - City["City"] -->|"latitude/longitude"| DistanceCalc["Distance Calculation"] - DistanceCalc --> Team["Nearest Soccer Team"] -``` - -This geographic coupling enables the sports-style leaderboard concept. - ---- - -### LanguageService - -**Purpose:** -- Loads languages from `data/languages.csv` -- Supports autocomplete -- Provides default language (Java) - -Languages are critical for filtering GitHub searches. - ---- - -### ReferencePopulationService - -**Purpose:** -- Post-processes regions after startup -- Injects fully populated State and City references into Region objects - -```mermaid -flowchart TD - RegionService --> ReferencePopulationService - StateService --> ReferencePopulationService - CityService --> ReferencePopulationService - ReferencePopulationService --> UpdatedRegions["Enriched Regions"] -``` - -This avoids circular dependencies during initial CSV loading. - ---- - -## 2. GitHub Integration & Contributor Scoring - -### GithubService - -**Purpose:** -- Builds GraphQL queries -- Executes GitHub API calls -- Manages concurrency and batching -- Calculates contributor scores -- Enriches contributors with city and team data - -### High-Level Flow - -```mermaid -flowchart TD - Request["Contributor Request"] --> TargetCities["Resolve Target Cities"] - TargetCities --> Batch["Batch by Concurrency"] - Batch --> AsyncCalls["Async GitHub Calls"] - AsyncCalls --> ProcessUsers["Process & Score Users"] - ProcessUsers --> Merge["Merge & Deduplicate"] - Merge --> Sorted["Sort by Score"] -``` - -### Key Capabilities - -#### 1. Concurrency & Priority - -Two thread pools: -- High priority executor -- Low priority executor - -Requests are batched based on configurable `github.api.concurrency`. - ---- - -#### 2. Rate Limit Handling - -GithubService integrates with GithubTokenRateManager: - -- Selects best available token -- Updates rate limits from response headers -- Switches tokens when limits are hit -- Handles: - - Timeout - - Rate limit exceeded - - Secondary rate limit - - Forbidden or expired tokens - ---- - -#### 3. Caching Integration - -All GraphQL calls are wrapped with CacheServiceAbs: - -- Cache key includes city, language, and page number -- Empty results are cached -- Reduces repeated API load - ---- - -#### 4. Contributor Scoring Formula - -Score formula: - -```text -score = commits × max(starsReceived, 1) × recencyMultiplier -``` - -Where: -- `recencyMultiplier` ranges from 1.0 to 2.0 -- Based on commit activity within the past year - -This ensures: -- Active developers rank higher -- High-impact repositories increase ranking - ---- - -#### 5. Data Enrichment - -Each contributor is enriched with: - -- City (matched from GitHub location) -- Nearest soccer team -- Social links (GitHub, email, website, Twitter, Mastodon, Bluesky, etc.) -- Language-specific repository statistics - -The service also detects social media platforms dynamically from URLs. - ---- - -### Target City Resolution - -GithubService resolves intersections between: - -- City -- State -- Region -- Soccer team - -```mermaid -flowchart TD - Filters["Filters Provided"] --> CityFilter - Filters --> StateFilter - Filters --> RegionFilter - Filters --> TeamFilter - CityFilter --> Intersect - StateFilter --> Intersect - RegionFilter --> Intersect - TeamFilter --> Intersect - Intersect --> FinalCities["Final Target Cities"] -``` - -This flexible intersection logic enables advanced geographic filtering. - ---- - -## 3. Hiring & LinkedIn Integration - -### HiringService - -**Purpose:** -- Exposes hiring manager profile -- Retrieves job openings -- Uses caching with refresh interval - -Profile generation flow: - -```mermaid -flowchart TD - CacheCheck["Check Cache"] -->|"miss"| GithubProfile["Fetch GitHub Profile"] - GithubProfile --> BuildProfile["Build HiringManagerProfile"] - BuildProfile --> StoreCache["Store in Cache"] - StoreCache --> ReturnProfile["Return Response"] -``` - -If cache exists and is valid, GitHub is not called. - ---- - -### LinkedInService - -**Purpose:** -- Retrieves organization job postings -- Performs OAuth client credentials flow -- Parses LinkedIn updates API -- Caches job results - -If LinkedIn fails, HiringService falls back to default static job entries. - ---- - -## 4. Cache Warm-Up & Readiness - -### PreCacheService - -**Purpose:** -- Automatically runs after startup -- Iterates through all languages -- Triggers contributor loading -- Marks cache as ready - -```mermaid -flowchart TD - Startup["Application Startup"] --> PreCacheService - PreCacheService --> ForEachLanguage["Iterate Languages"] - ForEachLanguage --> ContributorController - ContributorController --> GithubService - GithubService --> CacheFilled["Cache Filled"] -``` - -This ensures the leaderboard is responsive immediately after deployment. - ---- - -# Internal Dependency Graph - -```mermaid -flowchart LR - GithubService --> CityService - GithubService --> LanguageService - GithubService --> SoccerTeamService - GithubService --> CacheService - GithubService --> RateManager - - CityService --> StateService - CityService --> SoccerTeamService - - RegionService --> StateService - RegionService --> CityService - - ReferencePopulationService --> RegionService - ReferencePopulationService --> StateService - ReferencePopulationService --> CityService - - HiringService --> GithubService - HiringService --> LinkedInService - HiringService --> CacheService -``` - ---- - -# Lifecycle & Initialization Order - -1. CSV-based services load data at startup (`@PostConstruct`). -2. ReferencePopulationService enriches region references. -3. PreCacheService triggers initial contributor loading. -4. Cache is marked ready. - -This design ensures: -- Deterministic reference data -- No external calls required for base geography -- GitHub load distributed across tokens -- Fast subsequent responses via cache - ---- - -# Design Characteristics - -## Strengths - -- Clear separation of concerns -- Resilient external API handling -- Multi-token rate limit management -- Intelligent contributor scoring -- Geographic and sports-themed enrichment -- Startup cache warm-up for performance - -## Trade-Offs - -- Heavy reliance on GitHub GraphQL schema stability -- CSV-based static reference data requires redeploy for updates -- LinkedIn API changes may affect job parsing - ---- - -# Summary - -The **Service Layer** is the orchestration engine of Major League GitHub. It: - -- Connects geography to developer data -- Applies scoring logic to rank contributors -- Safely consumes external APIs -- Maintains cache efficiency -- Powers both leaderboard and hiring features - -It transforms raw GitHub and LinkedIn data into a sports-inspired, geographically aware developer ranking system. \ No newline at end of file diff --git a/docs/reference/architecture/webpack-plugins/webpack-plugins.md b/docs/reference/architecture/webpack-plugins/webpack-plugins.md index 38112da..2d7762e 100644 --- a/docs/reference/architecture/webpack-plugins/webpack-plugins.md +++ b/docs/reference/architecture/webpack-plugins/webpack-plugins.md @@ -1,226 +1,225 @@ # Webpack Plugins -The **Webpack Plugins** module contains custom build-time extensions for the Major League GitHub frontend. These plugins enhance the Webpack compilation lifecycle by generating static assets and SEO-related files automatically during the build process. +The **Webpack Plugins** module contains custom build-time extensions used by the Major League GitHub frontend. These plugins enhance the Webpack compilation process by generating static assets that are not directly produced by the React application itself. -This module is part of the frontend toolchain and operates entirely at build time. It does not ship runtime code to the browser. Instead, it integrates with Webpack’s plugin system to: +This module currently provides: -- Generate a production-ready `favicon.ico` from an SVG source -- Produce SEO-critical files such as `sitemap.xml` and `robots.txt` +- **FaviconGeneratorPlugin** – Automatically generates a `favicon.ico` file from an SVG source. +- **SeoFilesPlugin** – Dynamically generates `sitemap.xml` and `robots.txt` during the Webpack build. -By encapsulating this logic in custom plugins, the project ensures consistent asset generation across local development and CI/CD pipelines. +Together, these plugins ensure that branding and SEO-related assets are always consistent, up to date, and environment-aware. --- -## Architectural Overview - -The Webpack Plugins module integrates directly with the Webpack compiler lifecycle. Each plugin hooks into specific compilation phases to inject or transform build artifacts. - -```mermaid -flowchart TD - Dev["Developer Runs Build"] --> Webpack["Webpack Compiler"] - Webpack -->|"beforeRun / watchRun"| FaviconPlugin["FaviconGeneratorPlugin"] - Webpack -->|"emit"| SeoPlugin["SeoFilesPlugin"] - - FaviconPlugin --> FileSystem["File System"] - SeoPlugin --> Assets["Compilation Assets"] +## Module Responsibilities - Assets --> Output["Build Output Directory"] - FileSystem --> Output -``` +The Webpack Plugins module is responsible for: -### Key Characteristics +1. Extending the Webpack build lifecycle via custom hooks. +2. Generating derived static assets (ICO from SVG). +3. Injecting SEO-related files directly into the build output. +4. Ensuring build artifacts remain synchronized with source files. -- **Build-time execution only** -- **Zero runtime overhead** in the browser bundle -- **Deterministic asset generation** -- **CI/CD friendly** +These plugins operate purely at **build time** and do not affect runtime performance in the browser. --- -## Plugin Lifecycle Integration - -Webpack exposes lifecycle hooks that plugins can subscribe to. The two plugins in this module use different hooks depending on their responsibilities. +## Architectural Overview ```mermaid -flowchart LR - Compiler["Webpack Compiler"] --> BeforeRun["beforeRun Hook"] - Compiler --> WatchRun["watchRun Hook"] - Compiler --> Emit["emit Hook"] +flowchart TD + Dev["Developer Runs Build"] --> Webpack["Webpack Compiler"] - BeforeRun --> FaviconPlugin["FaviconGeneratorPlugin"] - WatchRun --> FaviconPlugin - Emit --> SeoPlugin["SeoFilesPlugin"] -``` + subgraph plugins["Webpack Plugins Module"] + direction TB + FaviconPlugin["FaviconGeneratorPlugin"] + SeoPlugin["SeoFilesPlugin"] + end -- **FaviconGeneratorPlugin** runs before compilation starts (both normal and watch mode). -- **SeoFilesPlugin** runs during the `emit` phase to inject generated files into the output bundle. + Webpack -->|"beforeRun / watchRun"| FaviconPlugin + Webpack -->|"emit"| SeoPlugin ---- + FaviconPlugin -->|"Generates"| IcoFile["favicon.ico"] + SeoPlugin -->|"Injects"| Sitemap["sitemap.xml"] + SeoPlugin -->|"Injects"| Robots["robots.txt"] -## FaviconGeneratorPlugin + IcoFile --> Output["Build Output Directory"] + Sitemap --> Output + Robots --> Output +``` + +### Key Points -**Core Component:** -`major-league-github.frontend.webpack-plugins.favicon-generator-plugin.FaviconGeneratorPlugin` +- Plugins hook into the **Webpack compiler lifecycle**. +- Output files are injected into the final build artifact. +- No runtime code changes are required in the React application. -### Purpose +--- -Automatically generates a `favicon.ico` file from an SVG source file during the build process. +# FaviconGeneratorPlugin -This ensures: +## Purpose -- A single source of truth (`favicon.svg`) -- Automatic regeneration when the SVG changes -- Consistent output for all environments +The **FaviconGeneratorPlugin** ensures that a `favicon.ico` file is always generated from a source SVG file. This prevents manual conversion steps and keeps the favicon aligned with the latest branding updates. -### Configuration Options +## Core Features -| Option | Description | Default | -|---------|------------|----------| -| `svgPath` | Path to the source SVG file | `public/favicon.svg` | -| `icoPath` | Output path for generated ICO file | `public/favicon.ico` | -| `size` | Icon size in pixels | `60` | +- Converts SVG → PNG → ICO +- Skips regeneration if the ICO file is newer than the SVG +- Works in both normal build mode and watch mode +- Automatically creates output directories if missing -### Internal Workflow +## Build Lifecycle Integration ```mermaid flowchart TD - Start["Plugin Triggered"] --> CheckSVG["Check if SVG Exists"] + Start["Webpack Build Starts"] --> Hook1["beforeRun Hook"] + Start --> Hook2["watchRun Hook"] + + Hook1 --> Generate["generateFavicon()"] + Hook2 --> Generate + + Generate --> CheckSVG{"SVG Exists?"} CheckSVG -->|"No"| Warn["Log Warning"] - CheckSVG -->|"Yes"| CheckICO["Check if ICO Exists"] - CheckICO --> CompareTime["Compare Modification Time"] - CompareTime -->|"ICO Newer"| Skip["Skip Generation"] - CompareTime -->|"SVG Newer"| Convert["Convert SVG to PNG via sharp"] - Convert --> ToICO["Convert PNG to ICO via to-ico"] - ToICO --> Save["Write favicon.ico to Disk"] - Save --> End["Done"] -``` + CheckSVG -->|"Yes"| CheckTime{"ICO Newer?"} -### Optimization Strategy + CheckTime -->|"Yes"| Skip["Skip Generation"] + CheckTime -->|"No"| Convert["Convert SVG → PNG → ICO"] -The plugin avoids unnecessary work by: + Convert --> Save["Write favicon.ico"] + Save --> End["Continue Build"] + Skip --> End + Warn --> End +``` -- Checking if the SVG file exists -- Comparing modification timestamps -- Skipping regeneration if the ICO file is already up to date +## Internal Workflow -This improves incremental build performance, especially in watch mode. +1. Resolve SVG and ICO paths relative to the Webpack context. +2. Validate that the SVG file exists. +3. Compare modification timestamps. +4. Use: + - `sharp` for SVG → PNG conversion. + - `to-ico` for PNG → ICO conversion. +5. Persist the generated `favicon.ico` to disk. -### External Dependencies +## Configuration Options -- **sharp** – Image processing (SVG → PNG conversion) -- **to-ico** – Converts PNG buffer to ICO format -- **fs / path** – Node.js filesystem utilities +| Option | Default | Description | +|--------|---------|------------| +| `svgPath` | `public/favicon.svg` | Path to the source SVG file | +| `icoPath` | `public/favicon.ico` | Output path for the ICO file | +| `size` | `60` | Resize dimension before ICO conversion | -### Error Handling +## Why This Matters -- Logs warnings if the SVG file is missing -- Throws build errors if image conversion fails -- Ensures output directory exists before writing files +- Prevents stale favicon artifacts. +- Ensures SVG remains the single source of truth. +- Reduces manual asset management errors. --- -## SeoFilesPlugin - -**Core Component:** -`major-league-github.frontend.webpack-plugins.seo-files-plugin.SeoFilesPlugin` +# SeoFilesPlugin -### Purpose +## Purpose -Generates SEO-related static files during the Webpack `emit` phase: +The **SeoFilesPlugin** generates search engine optimization files dynamically during the Webpack build: - `sitemap.xml` - `robots.txt` -These files are injected directly into the Webpack compilation assets and included in the final output bundle. - -### Configuration Options - -| Option | Description | Default | -|---------|------------|----------| -| `baseUrl` | Base site URL used in generated files | `https://www.mlg.soccer` | +This ensures that deployments always include consistent and environment-aware SEO metadata. -### Internal Workflow +## Build Lifecycle Integration ```mermaid flowchart TD - EmitStart["emit Hook Triggered"] --> DateGen["Generate Current Date"] - DateGen --> Sitemap["Build sitemap.xml Content"] - DateGen --> Robots["Build robots.txt Content"] - Sitemap --> Inject1["Add sitemap.xml to compilation.assets"] - Robots --> Inject2["Add robots.txt to compilation.assets"] - Inject1 --> Done["Assets Ready for Output"] - Inject2 --> Done -``` + Build["Webpack Emit Phase"] --> EmitHook["emit Hook"] + EmitHook --> Date["Compute Current Date"] + Date --> SitemapGen["Generate sitemap.xml"] + Date --> RobotsGen["Generate robots.txt"] -### Generated Artifacts + SitemapGen --> Inject1["Add to compilation.assets"] + RobotsGen --> Inject2["Add to compilation.assets"] -#### sitemap.xml + Inject1 --> Output["Build Output Directory"] + Inject2 --> Output +``` -- Includes homepage URL -- Sets daily change frequency -- Sets priority to `1.0` -- Uses current date as `lastmod` +## Generated Files -#### robots.txt +### sitemap.xml -- Allows all crawlers -- Disallows `/api/` routes -- References generated sitemap +Includes: +- Base URL +- Current build date as `` +- Change frequency +- Priority value -### Design Considerations +### robots.txt -- Uses Webpack’s asset injection system -- Avoids filesystem writes during build -- Ensures generated files are part of the final artifact +Includes: +- Allow all crawlers +- Disallow `/api/` +- Reference to sitemap location ---- +## Configuration Options -## Responsibilities Within the Frontend Architecture +| Option | Default | Description | +|--------|---------|------------| +| `baseUrl` | `https://www.mlg.soccer` | Root URL used in sitemap and robots file | -The Webpack Plugins module supports the frontend build pipeline by providing: +## Design Characteristics -| Concern | Responsibility | -|----------|----------------| -| Branding | Generate consistent favicon from SVG | -| SEO | Provide sitemap and crawler configuration | -| Automation | Eliminate manual asset management | -| Performance | Avoid redundant file generation | - -It complements frontend components, hooks, services, and types by enhancing the production build rather than modifying runtime behavior. +- Generated entirely in memory during compilation. +- Injected directly into `compilation.assets`. +- No filesystem writes required. +- Automatically reflects the current build date. --- -## Separation of Concerns +# Interaction with the Frontend Application -```mermaid -flowchart LR - Runtime["Frontend Application Code"] --> Browser["Browser Runtime"] - BuildTime["Webpack Plugins"] --> Output["Static Build Artifacts"] +Although these plugins live alongside the frontend codebase, they: - Runtime -.->|"No Direct Dependency"| BuildTime -``` +- Do not modify React components. +- Do not impact bundle size directly. +- Operate strictly during the Webpack compilation phase. -- The application code does not depend on these plugins. -- The plugins do not modify application logic. -- They operate strictly at build time. +They complement: ---- +- **Frontend Components** (UI rendering) +- **Frontend Services** (API communication) +- **Frontend Types** (TypeScript domain models) -## Benefits of the Approach - -1. **Single Source of Truth** – SVG-based favicon management. -2. **SEO Automation** – Always up-to-date sitemap and robots configuration. -3. **Build Consistency** – Works identically in local and CI environments. -4. **Incremental Efficiency** – Avoids redundant image processing. -5. **Clear Responsibility Boundary** – Isolated build-time concerns. +by ensuring that build artifacts meet production readiness standards. --- +# Build-Time vs Runtime Responsibilities + +```mermaid +flowchart LR + subgraph BuildTime["Build Time"] + WP["Webpack"] --> FP["FaviconGeneratorPlugin"] + WP --> SP["SeoFilesPlugin"] + end + + subgraph Runtime["Browser Runtime"] + React["React Application"] + API["Backend API"] + end + + FP --> Assets["Static Assets"] + SP --> Assets + Assets --> React +``` + ## Summary -The **Webpack Plugins** module encapsulates build-time automation logic for the Major League GitHub frontend. It enhances the Webpack pipeline through two focused plugins: +The **Webpack Plugins** module enhances the frontend build pipeline by: -- **FaviconGeneratorPlugin** – Converts SVG to ICO efficiently and conditionally. -- **SeoFilesPlugin** – Injects SEO-critical static files during compilation. +- Automating favicon generation from SVG sources. +- Ensuring SEO metadata is always present and correct. +- Keeping branding and indexing artifacts synchronized with each build. -Together, they ensure the application’s static assets and search engine configuration remain accurate, automated, and production-ready without introducing runtime complexity. \ No newline at end of file +By embedding these concerns directly into the Webpack lifecycle, the system ensures consistent, reproducible, and production-ready frontend builds without adding runtime complexity. \ No newline at end of file