Learning Lens

Privacy-first learning observation for educators — driven by a full Metal 3D game engine, not 2D SwiftUI toys.

Every scored activity (and every practice round) runs through Apple Metal: an MTKView, custom shaders, depth-buffered meshes, and 3D raycast touch. SwiftUI only owns the HUD chrome around the canvas. Task evidence stays on-device; outputs are bounded observation cards — never diagnoses.

Learning Lens does not diagnose learning disabilities, screen for disorders, infer emotion, or replace a qualified professional. Educators interpret evidence; software describes task behavior.

Public repo: https://github.com/moegon/LearningLens

---

Why Metal

Classroom sessions need clear motion, spatial targets, and reliable hit-testing — the same frame loop that drives the visuals also drives the timing evidence used for observations.

Layer	Owns
Metal (LearningLensMetal)	Real-time 3D presentation, meshes, lighting, camera, touch rays
Core (LearningLensCore)	Generators, scoring, observation language, practice recipes
SwiftUI shell	Menu, progress label, observation cards, session navigation

flowchart LR
  UI["SwiftUI session shell"] --> MTK["MTKView + MetalRenderer"]
  MTK --> Scene["MetalGameScene"]
  Scene --> Events["GameEvent stream"]
  Events --> Core["LearningLensCore scoring"]
  Core --> Cards["Educator observation cards"]
  Cards --> Practice["Unscored Metal practice"]

Loading

There is no SceneKit / SpriteKit / UIKit drawing path for games. Play always goes through MetalGameCanvas → MetalRenderer → a concrete MetalGameScene.

---

Metal engine (how play works)

Package: apple/Packages/LearningLensMetal

Piece	Role
MetalRenderer	MTKViewDelegate draw loop, depth stencil, perspective camera, lit fragment shader (runtime-compiled MSL)
Mesh bank	Shared GPU buffers for box, sphere, and capsule primitives
RenderObject	Per-frame instance: mesh, transform, color, highlight / emissive pulse
TouchRay + raycaster	Screen tap → world ray → closest sphere-bound intersection
MetalGameScene	Protocol: update, touch handlers, objects, camera, drainEvents()
MetalGameCanvas	SwiftUI UIViewRepresentable / NSViewRepresentable bridge to MTKView
BeatCuePlayer	On-device audio cues for Beat Bridges (no cloud)

Each frame:

1. Scene update(deltaTime:) advances trial state.
2. Renderer builds MVP uniforms and draws up to 64 objects with simple directional + rim lighting.
3. Touches become TouchRays; scenes emit typed GameEvents (targetHit, pathSampled, beatHit, …).
4. Host views map finished scenes into Core result models; Core builds observational cards.

Practice rounds use the same Metal pipeline with PracticeRecipe difficulty knobs — results never rewrite the observation profile.

---

Five Metal games

SessionPackBuilder shuffles all five titles for a full offline session. Educators can also launch any single game from the main menu.

Game	Metal mechanic	Observation domain(s)
Prism Memory	Glowing prisms light in sequence; learner taps recall order	Visual working memory, sequencing
Ribbon Run	Trace a glowing 3D ribbon on the ground plane	Path following
Orbit Tap	Tap spawned 3D orbs as they appear	Tap-target timing (hit rate + latency bands)
Rule Shift Sort	Sort tokens by color, then an announced switch to shape	Rule shifting (task behavior only)
Beat Bridges	Tap stepping stones near slow due beats	Rhythm sequencing

Screenshots (current shell; Metal canvas is the play surface):

Welcome	Pattern lineage	Observations

---

Try the iOS beta (TestFlight)

Public invite: Join Learning Lens on TestFlight

1. On an iPhone or iPad, install TestFlight.
2. Open the invite on that device: https://testflight.apple.com/join/e3Q3mtsz
   (or itms-beta://testflight.apple.com/join/e3Q3mtsz)
3. Tap Accept, then Install.

Privacy policy: https://moegon.github.io/LearningLens/privacy/

---

Session workflow

1. Educator starts a session — no child login, no account, no network required.
2. Learner plays the shuffled five-game Metal pack.
3. Educator view shows evidence-linked observation cards, caveats, adaptations, and Play practice CTAs.
4. Optional practice = same Metal scenes, unscored.
5. Discard clears in-memory session data.

---

Product posture (brief)

Audience	Value
Educators & learning-support staff	Transparent task observations + low-risk next-activity adaptations
Learners	Short game-like session; no student accounts or lasting child profile
Schools / privacy leads	On-device measurement; no raw media upload; non-diagnostic framing

Refused: disability labels, emotion inference, camera / biometrics, classroom ranking, automated placement.

Control	Behavior
Camera / mic / biometrics	Not in MVP; research path gated by ADR-0002
Raw media	Never uploaded or persisted
Model payloads	Aggregate de-identified observations only (optional); no names, images, notes
Scoring authority	LearningLensCore owns canonical cards; backend may only rephrase

Details: privacy · child safety · camera ADR

---

How to run

Requirements: Xcode 26+, Swift 6, Python 3.13+, uv.

make bootstrap
make test
open LearningLens.xcworkspace

Run scheme LearningLensiOS or LearningLensMac. After project.yml changes: make xcodegen.

Optional educator-insight backend:

cp .env.example .env
make backend-run

Without OPENAI_API_KEY (or with OPENAI_ENABLED=false), the backend returns a labelled deterministic local summary. Never commit .env.

---

Repository map

Path	Purpose
apple/Packages/LearningLensMetal	Full Metal renderer + five 3D game scenes
apple/Packages/LearningLensCore	Generators, scoring, profiles, practice recipes
apple/App	SwiftUI shell (MetalGameHostView, menu, observations)
backend/	Optional FastAPI educator-insight service
docs/governance/	Privacy & child-safety controls
docs/bmad/	PRD / architecture

Architecture sketch: docs/bmad/planning-artifacts/architecture.md

License

Apache-2.0. See LICENSE.