From 5e4f4d2bdab801df8e7eb07ef6a8f2bf626d3098 Mon Sep 17 00:00:00 2001 From: calebrob6 Date: Thu, 6 Aug 2026 15:52:07 +0000 Subject: [PATCH 1/4] feat(ui): improve labeling map controls Enable swipe comparison by default, standardize map shortcuts with collapsible help, and add a gated misclassification review mode to the interactive labeler. Document the behavior and validation plan across the relevant usage guides and feature spec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59694b49-6899-45d5-b492-c0b2dc13cd5e --- docs/usage/damage-mapping.md | 4 +- docs/usage/rapid-building-assessment.md | 13 +- .../features/labeling-view-controls/README.md | 54 +++++ .../features/labeling-view-controls/design.md | 66 ++++++ spec/features/labeling-view-controls/plan.md | 41 ++++ .../labeling-view-controls/test-plan.md | 58 +++++ .../labeling-view-controls/user-stories.md | 94 ++++++++ ui/src/AppContext.jsx | 2 +- .../BuildingValidation/BuildingValidation.jsx | 17 +- .../BuildingValidationRightPanel.jsx | 3 + .../Components/HelpDocs/HelpDocsLabeling.jsx | 2 +- .../Components/HelpDocs/HelpDocsResults.jsx | 4 + .../InteractiveLabeler/InteractiveLabeler.jsx | 218 ++++++++++++++---- ui/src/Components/KeyboardShortcutHelp.jsx | 143 ++++++++++++ .../LabelingTool/LabelingToolLeftPanel.jsx | 17 +- ui/src/Components/LabelingTool/LeftPanel.jsx | 15 -- ui/src/Components/Visualizer/InfoPanel.jsx | 3 + ui/src/Components/Visualizer/Visualizer.jsx | 12 +- .../VisualizerInformationMobile.jsx | 4 + ui/src/Components/keyboardShortcuts.js | 62 +++++ 20 files changed, 746 insertions(+), 86 deletions(-) create mode 100644 spec/features/labeling-view-controls/README.md create mode 100644 spec/features/labeling-view-controls/design.md create mode 100644 spec/features/labeling-view-controls/plan.md create mode 100644 spec/features/labeling-view-controls/test-plan.md create mode 100644 spec/features/labeling-view-controls/user-stories.md create mode 100644 ui/src/Components/KeyboardShortcutHelp.jsx create mode 100644 ui/src/Components/keyboardShortcuts.js diff --git a/docs/usage/damage-mapping.md b/docs/usage/damage-mapping.md index 3ce0cc92..df533830 100644 --- a/docs/usage/damage-mapping.md +++ b/docs/usage/damage-mapping.md @@ -57,8 +57,8 @@ labels, or the arrow next to **Save** to save and start training in one step. To make damage easier to see while you label, tune the view of the pre-/post-event imagery: **Opacity**, **Contrast**, **Hue Rotation**, and **Saturation** sliders (with **Reset**). -Toggle between post- and pre-event imagery with the imagery switch or `Ctrl+Alt+C` — if you -didn't upload pre-event imagery, the tool falls back to the Azure Basemap. +Toggle between post- and pre-event imagery with the imagery switch. Press `A` for pre-event +(or the Azure Basemap fallback) and `D` for post-event imagery. ![Labeling imagery properties panel](../_static/usage/labeling/labeling-imagery-properties.png) diff --git a/docs/usage/rapid-building-assessment.md b/docs/usage/rapid-building-assessment.md index b499ca57..04e4ab5f 100644 --- a/docs/usage/rapid-building-assessment.md +++ b/docs/usage/rapid-building-assessment.md @@ -83,9 +83,10 @@ panel with the class selector, counts, a view toggle, and quality metrics. - **Classes:** **Intact** (green), **Damaged** (red), **Cloudy** (purple) for obscured buildings; unlabeled buildings are gray. - **Shortcuts:** `1`/`2`/`3` pick a class, `T` cycles, `P` toggles Labeled/Predicted view, - `Space` shows/hides footprints. + `Space` shows/hides footprints, and `Ctrl+drag` box-labels buildings. Swipe comparison is + on by default; `A`/`S`/`D` move its divider left/to an even split/right. -Once you've labeled at least **3 buildings across 2+ classes**, an in-browser model +Once you've labeled at least **3 buildings in each of 2+ classes**, an in-browser model (logistic regression, WebGPU-accelerated when available) trains automatically and predicts damage for every building in view. Toggle **View: Labeled ↔ Predicted** to compare. The panel shows holdout **precision / recall / F1 for the Damaged class** so you can watch @@ -93,6 +94,11 @@ quality improve as you label diverse examples. ![The Interactive Labeler in Predicted view, with holdout metrics](../_static/usage/interactive/labeler-predicted.png) +Under **Advanced**, **Show misclassified buildings** trains or reuses the current in-browser +model and highlights only human-labeled buildings whose current prediction differs. Correctly +classified and unlabeled buildings are not highlighted. This view is mutually exclusive with +Predicted and Uncertainty views and turns off if labels fall below the training threshold. + ## Step 3 — Predict all buildings - **Save labels** persists your manual labels so you can resume later (no full prediction). @@ -115,7 +121,8 @@ sample of footprints (~200 by default) with the pre-/post-event imagery. - Select a building and label it **Damaged**, **Not Damaged**, or **Unknown**. - **Shortcuts:** `1`/`2`/`3` to label; arrow keys to move Prev/Next (auto-advances to the - next unlabeled). Filter by label status; a progress bar tracks your coverage. + next unlabeled); `A` shows pre-event imagery (or basemap) and `D` shows post-event imagery. + Filter by label status; a progress bar tracks your coverage. - **Save Labels** to persist, or **Download GeoJSON** to export the labeled sample. These human labels are the **ground truth** the reports compare predictions against diff --git a/spec/features/labeling-view-controls/README.md b/spec/features/labeling-view-controls/README.md new file mode 100644 index 00000000..fc225219 --- /dev/null +++ b/spec/features/labeling-view-controls/README.md @@ -0,0 +1,54 @@ +# Feature: Labeling View Controls + +**Status:** implemented +**Date:** 2026-08-05 +**Priority:** P1 +**Work items:** [#47](https://github.com/microsoft/haste/issues/47), +[#100](https://github.com/microsoft/haste/issues/100), +[#104](https://github.com/microsoft/haste/issues/104) + +## Contents + +- [Summary](#summary) +- [Success criteria](#success-criteria) +- [Scope](#scope) +- [Documents](#documents) +- [Issue note](#issue-note) + +## Summary + +Standardize visible keyboard shortcut help and imagery-comparison keys across +Results, Interactive Labeler, Labeling Tool, and Building Validation. Enable +the Interactive Labeler swipe map by default and add an Advanced mode that +highlights human labels that disagree with the current in-browser model. + +## Success Criteria + +- [x] All four views use shared, visible shortcut definitions. +- [x] Shortcuts ignore typing targets and accept lowercase or uppercase letters. +- [x] Imagery comparison uses `A` for pre/left, `S` for split where available, + and `D` for post/right. +- [x] Interactive swipe starts enabled. +- [x] Misclassified mode highlights only labeled prediction mismatches and is + mutually exclusive with Predicted and Uncertainty views. +- [x] Misclassified mode disables and turns off below the training threshold. + +## Scope + +The implementation is UI-only. It reuses Azure Maps, the existing in-browser +model, cached labels/predictions, and feature-state paint expressions. It adds +no API contract or dependency. + +## Documents + +| Document | Purpose | Status | +|---|---|---| +| [design.md](design.md) | UI and map-expression design | implemented | +| [user-stories.md](user-stories.md) | Acceptance criteria and ownership | implemented | +| [plan.md](plan.md) | Phased execution status | implemented | +| [test-plan.md](test-plan.md) | Validation scenarios | ready for validation | + +## Issue Note + +Issue [#104](https://github.com/microsoft/haste/issues/104) tracks enabling +swipe comparison by default in the Interactive Labeler. diff --git a/spec/features/labeling-view-controls/design.md b/spec/features/labeling-view-controls/design.md new file mode 100644 index 00000000..7b21a3af --- /dev/null +++ b/spec/features/labeling-view-controls/design.md @@ -0,0 +1,66 @@ +# Design: Labeling View Controls + +## Contents + +- [Overview](#overview) +- [Shared shortcuts](#shared-shortcuts) +- [Imagery behavior](#imagery-behavior) +- [Misclassified mode](#misclassified-mode) +- [State updates](#state-updates) +- [Accessibility](#accessibility) + +## Overview + +The change stays within `ui/src/Components/`. A shared shortcut data module +defines the labels shown in each view and a shared Fluent UI component renders +them. Each existing view retains its own map and domain behavior. + +## Shared Shortcuts + +`keyboardShortcuts.js` exports view-specific shortcut arrays and the common +typing-target guard. `KeyboardShortcutHelp.jsx` renders those arrays with +semantic keyboard labels. + +| View | Shortcuts | +|---|---| +| Results / Visualizer | `A`, `S`, `D` | +| Interactive Labeler | `1/2/3`, `T`, `P`, `Space`, `Ctrl+drag`, `A/S/D` | +| Labeling Tool | `A`, `D` | +| Building Validation | `1/2/3`, arrows, `A`, `D` | + +## Imagery Behavior + +Swipe views retain divider positions: `A` moves left, `S` centers, and `D` +moves right. Single-map views use `A` to reveal pre-event imagery (or the +basemap fallback) and `D` to reveal post-event imagery. They do not implement +an unsupported split state. + +## Misclassified Mode + +The Advanced toggle is enabled when at least three valid labels exist in each +of two classes, matching the existing `canTrain` condition. Enabling the mode +trains or reuses the existing on-demand model and predicts the viewport. + +The fill expression requires both a valid human `label` feature-state and a +valid `pred` feature-state, then checks that they differ. Mismatches receive a +distinct orange fill. Correctly classified and unlabeled buildings remain +transparent and are not represented as misclassified. + +Predicted, Uncertainty, and Misclassified modes are mutually exclusive. The +misclassified toggle turns off automatically if labels fall below the +training threshold. + +## State Updates + +The mode reuses `labeledMapRef`, `predictionsMapRef`, `trainedModelRef`, and +the existing `maybeTrainAndPredict` path. Viewport hydration reapplies label +and prediction state before evaluating the expression. Label additions, +box-labeling, removals, retraining, full prediction, and clearing all trigger +the same hydration or feature-state reset paths. + +## Accessibility + +Letter comparisons normalize `event.key` to lowercase, so uppercase and +lowercase work. Global handlers return for editable controls, buttons, links, +and Fluent UI switch controls so they do not override typing, activation, or +browser shortcuts. Visible help uses `kbd`, `dt`, and `dd` semantics. diff --git a/spec/features/labeling-view-controls/plan.md b/spec/features/labeling-view-controls/plan.md new file mode 100644 index 00000000..c15cb1b0 --- /dev/null +++ b/spec/features/labeling-view-controls/plan.md @@ -0,0 +1,41 @@ +# Plan: Labeling View Controls + +## Contents + +- [Phase 1: Specification](#phase-1-specification) +- [Phase 2: Shared controls](#phase-2-shared-controls) +- [Phase 3: View integration](#phase-3-view-integration) +- [Phase 4: Validation](#phase-4-validation) + +## Phase 1: Specification + +| Task | Agent | Story | Status | +|---|---|---|---| +| Define shortcut, swipe-default, and disagreement behavior | `ui` | US-001–004 | done | +| Define UI validation coverage | `ui` | US-001–004 | done | + +## Phase 2: Shared Controls + +| Task | Agent | Story | Status | +|---|---|---|---| +| Add shared shortcut definitions and typing guard | `ui` | US-001/002 | done | +| Add shared Fluent UI shortcut-help component | `ui` | US-001 | done | + +## Phase 3: View Integration + +| Task | Agent | Story | Status | +|---|---|---|---| +| Results / Visualizer shortcut help and guarded handler | `ui` | US-001/002 | done | +| Interactive shortcuts and default swipe | `ui` | US-001–003 | done | +| Labeling Tool `A`/`D` imagery controls | `ui` | US-001/002 | done | +| Building Validation imagery keys and help | `ui` | US-001/002 | done | +| Advanced misclassified mode | `ui` | US-004 | done | +| Update in-app and usage help | `ui` | US-001/002 | done | + +## Phase 4: Validation + +| Task | Agent | Story | Status | +|---|---|---|---| +| Run UI lint | `ui` | US-001–004 | blocked — ESLint 9 does not load the repository `.eslintrc.cjs` | +| Run production build | `ui` | US-001–004 | done | +| Execute browser scenarios in test plan | `ui-validation` | US-001–004 | pending | diff --git a/spec/features/labeling-view-controls/test-plan.md b/spec/features/labeling-view-controls/test-plan.md new file mode 100644 index 00000000..5ac3e932 --- /dev/null +++ b/spec/features/labeling-view-controls/test-plan.md @@ -0,0 +1,58 @@ +# Test Plan: Labeling View Controls + +## Contents + +- [Strategy](#strategy) +- [Shortcut scenarios](#shortcut-scenarios) +- [Misclassified scenarios](#misclassified-scenarios) +- [Regression scenarios](#regression-scenarios) +- [Sign-off](#sign-off) + +## Strategy + +The `ui` agent validates static quality with ESLint and a production Vite +build. The `ui-validation` agent validates user behavior in a browser with +Playwright, using mocked app/API data where practical. + +## Shortcut Scenarios + +| ID | View | Scenario | Expected | +|---|---|---|---| +| UI-001 | Results | Press `A`, `S`, `D` | Divider moves left, center, right | +| UI-002 | Interactive | Press `1/2/3`, `T`, `P`, `Space` | Documented action runs | +| UI-003 | Interactive | Use `Ctrl+drag`, then `A/S/D` | Box label and swipe actions run | +| UI-004 | Labeling Tool | Press `A`, `D` | Pre/post selection works | +| UI-005 | Validation | Press `1/2/3`, arrows, `A/D` | Label, navigation, and imagery actions work | +| UI-006 | All | Repeat letter keys with Shift/Caps Lock | Same action runs | +| UI-007 | All | Use shortcut keys in editable or interactive controls | No global shortcut overrides the control | +| UI-008 | All | Inspect visible shortcut help | Text matches behavior | + +## Misclassified Scenarios + +| ID | Scenario | Expected | +|---|---|---| +| UI-009 | Fewer than 3 labels in 2 classes | Toggle disabled with training-threshold copy | +| UI-010 | Enable at threshold | Model trains on demand; mismatch legend appears | +| UI-011 | Labeled prediction differs | Building receives misclassified emphasis | +| UI-012 | Labeled prediction matches | Building has no misclassified fill | +| UI-013 | Building has prediction but no human label | Building has no misclassified fill | +| UI-014 | Change a label or pan to new tiles | Prediction/mismatch display refreshes | +| UI-015 | Retrain or run full prediction | Current prediction state drives mismatch display | +| UI-016 | Enable Predicted or Uncertainty | Misclassified turns off | +| UI-017 | Clear labels or fall below threshold | Misclassified turns off and disables | + +## Regression Scenarios + +| ID | Scenario | Expected | +|---|---|---| +| REG-001 | Open Interactive Labeler | Swipe starts on with pre-left/post-right labels | +| REG-002 | Disable and re-enable swipe | Labeling interactions and map camera still work | +| REG-003 | Hide/show footprints | Labels and predictions remain cached | +| REG-004 | Save and restore labels | Hydration restores labels and mismatch evaluation | + +## Sign-off + +- [ ] `cd ui && npm run lint` (blocked: ESLint 9 requires an + `eslint.config.*`; this repository currently provides `.eslintrc.cjs`) +- [x] `cd ui && npm run build` +- [ ] `ui-validation` Playwright scenarios pass. diff --git a/spec/features/labeling-view-controls/user-stories.md b/spec/features/labeling-view-controls/user-stories.md new file mode 100644 index 00000000..e2d8e938 --- /dev/null +++ b/spec/features/labeling-view-controls/user-stories.md @@ -0,0 +1,94 @@ +# User Stories: Labeling View Controls + +## Contents + +- [Stories](#stories) +- [Agent assignment](#agent-assignment) +- [Out of scope](#out-of-scope) +- [Traceability](#traceability) + +## Stories + +### US-001: Discover shortcuts in every labeling-related view + +**As an** analyst, **I want** visible, consistent shortcut help, **so that** I +can work quickly without guessing controls. + +```gherkin +Given I open Results, Interactive Labeler, Labeling Tool, or Building Validation +Then the view shows shortcut help that matches its implemented behavior +And letter shortcuts work in lowercase and uppercase +And shortcuts do not run while I type in an editable control +``` + +### US-002: Compare imagery consistently + +**As an** analyst, **I want** consistent imagery keys, **so that** comparison +behavior transfers between views. + +```gherkin +Given a view supports swipe comparison +When I press A, S, or D +Then the divider moves left, split, or right respectively +``` + +```gherkin +Given a view uses a single map +When I press A or D +Then pre-event/basemap or post-event imagery is shown respectively +And no split behavior is added +``` + +### US-003: Start Interactive Labeler in swipe mode + +**As an** analyst, **I want** pre/post swipe available immediately, **so that** +I can compare imagery while assigning labels. + +```gherkin +Given the Interactive Labeler finishes loading +Then Swipe (pre-event) is on by default +And pre imagery is on the left and post imagery is on the right +``` + +### US-004: Review model disagreements + +**As an** analyst, **I want** to highlight model disagreements, **so that** I +can focus corrections on useful examples. + +```gherkin +Given at least 3 valid labels exist in at least 2 classes +When I enable Show misclassified buildings under Advanced +Then the current model trains on demand or is reused +And only buildings with a human label whose prediction differs are emphasized +And the legend explains the mode +``` + +```gherkin +Given Misclassified mode is active +When Predicted or Uncertainty view is enabled +Then Misclassified mode turns off +When labels fall below the training threshold +Then Misclassified mode turns off and becomes disabled +``` + +## Agent Assignment + +| Story | Implementing agent | Validating agent | UI path | +|---|---|---|---| +| US-001 | `ui` | `ui-validation` | shared shortcut component and four views | +| US-002 | `ui` | `ui-validation` | Visualizer, Interactive Labeler, Labeling Tool, Building Validation | +| US-003 | `ui` | `ui-validation` | `InteractiveLabeler/InteractiveLabeler.jsx` | +| US-004 | `ui` | `ui-validation` | `InteractiveLabeler/InteractiveLabeler.jsx` | + +## Out of Scope + +- API or persistence changes. +- A split view for single-map labeling or validation. +- A new model training implementation. + +## Traceability + +US-001 and US-002 implement +[#47](https://github.com/microsoft/haste/issues/47). US-004 implements +[#100](https://github.com/microsoft/haste/issues/100). US-003 implements +[#104](https://github.com/microsoft/haste/issues/104). diff --git a/ui/src/AppContext.jsx b/ui/src/AppContext.jsx index 1e454762..c9c12d92 100644 --- a/ui/src/AppContext.jsx +++ b/ui/src/AppContext.jsx @@ -140,7 +140,7 @@ export const AppProvider = ({ children }) => { target: `#postEventImagery`, title: "Imagery toggle", content: - "Click on the toggle to switch between post event and pre event imagery. If you did not upload pre event imagery, the tool will default to Azure Basemap. You can also use the keyboard shortcut - Ctrl+Alt+c", + "Click the toggle to switch imagery. Press A for pre event (or Azure Basemap) and D for post event.", }, { type: "teachingBubble", diff --git a/ui/src/Components/BuildingValidation/BuildingValidation.jsx b/ui/src/Components/BuildingValidation/BuildingValidation.jsx index 05fcea25..3ba38b89 100644 --- a/ui/src/Components/BuildingValidation/BuildingValidation.jsx +++ b/ui/src/Components/BuildingValidation/BuildingValidation.jsx @@ -311,9 +311,11 @@ const BuildingValidation = () => { } }, [showFill, isMapReady]); - // Toggle post-event imagery layer so the user can compare the - // post-event view against the basemap satellite underneath. + // A single-map comparison: post-event on, or pre-event/basemap when off. useEffect(() => { + if (preImageryRef.current) { + preImageryRef.current.setOptions({ visible: !showPostImagery }); + } if (postImageryRef.current) { postImageryRef.current.setOptions({ visible: showPostImagery }); } @@ -487,12 +489,14 @@ const BuildingValidation = () => { // Keyboard shortcuts: // 1 / 2 / 3 — assign Damaged / NotDamaged / Unknown // ArrowLeft / ArrowRight — move through the filtered list + // A / D — show pre (or basemap) / post imagery // The right-panel toggles and dropdown remain focusable; the - // INPUT/TEXTAREA/SELECT guard keeps shortcuts from hijacking typing. + // editable-target guard keeps shortcuts from hijacking typing. useEffect(() => { const keyMap = { "1": "Damaged", "2": "NotDamaged", "3": "Unknown" }; function onKeyDown(e) { - if (["INPUT", "TEXTAREA", "SELECT"].includes(e.target.tagName)) return; + if (shouldIgnoreShortcut(e)) return; + if (e.ctrlKey || e.altKey || e.metaKey) return; const labelValue = keyMap[e.key]; if (labelValue) { handleLabel(labelValue); @@ -504,6 +508,10 @@ const BuildingValidation = () => { } else if (e.key === "ArrowRight") { e.preventDefault(); navigateInFilter(1); + } else { + const key = e.key.toLowerCase(); + if (key === "a") setShowPostImagery(false); + else if (key === "d") setShowPostImagery(true); } } window.addEventListener("keydown", onKeyDown); @@ -596,6 +604,7 @@ const BuildingValidation = () => { setShowFill={setShowFill} showPostImagery={showPostImagery} setShowPostImagery={setShowPostImagery} + hasPreImagery={!!preImageryRef.current} hasPostImagery={!!postImageryRef.current} /> )} diff --git a/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx b/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx index 1bcdc932..c1728081 100644 --- a/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx +++ b/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx @@ -143,6 +143,7 @@ const BuildingValidationRightPanel = ({ setShowFill, showPostImagery, setShowPostImagery, + hasPreImagery, hasPostImagery, }) => { const styles = useStyles(); @@ -294,6 +295,8 @@ const BuildingValidationRightPanel = ({
■ Unknown
+ + {/* Actions */}
)} diff --git a/ui/src/Components/KeyboardShortcutHelp.jsx b/ui/src/Components/KeyboardShortcutHelp.jsx new file mode 100644 index 00000000..e2a39336 --- /dev/null +++ b/ui/src/Components/KeyboardShortcutHelp.jsx @@ -0,0 +1,143 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. +import { useState } from "react"; +import { ActionButton } from "@fluentui/react"; +import PropTypes from "prop-types"; + +const rootStyle = { + color: "#605e5c", + fontSize: 11, + marginTop: 8, +}; + +const itemStyle = { + display: "block", + height: "auto", + margin: 0, + padding: "0 0 8px", + position: "relative", + width: "100%", +}; + +const keysStyle = { + alignItems: "center", + display: "flex", + flexWrap: "wrap", + gap: 4, + lineHeight: "20px", + minHeight: 20, + position: "relative", + width: "100%", +}; + +const keyGroupStyle = { + alignItems: "center", + display: "inline-flex", + gap: 4, +}; + +const keyStyle = { + background: "#f3f2f1", + border: "1px solid #c8c6c4", + borderRadius: 3, + color: "#323130", + display: "inline-block", + fontFamily: "inherit", + fontSize: 10, + fontWeight: 600, + lineHeight: "16px", + minWidth: 18, + padding: "0 4px", + textAlign: "center", + whiteSpace: "nowrap", +}; + +const separatorStyle = { + color: "#8a8886", + display: "inline-block", + lineHeight: "16px", + whiteSpace: "pre", +}; + +const descriptionStyle = { + clear: "both", + display: "block", + fontSize: 11, + height: "auto", + lineHeight: "16px", + margin: "3px 0 0", + overflow: "visible", + overflowWrap: "break-word", + position: "relative", + whiteSpace: "normal", + width: "100%", +}; + +const KeyboardShortcutHelp = ({ + shortcuts, + title = "Keyboard shortcuts", +}) => { + const [isExpanded, setIsExpanded] = useState(true); + + return ( +
+ setIsExpanded((expanded) => !expanded)} + styles={{ + root: { + color: "#323130", + fontSize: 11, + fontWeight: 600, + height: 24, + padding: 0, + }, + icon: { fontSize: 10 }, + }} + > + {title} + + {isExpanded && ( +
+ {shortcuts.map((shortcut) => ( +
+
+ {shortcut.keys.map((key, index) => ( + + {index > 0 && ( + + {shortcut.separator || "/"} + + )} + {key} + + ))} +
+
{shortcut.description}
+
+ ))} +
+ )} +
+ ); +}; + +KeyboardShortcutHelp.propTypes = { + shortcuts: PropTypes.arrayOf( + PropTypes.shape({ + keys: PropTypes.arrayOf(PropTypes.string).isRequired, + separator: PropTypes.string, + description: PropTypes.string.isRequired, + }) + ).isRequired, + title: PropTypes.string, +}; + +export default KeyboardShortcutHelp; diff --git a/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx b/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx index 348b912e..c5cee8cb 100644 --- a/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx +++ b/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx @@ -13,6 +13,11 @@ import { saveLabels, checkLabelsState } from "./LabelingToolHelper"; import PropType from "prop-types"; import { useNavigate } from "react-router-dom"; +import KeyboardShortcutHelp from "../KeyboardShortcutHelp"; +import { + LABELING_TOOL_SHORTCUTS, + shouldIgnoreShortcut, +} from "../keyboardShortcuts"; const LabelingToolLeftPanel = ({ mapRef, @@ -168,19 +173,21 @@ const LabelingToolLeftPanel = ({ }, [eventImageryVisibilityState]); - // need a listener to update the post event imagery layer visibility on ctrl + p + // A/D are the standard pre/post controls. useEffect(() => { const handleKeyDown = (e) => { - if (e.ctrlKey && e.altKey && e.key === "c") { - setEventImageryVisibilityState((prevState) => !prevState); - } + if (shouldIgnoreShortcut(e)) return; + const key = e.key.toLowerCase(); + if (e.ctrlKey || e.altKey || e.metaKey) return; + if (key === "a") setEventImageryVisibilityState(false); + else if (key === "d") setEventImageryVisibilityState(true); }; window.addEventListener("keydown", handleKeyDown); return () => { window.removeEventListener("keydown", handleKeyDown); }; - }, [eventImageryVisibilityState]); + }, []); // Close the imagery settings panel when the user clicks on the map. useEffect(() => { diff --git a/ui/src/Components/LabelingTool/LeftPanel.jsx b/ui/src/Components/LabelingTool/LeftPanel.jsx index d3a98119..40330df3 100644 --- a/ui/src/Components/LabelingTool/LeftPanel.jsx +++ b/ui/src/Components/LabelingTool/LeftPanel.jsx @@ -164,21 +164,6 @@ const LeftPanel = ({ // eslint-disable-next-line react-hooks/exhaustive-deps }, [eventImageryVisibilityState]); - - // need a listener to update the post event imagery layer visibility on ctrl + p - useEffect(() => { - const handleKeyDown = (e) => { - if (e.ctrlKey && e.altKey && e.key === "c") { - setEventImageryVisibilityState((prevState) => !prevState); - } - }; - - window.addEventListener("keydown", handleKeyDown); - return () => { - window.removeEventListener("keydown", handleKeyDown); - }; - }, [eventImageryVisibilityState]); - const navigate = useNavigate(); const handleBackNavigation = () => { diff --git a/ui/src/Components/Visualizer/InfoPanel.jsx b/ui/src/Components/Visualizer/InfoPanel.jsx index 8aa3d9fc..4406b477 100644 --- a/ui/src/Components/Visualizer/InfoPanel.jsx +++ b/ui/src/Components/Visualizer/InfoPanel.jsx @@ -12,6 +12,8 @@ import { FluentIcon } from "../../util/icons"; import { useState, useContext } from "react"; import PropType from "prop-types"; import { AppContext } from "../../AppContext"; +import KeyboardShortcutHelp from "../KeyboardShortcutHelp"; +import { VISUALIZER_SHORTCUTS } from "../keyboardShortcuts"; const DAMAGE_LEGEND = [ { label: "0 - 20% damaged", color: "#FFFFFF" }, @@ -143,6 +145,7 @@ const InfoPanel = ({ Reset map position + diff --git a/ui/src/Components/Visualizer/Visualizer.jsx b/ui/src/Components/Visualizer/Visualizer.jsx index 9126ce4c..6b79cfd6 100644 --- a/ui/src/Components/Visualizer/Visualizer.jsx +++ b/ui/src/Components/Visualizer/Visualizer.jsx @@ -11,6 +11,7 @@ import { convertDateToString } from "../../util/conversion"; import VisualizerImageryControls from "./VisualizerImageryControls" import "../../assets/css/visualizer.css"; import { getAzureMapsAuthOptions } from "../../util/azureMapsAuth"; +import { shouldIgnoreShortcut } from "../keyboardShortcuts"; const Visualizer = ({ setModalComponent }) => { @@ -228,16 +229,7 @@ const Visualizer = ({ setModalComponent }) => { }, []); const handleKeyboardShortcuts = (event) => { - // Ignore when typing in an input/textarea/contenteditable element. - const target = event.target; - const tag = target?.tagName; - if ( - tag === "INPUT" || - tag === "TEXTAREA" || - target?.isContentEditable - ) { - return; - } + if (shouldIgnoreShortcut(event)) return; if (event.ctrlKey || event.altKey || event.metaKey) { return; } diff --git a/ui/src/Components/Visualizer/VisualizerInformationMobile.jsx b/ui/src/Components/Visualizer/VisualizerInformationMobile.jsx index 0fe8e914..45efa7c7 100644 --- a/ui/src/Components/Visualizer/VisualizerInformationMobile.jsx +++ b/ui/src/Components/Visualizer/VisualizerInformationMobile.jsx @@ -11,6 +11,8 @@ import { FluentIcon } from "../../util/icons"; import { useState } from "react"; import PropType from "prop-types"; +import KeyboardShortcutHelp from "../KeyboardShortcutHelp"; +import { VISUALIZER_SHORTCUTS } from "../keyboardShortcuts"; const VisualizerInformationMobile = ({ visualizerResults, @@ -120,6 +122,8 @@ const VisualizerInformationMobile = ({ ) } /> +
+ diff --git a/ui/src/Components/keyboardShortcuts.js b/ui/src/Components/keyboardShortcuts.js new file mode 100644 index 00000000..3e507b28 --- /dev/null +++ b/ui/src/Components/keyboardShortcuts.js @@ -0,0 +1,62 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +export const VISUALIZER_SHORTCUTS = [ + { + keys: ["A", "S", "D"], + description: "Move the swipe divider left / split / right", + }, +]; + +export const INTERACTIVE_LABELER_SHORTCUTS = [ + { + keys: ["1", "2", "3"], + description: "Set Intact / Damaged / Cloudy", + }, + { keys: ["T"], description: "Cycle the selected class" }, + { keys: ["P"], description: "Toggle Labeled / Predicted view" }, + { keys: ["Space"], description: "Show / hide footprints" }, + { + keys: ["Ctrl", "drag"], + separator: " + ", + description: "Box-label buildings", + }, + { + keys: ["A", "S", "D"], + description: "With Swipe on: move the divider left / split / right", + }, +]; + +export const LABELING_TOOL_SHORTCUTS = [ + { + keys: ["A", "D"], + description: "Show pre (or basemap) / post imagery", + }, +]; + +export const BUILDING_VALIDATION_SHORTCUTS = [ + { + keys: ["1", "2", "3"], + description: "Label Damaged / Not Damaged / Unknown", + }, + { + keys: ["←", "→"], + description: "Previous / next building", + }, + { + keys: ["A", "D"], + description: "Show pre (or basemap) / post imagery", + }, +]; + +export function shouldIgnoreShortcut(event) { + const target = event?.target; + const tagName = target?.tagName?.toUpperCase(); + return ( + ["INPUT", "TEXTAREA", "SELECT", "BUTTON", "A"].includes(tagName) || + target?.isContentEditable === true || + target?.closest?.( + "button, a, [role='button'], [role='link'], [role='switch']" + ) != null + ); +} From 6df3a8fb62dfcd6bb263be6d78ec8ae58cf3bd7d Mon Sep 17 00:00:00 2001 From: calebrob6 Date: Fri, 7 Aug 2026 04:10:24 +0000 Subject: [PATCH 2/4] fix(ui): adopt Fluent UI v9 shortcut control Migrate the collapsible keyboard shortcut header to the Fluent UI v9 Button and shared icon adapter introduced by the main UI restyle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59694b49-6899-45d5-b492-c0b2dc13cd5e --- ui/src/Components/KeyboardShortcutHelp.jsx | 33 ++++++++++++---------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/ui/src/Components/KeyboardShortcutHelp.jsx b/ui/src/Components/KeyboardShortcutHelp.jsx index e2a39336..bb575c00 100644 --- a/ui/src/Components/KeyboardShortcutHelp.jsx +++ b/ui/src/Components/KeyboardShortcutHelp.jsx @@ -1,7 +1,8 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT License. import { useState } from "react"; -import { ActionButton } from "@fluentui/react"; +import { Button } from "@fluentui/react-components"; +import { FluentIcon } from "../util/icons"; import PropTypes from "prop-types"; const rootStyle = { @@ -81,25 +82,27 @@ const KeyboardShortcutHelp = ({ return (
- + } onClick={() => setIsExpanded((expanded) => !expanded)} - styles={{ - root: { - color: "#323130", - fontSize: 11, - fontWeight: 600, - height: 24, - padding: 0, - }, - icon: { fontSize: 10 }, + size="small" + style={{ + color: "#323130", + fontSize: 11, + fontWeight: 600, + height: 24, + minWidth: 0, + padding: 0, }} > {title} - + {isExpanded && (
{shortcuts.map((shortcut) => ( From 6fe0b39b825ba46c18c5e7772ba916161b7c3137 Mon Sep 17 00:00:00 2001 From: calebrob6 Date: Fri, 7 Aug 2026 20:48:53 +0000 Subject: [PATCH 3/4] fix(ui): restore shortcut help after restyle Restore shortcut-help imports and render points lost during the main rebase, and migrate the misclassification controls fully to Fluent UI v9. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59694b49-6899-45d5-b492-c0b2dc13cd5e --- .../BuildingValidationRightPanel.jsx | 13 ++- .../InteractiveLabeler/InteractiveLabeler.jsx | 81 ++++++++++--------- .../LabelingTool/LabelingToolLeftPanel.jsx | 1 + 3 files changed, 54 insertions(+), 41 deletions(-) diff --git a/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx b/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx index c1728081..7de3eb9a 100644 --- a/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx +++ b/ui/src/Components/BuildingValidation/BuildingValidationRightPanel.jsx @@ -10,6 +10,8 @@ import { makeStyles, tokens, } from "@fluentui/react-components"; +import KeyboardShortcutHelp from "../KeyboardShortcutHelp"; +import { BUILDING_VALIDATION_SHORTCUTS } from "../keyboardShortcuts"; const LABEL_OPTIONS = [ { value: "Damaged", label: "Damaged (1)", color: "#C50F1F" }, @@ -203,7 +205,13 @@ const BuildingValidationRightPanel = ({ onChange={(_e, data) => setShowFill(!!data.checked)} /> setShowPostImagery(!!data.checked)} disabled={!hasPostImagery} @@ -288,7 +296,7 @@ const BuildingValidationRightPanel = ({ {/* Legend */}
-
Legend · Hotkeys: 1 / 2 / 3 · ← →
+
Legend
■ Unlabeled
■ Damaged
■ Not Damaged
@@ -338,6 +346,7 @@ BuildingValidationRightPanel.propTypes = { setShowFill: PropTypes.func.isRequired, showPostImagery: PropTypes.bool.isRequired, setShowPostImagery: PropTypes.func.isRequired, + hasPreImagery: PropTypes.bool, hasPostImagery: PropTypes.bool, }; diff --git a/ui/src/Components/InteractiveLabeler/InteractiveLabeler.jsx b/ui/src/Components/InteractiveLabeler/InteractiveLabeler.jsx index a29d3d5e..88ec5a89 100644 --- a/ui/src/Components/InteractiveLabeler/InteractiveLabeler.jsx +++ b/ui/src/Components/InteractiveLabeler/InteractiveLabeler.jsx @@ -1314,6 +1314,18 @@ const InteractiveLabeler = () => { Object.values(labeledMapRef.current).forEach((e) => { next[e.label] = (next[e.label] || 0) + 1; }); + const nextCanTrain = + [ + next[CLASS_INTACT], + next[CLASS_DAMAGED], + next[CLASS_CLOUDY], + ].filter((count) => count >= MIN_PER_CLASS).length >= 2; + canTrainRef.current = nextCanTrain; + if (!nextCanTrain) { + setUncertaintyOn(false); + setMisclassifiedOn(false); + setViewMode("label"); + } setCounts(next); } @@ -1997,28 +2009,12 @@ const InteractiveLabeler = () => { const totalLabeled = counts[0] + counts[1] + counts[2]; // Predicted, Uncertainty, and Misclassified views need a trained model, // which needs at least MIN_PER_CLASS labels in 2+ classes. - const trainableClassCounts = getValidLabeledEntries().reduce( - (classCounts, entry) => { - classCounts[entry.label] = (classCounts[entry.label] || 0) + 1; - return classCounts; - }, - {} - ); const canTrain = - Object.values(trainableClassCounts).filter((n) => n >= MIN_PER_CLASS) - .length >= 2; - - // If labels drop back below the trainable threshold (e.g. after clearing), - // fall back to the Labeled view so we don't sit in a now-disabled Predicted - // Uncertainty, or Misclassified view with no model behind it. - useEffect(() => { - canTrainRef.current = canTrain; - if (!canTrain) { - setUncertaintyOn(false); - setMisclassifiedOn(false); - setViewMode("label"); - } - }, [canTrain]); + [ + counts[CLASS_INTACT], + counts[CLASS_DAMAGED], + counts[CLASS_CLOUDY], + ].filter((count) => count >= MIN_PER_CLASS).length >= 2; return (
@@ -2204,8 +2200,11 @@ const InteractiveLabeler = () => { disabled={!canTrain} onChange={(_e, data) => { setViewMode(data.checked ? "predict" : "label"); - // Predicted view and Uncertainty view are mutually exclusive. - if (data.checked) setUncertaintyOn(false); + // Model-driven review views are mutually exclusive. + if (data.checked) { + setUncertaintyOn(false); + setMisclassifiedOn(false); + } }} style={{ marginTop: 12 }} /> @@ -2421,9 +2420,11 @@ const InteractiveLabeler = () => { disabled={!canTrain} onChange={(_e, data) => { setUncertaintyOn(!!data.checked); - // Uncertainty view and Predicted view are mutually exclusive; - // switching this on drops the map back to the Labeled view. - if (data.checked) setViewMode("label"); + // Model-driven review views are mutually exclusive. + if (data.checked) { + setViewMode("label"); + setMisclassifiedOn(false); + } }} />
@@ -2432,17 +2433,15 @@ const InteractiveLabeler = () => { A legend appears on the map.
- + - { - setMisclassifiedOn(!!checked); - if (checked) { + onChange={(_e, data) => { + setMisclassifiedOn(!!data.checked); + if (data.checked) { // Training is on demand: hydration trains/reuses the model // and refreshes predictions for the visible buildings. setViewMode("label"); @@ -2451,7 +2450,10 @@ const InteractiveLabeler = () => { } }} /> -
+
Trains or reuses the current model when enabled, then highlights only human-labeled buildings whose predicted class differs. Correct and unlabeled buildings stay unhighlighted. @@ -2461,11 +2463,12 @@ const InteractiveLabeler = () => {
- Click a building to label it · right-click to clear ·{" "} - Ctrl+drag to box-label · 1/2/ - 3 set class · P toggle view ·{" "} - Space show/hide footprints · with swipe on,{" "} - A/S/D snap divider left/center/right +
+ Click a building to label it · right-click to clear it +
+
)} diff --git a/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx b/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx index c5cee8cb..da11902d 100644 --- a/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx +++ b/ui/src/Components/LabelingTool/LabelingToolLeftPanel.jsx @@ -381,6 +381,7 @@ const LabelingToolLeftPanel = ({ } />
+
)} From 856db4c522d9fcc03f12fe389a66e8359d5d1be1 Mon Sep 17 00:00:00 2001 From: calebrob6 Date: Fri, 7 Aug 2026 21:00:41 +0000 Subject: [PATCH 4/4] docs: add keyboard shortcuts reference Replace the implementation spec with a user-facing keyboard shortcuts page and link it from the User Guide navigation and workflow documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 59694b49-6899-45d5-b492-c0b2dc13cd5e --- docs/_toc.yml | 1 + docs/usage/damage-mapping.md | 3 +- docs/usage/keyboard-shortcuts.md | 66 +++++++++++++ docs/usage/overview.md | 2 + docs/usage/rapid-building-assessment.md | 1 + .../features/labeling-view-controls/README.md | 54 ----------- .../features/labeling-view-controls/design.md | 66 ------------- spec/features/labeling-view-controls/plan.md | 41 -------- .../labeling-view-controls/test-plan.md | 58 ------------ .../labeling-view-controls/user-stories.md | 94 ------------------- 10 files changed, 72 insertions(+), 314 deletions(-) create mode 100644 docs/usage/keyboard-shortcuts.md delete mode 100644 spec/features/labeling-view-controls/README.md delete mode 100644 spec/features/labeling-view-controls/design.md delete mode 100644 spec/features/labeling-view-controls/plan.md delete mode 100644 spec/features/labeling-view-controls/test-plan.md delete mode 100644 spec/features/labeling-view-controls/user-stories.md diff --git a/docs/_toc.yml b/docs/_toc.yml index dba5c7e0..b4d283d0 100644 --- a/docs/_toc.yml +++ b/docs/_toc.yml @@ -4,6 +4,7 @@ parts: - caption: User Guide chapters: - file: usage/overview + - file: usage/keyboard-shortcuts - file: usage/rapid-building-assessment - file: usage/damage-mapping - file: usage/building-blocks diff --git a/docs/usage/damage-mapping.md b/docs/usage/damage-mapping.md index df533830..41eefefb 100644 --- a/docs/usage/damage-mapping.md +++ b/docs/usage/damage-mapping.md @@ -138,7 +138,8 @@ use the predictions. control, the predicted damage layer overlaid on both, and an optional **raw predictions** layer you can toggle on. Imagery sliders (opacity, contrast, hue, saturation) and keyboard shortcuts — `A` (all pre-event), `S` (split), `D` (all post-event) — help you inspect and -share the result. +share the result. See {doc}`Keyboard shortcuts ` for the complete +reference. ![The raw per-pixel predictions layer toggled on](../_static/usage/results/raw-predictions-layer.png) diff --git a/docs/usage/keyboard-shortcuts.md b/docs/usage/keyboard-shortcuts.md new file mode 100644 index 00000000..c69c062d --- /dev/null +++ b/docs/usage/keyboard-shortcuts.md @@ -0,0 +1,66 @@ +# Keyboard Shortcuts + +HASTE provides keyboard shortcuts for imagery comparison, labeling, and +building validation. Each supported view also includes a collapsible +**Keyboard shortcuts** section. + +## Contents + +- [General behavior](#general-behavior) +- [Results viewer](#results-viewer) +- [Interactive Labeler](#interactive-labeler) +- [Labeling Tool](#labeling-tool) +- [Building Validation](#building-validation) + +## General Behavior + +Letter shortcuts are not case-sensitive. HASTE ignores global shortcuts while +you type in a field or interact with a button, link, dropdown, or switch. + +The imagery comparison keys use the same direction across views: + +- `A` shows pre-event imagery or moves a swipe divider left. +- `S` centers a swipe divider when the view supports split comparison. +- `D` shows post-event imagery or moves a swipe divider right. + +## Results Viewer + +| Shortcut | Action | +|---|---| +| `A` | Move the swipe divider left to show pre-event imagery. | +| `S` | Center the swipe divider for an even comparison. | +| `D` | Move the swipe divider right to show post-event imagery. | + +## Interactive Labeler + +| Shortcut | Action | +|---|---| +| `1` | Select **Intact**. | +| `2` | Select **Damaged**. | +| `3` | Select **Cloudy**. | +| `T` | Cycle through the available classes. | +| `P` | Toggle between labeled and predicted views when the model can train. | +| `Space` | Show or hide building footprints. | +| `Ctrl` + drag | Box-label buildings. | +| `A` | Move the enabled swipe divider left. | +| `S` | Center the enabled swipe divider. | +| `D` | Move the enabled swipe divider right. | + +## Labeling Tool + +| Shortcut | Action | +|---|---| +| `A` | Show pre-event imagery, or the basemap when pre-event imagery is unavailable. | +| `D` | Show post-event imagery. | + +## Building Validation + +| Shortcut | Action | +|---|---| +| `1` | Label the selected building **Damaged**. | +| `2` | Label the selected building **Not Damaged**. | +| `3` | Label the selected building **Unknown**. | +| `Left Arrow` | Select the previous building in the current filter. | +| `Right Arrow` | Select the next building in the current filter. | +| `A` | Show pre-event imagery, or the basemap when pre-event imagery is unavailable. | +| `D` | Show post-event imagery. | diff --git a/docs/usage/overview.md b/docs/usage/overview.md index 24f38bb3..ec4e9cf3 100644 --- a/docs/usage/overview.md +++ b/docs/usage/overview.md @@ -37,3 +37,5 @@ continuous, pixel-level damage raster. Both workflows start the same way — create a project and add an image layer — then diverge. Those shared building blocks have their own pages: {doc}`Projects `, {doc}`Image layers `, and the {doc}`Model catalog `. +See {doc}`Keyboard shortcuts ` for the controls available +across map and labeling views. diff --git a/docs/usage/rapid-building-assessment.md b/docs/usage/rapid-building-assessment.md index 04e4ab5f..f4cbcf74 100644 --- a/docs/usage/rapid-building-assessment.md +++ b/docs/usage/rapid-building-assessment.md @@ -85,6 +85,7 @@ panel with the class selector, counts, a view toggle, and quality metrics. - **Shortcuts:** `1`/`2`/`3` pick a class, `T` cycles, `P` toggles Labeled/Predicted view, `Space` shows/hides footprints, and `Ctrl+drag` box-labels buildings. Swipe comparison is on by default; `A`/`S`/`D` move its divider left/to an even split/right. + See {doc}`Keyboard shortcuts ` for the complete reference. Once you've labeled at least **3 buildings in each of 2+ classes**, an in-browser model (logistic regression, WebGPU-accelerated when available) trains automatically and predicts diff --git a/spec/features/labeling-view-controls/README.md b/spec/features/labeling-view-controls/README.md deleted file mode 100644 index fc225219..00000000 --- a/spec/features/labeling-view-controls/README.md +++ /dev/null @@ -1,54 +0,0 @@ -# Feature: Labeling View Controls - -**Status:** implemented -**Date:** 2026-08-05 -**Priority:** P1 -**Work items:** [#47](https://github.com/microsoft/haste/issues/47), -[#100](https://github.com/microsoft/haste/issues/100), -[#104](https://github.com/microsoft/haste/issues/104) - -## Contents - -- [Summary](#summary) -- [Success criteria](#success-criteria) -- [Scope](#scope) -- [Documents](#documents) -- [Issue note](#issue-note) - -## Summary - -Standardize visible keyboard shortcut help and imagery-comparison keys across -Results, Interactive Labeler, Labeling Tool, and Building Validation. Enable -the Interactive Labeler swipe map by default and add an Advanced mode that -highlights human labels that disagree with the current in-browser model. - -## Success Criteria - -- [x] All four views use shared, visible shortcut definitions. -- [x] Shortcuts ignore typing targets and accept lowercase or uppercase letters. -- [x] Imagery comparison uses `A` for pre/left, `S` for split where available, - and `D` for post/right. -- [x] Interactive swipe starts enabled. -- [x] Misclassified mode highlights only labeled prediction mismatches and is - mutually exclusive with Predicted and Uncertainty views. -- [x] Misclassified mode disables and turns off below the training threshold. - -## Scope - -The implementation is UI-only. It reuses Azure Maps, the existing in-browser -model, cached labels/predictions, and feature-state paint expressions. It adds -no API contract or dependency. - -## Documents - -| Document | Purpose | Status | -|---|---|---| -| [design.md](design.md) | UI and map-expression design | implemented | -| [user-stories.md](user-stories.md) | Acceptance criteria and ownership | implemented | -| [plan.md](plan.md) | Phased execution status | implemented | -| [test-plan.md](test-plan.md) | Validation scenarios | ready for validation | - -## Issue Note - -Issue [#104](https://github.com/microsoft/haste/issues/104) tracks enabling -swipe comparison by default in the Interactive Labeler. diff --git a/spec/features/labeling-view-controls/design.md b/spec/features/labeling-view-controls/design.md deleted file mode 100644 index 7b21a3af..00000000 --- a/spec/features/labeling-view-controls/design.md +++ /dev/null @@ -1,66 +0,0 @@ -# Design: Labeling View Controls - -## Contents - -- [Overview](#overview) -- [Shared shortcuts](#shared-shortcuts) -- [Imagery behavior](#imagery-behavior) -- [Misclassified mode](#misclassified-mode) -- [State updates](#state-updates) -- [Accessibility](#accessibility) - -## Overview - -The change stays within `ui/src/Components/`. A shared shortcut data module -defines the labels shown in each view and a shared Fluent UI component renders -them. Each existing view retains its own map and domain behavior. - -## Shared Shortcuts - -`keyboardShortcuts.js` exports view-specific shortcut arrays and the common -typing-target guard. `KeyboardShortcutHelp.jsx` renders those arrays with -semantic keyboard labels. - -| View | Shortcuts | -|---|---| -| Results / Visualizer | `A`, `S`, `D` | -| Interactive Labeler | `1/2/3`, `T`, `P`, `Space`, `Ctrl+drag`, `A/S/D` | -| Labeling Tool | `A`, `D` | -| Building Validation | `1/2/3`, arrows, `A`, `D` | - -## Imagery Behavior - -Swipe views retain divider positions: `A` moves left, `S` centers, and `D` -moves right. Single-map views use `A` to reveal pre-event imagery (or the -basemap fallback) and `D` to reveal post-event imagery. They do not implement -an unsupported split state. - -## Misclassified Mode - -The Advanced toggle is enabled when at least three valid labels exist in each -of two classes, matching the existing `canTrain` condition. Enabling the mode -trains or reuses the existing on-demand model and predicts the viewport. - -The fill expression requires both a valid human `label` feature-state and a -valid `pred` feature-state, then checks that they differ. Mismatches receive a -distinct orange fill. Correctly classified and unlabeled buildings remain -transparent and are not represented as misclassified. - -Predicted, Uncertainty, and Misclassified modes are mutually exclusive. The -misclassified toggle turns off automatically if labels fall below the -training threshold. - -## State Updates - -The mode reuses `labeledMapRef`, `predictionsMapRef`, `trainedModelRef`, and -the existing `maybeTrainAndPredict` path. Viewport hydration reapplies label -and prediction state before evaluating the expression. Label additions, -box-labeling, removals, retraining, full prediction, and clearing all trigger -the same hydration or feature-state reset paths. - -## Accessibility - -Letter comparisons normalize `event.key` to lowercase, so uppercase and -lowercase work. Global handlers return for editable controls, buttons, links, -and Fluent UI switch controls so they do not override typing, activation, or -browser shortcuts. Visible help uses `kbd`, `dt`, and `dd` semantics. diff --git a/spec/features/labeling-view-controls/plan.md b/spec/features/labeling-view-controls/plan.md deleted file mode 100644 index c15cb1b0..00000000 --- a/spec/features/labeling-view-controls/plan.md +++ /dev/null @@ -1,41 +0,0 @@ -# Plan: Labeling View Controls - -## Contents - -- [Phase 1: Specification](#phase-1-specification) -- [Phase 2: Shared controls](#phase-2-shared-controls) -- [Phase 3: View integration](#phase-3-view-integration) -- [Phase 4: Validation](#phase-4-validation) - -## Phase 1: Specification - -| Task | Agent | Story | Status | -|---|---|---|---| -| Define shortcut, swipe-default, and disagreement behavior | `ui` | US-001–004 | done | -| Define UI validation coverage | `ui` | US-001–004 | done | - -## Phase 2: Shared Controls - -| Task | Agent | Story | Status | -|---|---|---|---| -| Add shared shortcut definitions and typing guard | `ui` | US-001/002 | done | -| Add shared Fluent UI shortcut-help component | `ui` | US-001 | done | - -## Phase 3: View Integration - -| Task | Agent | Story | Status | -|---|---|---|---| -| Results / Visualizer shortcut help and guarded handler | `ui` | US-001/002 | done | -| Interactive shortcuts and default swipe | `ui` | US-001–003 | done | -| Labeling Tool `A`/`D` imagery controls | `ui` | US-001/002 | done | -| Building Validation imagery keys and help | `ui` | US-001/002 | done | -| Advanced misclassified mode | `ui` | US-004 | done | -| Update in-app and usage help | `ui` | US-001/002 | done | - -## Phase 4: Validation - -| Task | Agent | Story | Status | -|---|---|---|---| -| Run UI lint | `ui` | US-001–004 | blocked — ESLint 9 does not load the repository `.eslintrc.cjs` | -| Run production build | `ui` | US-001–004 | done | -| Execute browser scenarios in test plan | `ui-validation` | US-001–004 | pending | diff --git a/spec/features/labeling-view-controls/test-plan.md b/spec/features/labeling-view-controls/test-plan.md deleted file mode 100644 index 5ac3e932..00000000 --- a/spec/features/labeling-view-controls/test-plan.md +++ /dev/null @@ -1,58 +0,0 @@ -# Test Plan: Labeling View Controls - -## Contents - -- [Strategy](#strategy) -- [Shortcut scenarios](#shortcut-scenarios) -- [Misclassified scenarios](#misclassified-scenarios) -- [Regression scenarios](#regression-scenarios) -- [Sign-off](#sign-off) - -## Strategy - -The `ui` agent validates static quality with ESLint and a production Vite -build. The `ui-validation` agent validates user behavior in a browser with -Playwright, using mocked app/API data where practical. - -## Shortcut Scenarios - -| ID | View | Scenario | Expected | -|---|---|---|---| -| UI-001 | Results | Press `A`, `S`, `D` | Divider moves left, center, right | -| UI-002 | Interactive | Press `1/2/3`, `T`, `P`, `Space` | Documented action runs | -| UI-003 | Interactive | Use `Ctrl+drag`, then `A/S/D` | Box label and swipe actions run | -| UI-004 | Labeling Tool | Press `A`, `D` | Pre/post selection works | -| UI-005 | Validation | Press `1/2/3`, arrows, `A/D` | Label, navigation, and imagery actions work | -| UI-006 | All | Repeat letter keys with Shift/Caps Lock | Same action runs | -| UI-007 | All | Use shortcut keys in editable or interactive controls | No global shortcut overrides the control | -| UI-008 | All | Inspect visible shortcut help | Text matches behavior | - -## Misclassified Scenarios - -| ID | Scenario | Expected | -|---|---|---| -| UI-009 | Fewer than 3 labels in 2 classes | Toggle disabled with training-threshold copy | -| UI-010 | Enable at threshold | Model trains on demand; mismatch legend appears | -| UI-011 | Labeled prediction differs | Building receives misclassified emphasis | -| UI-012 | Labeled prediction matches | Building has no misclassified fill | -| UI-013 | Building has prediction but no human label | Building has no misclassified fill | -| UI-014 | Change a label or pan to new tiles | Prediction/mismatch display refreshes | -| UI-015 | Retrain or run full prediction | Current prediction state drives mismatch display | -| UI-016 | Enable Predicted or Uncertainty | Misclassified turns off | -| UI-017 | Clear labels or fall below threshold | Misclassified turns off and disables | - -## Regression Scenarios - -| ID | Scenario | Expected | -|---|---|---| -| REG-001 | Open Interactive Labeler | Swipe starts on with pre-left/post-right labels | -| REG-002 | Disable and re-enable swipe | Labeling interactions and map camera still work | -| REG-003 | Hide/show footprints | Labels and predictions remain cached | -| REG-004 | Save and restore labels | Hydration restores labels and mismatch evaluation | - -## Sign-off - -- [ ] `cd ui && npm run lint` (blocked: ESLint 9 requires an - `eslint.config.*`; this repository currently provides `.eslintrc.cjs`) -- [x] `cd ui && npm run build` -- [ ] `ui-validation` Playwright scenarios pass. diff --git a/spec/features/labeling-view-controls/user-stories.md b/spec/features/labeling-view-controls/user-stories.md deleted file mode 100644 index e2d8e938..00000000 --- a/spec/features/labeling-view-controls/user-stories.md +++ /dev/null @@ -1,94 +0,0 @@ -# User Stories: Labeling View Controls - -## Contents - -- [Stories](#stories) -- [Agent assignment](#agent-assignment) -- [Out of scope](#out-of-scope) -- [Traceability](#traceability) - -## Stories - -### US-001: Discover shortcuts in every labeling-related view - -**As an** analyst, **I want** visible, consistent shortcut help, **so that** I -can work quickly without guessing controls. - -```gherkin -Given I open Results, Interactive Labeler, Labeling Tool, or Building Validation -Then the view shows shortcut help that matches its implemented behavior -And letter shortcuts work in lowercase and uppercase -And shortcuts do not run while I type in an editable control -``` - -### US-002: Compare imagery consistently - -**As an** analyst, **I want** consistent imagery keys, **so that** comparison -behavior transfers between views. - -```gherkin -Given a view supports swipe comparison -When I press A, S, or D -Then the divider moves left, split, or right respectively -``` - -```gherkin -Given a view uses a single map -When I press A or D -Then pre-event/basemap or post-event imagery is shown respectively -And no split behavior is added -``` - -### US-003: Start Interactive Labeler in swipe mode - -**As an** analyst, **I want** pre/post swipe available immediately, **so that** -I can compare imagery while assigning labels. - -```gherkin -Given the Interactive Labeler finishes loading -Then Swipe (pre-event) is on by default -And pre imagery is on the left and post imagery is on the right -``` - -### US-004: Review model disagreements - -**As an** analyst, **I want** to highlight model disagreements, **so that** I -can focus corrections on useful examples. - -```gherkin -Given at least 3 valid labels exist in at least 2 classes -When I enable Show misclassified buildings under Advanced -Then the current model trains on demand or is reused -And only buildings with a human label whose prediction differs are emphasized -And the legend explains the mode -``` - -```gherkin -Given Misclassified mode is active -When Predicted or Uncertainty view is enabled -Then Misclassified mode turns off -When labels fall below the training threshold -Then Misclassified mode turns off and becomes disabled -``` - -## Agent Assignment - -| Story | Implementing agent | Validating agent | UI path | -|---|---|---|---| -| US-001 | `ui` | `ui-validation` | shared shortcut component and four views | -| US-002 | `ui` | `ui-validation` | Visualizer, Interactive Labeler, Labeling Tool, Building Validation | -| US-003 | `ui` | `ui-validation` | `InteractiveLabeler/InteractiveLabeler.jsx` | -| US-004 | `ui` | `ui-validation` | `InteractiveLabeler/InteractiveLabeler.jsx` | - -## Out of Scope - -- API or persistence changes. -- A split view for single-map labeling or validation. -- A new model training implementation. - -## Traceability - -US-001 and US-002 implement -[#47](https://github.com/microsoft/haste/issues/47). US-004 implements -[#100](https://github.com/microsoft/haste/issues/100). US-003 implements -[#104](https://github.com/microsoft/haste/issues/104).