Skip to content

applib: touch drag-to-scroll and tap-to-select for ScrollLayer / MenuLayer#1773

Open
tsutomu1984-hub wants to merge 2 commits into
coredevices:mainfrom
tsutomu1984-hub:touch-scroll-menu
Open

applib: touch drag-to-scroll and tap-to-select for ScrollLayer / MenuLayer#1773
tsutomu1984-hub wants to merge 2 commits into
coredevices:mainfrom
tsutomu1984-hub:touch-scroll-menu

Conversation

@tsutomu1984-hub

Copy link
Copy Markdown

Summary

On touch-enabled platforms, make ScrollLayer and MenuLayer respond to
touch: a vertical drag scrolls the content, and a tap selects a menu
row (equivalent to a physical SELECT press). Existing apps that use these
widgets get this for free — no app changes, no new settings, and no public SDK
surface is added.

Motivation

Pebble Time 2 (and other CONFIG_TOUCH boards) have a touchscreen, but the
core scrolling widgets were button-only. This adds the two most common touch
interactions — scroll and select — at the widget level, so the bulk of the
system UI and third-party apps built on ScrollLayer/MenuLayer become
touch-usable without modification.

Approach

  • Guarded by CONFIG_TOUCH (the firmware Kconfig symbol, as used by the
    neighbouring layer.c / event_loop.c), so the code is compiled out on
    non-touch boards.
  • No new public API. The scroll layer subscribes to the touch service from
    inside its existing click-config provider
    (scroll_layer_click_config_provider), which already runs on every window
    appear. This means the focused scroll layer re-subscribes as the touch
    handler exactly when it takes over the buttons, and it needs no change to the
    app-facing API.

ScrollLayer (drag + tap dispatch)

  • Touchdown records the start Y and the start content offset and stops any
    running scroll animation, so the content follows the finger from where it
    was.
  • Position updates are ignored until the finger has moved more than a 10px
    threshold; past that the gesture is a confirmed drag and ΔY is applied to the
    content offset via the existing internal setter, which hard-clamps to the
    content bounds (honouring clips_content_offset), matching button scrolling.
  • Liftoff: if the gesture never became a drag and lifted off within 300ms
    it is dispatched as a tap to an optional internal handler; a confirmed drag
    just finalises the position.
  • The subscription is torn down in scroll_layer_deinit() so a destroyed
    layer can never be referenced by a pending touch event.

MenuLayer (tap-to-select)

  • Registers an internal tap handler on its scroll layer. On a tap, the tap's
    content-space Y coordinate is mapped to a row by walking the existing cell
    iterator. Section headers and separator gaps are ignored. A hit row is
    selected via menu_layer_set_selected_index() and then fires the same
    select_click callback as a physical SELECT press
    — identical handling, no
    separate code path.

File-scope static gesture state (design premise)

The in-flight gesture state and the active tap handler are held in file-scope
statics
, not in the ScrollLayer/MenuLayer structs. This relies on a
deliberate premise:

There is a single physical touchscreen, so only one touch gesture is ever
in flight at a time
(touch events are delivered to the single focused
subscriber).

Because of this, per-layer storage is unnecessary — and, importantly, it keeps
the widget structs unchanged: no struct growth and therefore no ABI impact
(applib_malloc.json sizes and the size_2x/size_3x static asserts are
untouched, which matters because MenuLayer currently has zero size_2x
headroom).

Watchface restriction: not affected

This targets watchapp contexts only and does not interfere with the touch
service's watchface restriction. The touch service already returns no per-task
state for watchfaces (sys_app_is_watchface()), so a ScrollLayer/MenuLayer
living in a watchface simply subscribes to nothing — the behaviour is inert
there by construction. The existing Settings → Display → Touch toggle
(touch_service_set_globally_enabled) continues to gate touch globally; no new
setting is introduced.

Out of scope for v1 (open problems)

These are intentionally not implemented and left as follow-ups:

  • Rubber-banding / over-scroll resistance at the ends — v1 hard-clamps only.
  • Touch long-press (SELECT-long-press equivalent).
  • Button/touch mutual-exclusion when both are used at once.
  • Live follow-through of the Settings → Display → Touch toggle mid-app
    (subscription is decided at window-appear time).
  • Paging-mode interaction — a drag applies the offset directly and does not
    page-align.

A separate, independent line of work (kept on a different branch) explores a
kernel-level "gesture → button synthesis" bridge to help apps that use their
own click handlers instead of ScrollLayer/MenuLayer; it is not part of
this PR and does not touch this code.

Testing

  • Builds: qemu_emery (touch), qemu_flint (non-touch, code correctly
    compiled out), and obelix@pvt/@slot1 (real Pebble Time 2 target) all
    build cleanly; the feature's symbols are present on touch targets and absent
    on non-touch.
  • Real hardware: sideloaded onto a Pebble Time 2 and confirmed that touch
    vertical scrolling works
    in the launcher / Settings menus.
  • Logic: the drag/tap state machine and the row hit-test interval logic are
    covered by a standalone check.
  • Note: the bundled qemu-pebble build does not raise the touch IRQ, so the
    end-to-end touch path could not be exercised purely in QEMU; verification was
    done on real hardware plus the logic checks above.

Compatibility

  • No new SDK function is exported (exported_symbols.json and the SDK revision
    are unchanged); the only added function is @internal and used solely by
    MenuLayer.
  • No new settings; existing ScrollLayer/MenuLayer apps benefit unmodified.
  • No struct/ABI changes.

tsutomu1984-hub and others added 2 commits July 22, 2026 02:30
On touch platforms (CONFIG_TOUCH), subscribe to the touch service from
the scroll layer's click config provider, so a focused scroll layer
receives touch events by riding along on the existing click setup
without any new public API.

A vertical drag past a 10px threshold scrolls the content, hard-clamped
to the content bounds (honoring clips_content_offset); any running
scroll animation is stopped on touchdown so the content follows the
finger. A near-stationary touch that lifts off within 300ms is
dispatched as a tap to an optional internal handler, letting
higher-level layers (e.g. MenuLayer) react to taps.

Gesture state and the active tap handler are file-static: a single
physical touchscreen means only one gesture is ever in flight, so no
per-layer storage (and no ABI-affecting struct growth) is needed. The
subscription is torn down in scroll_layer_deinit to avoid a dangling
reference to a destroyed layer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: tsutomu sasaki <tsutomu1984@gmail.com>
Register a tap handler on the menu's scroll layer (CONFIG_TOUCH). A tap
is mapped from its content-space Y coordinate to a row by walking the
existing cell iterator; section headers and separator gaps are ignored.
A tapped row is selected via menu_layer_set_selected_index and then
fires the same select_click callback as a physical SELECT press.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: tsutomu sasaki <tsutomu1984@gmail.com>
@ericmigi

Copy link
Copy Markdown
Collaborator

really cool! any videos of this in action?

@jplexer jplexer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing apps that use these widgets get this for free

we should never change existing runtime behavior for third party apps if we can avoid it so this is definitely a no-go. having this enable-able (via code) and turning it on for system apps would be better imho

@vvzvlad

vvzvlad commented Jul 22, 2026

Copy link
Copy Markdown

Oh, this is awesome! I'd been designing touch scrolling myself but hadn't gotten around to implementing it yet.
Mind if I jump in — even though I'm not on the Pebble team?
While reading through the code I noticed a few things — more food for thought than complaints, but some of them seem fairly important:

1)Wake-up. The screen always shows the menu, but the backlight times out — so a tap that's just meant to wake the watch also fires select_click on whatever row is under your finger. Maybe a touch that woke the screen itself shouldn't count as input? Otherwise it's easy to hit the wrong thing.

2)selection_will_change. The tap handler calls menu_layer_set_selected_index() + select_click directly, but selection_will_change only fires from the wrap path (menu_layer.c:138) and set_selected_next (:1308) — never from set_selected_index. Because of that, a finger can select a row the buttons can't (like Timezone in settings/time.c:403-409). Shouldn't the tap honor that veto too?

3)Small thing with the tap coordinate. You take it from the Liftoff, but on recovery the driver sometimes emits FingerUp with (0,0) (cst816.c:304,314) — and then the tap flies off to the top row. Safer to take it from the last PositionUpdate?

4)touch_service_unsubscribe() in scroll_layer_deinit. It's called unconditionally, but there's only one subscriber per task — so deiniting one layer tears down the subscription of another, still-focused one. You clear s_touch_gesture under a condition, but not the unsubscribe; probably worth making it symmetric.

5)About notifications. Looks like they don't get covered here — they're on SwapLayer, not ScrollLayer. Same with the action menu (ActionMenuLayer) — it has its own click config provider (action_menu_layer.c:742), so it bypasses scroll_layer_click_config_provider and won't get touch either.

Overall the approach is great, but my hunch is that covering all of this properly (the wake gate, the row veto, notifications, the action menu) would be easier on the recognizer framework (applib/ui/recognizer/) — that's exactly what it's for. Though it needs fixing first: there's a use-after-free in recognizer_reset — flags = 0 wipes the is_owned bit. Maybe it's worth building on that directly?

@tsutomu1984-hub

Copy link
Copy Markdown
Author

really cool! any videos of this in action?
Thanks! Here's a quick video — this actually shows

vid__touch.drag-to-scroll.and.tap-to-select.mp4

combined
with a few follow-up PRs I just opened (#1784, #1785, #1786) that
build on it: touch drag-to-scroll on the notification body, tap/swipe
synthesis for apps with their own click handlers (e.g. Music's
action bar), and touch support for the action menu. Individually
they're smaller/scoped changes, but together they round out touch
support across most of the system UI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants