Skip to content

Stabilize the single-shift logit transform - #23

Open
pedrocastanha wants to merge 2 commits into
extropic-ai:mainfrom
pedrocastanha:fix/stable-single-shift
Open

Stabilize the single-shift logit transform#23
pedrocastanha wants to merge 2 commits into
extropic-ai:mainfrom
pedrocastanha:fix/stable-single-shift

Conversation

@pedrocastanha

Copy link
Copy Markdown

Summary

  • rewrite the single-shift logit transform with a numerically stable, algebraically equivalent expression
  • preserve the input dtype when constructing the log(2) constant
  • add regression coverage for saturated float32 logits and the end-to-end gradient path

Why

The previous expression can overflow for large negative logits and suffer cancellation for positive logits, producing infinities at ordinary saturated float32 values. The new implementation uses logaddexp and remains finite across the tested range from -100 to 100.

Validation

  • pytest tests/test_gradients.py -q -k 'single_shifted_theta or single_gradient_is_finite'
  • existing param_shift_single simulator gradient tests
  • isort, Black, Flake8, and Pyright through pre-commit

Closes #16

Copilot AI lite review requested due to automatic review settings August 6, 2026 16:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stabilize the single-shift logit transform for large positive and negative theta

2 participants