Skip to content

Add activation checkpoint offloading #63

@jon-chuang

Description

@jon-chuang

JAX now supports saved activation offloading. These policies should be expressible in praxis and thus downstream libraries like paxml.

Maxtext already has such integration.
https://github.com/google/maxtext/blob/ebd39aa64d670fa13a313b6f776e01ad9e450321/MaxText/layers/models.py#L231

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions