Skip to content

Notebook 27 — Train SAE on a fine-tuned model (LoRA) #5

@caiovicentino

Description

@caiovicentino

User has a LoRA-fine-tuned checkpoint, wants to train an SAE on its residuals. Document the PEFT hook path + warn about LoRA merging. Target Gemma-2-2B + a small LoRA.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions