Skip to content
This repository was archived by the owner on Aug 1, 2024. It is now read-only.
This repository was archived by the owner on Aug 1, 2024. It is now read-only.

Why normalize twice since the target encoder also normalize the feature at last? #67

@jianghaojun

Description

@jianghaojun

ijepa/src/train.py

Lines 295 to 298 in 52c1ae9

def forward_target():
with torch.no_grad():
h = target_encoder(imgs)
h = F.layer_norm(h, (h.size(-1),)) # normalize over feature-dim

if self.norm is not None:
x = self.norm(x)
return x

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions