feat: Enable servekit for Kimi-K3 (quantization issue). #9

Open

Labels

opened

Currently, Kimi K3 takes 100 minutes to load on the CSCS cluster. Servekit would help there, but we don't implement PP presharding.

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests