Clarification on reproducing COCO FID-30K results and fine-tuned VAE decoder weights

[Theo-polis]

Hi STARFlow team,

Thank you for releasing the code and checkpoints. I am trying to reproduce the text-to-image COCO FID-30K result reported in Table 3 of the paper, and I would appreciate some clarification on the exact evaluation setup.

I used the released text-to-image checkpoint:

• ckpts/starflow_3B_t2i_256x256.pth
• config: configs/starflow_3B_t2i_256x256.yaml
• VAE: stabilityai/sd-vae-ft-ema
• --finetuned_vae none
• COCO val2014 captions/images for FID-30K-style evaluation
• CFG changed from 0.1 to 4.0

With this setup, I get a COCO FID around 20, while the paper reports a substantially better COCO FID-30K number. I understand that exact FID can depend on the caption subset, real-image statistics, preprocessing, and Inception implementation, so I wanted to ask about the intended reproduction setup.

I have a few specific questions:

1. Which checkpoint corresponds to the COCO FID-30K result in Table 3?

   • Is it the released starflow_3B_t2i_256x256.pth, or a different 3.8B / FullData checkpoint?

2. What is the exact COCO FID-30K evaluation protocol?

   • COCO 2014 val or COCO 2017 val?
   • Fixed 30K caption subset, or randomly sampled captions?
   • Are generated images compared against all COCO val images or a 30K subset?
   • Which FID implementation / real statistics file was used?

3. The paper mentions fine-tuning the decoder over noisy latents. The code also has finetune_decoder.py and --finetuned_vae, whose default appears to look for a local fine-tuned VAE checkpoint such as logs/vae/pytorch_model.bin.

   • Was a fine-tuned VAE decoder used for the reported COCO FID-30K result?
   • If so, are there plans to release the fine-tuned VAE decoder weights or provide instructions for reproducing them?

4. If the released checkpoint is mainly intended for qualitative generation rather than exact Table 3 reproduction, could you share the expected COCO FID range for the public checkpoint using the recommended evaluation pipeline?

Thanks again for the release. Any details on the evaluation recipe or missing checkpoints would be very helpful for reproducibility.