AI Trends

Trending story

English translation is being prepared.

1 reports1 story updatesUpdated Nov 19 02:53

Understand this story

Story summary

The English summary is being updated. See the reporting timeline below.

Latest development · Report

Why Fine-Tuning Jobs Take Longer Than FLOPs Estimates

Developments

1 confirmed developments

  1. English translation is being prepared.

    Source:Hugging Face

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 19

  1. Hugging FaceFirst-party

    Why Fine-Tuning Jobs Take Longer Than FLOPs Estimates

    Fine-tuning jobs often take longer than FLOPs estimates because environment setup, model loading, tokenization, evaluation, checkpoint uploads, and shared-infrastructure I/O are all excluded from training-loop computation. A CatBoost predictor based on 272 SFT runs and 24 features shows that evaluation sample size is the most important factor; even the best 30% MFU baseline has a mean error of more than 12 hours on jobs averaging 23.43 h.

Related stories

  1. Related storiesThinking Machines Lab introduces the Inkling 2 Small end-to-end 4-bit training recipe
  2. Related storiesZyphra proposes sparse attention method ZHA
  3. Related storiesNvidia releases Nemotron 3.5 Super VL and Bolt models
  4. Related storiesEnglish translation is being prepared.