Dongxi NLP explains Instruction Tuning and SFT training objectives

In "Post-Training Series Part 2", Dongxi NLP explains Instruction Tuning: SFT follows the pretraining training objective, i.e., predicting the next token at each position, using cross-entropy loss to measure the gap between the prediction and the correct token, and increasing the probability of the correct token. On the data side, the model's chat template is used to convert each pair of data into a token sequence in dialogue format, and then a loss mask is used to mark the tokens for which loss needs to be computed; this is an important part of the recipe.