Dongxi NLP explains Instruction Tuning and SFT training objectives
In "Post-Training Series Part 2", Dongxi NLP explains Instruction Tuning: SFT follows the pretraining training objective, i.e., predicting the next token at each position, using cross-entropy loss to measure the gap between the prediction and the correct token, and increasing the probability of the correct token. On the data side, the model's chat template is used to convert each pair of data into a token sequence in dialogue format, and then a loss mask is used to mark the tokens for which loss needs to be computed; this is an important part of the recipe.
View original posthttps://t.co/Y3AP0ziejN
View original
https://t.co/Y3AP0ziejN
Body · AI translation
Instruction Tuning or SFT follows the pretraining training objective:
Predict the next token at each position, measure the gap between the prediction and the correct token with cross-entropy loss, and improve the probability of the correct token through training.
Among them, use the model's chat template to convert each pair of data into a token sequence in dialogue format, then use a loss mask (marking the tokens for which loss needs to be computed, which is an important part of the recipe.
Let's look at the second article in the post-training series, Instruction Tuning