Q 函数预训练对 RL 微调帮助有限
Pretraining a Q-function often doesn’t actually help RL finetuning, compared to initializing Q from scratch.
AI 导读
与从零初始化 Q 函数相比，预训练 Q 函数通常并不能真正帮助 RL 微调。 我们发现，在来自多样化策略的数据上预训练 Q 函数，对于从预训练中获得改进至关重要。
查看原帖Pretraining has worked remarkably well across domains We show this doesn’t hold for Q-functions in online RL from a pretrained policy — and propose IPE, a more effective way to learn Q-functions for online RL fine-tuning (1/6) https://t.co/G5SYHmgz83
正文
当前显示原文，中文全文译文尚未就绪。
Pretraining a Q-function often doesn’t actually help RL finetuning, compared to initializing Q from scratch.
We find that pretraining Q-functions on data from diverse policies is critical to see improvements from pretraining.