Google DeepMind paper: Training models to write moral arguments makes them more decisive but does not improve their judgment

Understand this story

AI overview

Google DeepMind published a paper investigating the effects of training Gemma-4 4B with reinforcement learning to write moral arguments. The results showed that on ambiguous cases in MoralChoice, the proportion of answers that changed due to swapping the order of options dropped from 14.3% to 6.6%, but the proportion of sycophantic agreement with users only went from 45.3% to 44.0%, and accuracy on moral fable questions actually fell from 56.5% to 53.4%. Based on this, the paper recommends that when evaluating advice- or judgment-type tasks, consistency and correctness should be tested separately.

AI-generated from public reports

Latest development · ReportOct 30 00:23

Google DeepMind: Training models to write moral arguments makes them more decisive but does not improve judgment

Developments

1 confirmed developments

Oct 30· 1 reports
Google DeepMind paper: Training models to write moral arguments makes them more decisive, but their judgments do not improve
Source:Rohan Paul

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Oct 30

00:23
Rohan Paul
Google DeepMind: Training models to write moral arguments makes them more decisive but does not improve judgment
A new Google DeepMind paper found that after training Gemma-4 4B with reinforcement learning to write moral arguments, the proportion of cases where the model changed its answer on ambiguous MoralChoice cases due to a swap in option order dropped from 14.3% to 6.6%, but the proportion of sycophantic agreement with users only changed from 45.3% to 44.0%, and accuracy on moral fable questions actually fell from 56.5% to 53.4%. The paper recommends that when evaluating advice or judgment tasks, consistency and correctness be tested separately.