←新闻详情

面壁智能 OpenBMB更新于 10月20日 22:30

面壁智能提出 Trident:解决多模态检索文本偏好问题

🔱 Why can irrelevant text outrank an image that actually contains the answer?

AI 导读

面壁智能(OpenBMB)提出 Trident,用 Multi-Positive View InfoNCE 联合训练 Text/Image/Fused 三视图,解决多模态检索中无关文本得分高于含答案图像的“Chaos in the Text”现象。在 6 个视觉文档检索基准上,Trident-Qwen3VL-2B 将 Mix Avg NDCG@10 从 40.12 提升至 73.90。分析显示,多模态检索器在纯文本或纯图像语料上表现良好,但图文混合时性能明显下降,原因是文本常获得比语义对齐图像更高的相似度分数。

正文

当前显示原文,中文全文译文尚未就绪。

🔱 Why can irrelevant text outrank an image that actually contains the answer?

We find that multimodal retrievers can perform well on text-only and image-only corpora, yet degrade sharply when text and images are mixed.

We call this phenomenon "Chaos in the Text."

Our analysis reveals a systematic Modality Preference: text often receives higher similarity scores than semantically aligned images. We introduce Trident, which jointly trains Text / Image / Fused views with Multi-Positive View InfoNCE.

On 6 visual-document retrieval benchmarks, Trident-Qwen3VL-2B improves Mix Avg NDCG@10 from 40.12 → 73.90.

来源

面壁智能 OpenBMB