Recent advances in Multimodal In-Context Learning (M-ICL) for Multimodal Large Language Models (MLLMs) have attracted considerable attention. These developments primarily focus on configuring an in-context sequence for a given test case based on instance-level semantic similarity. However, high similarity among demonstrations in the sequence introduces inductive biases, which may mislead MLLMs and ultimately degrade their overall performance. To address this, we propose a novel cluster-based in-context configuration method that adaptively groups candidate data and selects demonstrations from each cluster. This method enhances the diversity within the sequence while preserving semantic consistency, enabling MLLMs to focus on the main intent of the demonstrations. The experimental results on four Visual Question Answering (VQA) benchmarks, including OK-VQA, VQAv2, VizWiz, and TextVQA …
本文针对多模态大语言模型在多模态上下文学习中的演示序列配置问题，提出一种基于聚类的上下文配置方法。该方法自适应地对候选数据进行分组，并从每个簇中选取演示样本，以增强序列内多样性并保持语义一致性，从而减少高相似演示带来的归纳偏置，使模型更关注演示的主要意图。在OK-VQA、VQAv2、VizWiz和TextVQA四个视觉问答基准上的实验验证了其有效性。