Authors

* External authors

Venue

Date

Share

Analysis of Multi-Source Language Training in Cross-Lingual Transfer

Seong Hoon Lim*

Taejun Yun*

Jinhyeon Kim*

Jihun Choi

Taeuk Kim

* External authors

ACL-2024

2024

Abstract

The successful adaptation of multilingual language models (LMs) to a specific language-task pair critically depends on the availability of data tailored for that condition. While cross-lingual transfer (XLT) methods have contributed to addressing this data scarcity problem, there still exists ongoing debate about the mechanisms behind their effectiveness. In this work, we focus on one of promising assumptions about inner workings of XLT, that it encourages multilingual LMs to place greater emphasis on language-agnostic or task-specific features. We test this hypothesis by examining how the patterns of XLT change with a varying number of source languages involved in the process. Our experimental findings show that the use of multiple source languages in XLT-a technique we term Multi-Source Language Training (MSLT)-leads to increased mingling of embedding spaces for different languages, supporting the claim that XLT benefits from making use of language-independent information. On the other hand, we discover that using an arbitrary combination of source languages does not always guarantee better performance. We suggest simple heuristics for identifying effective language combinations for MSLT and empirically prove its effectiveness.

Related Publications

Literature-based Hypothesis Generation: Predicting the evolution of scientific literature to support scientists

AI4X, 2025
Tarek R Besold, Uchenna Akujuobi, Samy Badreddine, Jihun Choi, Hatem ElShazly, Frederick Gifford, Chrysa Iliopoulou, Kana Maruyama, Kae Nagano, Pablo Sanchez Martin, Thiviyan Thanapalasingam, Alessandra Toniato, Christoph Wehner

Science is advancing at an increasingly quick pace, as evidenced, for instance, by the exponential growth in the number of published research articles per year [1]. On the one hand, this poses anincreasingly pressing challenge: Effectively navigating this ever-growing body o…

Link prediction for hypothesis generation: an active curriculum learning infused temporal graph-based approach

AIR, 2024
Uchenna Akujuobi, Priyadarshini Kumari, Jihun Choi, Samy Badreddine, Kana Maruyama, Sucheendra K Palaniappan*, Tarek R Besold

Over the last few years Literature-based Discovery (LBD) has regained popularity as a means to enhance the scientific research process. The resurgent interest has spurred the development of supervised and semi-supervised machine learning models aimed at making previously imp…

CookingSense: A Culinary Knowledgebase with Multidisciplinary Assertions

LREC-Coling, 2024
Donghee Choi*, Mogan Gim*, Donghyeon Park*, Mujeen Sung, Hyunjae Kim, Jaewoo Kang*, Jihun Choi

This paper introduces CookingSense, a descriptive collection of knowledge assertions in the culinary domain extracted from various sources, including web data, scientific papers, and recipes, from which knowledge covering a broad range of aspects is acquired. CookingSense is…

  • HOME
  • Publications
  • Analysis of Multi-Source Language Training in Cross-Lingual Transfer

JOIN US

Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.