Internship | Cross-lingual transfer for Dutch LLM pretraining

Data scarcity is a major challenge for developing European sovereign foundation models, especially for low-resource languages where additional training data is inherently limited. A promising solution is cross-lingual transfer: training models on related languages can improve performance on a target language more effectively than adding more data from the target language itself.

The recent ATLAS study (Longpre et al., 2025) is the largest empirical investigation of this effect to date, training 774 multilingual models across 50 languages and introducing the Bilingual Transfer Score (BTS) to measure how much one language benefits another. The study found that shared language family and script are the strongest predictors of transfer effectiveness.

However, Dutch was not included in the ATLAS language set. This creates both a scientific and practical gap. ATLAS's predictors cannot distinguish whether English or German should be a better source language for Dutch, as both are Germanic languages written in the Latin script. Yet, in practice, one may provide substantially more benefit than the other.

Answering this question is directly relevant for Dutch and European sovereign LLM initiatives, which must decide how to balance Dutch, English and other related languages within pretraining datasets to maximize performance.

Objective

The goal of this project is to extend ATLAS's methodology to Dutch and investigate:

Whether Dutch benefits more from English or German during pretraining, and whether this can be explained by ATLAS's language-family and script-based predictors.
Which Dutch-English-German data mixture yields the best combined Dutch and English performance under a fixed compute budget.

Approach
Phase 1: Extending Bilingual Transfer Scores to Dutch
Study and reproduce ATLAS's BTS methodology.
Train a Dutch-only baseline model and several bilingual variants, including Dutch+English and Dutch+German, with one or two additional control languages.
Compute BTS scores relative to Dutch.
Compare transfer effectiveness across source languages and assess whether ATLAS's existing language-family and script categories sufficiently explain the results.

Phase 2: Optimizing multilingual data mixtures
Run a small experimental sweep over Dutch-English-German data mixtures.
Evaluate each model on Dutch and English test sets using ATLAS-style metrics.
Compare the resulting performance against simple baselines such as Dutch-only and equal-split mixtures.
Analyze whether optimal mixture ratios can be predicted from pairwise transfer scores or require direct experimentation.

Research questions
1. Does Dutch benefit more from English or German pretraining data, and can this be explained by ATLAS's existing predictors?
2. What Dutch-English-German data mixture maximizes combined Dutch and English performance under a fixed compute budget?

• Deliverables
  • A reproducible Dutch-focused replication of the ATLAS BTS methodology.
  • An empirical comparison of English-to-Dutch versus German-to-Dutch transfer.
  • A recommended Dutch-English-German pretraining mixture for sovereign LLM development.
  • A thesis discussing the methodology, findings and limitations of scaling conclusions from small proxy models.

This assignment is aimed at students of the MSc Artificial Intelligence or adjacent fields. There are supervisor meetings once a week, possibly combined with other students to share knowledge across projects.

An internship at TNO means working in an environment where substance and impact are central. You will become part of a knowledge organisation where research and practice come together, and where experts collaborate on solutions to current societal and technological challenges.

Your internship is a period in which you can discover what suits you, where your strengths lie and what you would like to learn next. You are part of a professional working environment, gain insight into how things work in practice, and have the opportunity to build experience that goes beyond this internship alone. For many students, an internship is therefore also a first step in discovering whether TNO could be a potential next step after graduation.

In addition, we offer you:

• A professional and innovative internship environment in which you actively contribute to societal and technological challenges, working alongside leading experts in your field.
• Personal and dedicated supervision, with focus on your learning objectives, development and study assignment.
• Room to develop: you gain relevant work experience, develop both your subject‑specific and professional skills, and build a valuable network.
• Use of a laptop and the facilities you need to perform your work effectively.
• A monthly internship allowance of € 615 for a full-time internship, for MSc, BSc and vocational education (MBO) students.
• Up to eight hours of leave per internship month for a full-time internship, allowing you to balance your internship with your studies and personal life.
• A contribution towards travel expenses if you are not entitled to a student travel card.
• A free membership to Jong TNO: the network for young colleagues, where you can meet other TNO colleagues and participate in sports activities, professional and personal development activities, and social events such as the annual ski trip.

Our people are at the heart of TNO. Their curiosity, expertise and entrepreneurial mindset make it possible to deliver high-impact research and innovations that contribute to society’s sustainable wellbeing and prosperity. That is why we invest in an inspiring and inclusive working environment where colleagues can excel, have autonomy and continue to grow.

Your talent and ambition have every opportunity to flourish at TNO. You work with experts (both within and beyond TNO), have access to advanced technology and the freedom to explore, experiment and innovate. Our strength lies in independence, reliability and collaboration. We find each other in wonder and ingenuity. We are driven to push boundaries. By working with businesses and government, and by connecting different perspectives, we strengthen our innovative capability and create responsible, meaningful results.

At TNO, we believe this is our time to help society, government and business move forward faster. Together with driven colleagues, you turn knowledge into concrete innovations or ventures that truly make a difference by combining the power of science and entrepreneurship. And in doing so, you make your mark on our time.

You can apply for this internship position until the 2nd of November, 2026. After submitting your application, you will receive a confirmation by email.

TNO will arrange an appropriate internship agreement. If you have any questions about the content of the internship or the application procedure, you can contact the person listed below.

If you start an internship with us, we will also ask you to provide a Certificate of Conduct (VOG).

Important to know before applying:

• Before the start of the internship, an internship agreement must be signed. For students enrolled at a Dutch university or university of applied sciences, TNO uses the UNL-template, supplemented with several TNO‑specific agreements. For students from foreign educational institutions and MBO (secondary vocational education) programmes, the TNO internship agreement applies. TNO does not sign any other internship agreements.
• Before the start of the internship, the educational institution will need to confirm in writing that:
  1. you are enrolled at the educational institution for the duration of the internship, and;
  2. the internship forms part of your study programme.

The confirmation of educational institution takes place by signing the UNL template or forms prepared by TNO.

• Interns at TNO must have a registered residential address in the Netherlands at the start of the internship. Carrying out internship activities from abroad is not possible.
• Are you an international student? Please note that you must have a BSN (Dutch citizen service number) before the start of your internship. Applying for a BSN can take several weeks, so make sure to start this process well in advance.