Internship | Foundation models for robotics

Foundation models are AI models that are trained on a large amount of data and can be adapted to a wide range of applications in different domains (https://en.wikipedia.org/wiki/Foundation/_model). The most wide-known foundation model is currently ChatGPT, developed by OpenAI. Other foundation models available to the public are DALL-E and BERT. Some models are language-only (BERT, ChatGPT), some vision only (DALL-E), but also a combination of language and vision can be found. Currently, new variants are developed combining different modalities and tasks including planning. Foundation models have the potential to improve many AI capabilities of autonomous systems, and can revolutionize robotics. In other words, foundation models have the potential for disruptive advantages in this field. For an overview of current developments, see e.g. https://robotics-fm-survey.github.io/.

The deployment of foundation models for robotics is promising, but also challenging. Your role would be 1) to implement a state-of-the-art foundation model in our robotics pipeline, and 2) to investigate the advantages and challenges with respect to the earlier used more traditional deep learning approaches in this pipeline. Typical questions are: in what situations will this foundation model improve the performance, what are its limitations, and how can this be optimized. Depending on your interest, also the risks of using foundation models and its compliance to legal and ethical values can be investigated.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (50 people) specialized in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from physics to artificial intelligence. Intelligent Imaging is a young and growing department which has built up a lot of expertise over the past years in AI and deep learning.

Keywords for this project include: foundation models, robotics, vision, planning.

You are in the final stages of your Master's in artificial intelligence, computer science, physics, mathematics, electrical engineering, robotics, or a similar field and have some track record in the field of computer vision or robotics.

The project duration is flexible within the range of 6 to 12 months. Please mention in your application whether you are looking for an internship or a graduation project, the preferred duration of your project and your desired starting date.

You want to work on the precursor of your career; a work placement gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

• A highly professional, innovative working environment, with top experts as colleagues.
• A suitable internship allowance (615 euro for wo- and hbo-students, 390 euro for mbo-students, for a full-time internship).
• Eight hours of leave per internship month (for a full-time internship).
• A free membership of Jong TNO, where you can meet TNO-colleagues and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
• Use of a laptop.
• An allowance for travel expenses in case you don’t receive an OV-card.

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. Since 1932, we have been making knowledge and technology available for the common good. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world. Read more about TNO as an employer.

At TNO we encourage an inclusive work environment, where you can be yourself. Whatever your story and whatever unique qualities you bring to the table. It is by combining our unique strengths and perspectives that we are able to develop innovations that make a real difference in society. Want to know more? Read what steps we are taking in the area of diversity and inclusion.

After the first CV selection, the application process will be conducted by the concerning department. TNO will provide a suitable internship agreement. If you have any questions about this vacancy, you can contact the contact person mentioned below.

Important to be aware of before applying:

• Before the start of the internship, the internship agreement from TNO needs to be signed. For students at a college or university based in the Netherlands, TNO uses the UNL-template (supplemented with a number of specific agreements from TNO). For students of foreign and MBO educational institutions, the TNO internship agreement applies. TNO does not sign any other internship agreements.
• Before the start of the internship, the educational institution will need to confirm in writing that:

1. you are enrolled at the educational institution during the internship, and;
2. the internship takes place as part of the programme of the study.

• The confirmation of educational institution takes place by signing the UNL template or forms prepared by TNO.
• Interns at TNO must be in possession of a Dutch residential address at the start of the internship. Performance of internship activities from abroad is not possible.