Type dienstverband:
Internship and graduation project
Locatie:
Den Haag
Opleidingsniveau:
Master
Uren per week:
Fulltime – 40

Internship | Retrieval-augmented generation (RAG) for computer vision applications

About this position

Retrieval-augmented generation (RAG) is very successful in language models by having external knowledge flow into the model via on-the-fly retrieval. Can we improve vision models in a similar way? This paper “Improving Image Recognition by Retrieving from Web-Scale Image-Text Data” (https://arxiv.org/abs/2304.05173) showed that it is possible, even up to large scales. They found that retrieval is especially helpful for rare classes (i.e. long tails). That is interesting because a lot of real-world applications involve rare classes, as sometimes the most relevant/interesting concepts are occurring the least (a needle in the haystack). Can we leverage this retrieval from an external memory for smaller-sized datasets?

What will be your role?

In this project, you will:

  • Explore the potential of RAG for computer vision, e.g. image classification.
  • Research the methodologies to incorporate RAG into computer vision models.
  • Investigate how RAG can be effectively leveraged in smaller datasets.
  • Explore for which sizes of datasets and various degrees of class imbalance RAG can be helpful.
  • Explore which other tasks RAG could be relevant for (e.g. one/few-shot learning, continual learning).

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (50 people) specialized in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from physics to artificial intelligence. Intelligent Imaging is a young and growing department where we have built up a lot of expertise over the past years in AI and deep learning.

Keywords to describe this project include: retrieval augmented generation, deep learning, computer vision.

What we expect from you

You are in the final stages of your master’s in artificial intelligence, computer science, physics, mathematics, electrical engineering, or a similar field, you have experience with programming (Python) and prior experience with computer vision and deep learning.

The project duration is flexible within the range of 6 to 12 months. Please mention in your application whether you are looking for an internship or a graduation project, the preferred duration of your project and your desired starting date.

What you'll get in return

You want to work on the precursor of your career; a work placement gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

  • A highly professional, innovative working environment, with top experts as colleagues.
  • A suitable internship allowance (615 euro for wo- and hbo-students, 390 euro for mbo-students, for a full-time internship).
  • Eight hours of leave per internship month (for a full-time internship).
  • A free membership of Jong TNO, where you can meet TNO-colleagues and join several activities, such as sports activities, (work-related) courses or the yearly ski-trip.
  • Use of a laptop.
  • An allowance for travel expenses in case you don’t receive an OV-card.

TNO as an employer

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. Since 1932, we have been making knowledge and technology available for the common good. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world. Read more about TNO as an employer.

At TNO we encourage an inclusive work environment, where you can be yourself. Whatever your story and whatever unique qualities you bring to the table. It is by combining our unique strengths and perspectives that we are able to develop innovations that make a real difference in society. Want to know more? Read what steps we are taking in the area of diversity and inclusion.

The selection process

After the first CV selection, the application process will be conducted by the concerning department. TNO will provide a suitable internship agreement. If you have any questions about this vacancy, you can contact the contact person mentioned below.

For this internship vacancy it is required that the AIVD issues a security clearance (VGB) after conducting a security screening. Take into account that this process may take about 8 weeks. If you have been abroad for more than 6 consecutive months, or if you do not have the Dutch nationality, it may take longer. Read more about security screening on the AIVD website.

Important to be aware of before applying:

  • As stated in the internship policy, TNO doesn’t sign contracts from educational institutions. Before the start of the internship, the internship agreement from TNO needs to be signed, by the intern, TNO and the educational institution. This agreement is established in cooperation with Universiteiten van Nederland according to the UNL-template. If you are not a student at a Dutch college or university, or if you are a mbo-student, the TNO internship agreement will apply. In that case, a ‘proof of educational institution’ and a ‘proof of internship as part of course’ is needed instead of a signature from the educational institution.
  • Furthermore, you have to be registered at an educational institution to be eligible to do a paid internship at TNO. If you are not studying at a university you are only eligible for a work experience internship (unpaid, maximum of 8 weeks).
  • Interns at TNO must be in possession of a Dutch residential address at the start of the internship. It is not possible to work from abroad.

Has this job opening sparked your interest?

Then we’d like to hear from you! Please contact us for more information about the job or the selection process. To apply, please upload your CV and covering letter using the ‘apply now’ button.