Name:
Upgrading the Newsroom
Description:
Automated image selection system for news articles
Professor — Lab:
Karl AbererDistributed Information Systems Laboratory
Contact:
RĂ©mi Lebret

Home page:
Upgrading the Newsroom
Layman description:
It is an automated system that can recommend suitable images to illustrate news articles by analyzing the text content of the articles. The system can handle inputs in multiple languages like German and French by using aligned multilingual word embeddings. It uses advanced neural network techniques like attention to identify the most relevant parts of the text for retrieving matching images from a large database.
Technical description:
The system fuses multiple textual sources (caption, body, headline, lead) from news articles using a hierarchical attention mechanism to retrieve relevant images. It utilizes subword embeddings and self-attention to better encode entities and capture important keywords within texts. The model is trained on a large-scale multimodal multilingual dataset of over 500k German and French news article-image pairs in a weakly-supervised manner.
Papers:
Project status:
unknown — entered showcase: 2024-03-19 — entry updated: 2024-03-19

Code quality:
This project has not yet been evaluated by the C4DT Factory team. We will be happy to evaluate it upon request.
Project type:
Application