Master Thesis – Enhanced Reasoning in Vision-Language Models Using Structured Visual Representations

Machine Learning, Computer Vision, Computer Science

2 Master students, 30 hp each

Vision-Language Models (VLMs) have demonstrated strong capabilities in combining visual information with natural-language reasoning. However, most VLMs perform their reasoning primarily in the language space after encoding the visual input, which can make it difficult to explicitly preserve and access fine-grained visual information such as object locations, identities, shapes, and movements. Structured visual representations could provide additional information that the model can utilize throughout its reasoning process. Modern surveillance camera systems can provide not only video footage but also structured metadata, such as bounding boxes, object identities, tracking information, trajectories, and velocity vectors.

This thesis will investigate how structured visual information can be integrated into VLMs to support more detailed and interpretable visual reasoning. This may include leveraging existing metadata provided by surveillance systems to complement visual input and improve the model's understanding of complex scenes.

Different approaches to incorporating such information will be explored, including how it can be represented, aligned with visual input, and utilized during training and inference. The aim is to determine whether incorporating structured visual information can improve the model's ability to understand video footage and answer questions requiring object-level, spatial, and temporal reasoning.

• Investigate state-of-the-art methods for integrating structured visual information into Vision-Language Models.
• Explore the use of existing metadata from surveillance systems, as well as methods for generating training data from in-house and public datasets.
• Fine-tune one or more open-source VLMs using methods such as instruction fine-tuning and Parameter-Efficient Fine-Tuning (PEFT), e.g. LoRA.
• Investigate how intermediate visual representations can be incorporated into the model's reasoning process.
• Evaluate whether structured visual information improves the model's ability to answer questions requiring spatial, object-level, and temporal reasoning.
• Compare models trained with and without structured visual information.
• Investigate the interpretability and limitations of the resulting approach.

Ruthardt, Jona, et al. "Steerable visual representations." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026. https://arxiv.org/abs/2604.02327v1

Qin, Yiming, et al. "Chain-of-visual-thought: Teaching vlms to see and think better with continuous visual tokens." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026. https://arxiv.org/pdf/2511.19418

For this Thesis proposal we target 2 students with a strong interest in machine learning, Vision Language models, problem solving and programming. You are studying a Master Program in Electronics Engineering, Mathematics, Computer Science, Biomedical Engineering or Engineering Physics.

You are valuable to us – how nice that you are interested in one of our proposals! There are a few things for you to keep in mind when applying.

• Applications are accepted in both Swedish and English, and you apply via the proposal advert.
• The announced thesis is open only to students affiliated with a Swedish University/College either directly or via an exchange program.
• When the thesis proposal states that it includes two students working together, we would like you to apply in pairs. In these cases, send one application each but make sure to clearly state in your application who your co-applicant is. If you have any questions regarding this, please do not hesitate to contact us.
• Please attach your CV, Cover letter and University/College grade summary.

Magnus Svensson, Engineering Manager, Magnus.Svensson@axis.com, +46 46 2879187

Certain roles at Axis require background checks, which means applicable verifications will be done in these recruitments. Notice will be provided before we take any action.

We enable a smarter and safer world by creating innovative solutions for improving security and business performance. As a network technology company and industry leader, we offer solutions in video surveillance, access control, intercom, and audio systems, enhanced by intelligent analytics applications.

With around 6,000 committed employees in over 50 countries, we collaborate with partners worldwide. Together, we thrive in our friendly, open, and collaborative culture and inspire each other to think beyond the expected. United by our commitment to inclusion, diversity, and sustainability, we consistently seek to develop our skills and way of working.

For more information about Axis, please visit our website www.axis.com.

Listen to Axis Communications Playlist