Researcher (Assistant Professor) in Computer Vision at the Istituto Italiano di Tecnologia (IIT). Stuart's research focus is on Visual Reasoning to understand the layout of visual content from Iconography (e.g. Sketches) to 3D Scene understanding and their implications on methods of interaction. He is involved in the coordination and implementation of the MEMEX EU H2020 project for increasing social inclusion with Cultural Heritage. Stuart has previously held PostDoc positions at IIT, University College London (UCL) and the University of Surrey. Also at the University of Surrey Stuart was awarded his PhD. Stuart continues to hold an honorary position at UCL and UCL Digital Humanities and collaborates with researchers across universities and institutes, reviewing and organising workshops.

Stuart James

Research Scientist
Developer
Blogger
Runner
Climber
Drummer

 

 

 

Research interests

My research activities fit broadly into Spatial Reasoning — how we can reason about the layout of objects in space in both 2D and 3D to provide insight or retrieve relevant information.


Exploring using Depth and Knowledge to answer questions specifically related to the layout of a 3D scene from a 2D perspective.

Visual Question & Answering

Reconstructing the semantic relational structure of the scene using geometry and knowledge. Providing advanced interaction for questioning and reasoning.

Scene Graph

Use free-hand sketching with a controller in an Immersive Environment (VR). We have explored the task of retrieval to explore this unique modality.

Sketch Retrieval in Virtual Reality

We have explored using sketches to search collections of videos using Visual Storyboarding to express the sequence of events in the target clip.

Sketch based Retrieval

We are using sequences to retrieve information providing a broader context than a one-off search. We have demonstrated through Free-Hand storyboarding and storey synthesis.

Visual Narratives and Stories

Detection, Representation and Reasoning on Sketch, Line, Hatching, Motifs or higher level with signs or icons.

Abstract & Iconography Reasoning

Detection, Representation and Reasoning on Sketch, Line, Hatching, Motifs or higher level with signs or icons.

Knowledge Retrieval & Reasoning

Cultural Heritage & Digital Humanities

Assistive Technologies

Robotics

Research Group & Collaborators

Research Topic: Causality

Supervisor with Dr Jacopo Cavazza (IIT) and Dr Alessio Del Bue (IIT)

Davide Talon

PhD Student

Research Topic: Supporting the technical development of the MEMEX Project

Supervisor with Dr Alessio Del Bue (IIT)

Sarim Mehdi

Technician

Research Topic: Localising ourselves and objects for Social Good

Collaborator with Dr Alessio Del Bue (IIT)

Dr Matteo Taiana

PostDoc Collaborator

Research Topic: Visual scene understanding through geometry

Supervisor with Dr Alessio Del Bue (IIT)

Mohamed Dahy Abdelaher Elkhouly

PhD Student

Research Topic: Immersive spatial interaction within Virtual Reality

Collaborator with Prof. Anthony Steed (UCL)

Daniele Giunchi

External Collaborator

All associates in research lines PAVIS and VGM at Istituto Italiano di Tecnologia

Openings

Looking to do a PhD?

Our group is always looking for good PhD candidates, so if you are interested in doing a PhD in Visual Reasoning please contact me to discuss the options. We have yearly calls in the Italian education system and other options depending on the type of PhD. For more details review research areas and publications especially before making an inquiry or application.

Recent calls:

2020 University of Genova - Cycle XXXVI - Theme F [Now closed]

Current Open Positions

PostDoc Researcher on the MEMEX Project

This open position, up to two years, is financed in the context of the H2020 European Project MEMEX (MEMories and EXperiences for inclusive digital storytelling – Grant Agreement 870743) – an interdisciplinary effort to provide Augmented Reality (AR) systems to impact the European society at large.

The candidate will have the opportunity to work on realistic problems involving leading European research centers and companies with the aim to create a novel AR platform able to understand the semantic scene structure of the environment using an RGB video stream from a smartphone.

Apply Online

Latest Blog Post

03 Dec 2020 . research . Three great videos about the pilot locations of MEMEX Comments

Snapshot of MEMEX Project news

We recently published three videos on the pilot locations, Paris, Barcelona and Spain, of the MEMEX Project. This challenging collaboration effort came together during the second wave of COVID so a massive effort was needed by the Michael Culture Association, NOHO Limited, Fundacio Interarts per a la Cooperacio Cultural Internacional (InterArts), Mapas das Ideias and Dédale.

Read More

Previous posts

newsfeed

  • August 2020

    Organising VISART V: Vision for Art

  • December 2019

    Coordinating and Implementing MEMEX EU Project

  • December 2019

    Rejoined Visual Geometry and Modelling Lab of IIT

  • January 2019

    Researcher (Assist. Prof) in Cultural Heritage in IIT

  • April 2017

    Moved to Istituto Italiano di Tecnologia (IIT)

  • April 2017

    Became Honorary Research Associate at UCL

  • October 2016

    Joined UCL Centre for Digital Humanities

  • April 2016

    Invited to Rank Prize Symposium on Computer Vision and Video Effects

  • April 2016

    Graduated as Doctor of Philosophy

  • October 2015

    Research Associate at UCL with Prof. Tim Weyrich

  • September 2015

    Defended PhD Thesis

Latest Publication

2021 Consistent Mesh Colors for Multi-View Reconstructed 3D Scenes

Specular highlights are commonplace in images, however, methods for detecting them and in turn removing the phenomenon are particularly challenging. A reason for this, is due to the difficulty of creating a dataset for training or evaluation, as in the real-world we lack the necessary control over the environment. Therefore, we propose a novel physically-based rendered LIGHT Specularity (LIGHTS) Dataset for the evaluation of the specular highlight detection task. Our dataset consists of 18 high quality architectural scenes, where each scene is rendered with multiple views. In total we have 2,603 views with an average of 145 views per scene. Additionally we propose a simple aggregation based method for specular highlight detection that outperforms prior work by 3.6% in two orders of magnitude less time on our dataset.

Accepted at arXiv in preprint.

See full publication index

Contact

To find out more about our research you can find me at...

Visual Geometry and Modelling Lab

Center for Human Technologies - Istituto Italiano di Tecnologia
Via Enrico Melen, 83, 16152 Genova GE