{ Stuart James }

Research interests

My research activities fit broadly into Spatial Reasoning — how we can reason about the layout of objects in space in both 2D and 3D to provide insight or retrieve relevant information. My research has a keen interest on varied data types including those from the Humanities such as Art and Cultural Heritage.

Exploring using **Depth** and **Knowledge** to answer questions specifically related to the layout of a 3D scene from a 2D perspective.

Visual Question & Answering

Detection, Representation and Reasoning on simplified representations or symbols such as Sketch, Line, Hatching, Motifs or icons.

Abstract & Iconography Reasoning

Identifying and retrieving relevant knowledge held within Knowledge Graphs to support Computer Vision tasks such as Visual Question and Answering or reasoning on location.

Knowledge Retrieval & Reasoning

Reconstructing the semantic relational structure of the scene using geometry and knowledge. Providing advanced interaction for questioning and reasoning.

Scene Graph

Principally on layout of content in 2D or 3D and how to make decisions that influence about a path or option linked with Visual Question and Answering

Planning & Reasoning

We have explored using sketches to search collections of videos using Visual Storyboarding to express the sequence of events in the target clip.

Sketch based Retrieval

We are using sequences to retrieve information providing a broader context than a one-off search. We have demonstrated through Free-Hand storyboarding and storey synthesis.

Visual Narratives and Stories

Within VR we explored the use free-hand sketching in an Immersive Environment (VR) with multiple modalities for the task of retrieval.

Interaction in Virtual Reality

Providing storytelling experiences overlaying information of surrounding Cultural Heritage and the stories of the particpants in the MEMEX Project.

Interaction in Augmented Reality

Cultural Heritage & Digital Humanities

Assistive Technologies

Robotics

Openings

Looking to do a PhD?

Our group is always looking for good PhD candidates, so if you are interested in doing a PhD in Visual Reasoning please contact me to discuss the options. We have yearly calls in the Italian education system and other options depending on the type of PhD. For more details review research areas and publications especially before making an inquiry or application.

Theme

Machine ability to detect objects within images has surpassed human ability, however, when posed with relatively simple more complex tasks machines quickly struggle. This theme focuses on developing AI systems that are able to access knowledge stored in Knowledge Graphs to understand the world around the camera view. Few works have successfully integrated knowledge in Computer Vision systems and knowledge graphs provide one avenue. This research will study methods to integrate knowledge for user interaction via retrieval or visual question and answering within real world environments. In particular, shallow and deep graph-based methodologies are promising computational framework to include external knowledge and also maintaining a high degree of interpretability, a necessary feature for modern AI systems.

Call Links

Latest Blog Post

We recently published three videos on the pilot locations, Paris, Barcelona and Spain, of the MEMEX Project. This challenging collaboration effort came together during the second wave of COVID so a massive effort was needed by the Michael Culture Association, NOHO Limited, Fundacio Interarts per a la Cooperacio Cultural Internacional (InterArts), Mapas das Ideias and Dédale.

newsfeed