Simo-Serra Lab is accepting students enrolled in the Department of Computer Science and Engineering. At the current time, only undergraduates students are accepted. Supervision can be carried out either in Japanese or in English.

It is highly recommended for students to have familiarity/interest in python programming and Linux before joining the lab, as all development is done on a Linux cluster using predominantly the python programming language. In the case of not being familiar with Python and Linux, a willingness to learn is essential.

Upon joining the laboratory, students are expected to propose a research topic related to one of the below research areas. It is recommended to choose a topic in which you are interested in. Some previous student topics are:

• Vectorization of rough sketches project site
• Captioning game play videos project site
• Restoration of deteriorated line drawings project site
• Fashion style classification dataset
• Semantic segmentation of aerial images paper
• Enhancement of photos paper
• Thesis from previous years

Research Areas

• AI-Aided Content Design (Computer Graphics)
  • Interactive Neural Networks
  • Human-Computer Interface (HCI)
  • Optimization
• Representation Learning (Machine Learning)
  • Structured Prediction
  • Multi-Modal Learning
  • Weak Supervision
  • Model Compression
• Image Processing (Computer Vision)
  • Fully Convolutional Networks
  • Hierarchical Models
  • Transformers / Attention

Research Topics

Our research focuses on creating smart tools and interfaces for supporting content design using machine learning techniques with an objective of allowing users to not only create content, such as illustrations or movies, faster, but also of higher quality. This is done by employing and developing the cutting edge machine learning techniques, such as convolutional neural networks (deep learning) and conditional random field models. We focus on practical problems with high impact in the daily workflow of creators.

In particular, most of the research is focused on, but not limited to, image translation tasks, in which the input is an image, such as a rough sketch, and the output is also an image, such as a line drawing. These tasks are characterized by needing to process large amounts of visual data as a single image is composed by millions of pixels, for which convolution operators have shown high performance. As humans are primarily visual creatures, many different content design tasks can be formulated as image translation tasks such as colorization of black and white images, inking of rough sketches, or completing regions of an image.

Much of the recent research is based on deep learning and in particular convolutional neural networks given their high performance in a wide variety of tasks. However, despite seeming like a one-fits-all solution, there are many issues that have to be resolved when designing models such as computational cost, interactivity, representation capacity, optimization paradigm… It is to be noted that most of the research can be formulated as energy minimization optimization problems and thus optimization plays a fundamental role.

Additional Links

• Thesis from Previous Years
• Research Projects
• Publications