Research

In recent years, deep learning has been used as a synonym for artificial intelligence, and different modalities (numerical, image, and linguistic information) can now be handled in a unified way under representation learning. As a result, the boundaries between natural language processing and image processing have almost disappeared, and we have now achieved information processing that handles multimodal information in the same way as we do in our minds. In the Kobayashi lab, we mainly research the three topics of natural language processing, brain emotion analysis, and multimodal information processing.

Natural Language Processing (NLP)

The natural language processing research conducted in the Kobayashi Lab is based on the research trends in the field, and we are engaged in natural language processing research based on linguistics.
In particular, Kobayashi himself has specialized in and conducted research on systemic functional linguistics since he was a Ph.D. student at the University of Sydney’s Department of Linguistics.
Since then, he has been involved in natural language processing based on statistical machine learning, and is currently working on natural language processing research based on deep learning, which is now mainstream.

Examples of Research Themes

  • Data-to-Text
  • Dialogue Processing Using Knowledge
  • Semantic Interpretation of Adverbs
  • Temporal Common Sense in Language
  • Caption Generation
  • Japanese Language Learning Support
  • Author estimation
  • Syntactic analysis using Transformer
  • Sentence style conversion
  • Natural language inference
  • Explanation of language rules AI

Brain Emotion Analysis

Brain decoding using deep learning models to verbalize information in the brain has become a representative method of the Kobayashi Lab and has been published in overseas journals. We are actively pursuing research at the intersection of neuroscience and artificial intelligence, and in recent years, the fusion of neuroscience and artificial intelligence has attracted attention. We believe that a considerable number of research topics in the new academic field of creating new disciplines will be related to information processing in the human brain, and that research will continue to develop in the future, looking beyond current artificial intelligence technology.

Examples of Research Themes

  • Elucidation of the mechanisms of time generation in the human brain
  • To develop BrainBERT, a new general-purpose language model that maps language to brain activity.
  • To elucidate the information processing mechanisms in the human brain that determine whether a tanka poem is “poetic” or “unpoetic” as it is written.
  • To elucidate human perception of the world from the perspective of prediction
  • To construct a customized information delivery method based on the elucidation of the information processing mechanism in the brain, taking into account gender and individual differences

Multimodal Information Processing

Intelligence requires not only the brain, but also physicality. Since about 2014, we have been conducting joint research with top researchers in Japan who are incorporating machine learning into robotics, and we have established a working group called “Language and Robotics” to promote the development of technologies that will enable robots, which will be increasingly important in the future, to process language and perform intelligent tasks, and we are working to revitalize this new field.

Examples of Research Themes

  • Development of a method for expressing actions in language by capturing the correspondence between the meaning of adverbs that express actions and the meaning of adverbs that express actions
  • Development of a model that incorporates conventional syntactic structure analysis methods into a deep learning model
  • A method for generating natural language sentences by using questions in image question-answering as control signals for generating natural language sentences
  • A method for generating natural language sentences that match the speaker’s intentions based on the pen’s drawn traces
  • Development of technology that judges interactions between multiple people from images and explains them in language