Publications

Efficient Construction of Model Family through Progressive Training Using Model Expansion

COLM2025

Foundation model / LLM

By : Kazuki Yano, Sho Takase, Sosuke Kobayashi, Shun Kiyono, Jun Suzuki

Spike No More: Stabilizing the Pre-training of Large Language Models

COLM2025

Foundation model / LLM

By : Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars

COLM2025

Foundation model / LLM

By : Rei Higuchi, Ryotaro Kawata, Naoki Nishikawa, Kazusato Oko, Shoichiro Yamaguchi, Sosuke Kobayashi, Seiya Tokui, Kohei Hayashi, Daisuke Okanohara, Taiji Suzuki

User-Guided Correction of Reconstruction Errors in Structure-from-Motion

IUI2025

Human Computer Interaction (HCI)/ Human Robot Interaction (HRI)

By : Sotaro Kanazawa, Jinyao Zhou, Yuta Kikuchi, Sosuke Kobayashi, Chunyu Li, Fabrice Matulic, Takeo Igarashi, Keita Higuchi

B2T Connection: Serving Stability and Performance in Deep Transformers

Findings of ACL 2023

Natural Language Processing / Speech Processing

By : Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

Decomposing NeRF for Editing via Feature Field Distillation

NeurIPS 2022

Computer Vision

Machine Learning / Deep Learning

By : Sosuke Kobayashi, Eiichi Matsumoto, Vincent Sitzmann

VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users’ Daily Lives

CHI 2022

Human Computer Interaction (HCI)/ Human Robot Interaction (HRI)

Natural Language Processing / Speech Processing

By : Riku Arakawa, Hiromu Yakura, Sosuke Kobayashi

Instance-Based Neural Dependency Parsing

TACL 2021

Natural Language Processing / Speech Processing

By : Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui

SHAPE: Shifted Absolute Position Embedding for Transformers

EMNLP 2021

Natural Language Processing / Speech Processing

By : Shun Kiyono, Sosuke Kobayashi, Jun Suzuki, Kentaro Inui

All Word Embeddings from One Embedding

NeurIPS 2020

Machine Learning / Deep Learning

Natural Language Processing / Speech Processing

By : Sho Takase, Sosuke Kobayashi

Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN

IJCV 2020

Computer Vision

Machine Learning / Deep Learning

By : Masaki Saito, Shunta Saito, Masanori Koyama, Sosuke Kobayashi

Instance-based Learning of Span Representations: A Case Study through Named Entity Recognition

ACL 2020

Natural Language Processing / Speech Processing

By : Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, Kentaro Inui