Kirato Yoshihara

Gradient Descent towards a big impact.

About

Hi there! I'm Kirato — a builder with research at heart, currently a final-year undergrad at The University of Osaka. Now at MBZUAI and AIST, previously at Preferred Networks.

Research Interests

Foundation Models, Efficient Learning, Representation Learning

Publications

Different Layers, Different Manifolds: Module-Wise Weight-Space Geometry in Transformer Optimization

Kirato Yoshihara
Weight-Space Symmetries (WSS) @ ICML 2026

VesselBridge3D: A Foundation Model Adaptation Framework for Label-Efficient 3D Vessel Segmentation

Kirato Yoshihara, Yohei Sugawara, Yuta Tokuoka, Lihang Hong
SWITCH+ Workshop @ MICCAI 2026 · Spotlight

Modular Norm RandOpt: Population-Efficient Ensembling through Architecture-Aware Perturbations

Kirato Yoshihara, Hiroaki Hamade
Under review at ICLR 2027

Essays

A ligand in a blue protein binding pocket
9/8 2026

Verifiable Docking: Repairing Only What Needs to Move

A docking pose can be almost right and still clash with the protein. Can the clash itself tell us what to repair?
Surface normal visualization for Rethinking 0x5F3759DF for GPUs
5/13 2026

Rethinking 0x5F3759DF for GPUs

The old spell was 0x5F3759DF. We recast it for GPUs, FP16/BF16, and RMSNorm, and discovered that modern magic may need a lookup table.
Assignments of different layers to different manifolds
4/17 2026

Different Layers, Different Manifolds: Scaling Manifold Muon to GPT-2

Manifold Muon constrains weights to have nice spectral properties. But which manifold should each layer live on? We tried five assignments on GPT-2 and only three survived.
# Chimera estimator loss = k3.detach() + k1 - k1.detach() # forward: k3 # backward: k1
4/9 2026

The Chimera Estimator: Fixing KL Divergence Gradients in High Dimensions

Schulman's k3 estimator has nice forward properties but exploding gradient variance. What if we use k3 for the forward pass and k1 for the backward?

Fun Facts about Me

Mac or Windows?
Mac
Coffee or tea?
Coffee
City or nature?
Nature — a.k.a. Nagano
Favorite editor?
Claude
Best model weights?
model_final_v4_actually_best_20260409_0342_newer_important.pth

Contact