Suhas Kotha

Hello! I am a PhD student at Stanford, fortunate to be advised by Percy Liang and Tatsunori Hashimoto. I am currently trying to understand how to do deep learning when compute is infinite relative to data. I previously studied CS at CMU, fortunate to be advised by Aditi Raghunathan.

Papers

Content

Fresh work/ideas

• 💜 Perturbation Invariant Gradients: Permutation Case Study
• 💜 Self-Improvement via Synthetic Data Augmentation
• Optimal Experimental Design with Compute Constraints
• Highest Likelihood Decoding
• 💜 Lower Diversity Accelerates Convergence for Regression
• Synthesizing Optimal Parallel Collectives via SAT
• The Type Theory of Transformers

Challenges

• Language Data Ordering and Filtering
• Emergence as Prediction Across Scale

Paper Analyses

• Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo
• The Optimization Trajectory of Two-Layer Linear Neural Networks
• Convex Optimization Theory Agrees with Learning Rate Scheduling
• Transformers are RNNs
• Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
• A Universal Law of Robustness via Isoperimetry

Introductions

• 💜 Off-policy RL
• Some Intuition Behind Double Descent
• Linear Policies are Optimal (for simple dynamics and costs)
• Second Law of Thermodynamics
• Elegance of Linear Regression
• Understanding Reinforcement Learning from Human Feedback
• Convex Geometry for Computers