Suhas Kotha
Hello! I am a PhD student at Stanford, currently researching how to do deep learning when compute is infinite relative to data. I am fortunate to be advised by Percy Liang and Tatsunori Hashimoto. I previously studied CS at CMU, fortunate to be advised by Aditi Raghunathan.
Papers
Content
Fresh work/ideas
- Perturbation Invariant Gradients: Permutation Case Study
- 💜 Self-Improvement via Synthetic Data Augmentation
- Optimal Experimental Design with Compute Constraints
- Highest Likelihood Decoding
- 💜 Lower Diversity Accelerates Convergence for Regression
- Synthesizing Optimal Parallel Collectives via SAT
- The Type Theory of Transformers
Paper Analyses
- From Entropy to Epiplexity (Interestingness Part 2)
- Prediction with a Short Memory
- Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo
- The Optimization Trajectory of Two-Layer Linear Neural Networks
- Convex Optimization Theory Agrees with Learning Rate Scheduling
- Transformers are RNNs
- Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
- A Universal Law of Robustness via Isoperimetry
Introductions
- Algorithmic Information Theory (Interestingness Part 1)
- Asymptotic Efficiency of Kernel Density Estimation
- 💜 Pre-training and RL as minimizing KL
- 💜 Off-policy RL
- Some Intuition Behind Double Descent
- Linear Policies are Optimal (for simple dynamics and costs)
- Second Law of Thermodynamics
- Elegance of Linear Regression
- Understanding Reinforcement Learning from Human Feedback
- Convex Geometry for Computers
Challenges
