See which layers of your fine-tune are over- or under-trained
Paste a public Hugging Face model. AlphaCheck fits a power law to the eigenvalue spectrum of every weight matrix and flags each layer underfit, healthy or overfit. No eval data, no GPU, nothing to install.
How to read α
Ranges are WeightWatcher's published thresholds. The health score is the share of matrices in the healthy range.
Method
For each weight matrix W in the transformer blocks (attention and MLP projections), we compute the eigenvalues of WᵀW and fit p(λ) ∝ λ−α to the tail, choosing the tail start by the smallest Kolmogorov-Smirnov distance.
This is Heavy-Tailed Self-Regularization, from Martin, Peng & Mahoney, Nature Communications (2021), with the power-law fit of Clauset, Shalizi & Newman (2009). The open-source WeightWatcher does the same analysis on your own machine; AlphaCheck runs it for you on our server, straight from the Hub.
Weights stream one matrix at a time from Hugging Face. A 0.5B model takes about half a minute; a 7B model takes around half an hour on our current server, and the report fills in as it goes.
What it can analyze
- Public, ungated text models on Hugging Face, up to 13B parameters
- Weights in safetensors format (bf16, fp16 or fp32)
- Full models, including merged fine-tunes. LoRA-only adapters, vision and multimodal models, and non-transformer architectures are not supported.
Subscribers can add a base model to see the α change of every matrix, which shows where fine-tuning moved the weights. $9/month, cancel anytime. The first 3 reports are free.