chore: reorganize repo structure

- Move root-level analysis docs → docs/
- Move start_xiaoyi.sh + test_xiaoyi2.py → link/
- Move docs/WEB_UI_REQUIREMENTS.md → webui/
- Remove papers/training-events/ (research output, not core platform code)
- Remove fizzbuzz.py, pyproject.toml (stale)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fizzbuzz.py

@@ -1,9 +0,0 @@
-for i in range(1, 21):
-    if i % 3 == 0 and i % 5 == 0:
-        print('FizzBuzz')
-    elif i % 3 == 0:
-        print('Fizz')
-    elif i % 5 == 0:
-        print('Buzz')
-    else:
-        print(i)

papers/training-events/README.md

@@ -1,61 +0,0 @@
-# Training as Structured Event Sequence
-
-**Status**: Draft v0.3, May 2026 — experiments added
-
-## Files
-
-- `main.tex` — Full paper (579 lines, 10pt twocolumn, v0.3, with experimental results)
-- `measure_events.py` — E1-E5 event detector; `python measure_events.py` for smoke test
-- `test_measure_events.py` — pytest suite; `pytest test_measure_events.py`
-- `demo_synthetic.py` — End-to-end demo on synthetic MLP; `python demo_synthetic.py`
-- `requirements.txt` — Dependencies: torch, numpy, pyhessian
-
-See also: `training_events.py` + `test_training_events.py` in project root
-(full 5-event detector with PSim, plasticity gap, 26 tests).
-
-## The Five Events (v0.2)
-
-```
-E1 LF  Label Fitting          train_loss < epsilon_LF
-E2 SE  Spectral Edge          Hessian gap gamma_k > delta_gap   [Davis-Kahan]
-E3 CSC Curvature Subspace     ||Pi_k delta_theta||^2/||delta_theta||^2 > rho_cap
-E4 GA  Geometric Alignment    PSim_k(H, G) > alpha_align         [Gronwall]
-E5 FE  Functional Events      test-accuracy jump OR neural collapse (NC1)
-```
-
-Causal chain: E1 => E2 (Weyl) => E3 (Gronwall) => E4 (covariance tracking) => E5
-
-## Unified Phenomena
-
-| Phenomenon | E1 | E2 | E3 | E4 | E5 / outcome |
-|---|---|---|---|---|---|
-| Edge of Stability | short | short | ≈E2 | coincides | oscillatory loss |
-| Grokking | early | delayed | long gap | long gap | delayed test jump |
-| Neural Collapse | early | moderate | moderate | moderate | ETF geometry |
-| Double Descent | at interp. | moderate | moderate | capacity-dep. | test error trough |
-| **Loss of Plasticity** | ✓ | ✓ | ✓ | **fails** | no generalization |
-
-## Experimental Agenda (5 experiments)
-
-1. **Exp-1** Temporal ordering across ResNet/ViT/MLP on CIFAR-10/100
-2. **Exp-2** Grokking: spectral gap + PSim tracking predicts test jump
-3. **Exp-3** Alignment regularizer accelerates Neural Collapse
-4. **Exp-4** Failed E4 predicts catastrophic forgetting in Split-CIFAR-100
-5. **Exp-5** Sharpening rate predicts t_SE
-
-## Quick Start
-
-```bash
-pip install -r requirements.txt
-python measure_events.py      # smoke test
-pytest test_measure_events.py # unit tests
-python demo_synthetic.py      # end-to-end synthetic demo
-
-# Full test suite (project root)
-pytest test_training_events.py  # 26/26
-```
-
-## Experiments (v0.3 new)
-- `run_experiments.py` — Runs Exp-1/Exp-2 end-to-end; results in `results/`
-- `tables.tex` — LaTeX tables for Exp-1/2/4 (auto-generated)
-- `plot_figures.py` — Plotting script for figures

papers/training-events/SUBMISSION_CHECKLIST.md

@@ -1,59 +0,0 @@
-# Submission Checklist — Training as Event Sequence
-
-**Paper version**: v0.3 (2026-05-20)  
-**Target venue**: NeurIPS 2027 (deadline ~May 2027) / ICLR 2027 (deadline ~Sep 2026)
-
-## Current Status
-
-### ✅ Done
-- [x] Theory: E1–E5 definitions and causal ordering proofs (Prop 1–3)
-- [x] Unification: 5 phenomena (EoS, Grokking, NC, DD, LoP) unified  
-- [x] Empirical Protocol section with Event Monitor algorithm
-- [x] Exp-1: Temporal ordering 9/9 (rate=1.00) on synthetic MLP
-- [x] Exp-2: Grokking spectral gap growing (mod-23, pilot 90 epochs)
-- [x] Exp-3: Gradient alignment E4 fires before E5 (pilot)
-- [x] Exp-4: Forgetting 5-80% + synthetic Δψ=-0.922 proof-of-concept
-- [x] LaTeX tables (Exp-1/2/4)
-- [x] PDF compiled: 7 pages, 371KB
-- [x] All tests: 35/35 passing
-
-### 🔲 Required before submission
-- [ ] **Exp-1 (GPU)**: ResNet-20/ViT-S/MLP on CIFAR-10 — run `exp1_cifar10_resnet.py`
-- [ ] **Exp-2 (full)**: Extend grokking to 500+ epochs for E5 jump observation
-- [ ] **Exp-4 (Hessian)**: True Hessian PSim via top-k eigenvectors (pyhessian)
-- [ ] **Exp-5**: Sharpening rate predictor on CIFAR-10 (harder data)
-- [ ] Update all tables with GPU experiment results
-- [ ] Add figures (training curves, event timeline diagram)
-- [ ] Polish writing / related work
-- [ ] Spellcheck + grammar check
-- [ ] arXiv preprint upload
-
-## Venue Requirements
-
-### NeurIPS 2027
-- 8 pages main + unlimited appendix
-- LaTeX template: neurips_2027
-- Anonymous submission required
-- Code/data sharing encouraged
-
-### ICLR 2027  
-- No page limit  
-- Openreview.net submission
-- Anonymous submission required
-- Code reproducibility emphasized
-
-## Files
-
-```
-papers/training-events/
-  main.tex                    — Full paper (v0.3, 586 lines)
-  tables.tex                  — LaTeX tables for Exp-1/2/4
-  main.pdf                    — Compiled (7 pages, 371KB)
-  measure_events.py           — E1-E5 event detector
-  demo_synthetic.py           — End-to-end demo (E1→E5 SATISFIED)
-  run_experiments.py          — Exp-1/Exp-2 runner
-  exp1_cifar10_resnet.py      — Full Exp-1 (GPU, CIFAR-10)
-  plot_figures.py             — Figure generation
-  test_measure_events.py      — 9/9 tests
-  results/                    — Experiment outputs (JSON)
-```

papers/training-events/demo_synthetic.py

@@ -1,52 +0,0 @@
-"""demo_synthetic.py — 5-event sequence demo, calibrated thresholds."""
-import torch, torch.nn as nn, torch.nn.functional as F
-from measure_events import (
-    effective_rank, spectral_ratio, projection_power,
-    positive_hessian, EventDetector, EventRecord)
-torch.manual_seed(42)
-D, H, C, N = 4, 8, 2, 200
-X = torch.randn(N, D)
-y = (X[:, 0] + X[:, 1] > 0).long()
-model = torch.nn.Sequential(nn.Linear(D,H), nn.ReLU(), nn.Linear(H,C))
-opt = torch.optim.SGD(model.parameters(), lr=0.1, weight_decay=1e-3)
-loss_fn = nn.CrossEntropyLoss()
-P = sum(p.numel() for p in model.parameters())  # 58
-def _up(pf):
-    i=0; W1=pf[i:i+H*D].reshape(H,D); i+=H*D; b1=pf[i:i+H]; i+=H
-    W2=pf[i:i+C*H].reshape(C,H); i+=C*H; b2=pf[i:i+C]
-    return W1,b1,W2,b2
-def _fwd(pf,x):
-    W1,b1,W2,b2=_up(pf); return torch.relu(x@W1.T+b1)@W2.T+b2
-def pf(): return torch.cat([p.detach().flatten() for p in model.parameters()])
-def hess():
-    p=pf().requires_grad_(True); g=torch.autograd.grad(loss_fn(_fwd(p,X),y),p,create_graph=True)[0]
-    H=torch.zeros(P,P)
-    for i in range(P): H[i]=torch.autograd.grad(g[i],p,retain_graph=True)[0].detach()
-    return H
-def G():
-    Gm=torch.zeros(P,P); p=pf().requires_grad_(True)
-    for xi,yi in zip(X,y):
-        g=torch.autograd.grad(loss_fn(_fwd(p,xi.unsqueeze(0)),yi.unsqueeze(0)),p)[0]
-        Gm+=torch.outer(g.detach(),g.detach())
-    return Gm/N
-def nc2():
-    W1,b1,_,_=_up(pf())
-    with torch.no_grad():
-        f=torch.relu(X@W1.T+b1)
-        mu0=F.normalize(f[y==0].mean(0),dim=0); mu1=F.normalize(f[y==1].mean(0),dim=0)
-        return abs((mu0@mu1).item()+1.0)
-# Thresholds calibrated: E1@50, E2-E4@150, E5@250
-det=EventDetector(delta1=0.06,rho_star=6.5,pi_star=0.998,psi_star=0.998,nc2_eps=1.124)
-print(f"P={P}"); print(f"{'Step':>5}|{'Acc':>6}|{'erank':>6}|{'rho':>7}|{'Pi':>6}|{'NC2':>6}")
-for step in range(1,401):
-    opt.zero_grad(); loss_fn(model(X),y).backward(); opt.step()
-    if step%50==0:
-        acc=(model(X).argmax(1)==y).float().mean().item()
-        Gm=G(); Hm=hess(); Hp=positive_hessian(Hm)
-        er=effective_rank(Gm); sr=spectral_ratio(Gm,k=2)
-        pi=projection_power(Hp,Gm,k=4); nc=nc2()
-        print(f"{step:5d}|{acc:6.3f}|{er:6.3f}|{sr:7.3f}|{pi:6.3f}|{nc:6.3f}")
-        det.update(EventRecord(step=step,train_acc=acc,erank=er,spectral_ratio=sr,
-                               pi_H_G=pi,triple_align=pi,nc2=nc))
-det.summary()
-

papers/training-events/exp1_cifar10_resnet.py

@@ -1,158 +0,0 @@
-#!/usr/bin/env python3
-"""
-Exp-1 (full): Temporal ordering on CIFAR-10 with ResNet-20.
-Requires: GPU, torchvision, pyhessian
-
-Usage:
-    python3 exp1_cifar10_resnet.py --device cuda
-    python3 exp1_cifar10_resnet.py --device cpu  # very slow
-
-Expected output (per config, lr in {0.01, 0.1, 0.5}):
-    t_E1 < t_E2 < t_E3 < t_E4 < t_E5
-"""
-import argparse, json, os, time
-import torch, torch.nn as nn
-import numpy as np
-from pathlib import Path
-
-# ── ResNet-20 for CIFAR-10 ────────────────────────────────────────────────────
-
-class BasicBlock(nn.Module):
-    def __init__(self, in_c, out_c, stride=1):
-        super().__init__()
-        self.conv1 = nn.Conv2d(in_c, out_c, 3, stride=stride, padding=1, bias=False)
-        self.bn1 = nn.BatchNorm2d(out_c)
-        self.conv2 = nn.Conv2d(out_c, out_c, 3, padding=1, bias=False)
-        self.bn2 = nn.BatchNorm2d(out_c)
-        self.shortcut = nn.Sequential()
-        if stride != 1 or in_c != out_c:
-            self.shortcut = nn.Sequential(
-                nn.Conv2d(in_c, out_c, 1, stride=stride, bias=False),
-                nn.BatchNorm2d(out_c))
-    def forward(self, x):
-        out = torch.relu(self.bn1(self.conv1(x)))
-        out = self.bn2(self.conv2(out))
-        return torch.relu(out + self.shortcut(x))
-
-class ResNet20(nn.Module):
-    def __init__(self, num_classes=10):
-        super().__init__()
-        self.conv0 = nn.Conv2d(3, 16, 3, padding=1, bias=False)
-        self.bn0 = nn.BatchNorm2d(16)
-        self.layer1 = self._make(16, 16, 3, stride=1)
-        self.layer2 = self._make(16, 32, 3, stride=2)
-        self.layer3 = self._make(32, 64, 3, stride=2)
-        self.fc = nn.Linear(64, num_classes)
-    def _make(self, in_c, out_c, n, stride):
-        layers = [BasicBlock(in_c, out_c, stride)]
-        for _ in range(n-1): layers.append(BasicBlock(out_c, out_c))
-        return nn.Sequential(*layers)
-    def forward(self, x):
-        x = torch.relu(self.bn0(self.conv0(x)))
-        x = self.layer1(x); x = self.layer2(x); x = self.layer3(x)
-        x = x.mean([2,3])  # global avg pool
-        return self.fc(x)
-
-# ── Spectral quantities ────────────────────────────────────────────────────────
-
-def multi_batch_grads(model, loader, device, n_batches=16):
-    """(n_batches, P) gradient matrix via mini-batch sampling."""
-    grads = []
-    it = iter(loader)
-    for _ in range(n_batches):
-        try: xb, yb = next(it)
-        except StopIteration: it = iter(loader); xb, yb = next(it)
-        xb, yb = xb.to(device), yb.to(device)
-        model.zero_grad()
-        nn.CrossEntropyLoss()(model(xb), yb).backward()
-        grads.append(torch.cat([p.grad.detach().flatten() for p in model.parameters() if p.grad is not None]))
-    return torch.stack(grads)
-
-def spectral_features(G, K=10):
-    U, S, Vh = torch.linalg.svd(G, full_matrices=False)
-    gap = S[0].item()/S[K].item() if len(S)>K else 0.0
-    return gap, Vh[:K].T  # V_top: (P, K)
-
-def psim(V1, V2, K=10):
-    return (torch.norm(V1.T @ V2,'fro')**2/K).item()
-
-# ── Training loop ─────────────────────────────────────────────────────────────
-
-def run(lr=0.1, n_epochs=200, device='cuda', data_root='./data', seed=0):
-    import torchvision, torchvision.transforms as T
-    torch.manual_seed(seed)
-    tr_tf = T.Compose([T.RandomCrop(32,4), T.RandomHorizontalFlip(), T.ToTensor(),
-                        T.Normalize((0.491,0.482,0.447),(0.247,0.244,0.262))])
-    te_tf = T.Compose([T.ToTensor(), T.Normalize((0.491,0.482,0.447),(0.247,0.244,0.262))])
-    tr_ds = torchvision.datasets.CIFAR10(data_root, train=True,  transform=tr_tf, download=True)
-    te_ds = torchvision.datasets.CIFAR10(data_root, train=False, transform=te_tf, download=True)
-    tr_ld = torch.utils.data.DataLoader(tr_ds, batch_size=128, shuffle=True,  num_workers=2)
-    te_ld = torch.utils.data.DataLoader(te_ds, batch_size=256, shuffle=False, num_workers=2)
-    model = ResNet20().to(device)
-    opt = torch.optim.SGD(model.parameters(), lr=lr, momentum=0.9, weight_decay=1e-4)
-    sch = torch.optim.lr_scheduler.MultiStepLR(opt, [100,150], gamma=0.1)
-    crit = nn.CrossEntropyLoss()
-    K=10; EPS_LF=0.15; GAP_TH=3.0; PROJ_TH=0.55; PSIM_TH=0.45; JUMP_TH=0.03
-    t_E=[None]*5; prev_te=0.0
-    for ep in range(1, n_epochs+1):
-        model.train()
-        eloss=0; nb=0
-        for xb,yb in tr_ld:
-            xb,yb=xb.to(device),yb.to(device)
-            opt.zero_grad(); l=crit(model(xb),yb); l.backward(); opt.step()
-            eloss+=l.item(); nb+=1
-        sch.step()
-        tloss=eloss/nb
-        with torch.no_grad():
-            tr_acc=sum((model(xb.to(device)).argmax(1)==yb.to(device)).float().mean().item() for xb,yb in tr_ld)/len(tr_ld)
-            te_acc=sum((model(xb.to(device)).argmax(1)==yb.to(device)).float().mean().item() for xb,yb in te_ld)/len(te_ld)
-        if ep%10==0:
-            G=multi_batch_grads(model,tr_ld,device)
-            gap,V_H=spectral_features(G,K)
-            G2=multi_batch_grads(model,tr_ld,device)
-            _,V_G=spectral_features(G2,K)
-            # Gradient projection
-            model.zero_grad(); xb,yb=next(iter(tr_ld)); crit(model(xb.to(device)),yb.to(device)).backward()
-            gf=torch.cat([p.grad.detach().flatten() for p in model.parameters() if p.grad is not None])
-            proj_r=(V_H@(V_H.T@gf)).norm()/(gf.norm()+1e-8)
-            ps=psim(V_H,V_G,K)
-            if not t_E[0] and tloss<EPS_LF: t_E[0]=ep; print(f'  E1@{ep}')
-            if t_E[0] and not t_E[1] and gap>GAP_TH: t_E[1]=ep; print(f'  E2@{ep} gap={gap:.2f}')
-            if t_E[1] and not t_E[2] and proj_r.item()>PROJ_TH: t_E[2]=ep; print(f'  E3@{ep} proj={proj_r.item():.3f}')
-            if t_E[2] and not t_E[3] and ps>PSIM_TH: t_E[3]=ep; print(f'  E4@{ep} psim={ps:.3f}')
-            if t_E[3] and not t_E[4] and (te_acc-prev_te)>JUMP_TH: t_E[4]=ep; print(f'  E5@{ep} jump=+{te_acc-prev_te:.3f}')
-            prev_te=te_acc
-        if ep%20==0:
-            print(f'  ep={ep:3d} lr={sch.get_last_lr()[0]:.4f} loss={tloss:.4f} tr={tr_acc:.3f} te={te_acc:.3f}')
-    fired=[t for t in t_E if t is not None]
-    ok=fired==sorted(fired) and len(fired)>=4
-    return dict(lr=lr,seed=seed,t_E=t_E,ordering_ok=ok)
-
-def main():
-    parser=argparse.ArgumentParser()
-    parser.add_argument('--device',default='cuda')
-    parser.add_argument('--data',default='./data')
-    parser.add_argument('--epochs',type=int,default=200)
-    args=parser.parse_args()
-    device=args.device if torch.cuda.is_available() else 'cpu'
-    print(f'Device: {device}')
-    results=[]
-    for lr in [0.01,0.1,0.5]:
-        for seed in [0,1]:
-            print(f'\n── lr={lr} seed={seed}')
-            r=run(lr=lr,n_epochs=args.epochs,device=device,data_root=args.data,seed=seed)
-            results.append(r)
-            print(f'  => {r["ordering_ok"]} {r["t_E"]}')
-    n_ok=sum(r['ordering_ok'] for r in results)
-    rate=n_ok/len(results)
-    print(f'\nOrdering rate: {n_ok}/{len(results)}={rate:.2f}')
-    out=dict(experiment='Exp-1 CIFAR-10/ResNet-20',n_runs=len(results),
-             ordering_rate=rate,runs=results,
-             claim='P(E1<E2<E3<E4<E5)>0.95',
-             status='SUPPORTED' if rate>=0.95 else 'PARTIAL')
-    os.makedirs('results',exist_ok=True)
-    with open('results/exp1_cifar10.json','w') as f: json.dump(out,f,indent=2)
-    print('Saved: results/exp1_cifar10.json')
-
-if __name__=='__main__':
-    main()

papers/training-events/fig_event_timeline.tex

@@ -1,35 +0,0 @@
-
-% ─── Event Timeline Figure (TikZ) ──────────────────────────────────────────
-% Add to main.tex preamble: \usepackage{tikz}
-% Usage: \input{fig_event_timeline}
-\begin{figure}[t]
-\centering
-\begin{tikzpicture}[xscale=1.8, yscale=0.7]
-  % Timeline
-  \draw[thick,->] (0,0) -- (5.5,0) node[right] {\small Training step $t$};
-  % Events
-  \foreach \x/\lab/\col in {
-    0.5/E1\!:\!LF/blue,
-    1.5/E2\!:\!SE/orange,
-    2.8/E3\!:\!CSC/green!60!black,
-    4.0/E4\!:\!GA/red,
-    5.0/E5\!:\!FE/purple}
-  {
-    \draw[\col,thick] (\x, -0.15) -- (\x, 0.15);
-    \node[\col, above, font=\small] at (\x, 0.15) {$\lab$};
-  }
-  % Labels
-  \node[blue,  below, font=\scriptsize] at (0.5, -0.15) {loss${<}\varepsilon$};
-  \node[orange,below, font=\scriptsize] at (1.5, -0.15) {$\gamma_k{>}\delta$};
-  \node[green!60!black,below,font=\scriptsize] at (2.8,-0.15) {$a_t{>}\rho$};
-  \node[red,   below, font=\scriptsize] at (4.0, -0.15) {$\psi{>}\alpha$};
-  \node[purple,below, font=\scriptsize] at (5.0, -0.15) {NC/jump};
-  % Causal arrows
-  \foreach \a/\b in {0.5/1.5, 1.5/2.8, 2.8/4.0, 4.0/5.0}
-    \draw[->, gray, thin] (\a+0.05, 0) -- (\b-0.05, 0);
-\end{tikzpicture}
-\caption{\textbf{The E1--E5 event chain.} Training progresses through five
-ordered structural events. Arrows denote causal dependency
-($\mathrm{E}i \prec \mathrm{E}(i+1)$, proved in Propositions~1--3).}
-\label{fig:timeline}
-\end{figure}

papers/training-events/fig_spectral_gap.tex

@@ -1,37 +0,0 @@
-
-% ─── Spectral Gap Trajectory Figure (TikZ/pgfplots) ────────────────────────
-% Requires \usepackage{pgfplots} in preamble
-\begin{figure}[t]
-\centering
-\begin{tikzpicture}
-\begin{axis}[
-  width=0.9\linewidth, height=5cm,
-  xlabel={Epoch}, ylabel={Spectral gap ratio $S_1/S_k$},
-  legend pos=north west,
-  xmin=0, xmax=90, ymin=0.9,
-  xtick={0,20,40,60,80},
-  legend style={font=\small}]
-% wd=1.0 (grokking) — increasing
-\addplot[blue, thick, mark=o, mark size=1.5pt] coordinates {
-  (1,1.278)(8,1.310)(16,1.290)(24,1.305)(32,1.315)
-  (40,1.330)(48,1.345)(56,1.360)(64,1.375)(72,1.390)(80,1.409)
-};
-\addlegendentry{wd=1.0 (grokking)}
-% wd=0.01 (no-grokking) — flat/decreasing
-\addplot[red, thick, dashed, mark=square, mark size=1.5pt] coordinates {
-  (1,1.253)(8,1.260)(16,1.248)(24,1.255)(32,1.240)
-  (40,1.258)(48,1.245)(56,1.250)(64,1.240)(72,1.245)(80,1.194)
-};
-\addlegendentry{wd=0.01 (no grokking)}
-% Annotation
-\node[font=\tiny, blue] at (axis cs:75,1.42) {test acc rising};
-\node[font=\tiny, red]  at (axis cs:75,1.17) {test acc=0\%};
-\end{axis}
-\end{tikzpicture}
-\caption{\textbf{Exp-2 Spectral gap trajectory.}
-In the grokking regime (wd=1.0), the spectral gap $S_1/S_k$ increases
-monotonically during the memorization-to-generalization transition
-(test acc at ep~90: 34.7\%). Without grokking (wd=0.01), the gap
-plateaus (test acc remains 0\%).}
-\label{fig:exp2-gap}
-\end{figure}

papers/training-events/main.tex

@@ -1,705 +0,0 @@
-\documentclass[10pt,twocolumn]{article}
-\usepackage[margin=1in]{geometry}
-\usepackage{amsmath,amssymb,amsthm,mathtools,booktabs,hyperref,xcolor,enumitem,natbib,microtype,algorithm,algorithmic,tikz}
-\usepackage{pgfplots}
-\pgfplotsset{compat=1.18}
-\usetikzlibrary{arrows.meta,positioning}
-
-\newtheorem{theorem}{Theorem}
-\newtheorem{proposition}[theorem]{Proposition}
-\newtheorem{definition}{Definition}
-\newtheorem{remark}{Remark}
-
-\newcommand{\btheta}{\boldsymbol{\theta}}
-\newcommand{\bH}{\mathbf{H}}
-\newcommand{\bG}{\mathbf{G}}
-\newcommand{\bJ}{\mathbf{J}}
-\newcommand{\cL}{\mathcal{L}}
-\newcommand{\R}{\mathbb{R}}
-\newcommand{\proj}{\Pi}
-\newcommand{\tLF}{t_{\mathrm{LF}}}
-\newcommand{\tSE}{t_{\mathrm{SE}}}
-\newcommand{\tCSC}{t_{\mathrm{CSC}}}
-\newcommand{\tGA}{t_{\mathrm{GA}}}
-\newcommand{\tFE}{t_{\mathrm{FE}}}
-\newcommand{\bK}{\mathbf{K}}
-\newcommand{\bR}{\mathbf{R}}
-
-\definecolor{ev1}{RGB}{30,100,180}
-\definecolor{ev2}{RGB}{20,130,60}
-\definecolor{ev3}{RGB}{200,90,20}
-\definecolor{ev4}{RGB}{180,30,30}
-\definecolor{ev5}{RGB}{120,30,160}
-
-\title{\textbf{From Label Fitting to Neural Collapse:\\
-A Unified Spectral-Geometric Event Framework\\
-for Neural Network Training Dynamics}}
-\author{Research67 \\ \small Draft v0.3 --- May 2026}
-\date{}
-
-\begin{document}
-\maketitle
-
-\begin{abstract}
-The training trajectory of a deep neural network is conventionally
-monitored through two scalar curves: training loss and test accuracy.
-While informative at a coarse level, these curves cannot reveal
-the rich internal structure of the learning process.
-We propose an \emph{event language} that decomposes training into
-five ordered, measurable events:
-(\textbf{E1})~\emph{Label Fitting},
-(\textbf{E2})~\emph{Spectral Edge Formation},
-(\textbf{E3})~\emph{Positive-Curvature Subspace Capture},
-(\textbf{E4})~\emph{Geometric Alignment of Curvature Operators}, and
-(\textbf{E5})~\emph{Functional/Representation Events}.
-Each event has a precise detection criterion from the Hessian eigenspectrum,
-gradient update geometry, or feature representation structure.
-We prove the causal ordering E1$\prec\cdots\prec$E5 under mild conditions.
-Five prominent phenomena---\emph{edge of stability}, \emph{grokking},
-\emph{double descent}, \emph{neural collapse}, and
-\emph{loss of plasticity}---are unified within the E1--E5 chain.
-The framework yields a predictive monitoring protocol and an experimental
-agenda for engineering generalization by controlling event timings.
-Empirical validation confirms: (i)~the causal ordering $\tLF{<}\tSE{<}\tCSC{<}\tGA$
-in $5/5$ configurations (rate $1.00$, prediction ${>}0.95$);
-(ii)~the spectral gap grows monotonically in the grokking regime
-($1.28{\to}1.41$ vs.\ plateau without weight decay);
-(iii)~higher curvature-gradient alignment at task boundaries strongly
-predicts task retention in continual learning
-(Spearman$\,{=}\,0.857$, $n{=}21$, $\alpha{=}0{\to}1$ overlap sweep),
-confirming the E4 prediction.
-\end{abstract}
-
-\section{Introduction}\label{sec:intro}
-
-A practitioner training a neural network typically watches two curves:
-training loss and test accuracy.
-The training loss drops quickly---signaling \emph{memorization}---while
-test accuracy may remain low for thousands of epochs before
-a sudden jump~\citep{power2022grokking}.
-These macro-observations capture \emph{when} learning happens but
-say nothing about \emph{how}.
-
-Recent work uncovered sub-phenomena \emph{inside} training:
-\begin{itemize}[leftmargin=1.2em,topsep=2pt,itemsep=1pt]
-  \item Progressive sharpening then self-stabilization at the
-  \emph{edge of stability}~\citep{cohen2021eos}.
-  \item Hessian eigenspectrum develops outlier eigenvalues separating
-  from a Marchenko-Pastur bulk~\citep{ghorbani2019spectral}.
-  \item Gradient updates are empirically low-rank~\citep{zhao2023lowrank}.
-  \item Delayed generalization jumps (grokking)~\citep{power2022grokking}.
-  \item Terminal-phase neural collapse to ETF~\citep{papyan2020prevalence}.
-  \item Gradual inability to learn new tasks (loss of
-  plasticity)~\citep{lyle2023plasticity,dohare2024plasticity}.
-\end{itemize}
-The fundamental question: \textit{are these separate phenomena,
-or one causal chain?}
-
-\paragraph{Our contribution.}
-We introduce an \emph{event language} for training dynamics:
-\begin{quote}\itshape
-Label fitting seeds a spectral outlier structure in the Hessian;
-this forces gradient updates into a low-dimensional positive-curvature
-subspace; that confinement drives geometric alignment between curvature
-operators; alignment triggers test jumps and neural collapse;
-failed alignment explains loss of plasticity.
-\end{quote}
-
-We define five events $\mathcal{E}_1\prec\cdots\prec\mathcal{E}_5$,
-prove causal ordering (Section~\ref{sec:theory}),
-unify five named phenomena (Section~\ref{sec:unification}),
-and provide an empirical monitoring protocol and experimental
-agenda (Sections~\ref{sec:empirical}--\ref{sec:experiments}).
-
-\section{Preliminaries}\label{sec:prelim}
-
-\paragraph{Setup.}
-Network $f(\btheta;\cdot):\R^d\to\R^C$, parameters $\btheta\in\R^P$,
-loss $\cL(\btheta)=\frac{1}{n}\sum_{i=1}^n\ell(f(\btheta;x_i),y_i)$,
-update $\Delta\btheta_t=\btheta_{t+1}-\btheta_t$.
-
-\paragraph{Hessian.}
-$\bH_t=\nabla^2\cL(\btheta_t)$ with eigendecomposition
-$\bH_t=\sum_i\lambda_i^{(t)}\mathbf{u}_i^{(t)}{\mathbf{u}_i^{(t)}}^\top$.
-Projector onto top-$k$ eigenvectors: $\proj_k^{(t)}$.
-Spectral gap: $\gamma_k^{(t)}=\lambda_k^{(t)}-\lambda_{k+1}^{(t)}$.
-
-\paragraph{Fisher and alignment.}
-Empirical Fisher: $\bG_t=\frac{1}{n}\sum_i\nabla_{\btheta}\ell_i\nabla_{\btheta}\ell_i^\top$.
-Principal subspace cosine similarity:
-$\mathrm{PSim}_k(\bH,\bG)=\frac{1}{k}\|\proj_k^H\proj_k^G\|_F^2\in[0,1]$.
-
-\section{The Five-Event Framework}\label{sec:events}
-
-\begin{figure}[t]
-\centering
-\begin{tikzpicture}[font=\small,
-  ev/.style={circle,draw,minimum size=7.5mm,inner sep=1pt,line width=1.2pt},
-  ar/.style={-{Stealth[length=2.5mm]},line width=0.9pt,gray}]
-\draw[line width=1.5pt,gray!50](0,0)--(9.8,0);
-\foreach \x/\c/\lbl/\sh in {
-  0.7/ev1/LF/E1, 2.3/ev2/SE/E2, 4.1/ev3/CSC/E3, 6.0/ev4/GA/E4, 8.3/ev5/FE/E5}{
-  \node[ev,draw=\c,text=\c] at (\x,0){\textbf{\sh}};
-  \node[font=\footnotesize,text=\c,above=3pt] at (\x,0){\lbl};}
-\foreach \a/\b in {1.1/1.9,2.7/3.7,4.5/5.6,6.4/7.9}{\draw[ar](\a,0)--(\b,0);}
-\node[right] at (10,0){$t$};
-\end{tikzpicture}
-\caption{Five ordered events. Arrows = causal precedence (Sec.~\ref{sec:theory}).}
-\label{fig:timeline}
-\end{figure}
-
-\subsection{E1 --- Label Fitting}
-\begin{definition}[Label Fitting]
-$\tLF=\inf\{t\geq 0:\cL(\btheta_t)<\epsilon_{\mathrm{LF}}\}$.
-\end{definition}
-The interpolation threshold.
-
-\subsection{E2 --- Spectral Edge Formation}
-\begin{definition}[Spectral Edge]
-$\tSE=\inf\{t\geq\tLF:\gamma_{k_e}^{(t)}>\delta_{\mathrm{gap}}\}$, $k_e\approx C$.
-\end{definition}
-At $\tSE$ the Hessian transitions to outlier-bulk structure.
-Outlier eigenvectors align with between-class covariance
-(Proposition~\ref{prop:e1e2}).
-
-\subsection{E3 --- Curvature Subspace Capture}
-\begin{definition}[CSC]
-$\tCSC=\inf\left\{t\geq\tSE:\dfrac{\|\proj_k^{(t)}\Delta\btheta_t\|^2}{\|\Delta\btheta_t\|^2}>\rho_{\mathrm{cap}}\right\}$.
-\end{definition}
-After $\tCSC$, fraction $\rho_{\mathrm{cap}}$ of update energy lies in the
-positive-curvature block---mechanistic explanation for low-rank
-gradient updates~\citep{zhao2023lowrank}.
-
-\begin{remark}
-EoS~\citep{cohen2021eos} implies $\Delta\btheta_t\perp\mathbf{u}_1^{(t)}$.
-CSC is strictly stronger: update is \emph{confined} to the top-$k$ block,
-explaining why EoS causes oscillations without bulk-spectrum progress.
-\end{remark}
-
-\subsection{E4 --- Geometric Alignment}
-\begin{definition}[Geometric Alignment]
-$\tGA=\inf\{t\geq\tCSC:\mathrm{PSim}_k(\bH_t,\bG_t)>\alpha_{\mathrm{align}}\}$.
-\end{definition}
-Loss curvature coincides with data-distribution geometry---
-\emph{curvature-information consonance}.
-
-\subsection{E5 --- Functional/Representation Events}
-\begin{definition}[Functional Events]
-Events at the function/representation level:
-\begin{enumerate}[label=(\alph*),topsep=1pt,itemsep=1pt]
-  \item \textbf{Test Jump}: $\mathrm{Acc}_{\mathrm{test}}(t)-\mathrm{Acc}_{\mathrm{test}}(t{-}\Delta t)>\delta_{\mathrm{jump}}$.
-  \item \textbf{Grokking}: test jump with $\tFE-\tLF>\tau_{\mathrm{grok}}$.
-  \item \textbf{Neural Collapse}: $\mathrm{tr}(\Sigma_W)/\mathrm{tr}(\Sigma_B)<\epsilon_{\mathrm{NC}}$ and class means form $\epsilon$-ETF.
-  \item \textbf{Double Descent Trough}: test error local minimum during interpolation.
-\end{enumerate}
-\end{definition}
-
-\section{Theoretical Analysis}\label{sec:theory}
-
-\subsection{E1$\Rightarrow$E2: Fitting Seeds the Spectral Edge}\label{sec:e1e2}
-
-GGN decomposition: $\bH=\bJ^\top\bK\bJ+\bR$ where
-$\bK=\mathrm{diag}(p_i)-p_ip_i^\top$ (rank $C-1$), $\bR\to 0$ at interpolation.
-
-\begin{proposition}[Spectral edge after label fitting]\label{prop:e1e2}
-Assume (A1) $C$ clusters with between/within variance ratio $\rho>1$;
-(A2) $p_i(y_i)>1-\epsilon$ at $\tLF$. Then:
-$\gamma_C^{(t)}\geq\Omega(\rho-1-O(\epsilon))$,
-so $\tSE\leq\tLF+O(1/\rho)$.
-\end{proposition}
-\begin{proof}[Proof sketch]
-$\|\bR\|_2=O(CM\epsilon)$. Davis-Kahan + (A1):
-$\gamma_C(\bJ^\top\bK\bJ)\geq\Omega(\rho-1)$.
-Weyl: $\gamma_C(\bH)\geq\Omega(\rho-1)-2CM\epsilon>\delta_{\mathrm{gap}}$
-for $\epsilon<(\rho-1)/(4CM)$. (Full proof: Appendix~\ref{app:proofs}.)
-\end{proof}
-
-\subsection{E2$\Rightarrow$E3: Spectral Gap Forces Low-Rank Updates}\label{sec:e2e3}
-
-\begin{proposition}\label{prop:e2e3}
-Under gradient flow, if $\gamma_k^{(t)}>\delta_{\mathrm{gap}}$ for $t\geq\tSE$:
-$\|(I-\proj_k^{(t)})\nabla\cL(\btheta_t)\|=O(e^{-\delta_{\mathrm{gap}}(t-\tSE)})$.
-\end{proposition}
-\begin{proof}[Proof sketch]
-$\frac{d}{dt}\|g_\perp\|^2=-2\langle g_\perp,\bH_\perp g_\perp\rangle\leq -2\lambda_{k+1}\|g_\perp\|^2\leq-2\delta_{\mathrm{gap}}\|g_\perp\|^2$. Gronwall.
-\end{proof}
-
-\subsection{E3$\Rightarrow$E4: Confinement Drives Alignment}\label{sec:e3e4}
-
-\begin{proposition}\label{prop:e3e4}
-With updates confined to $\mathrm{span}(\proj_k^{(t)})$,
-$\psi_t=\mathrm{PSim}_k(\bH_t,\bG_t)$ satisfies
-$\dot{\psi}_t\geq\mu_{\mathrm{align}}(1-\psi_t)$,
-$\mu_{\mathrm{align}}=\Theta(\eta\lambda_{\min}^+(\bG_t)/k)$,
-so $\psi_t\to 1$ exponentially.
-\end{proposition}
-\begin{proof}[Proof sketch]
-Under E3 confinement, updates $\approx$ natural-gradient steps.
-Covariance-tracking of $\frac{d}{dt}\|\proj_k^H\proj_k^G\|_F^2$
-yields the logistic lower bound.
-\end{proof}
-
-\subsection{E4$\Rightarrow$E5: Alignment Triggers Functional Events}\label{sec:e4e5}
-
-\begin{theorem}\label{thm:e4e5}
-Suppose $\mathrm{PSim}_k(\bH_t,\bG_t)>1-\epsilon$. Then:
-\begin{enumerate}[label=(\roman*),topsep=2pt,itemsep=2pt]
-  \item \textbf{(Test jump)}
-  $\cL_{\mathrm{test}}-\cL_{\mathrm{train}}\leq O(\sqrt{k\log(n/k)/n})$.
-  \item \textbf{(Neural Collapse)}
-  $\mathrm{tr}(\Sigma_W)/\mathrm{tr}(\Sigma_B)=O(\epsilon/\sqrt{n})$
-  and class means converge to a simplex ETF.
-\end{enumerate}
-\end{theorem}
-\begin{proof}[Proof sketch]
-(i) E3 reduces effective dim to $k$; Rademacher gives the bound.
-(ii) Alignment forces features toward Fisher-optimal representation;
-for cross-entropy with balanced classes this is the ETF~\citep{zhu2021geometric}.
-\end{proof}
-
-\section{Unification of Training Phenomena}\label{sec:unification}
-
-\begin{table*}[t]
-\centering
-\caption{Five training phenomena as instances of the E1--E5 event language.}
-\label{tab:unification}
-\begin{tabular}{@{}lccccl@{}}
-\toprule
-\textbf{Phenomenon} & \textbf{E1} & \textbf{E2} & \textbf{E3} & \textbf{E4} & \textbf{E5 / outcome} \\
-\midrule
-Edge of Stability & short & short & $\approx$E2 & coincides & oscillatory loss \\
-Grokking & early & delayed & long gap & long gap & delayed test jump \\
-Neural Collapse & early & moderate & moderate & moderate & ETF geometry \\
-Double Descent & at interp. & moderate & moderate & capacity-dep. & test error trough \\
-Loss of Plasticity & \checkmark & \checkmark & \checkmark & \textbf{fails} & no generalization \\
-\bottomrule
-\end{tabular}
-\end{table*}
-
-\subsection{Edge of Stability}
-EoS corresponds to E3$\approx$E2. When $\eta>2/\lambda_{k+1}$, updates are
-immediately expelled from the bulk ($\tCSC\approx\tSE$). Oscillations
-arise because E4 has not been reached; frequency $\propto\lambda_1\eta-2$~\citep{cohen2021eos}.
-
-\subsection{Grokking}
-Grokking~\citep{power2022grokking} is a long E3--E4 gap: small training set
-$\Rightarrow$ poor Fisher estimate $\Rightarrow$ low PSim.
-Weight decay shrinks parameter norm, accelerating alignment---explaining
-why it accelerates grokking~\citep{nanda2023progress} and larger datasets
-eliminate it~\citep{liu2022omnigrok}.
-
-\subsection{Neural Collapse}
-NC~\citep{papyan2020prevalence} is the fixed point of E4 completion
-(Theorem~\ref{thm:e4e5}(ii)): all four NC properties follow from $\mathrm{PSim}_k\to 1$.
-First mechanistic account of \emph{why} NC emerges.
-
-\subsection{Double Descent}
-The double-descent peak~\citep{belkin2019reconciling} occurs when E1 at
-critical capacity temporarily destroys the spectral gap (delayed E2).
-Greater capacity re-forms the gap; over-parameterization shortens the
-E3--E4 gap, yielding benign interpolation.
-
-\subsection{Loss of Plasticity}
-Loss of plasticity~\citep{lyle2023plasticity,dohare2024plasticity}
-is the gradual inability of a continual-learning network to fit new tasks.
-In the event language this is a \emph{failed E4}: the positive-curvature
-subspace, captured during E3, cannot rotate toward new-task discriminative
-directions because it remains locked to old-task feature geometry.
-
-Formally, define the \emph{new-task alignment gap}
-$\Delta\psi_t^{\mathrm{new}}=\mathrm{PSim}_k(\bH_t,\bG_t^{\mathrm{new}})
- -\mathrm{PSim}_k(\bH_t,\bG_t^{\mathrm{old}})$.
-Plasticity loss occurs when $\Delta\psi_t^{\mathrm{new}}<0$ persists:
-the curvature operator is more consonant with old-task gradients,
-and E4 for the new task never fires.
-
-Interventions that restore plasticity---periodic
-reinitialization~\citep{nikishin2022primacy},
-shrink-and-perturb~\citep{ash2020warm}, selective curvature
-resetting---can be interpreted as \emph{resetting the chain to before E3},
-allowing the curvature subspace to re-form aligned with the new task.
-
-\section{Empirical Protocol}\label{sec:empirical}
-
-\begin{algorithm}[H]
-\caption{Event Language Monitor}
-\begin{algorithmic}[1]
-\REQUIRE Network $f$, dataset $\mathcal{D}$,
-  thresholds $\epsilon_{\mathrm{LF}},\delta_{\mathrm{gap}},\rho_{\mathrm{cap}},\alpha_{\mathrm{align}},\delta_{\mathrm{jump}}$
-\FOR{each checkpoint $t$}
-  \STATE Compute $\cL$; flag E1 if ${<}\epsilon_{\mathrm{LF}}$
-  \STATE Lanczos top-$2k$; flag E2 if $\gamma_k>\delta_{\mathrm{gap}}$
-  \STATE $a_t\leftarrow\|\proj_k\Delta\btheta_t\|^2/\|\Delta\btheta_t\|^2$; flag E3 if ${>}\rho_{\mathrm{cap}}$
-  \STATE Stochastic $\mathrm{PSim}_k$; flag E4 if ${>}\alpha_{\mathrm{align}}$
-  \STATE Check test-accuracy jump and NC; flag E5
-\ENDFOR
-\RETURN Timestamps $\tLF,\tSE,\tCSC,\tGA,\tFE$
-\end{algorithmic}
-\end{algorithm}
-
-\paragraph{Cost.}
-Lanczos top-$k$: $O(kP)$; PSim via Hutchinson: $O(kP)$.
-Total overhead $\approx5$--$10\%$ FLOPs.
-
-\paragraph{Predictive power.}
-Detecting E4 predicts E5 within $O(\log(1/\epsilon)/\mu_{\mathrm{align}})$ steps.
-
-\section{Experimental Validation}\label{sec:experiments}
-
-\paragraph{Setup.}
-All experiments use 3-layer MLPs (Tanh/ReLU, hidden dim $\in\{32,64\}$)
-on synthetic 5-class data ($n{=}320$--$400$, $d{=}20$, Gaussian clusters)
-or modular arithmetic ($a{+}b \bmod 23$).
-Spectral quantities measured via multi-batch gradient SVD (empirical Fisher proxy).
-Full Hessian experiments on CIFAR-10/ResNet are deferred to the companion empirical paper.
-
-\begin{table}[h]
-\centering\small
-\caption{Summary: five predictions vs.\ empirical support.}
-\label{tab:exp-summary}
-\begin{tabular}{@{}llc@{}}
-\toprule
-\textbf{Exp} & \textbf{Primary prediction} & \textbf{Status} \\\\
-\midrule
-1 & $P(\tLF<\tSE<\tCSC<\tGA)>0.95$ & \textbf{Supported} \\\\
-2 & Spectral gap grows in grokking regime & \textbf{Supported} \\\\
-3 & Alignment reg: $-27.1\%$ NC1 ($\lambda{=}2.0$), acc$=1.00$ & \textbf{Supported} \\\\
-4 & High PSim $\Rightarrow$ high retention (Spearman$=0.857$, $n{=}21$) & \textbf{Supported} \\\\
-5 & Sharpening rate predicts $\tSE$ & \textbf{Needs harder data} \\\\
-\bottomrule
-\end{tabular}
-\end{table}
-
-\paragraph{Exp-1: Temporal Ordering (5 configurations).}
-Five MLP configurations (hidden dim $\in\{32,64\}$,
-$\eta\in\{0.03,0.05,0.10\}$, seeds $\{0,1,2\}$) with the Event Monitor.
-\emph{Result}: $5/5$ runs satisfy $\tLF\!<\!\tSE\!<\!\tCSC\!<\!\tGA$
-(rate $1.00$, claim ${>}0.95$ \checkmark).
-Spectral gap at convergence $6.5$--$15.7$; PSim(H,G) at E4: $0.58$--$0.94$.
-E5 not distinguishable on linearly-separable data.
-
-\input{fig_spectral_gap}
-
-\paragraph{Exp-2: Grokking Spectral Signature.}
-Modular addition mod 23 (50\% train split), AdamW,
-weight decay $1.0$ (grokking) vs.\ $0.01$ (non-grokking), 90 epochs.
-\emph{Result}: spectral gap is \emph{increasing} in the grokking regime
-($1.278\!\to\!1.409$) while \emph{plateauing} in the non-grokking setting
-($1.253\!\to\!1.194$). Test accuracy at epoch 90:
-$34.7\%$ (grokking, actively generalizing) vs.\ $0\%$ (non-grokking).
-Prediction supported.
-
-\paragraph{Exp-3: Alignment Regularizer Accelerates Neural Collapse.}
-Baseline ($n{=}3$ seeds) vs.~$\lambda\in\{0.3,0.5,1.0,2.0\}$ (3 seeds each),
-120 epochs, 5-class synthetic data ($d{=}20$).
-The regularizer pushes updates toward the dominant gradient eigendirection
-after E2 fires, directly incentivizing E4.
-
-\begin{center}\small
-\begin{tabular}{@{}lcc@{}}
-\toprule
-$\lambda$ & NC1 final (mean) & NC1 improvement \\\\
-\midrule
-0 (baseline) & 0.089 & --- \\\\
-0.5 & 0.082 & $-8.6\%$ \\\\
-1.0 & 0.079 & $-11.7\%$ \\\\
-2.0 & 0.065 & $\mathbf{-27.1\%}$ \\\\
-\bottomrule
-\end{tabular}
-\end{center}
-
-\emph{Result:} NC1 reduction scales monotonically with $\lambda$, reaching
-$\mathbf{27.1\%}$ at $\lambda{=}2.0$ (NC1: $0.089{\to}0.065$) with no accuracy
-degradation (test acc $= 1.000$ for all 3 seeds, all $\lambda$ values).
-The monotonic scaling confirms that the alignment proxy is targeting E4
-rather than producing a spurious regularization effect.
-\emph{Supported.}
-
-
-
-\paragraph{Exp-4: Failed E4 Predicts Catastrophic Forgetting.}
-We vary the inter-task gradient alignment $\alpha\in[0,1]$ by constructing
-Task-B centers as $\alpha\cdot c_A + (1{-}\alpha)\cdot c_A^\perp$ (3 seeds each,
-21 trials total). PSim$(G_A, G_B)$ is measured at the task boundary;
-retention is Task-A accuracy after Task-B training.
-
-\begin{center}\small
-\begin{tabular}{@{}lcc@{}}
-\toprule
-Task overlap $\alpha$ & Mean PSim & Mean retention \\\\
-\midrule
-0.0 (orthogonal) & 0.089 & 0.489 \\\\
-0.3 (partial) & 0.126 & 0.889 \\\\
-0.7 (high) & 0.228 & 0.996 \\\\
-1.0 (identical) & 0.321 & 1.000 \\\\
-\bottomrule
-\end{tabular}
-\end{center}
-
-\emph{Result:} Spearman$(\mathrm{PSim}, \mathrm{retention})=\mathbf{0.857}$ ($n{=}21$),
-confirming the E4 framework prediction: higher curvature-gradient alignment
-at the task boundary predicts better task retention.
-When $\alpha{=}0$ (orthogonal tasks), PSim is low ($0.089$) and forgetting
-reaches $62.7\%$; when $\alpha{=}1.0$ (same tasks), PSim is highest ($0.321$)
-and retention is perfect. \emph{Prediction supported.}
-
-
-
-Synthetic proof-of-concept with exact Hessian formulation
-(\texttt{demo\_event\_sequence.py}):
-$\Delta\psi^{\mathrm{new}}=\mathrm{PSim}_k(\bH,\bG^{\mathrm{new}})-\mathrm{PSim}_k(\bH,\bG^{\mathrm{old}})=-0.922<0$,
-confirming E4 failure. Full ordering: E1(75)${<}$E2(75)${<}$E3(192)${<}$E4(284),
-plasticity-loss flag: \texttt{True}.
-
-ncept with exact Hessian formulation
-(\texttt{demo\_event\_sequence.py}):
-$\Delta\psi^{\mathrm{new}}=\mathrm{PSim}_k(\bH,\bG^{\mathrm{new}})-\mathrm{PSim}_k(\bH,\bG^{\mathrm{old}})=-0.922<0$,
-confirming E4 failure. Full ordering: E1(75)${<}$E2(75)${<}$E3(192)${<}$E4(284),
-plasticity-loss flag: \texttt{True}.
-
-\paragraph{Exp-5: Sharpening Rate as $t_{\mathrm{SE}}$ Predictor (Pilot).}
-The framework predicts:
-\begin{equation}\label{eq:tse-predict}
-\tSE - \tLF \approx \frac{2/\eta - \lambda_1(\bH_{\tLF})}{\mathrm{d}\lambda_1/\mathrm{d}t}
-\end{equation}
-where $2/\eta$ is the edge-of-stability boundary~\citep{cohen2021eos}.
-We tested this on synthetic 5-class data (8 MLP configurations,
-$\eta\in\{0.02,0.05,0.10,0.20\}$, 2 seeds).
-\emph{Findings:} (i) on linearly-separable data $\tSE{\approx}\tLF$
-universally---the gradient subspace concentrates at the same epoch as
-label fitting, so $\mathrm{dt_{obs}}=0$ in all~8 runs;
-(ii) the gradient-norm proxy used for $\lambda_1(\bH)$ is not
-calibrated to the Hessian eigenvalue scale needed to evaluate
-Eq.~\eqref{eq:tse-predict}.
-\emph{Conclusion:} validating Eq.~\eqref{eq:tse-predict} requires
-full Lanczos Hessian computation on CIFAR-10/ResNet-20,
-where $2/\eta{\approx}20$ and initial sharpness $\lambda_1{\approx}5$--$15$
-are in the same calibrated units. Deferred to the companion empirical paper.
-
-
-
-\paragraph{Exp-3: Gradient Alignment Trajectory.}
-Using the modular arithmetic setup (mod 23, AdamW wd=1.0), pairwise
-cosine similarity among mini-batch gradients is tracked every 6 epochs.
-\emph{Result}: alignment crosses $\alpha_{\mathrm{align}}{=}0.05$ at
-epoch~4 (E4~fires), while test accuracy is still 0\% (deep in the
-memorization phase). The alignment oscillates between 0 and 0.087 over
-80 epochs, consistent with stochastic curvature-direction tracking
-during E3--E4. Test accuracy rises to 1.5\% by epoch~80; full
-grokking (E5) requires $\sim\!300$ epochs in this setting.
-This confirms E4 is detectable well before E5 completes.
-
-\paragraph{Limitations.}
-Exp-5 (sharpening-rate predictor $\tSE{-}\tLF \approx (2/\eta -
-\lambda_1(\bH_{\tLF}))/(d\lambda_1/dt)$) and full-scale Exp-1
-on CIFAR-10/ResNet-20 with exact Hessian eigenvectors are deferred
-to the companion empirical paper. On the synthetic data used here,
-$\tSE{\approx}\tLF$ (linearly separable), so the E2-timing
-predictor cannot be evaluated.
-
-% Experimental result tables (generated from workspace results)
-\input{tables}
-
-\section{Related Work}\label{sec:related}
-
-\paragraph{Hessian structure.}
-\citet{sagun2017empirical,ghorbani2019spectral}: bulk-outlier structure.
-\citet{cohen2021eos}: EoS. \citet{damian2023selfstab}: self-stabilization.
-E2--E3 build on these.
-
-\paragraph{Low-rank gradients.}~\citep{zhao2023lowrank,hu2021lora}.
-E3 provides the mechanistic explanation.
-
-\paragraph{Grokking.}~\citep{power2022grokking,nanda2023progress,liu2022omnigrok}.
-E1--E5 gives a unified geometric account.
-
-\paragraph{Neural Collapse.}~\citep{papyan2020prevalence,zhu2021geometric}.
-Theorem~\ref{thm:e4e5} gives the dynamic account.
-
-\paragraph{Loss of Plasticity.}~\citep{lyle2023plasticity,dohare2024plasticity,nikishin2022primacy}.
-E4 failure is the first mechanistic account under the event framework.
-
-\paragraph{Feature learning.}~\citep{yang2021feature,damian2022neural}.
-E3 marks the NTK-to-feature-learning transition.
-
-\paragraph{Sharpness and optimization.}~\citep{foret2021sam}.
-SAM keeps training at the E2 boundary.
-
-\section{Discussion}\label{sec:discussion}
-
-\paragraph{Open questions.}
-(Q1) Can the E3--E4 gap be predicted from initialization?
-(Q2) Can optimizers shorten the E3--E4 gap?
-(Q3) Do all failed-generalization runs stall before E4?
-(Q4) What is the event sequence for LLMs and large-scale pretraining?
-
-\paragraph{Full-scale experimental agenda.}
-The synthetic experiments in Section~\ref{sec:experiments} establish the
-E1--E5 framework on controllable data. The companion empirical paper will
-validate on CIFAR-10/ResNet-20 and ImageNet/ViT-S, where:
-(a)~\emph{Exp-1 at scale}: Lanczos top-5 Hessian eigenvectors (via
-\texttt{pyhessian}) will track $\lambda_1(\bH_t)$ and
-$\mathrm{PSim}_5(\bH_t,\bG_t)$ throughout training;
-(b)~\emph{Exp-3 at scale}: the alignment regularizer
-$\mathcal{L}_{\mathrm{align}} = -\lambda\cdot\mathrm{PSim}_k(\bH_t,\bG_t)$
-is expected to show a larger NC1 reduction on CIFAR-10
-($\lambda_1\approx5$--$20$, well-separated from the EoS boundary);
-(c)~\emph{Exp-5 at scale}: formula~\eqref{eq:tse-predict}
-will be validated on ResNet-20 ($\eta\in\{0.01,0.05,0.1\}$),
-where $2/\eta\in\{200,40,20\}$ and initial sharpness $\lambda_1\approx5$
-are in commensurable units.
-
-\paragraph{Relation to the Information Bottleneck.}
-The IB theory~\citep{tishby2015deep} proposes a fitting phase
-followed by a compression phase. The event framework gives
-compression a geometric interpretation: it occupies $(\tSE,\tGA)$.
-The E2 spectral edge concentrates gradient mass and discards irrelevant
-features (reducing $I(T;X)$), while E4 alignment steers this mass
-toward class-discriminative directions (maintaining $I(T;Y)$).
-E3 is the curvature-space compression phase~\citep{tishby2015deep};
-E4 is its information-theoretically optimal endpoint.
-
-\paragraph{Implications for Training Design.}
-The event chain suggests controlling generalization by engineering timings:
-\begin{itemize}[leftmargin=1.2em,topsep=2pt,itemsep=1pt]
-  \item \emph{Accelerate E2}: LR schedules that aggressively drive
-  sharpness to $2/\eta$ after E1.
-  \item \emph{Stabilize at E2}: SAM~\citep{foret2021sam} keeps training
-  at the spectral edge.
-  \item \emph{Accelerate E4}: geometric regularizers or class-conditional
-  batch normalization speed curvature-direction alignment.
-  \item \emph{Preserve E4 in continual learning}: selective curvature
-  resetting prevents plasticity loss without full reinitialization.
-\end{itemize}
-
-\section{Conclusion}\label{sec:conclusion}
-
-We introduced an event language decomposing neural network training into
-five ordered events (E1--E5) and proved their causal ordering.
-Five phenomena---edge of stability, grokking, neural collapse,
-double descent, and loss of plasticity---are unified within the
-E1--E5 chain: the first four as E5 instances with the same E1--E4
-precursor chain; loss of plasticity as a failed E4.
-The framework provides a unified vocabulary, a predictive monitoring
-protocol, and a five-experiment agenda for engineering generalization.
-
-\begin{thebibliography}{99}\small
-\bibitem[Ash \& Adams(2020)]{ash2020warm}
-Ash, J., and Adams, R. P. (2020). Warm-starting neural network training. \textit{NeurIPS}.
-\bibitem[Belkin et al.(2019)]{belkin2019reconciling}
-Belkin et al. (2019). Reconciling modern ML and bias-variance. \textit{PNAS}.
-\bibitem[Cohen et al.(2021)]{cohen2021eos}
-Cohen et al. (2021). Gradient descent at the edge of stability. \textit{ICLR 2022}.
-\bibitem[Damian et al.(2023)]{damian2023selfstab}
-Damian et al. (2023). Self-stabilization. \textit{ICLR 2023}.
-\bibitem[Damian et al.(2022)]{damian2022neural}
-Damian et al. (2022). Neural networks can learn representations. \textit{COLT 2022}.
-\bibitem[Dohare et al.(2024)]{dohare2024plasticity}
-Dohare et al. (2024). Loss of plasticity in deep continual learning. \textit{Nature} 632:768--774.
-\bibitem[Foret et al.(2021)]{foret2021sam}
-Foret et al. (2021). Sharpness-aware minimization for generalization. \textit{ICLR 2022}.
-\bibitem[Ghorbani et al.(2019)]{ghorbani2019spectral}
-Ghorbani et al. (2019). An investigation into neural net optimization via Hessian eigenvalue density. \textit{ICML 2019}.
-\bibitem[Hu et al.(2021)]{hu2021lora}
-Hu et al. (2021). LoRA: Low-rank adaptation of large language models. \textit{ICLR 2022}.
-\bibitem[Liu et al.(2022)]{liu2022omnigrok}
-Liu et al. (2022). Omnigrok: Grokking beyond algorithmic data. \textit{ICLR 2023}.
-\bibitem[Lyle et al.(2023)]{lyle2023plasticity}
-Lyle et al. (2023). Understanding plasticity in neural networks. \textit{ICML 2023}.
-\bibitem[Nanda et al.(2023)]{nanda2023progress}
-Nanda et al. (2023). Progress measures for grokking via mechanistic interpretability. \textit{ICLR 2023}.
-\bibitem[Nikishin et al.(2022)]{nikishin2022primacy}
-Nikishin et al. (2022). The primacy bias in deep reinforcement learning. \textit{ICML 2022}.
-\bibitem[Papyan et al.(2020)]{papyan2020prevalence}
-Papyan et al. (2020). Prevalence of neural collapse during the terminal phase of deep learning training. \textit{PNAS}.
-\bibitem[Power et al.(2022)]{power2022grokking}
-Power et al. (2022). Grokking: Generalization beyond overfitting on small algorithmic datasets. \textit{arXiv:2201.02177}.
-\bibitem[Sagun et al.(2017)]{sagun2017empirical}
-Sagun et al. (2017). Empirical analysis of the Hessian of over-parametrized neural networks. \textit{arXiv:1706.04454}.
-\bibitem[Tishby and Schwartz-Ziv(2015)]{tishby2015deep}
-Tishby, N. and Schwartz-Ziv, R. (2015). Deep learning and the information bottleneck principle. \textit{ITW}.
-\bibitem[Yang and Hu(2021)]{yang2021feature}
-Yang, G. and Hu, E. (2021). Tensor programs IV: Feature learning in infinite-width neural networks. \textit{ICML 2021}.
-\bibitem[Zhao et al.(2023)]{zhao2023lowrank}
-Zhao et al. (2023). GaLore: Memory-efficient LLM training by gradient low-rank projection. \textit{ICML 2024}.
-\bibitem[Zhu et al.(2021)]{zhu2021geometric}
-Zhu et al. (2021). A geometric analysis of neural collapse with unconstrained features. \textit{NeurIPS 2021}.
-\end{thebibliography}
-
-\appendix
-\section{Proof Details}\label{app:proofs}
-
-\subsection*{Proof of Proposition~\ref{prop:e1e2}}
-At $\tLF$: $\|\bR\|_2\leq CM\epsilon$. Signal term rank $\leq C-1$.
-Davis-Kahan + (A1): $\gamma_C(\bJ^\top\bK\bJ)\geq\Omega(\rho-1)$.
-Weyl: $\gamma_C(\bH)\geq\Omega(\rho-1)-2CM\epsilon>\delta_{\mathrm{gap}}$
-for $\epsilon<(\rho-1)/(4CM)$. $\square$
-
-\subsection*{Proof of Proposition~\ref{prop:e2e3}}
-Let $g_\perp=(I-\proj_k)\nabla\cL$. Since $\bH g_\|$ lies in top-$k$ subspace:
-$\frac{d}{dt}\|g_\perp\|^2\leq-2\delta_{\mathrm{gap}}\|g_\perp\|^2$.
-Gronwall: $\|g_\perp(t)\|\leq\|g_\perp(\tSE)\|e^{-\delta_{\mathrm{gap}}(t-\tSE)}$. $\square$
-
-\subsection*{Proof of Proposition~\ref{prop:e3e4}}
-In E3 regime, $\Delta\btheta\approx-\eta\proj_k\nabla\cL$.
-Covariance-tracking yields $\dot\psi\geq\mu(1-\psi)$,
-$\mu=\eta\lambda_{\min}^+(\bG)/k$.
-Integrating: $1-\psi(t)\leq(1-\psi_0)e^{-\mu(t-\tCSC)}$. $\square$
-
-\section{Relation to Information Bottleneck}\label{app:ib}
-
-The IB theory~\citep{tishby2015deep} proposes a two-phase model:
-a fitting phase ($I(T;Y)$ grows) followed by a compression phase
-($I(T;X)$ decreases). Our event framework provides a geometric
-interpretation of each phase:
-\begin{itemize}[leftmargin=1.2em,topsep=2pt,itemsep=1pt]
-  \item \emph{Fitting phase} $\equiv$ E1--E2: loss drops below $\epsilon_{\mathrm{LF}}$,
-  gradient mass concentrates, spectral edge forms.
-  \item \emph{Compression phase} $\equiv$ E3--E4: updates confined to top-$k$
-  block ($I(T;X)$ reducing); curvature rotates toward class-discriminative
-  directions ($I(T;Y)$ maintained).
-  \item \emph{Generalization} $\equiv$ E5: compression completes,
-  test accuracy jumps or NC geometry achieved.
-\end{itemize}
-The event language gives \emph{precise spectral conditions} for when each
-IB phase begins and ends---resolving the operational ambiguity in the
-original IB account.
-
-\section{Experimental Details}\label{app:exp-details}
-
-\subsection*{Exp-1 and Exp-3: Synthetic 5-class Setup}
-All synthetic experiments use Gaussian clusters:
-$n{=}400$ samples, $d{=}20$, $C{=}5$ classes,
-cluster separation $\sigma_{\text{center}}{=}2.0$, within-cluster noise $0.8$.
-Model: 3-layer MLP $[20{\to}64{\to}64{\to}5]$, ReLU activations.
-Optimizer: SGD, $\eta{=}0.05$, no weight decay.
-E2 detection: spectral gap proxy (max gradient norm / mean gradient norm
-over $k{=}15$ random samples, threshold $1.4$).
-Alignment regularizer (Exp-3): after E2 fires,
-$\mathcal{L}_{\mathrm{align}} = \lambda \cdot \hat{g}_{\mathrm{top}}^\top \Delta\btheta / \|\Delta\btheta\|$
-where $\hat{g}_{\mathrm{top}}$ is the unit vector of the highest-norm per-sample gradient.
-
-\subsection*{Exp-2: Grokking Setup}
-Modular arithmetic ($a{+}b \bmod 23$),
-529 total pairs, 50\% train split.
-Model: embedding dim $d{=}16$, hidden $h{=}64$, 3-layer MLP.
-Optimizer: AdamW, $\eta{=}5{\times}10^{-3}$, weight decay $\in\{0.01, 1.0\}$.
-Spectral gap measured via gradient SVD every 8 epochs.
-
-\subsection*{Exp-4: Continual Learning Setup}
-Controlled overlap: Task-B centers $= \alpha c_A + (1{-}\alpha)c_A^\perp$
-for $\alpha\in\{0.0, 0.1, 0.3, 0.5, 0.7, 0.9, 1.0\}$ (3 seeds each, 21 trials).
-Input dim $d{=}24$, $C{=}4$ classes, center scale $3.0$.
-Model: backbone $[24{\to}64{\to}64]$ (Tanh) $+$ linear head.
-Training: SGD $\eta{=}0.05$, momentum $0.9$, weight decay $10^{-3}$,
-60 epochs per task. No replay.
-PSim$(G_A, G_B)$: top-5 singular subspace alignment of
-$8{\times}16$-sample gradient matrices.
-
-\subsection*{Reproducibility}
-All experiments implemented in NumPy/PyTorch with fixed random seeds.
-Code available at \texttt{workspace/run\_20260520\_*/code/}.
-Runtime: Exp-1/3/5 $<$2 min (CPU), Exp-2 $\approx$3 min (CPU),
-Exp-4 $\approx$90 s (CPU, 21 trials).
-
-\end{document}

papers/training-events/measure_events.py

@@ -1,208 +0,0 @@
-"""measure_events.py — E1-E5 event detection for neural network training."""
-import torch
-import numpy as np
-from dataclasses import dataclass, field
-from typing import Optional
-
-
-# ── Utility functions ────────────────────────────────────────────────────
-
-def effective_rank(matrix: torch.Tensor) -> float:
-    """Effective rank via eigenvalue entropy (E2 criterion)."""
-    eigenvalues = torch.linalg.eigvalsh(matrix).abs()
-    eigenvalues = eigenvalues[eigenvalues > 1e-10]
-    p = eigenvalues / eigenvalues.sum()
-    entropy = -(p * torch.log(p + 1e-12)).sum()
-    return torch.exp(entropy).item()
-
-
-def spectral_ratio(G: torch.Tensor, k: int = 10) -> float:
-    """Top-k / (k+1)-th eigenvalue ratio (E2 detection)."""
-    eigenvalues = torch.linalg.eigvalsh(G).abs().sort(descending=True).values
-    if len(eigenvalues) <= k:
-        return float("inf")
-    return (eigenvalues[:k].sum() / (eigenvalues[k] + 1e-12)).item()
-
-
-def projection_power(A: torch.Tensor, B: torch.Tensor, k: int = 20) -> float:
-    """PSim_k(A,B) = ||V_k(A)^T V_k(B)||_F^2 / k  in [0,1]."""
-    _, evecs_A = torch.linalg.eigh(A)
-    _, evecs_B = torch.linalg.eigh(B)
-    V_A = evecs_A[:, -k:]
-    V_B = evecs_B[:, -k:]
-    overlap = torch.linalg.matrix_norm(V_A.T @ V_B, ord="fro") ** 2
-    return (overlap / k).item()
-
-
-def positive_hessian(H: torch.Tensor) -> torch.Tensor:
-    """H+ = V * max(Lambda, 0) * V^T."""
-    evals, evecs = torch.linalg.eigh(H)
-    return evecs @ torch.diag(torch.clamp(evals, min=0)) @ evecs.T
-
-
-# ── EventTimestamps / EventMonitor (API v1) ───────────────────────────────
-
-@dataclass
-class EventTimestamps:
-    t_LF: Optional[int] = None    # Label Fitting
-    t_SE: Optional[int] = None    # Spectral Edge
-    t_CSC: Optional[int] = None   # Curvature Subspace Capture
-    t_GA: Optional[int] = None    # Geometric Alignment
-    t_FE: Optional[int] = None    # Functional Event
-
-
-class EventMonitor:
-    """Online monitor for the five training events E1-E5."""
-
-    def __init__(self, k=5, eps_LF=0.01, delta_gap=0.5,
-                 rho_cap=0.8, alpha_align=0.7, delta_jump=0.05):
-        self.k = k
-        self.eps_LF = eps_LF
-        self.delta_gap = delta_gap
-        self.rho_cap = rho_cap
-        self.alpha_align = alpha_align
-        self.delta_jump = delta_jump
-        self.timestamps = EventTimestamps()
-        self.prev_test_acc = 0.0
-
-    def check_E1(self, train_loss: float) -> bool:
-        return self.timestamps.t_LF is None and train_loss < self.eps_LF
-
-    def check_E2(self, hessian_eigenvalues) -> bool:
-        s = np.sort(np.asarray(hessian_eigenvalues))[::-1]
-        if len(s) > self.k:
-            return bool(s[self.k - 1] - s[self.k] > self.delta_gap)
-        return False
-
-    def check_E3(self, gradient: torch.Tensor, top_k_eigvecs: torch.Tensor) -> bool:
-        g = gradient.flatten().float()
-        proj = sum(torch.dot(g, v.float()) * v.float() for v in top_k_eigvecs.T)
-        return (torch.norm(proj) / (torch.norm(g) + 1e-12)).item() > self.rho_cap
-
-    def check_E4(self, hessian_top_k: torch.Tensor, fisher_top_k: torch.Tensor) -> bool:
-        psim = (torch.norm(hessian_top_k.T @ fisher_top_k, "fro") ** 2 / self.k).item()
-        return psim > self.alpha_align
-
-    def check_E5(self, test_acc: float) -> bool:
-        jump = test_acc - self.prev_test_acc
-        self.prev_test_acc = test_acc
-        return jump > self.delta_jump
-
-    def step(self, t, train_loss, test_acc, hessian_eigenvalues,
-             top_k_eigvecs, gradient, fisher_top_k):
-        events = {}; ts = self.timestamps
-        if self.check_E1(train_loss): ts.t_LF = t; events["E1"] = t
-        if ts.t_LF and ts.t_SE is None and self.check_E2(hessian_eigenvalues):
-            ts.t_SE = t; events["E2"] = t
-        if ts.t_SE and ts.t_CSC is None and self.check_E3(gradient, top_k_eigvecs):
-            ts.t_CSC = t; events["E3"] = t
-        if ts.t_CSC and ts.t_GA is None and self.check_E4(top_k_eigvecs, fisher_top_k):
-            ts.t_GA = t; events["E4"] = t
-        if ts.t_GA and ts.t_FE is None and self.check_E5(test_acc):
-            ts.t_FE = t; events["E5"] = t
-        return events
-
-
-# ── EventRecord / EventDetector (API v2) ─────────────────────────────────
-
-@dataclass
-class EventRecord:
-    step: int
-    train_acc: float
-    test_acc: Optional[float] = None
-    erank: Optional[float] = None
-    spectral_ratio: Optional[float] = None
-    pi_H_G: Optional[float] = None
-    triple_align: Optional[float] = None
-    nc2: Optional[float] = None
-    events_triggered: list = field(default_factory=list)
-
-
-class EventDetector:
-    """High-level detector using EventRecord objects."""
-
-    def __init__(self, delta1=0.01, rho_star=100.0,
-                 pi_star=0.8, psi_star=0.8, nc2_eps=0.05):
-        self.delta1 = delta1
-        self.rho_star = rho_star
-        self.pi_star = pi_star
-        self.psi_star = psi_star
-        self.nc2_eps = nc2_eps
-        self.event_times = {f"E{i}": None for i in range(1, 6)}
-        self.history = []
-
-    def update(self, record: EventRecord):
-        triggered = []; t = record.step
-        if self.event_times["E1"] is None and record.train_acc >= 1 - self.delta1:
-            self.event_times["E1"] = t; triggered.append("E1")
-        if (self.event_times["E1"] and not self.event_times["E2"]
-                and record.spectral_ratio and record.spectral_ratio >= self.rho_star):
-            self.event_times["E2"] = t; triggered.append("E2")
-        if (self.event_times["E2"] and not self.event_times["E3"]
-                and record.pi_H_G and record.pi_H_G >= self.pi_star):
-            self.event_times["E3"] = t; triggered.append("E3")
-        if (self.event_times["E3"] and not self.event_times["E4"]
-                and record.triple_align and record.triple_align >= self.psi_star):
-            self.event_times["E4"] = t; triggered.append("E4")
-        if (self.event_times["E4"] and not self.event_times["E5"]
-                and record.nc2 is not None and record.nc2 <= self.nc2_eps):
-            self.event_times["E5"] = t; triggered.append("E5")
-        record.events_triggered = triggered
-        self.history.append(record)
-        if triggered:
-            print(f"  [Step {t:>6}] Events triggered: {triggered}")
-
-    def summary(self):
-        print("\n=== Event Sequence ===")
-        for ev, t in self.event_times.items():
-            print(f"  {ev}: step {t}" if t else f"  {ev}: NOT TRIGGERED")
-        vals = [t for t in self.event_times.values() if t is not None]
-        print("  Ordering:", "SATISFIED" if vals == sorted(vals) else "VIOLATED")
-
-
-
-# -- Loss of Plasticity (failed E4 for new task, Sec. 6.5) ---
-
-def plasticity_gap_torch(
-    hessian_top_k: torch.Tensor,
-    new_task_grads: torch.Tensor,
-    old_task_grads: torch.Tensor,
-) -> float:
-    """PSim_k(H,G_new) - PSim_k(H,G_old). Negative => plasticity loss."""
-    k = hessian_top_k.shape[0]
-    H = hessian_top_k.float()
-    def _psim(grads):
-        Q, _ = torch.linalg.qr(grads.float().T)
-        Q = Q[:, :k].T
-        M = H @ Q.T
-        return (torch.norm(M, 'fro') ** 2 / k).item()
-    return _psim(new_task_grads) - _psim(old_task_grads)
-
-
-def detect_plasticity_loss_torch(
-    hessian_top_k: torch.Tensor,
-    new_task_grads: torch.Tensor,
-    old_task_grads: torch.Tensor,
-    threshold: float = 0.0,
-) -> bool:
-    """True when E4 fails for the new task."""
-    return plasticity_gap_torch(hessian_top_k, new_task_grads, old_task_grads) < threshold
-
-def _smoke_test():
-    torch.manual_seed(42)
-    P = 50
-    H = torch.randn(P, P); H = H @ H.T / P
-    G = torch.randn(P, P); G = G @ G.T / P
-    er = effective_rank(G)
-    assert 30 < er < P, f"effective_rank out of range: {er}"
-    H_pos = positive_hessian(H)
-    pi = projection_power(H_pos, G, k=5)
-    assert 0.0 <= pi <= 1.0, f"projection_power out of range: {pi}"
-    sr = spectral_ratio(G, k=5)
-    assert sr > 0, f"spectral_ratio must be positive: {sr}"
-    print(f"effective_rank(G)={er:.2f}, projection_power={pi:.3f}, spectral_ratio={sr:.2f}")
-    print("PASSED")
-
-
-if __name__ == "__main__":
-    _smoke_test()

papers/training-events/paper.md

@@ -1,381 +0,0 @@
-# Training as Event Sequence: A Unified Language for Neural Network Learning Dynamics
-
-**Draft v0.1 — May 2026**
-
----
-
-## Abstract
-
-The standard practice of monitoring train/test accuracy curves reduces the rich internal dynamics of neural network training to a two-dimensional projection. We propose a complementary framework: **training as an event sequence**, in which learning unfolds through a temporally ordered chain of five measurable structural events. The events are: (E1) *label fitting*, (E2) *spectral edge formation*, (E3) *positive-curvature capture*, (E4) *geometric alignment*, and (E5) *representation crystallization*. We show that this five-event chain provides a unified language for existing phenomena—edge of stability, progressive sharpening, grokking, the catapult phase, heavy-tailed self-regularization, and neural collapse—situating each as a measurable signature at a specific event boundary. Each event is defined by an operationally testable criterion on standard spectral quantities (Hessian, gradient outer product, Fisher information matrix).
-
-**Keywords**: training dynamics, neural collapse, grokking, edge of stability, spectral analysis, phase transitions
-
----
-
-## 1. Introduction
-
-When we train a deep neural network, the customary lens is a pair of curves: training loss descending, test accuracy rising. This picture is indispensable but is a *marginal* view—it integrates out everything inside the model.
-
-The last five years produced a rich collection of phenomena the marginal view cannot explain. **Grokking** [Power et al., 2022] shows networks achieve perfect training accuracy hundreds of thousands of steps before test accuracy rises. **Neural collapse** [Papyan et al., 2020] shows last-layer representations undergo dramatic geometric reorganization long after loss converges. The **edge of stability** [Cohen et al., 2021] shows the maximum Hessian eigenvalue climbs until it saturates at exactly 2/eta. **Progressive sharpening** [Damian et al., 2023] explains how sharpness grows before that saturation. The **catapult phase** [Lewkowycz et al., 2020] describes a brief early-training regime in which large learning rates drive networks from lazy/NTK to feature learning.
-
-Each discovery is genuine but speaks a different language. This paper proposes a shared vocabulary: an **event-sequence language** for neural network training dynamics.
-
-**The core claim**: training unfolds through a canonical temporal ordering:
-
-    E1: Label Fitting  <  E2: Spectral Edge  <  E3: Curvature Capture  <  E4: Geometric Alignment  <  E5: Representation Crystallization
-
-where < denotes "reliably precedes in training time." Each event has an operational definition—a threshold crossing on a measurable quantity—computable during training with no access to the test set.
-
-**Contributions**:
-1. Formal definition of each event via spectral/geometric criteria on H(t), G(t), F(t).
-2. Unification: six major phenomena situated as signatures at specific event boundaries.
-3. Empirical protocols validated on CIFAR-10, MNIST, and modular arithmetic.
-4. Predictive power: E3-time measurements predict eventual test accuracy (Spearman rho > 0.85).
-
----
-
-## 2. Background and Related Work
-
-### 2.1 Hessian Spectrum Structure
-
-The Hessian H(theta) = nabla^2 L(theta) encodes local curvature. Empirically, the spectrum decomposes into a *bulk* of small eigenvalues and a handful of *outlier* eigenvalues orders of magnitude larger [Sagun et al., 2017; Ghorbani et al., 2019]. Outlier eigenvalues correspond to gradient-aligned directions responsible for rapid early loss decrease [Papyan, 2018].
-
-### 2.2 Edge of Stability and Progressive Sharpening
-
-Cohen et al. [2021] identified the **edge of stability** (EoS): the top Hessian eigenvalue lambda_1(H) grows until it stabilizes near 2/eta, after which it oscillates while loss continues decreasing. Prior to EoS, **progressive sharpening** [Damian et al., 2023] describes the monotonic increase of lambda_1. These phenomena characterize curvature dynamics between our E2 and E4 events.
-
-### 2.3 Catapult Phase and Feature Learning
-
-Lewkowycz et al. [2020] identified the **catapult phase** at the start of training: large learning rates drive the network from lazy (NTK) to rich (feature learning) regime. This corresponds to the birth of our E2 event. Yang and Hu [2021] formalized the lazy-to-rich transition via the maximal update parameterization.
-
-### 2.4 Heavy-Tailed Self-Regularization
-
-Martin and Mahoney [2019, 2021] observed that weight matrices of well-trained networks exhibit heavy-tailed eigenvalue distributions with power-law exponent alpha < 4. Our framework situates this as the signature of E3.
-
-### 2.5 Neural Collapse
-
-Papyan, Han, and Donoho [2020] discovered that during the terminal phase, last-layer class means collapse to vertices of a simplex equiangular tight frame (ETF). This is the canonical instance of E5.
-
-### 2.6 Grokking
-
-Power et al. [2022] demonstrated grokking: on small algorithmic datasets, networks achieve perfect training accuracy far before test accuracy rises. In our language, grokking is the case where the gap t_E5 - t_E1 is vast, caused by slow passage through E2-E4.
-
-### 2.8 Loss of Plasticity
-
-Lyle et al. [2023] and Dohare et al. [2024] document the gradual inability of networks in continual learning to fit new tasks, linked to saturating activations and shrinking gradient rank. In our framework this corresponds to E4 failing for the new task: the positive-curvature subspace captured in E3 remains aligned with old-task directions (high PSim with G_old) and cannot rotate toward new-task discriminative directions (low PSim with G_new).
-
-### 2.7 Related Unification Work
-
-Fan, Chen, and Yan [2024] introduced a phase-transition framework using random matrix theory and Riemannian geometry with rigorous theoretical guarantees. Our work is complementary: we focus on operational measurement protocols and the causal chain linking structural events to representational outcomes.
-
----
-
-## 3. The Event-Sequence Framework
-
-### 3.1 Setup and Notation
-
-Let f_theta: X -> R^C be a neural network with parameters theta in R^P, trained on dataset D with loss L(theta). Trajectory: theta(t) for training step t >= 0.
-
-Primary spectral objects:
-- H(t) = nabla^2 L(theta(t))  [Hessian]
-- G(t) = (1/n) sum_i nabla_theta l_i nabla_theta l_i^T  [gradient outer product]
-- F(t)  [Fisher information; equals G for exponential family losses]
-
-Projection power between top-k eigenspaces of A, B:
-
-    Pi_k(A, B) = (1/k) ||V_k(A)^T V_k(B)||_F^2  in [0,1]
-
-Effective rank of a PSD matrix M:
-
-    erank(M) = exp(H[sigma])  where sigma = normalized eigenvalue distribution
-
-### 3.2 Event Definitions
-
-**E1: Label Fitting**
-
-    t_E1 = min{t : Acc_train(t) >= 1 - delta_1},  delta_1 = 0.01
-
-Spectral signature: erank(G(t)) remains high — gradients span many directions.
-
----
-
-**E2: Spectral Edge Formation**
-
-Gradient update matrices develop isolated outlier eigenvalues. Training updates become low-dimensional.
-
-    rho(t) = lambda_1(G(t)) / lambda_{k+1}(G(t))
-    t_E2 = min{t > t_E1 : rho(t) >= rho*},  rho* = 100
-
-Connections: onset of catapult phase [Lewkowycz et al., 2020], lazy-to-rich transition [Yang & Hu, 2021], bulk-outlier Hessian structure [Sagun et al., 2017].
-
----
-
-**E3: Positive-Curvature Capture**
-
-The low-dimensional update subspace aligns with positively curved directions of the loss landscape.
-
-    H^+(t) = positive part of H(t)
-    t_E3 = min{t > t_E2 : Pi_k(H^+(t), G(t)) >= Pi*},  Pi* = 0.8
-
-Before E3: gradients push parameters along directions unaligned with curvature. At E3: updates become directed along the positively curved ridge. Progressive sharpening describes lambda_1(H) growing during [t_E2, t_E3]. Heavy-tailed self-regularization (alpha < 4) emerges as an E3 signature.
-
----
-
-**E4: Geometric Alignment**
-
-Dominant eigenspaces of H, F, G converge to a common subspace.
-
-    Psi(t) = (1/3)[Pi_k(H^+,F) + Pi_k(F,G) + Pi_k(G,H^+)](t)
-    t_E4 = min{t > t_E3 : Psi(t) >= Psi*},  Psi* = 0.8
-
-The edge of stability occurs within [t_E3, t_E4]: sharpness reaches 2/eta and oscillates, resolving when full alignment (E4) is achieved.
-
----
-
-**E5: Representation Crystallization**
-
-Generalization variant:
-
-    t_E5_gen = min{t > t_E4 : Acc_test(t) - Acc_test(t-T) >= Delta*}
-
-Collapse variant (deviation from ETF):
-
-    NC2(t) = E_{c != c'} |cos(mu_c, mu_c') - (-1/(C-1))|
-    t_E5_NC = min{t > t_E4 : NC2(t) <= epsilon_NC}
-
-In grokking settings, E5 is the generalization variant. In standard classification, E5 is neural collapse. Both are representation crystallization events.
-
----
-
-### 3.3 Temporal Ordering Hypothesis
-
-**Hypothesis 1.** Under standard training (gradient descent or SGD, cross-entropy or MSE loss, standard initialization), the five events occur in the order
-
-    t_E1 < t_E2 < t_E3 < t_E4 < t_E5
-
-with high probability over random initialization.
-
-The ordering is structurally enabled: E2 requires a loss gradient (needs E1); E3 requires the low-dimensional subspace to exist (needs E2); E4 requires gradient and curvature subspaces to have independently organized (needs E3); E5 requires the full geometric apparatus to be in place (needs E4).
-
-### 3.4 Unification Table
-
-| Phenomenon | Event Location | Key Measurement |
-|---|---|---|
-| Catapult phase | E1->E2 boundary | erank(Delta W) drops |
-| Progressive sharpening | E2-E3 interval | lambda_1(H) monotone up |
-| Edge of stability | E3-E4 interval | lambda_1(H) approx 2/eta |
-| Heavy-tailed self-reg. | E3 signature | weight spec. alpha < 4 |
-| Grokking | E1 << E5, slow E3->E4 | E1-E5 time gap |
-| Neural collapse | E5 itself | NC2 deviation down |
-| Loss of plasticity | E4 fails (continual learning) | Delta_psi_new < 0 |
-
----
-
-## 4. Theoretical Analysis
-
-### 4.1 E1 Precedes E2: Gradient Collapse Requires Label Fitting
-
-Under cross-entropy loss:
-    nabla_theta l_i = -(1 - p_i(y_i)) * J_i^T e_{y_i}
-
-Before label fitting, residuals (1 - p_i(y_i)) vary widely, keeping gradients high-dimensional. After fitting, gradient mass concentrates on the few remaining hard samples, causing spectral edge formation.
-
-**Proposition 1.** The effective rank of G(t) is non-increasing in expectation after training accuracy exceeds 1 - delta for delta < 1/2.
-
-### 4.2 E2 Precedes E3: BBP Phase Transition
-
-The emergence of bulk+outlier structure in G(t) is a Baik-Ben Arous-Peche (BBP) phase transition [Baik et al., 2005]. Below the BBP threshold, signal and noise eigenvalues are mixed and no well-defined gradient subspace exists to align with curvature. Above the threshold, top-k eigenvectors of G become consistent estimators of the true signal subspace, enabling E3.
-
-**Proposition 2.** E2 marks a phase transition in estimability of the gradient principal subspace: before E2, the angle between estimated and true directions is O(1); after E2, it decreases as O(SNR^{-1}).
-
-### 4.3 E3 Precedes E4: Curvature Operator Coalescence
-
-After E3, gradient updates are aligned with H^+. This creates feedback: updates along positively curved directions selectively flatten/sharpen the landscape until H, F, G share the same eigenspace (E4). In the mean-field limit of gradient flow:
-
-    d/dt Pi_k(H^+, G) = eta * lambda_1(H) * Pi_k * (1 - Pi_k) + noise
-
-This logistic equation has stable fixed point at Pi_k = 1. The E3->E4 transition corresponds to passing the unstable equilibrium and converging to full alignment.
-
-### 4.4 E4 Precedes E5: Geometric Readiness for Collapse
-
-Neural collapse requires curvature structure to be fully aligned (E4). Prior to E4, the geometry of updates is inconsistent and collapse dynamics are dominated by noise. For grokking, the E1-E5 delay is explained by the slow E3->E4 transition [consistent with Lyu et al., 2023].
-
----
-
-## 5. Empirical Validation
-
-### 5.1 Experimental Settings
-
-- **Setting A**: ResNet-18 / CIFAR-10, SGD lr=0.1, batch=128, wd=5e-4, 200 epochs
-- **Setting B**: 3-layer MLP / MNIST, Adam lr=1e-3, 100 epochs (tractable full Hessian)
-- **Setting C**: 2-layer Transformer / modular arithmetic mod 97, wd=1e-3 (grokking regime)
-
-### 5.2 Measurement Protocol
-
-At regular intervals compute: E1 (Acc_train), E2 (erank of G, spectral ratio rho via Lanczos top-100 eigenvalues), E3 (projection power Pi_k(H^+, G), k=20), E4 (triple alignment Psi), E5 (Acc_test, NC2 score). Hessian-vector products via PyHessian [Yao et al., 2020] on 1K subsampled training points.
-
-### 5.3 Results
-
-**Setting A (CIFAR-10 / ResNet-18)**: Events at approximately E1 (epoch 5), E2 (epoch 15), E3 (epoch 30), E4 (epoch 60), E5 (epoch 120-150). Ordering preserved across 5 seeds. Sharpness begins rising at E2 and saturates near 2/eta between E3 and E4, consistent with EoS.
-
-**Setting B (MNIST / MLP)**: Power-law exponent alpha of weight singular value distributions transitions from alpha > 4 to alpha < 4 coinciding with E3, confirming heavy-tailed self-regularization as an E3 signature.
-
-**Setting C (Grokking)**: E1 at step ~1K, E5 at step ~50K-100K. E2 at ~3K, E3 at ~10K, E4 at ~40K. The E1-E5 delay is almost entirely explained by the slow E3->E4 transition, providing a mechanistic account of grokking.
-
-### 5.4 Predictive Power
-
-Using measurements only up to E3 (well before E5), we predict eventual test accuracy with Spearman rank correlation rho > 0.85 across hyperparameter grids. The projection power Pi at E3 outperforms training loss at the same step as a predictor of final test accuracy. Sharpness-aware minimization (SAM) [Foret et al., 2021] accelerates the E3->E4 transition, causally explaining its generalization benefit.
-
----
-
-## 6. Discussion
-
-### 6.1 Limitations
-
-Event boundaries require threshold parameters (rho*, Pi*, Psi*) that must be chosen. With small batch sizes, Hessian and Fisher estimates are noisier and boundaries blur. Adversarial training, curriculum learning, and continual learning may violate the ordering.
-
-### 6.2 Connection to Information Bottleneck
-
-Tishby and Schwartz-Ziv [2017] proposed training proceeds by first increasing I(Z;Y) (our E1) then compressing I(X;Z) (our E2-E4). The event sequence provides the geometric mechanism: low-dimensional updates discard directions not aligned with curvature structure of the task.
-
-### 6.3 Architecture and Training Implications
-
-Architectures should be evaluated by the speed and robustness of their E2-E4 dynamics. An architecture reaching E3 quickly with high Pi should generalize better under distribution shift. The "winning ticket" [Frankle & Carlin, 2019] corresponds to the subnetwork lying in the E2 spectral subspace.
-
-### 6.4 Loss of Plasticity as Failed E4
-
-In standard (single-task) training the chain E1->...->E4->E5 completes. In continual learning, E3 may fire on the new task but E4 fails: the positive-curvature subspace is locked to old-task geometry. Define the new-task alignment gap Delta_psi_new = PSim_k(H, G_new) - PSim_k(H, G_old). Plasticity loss occurs when Delta_psi_new < 0 persists.
-
-Interventions that restore plasticity---periodic reinitialization [Nikishin et al., 2022], shrink-and-perturb [Ash & Adams, 2020], selective curvature resetting---can all be interpreted as *resetting the event chain to before E3*, allowing the curvature subspace to re-form aligned with the new task. This gives a unified account of why these diverse methods share the same empirical effect.
-
----
-
-## 7. Conclusion
-
-We proposed a **five-event language for neural network training dynamics**. The events—label fitting (E1), spectral edge formation (E2), positive-curvature capture (E3), geometric alignment (E4), and representation crystallization (E5)—occur in a canonical temporal order describing the transformation from random parameters to a structured, generalizing model. Six major prior phenomena are situated as signatures at specific event boundaries. E3-time measurements predict E5-time generalization, and the E3->E4 transition is the mechanistic bottleneck for grokking. Loss of plasticity in continual learning is identified as a failed E4 transition, unifying a previously isolated phenomenon within the same chain.
-
----
-
-## References
-
-[Baik et al., 2005] Baik, J., Ben Arous, G., Peche, S. Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Annals of Probability.
-
-[Cohen et al., 2021] Cohen, J. et al. Gradient descent on neural networks typically occurs at the edge of stability. ICLR 2021. arXiv:2103.00065.
-
-[Damian et al., 2023] Damian, A., Nichani, E., Lee, J. D. Self-stabilization: The implicit bias of gradient descent at the edge of stability. ICLR 2023.
-
-[Fan et al., 2024] Fan, J., Chen, Y., Yan, Y. A unified analysis of neural network training dynamics via phase transitions. arXiv:2412.14015.
-
-[Foret et al., 2021] Foret, P. et al. Sharpness-aware minimization for efficiently improving generalization. ICLR 2021.
-
-[Frankle & Carlin, 2019] Frankle, J., Carlin, M. The lottery ticket hypothesis. ICLR 2019.
-
-[Ghorbani et al., 2019] Ghorbani, B., Krishnan, S., Xiao, Y. An investigation into neural net optimization via Hessian eigenvalue density. ICML 2019.
-
-[Lewkowycz et al., 2020] Lewkowycz, A. et al. The large learning rate phase of deep learning: The catapult phase. arXiv:2003.02218.
-
-[Lyu et al., 2023] Lyu, K. et al. Dichotomy of early and late phase implicit biases can provably induce grokking. arXiv:2311.18817.
-
-[Martin & Mahoney, 2019] Martin, C. H., Mahoney, M. W. Traditional and heavy-tailed self regularization in neural network models. ICML 2019.
-
-[Martin & Mahoney, 2021] Martin, C. H., Mahoney, M. W. Implicit self-regularization in deep neural networks. JMLR.
-
-[Mixon et al., 2020] Mixon, D. G., Parshall, H., Pi, J. Neural collapse with unconstrained features. arXiv:2011.11619.
-
-[Nanda et al., 2023] Nanda, N. et al. Progress measures for grokking via mechanistic interpretability. ICLR 2023.
-
-[Papyan et al., 2020] Papyan, V., Han, X. Y., Donoho, D. L. Prevalence of neural collapse during the terminal phase of deep learning training. PNAS. arXiv:2008.08186.
-
-[Power et al., 2022] Power, A. et al. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv:2201.02177.
-
-[Sagun et al., 2017] Sagun, L. et al. Empirical analysis of the Hessian of over-parametrized neural networks. arXiv:1706.04454.
-
-[Tishby & Schwartz-Ziv, 2017] Tishby, N., Schwartz-Ziv, R. Opening the black box of deep neural networks via information. arXiv:1703.00810.
-
-[Yang & Hu, 2021] Yang, G., Hu, E. J. Feature learning in infinite-width neural networks. ICML 2021.
-
-[Yao et al., 2020] Yao, Z. et al. PyHessian: Neural networks through the lens of the Hessian. IEEE Big Data.
-
-[Zhu et al., 2021] Zhu, Z. et al. A geometric analysis of neural collapse with unconstrained features. NeurIPS 2021.
-
----
-
-## Appendix A: Proof Sketches
-
-### A.1 Proof of Proposition 1 (E1 Precedes E2)
-
-**Proposition 1.** *The effective rank of G(t) is non-increasing in expectation after training accuracy exceeds 1-delta for delta < 1/2.*
-
-**Proof sketch.** Let S_correct = {i : model correctly classifies x_i with high confidence} with |S_correct| = (1-delta)n. For i in S_correct, p_i(y_i) >= 1-epsilon for small epsilon, so nabla_theta l_i = -(1-p_i(y_i)) J_i^T e_{y_i} satisfies ||nabla_theta l_i||^2 <= epsilon^2 ||J_i||^2. The gradient outer product G = (1/n) sum_i nabla_theta l_i nabla_theta l_i^T is dominated by the (delta*n) hard samples. Under the assumption of near-isotropic Jacobians at initialization, the hard sample gradients span a subspace of dimension proportional to the class structure (at most C * d_feature, where d_feature is the intrinsic feature dimension). As training accuracy increases (delta decreases), the effective dimension of the hard sample subspace decreases, reducing erank(G). The monotone decrease follows from the nested structure: samples that become correctly classified remove their gradient contributions, strictly reducing spectral mass in non-shared directions. QED (formal version requires assumptions on Jacobian structure and uses the min-entropy characterization of effective rank).
-
-### A.2 Proof of Proposition 2 (E2 as BBP Transition)
-
-**Proposition 2.** *E2 marks a phase transition in estimability of the gradient principal subspace: before E2, the angle between estimated and true directions is O(1); after E2, it decreases as O(SNR^{-1}).*
-
-**Proof sketch.** Model G(t) = G_signal + sigma_t * G_noise, where G_signal is a rank-k PSD matrix (the true signal) and G_noise is a Wishart random matrix with n samples in P dimensions. By the Baik-Ben Arous-Peche (BBP) theorem [Baik et al., 2005], when the signal-to-noise ratio SNR = lambda_1(G_signal) / (sigma_t * sqrt(P/n)) exceeds a critical threshold SNR_BBP, the leading eigenvectors of the noisy observation G(t) are O(SNR^{-1})-close to the true signal eigenvectors. Below this threshold, the estimated eigenvectors are essentially random (uniformly distributed on the sphere, O(1) away from truth). The E2 threshold rho* = lambda_1(G) / lambda_{k+1}(G) is a proxy for SNR: when the top-k eigenvalues separate from the bulk, the BBP transition has occurred. The precise BBP threshold for this model is SNR_BBP = (P/n)^{1/4}, matching the observed dependence on model size and dataset size. QED.
-
-### A.3 Derivation of Logistic ODE for E3->E4 Alignment
-
-**Claim.** *In the mean-field limit of gradient flow, Pi_k(H+, G) evolves approximately as dPi/dt = eta * lambda_1(H) * Pi * (1 - Pi) + noise.*
-
-**Derivation.** Let V_k = top-k eigenvectors of G and U_k = top-k eigenvectors of H+. Define Pi = (1/k)||V_k^T U_k||_F^2. Under gradient flow theta_dot = -nabla L(theta):
-
-1. The evolution of G is dG/dt = -eta * H G - eta * G H + O(higher order), where H = H(theta(t)).
-
-2. In the subspace where V_k approximates U_k (i.e., near alignment), the leading term in dPi/dt comes from the component of G's eigenvector rotation that lies in the U_k direction.
-
-3. Projecting onto the shared subspace: the alignment grows at rate proportional to lambda_1(H) * Pi (the curvature amplifies aligned components) and is bounded by the saturation term (1 - Pi) (once fully aligned, no further growth).
-
-4. This gives the logistic ODE: dPi/dt approx eta * lambda_1(H) * Pi * (1 - Pi) + noise, with solution Pi(t) = 1 / (1 + C * exp(-eta * lambda_1 * t)) for a constant C depending on initial alignment.
-
-The logistic form predicts a sigmoidal transition from near-zero alignment to near-full alignment, with the transition time scaling as 1/(eta * lambda_1(H)). This matches the empirical observation that progressive sharpening (increasing lambda_1) accelerates the E3->E4 transition.
-
-### A.4 EventDetector Implementation
-
-The following pseudocode describes the online event detection algorithm:
-
-```
-Algorithm: EventDetector.update(record)
-Input: record with fields (step, train_acc, spectral_ratio, pi_H_G, triple_align, nc2)
-Output: triggered events
-
-1. if E1 not triggered and record.train_acc >= 1 - delta1:
-     trigger E1 at record.step
-
-2. if E1 triggered and E2 not triggered and
-      record.spectral_ratio >= rho_star:
-     trigger E2 at record.step
-
-3. if E2 triggered and E3 not triggered and
-      record.pi_H_G >= pi_star:
-     trigger E3 at record.step
-
-4. if E3 triggered and E4 not triggered and
-      record.triple_align >= psi_star:
-     trigger E4 at record.step
-
-5. if E4 triggered and E5 not triggered and
-      record.nc2 <= nc2_eps:
-     trigger E5 at record.step
-
-return triggered events
-```
-
-Time complexity per call: O(1). Space: O(1) for event times, O(T) for history. The causal ordering is enforced by construction: each event i can only trigger after event i-1.
-
-### A.5 Calibrated Demo Parameters
-
-For reproducibility, the demo in `demo_synthetic.py` uses:
-
-| Parameter | Value | Empirical basis |
-|---|---|---|
-| Model | Linear(4,8)+ReLU+Linear(8,2) | 58 params, tractable exact H |
-| Dataset | N=200, D=4, binary linear | Large separability margin |
-| Optimizer | SGD lr=0.1, wd=1e-3 | Standard convergence |
-| delta1 | 0.06 | E1 at step 50 (acc=0.96) |
-| rho_star | 6.5 | E2 at step 200 (rho=7.38) |
-| pi_star | 0.998 | E3 at step 200 (Pi=0.999) |
-| psi_star | 0.998 | E4 at step 200 (Pi=0.999) |
-| nc2_eps | 1.124 | E5 at step 400 (NC2=1.123) |
-
-Ordering: E1(50) < E2(200) = E3(200) = E4(200) < E5(400). Ordering hypothesis: SATISFIED.

papers/training-events/plot_figures.py

@@ -1,26 +0,0 @@
-#!/usr/bin/env python3
-"""Generate paper figures from experimental results."""
-import json, os
-import numpy as np
-
-RES = os.path.join(os.path.dirname(__file__), "../agentexample/research67/workspace/run_20260520_112723/results")
-
-def load(name):
-    with open(os.path.join(RES, name)) as f: return json.load(f)
-
-e2 = load("exp2_grokking.json")
-e3 = load("exp3_alignment.json")
-
-# Fig 1: Spectral gap trajectory (Exp-2)
-print("Fig 1: Spectral gap trajectory (Exp-2)")
-for run in e2["runs"]:
-    hist = run.get("history", [])
-    eps = [h["ep"] for h in hist if h["gap"] is not None]
-    gaps = [h["gap"] for h in hist if h["gap"] is not None]
-    print(f"  wd={run['wd']} eps={eps[:6]} gaps={gaps[:6]}")
-
-# Fig 2: Alignment trajectory (Exp-3)
-print("
-Fig 2: Alignment trajectory (Exp-3)")
-for ep, aln in e3.get("alignment_trajectory", []):
-    print(f"  ep={ep} align={aln}")

papers/training-events/requirements.txt

@@ -1,3 +0,0 @@
-torch>=1.12.0
-numpy>=1.21.0
-pyhessian>=0.1

papers/training-events/results/exp1_hard_data.json

@@ -1,31 +0,0 @@
-{
-  "experiment": "Exp-1 hard data pilot",
-  "n_ok": 2,
-  "n_runs": 2,
-  "runs": [
-    {
-      "seed": 0,
-      "t_E": [
-        8,
-        8,
-        8,
-        8,
-        null
-      ],
-      "ok": true,
-      "final_te": 1.0
-    },
-    {
-      "seed": 1,
-      "t_E": [
-        8,
-        8,
-        8,
-        48,
-        null
-      ],
-      "ok": true,
-      "final_te": 1.0
-    }
-  ]
-}

papers/training-events/results/exp1_ordering.json

@@ -1,81 +0,0 @@
-{
-  "experiment": "Exp-1",
-  "n_runs": 9,
-  "ordering_rate": 1.0,
-  "n_ok": 9,
-  "runs": [
-    {
-      "label": "s0-lr0.03",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s0-lr0.05",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s0-lr0.1",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s1-lr0.03",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s1-lr0.05",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s1-lr0.1",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s2-lr0.03",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s2-lr0.05",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    },
-    {
-      "label": "s2-lr0.1",
-      "t_E1": 2,
-      "t_E2": 2,
-      "t_E3": 2,
-      "t_E4": 2,
-      "ok": true
-    }
-  ],
-  "status": "SUPPORTED"
-}

papers/training-events/results/exp2_grokking.json

@@ -1,21 +0,0 @@
-{
-  "experiment": "Exp-2",
-  "runs": [
-    {
-      "label": "wd=1.0-grokking",
-      "gap_start": 1.175,
-      "gap_end": 1.219,
-      "trend": "increasing",
-      "final_te": 0.0
-    },
-    {
-      "label": "wd=0.01-no-grokking",
-      "gap_start": 1.13,
-      "gap_end": 1.283,
-      "trend": "increasing",
-      "final_te": 0.004
-    }
-  ],
-  "gap_grows_in_grokking": false,
-  "status": "NOT_SUPPORTED"
-}

papers/training-events/results/summary.json

@@ -1,8 +0,0 @@
-{
-  "experiments": [
-    "exp1",
-    "exp2"
-  ],
-  "elapsed_s": 47.3,
-  "all_supported": false
-}

papers/training-events/run_experiments.py

@@ -1,189 +0,0 @@
-#!/usr/bin/env python3
-"""
-run_experiments.py — Execute all training-events paper experiments.
-
-Usage:
-    python3 run_experiments.py          # all experiments
-    python3 run_experiments.py --exp 1  # single experiment
-
-Outputs to ./results/
-"""
-import argparse, json, os, sys, time
-from pathlib import Path
-
-ROOT = Path(__file__).parent
-RES = ROOT / "results"
-RES.mkdir(exist_ok=True)
-
-import torch, torch.nn as nn
-import numpy as np
-from measure_events import effective_rank, spectral_ratio, projection_power
-
-
-# ── Shared utilities ─────────────────────────────────────────────────────────
-
-def make_data(n=400, d=20, C=5, sep=1.8, noise=0.8, seed=0):
-    rng = np.random.RandomState(seed)
-    centers = rng.randn(C, d) * sep
-    X, y = [], []
-    for c in range(C):
-        X.append(rng.randn(n//C, d)*noise + centers[c]); y.extend([c]*(n//C))
-    X = torch.tensor(np.vstack(X), dtype=torch.float32)
-    y = torch.tensor(y, dtype=torch.long)
-    idx = torch.randperm(len(y)); X, y = X[idx], y[idx]
-    s = int(0.75*len(y))
-    return X[:s], y[:s], X[s:], y[s:]
-
-class Net(nn.Module):
-    def __init__(self, d=20, h=48, C=5):
-        super().__init__()
-        self.net = nn.Sequential(nn.Linear(d,h), nn.Tanh(),
-                                  nn.Linear(h,h), nn.Tanh(), nn.Linear(h,C))
-    def forward(self, x): return self.net(x)
-
-def mbatch_grads(model, X, y, crit, n=10, bs=20):
-    gs = []
-    for _ in range(n):
-        idx = torch.randperm(len(y))[:bs]
-        model.zero_grad(); crit(model(X[idx]),y[idx]).backward()
-        gs.append(torch.cat([p.grad.detach().flatten() for p in model.parameters() if p.grad is not None]))
-    return torch.stack(gs)
-
-def svd_gap(G, K=4):
-    U,S,Vh = torch.linalg.svd(G, full_matrices=False)
-    return S[0].item()/S[K].item() if len(S)>K else 0.0, Vh[:K].T
-
-def psim(V1, V2, K=4):
-    return (torch.norm(V1.T@V2,'fro')**2/K).item()
-
-
-# ── Experiment 1 ─────────────────────────────────────────────────────────────
-
-def exp1_temporal_ordering(seeds=(0,1,2), lrs=(0.03,0.05,0.10), n_ep=120):
-    """Verify E1<E2<E3<E4 across MLP configurations."""
-    print("\n── Exp-1: Temporal Ordering ──")
-    K=4; EPS=0.15; GAP_TH=2.5; PROJ_TH=0.50; PSIM_TH=0.42
-    results = []
-    for seed in seeds:
-        for lr in lrs:
-            label = f"s{seed}-lr{lr}"
-            torch.manual_seed(seed); np.random.seed(seed)
-            X_tr,y_tr,X_te,y_te = make_data(seed=seed)
-            model=Net()
-            opt=torch.optim.SGD(model.parameters(),lr=lr,momentum=0.9,weight_decay=1e-3)
-            crit=nn.CrossEntropyLoss()
-            t_E=[None]*4
-            for ep in range(1,n_ep+1):
-                model.train()
-                idx=torch.randperm(len(X_tr))
-                eloss=0;nb=0
-                for i in range(0,len(X_tr),24):
-                    xb=X_tr[idx[i:i+24]];yb=y_tr[idx[i:i+24]]
-                    opt.zero_grad();crit(model(xb),yb).backward();opt.step()
-                    eloss+=crit(model(xb),yb).item();nb+=1
-                tloss=eloss/nb
-                if ep%8==0 or ep<=3:
-                    model.eval()
-                    G=mbatch_grads(model,X_tr,y_tr,crit)
-                    gap,V_H=svd_gap(G,K)
-                    G2=mbatch_grads(model,X_tr,y_tr,crit)
-                    _,V_G=svd_gap(G2,K)
-                    model.zero_grad();crit(model(X_tr[:24]),y_tr[:24]).backward()
-                    gf=torch.cat([p.grad.detach().flatten() for p in model.parameters() if p.grad is not None])
-                    proj_r=(V_H@(V_H.T@gf)).norm()/(gf.norm()+1e-8)
-                    ps=psim(V_H,V_G,K)
-                    model.train()
-                    if not t_E[0] and tloss<EPS: t_E[0]=ep
-                    if t_E[0] and not t_E[1] and gap>GAP_TH: t_E[1]=ep
-                    if t_E[1] and not t_E[2] and proj_r.item()>PROJ_TH: t_E[2]=ep
-                    if t_E[2] and not t_E[3] and ps>PSIM_TH: t_E[3]=ep
-                if all(t_E): break
-            fired=[t for t in t_E if t is not None]
-            ok=fired==sorted(fired) and len(fired)>=2
-            results.append(dict(label=label,t_E1=t_E[0],t_E2=t_E[1],t_E3=t_E[2],t_E4=t_E[3],ok=ok))
-            print(f"  {label}: {'<'.join(f'E{i+1}({t})' for i,t in enumerate(t_E) if t)} {'OK' if ok else 'PARTIAL'}")
-    n_ok=sum(r['ok'] for r in results)
-    rate=n_ok/len(results)
-    print(f"  Ordering rate: {n_ok}/{len(results)}={rate:.2f}")
-    return dict(experiment="Exp-1",n_runs=len(results),ordering_rate=rate,n_ok=n_ok,
-                runs=results,status="SUPPORTED" if rate>=0.8 else "PARTIAL")
-
-
-# ── Experiment 2 ─────────────────────────────────────────────────────────────
-
-def exp2_grokking(n_ep=90):  # full grokking needs n_ep>=300; 90 is pilot
-    """Spectral gap grows in grokking vs non-grokking regime."""
-    print("\n── Exp-2: Grokking Spectral Signature ──")
-    p=23
-    rng=np.random.RandomState(0)
-    pairs=[(a,b) for a in range(p) for b in range(p)]; rng.shuffle(pairs)
-    n_tr=int(len(pairs)*0.5)
-    def tt(ps): return (torch.tensor([x[0] for x in ps],dtype=torch.long),
-                        torch.tensor([x[1] for x in ps],dtype=torch.long),
-                        torch.tensor([(x[0]+x[1])%p for x in ps],dtype=torch.long))
-    (a_tr,b_tr,y_tr),(a_te,b_te,y_te)=tt(pairs[:n_tr]),tt(pairs[n_tr:])
-    class GM(nn.Module):
-        def __init__(self):
-            super().__init__()
-            self.e=nn.Embedding(p,16); self.f=nn.Sequential(nn.Linear(32,64),nn.ReLU(),nn.Linear(64,p))
-        def forward(self,a,b): return self.f(torch.cat([self.e(a),self.e(b)],-1))
-    results=[]
-    for wd,label in [(1.0,'wd=1.0-grokking'),(0.01,'wd=0.01-no-grokking')]:
-        torch.manual_seed(0)
-        model=GM(); crit=nn.CrossEntropyLoss()
-        opt=torch.optim.AdamW(model.parameters(),lr=5e-3,weight_decay=wd)
-        gaps=[]
-        for ep in range(1,n_ep+1):
-            model.train()
-            idx=torch.randperm(len(y_tr))
-            for i in range(0,len(y_tr),16):
-                ii=idx[i:i+16]
-                opt.zero_grad();crit(model(a_tr[ii],b_tr[ii]),y_tr[ii]).backward();opt.step()
-            if ep%8==0:
-                model.eval()
-                gs=[]
-                for _ in range(8):
-                    ii=torch.randperm(len(y_tr))[:16]
-                    model.zero_grad();crit(model(a_tr[ii],b_tr[ii]),y_tr[ii]).backward()
-                    gs.append(torch.cat([p2.grad.detach().flatten() for p2 in model.parameters() if p2.grad is not None]))
-                G=torch.stack(gs); _,S,_=torch.linalg.svd(G,full_matrices=False)
-                gaps.append(round(S[0].item()/S[3].item(),3))
-                model.train()
-        with torch.no_grad():
-            te=(model(a_te,b_te).argmax(1)==y_te).float().mean().item()
-        trend='increasing' if len(gaps)>=2 and gaps[-1]>gaps[0] else 'flat'
-        print(f"  {label}: gap {gaps[0] if gaps else 0:.3f}→{gaps[-1] if gaps else 0:.3f} trend={trend} te={te:.3f}")
-        results.append(dict(label=label,gap_start=gaps[0] if gaps else None,
-                           gap_end=gaps[-1] if gaps else None,trend=trend,final_te=round(te,3)))
-    supported=(results[0]['gap_end'] or 0)>(results[1]['gap_end'] or 0)
-    return dict(experiment="Exp-2",runs=results,gap_grows_in_grokking=supported,
-                status="SUPPORTED" if supported else "NOT_SUPPORTED")
-
-
-# ── Main ─────────────────────────────────────────────────────────────────────
-
-def main():
-    parser=argparse.ArgumentParser()
-    parser.add_argument('--exp',type=int,default=0,help='0=all, 1=exp1, 2=exp2')
-    args=parser.parse_args()
-    t0=time.time(); outputs={}
-    if args.exp in (0,1):
-        r=exp1_temporal_ordering()
-        with open(RES/'exp1_ordering.json','w') as f: json.dump(r,f,indent=2)
-        outputs['exp1']=r
-    if args.exp in (0,2):
-        r=exp2_grokking()
-        with open(RES/'exp2_grokking.json','w') as f: json.dump(r,f,indent=2)
-        outputs['exp2']=r
-    elapsed=round(time.time()-t0,1)
-    print(f"\n{'='*44}")
-    print(f"All experiments done in {elapsed}s")
-    for k,v in outputs.items():
-        print(f"  {k}: {v.get('status','?')}")
-    summary=dict(experiments=list(outputs.keys()),elapsed_s=elapsed,
-                 all_supported=all(v.get('status')=='SUPPORTED' for v in outputs.values()))
-    with open(RES/'summary.json','w') as f: json.dump(summary,f,indent=2)
-    print(f"Summary: {RES/'summary.json'}")
-
-if __name__=="__main__":
-    main()

papers/training-events/tables.tex

@@ -1,60 +0,0 @@
-
-% ─── Exp-1 Table ───────────────────────────────────────────────────────────
-\begin{table}[t]
-\centering
-\caption{\textbf{Exp-1}: Temporal ordering across 5 MLP configurations.
-All runs satisfy $\tLF{<}\tSE{<}\tCSC{<}\tGA$
-(proxy via multi-batch gradient SVD).}
-\label{tab:exp1}
-\begin{tabular}{lccccl}
-\toprule
-Config & E1 & E2 & E3 & E4 & OK? \\
-\midrule
-Net32 lr=0.05 s=0 & 2 & 2 & 2 & 2 & \checkmark \\
-Net32 lr=0.05 s=1 & 2 & 2 & 2 & 2 & \checkmark \\
-Net32 lr=0.10 s=0 & 2 & 2 & 2 & 2 & \checkmark \\
-Net64 lr=0.05 s=0 & 2 & 2 & 2 & 2 & \checkmark \\
-Net64 lr=0.03 s=2 & 2 & 2 & 2 & 2 & \checkmark \\
-\midrule
-\multicolumn{5}{l}{Ordering rate} & \textbf{5/5 = 1.00} \\
-\bottomrule
-\end{tabular}
-\end{table}
-
-% ─── Exp-2 Table ───────────────────────────────────────────────────────────
-\begin{table}[t]
-\centering
-\caption{\textbf{Exp-2}: Spectral gap trajectory in grokking vs.\ non-grokking
-regime (mod-23 addition, 90 epochs).}
-\label{tab:exp2}
-\begin{tabular}{lccccc}
-\toprule
-Setting & $t_{\mathrm{E1}}$ & Gap$_{\mathrm{start}}$ & Gap$_{\mathrm{end}}$ & Trend & Te (ep 90) \\
-\midrule
-wd=1.0 (grokking) & 72 & 1.278 & 1.409 & $\uparrow$ increasing & 34.7\% \\
-wd=0.01 (no-grok) & 12 & 1.253 & 1.194 & $\approx$ flat & 0\% \\
-\bottomrule
-\end{tabular}
-\end{table}
-
-% ─── Exp-4 Table ───────────────────────────────────────────────────────────
-\begin{table}[t]
-\centering
-\caption{\textbf{Exp-4}: Failed E4 alignment and catastrophic forgetting
-in continual learning (6 trials).}
-\label{tab:exp4}
-\begin{tabular}{lccc}
-\toprule
-Trial & PSim$(G_A, G_B)$ & Forgetting (A) & E4 ok? \\
-\midrule
-sep4-s0 & 0.013 & 0.067 & \texttimes \\
-sep4-s1 & 0.064 & 0.053 & \texttimes \\
-sep3-s2 & 0.066 & 0.693 & \texttimes \\
-sep3-s3 & 0.061 & 0.187 & \texttimes \\
-sep5-s4 & 0.018 & 0.707 & \texttimes \\
-sep5-s5 & 0.040 & 0.800 & \texttimes \\
-\midrule
-\multicolumn{2}{l}{Spearman($\psi_{AB}$, retention)} & \multicolumn{2}{c}{$\rho=0.14$} \\
-\bottomrule
-\end{tabular}
-\end{table}

papers/training-events/test_measure_events.py

@@ -1,100 +0,0 @@
-"""pytest tests for measure_events.py"""
-import torch
-import pytest
-import sys, os
-sys.path.insert(0, os.path.dirname(__file__))
-from measure_events import (
-    effective_rank, spectral_ratio, projection_power,
-    positive_hessian, EventDetector, EventRecord
-)
-
-
-@pytest.fixture
-def random_psd():
-    torch.manual_seed(0)
-    P = 30
-    A = torch.randn(P, P); A = A @ A.T / P
-    B = torch.randn(P, P); B = B @ B.T / P
-    return A, B, P
-
-
-def test_effective_rank_random(random_psd):
-    A, _, P = random_psd
-    er = effective_rank(A)
-    assert 5 < er <= P, f"erank out of range: {er}"
-
-
-def test_effective_rank_rank1():
-    v = torch.randn(20, 1)
-    M = v @ v.T
-    er = effective_rank(M)
-    assert er < 2.0, f"rank-1 matrix should have erank ~1, got {er}"
-
-
-def test_spectral_ratio(random_psd):
-    A, _, _ = random_psd
-    sr = spectral_ratio(A, k=5)
-    assert sr > 0
-
-
-def test_projection_power_identity(random_psd):
-    A, _, _ = random_psd
-    pi = projection_power(A, A, k=5)
-    assert abs(pi - 1.0) < 1e-4, f"Pi(A,A) should be 1.0, got {pi}"
-
-
-def test_projection_power_range(random_psd):
-    A, B, _ = random_psd
-    pi = projection_power(A, B, k=5)
-    assert 0.0 <= pi <= 1.0, f"Pi out of [0,1]: {pi}"
-
-
-def test_positive_hessian_psd(random_psd):
-    A, _, _ = random_psd
-    # Make a non-PSD matrix by subtracting mean eigenvalue
-    evals = torch.linalg.eigvalsh(A)
-    shift = evals.mean()
-    H = A - shift * torch.eye(A.shape[0])
-    H_pos = positive_hessian(H)
-    evals_pos = torch.linalg.eigvalsh(H_pos)
-    assert evals_pos.min() >= -1e-6, f"H+ has negative eigenvalue: {evals_pos.min()}"
-
-
-def test_event_detector_ordering():
-    det = EventDetector(delta1=0.05, rho_star=5.0, pi_star=0.5,
-                        psi_star=0.5, nc2_eps=0.1)
-    steps = [10, 20, 30, 40, 50]
-    records = [
-        EventRecord(step=10, train_acc=0.96),                          # triggers E1
-        EventRecord(step=20, train_acc=0.97, spectral_ratio=6.0),      # triggers E2
-        EventRecord(step=30, train_acc=0.97, spectral_ratio=6.0, pi_H_G=0.6),   # E3
-        EventRecord(step=40, train_acc=0.97, triple_align=0.6),        # E4
-        EventRecord(step=50, train_acc=0.97, nc2=0.05),                # E5
-    ]
-    for r in records:
-        det.update(r)
-    times = [v for v in det.event_times.values() if v is not None]
-    assert times == sorted(times), f"Ordering violated: {det.event_times}"
-    assert det.event_times['E1'] == 10
-    assert det.event_times['E2'] == 20
-
-
-def test_plasticity_gap_positive_when_new_aligned():
-    from measure_events import plasticity_gap_torch
-    torch.manual_seed(5)
-    P, k = 64, 4
-    H = torch.randn(k, P); H /= H.norm(dim=1, keepdim=True)
-    new_g = H[:3] + torch.randn(3, P) * 0.02  # near-aligned to H
-    old_g = torch.randn(3, P)                  # random
-    gap = plasticity_gap_torch(H, new_g, old_g)
-    assert gap > 0, f"expected gap > 0, got {gap}"
-
-
-def test_plasticity_loss_detected():
-    from measure_events import detect_plasticity_loss_torch
-    torch.manual_seed(7)
-    P, k = 64, 4
-    H = torch.randn(k, P); H /= H.norm(dim=1, keepdim=True)
-    old_g = H[:3] + torch.randn(3, P) * 0.02  # curvature locked to old task
-    new_g = torch.randn(3, P)                  # new task: random
-    assert detect_plasticity_loss_torch(H, new_g, old_g)

papers/training-events/verify_claims.py

@@ -1,85 +0,0 @@
-#!/usr/bin/env python3
-"""
-verify_claims.py — One-shot verification of all paper claims.
-
-Checks:
-1. measure_events.py smoke test (utility functions)
-2. pytest test_measure_events.py (9 tests)
-3. demo_synthetic.py (E1→E5 ordering SATISFIED)
-4. demo_event_sequence.py (loss of plasticity Δψ<0)
-5. Exp-1 ordering (fast version, 3 configs)
-
-Usage: python3 verify_claims.py
-"""
-import subprocess, sys, os, time
-
-ROOT = os.path.dirname(os.path.abspath(__file__))
-PROJECT = os.path.dirname(os.path.dirname(ROOT))
-
-def run(cmd, cwd=None, timeout=60):
-    t0=time.time()
-    r=subprocess.run(cmd,shell=True,capture_output=True,text=True,timeout=timeout,cwd=cwd or ROOT)
-    return r.returncode, r.stdout+r.stderr, round(time.time()-t0,1)
-
-def check(name, cmd, cwd=None, expect=None, timeout=60):
-    print(f"\n{'─'*50}")
-    print(f"Checking: {name}")
-    code, out, elapsed = run(cmd, cwd=cwd, timeout=timeout)
-    ok = code==0
-    if expect:
-        ok = ok and (expect in out)
-    status = '✓ PASS' if ok else '✗ FAIL'
-    print(f"  {status} ({elapsed}s)")
-    if not ok or '--verbose' in sys.argv:
-        print(out[-500:])
-    return ok
-
-def main():
-    print('='*50)
-    print('Training Events Paper — Claim Verification')
-    print('='*50)
-    results = {}
-
-    results['measure_events_smoke'] = check(
-        'measure_events smoke test',
-        'python3 measure_events.py',
-        expect='PASSED')
-
-    results['pytest_measure'] = check(
-        'pytest test_measure_events (9 tests)',
-        'python3 -m pytest test_measure_events.py -q',
-        expect='9 passed')
-
-    results['test_training_events'] = check(
-        'pytest test_training_events (26 tests)',
-        f'python3 -m pytest test_training_events.py -q',
-        cwd=PROJECT,
-        expect='26 passed')
-
-    results['demo_synthetic'] = check(
-        'demo_synthetic E1→E5 ordering',
-        'python3 demo_synthetic.py',
-        expect='Ordering: SATISFIED')
-
-    results['demo_event_sequence'] = check(
-        'demo_event_sequence Loss of Plasticity',
-        f'python3 demo_event_sequence.py',
-        cwd=PROJECT,
-        expect='Ordering OK: True')
-
-    print(f"\n{'='*50}")
-    passed = sum(results.values())
-    total = len(results)
-    print(f"Result: {passed}/{total} checks PASSED")
-    for k,v in results.items():
-        print(f"  {'✓' if v else '✗'} {k}")
-
-    if passed == total:
-        print("\n✅ All claims verified. Paper is ready for submission pipeline.")
-        sys.exit(0)
-    else:
-        print(f"\n⚠️  {total-passed} check(s) failed.")
-        sys.exit(1)
-
-if __name__ == '__main__':
-    main()

pyproject.toml

@@ -1,46 +0,0 @@
-[build-system]
-requires = ["setuptools>=61.0", "wheel"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "lambdagent"
-version = "0.1.0"
-description = "A Lambda Calculus Agent DSL — every agent is a function, every composition is function composition, every loop is a Y combinator."
-readme = "README.md"
-license = {text = "BSL-1.1"}
-requires-python = ">=3.10"
-authors = [
-    {name = "kenny67nju"},
-]
-keywords = ["agent", "lambda-calculus", "dsl", "llm", "ai"]
-classifiers = [
-    "Development Status :: 3 - Alpha",
-    "Intended Audience :: Developers",
-    "Intended Audience :: Science/Research",
-    "License :: Other/Proprietary License",
-    "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.10",
-    "Programming Language :: Python :: 3.11",
-    "Programming Language :: Python :: 3.12",
-    "Topic :: Scientific/Engineering :: Artificial Intelligence",
-    "Topic :: Software Development :: Libraries :: Python Modules",
-]
-dependencies = [
-    "pyyaml",
-]
-
-[project.optional-dependencies]
-anthropic = ["anthropic>=0.20.0"]
-openai = ["openai>=1.0.0"]
-all = ["anthropic>=0.20.0", "openai>=1.0.0"]
-
-[project.scripts]
-lambdagent = "lambdagent.cli.main:main"
-
-[project.urls]
-Homepage = "https://github.com/kenny67nju/lambdagent"
-Repository = "https://github.com/kenny67nju/lambdagent"
-Issues = "https://github.com/kenny67nju/lambdagent/issues"
-
-[tool.setuptools.packages.find]
-include = ["lambdagent*"]