The problem

When architecturally distinct models reach similar predictions, do they learn the same abstractions — or do similarity metrics just flatter architectural priors? Trained CKA heatmaps look convergent, but a similarity score means nothing without knowing where randomly-initialized networks already sit.

Technical approach

Eight model families trained from scratch under matched conditions: same dataset and splits, a 2–5M-parameter capacity band for the deep models, five seeds, frozen evaluation. Penultimate-layer representations probed with linear CKA against trained, untrained single-init, untrained multi-init, and cross-seed references, plus linear-probe transfer diagnostics — 40 frozen runs with full provenance.

Technical highlights

• The mirage, quantified: trained inter-architecture CKA mean 0.6853 vs an untrained-floor 0.6731 — training moves cross-architecture similarity by ~0.012.
• A genuine gap survives: deep models sit at CKA 0.11–0.13 against classical DSP features — a +0.51 to +0.61 novelty gap — while classical SVM-RBF still wins the task (0.399 macro-F1).
• Measurement discipline as the contribution: convergence claims need untrained floors, same-architecture ceilings, and feature-group ablations.

Why it matters here

For ML/AI research audiences: an interpretability result about the tools we use to make interpretability claims.