Pratinav Seth

← Publications / Safety Post-Training & Alignment

C-ΔΘ: Circuit-Restricted Weight Arithmetic for Selective Refusal

Aditya Kasliwal, Pratinav Seth, Vinay Kumar Sankarapu

Mech Interp Workshop, ICML 20262026My role: Senior authorWorkshop1 citation
PDFarXivScholar
C-ΔΘ

Abstract

Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity. Activation steering is widely used, but it requires runtime hooks and scales cost with the number of generations; conditional variants improve selectivity by gating when steering is applied but still retain an inference-time control path. We ask whether selective refusal can be moved entirely offline: can a mechanistic understanding of category-specific refusal be distilled into a circuit-restricted weight update that deploys as a standard checkpoint? We propose C-ΔΘ (Circuit-Restricted Weight Arithmetic), which (i) localizes refusal-causal computation as a sparse circuit using EAP-IG and (ii) computes a constrained weight update ΔΘ_C supported only on that circuit (typically <5% of parameters). Applying ΔΘ_C yields a drop-in edited checkpoint with no inference-time hooks, shifting cost from per request intervention to a one-time offline update. We evaluate category-targeted selectivity and capability retention on refusal and utility benchmarks.

Accepted at the Mechanistic Interpretability Workshop, ICML 2026 (in-person poster; <15% acceptance rate)

Cite

@inproceedings{kasliwal2026circuitrestricted,
  title         = {{C-ΔΘ: Circuit-Restricted Weight Arithmetic for Selective Refusal}},
  author        = {Aditya Kasliwal and Pratinav Seth and Vinay Kumar Sankarapu},
  booktitle     = {Mech Interp Workshop, ICML 2026},
  year          = {2026},
  eprint        = {2602.04521},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2602.04521}
}
All →