Data that drives
the research.
Curated evaluation datasets for LLM safety research. Published openly on Hugging Face — used by PICK for refusal ablation benchmarking, available for the wider research community.
The control
group.
A curated set of benign, non-harmful prompts used to establish the model's baseline refusal behavior and verify that capability is preserved after ablation. These prompts span everyday conversation, factual questions, coding tasks, and creative writing — all designed to elicit normal, helpful responses. Used as the reference distribution for KL divergence computation.
The test
set.
A curated set of 800 harmful prompts spanning multiple categories, designed to test model refusal behavior. Each prompt is crafted to elicit refusal from safety-aligned models. Used as the primary evaluation metric for PICK: measure refusal rate before and after ablation. The goal is not to make models comply — it's to measure precisely how much refusal behavior has been surgically removed.
Run the model on safe_prompt to capture the reference output distribution. This establishes the "normal" behavior — the distribution we want to preserve. Used as the target for KL divergence minimization during component selection.
Run the model on unsafe_prompt before and after ablation. Count refusals. The refusal reduction rate — how many prompts that were previously refused now receive a direct answer — is the primary evaluation metric for PICK.