Can CP work on non-exchangeable data?
Let
[
S_1,\dots,S_n,S_{n+1}
]
be the calibration scores plus the test score. They are exchangeable if, for every permutation (\pi) of ({1,\dots,n+1}),
[
(S_1,\dots,S_n,S_{n+1})
\overset{d}{=}
(S_{\pi(1)},\dots,S_{\pi(n+1)}).
]
So the joint distribution must be invariant to reordering.
Equivalently: before seeing the values, no index is special. The test score is not systematically larger, smaller, more variable, or differently dependent than the calibration scores.
You usually do not prove exchangeability from the observed scores alone. You justify it from the data-generating process. Finite data can falsify or diagnose violations, but cannot certify full exchangeability.
The clean sufficient condition for split conformal is:
[