Can CP work on non-exchangeable data?

The core anchor to understand the vanilla split CP is:

“Once particular calibration samples has been drawn (samples from the unknown residual / non-conformity score distribution), the ratio of the mass lying under the 1-alpha quantile of the cal_set follows Beta distribution.”

$mass = F(\hat{q})$

$value = F^{-1}(mass)$

Coverage is a mass statement, while the interval width is a value statement.

1. Rigorous definition

Let

[

S_1,\dots,S_n,S_{n+1}

]

be the calibration scores plus the test score. They are exchangeable if, for every permutation (\pi) of ({1,\dots,n+1}),

[

(S_1,\dots,S_n,S_{n+1})

\overset{d}{=}

(S_{\pi(1)},\dots,S_{\pi(n+1)}).

]

So the joint distribution must be invariant to reordering.

Equivalently: before seeing the values, no index is special. The test score is not systematically larger, smaller, more variable, or differently dependent than the calibration scores.