Empty predictive sets about conformal_classification HOT 3 CLOSED

Hi Linda,

Thank you for pointing this out! Sets of size zero are a mathematical oddity sometimes needed to achieve exact coverage (e.g. in the case we picked a coverage level lower than the model's accuracy). This is expected behavior, but I forgot to note it in the documentation.

For an immediate fix, you can just set randomized=False. Then sets of size zero shouldn't happen anymore, although the coverage will be slightly conservative.

In the longer term (over the next few days) I'll try to put another flag called "allow-zero-sets" or something that just never allows sets of size zero. This should be less conservative than randomized=False.

Let me know if this fixed your problem :)

Best,
Anastasios

P.S. I'll leave this issue open until I add the new flag, but for now, randomized=False should give you the desired behavior.

from conformal_classification.

Hi Anastasios,

thanks for your quick answer, explanation and suggestion! This fixes my problem for now, I'll use randomized=False until you've introduced the new flag.

Best,
Linda

from conformal_classification.

Hi Linda,
Apologies for the wait! You can see the new flag in example.py. :)
Best,
Anastasios

from conformal_classification.