hfawaz / cd-diagram Goto Github PK

In the example on your README.md example, don't you mean that cfl3 and cfl5 were the best overall (with average ranks around 1.5 and 2, respectively)?

Hello,
first, thanks for the open code!
I was wondering whether there's a way to calculate critical distance value as is calculated in the original implementation in Orange (by Demšar), where the Nemenyi or Bonferroni-Dunn test was used. Since your implementation only includes adjusted p-values, I assume it is not possible, as Holm's method doesn't have a table of critical values? (The only option is to visually inspect the diagram if that even makes sense?)
Thanks!

Below is the result when I am trying to compare four different classifiers.

['CoEye' 'TSF' 'cBOSS' 'Mr-SEQL']
CoEye 5.0
Mr-SEQL 61.0
TSF 8.0
cBOSS 10.0
dtype: float64
CoEye 3.279570
TSF 2.559140
cBOSS 2.537634
Mr-SEQL 1.623656
dtype: float64
('CoEye', 'Mr-SEQL', 1.6071230220582003e-11, True)
('Mr-SEQL', 'TSF', 1.5905280411437273e-07, True)
('Mr-SEQL', 'cBOSS', 1.3264426790567074e-06, True)
('CoEye', 'TSF', 1.6515911035945563e-05, True)
('CoEye', 'cBOSS', 4.231130849948383e-05, True)
('TSF', 'cBOSS', 0.4035623277672452, False)
Index(['CoEye', 'TSF', 'cBOSS', 'Mr-SEQL'], dtype='object')
[1, 2]

From the above result, it is clear that classifiers TSF and cBOSS are not significantly different. But I am not getting any clique for them in the CD.
Below is the CD diagram obtained.

it will be more useful if Friedman Nemenyi Test is embedded instead of Wilcoxon_holm. Thank you

Hi Hassan,

I've used your code for creating a critical difference diagram and got the following output. Am I misunderstanding something or should that extra line between M4 winner and M4 runner-up not be there?

Is this library can also be used for Bonferroni–Dunn post-hoc test?

Hi Hassan,
First of all, thank you for sharing the code.

I've run your code using several combinations of results, for all of them, I obtained interpretable diagrams except for one (I attach the results csv used - see the compressed file).

The thing is that it doesn't make sense, since the clf3 is a way better than the rest of the classifier.

Thank you again for your work.

Best,
David.

example.zip

can we plot cd-diagram for regression models