Cophenetic Correlation about nimfa HOT 2 CLOSED

Dear Naarkhoo,

that is actually expected behaviour. Please see the paper by Brunet et al. on cophenetic correlation coefficient-based measure.

To compute the cophenetic correlation scores (coph_cor is the name of the method in Nimfa), one need the consensus matrix. Consensus matrix is the average connectivity matrix over many runs. The last thing is crucial here.

If you call model.estimate_rank method, the factorization is repeated several times (depending on n_run parameter), each run produces a distinct connectivity matrix and these are then used for estimating the consensus matrix.

If you call fit.summary() then the cophenetic measure is computed based on single current factorized model, which has only one connectivity matrix and the consensus matrix is then trivial: all of its entries are equal to 1. Notice that the entries in consensus matrix range from 0 to 1 and reflect the probability that samples i and j are clustered together. Entries equal to 1 indicate perfectly stable clustering and no varying among the runs (obviously, as we have only one factorized model there aren't any variations here). In short, fit.summary() returns perfect stability (cophenetic = 1) because we have access to matrix factors from a single run and based on that data alone the stability of clustering is perfect. That is, its estimation is useless and factorization should be repeated several times.

• See Nimfa's documentation on estimate_rank method at http://nimfa.biolab.si/nimfa.models.nmf.html. There is a note that estimate_rank tracks matrix factors because it needs them for computing cophentic correlation coefficient.
• See paper 'Metagenes and molecular pattern discovery using matrix factorization' by Brunet et al., which describes the method implemented in Nimfa for evaluating the stability of clustering using cophenetic correlation coefficient. Method of Brunet et al. is computed s the Pearson correlation of two distance matrices: the distance between samples induced by the consensus matrix, and the distance between samples induced by the linkage used in the reordering of of consensus matrix.

from nimfa.

Thanks, Now, It makes a lot of sense. I will look into those paper.

from nimfa.