microsoft / zero-shot-scfoundation Goto Github PK

Hi,
Thank you for the great work! We wonder if you have plans to release all the data you have used in the preprint?

Quick question about Figure 6c (above). From the paper, it seems that the highly expressed genes are of rank 1.0 and the lowly expressed genes of rank 0.0. This would be in line with the results of scGPT — the model does well at predicting highly expressed genes and poorly at predicting lowly expressed ones.
Now, as far as I understand, in Geneformer, the highly expressed genes are at the front of the list (source) and in your code you eventually divide the rank by the maximum rank (source) so, wouldn't that mean that Geneformer does better at predicting lowly expressed genes and worse at predicting highly expressed ones?

Here are also two plots that I created independently that would confirm my assessment:

1. Here I'm taking the cross entropy with reduction='none' and then plot the mean cross entropy across a batch of cells. It suggests that the model is less confidence for higher ranked genes.
   

2. Here I'm plotting the sliding window accuracy and f1 score (50 genes at a time) from the top of the list to the bottom. It also suggests that the model does better for the highly expressed genes.
   

Can you confirm that you didn't alter the way Geneformer ranks genes in your assessment?

Hi kzkedzierska,

I've been exploring your paper and I'm really impressed by your work. Great stuff!

I found the pancreas dataset link in the paper (thanks for that!), but I'm hitting a bit of a wall trying to track down the other four datasets you mentioned. I've followed the references as best as I can, yet it seems like I’m missing something or maybe they're not available at the referenced locations.

Could you point me in the right direction to find those datasets? Any guidance or direct links would be super helpful.

Thanks a bunch for your time, and for the great contributions you're making to the field!

Cheers,
renly0313