haythamfayek / matdl Goto Github PK

Mish is a new novel activation function proposed in this paper.
It has shown promising results so far and has been adopted in several packages including:

• TensorFlow-Addons
• SpaCy (Tok2Vec Layer)
• Thinc - SpaCy's official NLP based ML library
• Echo AI
• CNTKX - Extension of Microsoft's CNTK
• FastAI-Dev
• Darknet
• Yolov3
• BeeDNN - Library in C++
• Gen-EfficientNet-PyTorch
• dnet

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10:

Per JOSS requirements, please add "clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support" to the README

One slightly annoying thing about the CNN demo is that I don't know what format the MNIST data is in (mnist_uint8.mat). It would be nice if you could add either the data file itself (hosted somewhere) or a script that downloads MNIST data from the web and converts it into this mat file format.

(would just make it easier to quickly run your demo)