hypnopump / e-swish Goto Github PK
View Code? Open in Web Editor NEWCode for reproducibility of the E-swish paper experiments
Code for reproducibility of the E-swish paper experiments
It was published in Arxiv before your paper, so it should be cited and commented, in my opinion. They have found (or "found") through a search, the swish function with a beta factor inside the sigmoid, whereas you add one outside.
As far as I can see, for unconstrained weights a beta outside the sigmoid does exactly the same as increasing all the weights from the node, so the network will be able to represent exactly the same functions as pure swish (except the last layer may have no weights out). And a beta inside the sigmoid is equivalent to changing all the weights into the node (except the first layer may have no weights into it).
So basically, the beta parameters only affect the learning process, and will obviously interact with other learning parameters/choices and regularization. (Using SGD instead of Adam for a comparison based on another paper counts as such a choice.)
Please enlighten me if I am wrong.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.