digantamisra98 / evonorm Goto Github PK
View Code? Open in Web Editor NEWUnofficial PyTorch Implementation of EvoNorm
Home Page: https://arxiv.org/abs/2004.02967
License: MIT License
Unofficial PyTorch Implementation of EvoNorm
Home Page: https://arxiv.org/abs/2004.02967
License: MIT License
How to judge which evonorm sample2d or evonorm batch2d to use
Hi, thank you for the nice implementation. I've got a question about how you use the EvoNorm S0 in the BasicBlock (your block definition):
self.evo = EvoNorm2D(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = norm_layer(planes)
What would be the norm_layer in this case? My understanding of the original paper is that the norm_lyaer(planes) has to actually be EvoNorm2D(planes, non_linear=False). At least that is what I understood from Figure 5's capture and a footnote there (see the original paper).
What is your take on that?
def __init__(self, input, non_linear = True, version = 'S0', efficien affine = True,
There is a space between efficien and affine. Not sure what should be written instead (erase "efficien" ?)
Hi,
to make easy use of the layer, I noticed that it can help to add a groups
argument to the initialization of the layer. This groups argument can then be given to the group_std
calculation to adjust for the amount of groups that one wants to define.
Otherwise you are stuck with the default value of 32
.
Kind regards
Implement and example could refer to impl and train code.
I didn't struggle with training settings btw.
Hi, i have read your code,but i want to know whether evo_norm could be used for sequence-data, for example, the dims in data just likes batch_size, seq_len, embedding_dims
.
In other words, N, C, H, W = x.size()
, does C
correspond to embedding_dims
?
作者你好,我在使用你的代码训练模型的时候,发现内存一直在涨,请问你是否发现有内存泄漏的问题?
I noticed that the first conv layer of ResNet is followed by BatchNorm Layer, I think it should also be followed by EvoNorm as well.
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) self.bn1 = nn.BatchNorm2d(64) self.relu = nn.ReLU(inplace=True)
I am using this on my own computer vision dataset. My image size is small enough that when the image size ends up being (batch_size,n_channels,1,1), the EvoNorm begins returning NaN's. I know the reason is because the width=channel=1 because when I make my network smaller with less convolution layers i.e. width,channel>1, the NaN's go away. Is there any reason why this is the case?
Maybe this is my misunderstanding. But it looks like evonorm2d.py line 55 which is using self.eps, but there is no self.eps definition at init function.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.