I downloaded models via the PyTorch Hub API and exported ONNX file well on a few weeks ago.
But now, I cannot do that.

The geffnet 3edcd4d, which was commited a week ago, uses SiLU activation with PyTorch 1.7.0, but recent ONNX opset 12 hasn't supported SiLU yet.
Therefore torch.onnx.export() of PyTorch 1.7.0 results in following error.

import torch

model = torch.hub.load(
    'rwightman/gen-efficientnet-pytorch', 'tf_efficientnet_b0',
    pretrained=True, exportable=True, verbose=False)
model.eval()
dummy_input = torch.randn(1,3,224,224)
model(dummy_input)
traced_model = torch.jit.trace(model, dummy_input)
torch_out = torch.onnx.export(
    model, dummy_input, 'tf_efficientnet_b0.onnx',
    export_params=True, verbose=False,opset_version=12)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/__init__.py", line 230, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 91, in export
    use_external_data_format=use_external_data_format)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 639, in _export
    dynamic_axes=dynamic_axes)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 421, in _model_to_graph
    dynamic_axes=dynamic_axes, input_names=input_names)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 203, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/__init__.py", line 263, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 930, in _run_symbolic_function
    symbolic_fn = _find_symbolic_in_registry(domain, op_name, opset_version, operator_export_type)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 888, in _find_symbolic_in_registry
    return sym_registry.get_registered_op(op_name, domain, opset_version)
  File "/home/k-tanabe/.local/lib/python3.6/site-packages/torch/onnx/symbolic_registry.py", line 111, in get_registered_op
    raise RuntimeError(msg)
RuntimeError: Exporting the operator silu to ONNX opset version 12 is not supported. Please open a bug to request ONNX export support for the missing operator

You have updated also geffnet/version.py from 1.0.0 to 1.0.1 in 3edcd4d.
Could you tag v1.0.0 on the previous revision e84f554, which supports ONNX file export with PyTorch 1.7.0.
Then I can down load v1.0.0 with a following command.

model = torch.hub.load(
    'rwightman/gen-efficientnet-pytorch:v1.0.0', 'tf_efficientnet_b0',
    pretrained=True, exportable=True, verbose=False)

Hi, i want to use efficientnet_pytorch.from_name(...) function in gen-efficientnet-pytorch.

from efficientnet_pytorch import EfficientNet
import torch
import torch.nn as nn

class LyftModel(nn.Module):
    def __init__(self, cfg: Dict):
        super(LyftModel, self).__init__()

        self.backbone = EfficientNet.from_name("efficientnet-b1")
        num_history_channels = (cfg["model_params"]["history_num_frames"] + 1) * 2
        num_in_channels = 3 + num_history_channels
        num_targets = 2 * cfg["model_params"]["future_num_frames"]
        self.backbone._conv_stem = nn.Conv2d(
            num_in_channels,
            self.backbone._conv_stem.out_channels,
            kernel_size=self.backbone._conv_stem.kernel_size,
            stride=self.backbone._conv_stem.stride,
            padding=self.backbone._conv_stem.padding,
            bias=False,
        )
        self.avg_pool = nn.AdaptiveAvgPool2d((1, 1))
        self.fea_bn = nn.BatchNorm1d(1280)
        self.fea_bn.bias.requires_grad_(False)
        self.binary_head = nn.Linear(1280, num_targets)
        self.dropout = nn.Dropout(p=0.2)
 

    def forward(self, x):

        img_feature = self.backbone.extract_features(x)
        img_feature = self.avg_pool(img_feature)
        img_feature = img_feature.view(img_feature.size(0), -1)
        fea = self.fea_bn(img_feature)
        fea = self.dropout(fea)
        output = self.binary_head(fea)

        return output

Thanks in advance.

I tested many mobilenetv3 models and got very low accuracy. What is the problem?

Hi Ross,

First of all, thank you for the awesome repo, it has been wonderful to use this and pytorch-image-models so far. I just have a couple of questions about ONNX export.

1. I (mistakenly) trained my EfficientNet model on the models found in pytorch-image-models and then realised that to export it to ONNX I needed to have
   a) used gen-efficientnet-pytorch and
   b) set the geffnet.config.set_exportable(True) beforehand as you have stated in your export script.

Is there any way I could perhaps still transfer the weights from a pytorch-image-models version of EfficientNet to an equivalent gen-efficientnet-pytorch version AND at the same time make it compatible with ONNX?

2. Is there any chance you could also add ONNX export somehow to the pytorch-image-models versions of EfficientNet? My code base immensely prefers to work with the attributes and methods available for models in that repository compared with the models in geffnet

3. Seconding the suggestion in issue #32 and would love to have the option for that for pytorch-image-models as well.

Thank you

https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet

Bringing this issue over from: tensorflow/tpu#738

I am also confused. The starting value for the gamma coefficient (resolution scaling) is 1.15 and the baseline image resolution is 224x224. So it makes sense that 1.15⁰ = 1 would be the scaling factor for B0 as 224x1 = 224. EfficientNetB1 would therefore be 1.15¹ = 1.15 and make the resolution for B1 224x1.15 = 257.6, but in the code it shows the B1 resolution is 240, and B2 is 260. I am also confused with the other coefficients as in the code it has:

params_dict = {

(width_coefficient, depth_coefficient, resolution, dropout_rate)

'efficientnet-b0': (1.0, 1.0, 224, 0.2),
'efficientnet-b1': (1.0, 1.1, 240, 0.2),
'efficientnet-b2': (1.1, 1.2, 260, 0.3),
'efficientnet-b3': (1.2, 1.4, 300, 0.3),
'efficientnet-b4': (1.4, 1.8, 380, 0.4),
'efficientnet-b5': (1.6, 2.2, 456, 0.4),
'efficientnet-b6': (1.8, 2.6, 528, 0.5),
'efficientnet-b7': (2.0, 3.1, 600, 0.5),
'efficientnet-b8': (2.2, 3.6, 672, 0.5),
'efficientnet-l2': (4.3, 5.3, 800, 0.5),
}

why is β (width) coefficient for B1 = 1.0? Shouldn't it be 1.1¹ = 1.1? And α (depth) coefficient is 1.1 when in the paper it has α as = 1.2.

Just hoping someone in this community could enlighten me

I export efficientnet_b0 onnx and set the 640x640 size , it will happen the error
'''
import onnx
import onnx_tensorrt.backend as backend
import numpy as np
model = onnx.load("efficientnet_b0.onnx")
engine = backend.prepare(model, device='CUDA:0')
input_data = np.random.random(size=(1, 3, 640, 640)).astype(np.float32)
output_data = engine.run(input_data)[0]
print(output_data)
print(output_data.shape)
'''

[Error]
[TensorRT] ERROR: Parameter check failed at: ../builder/Network.cpp::addPoolingNd::500, condition: allDimsGtEq(windowSize, 1) && volume(windowSize) < MAX_KERNEL_DIMS_PRODUCT
Traceback (most recent call last):
File "test_onnx.py", line 7, in
engine = backend.prepare(model, device='CUDA:0')
File "/opt/conda/lib/python3.6/site-packages/onnx_tensorrt-0.1.0-py3.6-linux-x86_64.egg/onnx_tensorrt/backend.py", line 218, in prepare
return TensorRTBackendRep(model, device, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/onnx_tensorrt-0.1.0-py3.6-linux-x86_64.egg/onnx_tensorrt/backend.py", line 94, in init
raise RuntimeError(msg)
RuntimeError: While parsing node number 8:
builtin_op_importers.cpp:1175 In function importGlobalAveragePool:
[8] Assertion failed: layer_ptr

@rwightman hi, thanks for your work firstly, it's a great repo.
I want to get the pretrained model for gen-efficientent_lite0, but the url is 'none'. I found a link ''tf_efficientnet_lite0'', is it converted from tensorflow checkpoint? And, what does ' v0.1-weights' means? Can I download it directly and use it normally？

Thanks for your sharing of so many pretrained models.

I am using your EfficientNet-b0 to train on CIFAR-100 using pytorch.My best result was only about 60% acc. I have tried so many hyperparameters and tricks,but still fail to work well.So I have to ask for your kind help.

Here is my settings:
I used torch.hub to load your pretrained model,and changed out features of last classifier layer to 100.My initial lr was 0.001 used SGD to optimize and reducelronplateau lr scheduler.

It would be great to receive your reply.Have a good day!

Have you tried to use half precision or apex to train a model from scratch? Is it converging?

No state_dict found in checkpoint file efficientnet_b1-533bc792.pth?

Can you complement the README.md with exact instructions on how to import Tensorflow checkpoints of EfficientNet models? I train EfficientNetB0-B4 models on TPUs now and it would be nice to import into your framework for evaluation. I love pytorch-image-models, it is very handy and I would like to compare the shipped tf_efficientnet_bX models with my new models.

Thanks!

Hello. I want to experiment with efficientnets - accelerate them further by replacing some convolution layers with other layers using trnsor decompositions. Is it better for me to use this repo or the other one? What is the difference between Efficientnet models in the two repos?

only want to use mobilenetv3 model，example mobilenetv3.py is only model，no creating model and load pretrain.Could you give me some advice?

Hi @rwightman,
Thanks first for this awesome repo. I'm trying to use your impl to get the AP pre-trained B5, but it's quite clearly failing to load the pretrained weights though with neither an error nor a confirm the weights were loaded. Is this a known issue or am I doing something wrong? (edit - ok I re-read the readme and think I misunderstood that AP implemented meant with pretrained weights available...anyway, if so then passing preTrained=True where no weights exist should ideally print a warning or error?)

1 - Installed via pip install geffnet
2 - Import geffnet
3 - model = geffnet.create_model('efficientnet_b5',num_classes = data.c,pretrained=True, drop_rate=0.2, drop_connect_rate=0.2)#, as_sequential=True)
Normally I'm used to seeing a "loading .pth and the progress bar here on a new instance, or a confirmation of weights loaded. I did not see either but no error either.
4 - When you go to train it becomes abundantly clear that it's working with a new init network (i.e. first epoch close to random, then verrry slow training progress. By contrast a pre-trained digs right in.

If possible, it would be great to get a confirmation message like in Melas impl once weights are loaded: "Loaded pretrained weights for efficientnet-b5" or if not a warning that it's a new network if pretrained=True was passed in?

Thanks much!
Less

Hey @rwightman ,

I tried to change the number of classes for my finetuning project with 'tf_efficientnet_b4_ap' and for some reason, when I change the number of classes, i get the following output:
=> Discarding pretrained classifier since num_classes != 1000
but I think it shouldn't be discarded for num_classes != 1000 because it should be solved by something as simple as
model = geffnet.create_model(modelname, pretrained=True) model.classifier(nn.Sequential(nn.Linear(1792, num_classes)))
If this cannot be done, could you please explain why?
I would honestly like to run tf_efficientnet instead of regular ones as they do provide better results.

Also, the adv_prop normalization could adapt to other normalizations, right?

Thanks

Hi there

Is there a recommended amount of GPU memory needed to train EfficientNets b0 to l2? Currently have another implementation of EfficientNet-b8 but running into memory issues on a 24GB GPU. Will I have the same issues with this implementation?

TIA

I tested tf_efficientnet accuracy with ImageNet 2012 _VAL.

And that is result
b0 = 0.699
b1 = 0.717
b2 = 0.721
b3 = 0.724
b4 = 0.717
b5 = 0.619
b6 = 0.694
b7 = 0.523

Why b4b7 was get lower accuracy than b0b3?
I think that weight has problem.
Can you check that's weight?

When I use this code

model = geffnet.create_model(
'efficientnet_b5',
num_classes=1000,
in_chans=3,
pretrained=True,
exportable=True)

But this error occured.

efficientnet b0~b3 => okay

efficientnet b4~b7 => URL is empty.

How to solved this problem?

It seems like the paper here

New top-1 high! Ported official TF EfficientNet AdvProp (https://arxiv.org/abs/1911.08609) weights and B8 model spec.

should be this one https://arxiv.org/abs/1911.09665

Hi @rwightman!

Thank you for your great implementation for efficientnet and its relatives.

I found the global param is _SCRIPTABLE and _EXPORTABLE in the config file. However, when I use torch hub to access the model, it is hard for me to acces these two variables and successful export the model. In this case I'm wondering is that possible to set _SCRIPTABLE and _EXPORTABLE as a parameter of EfficientNetBuilder?

Best,

After updating pytorch 1.3.0 -> 1.3.1
I got the following error

torch.hub.list('rwightman/gen-efficientnet-pytorch')

ImportError: cannot import name 'mobilenetv3_100' from 'geffnet' (/home/mario/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master/geffnet/__init__.py)

what is the validate dataset and how to organize the dir path tree?

I want use m = geffnet.create_model('mobilenetv3_large_100', pretrained=True) to replace
model = torch.hub.load('rwightman/gen-efficientnet-pytorch', 'efficientnet_b0', pretrained=True)
BTW, I use the pre-trained model aim to Fine-tuning the model.

Am i doing it right？

Thank you very much!
@rwightman

Is there a way around or a solution?

C:\Users\Kedar/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master\geffnet\conv2d_layers.py:39: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return max((math.ceil(i / s) - 1) * s + (k - 1) * d + 1 - i, 0)
C:\Users\Kedar/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master\geffnet\conv2d_layers.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return max((math.ceil(i / s) - 1) * s + (k - 1) * d + 1 - i, 0)
C:\Users\Kedar/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master\geffnet\conv2d_layers.py:63: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if pad_h > 0 or pad_w > 0:
Traceback (most recent call last):
File "", line 4, in
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx_init_.py", line 148, in export
strip_doc_string, dynamic_axes, keep_initializers_as_inputs)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\utils.py", line 66, in export
dynamic_axes=dynamic_axes, keep_initializers_as_inputs=keep_initializers_as_inputs)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\utils.py", line 416, in _export
fixed_batch_size=fixed_batch_size)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\utils.py", line 296, in _model_to_graph
fixed_batch_size=fixed_batch_size, params_dict=params_dict)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\utils.py", line 135, in _optimize_graph
graph = torch._C.jit_pass_onnx(graph, operator_export_type)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx_init.py", line 179, in _run_symbolic_function
return utils._run_symbolic_function(*args, **kwargs)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\utils.py", line 657, in _run_symbolic_function
return op_fn(g, *inputs, **attrs)
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\symbolic_helper.py", line 128, in wrapper
args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\symbolic_helper.py", line 128, in
args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
File "C:\Users\Kedar.conda\envs\conv\lib\site-packages\torch\onnx\symbolic_helper.py", line 81, in _parse_arg
"', since it's not constant, please try to make "
RuntimeError: Failed to export an ONNX attribute 'onnx::Div', since it's not constant, please try to make things (e.g., kernel size) static if possible

When I import geffnet in Google Colab.
I got some error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-1dbdac5fd1e4> in <module>()
----> 1 import geffnet

3 frames
/usr/local/lib/python3.6/dist-packages/geffnet/layers.py in versiontuple(v)
      5 
      6 def versiontuple(v):
----> 7     return tuple(map(int, (v.split("."))))[:3]
      8 
      9 

ValueError: invalid literal for int() with base 10: '0+cu100'

I think it is because the torch.version string is '1.3.0+cu100' in Google Colab.
Maybe we can use some regular expression to parse this??

> /usr/local/lib/python3.6/dist-packages/geffnet/layers.py(7)versiontuple()
      5 
      6 def versiontuple(v):
----> 7     return tuple(map(int, (v.split("."))))[:3]
      8 
      9 

ipdb> v
'1.3.0+cu100'
ipdb> v.split(".")
['1', '3', '0+cu100']

Traceback (most recent call last):
File "train.py", line 138, in
main()
File "train.py", line 116, in main
cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/builder.py", line 43, in build_detector
return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/builder.py", line 15, in build
return build_from_cfg(cfg, registry, default_args)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/utils/registry.py", line 76, in build_from_cfg
return obj_cls(**args)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/detectors/retinanet.py", line 16, in init
test_cfg, pretrained)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/detectors/single_stage.py", line 31, in init
self.init_weights(pretrained=pretrained)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/detectors/single_stage.py", line 35, in init_weights
self.backbone.init_weights(pretrained=pretrained)
File "/home/hs/hao/contextnet/mmdetection-master/mmdet/models/backbones/mixnet.py", line 837, in init_weights
load_checkpoint(self, pretrained, strict=False, logger=logger)
File "/home/hs/anaconda3/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 181, in load_checkpoint
'No state_dict found in checkpoint file {}'.format(filename))
RuntimeError: No state_dict found in checkpoint file /home/hs/hao/contextnet/mmdetection-master/weights/mixnet/mixnet_m-4647fc68.pth

Hi, thanks for the great work. Following my question here , I tried to convert to ONNX using this repo. But I got several errors.

By inputting this command just like your example, I got segmentation fault error :

sudo python3 onnx_export.py --model mobilenetv3_100 ./mobilenetv3_100.onnx
==> Creating PyTorch mobilenetv3_100 model
==> Exporting model to ONNX format at './mobilenetv3_100.onnx'
==> Loading and checking exported model from './mobilenetv3_100.onnx'
Segmentation fault

When I tried with efficientnet_b0 using checkpoint and not using checkpoint
sudo python3 onnx_export.py --model efficientnet_b0 ./efficientnet.onnx
or
sudo python3 onnx_export.py --model efficientnet_b0 --checkpoint ../train/model_best.pth.tar --num-classes 30 ./efficientnet.onnx

I got Couldn't export Python operator SwishAutoFn error.

==> Creating PyTorch efficientnet_b0 model
=> Loading checkpoint '../train/20191015-Deepeye36k-efficientnet_b0-224/model_best.pth.tar'
=> Loaded checkpoint '../train/20191015-Deepeye36k-efficientnet_b0-224/model_best.pth.tar'
==> Exporting model to ONNX format at './efficientnet.onnx'
Traceback (most recent call last):
  File "onnx_export.py", line 75, in <module>
    main()
  File "onnx_export.py", line 59, in main
    input_names=input_names, output_names=output_names)
  File "/home/ivan/.local/lib/python3.6/site-packages/torch/onnx/__init__.py", line 26, in _export
    result = utils._export(*args, **kwargs)
  File "/home/ivan/.local/lib/python3.6/site-packages/torch/onnx/utils.py", line 394, in _export
    operator_export_type, strip_doc_string, val_keep_init_as_ip)
RuntimeError: ONNX export failed: Couldn't export Python operator SwishAutoFn

Any help would be appreciated, thanks

When run python3 onnx_export.py --model tf_efficientnet_b6_ns ./tf_efficientnet_b6_ns.onnx, I got this error:
==> Loading and checking exported model from './tf_efficientnet_b6_ns.onnx'
Segmentation fault (core dumped)
how to fix this?

Hi @rwightman

Good morning. I am impressed by your work.

I try to export your model to onnx as following:

x = torch.rand(1, 3, 352, 1216)
torch.onnx.export(net, x, "./sqnet_3566.onnx")
print("export onnx successfully!")

But I encountered the following error message:

Using cache found in /home/paul/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master
/home/paul/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master/geffnet/conv2d_layers.py:47: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  return max((-(i // -s) - 1) * s + (k - 1) * d + 1 - i, 0)
/home/paul/pytorch/AdaBins/models/layers.py:19: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  embeddings = embeddings + self.positional_encodings[:embeddings.shape[2], :].T.unsqueeze(0)
Traceback (most recent call last):
  File "rk3566Test.py", line 273, in <module>
    export_pytorch_model()
  File "rk3566Test.py", line 55, in export_pytorch_model
    torch.onnx.export(net, x, "./sqnet_3566.onnx")
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/__init__.py", line 208, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/utils.py", line 92, in export
    use_external_data_format=use_external_data_format)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/utils.py", line 530, in _export
    fixed_batch_size=fixed_batch_size)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/utils.py", line 384, in _model_to_graph
    fixed_batch_size=fixed_batch_size, params_dict=params_dict)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/utils.py", line 188, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/__init__.py", line 241, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/utils.py", line 791, in _run_symbolic_function
    return symbolic_fn(g, *inputs, **attrs)
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in wrapper
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in <listcomp>
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/home/paul/rknn2/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 81, in _parse_arg
    "', since it's not constant, please try to make "
RuntimeError: Failed to export an ONNX attribute 'onnx::Cast', since it's not constant, please try to make things (e.g., kernel size) static if possible

I did change the _EXPORTABLE flag to True in config.py as following:

# Set to True if exporting a model with Same padding via ONNX
_EXPORTABLE = True

Is this the right way to do it? or I miss something else? Could you please help? Thanks a lot.

from geffnet.mobilenetv3 import mobilenetv3_rw, mobilenetv3_small_075
net = mobilenetv3_small_075()

########################modify###################
# self.global_pool = nn.AdaptiveAvgPool2d(1)
# self.conv_head = select_conv2d(in_chs, num_features, 1, padding=pad_type, bias=head_bias)
# self.act2 = act_layer(inplace=True)
# self.classifier = nn.Linear(num_features, num_classes)
def as_sequential(self):
layers = [self.conv_stem, self.bn1, self.act1]
layers.extend(self.blocks)
# layers.extend([
# self.global_pool, self.conv_head, self.act2,
# nn.Flatten(), nn.Dropout(self.drop_rate), self.classifier])
return nn.Sequential(*layers)

def features(self, x):
    x = self.conv_stem(x)
    x = self.bn1(x)
    x = self.act1(x)
    x = self.blocks(x)
    # x = self.global_pool(x)
    # x = self.conv_head(x)
    # x = self.act2(x)
    return x
def forward(self, x):
    x = self.features(x)
    # x = x.flatten(1)
    if self.drop_rate > 0.:
        x = F.dropout(x, p=self.drop_rate, training=self.training)
    return x

I'm not sure if this model modification method is correct.
For complex models, I'm falling into the quagmire , and I hope to get some suggestions.

Did you test new Efficient-Net-B3-ap weights on ImageNet-val? I suppouse it doesn't give the same accuracy as paper...

Hi Ross,

Thanks for making this easily accessible repository !

I apologize in advance I'm opening an issue to ask about usage, but it'll pass a bit more until geffnet will have its own tag on stackoverflow 😊

I am interested in using pretrained models with a new dataset. The problem is that this new dataset is not yet in a good shape: images have different sizes, so when I try to load an ImageDataBunch I get warnings about this. I could apply size parameter, but I wanted to ensure that all the transformations are consistent with the model.

Is there a way to get the transforms needed for a model? I have seen the data folder, together with create_loader, but it is not clear how to use this with ImageDataBunch.

Any advice would be much appreciated !

Hey @rwightman, I need to use MobileNetv3-large pretrained model for my backbone but couldn't find the pretrained model in your repo! is still tf_mobilenetv3_large_100 available for download?

Thanks!

@rwightman Hi,

What is the simplest way to convert trained Pytorch-weights of EfficientNet-Lite model back to Tensorflow?
If I want to train EfficientNet-Lite model in Pytorch and then use it in TensorFlow (pb/tflite)?

I have done it by using https://github.com/onnx/onnx-tensorflow but it adds extra transpose layers and use multiple conv2d-layers instead of one GroupedConv2D, so Pytorch->ONNX->TensorFlow-model is 10x times slower than the native TensorFlow model: onnx/onnx-tensorflow#782

mobilenetv3_small_075 is still not available?

Hi,

I have tried to reproduce mnasnet.

Could you let me know how to set the training parameters?

or Can I get the same results with your reported ones by using your default setting in your training code?

Tx in advance.

dd

Hi, I successfully get ncnn int8 model of efficientnet_b0 followed by your code:
pytorch --> onnx --> ncnn --> ncnn int8

However, when I do it on tf_mobilenetv3_small_075 model, I got some error in the last step:
./ncnn2table --param tf_mobilenetv3_small_075-sim.param --bin tf_mobilenetv3_small_075-sim.bin --images imgs/ --output tf_mobilenetv3_small_075-sim.table --mean 127.5,127.5,127.5 --norm 0.0078125,0.0078125,0.0078125 --size 224,224 --thread 2 (which succeed in efficientnet_b0 model)
--- ncnn post training quantization tool --- 07:49:29 Nov 7 2019
param = 'tf_mobilenetv3_small_075-sim.param'
bin = 'tf_mobilenetv3_small_075-sim.bin'
images = '/nfs/project/lijian_i/onnx-simplifier-master/imgs/'
output = 'tf_mobilenetv3_small_075-sim.table'
mean = '127.5,127.5,127.5'
norm = '0.0078125,0.0078125,0.0078125'
size = '224,224'
thread = '2'
====> Quantize the parameters.
====> Quantize the activation.
====> step 1 : find the max value.
[1] 2429 segmentation fault (core dumped) /home/luban/ncnn-master/build/tools/quantize/ncnn2table --param --bin

I was wondering is that input sized changed in TF ported weight or other issues?
Thanks for advance!

I load the efficientnet_b0 by model = torch.hub.load('rwightman/gen-efficientnet-pytorch', 'efficientnet_b0', pretrained=True)
then I convert the model to onnx. and the eval result is top1_acc=0.65032, top5_acc=0.8718

Hi again,

As per my previous issue, efficientnet_b0 weights exported to ONNX perfectly. But I went to try tf_efficientnet_lite2 today and I have encountered a similar issue to #34 . However the thing which is confusing me is that it works on Colab and not on my local machine with the a virtual environment with the exact same torch 1.5.0+cu101 packaged installed via pip (as opposed to conda for the sake of comparison) and exact same code. Could you please point me in the right direction to fix this problem? Thank you!

Here is the Colab notebook for reference: https://colab.research.google.com/drive/1jG7mSPitb-acA7EgMFaMsG3u57c51Bbe

And this what my local machine is telling me each time I run the exact same code.

(venv) andrewl@albona:/albona/nobackup/andrewl/test$ python3 test.py 
/pytorch/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.
/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/geffnet/conv2d_layers.py:39: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  return max((math.ceil(i / s) - 1) * s + (k - 1) * d + 1 - i, 0)
/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/geffnet/conv2d_layers.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  return max((math.ceil(i / s) - 1) * s + (k - 1) * d + 1 - i, 0)
Traceback (most recent call last):
  File "test.py", line 23, in <module>
    output_names=['output'],
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/__init__.py", line 168, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/utils.py", line 69, in export
    use_external_data_format=use_external_data_format)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/utils.py", line 488, in _export
    fixed_batch_size=fixed_batch_size)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/utils.py", line 351, in _model_to_graph
    fixed_batch_size=fixed_batch_size, params_dict=params_dict)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/utils.py", line 154, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/__init__.py", line 199, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/utils.py", line 740, in _run_symbolic_function
    return op_fn(g, *inputs, **attrs)
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in wrapper
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in <listcomp>
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/nb/andrewl/anaconda3/envs/venv/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 81, in _parse_arg
    "', since it's not constant, please try to make "
RuntimeError: Failed to export an ONNX attribute 'onnx::Cast', since it's not constant, please try to make things (e.g., kernel size) static if possible

I've also collected my system information for you here:

Collecting environment information...
PyTorch version: 1.5.0+cu101
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Debian GNU/Linux 10 (buster)
GCC version: (Debian 8.3.0-6) 8.3.0
CMake version: Could not collect

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.2.148
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 418.74
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.16.2
[conda] numpy                     1.18.3                   pypi_0    pypi
[conda] torch                     1.5.0+cu101              pypi_0    pypi
[conda] torchvision               0.6.0+cu101              pypi_0    pypi

And the information from the Colab notebook:

Collecting environment information...
PyTorch version: 1.5.0+cu101
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.12.0

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.18.3
[pip3] torch==1.5.0+cu101
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.6.0+cu101
[conda] Could not collect

How long did it took to train efficientnet_b3?

The model creation fails with current PyPI release of this library (0.9.8).

When installed from the sources

pip install git+https://github.com/rwightman/gen-efficientnet-pytorch.git

There isn't any error and model is being instantiated properly. Seems like some fixes are not yet published to PyPI

Is this Apache licensed?

Hi, thanks for the great repo! What's the reason that the x.mul_(inner) line is commented out? Does the commented line return different results? Thanks!

Hi,
I want to convert efficientnet_b0.pth to efficientnet_b0.onnx, when I runpython onnx_export.py --model=efficientnet_b0 res.onnx --checkpoint=../efficientnet_b0.pth:

==> Creating PyTorch efficientnet_b0 model
=> Loading checkpoint '../efficientnet_b0.pth'
=> Loaded checkpoint '../efficientnet_b0.pth'
==> Exporting model to ONNX format at 'res.onnx'
==> Loading and checking exported model from 'res.onnx'
Segmentation fault (core dumped)

what should I do?

Hi,
Could you add a prerequisites section to your landing page?
Specifying supported operating systems, Python versions. PyTorch versions, CUDA support etc.?

Thanks.

When I try:
model = torch.hub.load('rwightman/gen-efficientnet-pytorch', 'efficientnet_b0', in_chans=1, num_features=16)

or

model = torch.hub.load('rwightman/gen-efficientnet-pytorch', 'efficientnet_b0', in_chans=1, num_features=16, pretrained=False)
I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\t-maserr\AppData\Local\Continuum\anaconda3\envs\python37\lib\site-packages\torch\hub.py", line 359, in load
    model = entry(*args, **kwargs)
  File "C:\Users\t-maserr/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master\geffnet\gen_efficientnet.py", line 636, in efficientnet_b0
    'efficientnet_b0', channel_multiplier=1.0, depth_multiplier=1.0, pretrained=pretrained, **kwargs)
  File "C:\Users\t-maserr/.cache\torch\hub\rwightman_gen-efficientnet-pytorch_master\geffnet\gen_efficientnet.py", line 430, in _gen_efficientnet
    **kwargs,
TypeError: type object got multiple values for keyword argument 'num_features

Hi,

Is it possible to use this efficientnet implementation as a feature extractor? If so how should it be done?

Thanks in advance

torch.hub.list('rwightman/gen-efficientnet-pytorch')
model = torch.hub.load('rwightman/gen-efficientnet-pytorch', 'efficientnet_b0', pretrained=True, num_classes=3)
scriptedmodel = torch.jit.script(model)

got:

=> Discarding pretrained classifier since num_classes != 1000
Traceback (most recent call last):
  File "./scripts/update_model.py", line 122, in <module>
    scriptedmodel = torch.jit.script(model)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/__init__.py", line 1516, in script
    return torch.jit._recursive.create_script_module(obj, torch.jit._recursive.infer_methods_to_compile)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/_recursive.py", line 318, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/_recursive.py", line 372, in create_script_module_impl
    script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/__init__.py", line 1900, in _construct
    init_fn(script_module)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/_recursive.py", line 353, in init_fn
    scripted = create_script_module_impl(orig_value, sub_concrete_type, infer_methods_to_compile)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/_recursive.py", line 376, in create_script_module_impl
    create_methods_from_stubs(concrete_type, stubs)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/_recursive.py", line 292, in create_methods_from_stubs
    concrete_type._create_methods(defs, rcbs, defaults)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/annotations.py", line 290, in try_ann_to_type
    torch.jit._recursive_compile_class(ann, loc)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/__init__.py", line 1359, in _recursive_compile_class
    _compile_and_register_class(obj, rcb, _qual_name)
  File "/home/security/.local/lib/python3.8/site-packages/torch/jit/__init__.py", line 1363, in _compile_and_register_class
    _jit_script_class_compile(qualified_name, ast, rcb)
RuntimeError: 
Tried to access nonexistent attribute or method 'saved_tensors' of type '__torch__.geffnet.activations.activations_me.SwishJitAutoFn'. Did you forget to initialize an attribute in __init__()?:
  File "/home/security/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master/geffnet/activations/activations_me.py", line 50
    @staticmethod
    def backward(ctx, grad_output):
        x = ctx.saved_tensors[0]
            ~~~~~~~~~~~~~~~~~ <--- HERE
        return swish_jit_bwd(x, grad_output)
'SwishJitAutoFn.backward' is being compiled since it was called from '__torch__.geffnet.activations.activations_me.SwishJitAutoFn'
  File "/home/security/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master/geffnet/activations/activations_me.py", line 63
    def forward(self, x):
        return SwishJitAutoFn.apply(x)
               ~~~~~~~~~~~~~~ <--- HERE
'__torch__.geffnet.activations.activations_me.SwishJitAutoFn' is being compiled since it was called from 'SwishMe.forward'
  File "/home/security/.cache/torch/hub/rwightman_gen-efficientnet-pytorch_master/geffnet/activations/activations_me.py", line 63
    def forward(self, x):
        return SwishJitAutoFn.apply(x)
               ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE