gmalivenko / pytorch2keras Goto Github PK
View Code? Open in Web Editor NEWPyTorch to Keras model convertor
Home Page: https://pytorch2keras.readthedocs.io/en/latest/
License: MIT License
PyTorch to Keras model convertor
Home Page: https://pytorch2keras.readthedocs.io/en/latest/
License: MIT License
Hi, thanks for great job.
I found particular problem:
when converting module of a type nn.Conv2d(1,output_channel,3,stride=(1,1),bias=True),
where output_channel >1 and groups =1,
in layers.py it is assumed that we deal with depthwise separable convolution (line 100),
and I got the following error (I changed input image from 3x320x320 to 1x320x320 in mobilinet.py):
File "mobilinet.py", line 136, in
k_model = pytorch_to_keras(model, input_var, (1, 224, 224,), verbose=True)
File "C:\ProgramData\Miniconda3\lib\site-packages\pytorch2keras-0.1.11-py3.5.egg\pytorch2keras\converter.py", line 314, in pytorch_to_keras
names
File "C:\ProgramData\Miniconda3\lib\site-packages\pytorch2keras-0.1.11-py3.5.egg\pytorch2keras\layers.py", line 147, in convert_conv
layers[scope_name] = conv(layers[input_name])
File "C:\ProgramData\Miniconda3\lib\site-packages\keras\engine\base_layer.py", line 436, in call
self.set_weights(self._initial_weights)
File "C:\ProgramData\Miniconda3\lib\site-packages\keras\engine\base_layer.py", line 1058, in set_weights
'provided weight shape ' + str(w.shape))
ValueError: Layer weight shape (3, 3, 1, 1) not compatible with provided weight shape (3, 3, 320, 1)
I temporally fixed this by changing line 100 to:
if n_groups == in_channels and n_groups>1:
TypeError Traceback (most recent call last)
in ()
395 output = model(input_var)
396
--> 397 k_model = pytorch_to_keras(model, input_var, (3, 224, 224,), verbose=True)
398
399 pytorch_output = output.data.numpy()
~\Downloads\Compressed\pytorch2keras-master\pytorch2keras\converter.py in pytorch_to_keras(model, args, input_shapes, change_ordering, training, verbose, short_names)
96
97 # _optimize_trace(trace, False)
---> 98 trace.set_graph(_optimize_graph(trace.graph(), False))
99
100 if verbose:
~\Downloads\Compressed\pytorch2keras-master\pytorch2keras\converter.py in _optimize_graph(graph, aten)
41 torch._C._jit_pass_peephole(graph)
42 torch._C._jit_pass_lint(graph)
---> 43 graph = torch._C._jit_pass_onnx(graph, aten)
44 torch._C._jit_pass_lint(graph)
45 torch._C._jit_pass_onnx_peephole(graph)
TypeError: _jit_pass_onnx(): incompatible function arguments. The following argument types are supported:
1. (arg0: torch::jit::Graph, arg1: torch._C._onnx.OperatorExportTypes) -> torch::jit::Graph
Invoked with: graph(%0 : Float(1, 3, 224, 224)
%1 : Float(64, 3, 7, 7)
%2 : Float(64)
%3 : Float(64)
%4 : Float(64)
%5 : Float(64)
%6 : Float(64)
%7 : Long()
%8 : Float(16, 64, 1, 1)
%9 : Float(16)
%10 : Float(16)
%11 : Float(16)
%12 : Float(16)
%13 : Float(16)
%14 : Long()
%15 : Float(8, 16, 1, 1)
%16 : Float(8)
%17 : Float(8)
%18 : Float(8)
%19 : Float(8)
%20 : Float(8)
%21 : Long()
%22 : Float(16, 8, 1, 3)
%23 : Float(16)
%24 : Float(16)
%25 : Float(16)
%26 : Float(16)
%27 : Float(16)
%28 : Long()
%29 : Float(16, 16, 3, 1)
%30 : Float(16)
%31 : Float(16)
%32 : Float(16)
%33 : Float(16)
%34 : Float(16)
%35 : Long()
%36 : Float(32, 16, 1, 1)
%37 : Float(32)
%38 : Float(32)
%39 : Float(32)
%40 : Float(32)
%41 : Float(32)
%42 : Long()
%43 : Float(32, 64, 1, 1)
%44 : Float(32)
%45 : Float(32)
%46 : Float(32)
%47 : Float(32)
%48 : Float(32)
%49 : Long()
%50 : Float(16, 32, 1, 1)
%51 : Float(16)
%52 : Float(16)
%53 : Float(16)
%54 : Float(16)
%55 : Float(16)
%56 : Long()
%57 : Float(8, 16, 1, 1)
%58 : Float(8)
%59 : Float(8)
%60 : Float(8)
%61 : Float(8)
%62 : Float(8)
%63 : Long()
%64 : Float(16, 8, 1, 3)
%65 : Float(16)
%66 : Float(16)
%67 : Float(16)
%68 : Float(16)
%69 : Float(16)
%70 : Long()
%71 : Float(16, 16, 3, 1)
%72 : Float(16)
%73 : Float(16)
%74 : Float(16)
%75 : Float(16)
%76 : Float(16)
%77 : Long()
%78 : Float(32, 16, 1, 1)
%79 : Float(32)
%80 : Float(32)
%81 : Float(32)
%82 : Float(32)
%83 : Float(32)
%84 : Long()
%85 : Float(16, 32, 1, 1)
%86 : Float(16)
%87 : Float(16)
%88 : Float(16)
%89 : Float(16)
%90 : Float(16)
%91 : Long()
%92 : Float(8, 16, 1, 1)
%93 : Float(8)
%94 : Float(8)
%95 : Float(8)
%96 : Float(8)
%97 : Float(8)
%98 : Long()
%99 : Float(16, 8, 1, 3)
%100 : Float(16)
%101 : Float(16)
%102 : Float(16)
%103 : Float(16)
%104 : Float(16)
%105 : Long()
%106 : Float(16, 16, 3, 1)
%107 : Float(16)
%108 : Float(16)
%109 : Float(16)
%110 : Float(16)
%111 : Float(16)
%112 : Long()
%113 : Float(32, 16, 1, 1)
%114 : Float(32)
%115 : Float(32)
%116 : Float(32)
%117 : Float(32)
%118 : Float(32)
%119 : Long()
%120 : Float(16, 32, 1, 1)
%121 : Float(16)
%122 : Float(16)
%123 : Float(16)
%124 : Float(16)
%125 : Float(16)
%126 : Long()
%127 : Float(8, 16, 1, 1)
%128 : Float(8)
%129 : Float(8)
%130 : Float(8)
%131 : Float(8)
%132 : Float(8)
%133 : Long()
%134 : Float(16, 8, 1, 3)
%135 : Float(16)
%136 : Float(16)
%137 : Float(16)
%138 : Float(16)
%139 : Float(16)
%140 : Long()
%141 : Float(16, 16, 3, 1)
%142 : Float(16)
%143 : Float(16)
%144 : Float(16)
%145 : Float(16)
%146 : Float(16)
%147 : Long()
%148 : Float(32, 16, 1, 1)
%149 : Float(32)
%150 : Float(32)
%151 : Float(32)
%152 : Float(32)
%153 : Float(32)
%154 : Long()
%155 : Float(16, 32, 1, 1)
%156 : Float(16)
%157 : Float(16)
%158 : Float(16)
%159 : Float(16)
%160 : Float(16)
%161 : Long()
%162 : Float(8, 16, 1, 1)
%163 : Float(8)
%164 : Float(8)
%165 : Float(8)
%166 : Float(8)
%167 : Float(8)
%168 : Long()
%169 : Float(16, 8, 1, 3)
%170 : Float(16)
%171 : Float(16)
%172 : Float(16)
%173 : Float(16)
%174 : Float(16)
%175 : Long()
%176 : Float(16, 16, 3, 1)
%177 : Float(16)
%178 : Float(16)
%179 : Float(16)
%180 : Float(16)
%181 : Float(16)
%182 : Long()
%183 : Float(32, 16, 1, 1)
%184 : Float(32)
%185 : Float(32)
%186 : Float(32)
%187 : Float(32)
%188 : Float(32)
%189 : Long()
%190 : Float(16, 32, 1, 1)
%191 : Float(16)
%192 : Float(16)
%193 : Float(16)
%194 : Float(16)
%195 : Float(16)
%196 : Long()
%197 : Float(8, 16, 1, 1)
%198 : Float(8)
%199 : Float(8)
%200 : Float(8)
%201 : Float(8)
%202 : Float(8)
%203 : Long()
%204 : Float(16, 8, 1, 3)
%205 : Float(16)
%206 : Float(16)
%207 : Float(16)
%208 : Float(16)
%209 : Float(16)
%210 : Long()
%211 : Float(16, 16, 3, 1)
%212 : Float(16)
%213 : Float(16)
%214 : Float(16)
%215 : Float(16)
%216 : Float(16)
%217 : Long()
%218 : Float(32, 16, 1, 1)
%219 : Float(32)
%220 : Float(32)
%221 : Float(32)
%222 : Float(32)
%223 : Float(32)
%224 : Long()
%225 : Float(32, 32, 1, 1)
%226 : Float(32)
%227 : Float(32)
%228 : Float(32)
%229 : Float(32)
%230 : Float(32)
%231 : Long()
%232 : Float(16, 32, 1, 1)
%233 : Float(16)
%234 : Float(16)
%235 : Float(16)
%236 : Float(16)
%237 : Float(16)
%238 : Long()
%239 : Float(32, 16, 1, 3)
%240 : Float(32)
%241 : Float(32)
%242 : Float(32)
%243 : Float(32)
%244 : Float(32)
%245 : Long()
%246 : Float(32, 32, 3, 1)
%247 : Float(32)
%248 : Float(32)
%249 : Float(32)
%250 : Float(32)
%251 : Float(32)
%252 : Long()
%253 : Float(64, 32, 1, 1)
%254 : Float(64)
%255 : Float(64)
%256 : Float(64)
%257 : Float(64)
%258 : Float(64)
%259 : Long()
%260 : Float(64, 32, 1, 1)
%261 : Float(64)
%262 : Float(64)
%263 : Float(64)
%264 : Float(64)
%265 : Float(64)
%266 : Long()
%267 : Float(32, 64, 1, 1)
%268 : Float(32)
%269 : Float(32)
%270 : Float(32)
%271 : Float(32)
%272 : Float(32)
%273 : Long()
%274 : Float(16, 32, 1, 1)
%275 : Float(16)
%276 : Float(16)
%277 : Float(16)
%278 : Float(16)
%279 : Float(16)
%280 : Long()
%281 : Float(32, 16, 1, 3)
%282 : Float(32)
%283 : Float(32)
%284 : Float(32)
%285 : Float(32)
%286 : Float(32)
%287 : Long()
%288 : Float(32, 32, 3, 1)
%289 : Float(32)
%290 : Float(32)
%291 : Float(32)
%292 : Float(32)
%293 : Float(32)
%294 : Long()
%295 : Float(64, 32, 1, 1)
%296 : Float(64)
%297 : Float(64)
%298 : Float(64)
%299 : Float(64)
%300 : Float(64)
%301 : Long()
%302 : Float(32, 64, 1, 1)
%303 : Float(32)
%304 : Float(32)
%305 : Float(32)
%306 : Float(32)
%307 : Float(32)
%308 : Long()
%309 : Float(16, 32, 1, 1)
%310 : Float(16)
%311 : Float(16)
%312 : Float(16)
%313 : Float(16)
%314 : Float(16)
%315 : Long()
%316 : Float(32, 16, 1, 3)
%317 : Float(32)
%318 : Float(32)
%319 : Float(32)
%320 : Float(32)
%321 : Float(32)
%322 : Long()
%323 : Float(32, 32, 3, 1)
%324 : Float(32)
%325 : Float(32)
%326 : Float(32)
%327 : Float(32)
%328 : Float(32)
%329 : Long()
%330 : Float(64, 32, 1, 1)
%331 : Float(64)
%332 : Float(64)
%333 : Float(64)
%334 : Float(64)
%335 : Float(64)
%336 : Long()
%337 : Float(32, 64, 1, 1)
%338 : Float(32)
%339 : Float(32)
%340 : Float(32)
%341 : Float(32)
%342 : Float(32)
%343 : Long()
%344 : Float(16, 32, 1, 1)
%345 : Float(16)
%346 : Float(16)
%347 : Float(16)
%348 : Float(16)
%349 : Float(16)
%350 : Long()
%351 : Float(32, 16, 1, 3)
%352 : Float(32)
%353 : Float(32)
%354 : Float(32)
%355 : Float(32)
%356 : Float(32)
%357 : Long()
%358 : Float(32, 32, 3, 1)
%359 : Float(32)
%360 : Float(32)
%361 : Float(32)
%362 : Float(32)
%363 : Float(32)
%364 : Long()
%365 : Float(64, 32, 1, 1)
%366 : Float(64)
%367 : Float(64)
%368 : Float(64)
%369 : Float(64)
%370 : Float(64)
%371 : Long()
%372 : Float(32, 64, 1, 1)
%373 : Float(32)
%374 : Float(32)
%375 : Float(32)
%376 : Float(32)
%377 : Float(32)
%378 : Long()
%379 : Float(16, 32, 1, 1)
%380 : Float(16)
%381 : Float(16)
%382 : Float(16)
%383 : Float(16)
%384 : Float(16)
%385 : Long()
%386 : Float(32, 16, 1, 3)
%387 : Float(32)
%388 : Float(32)
%389 : Float(32)
%390 : Float(32)
%391 : Float(32)
%392 : Long()
%393 : Float(32, 32, 3, 1)
%394 : Float(32)
%395 : Float(32)
%396 : Float(32)
%397 : Float(32)
%398 : Float(32)
%399 : Long()
%400 : Float(64, 32, 1, 1)
%401 : Float(64)
%402 : Float(64)
%403 : Float(64)
%404 : Float(64)
%405 : Float(64)
%406 : Long()
%407 : Float(32, 64, 1, 1)
%408 : Float(32)
%409 : Float(32)
%410 : Float(32)
%411 : Float(32)
%412 : Float(32)
%413 : Long()
%414 : Float(16, 32, 1, 1)
%415 : Float(16)
%416 : Float(16)
%417 : Float(16)
%418 : Float(16)
%419 : Float(16)
%420 : Long()
%421 : Float(32, 16, 1, 3)
%422 : Float(32)
%423 : Float(32)
%424 : Float(32)
%425 : Float(32)
%426 : Float(32)
%427 : Long()
%428 : Float(32, 32, 3, 1)
%429 : Float(32)
%430 : Float(32)
%431 : Float(32)
%432 : Float(32)
%433 : Float(32)
%434 : Long()
%435 : Float(64, 32, 1, 1)
%436 : Float(64)
%437 : Float(64)
%438 : Float(64)
%439 : Float(64)
%440 : Float(64)
%441 : Long()
%442 : Float(64, 64, 1, 1)
%443 : Float(64)
%444 : Float(64)
%445 : Float(64)
%446 : Float(64)
%447 : Float(64)
%448 : Long()
%449 : Float(32, 64, 1, 1)
%450 : Float(32)
%451 : Float(32)
%452 : Float(32)
%453 : Float(32)
%454 : Float(32)
%455 : Long()
%456 : Float(64, 32, 1, 3)
%457 : Float(64)
%458 : Float(64)
%459 : Float(64)
%460 : Float(64)
%461 : Float(64)
%462 : Long()
%463 : Float(64, 64, 3, 1)
%464 : Float(64)
%465 : Float(64)
%466 : Float(64)
%467 : Float(64)
%468 : Float(64)
%469 : Long()
%470 : Float(128, 64, 1, 1)
%471 : Float(128)
%472 : Float(128)
%473 : Float(128)
%474 : Float(128)
%475 : Float(128)
%476 : Long()
%477 : Float(128, 64, 1, 1)
%478 : Float(128)
%479 : Float(128)
%480 : Float(128)
%481 : Float(128)
%482 : Float(128)
%483 : Long()
%484 : Float(64, 128, 1, 1)
%485 : Float(64)
%486 : Float(64)
%487 : Float(64)
%488 : Float(64)
%489 : Float(64)
%490 : Long()
%491 : Float(32, 64, 1, 1)
%492 : Float(32)
%493 : Float(32)
%494 : Float(32)
%495 : Float(32)
%496 : Float(32)
%497 : Long()
%498 : Float(64, 32, 1, 3)
%499 : Float(64)
%500 : Float(64)
%501 : Float(64)
%502 : Float(64)
%503 : Float(64)
%504 : Long()
%505 : Float(64, 64, 3, 1)
%506 : Float(64)
%507 : Float(64)
%508 : Float(64)
%509 : Float(64)
%510 : Float(64)
%511 : Long()
%512 : Float(128, 64, 1, 1)
%513 : Float(128)
%514 : Float(128)
%515 : Float(128)
%516 : Float(128)
%517 : Float(128)
%518 : Long()
%519 : Float(64, 128, 1, 1)
%520 : Float(64)
%521 : Float(64)
%522 : Float(64)
%523 : Float(64)
%524 : Float(64)
%525 : Long()
%526 : Float(32, 64, 1, 1)
%527 : Float(32)
%528 : Float(32)
%529 : Float(32)
%530 : Float(32)
%531 : Float(32)
%532 : Long()
%533 : Float(64, 32, 1, 3)
%534 : Float(64)
%535 : Float(64)
%536 : Float(64)
%537 : Float(64)
%538 : Float(64)
%539 : Long()
%540 : Float(64, 64, 3, 1)
%541 : Float(64)
%542 : Float(64)
%543 : Float(64)
%544 : Float(64)
%545 : Float(64)
%546 : Long()
%547 : Float(128, 64, 1, 1)
%548 : Float(128)
%549 : Float(128)
%550 : Float(128)
%551 : Float(128)
%552 : Float(128)
%553 : Long()
%554 : Float(64, 128, 1, 1)
%555 : Float(64)
%556 : Float(64)
%557 : Float(64)
%558 : Float(64)
%559 : Float(64)
%560 : Long()
%561 : Float(32, 64, 1, 1)
%562 : Float(32)
%563 : Float(32)
%564 : Float(32)
%565 : Float(32)
%566 : Float(32)
%567 : Long()
%568 : Float(64, 32, 1, 3)
%569 : Float(64)
%570 : Float(64)
%571 : Float(64)
%572 : Float(64)
%573 : Float(64)
%574 : Long()
%575 : Float(64, 64, 3, 1)
%576 : Float(64)
%577 : Float(64)
%578 : Float(64)
%579 : Float(64)
%580 : Float(64)
%581 : Long()
%582 : Float(128, 64, 1, 1)
%583 : Float(128)
%584 : Float(128)
%585 : Float(128)
%586 : Float(128)
%587 : Float(128)
%588 : Long()
%589 : Float(64, 128, 1, 1)
%590 : Float(64)
%591 : Float(64)
%592 : Float(64)
%593 : Float(64)
%594 : Float(64)
%595 : Long()
%596 : Float(32, 64, 1, 1)
%597 : Float(32)
%598 : Float(32)
%599 : Float(32)
%600 : Float(32)
%601 : Float(32)
%602 : Long()
%603 : Float(64, 32, 1, 3)
%604 : Float(64)
%605 : Float(64)
%606 : Float(64)
%607 : Float(64)
%608 : Float(64)
%609 : Long()
%610 : Float(64, 64, 3, 1)
%611 : Float(64)
%612 : Float(64)
%613 : Float(64)
%614 : Float(64)
%615 : Float(64)
%616 : Long()
%617 : Float(128, 64, 1, 1)
%618 : Float(128)
%619 : Float(128)
%620 : Float(128)
%621 : Float(128)
%622 : Float(128)
%623 : Long()
%624 : Float(64, 128, 1, 1)
%625 : Float(64)
%626 : Float(64)
%627 : Float(64)
%628 : Float(64)
%629 : Float(64)
%630 : Long()
%631 : Float(32, 64, 1, 1)
%632 : Float(32)
%633 : Float(32)
%634 : Float(32)
%635 : Float(32)
%636 : Float(32)
%637 : Long()
%638 : Float(64, 32, 1, 3)
%639 : Float(64)
%640 : Float(64)
%641 : Float(64)
%642 : Float(64)
%643 : Float(64)
%644 : Long()
%645 : Float(64, 64, 3, 1)
%646 : Float(64)
%647 : Float(64)
%648 : Float(64)
%649 : Float(64)
%650 : Float(64)
%651 : Long()
%652 : Float(128, 64, 1, 1)
%653 : Float(128)
%654 : Float(128)
%655 : Float(128)
%656 : Float(128)
%657 : Float(128)
%658 : Long()
%659 : Float(64, 128, 1, 1)
%660 : Float(64)
%661 : Float(64)
%662 : Float(64)
%663 : Float(64)
%664 : Float(64)
%665 : Long()
%666 : Float(32, 64, 1, 1)
%667 : Float(32)
%668 : Float(32)
%669 : Float(32)
%670 : Float(32)
%671 : Float(32)
%672 : Long()
%673 : Float(64, 32, 1, 3)
%674 : Float(64)
%675 : Float(64)
%676 : Float(64)
%677 : Float(64)
%678 : Float(64)
%679 : Long()
%680 : Float(64, 64, 3, 1)
%681 : Float(64)
%682 : Float(64)
%683 : Float(64)
%684 : Float(64)
%685 : Float(64)
%686 : Long()
%687 : Float(128, 64, 1, 1)
%688 : Float(128)
%689 : Float(128)
%690 : Float(128)
%691 : Float(128)
%692 : Float(128)
%693 : Long()
%694 : Float(64, 128, 1, 1)
%695 : Float(64)
%696 : Float(64)
%697 : Float(64)
%698 : Float(64)
%699 : Float(64)
%700 : Long()
%701 : Float(32, 64, 1, 1)
%702 : Float(32)
%703 : Float(32)
%704 : Float(32)
%705 : Float(32)
%706 : Float(32)
%707 : Long()
%708 : Float(64, 32, 1, 3)
%709 : Float(64)
%710 : Float(64)
%711 : Float(64)
%712 : Float(64)
%713 : Float(64)
%714 : Long()
%715 : Float(64, 64, 3, 1)
%716 : Float(64)
%717 : Float(64)
%718 : Float(64)
%719 : Float(64)
%720 : Float(64)
%721 : Long()
%722 : Float(128, 64, 1, 1)
%723 : Float(128)
%724 : Float(128)
%725 : Float(128)
%726 : Float(128)
%727 : Float(128)
%728 : Long()
%729 : Float(128, 128, 1, 1)
%730 : Float(128)
%731 : Float(128)
%732 : Float(128)
%733 : Float(128)
%734 : Float(128)
%735 : Long()
%736 : Float(64, 128, 1, 1)
%737 : Float(64)
%738 : Float(64)
%739 : Float(64)
%740 : Float(64)
%741 : Float(64)
%742 : Long()
%743 : Float(128, 64, 1, 3)
%744 : Float(128)
%745 : Float(128)
%746 : Float(128)
%747 : Float(128)
%748 : Float(128)
%749 : Long()
%750 : Float(128, 128, 3, 1)
%751 : Float(128)
%752 : Float(128)
%753 : Float(128)
%754 : Float(128)
%755 : Float(128)
%756 : Long()
%757 : Float(256, 128, 1, 1)
%758 : Float(256)
%759 : Float(256)
%760 : Float(256)
%761 : Float(256)
%762 : Float(256)
%763 : Long()
%764 : Float(256, 128, 1, 1)
%765 : Float(256)
%766 : Float(256)
%767 : Float(256)
%768 : Float(256)
%769 : Float(256)
%770 : Long()
%771 : Float(128, 256, 1, 1)
%772 : Float(128)
%773 : Float(128)
%774 : Float(128)
%775 : Float(128)
%776 : Float(128)
%777 : Long()
%778 : Float(1000, 128)
%779 : Float(1000)) {
%785 : Float(1, 64, 110, 110) = aten::_convolution[stride=[2, 2], padding=[1, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%0, %1, %2), scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/Conv2d[conv]
%790 : Float(1, 64, 110, 110) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%785, %3, %4, %5, %6), scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/BatchNorm2d[bn]
%792 : Float(1, 64, 110, 110) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/ReLU[activ]
%795 : Float(1, 64, 55, 55), %796 : Long(1, 64, 55, 55) = aten::max_pool2d_with_indiceskernel_size=[3, 3], stride=[2, 2], padding=[0, 0], dilation=[1, 1], ceil_mode=1, scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/MaxPool2d[pool]
%802 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%795, %43, %44), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%807 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%802, %45, %46, %47, %48), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%809 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%811 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/ReLU[activ]
%817 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%795, %8, %9), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%822 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%817, %10, %11, %12, %13), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%824 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%830 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%824, %15, %16), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%835 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%830, %17, %18, %19, %20), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%837 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%843 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%837, %22, %23), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%848 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%843, %24, %25, %26, %27), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%850 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%856 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%850, %29, %30), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%861 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%856, %31, %32, %33, %34), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%863 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%869 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%863, %36, %37), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%874 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%869, %38, %39, %40, %41), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%876 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%877 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%876, %811), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]
%879 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/ReLU[activ]
%881 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/ReLU[activ]
%887 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%881, %50, %51), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%892 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%887, %52, %53, %54, %55), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%894 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%900 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%894, %57, %58), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%905 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%900, %59, %60, %61, %62), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%907 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%913 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%907, %64, %65), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%918 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%913, %66, %67, %68, %69), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%920 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%926 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%920, %71, %72), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%931 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%926, %73, %74, %75, %76), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%933 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%939 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%933, %78, %79), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%944 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%939, %80, %81, %82, %83), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%946 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%947 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%946, %881), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]
%949 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/ReLU[activ]
%951 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/ReLU[activ]
%957 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%951, %85, %86), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%962 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%957, %87, %88, %89, %90), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%964 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%970 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%964, %92, %93), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%975 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%970, %94, %95, %96, %97), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%977 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%983 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%977, %99, %100), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%988 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%983, %101, %102, %103, %104), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%990 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%996 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%990, %106, %107), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1001 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%996, %108, %109, %110, %111), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1003 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1009 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1003, %113, %114), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1014 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1009, %115, %116, %117, %118), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1016 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1017 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1016, %951), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]
%1019 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/ReLU[activ]
%1021 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/ReLU[activ]
%1027 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1021, %120, %121), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1032 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1027, %122, %123, %124, %125), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1034 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1040 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1034, %127, %128), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1045 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1040, %129, %130, %131, %132), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1047 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1053 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1047, %134, %135), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1058 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1053, %136, %137, %138, %139), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1060 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1066 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1060, %141, %142), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1071 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1066, %143, %144, %145, %146), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1073 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1079 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1073, %148, %149), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1084 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1079, %150, %151, %152, %153), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1086 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1087 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1086, %1021), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]
%1089 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/ReLU[activ]
%1091 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/ReLU[activ]
%1097 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1091, %155, %156), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1102 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1097, %157, %158, %159, %160), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1104 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1110 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1104, %162, %163), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1115 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1110, %164, %165, %166, %167), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1117 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1123 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1117, %169, %170), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1128 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1123, %171, %172, %173, %174), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1130 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%1136 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1130, %176, %177), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%1141 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1136, %178, %179, %180, %181), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1143 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%1149 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1143, %183, %184), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%1154 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1149, %185, %186, %187, %188), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1156 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%1157 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1156, %1091), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]
%1159 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/ReLU[activ]
%1161 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/ReLU[activ]
%1167 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1161, %190, %191), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%1172 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1167, %192, %193, %194, %195), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1174 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%1180 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1174, %197, %198), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%1185 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1180, %199, %200, %201, %202), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1187 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%1193 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1187, %204, %205), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%1198 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1193, %206, %207, %208, %209), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1200 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%1206 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1200, %211, %212), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%1211 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1206, %213, %214, %215, %216), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1213 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%1219 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1213, %218, %219), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%1224 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1219, %220, %221, %222, %223), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1226 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%1227 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1226, %1161), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]
%1229 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/ReLU[activ]
%1235 : Float(1, 64, 28, 28) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1229, %260, %261), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%1240 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1235, %262, %263, %264, %265), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%1242 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%1244 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/ReLU[activ]
%1250 : Float(1, 32, 28, 28) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1229, %225, %226), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%1255 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1250, %227, %228, %229, %230), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1257 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%1263 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1257, %232, %233), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%1268 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1263, %234, %235, %236, %237), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1270 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%1276 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1270, %239, %240), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%1281 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1276, %241, %242, %243, %244), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1283 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%1289 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1283, %246, %247), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%1294 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1289, %248, %249, %250, %251), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1296 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%1302 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1296, %253, %254), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%1307 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1302, %255, %256, %257, %258), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1309 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%1310 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1309, %1244), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]
%1312 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/ReLU[activ]
%1314 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/ReLU[activ]
%1320 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1314, %267, %268), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%1325 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1320, %269, %270, %271, %272), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1327 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%1333 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1327, %274, %275), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%1338 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1333, %276, %277, %278, %279), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1340 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%1346 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1340, %281, %282), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%1351 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1346, %283, %284, %285, %286), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1353 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%1359 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1353, %288, %289), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%1364 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1359, %290, %291, %292, %293), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1366 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%1372 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1366, %295, %296), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%1377 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1372, %297, %298, %299, %300), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1379 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%1380 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1379, %1314), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]
%1382 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/ReLU[activ]
%1384 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/ReLU[activ]
%1390 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1384, %302, %303), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%1395 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1390, %304, %305, %306, %307), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1397 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%1403 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1397, %309, %310), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%1408 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1403, %311, %312, %313, %314), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1410 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%1416 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1410, %316, %317), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%1421 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1416, %318, %319, %320, %321), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1423 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%1429 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1423, %323, %324), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1434 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1429, %325, %326, %327, %328), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1436 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1442 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1436, %330, %331), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1447 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1442, %332, %333, %334, %335), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1449 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1450 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1449, %1384), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]
%1452 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/ReLU[activ]
%1454 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/ReLU[activ]
%1460 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1454, %337, %338), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1465 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1460, %339, %340, %341, %342), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1467 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1473 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1467, %344, %345), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1478 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1473, %346, %347, %348, %349), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1480 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1486 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1480, %351, %352), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1491 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1486, %353, %354, %355, %356), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1493 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1499 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1493, %358, %359), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1504 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1499, %360, %361, %362, %363), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1506 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1512 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1506, %365, %366), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1517 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1512, %367, %368, %369, %370), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1519 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1520 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1519, %1454), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]
%1522 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/ReLU[activ]
%1524 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/ReLU[activ]
%1530 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1524, %372, %373), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1535 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1530, %374, %375, %376, %377), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1537 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1543 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1537, %379, %380), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1548 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1543, %381, %382, %383, %384), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1550 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1556 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1550, %386, %387), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1561 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1556, %388, %389, %390, %391), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1563 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%1569 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1563, %393, %394), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%1574 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1569, %395, %396, %397, %398), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1576 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%1582 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1576, %400, %401), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%1587 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1582, %402, %403, %404, %405), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1589 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%1590 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1589, %1524), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]
%1592 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/ReLU[activ]
%1594 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/ReLU[activ]
%1600 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1594, %407, %408), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%1605 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1600, %409, %410, %411, %412), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1607 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%1613 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1607, %414, %415), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%1618 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1613, %416, %417, %418, %419), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1620 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%1626 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1620, %421, %422), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%1631 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1626, %423, %424, %425, %426), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1633 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%1639 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1633, %428, %429), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%1644 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1639, %430, %431, %432, %433), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1646 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%1652 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1646, %435, %436), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%1657 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1652, %437, %438, %439, %440), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1659 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%1660 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1659, %1594), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]
%1662 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/ReLU[activ]
%1668 : Float(1, 128, 14, 14) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1662, %477, %478), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%1673 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1668, %479, %480, %481, %482), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%1675 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%1677 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/ReLU[activ]
%1683 : Float(1, 64, 14, 14) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1662, %442, %443), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%1688 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1683, %444, %445, %446, %447), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1690 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%1696 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1690, %449, %450), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%1701 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1696, %451, %452, %453, %454), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1703 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%1709 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1703, %456, %457), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%1714 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1709, %458, %459, %460, %461), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1716 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%1722 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1716, %463, %464), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%1727 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1722, %465, %466, %467, %468), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1729 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%1735 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1729, %470, %471), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%1740 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1735, %472, %473, %474, %475), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1742 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%1743 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1742, %1677), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]
%1745 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/ReLU[activ]
%1747 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/ReLU[activ]
%1753 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1747, %484, %485), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%1758 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1753, %486, %487, %488, %489), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1760 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%1766 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1760, %491, %492), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%1771 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1766, %493, %494, %495, %496), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1773 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%1779 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1773, %498, %499), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%1784 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1779, %500, %501, %502, %503), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1786 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%1792 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1786, %505, %506), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%1797 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1792, %507, %508, %509, %510), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1799 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%1805 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1799, %512, %513), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%1810 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1805, %514, %515, %516, %517), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1812 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%1813 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1812, %1747), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]
%1815 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/ReLU[activ]
%1817 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/ReLU[activ]
%1823 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1817, %519, %520), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%1828 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1823, %521, %522, %523, %524), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1830 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%1836 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1830, %526, %527), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%1841 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1836, %528, %529, %530, %531), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1843 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%1849 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1843, %533, %534), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%1854 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1849, %535, %536, %537, %538), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1856 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%1862 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1856, %540, %541), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1867 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1862, %542, %543, %544, %545), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1869 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1875 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1869, %547, %548), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1880 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1875, %549, %550, %551, %552), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1882 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1883 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1882, %1817), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]
%1885 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/ReLU[activ]
%1887 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/ReLU[activ]
%1893 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1887, %554, %555), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1898 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1893, %556, %557, %558, %559), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1900 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1906 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1900, %561, %562), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1911 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1906, %563, %564, %565, %566), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1913 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1919 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1913, %568, %569), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1924 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1919, %570, %571, %572, %573), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1926 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1932 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1926, %575, %576), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1937 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1932, %577, %578, %579, %580), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1939 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1945 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1939, %582, %583), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1950 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1945, %584, %585, %586, %587), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1952 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1953 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1952, %1887), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]
%1955 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/ReLU[activ]
%1957 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/ReLU[activ]
%1963 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1957, %589, %590), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1968 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1963, %591, %592, %593, %594), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1970 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1976 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1970, %596, %597), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1981 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1976, %598, %599, %600, %601), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1983 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1989 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1983, %603, %604), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1994 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1989, %605, %606, %607, %608), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1996 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%2002 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1996, %610, %611), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%2007 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2002, %612, %613, %614, %615), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2009 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%2015 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2009, %617, %618), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%2020 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2015, %619, %620, %621, %622), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2022 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%2023 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2022, %1957), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]
%2025 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/ReLU[activ]
%2027 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/ReLU[activ]
%2033 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2027, %624, %625), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%2038 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2033, %626, %627, %628, %629), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2040 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%2046 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2040, %631, %632), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%2051 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2046, %633, %634, %635, %636), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2053 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%2059 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2053, %638, %639), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%2064 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2059, %640, %641, %642, %643), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2066 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%2072 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2066, %645, %646), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%2077 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2072, %647, %648, %649, %650), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2079 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%2085 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2079, %652, %653), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%2090 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2085, %654, %655, %656, %657), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2092 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%2093 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2092, %2027), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]
%2095 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/ReLU[activ]
%2097 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/ReLU[activ]
%2103 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2097, %659, %660), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/Conv2d[conv]
%2108 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2103, %661, %662, %663, %664), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2110 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/ReLU[activ]
%2116 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2110, %666, %667), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/Conv2d[conv]
%2121 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2116, %668, %669, %670, %671), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2123 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/ReLU[activ]
%2129 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2123, %673, %674), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/Conv2d[conv]
%2134 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2129, %675, %676, %677, %678), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2136 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/ReLU[activ]
%2142 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2136, %680, %681), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/Conv2d[conv]
%2147 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2142, %682, %683, %684, %685), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2149 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/ReLU[activ]
%2155 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2149, %687, %688), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/Conv2d[conv]
%2160 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2155, %689, %690, %691, %692), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2162 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/ReLU[activ]
%2163 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2162, %2097), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]
%2165 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/ReLU[activ]
%2167 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/ReLU[activ]
%2173 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2167, %694, %695), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/Conv2d[conv]
%2178 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2173, %696, %697, %698, %699), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2180 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/ReLU[activ]
%2186 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2180, %701, %702), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/Conv2d[conv]
%2191 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2186, %703, %704, %705, %706), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2193 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/ReLU[activ]
%2199 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2193, %708, %709), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/Conv2d[conv]
%2204 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2199, %710, %711, %712, %713), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2206 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/ReLU[activ]
%2212 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2206, %715, %716), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/Conv2d[conv]
%2217 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2212, %717, %718, %719, %720), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2219 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/ReLU[activ]
%2225 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2219, %722, %723), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/Conv2d[conv]
%2230 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2225, %724, %725, %726, %727), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2232 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/ReLU[activ]
%2233 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2232, %2167), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]
%2235 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/ReLU[activ]
%2241 : Float(1, 256, 7, 7) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2235, %764, %765), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%2246 : Float(1, 256, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2241, %766, %767, %768, %769), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%2248 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%2250 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/ReLU[activ]
%2256 : Float(1, 128, 7, 7) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2235, %729, %730), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%2261 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2256, %731, %732, %733, %734), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2263 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%2269 : Float(1, 64, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2263, %736, %737), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%2274 : Float(1, 64, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2269, %738, %739, %740, %741), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2276 : Float(1, 64, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%2282 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2276, %743, %744), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%2287 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2282, %745, %746, %747, %748), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2289 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%2295 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2289, %750, %751), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%2300 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2295, %752, %753, %754, %755), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2302 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%2308 : Float(1, 256, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2302, %757, %758), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%2313 : Float(1, 256, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2308, %759, %760, %761, %762), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2315 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%2316 : Float(1, 256, 7, 7) = aten::add[alpha={1}](%2315, %2250), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]
%2318 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/ReLU[activ]
%2324 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2318, %771, %772), scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/Conv2d[conv]
%2329 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2324, %773, %774, %775, %776), scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/BatchNorm2d[bn]
%2331 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/ReLU[activ]
%2333 : Float(1, 128, 1, 1) = aten::avg_pool2dkernel_size=[7, 7], stride=[1, 1], padding=[0, 0], ceil_mode=0, count_include_pad=1, scope: SqueezeNext/Sequential[features]/AvgPool2d[final_pool]
%2334 : Long() = aten::sizedim=0, scope: SqueezeNext
%2335 : Long() = prim::Constantvalue={-1}, scope: SqueezeNext
%2336 : Dynamic = aten::stack[dim=0](%2334, %2335), scope: SqueezeNext
%2337 : Float(1, 128) = aten::view(%2333, %2336), scope: SqueezeNext
%2338 : Float(128!, 1000!) = aten::t(%778), scope: SqueezeNext/Linear[output]
%2339 : Float(1, 1000) = aten::expandsize=[1, 1000], implicit=1, scope: SqueezeNext/Linear[output]
%2340 : Float(1, 1000) = aten::addmm[beta={1}, alpha={1}](%2339, %2337, %2338), scope: SqueezeNext/Linear[output]
return (%2340);
}
, False
@nerox8664 Hi, we train a model with customized, called SegmentConsensus, layer by torch.0.4.0, and getting the following error:
Using TensorFlow backend.
Freezing BatchNorm2D except the first one.
Freezing BatchNorm2D except the first one.
Traceback (most recent call last):
File "pytorch_trn-res2keras-tf.py", line 27, in
keras_model = pytorch_to_keras(pytorch_model, input_var, (4, 3, 224, 224), verbose=True)
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 91, in pytorch_to_keras
trace, torch_out = torch.jit.get_trace_graph(model, tuple(args))
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 29, in set_training
yield
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 91, in pytorch_to_keras
trace, torch_out = torch.jit.get_trace_graph(model, tuple(args))
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/jit/init.py", line 255, in get_trace_graph
return LegacyTracedModule(f, nderivs=nderivs)(*args, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/jit/init.py", line 288, in forward
out = self.inner(*trace_inputs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self._slow_forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 479, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/intellif/framework_exchange/pytorch2keras/models.py", line 338, in forward
output = self.consensus(base_out)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self._slow_forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 479, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/intellif/framework_exchange/pytorch2keras/ops/basic_ops.py", line 47, in forward
return SegmentConsensus(self.consensus_type, self.dim)(input)
RuntimeError: Attempted to trace SegmentConsensus, but tracing of legacy functions is not supported
How to understand tracing of legacy functions is not supported
@nerox8664 Hi guys, after successfully getting the converted model, I got the error as following:
Using TensorFlow backend.
2018-06-28 11:24:45.607236: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-06-28 11:24:46.300184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
pciBusID: 0000:88:00.0
totalMemory: 10.91GiB freeMemory: 10.75GiB
2018-06-28 11:24:46.300260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-28 11:24:46.914014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-28 11:24:46.914079: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-06-28 11:24:46.914088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-06-28 11:24:46.914566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10390 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:88:00.0, compute capability: 6.1)
/home/wupeng/workshop/fashion-keypoint/venv-kp/local/lib/python2.7/site-packages/keras/models.py:252: UserWarning: No training configuration found in save file: the model was not compiled. Compile it manually.
warnings.warn('No training configuration found in save file: '
('image_path :', './data/train_0427/Images/dress/ca8b38cc7fb0ce9e35563766d12090b7.jpg')
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)
There are some problems with new PyTorch release:
What to do:
I want to apply InstanceNormalization after nn.ConvTranspose2d. However, an error appeared and it was impossible.
Can not it work with the current program?
Write down the test code and a error.
source code.
import numpy as np
import torch
import torch.nn as nn
from torch.autograd import Variable
from pytorch2keras.converter import pytorch_to_keras
from sys import exit
class TestMultipleInputs(nn.Module):
"""Module for Conv2d conversion testing
"""
def __init__(self, inp=10, out=16, kernel_size=3, bias=True):
super(TestMultipleInputs, self).__init__()
self.conv2d = nn.Conv2d(inp, out, kernel_size=kernel_size, bias=bias)
self.deconv2d = nn.ConvTranspose2d(inp, out, kernel_size=kernel_size, bias=bias)
self.in2d = nn.InstanceNorm2d(out)
def forward(self, x, y, z):
# return self.in2d(self.conv2d(x)) + self.in2d(self.conv2d(y)) + self.in2d(self.conv2d(z))
return self.in2d(self.deconv2d(x)) + self.in2d(self.deconv2d(y)) + self.in2d(self.deconv2d(z))
# return self.conv2d(x) + self.conv2d(y) + self.conv2d(z)
if __name__ == '__main__':
max_error = 0
kernel_size = np.random.randint(1, 7)
inp = np.random.randint(kernel_size + 1, 100)
out = np.random.randint(1, 100)
model = TestMultipleInputs(inp, out, kernel_size, inp % 2)
input_np = np.random.uniform(0, 1, (1, inp, inp, inp))
input_var = Variable(torch.FloatTensor(input_np))
input_var2 = Variable(torch.FloatTensor(input_np))
input_var3 = Variable(torch.FloatTensor(input_np))
print(input_var.size())
print(input_var2.size())
print(input_var3.size())
output = model(input_var, input_var2, input_var3)
k_model = pytorch_to_keras(model, [input_var, input_var2, input_var3], [(inp, inp, inp,), (inp, inp, inp,), (inp, inp, inp,)], verbose=True)
pytorch_output = output.data.numpy()
keras_output = k_model.predict([input_np, input_np, input_np])
an error.
Using TensorFlow backend.
torch.Size([1, 92, 92, 92])
torch.Size([1, 92, 92, 92])
torch.Size([1, 92, 92, 92])
graph(%0 : Float(1, 92, 92, 92)
%1 : Float(1, 92, 92, 92)
%2 : Float(1, 92, 92, 92)
%3 : Float(53, 92, 1, 1)
%4 : Float(92, 53, 1, 1)) {
%5 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%0, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
%6 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%8 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%5, %6, %7), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%9 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%1, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
%10 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%11 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%12 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%9, %10, %11), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%13 : Float(1, 53, 92, 92) = onnx::Add(%8, %12), scope: TestMultipleInputs
%14 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%2, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
%15 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%16 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%17 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%14, %15, %16), scope: TestMultipleInputs/InstanceNorm2d[in2d]
%18 : Float(1, 53, 92, 92) = onnx::Add(%13, %17), scope: TestMultipleInputs
return (%18);
}
[18 defined in (%18 : Float(1, 53, 92, 92) = onnx::Add(%13, %17), scope: TestMultipleInputs
)]
Graph outputs: ['18']
State dict: ['conv2d.weight', 'deconv2d.weight']
____
graph node: TestMultipleInputs/ConvTranspose2d[deconv2d]
type: onnx::ConvTranspose
inputs: ['input0']
outputs: ['TestMultipleInputs/ConvTranspose2d[deconv2d]']
name in state_dict: deconv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [1, 1], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting transposed convolution ...
2018-10-20 00:17:50.934191: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::Constant
inputs: []
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'value': tensor([ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1.])}
is_terminal: False
Converting constant ...
____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::Constant
inputs: []
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'value': tensor([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0.])}
is_terminal: False
Converting constant ...
____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::InstanceNormalization
inputs: ['5', '6', '7']
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'epsilon': 1e-05}
is_terminal: False
Converting instancenorm ...
Traceback (most recent call last):
File "multiple_layer_test.py", line 45, in <module>
k_model = pytorch_to_keras(model, [input_var, input_var2, input_var3], [(inp, inp, inp,), (inp, inp, inp,), (inp, inp, inp,)], verbose=True)
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/converter.py", line 182, in pytorch_to_keras
short_names
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/layers.py", line 626, in convert_instancenorm
layers[scope_name] = lambda_layer(layers[inputs[0]])
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/keras/engine/base_layer.py", line 457, in __call__
output = self.call(inputs, **kwargs)
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/keras/layers/core.py", line 682, in call
return self.function(inputs, **arguments)
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/layers.py", line 622, in target_layer
trainable=False)
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 183, in func_with_args
return func(*args, **current_args)
File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/normalization.py", line 120, in instance_norm
inputs.name, params_shape))
ValueError: Inputs deconv2d0.27781545017061593/transpose_1:0 has undefined channels dimension (?,).
In my environment.
Hi!
I have a problem in converting simple feedforward net
Here is my code:
from torch import nn, zeros, LongTensor, FloatTensor, onnx
from torch.autograd import Variable
import torch.nn.functional as F
from converter import pytorch_to_keras
class SimpleNet(nn.Module):
def __init__(self, input_size):
super(SimpleNet, self).__init__()
self.embedd = nn.Embedding(input_size, 100)
self.fc1 = nn.Linear(100, 200)
self.fc2 = nn.Linear(200, 200)
self.fc3 = nn.Linear(200, 300)
def forward(self, input):
r_0 = input.mm(self.embedd.weight)
r_1 = F.tanh(self.fc1(r_0).div(10) )
r_2 = F.tanh(self.fc2(r_1).div(10) ) + r_1
r_3 = F.tanh(self.fc3(r_2).div(10) )
return r_3
# prepare fake input
indices = [1,2,3,4,5]
input = zeros(1000).type(FloatTensor)
for i in indices:
input[i] = 1
input = Variable(input)
input = input.view(1, -1)
# create forward pass with fake input
simple_net = SimpleNet(1000)
out = simple_net(input)
k_model = pytorch_to_keras(simple_net, input, (1, 1000), verbose=True)
This gives me this output with error:
graph(%0 : Float(1, 1000)
%1 : Float(1000, 100)
%2 : Float(200, 100)
%3 : Float(200)
%4 : Float(200, 200)
%5 : Float(200)
%6 : Float(300, 200)
%7 : Float(300)) {
%8 : UNKNOWN_TYPE = Constant[value={0}](), scope: SimpleNet
%9 : Float(1, 100) = Gemm[alpha=1, beta=0, broadcast=1](%0, %1, %8), scope: SimpleNet
%10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
%11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
%13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
%14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
%15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
%17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
%18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
%19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
%20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
return (%22);
}
[22 defined in (%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
)]
['embedd.weight', 'fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias', 'fc3.weight', 'fc3.bias']
____
graph node: SimpleNet
type: Constant
inputs: ['input']
outputs: ['SimpleNet']
name in state_dict:
attrs: {'value':
0
[torch.FloatTensor of size 1]
}
is_terminal: False
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-34-ee7856e5a4c6> in <module>()
----> 1 k_model = pytorch_to_keras(simple_net, input, (1, 1000), verbose=True) #we should specify shape of the input tensor
~/workspace/pytorch2keras/pytorch2keras/converter.py in pytorch_to_keras(model, args, input_shape, change_ordering, training, verbose)
154 print('attrs:', node_attrs)
155 print('is_terminal:', node_id in graph_outputs)
--> 156 AVAILABLE_CONVERTERS[node_type](
157 node_attrs,
158 node_weights_name, node_id,
KeyError: 'Constant'
I should note, that i succesfully convert this model to onnx format:
torch_out = onnx._export(simple_net,
input,
"test_simple_net.onnx",
export_params=True,
verbose=True
)
graph(%0 : Float(1, 1000)
%1 : Float(1000, 100)
%2 : Float(200, 100)
%3 : Float(200)
%4 : Float(200, 200)
%5 : Float(200)
%6 : Float(300, 200)
%7 : Float(300)) {
%8 : UNKNOWN_TYPE = Constant[value={0}](), scope: SimpleNet
%9 : Float(1, 100) = Gemm[alpha=1, beta=0, broadcast=1](%0, %1, %8), scope: SimpleNet
%10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
%11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
%13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
%14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
%15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
%17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
%18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
%19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
%20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
return (%22);
}
Could you add "Constant" to AVAILABLE_CONVERTERS ?
thank you
It seems that there is an error in convert.convert_constant function, where it is defined as
def convert_constant(params, w_name, scope_name, inputs, layers, weights):
def target_layer(params=params):
return keras.backend.constant(np.float32(params['value']))
lambda_layer = keras.layers.Lambda(target_layer)
layers[scope_name] = lambda_layer(layers[inputs[0]])
while I believe the layers[inputs[0]]
argument to lambda_layer is supposed to be a dummy argument to bypass compatibility check in keras base_layer
and the params
variable from the convert_constant
scope should be the real argument and contains the actual constant values.
Therefore, the correct way to define target_layer should be:
def target_layer(x, params=params):
return keras.backend.constant(np.float32(params['value']))
instead.
Add new models to the test set.
With some stride / padding / kernel size combination results may be different.
For example:
stride: 2
padding: 1
kernel_size: 3
I convert the model from pytroch to keras successfully, but I can't save it, which report the following error
k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)],change_ordering=True, verbose=False)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/converter.py", line 341, in pytorch_to_keras
conf = model.get_config()
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/keras/engine/network.py", line 931, in get_config
return copy.deepcopy(config)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 215, in _deepcopy_list
append(deepcopy(a, memo))
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in <listcomp>
y = [deepcopy(a, memo) for a in x]
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in <listcomp>
y = [deepcopy(a, memo) for a in x]
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 169, in deepcopy
rv = reductor(4)
TypeError: can't pickle _thread.RLock objects
Describe the bug
When running the provided test for AlexNet, encountered KeyError: 'classifier.0.weight'
I checked what keys does weights dict have:
odict_keys(['features.0.weight', 'features.0.bias', 'features.3.weight', 'features.3.bias', 'features.6.weight', 'features.6.bias', 'features.8.weight', 'features.8.bias', 'features.10.weight', 'features.10.bias', 'classifier.1.weight', 'classifier.1.bias', 'classifier.4.weight', 'classifier.4.bias', 'classifier.6.weight', 'classifier.6.bias'])
To Reproduce
Simply run pytorch2keras/tests/models/alexnet.py
Logs
....
....
....
Converting flatten ...
graph node: AlexNet/Sequential[classifier]/Dropout[0]
node id: 104
type: onnx::Dropout
inputs: ['101']
outputs: ['AlexNet/Sequential[classifier]/Dropout[0]', 'AlexNet/Sequential[classifier]/Dropout[0]']
name in state_dict: classifier.0
attrs: {'ratio': 0.5}
is_terminal: False
Converting dropout ...
graph node: AlexNet/Sequential[classifier]/Dropout[0]
node id: 109
type: onnx::Gemm
inputs: ['104', 'input11', 'input12']
outputs: ['AlexNet/Sequential[classifier]/Dropout[0]']
name in state_dict: classifier.0
attrs: {'alpha': 1.0, 'beta': 1.0, 'transB': 1}
is_terminal: False
Converting Linear ...
Traceback (most recent call last):
File "validate_cnn.py", line 39, in
k_model = pytorch_to_keras(model, input_var, (3, 224, 224,), verbose=True)
File "C:\Users\xxxxxx\work\WPy-3670\python-3.6.7.amd64\lib\site-packages\pytorch2keras\converter.py", line 325, in pytorch_to_keras
names
File "C:\Users\xxxxxx\work\WPy-3670\python-3.6.7.amd64\lib\site-packages\pytorch2keras\linear_layers.py", line 34, in convert_gemm
W = weights[weights_name].numpy().transpose()
KeyError: 'classifier.0.weight'
Environment (please complete the following information):
graph node: CRNN type: onnx::Transpose inputs: ['68'] outputs: ['CRNN'] name in state_dict: attrs: {'perm': [2, 0, 1]} is_terminal: False Converting transpose ... !!! Cannot permute batch dimension. Result may be wrong !!! Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ben/.local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 143, in pytorch_to_keras if node_inputs[0] in model_inputs: IndexError: list index out of range
I got a model which has some layers as following:
State dict: ['base.conv1.weight', 'base.bn1.weight', 'base.bn1.bias', 'base.bn1.running_mean', 'base.bn1.running_var', 'base.layer1.0.conv1.weight', ..., 'local_conv_list.0.0.weight', 'local_conv_list.0.0.bias', 'local_conv_list.0.1.weight', 'local_conv_list.0.1.bias', 'local_conv_list.0.1.running_mean', ..., 'fc_list.0.weight', 'fc_list.0.bias', 'fc_list.1.weight', 'fc_list.1.bias', 'fc_list.2.weight', 'fc_list.2.bias', 'fc_list.3.weight', 'fc_list.3.bias', 'fc_list.4.weight', 'fc_list.4.bias', 'fc_list.5.weight', 'fc_list.5.bias']
The definition of the model is : ResNet50 + self.local_conv_list = nn.ModuleList()
where:
self.local_conv_list.append(nn.Sequential(
nn.Conv2d(2048, local_conv_out_channels, 1),
nn.BatchNorm2d(local_conv_out_channels),
nn.ReLU(inplace=True)
))
In converter.py
node_weights_name = '.'.join(
re.findall(r'\[([\w\d.]+)\]', node_scope_name)
)
this function parsing the string, such like:
['PCBModel/ResNet[base]/Sequential[layer4]/Bottleneck[2]/Conv2d[conv3]']
into the original pytorch layer name such as :
base.layer4.2.conv3
then converting the weights accordingly.
But when it comes to the layer which does not have certain naming pattern, such like:
'local_conv_list.0.1.weight',
graph node: PCBModel/Sequential/Conv2d[0]
name in state_dict: 0
the parsing results will be something wrong and it will never matched with any layer name in the State dict, therefore results in the converting error.
So, I got stuck here and was wondering how to handle it, is there anyone could give some constructive suggestion?
Close the bracket at https://github.com/nerox8664/pytorch2keras/blob/master/pytorch2keras/layers.py#L23
after reading all the issue, there was the branch which could support the pytorch0.3.0,
would you mind sharing this branch again.
Thanks
@nerox8664
Hi again!
Another problem with converting model to keras, if we have input as LongTensor of indices for nn.Embedding (not one-hot encoded input like in issue 11).
This is simple code for reproducing:
from torch import nn, zeros, LongTensor, FloatTensor, onnx
from torch.autograd import Variable
import torch.nn.functional as F
from converter import pytorch_to_keras
class SimpleNet(nn.Module):
def __init__(self, input_size):
super(SimpleNet, self).__init__()
self.embedd = nn.Embedding(input_size, 100)
self.fc1 = nn.Linear(100, 200)
self.fc2 = nn.Linear(200, 200)
self.fc3 = nn.Linear(200, 300)
def forward(self, input):
r_0 = self.embedd(input).sum(dim=1)
r_1 = F.tanh(self.fc1(r_0).div(10.))
r_2 = F.tanh(self.fc2(r_1).div(10.)) + r_1
r_3 = F.tanh(self.fc3(r_2).div(10.))
return r_3
input = Variable(LongTensor([[1,2,4,5]]))
simple_net = SimpleNet(1000)
out = simple_net(input)
k_model = pytorch_to_keras(simple_net, input, (1, 4), verbose=True)
And this is the output:
graph(%0 : Long(1, 4)
%1 : Float(1000, 100)
%2 : Float(200, 100)
%3 : Float(200)
%4 : Float(200, 200)
%5 : Float(200)
%6 : Float(300, 200)
%7 : Float(300)) {
%8 : Float(1, 4, 100) = Gather(%1, %0), scope: SimpleNet/Embedding[embedd]
%9 : Float(1, 100) = ReduceSum[axes=[1], keepdims=0](%8), scope: SimpleNet
%10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
%11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
%13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
%14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
%15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
%17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
%18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
%19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
%20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
%21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
return (%22);
}
[22 defined in (%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
)]
['embedd.weight', 'fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias', 'fc3.weight', 'fc3.bias']
____
graph node: SimpleNet/Embedding[embedd]
type: Gather
inputs: ['input']
outputs: ['SimpleNet/Embedding[embedd]']
name in state_dict: embedd
attrs: {}
is_terminal: False
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-43-20307fc45f89> in <module>()
----> 1 k_model = pytorch_to_keras(simple_net, input, (1, 4), verbose=True) #we should specify shape of the input tensor
~/workspace/pytorch2keras/pytorch2keras/converter.py in pytorch_to_keras(model, args, input_shape, change_ordering, training, verbose)
154 print('attrs:', node_attrs)
155 print('is_terminal:', node_id in graph_outputs)
--> 156 AVAILABLE_CONVERTERS[node_type](
157 node_attrs,
158 node_weights_name, node_id,
KeyError: 'Gather'
So, please, could you add Gather also to AVAILABLE_CONVERTERS ?
thanks
Should we simply add zeropadding before convolution transpose or we should use different advanced feature? :)
Output shapes in pytorch and keras are different
Getting this error:
KeyError: <tf.Tensor 'maxpool0.13212125626892746/transpose_1:0' shape=(?, 64, 56, 56) dtype=float32>
Continued test from issue #41, it seems like the exported keras model has lambda op. Is it correct?
output of print (k_model.summary())
in #41
Layer (type) Output Shape Param #
=================================================================
input0 (InputLayer) (None, 3, 224, 224) 0
_________________________________________________________________
lambda_1 (Lambda) (None, 3, 232, 232) 0 <-- lambda layer
_________________________________________________________________
conv1.conv2d0.19608361958321 (None, 32, 224, 224) 7808
_________________________________________________________________
in10.5890283736035935 (Lambd (None, 32, 224, 224) 0 <-- lambda layer
_________________________________________________________________
relu0.8399978862622054 (Acti (None, 32, 224, 224) 0
=================================================================
Total params: 7,808
Trainable params: 7,808
Non-trainable params: 0
Trying to make the generated keras model file to tfjs (which does not support lambda layer). Knowing that lambda can be any python codes. This would make it not portable to non-python frameworks, such as c++ or js based ones.
Don't know if it's logical to ask, but is it possible to workaround this requirements for lambda layer?
Thank you very much for your code.
I was able to convert my pytorch model to keras successfully. The summary of model was also correct. But when I wanted to save the keras model, I received the below error. I searched but didn't find anything useful. I appreciate any advice or suggestions.
TypeError: can't pickle SwigPyObject objects
Thanks,
Noosh
I've downgraded to 0.4.1 and it worked
During transfer, all layer names are changed, so it is confused to use because it is hard to find the layer name match the standard resnet101 name
After upgrade pytorch
to version 0.3, the test file alexnet.py
can not work correctly. How to solve this problem?
Add support of recurrent layers to the converter:
GRU
LSTM
when I do pip install pytorch2keras
in my virtual environment “conda,python3.6” I got an error:
`Collecting pytorch2keras
Downloading https://files.pythonhosted.org/packages/2f/61/9fd96f62814ddf7d0a3d60db85be6965451c7d6efd8d5c52b1877db68bd3/pytorch2keras-0.1.6.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/download.py", line 436, in get_file_content
with open(url, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-zk3qnkav/pytorch2keras/setup.py", line 16, in <module>
reqs = [str(ir.req) for ir in install_reqs]
File "/tmp/pip-install-zk3qnkav/pytorch2keras/setup.py", line 16, in <listcomp>
reqs = [str(ir.req) for ir in install_reqs]
File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/req/req_file.py", line 82, in parse_requirements
filename, comes_from=comes_from, session=session
File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/download.py", line 440, in get_file_content
'Could not open requirements file: %s' % str(exc)
pip._internal.exceptions.InstallationError: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zk3qnkav/pytorch2keras/
`
Feature request
A clear and concise description of what the problem is. Ex. My layer hasn't supported yet [...]
Additional context
Add any other context or screenshots about the feature request here.
I follow the installation steps but get the error , when from converter import pytorch_to_keras
Is it not suitable for python3 ?
It seems like 49dc427 breaks 0.2 compatibility
ImportError Traceback (most recent call last)
<ipython-input-9-9e4002883d78> in <module>()
1 import sys
2 sys.path.insert(0, '../pytorch2keras/pytorch2keras')
----> 3 from converter import pytorch_to_keras
4 k_model = pytorch_to_keras(model, x, (3, 224, 224), verbose=True)
5 #we should specify shape of the input tensor
~/projects/pytorch2keras/pytorch2keras/converter.py in <module>()
8 import torch.serialization
9 import contextlib
---> 10 from torch.jit import _unique_state_dict
11
12 from layers import AVAILABLE_CONVERTERS
ImportError: cannot import name '_unique_state_dict'
_unique_state_dict was added in 0.4 version: pytorch/pytorch@e6cbe84
Hi
Have there anybody successfully transferred the weights file to Keras?
Could you please share ResNet18 keras weights?
Thx!
Trying to convert an Instance Normalization layer but never successfully done so. Did a test script as the following:
package version:
pytorch2keras: 0.1.10
tensorflow: 1.12
pytorch: 0.4.1
numpy: 1.15.1
# pytorch2tf_instancenorm_test.py
import numpy as np
import torch
from torch.autograd import Variable
from pytorch2keras import converter
class Pytorch2KerasTestNet(torch.nn.Module):
def __init__(self):
super(Pytorch2KerasTestNet, self).__init__()
self.conv1 = ConvLayer(3, 32, kernel_size=9, stride=1)
self.in1 = torch.nn.InstanceNorm2d(32, affine=True)
self.relu = torch.nn.ReLU()
def forward(self, x):
y = self.relu(self.in1(self.conv1(x)))
return y
class ConvLayer(torch.nn.Module):
def __init__(self, in_channels, out_channels, kernel_size, stride):
super(ConvLayer, self).__init__()
reflection_padding = kernel_size // 2
self.reflection_pad = torch.nn.ReflectionPad2d(reflection_padding)
self.conv2d = torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride)
def forward(self, x):
out = self.reflection_pad(x)
print("conv2d")
out = self.conv2d(out)
return out
model = Pytorch2KerasTestNet()
input_np = np.random.uniform(0, 1, (1, 3, 224, 224))
input_var = Variable(torch.FloatTensor(input_np))
k_model = converter.pytorch_to_keras(model, input_var, [(3, 224, 224,)], verbose=True)
When running it using python pytorch2tf_instancenorm_test.py
, it results in the following error:
2018-12-05 20:57:56.505141: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Using TensorFlow backend.
conv2d
graph(%0 : Float(1, 3, 224, 224)
%1 : Float(32, 3, 9, 9)
%2 : Float(32)
%3 : Float(32)
%4 : Float(32)) {
%5 : Float(1, 3, 232, 232) = onnx::Pad[mode=reflect, pads=[0, 0, 4, 4, 0, 0, 4, 4]](%0), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
%6 : Float(1, 32, 224, 224) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[9, 9], pads=[0, 0, 0, 0], strides=[1, 1]](%5, %1, %2), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
%7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
%8 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
%9 : Float(1, 32, 224, 224) = onnx::InstanceNormalization[epsilon=1e-05](%6, %7, %8), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
%10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
return (%10);
}
[10 defined in (%10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
)]
[%5 : Float(1, 3, 232, 232) = onnx::Pad[mode=reflect, pads=[0, 0, 4, 4, 0, 0, 4, 4]](%0), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
, %6 : Float(1, 32, 224, 224) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[9, 9], pads=[0, 0, 0, 0], strides=[1, 1]](%5, %1, %2), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
, %7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %8 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %9 : Float(1, 32, 224, 224) = onnx::InstanceNormalization[epsilon=1e-05](%6, %7, %8), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
]
Graph outputs: ['10']
State dict: ['conv1.conv2d.weight', 'conv1.conv2d.bias', 'in1.weight', 'in1.bias']
____
graph node: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
id: 5
type: onnx::Pad
inputs: ['input0']
outputs: ['Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]']
name in state_dict: conv1.reflection_pad
attrs: {'mode': 'reflect', 'pads': [0, 0, 4, 4, 0, 0, 4, 4]}
is_terminal: False
Converting padding...
____
graph node: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
id: 6
type: onnx::Conv
inputs: ['5']
outputs: ['Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]']
name in state_dict: conv1.conv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [9, 9], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting convolution ...
____
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 7
type: onnx::Constant
inputs: []
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'value': tensor([0.8979, 0.3156, 0.1349, 0.2346, 0.8874, 0.7174, 0.3563, 0.0794, 0.9949,
0.6409, 0.7893, 0.0516, 0.6400, 0.9812, 0.6296, 0.0555, 0.8294, 0.1040,
0.7664, 0.8627, 0.5198, 0.5813, 0.6676, 0.4292, 0.9838, 0.8877, 0.0068,
0.0730, 0.9636, 0.5794, 0.4843, 0.7476])}
is_terminal: False
Converting constant ...
____
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 8
type: onnx::Constant
inputs: []
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'value': tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
is_terminal: False
Converting constant ...
____
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 9
type: onnx::InstanceNormalization
inputs: ['6', '7', '8']
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'epsilon': 1e-05}
is_terminal: False
Converting instancenorm ...
Traceback (most recent call last):
File "pytorch2tf_instancenorm_test.py", line 45, in <module>
k_model = converter.pytorch_to_keras(model, input_var, [(3, 224, 224,)], verbose=True)
File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 316, in pytorch_to_keras
names
File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/layers.py", line 653, in convert_instancenorm
layers[scope_name] = lambda_layer(layers[inputs[0]])
File "/home/myang/py3env/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 457, in __call__
output = self.call(inputs, **kwargs)
File "/home/myang/py3env/local/lib/python3.6/site-packages/keras/layers/core.py", line 687, in call
return self.function(inputs, **arguments)
File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/layers.py", line 646, in target_layer
param_initializers={'beta': tf.constant_initializer(beta), 'gamma': tf.constant_initializer(gamma)},
File "/home/myang/py3env/local/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py", line 208, in __init__
"tuple of values, or numpy.ndarray)." % type(value))
TypeError: Invalid type for initial value: <class 'tensorflow.python.framework.ops.Tensor'> (expected Python scalar, list or tuple of values, or numpy.ndarray).
The code is an excerpt from PyTorch Fast Style Transfer. Is this related to #37? Really trying to get it to work...
Hi, once again thanks for awesome work.
It really helps with shortening pytorch to production path (pytorch=>keras=>tf).
I was running my pytorch model based on inception4 architecture.
I encountered this error during the run:
ValueError: Unsuported padding size for convolution
I guess this is due to my architecture being an inception4 architecture - it has non-symmetric filters and this is solved differently in keras and pytorch. In keras it is solved via 'same' convolutions, in pytorch - via different paddings. Investigating deeper - I indeed found out that conv layers with assymmetric padding in pytorch are the culprits.
Just compare these links:
I overcame this by essentially a hack, but maybe you will give some advice / commit on how to do it properly?
In a nutshell I did this.
I can do a PR with this change, if you want.
if node.padding[0] != node.padding[1]:
# originally this line was not commented
# raise ValueError('Unsuported padding size for convolution')
# quick fix for inception architectures
# refer here for more info https://github.com/fchollet/keras/blob/master/keras/applications/inception_v3.py
border_mode = 'same'
else:
# this code initially was under no condition
padding = node.padding[0]
if padding > 0:
padding_name = output_name + '_pad'
padding_layer = keras.layers.ZeroPadding2D(
padding=node.padding,
name=padding_name
)
layers[padding_name] = padding_layer(layers[input_name])
input_name = padding_name
# this line below also was applied unconditionally
border_mode = 'valid'
Anyway - which proper solution would you suggest for this edge case?
Since my forward function in PyTorch has three inputs, I was wondering if the pytorch2keras converter can handle multiple variables? Does the converter support tuple inputs?
For example:
x is a (48,15)
y is (48,)
z is a tuple of two of these --> (2, 48, 128)
from converter import pytorch_to_keras
---> we should specify shape of the input tensor
k_model = pytorch_to_keras(model, (x, y, z), (???), verbose=True)
please,whether batchnorm1d is supported??
File "/usr/local/lib/python3.5/dist-packages/pytorch2keras/reshape_layers.py", line 122, in target_layer
return tf.squeeze(x, axis=axis)
NameError: name 'tf' is not defined
why happen to it,please!
Hi,
I followed the instruction to get keras model from a torch model. It is (probably) successful, and I save it and everything till here is just fine.
However, when I try to load again the saved keras model, I get the following error:
Using TensorFlow backend.
2018-12-13 11:49:04.256785: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "D:/Github packages/deep-head-pose-master-python3/deep-head-pose-master/core2/testing.py", line 4, in <module>
model = keras.models.load_model('keras_model.h5')
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 419, in load_model
model = _deserialize_model(f, custom_objects, compile)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 225, in _deserialize_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 458, in model_from_config return deserialize(config, custom_objects=custom_objects)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\layers\__init__.py", line 55, in deserialize printable_module_name='layer')
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\utils\generic_utils.py", line 145, in deserialize_keras_object list(custom_objects.items())))
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\network.py", line 1032, in from_config process_node(layer, node_data)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\network.py", line 991, in process_node layer(unpack_singleton(input_tensors), **kwargs)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\base_layer.py", line 457, in __call__ output = self.call(inputs, **kwargs)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\layers\core.py", line 687, in call return self.function(inputs, **arguments)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\pytorch2keras-0.1.12-py3.6.egg\pytorch2keras\layers.py", line 1330, in target_layer def target_layer(x):
NameError: name 'keras' is not defined
Any idea?
I have encounterd a new problem when I tried to run the code as follows:
import torch.nn as nn
from pytorch2keras.converter import pytorch_to_keras
import numpy as np
import torch
from torch.autograd import Variable
class CifarNet(nn.Module):
def __init__(self):
super(CifarNet, self).__init__()
self.conv1 = nn.Conv2d(3, 64, kernel_size=3)
self.conv2 = nn.Conv2d(64, 64, kernel_size=3)
self.conv3 = nn.Conv2d(64, 128, kernel_size=3)
self.conv4 = nn.Conv2d(128, 128, kernel_size=3)
self.pool = nn.MaxPool2d(2, 2)
self.relu = nn.ReLU(inplace=True)
self.fc1 = nn.Linear(3200, 256)
self.dropout = nn.Dropout(0.5)
self.fc2 = nn.Linear(256, 256)
self.fc3 = nn.Linear(256, 10)
def forward(self, x):
x = self.relu(self.conv1(x))
x = self.relu(self.conv2(x))
x = self.pool(x)
x = self.relu(self.conv3(x))
x = self.relu(self.conv4(x))
x = self.pool(x)
x = x.view(-1, 3200)
x = self.relu(self.fc1(x))
x = self.dropout(x)
x = self.relu(self.fc2(x))
x = self.fc3(x)
return x
# Load the trained model from file
trained_model = CifarNet()
trained_model.load_state_dict(torch.load('results/cifar_best.pth'))
input_np = np.random.uniform(0, 1, (1, 3, 32, 32))
input_var = Variable(torch.FloatTensor(input_np))
k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)], verbose=True)
which reported as follows:
Traceback (most recent call last):
File "pytorch4keras.py", line 40, in <module>
k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)], verbose=True)
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/converter.py", line 332, in pytorch_to_keras
names
File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/linear_layers.py", line 34, in convert_gemm
W = weights[weights_name].numpy().transpose()
KeyError: 'dropout.weight'
Is the dropout layer not include in the converter framework ?
Describe the bug
Multiple linear layers with multiple inputs do not work. When the input shapes are identical, it works (e.g both inputs expect 5 units or both expect 10 units, but when one is 5 and the other is 10 it does not work).
To Reproduce
class LayerTest(nn.Module):
def __init__(self):
super().__init__()
self.fc1 = nn.Linear(10, 10)
self.fc2 = nn.Linear(5, 10)
def forward(self, x, y):
x = self.fc1(x)
y = self.fc2(y)
return x + y
model = LayerTest().eval()
input_x = torch.FloatTensor(np.random.uniform(0, 1, (1, 10)))
input_y = torch.FloatTensor(np.random.uniform(0, 1, (1, 5)))
output = model(input_x, input_y)
k_model = pytorch_to_keras(model, [input_x, input_y], [(10,), (5,)], verbose=True)
Expected behavior
This should convert properly.
Logs
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/henry/code/altum/model-trainer/export/export.py", line 54, in <module>
k_model = pytorch_to_keras(model, [input_x, input_y], [(10,), (5,)], verbose=True)
File "/usr/local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 332, in pytorch_to_keras
names
File "/usr/local/lib/python3.6/site-packages/pytorch2keras/linear_layers.py", line 49, in convert_gemm
layers[scope_name] = dense(layers[inputs[0]])
File "/usr/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 436, in __call__
self.set_weights(self._initial_weights)
File "/usr/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 1057, in set_weights
'provided weight shape ' + str(w.shape))
ValueError: Layer weight shape (5, 10) not compatible with provided weight shape (10, 10)
Using TensorFlow backend.
Traceback (most recent call last):
File "pytorch_fpn-res2keras-tf.py", line 18, in
pytorch_model = torch.load('./_0620_80000.pth')
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/serialization.py", line 303, in load
return _load(f, map_location, pickle_module)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/serialization.py", line 469, in _load
result = unpickler.load()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1124: ordinal not in range(128)
@nerox8664 would u mind sharing all of your environment? I mean, for example, numpy'version included. The repo work well about a mouth ago, but now, it doesn't work. And, I have upgraded something for the past a mouth.
Thanks.
looking forward to any replies.
Is it possible to convert model from channels_first to channels_last(default Tensorflow tensor dim ordering)?
When I convert the customized network from Pytorch to Keras-tensorflow, the error is following:
Traceback (most recent call last):
File "pytorch_fpn-res2keras-tf.py", line 26, in
keras_model = pytorch_to_keras(pytorch_model, input_var, (3, 384, 384), verbose=True)
File "/data/xxxx/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 160, in pytorch_to_keras
AVAILABLE_CONVERTERS[node_type](
KeyError: u'prim::PythonOp'
@nerox8664 @justasbr
Looking forward to any replies. Thanks
When I try the tests/multiple_inputs.py
, it reports the following errors (I use pytorch 0.4):
TypeError: forward() missing 2 required positional arguments: 'y' and 'z'
What could be the problem? Thanks.
I was converting a model which using ResNet50 as backbone following by some slicing and avg pool and conv operation, but the problem is that it seems the "onnx::Slice" layer is not supported in the current version therefore results in the converting error. So I'd like to know if you are going to add the functionality for converting the slice layer?
I've added batchnorm layer to your dummy example and it crushed
the code:
from keras import backend as K
K.set_image_data_format('channels_first')
from pytorch2keras.converter import pytorch_to_keras
import torch
import torch.nn as nn
import numpy as np
from torch.autograd import Variable
class TestConv2d(nn.Module):
def __init__(self, inp=10, out=16, kernel_size=3):
super(TestConv2d, self).__init__()
self.conv2d = nn.Conv2d(inp, out, stride=1, kernel_size=kernel_size, bias=True)
self.bn_1 = nn.BatchNorm2d(num_features = out)
def forward(self, x):
x = self.conv2d(x)
x = self.bn_1(x)
return x
model = TestConv2d()
input_np = np.random.uniform(0, 1, (1, 10, 32, 32))
input_var = Variable(torch.FloatTensor(input_np))
# we should specify shape of the input tensor
k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True)
the error:
graph(%0 : Float(1, 10, 32, 32)
%1 : Float(16, 10, 3, 3)
%2 : Float(16)
%3 : Float(16)
%4 : Float(16)
%5 : Float(16)
%6 : Float(16)
%7 : Long()) {
%8 : Float(1, 16, 30, 30) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[0, 0, 0, 0], strides=[1, 1]](%0, %1, %2), scope: TestConv2d/Conv2d[conv2d]
%9 : Float(1, 16, 30, 30) = onnx::BatchNormalization[epsilon=1e-05, is_test=1, momentum=1](%8, %3, %4, %5, %6), scope: TestConv2d/BatchNorm2d[bn_1]
return (%9);
}
Graph inputs: ['0', '1', '2', '3', '4', '5', '6', '7']
Graph outputs: ['9']
State dict: ['conv2d.weight', 'conv2d.bias', 'bn_1.weight', 'bn_1.bias', 'bn_1.running_mean', 'bn_1.running_var', 'bn_1.num_batches_tracked']
____
graph node: TestConv2d/Conv2d[conv2d]
node id: 8
type: onnx::Conv
inputs: ['0', '1', '2']
outputs: ['TestConv2d/Conv2d[conv2d]']
name in state_dict: conv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [3, 3], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting convolution ...
____
graph node: TestConv2d/BatchNorm2d[bn_1]
node id: 9
type: onnx::BatchNormalization
inputs: ['8', '3', '4', '5', '6']
outputs: ['TestConv2d/BatchNorm2d[bn_1]']
name in state_dict: bn_1
attrs: {'epsilon': 1e-05, 'is_test': 1, 'momentum': 1.0}
is_terminal: True
Converting batchnorm ...
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
1575 try:
-> 1576 c_op = c_api.TF_FinishOperation(op_desc)
1577 except errors.InvalidArgumentError as e:
InvalidArgumentError: Shape must be rank 1 but is rank 0 for 'bn_10.9351925092536912/cond/Reshape_4' (op: 'Reshape') with input shapes: [1,16,1,1], [].
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-1-7f985261f348> in <module>()
24
25 # we should specify shape of the input tensor
---> 26 k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True)
~\Anaconda3\lib\site-packages\pytorch2keras\converter.py in pytorch_to_keras(model, args, input_shapes, change_ordering, training, verbose, names)
313 node_input_names,
314 layers, state_dict,
--> 315 names
316 )
317 if node_id in graph_outputs:
~\Anaconda3\lib\site-packages\pytorch2keras\normalization_layers.py in convert_batchnorm(params, w_name, scope_name, inputs, layers, weights, names)
59 name=tf_name
60 )
---> 61 layers[scope_name] = bn(layers[inputs[0]])
62
63
~\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
455 # Actually call the layer,
456 # collecting output(s), mask(s), and shape(s).
--> 457 output = self.call(inputs, **kwargs)
458 output_mask = self.compute_mask(inputs, previous_mask)
459
~\Anaconda3\lib\site-packages\keras\layers\normalization.py in call(self, inputs, training)
204 return K.in_train_phase(normed_training,
205 normalize_inference,
--> 206 training=training)
207
208 def get_config(self):
~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in in_train_phase(x, alt, training)
3121
3122 # else: assume learning phase is a placeholder tensor.
-> 3123 x = switch(training, x, alt)
3124 if uses_learning_phase:
3125 x._uses_learning_phase = True
~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in switch(condition, then_expression, else_expression)
3056 x = tf.cond(condition,
3057 then_expression_fn,
-> 3058 else_expression_fn)
3059 else:
3060 # tf.where needs its condition tensor
~\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py in new_func(*args, **kwargs)
452 'in a future version' if date is None else ('after %s' % date),
453 instructions)
--> 454 return func(*args, **kwargs)
455 return tf_decorator.make_decorator(func, new_func, 'deprecated',
456 _add_deprecated_arg_notice_to_docstring(
~\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py in cond(pred, true_fn, false_fn, strict, name, fn1, fn2)
2055 context_f = CondContext(pred, pivot_2, branch=0)
2056 context_f.Enter()
-> 2057 orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
2058 if orig_res_f is None:
2059 raise ValueError("false_fn must have a return value.")
~\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py in BuildCondBranch(self, fn)
1893 """Add the subgraph defined by fn() to the graph."""
1894 pre_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION) # pylint: disable=protected-access
-> 1895 original_result = fn()
1896 post_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION) # pylint: disable=protected-access
1897 if len(post_summaries) > len(pre_summaries):
~\Anaconda3\lib\site-packages\keras\layers\normalization.py in normalize_inference()
165 broadcast_gamma,
166 axis=self.axis,
--> 167 epsilon=self.epsilon)
168 else:
169 return K.batch_normalization(
~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in batch_normalization(x, mean, var, beta, gamma, axis, epsilon)
1906 # so it may have extra axes with 1, it is not needed and should be removed
1907 if ndim(mean) > 1:
-> 1908 mean = tf.reshape(mean, (-1))
1909 if ndim(var) > 1:
1910 var = tf.reshape(var, (-1))
~\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py in reshape(tensor, shape, name)
7432 if _ctx is None or not _ctx._eager_context.is_eager:
7433 _, _, _op = _op_def_lib._apply_op_helper(
-> 7434 "Reshape", tensor=tensor, shape=shape, name=name)
7435 _result = _op.outputs[:]
7436 _inputs_flat = _op.inputs
~\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
785 op = g.create_op(op_type_name, inputs, output_types, name=scope,
786 input_types=input_types, attrs=attr_protos,
--> 787 op_def=op_def)
788 return output_structure, op_def.is_stateful, op
789
~\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py in new_func(*args, **kwargs)
452 'in a future version' if date is None else ('after %s' % date),
453 instructions)
--> 454 return func(*args, **kwargs)
455 return tf_decorator.make_decorator(func, new_func, 'deprecated',
456 _add_deprecated_arg_notice_to_docstring(
~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in create_op(***failed resolving arguments***)
3153 input_types=input_types,
3154 original_op=self._default_original_op,
-> 3155 op_def=op_def)
3156 self._create_op_helper(ret, compute_device=compute_device)
3157 return ret
~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in __init__(self, node_def, g, inputs, output_types, control_inputs, input_types, original_op, op_def)
1729 op_def, inputs, node_def.attr)
1730 self._c_op = _create_c_op(self._graph, node_def, grouped_inputs,
-> 1731 control_input_ops)
1732
1733 # Initialize self._outputs.
~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
1577 except errors.InvalidArgumentError as e:
1578 # Convert to ValueError for backwards compatibility.
-> 1579 raise ValueError(str(e))
1580
1581 return c_op
ValueError: Shape must be rank 1 but is rank 0 for 'bn_10.9351925092536912/cond/Reshape_4' (op: 'Reshape') with input shapes: [1,16,1,1], [].
Ran /tests/resnet50.py and /tests/alexnet.py
For resnet50 I got this error
1.88754e+08
Max error: 1079778560.0
For alexnet I got this error
0.062452
Max error: 0.11586721986532211
Is this expected behavior, or is there anything wrong with my versions / set-up?
I am using Pytorch v0.4.1.post2. When executing my code, I get an ImportError saying that torch._C.ListType does not exist.
The error happens in converter.py. Basically the conditions used if version.parse('0.4.1') < version.parse(torch.__version__)
should only be true if the user is using Pytorch>=1.0.0 but are also true for versions v0.4.1.x.
I managed to work around this by replacing if version.parse('0.4.1') < version.parse(torch.__version__)
with if version.parse('1.0.0') <= version.parse(torch.__version__)
. I can PR if you want.
First of all thanks for this amazing converter, great jog ;)
So I have latest versions of pytorch, keras and this converter. And it seems like batch normalization layer isn't converted properly. Do you know why? Do you have the same issue?)
While running resnet18.py in tests/ I encountered the
KeyError: 'Layer is not currently supported.'
So is there any other way to convert resnet18 model from torch to keras?
Describe the bug
I am trying to convert a pytorch model that has a PixelShuffle layer.
The layer is converted as a Lambda layer, but unfortunately, it doesn't work properly.
Please note the model summary below where the PixelShuffle is supposed to get a (None, 256, 256, 256) tensor as its input and output a (None, 64, 512, 512) tensor. The PixelShuffle is translated to three steps (Lambda-Permute-Lambda). However, for some reason, the channels are manipulated (moved to the middle, last and then first again), which distorts the output image. Please note that the "image_data_format" of the keras is "channels_first".
To Reproduce
The model that I am trying to convert is from this repo:
https://github.com/nmhkahn/CARN-pytorch
To reproduce the workflow, please substitute the main function in sample.py with:
def main(cfg):
module = importlib.import_module("model.{}".format(cfg.model)) #it imports model.carn.py
net = module.Net(multi_scale=True, group = cfg.group)
print(json.dumps(vars(cfg), indent=4, sort_keys=True))
#state_dict = torch.load(cfg.ckpt_path)
state_dict = torch.load(cfg.ckpt_path, map_location='cpu')
new_state_dict = OrderedDict()
for k, v in state_dict.items():
name = k
new_state_dict[name] = v
net.load_state_dict(new_state_dict)
scale = cfg.scale
# device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device = torch.device("cpu")
net = net.to(device)
model = net
print(model)
# keras
input_np = np.random.uniform(0, 1, (1, 3, 256, 256))
input_var = Variable(torch.FloatTensor(input_np))
k_model = pytorch_to_keras(model, input_var, [(3, 256, 256,)], verbose=True)
k_model.summary(line_length=170)
input_img = '../data/img_001_SRF_2_LR.png'
img = Image.open(input_img).convert("RGB")
image = np.array(img, dtype=np.float32)[...,:3]
image = np.rollaxis(image, 2, 0) # channel first
image = image.astype(np.float32) / 255 # normalization
input_image = np.expand_dims(image, axis=0)
print("Image 1 shape: ", input_image.shape)
prediction = k_model.predict(input_image)
prediction = prediction[0]
print("prediction: ", prediction.shape)
prediction = np.rollaxis(prediction, 1, 0)
prediction = np.rollaxis(prediction, 2, 1)
prediction = prediction * 255 # denormalization
print("prediction: ", prediction.shape)
output_path = '../data/img_001_SRF_2_SR_keras.png'
print("min", np.min(prediction))
print("max", np.max(prediction))
result = Image.fromarray((prediction).astype(np.uint8))
result.save(output_path)
#k_model.save('my_model.h5')
Expected behavior
We are looking for a super-sesolution of the input image. The conversion of the PixelShuffle layer is supposed to get a (None, 256, 256, 256) tensor as its input and output a (None, 64, 512, 512). That is, channel will be divided by 4 and the width and height multiplied by 2.
Today I was struggling with converting my inception-based model.
Apart from the padding hack, I noticed a potential feature for your library.
You assume that only 2 inputs are concatenated into 1, but it case of my model I have 2 and 4. I guess in a more general case you may consider including a proper variation of the below modification into future versions:
def convert_concat(node, node_name, input_names, output_name, layers):
"""
Convert concatenation.
Args:
node: pytorch node element.
node_name: pytorch node name
input_name: pytorch input node name
output_name: pytorch output node name
layers: dictionary with keras tensors
"""
print('Conerting concat ...')
# basically a very shitty hack because my model contains only 2 or 4 concats
if len(input_names)==2:
model0 = layers[input_names[0]]
model1 = layers[input_names[1]]
cat = keras.layers.Concatenate(name=output_name, axis=node.dim)
layers[output_name] = cat([model0, model1])
elif len(input_names)==4:
model0 = layers[input_names[0]]
model1 = layers[input_names[1]]
model2 = layers[input_names[2]]
model3 = layers[input_names[3]]
cat = keras.layers.Concatenate(name=output_name, axis=node.dim)
layers[output_name] = cat([model0, model1, model2, model3])
Describe the bug
I am trying to convert a simple Pytorch Gender Classifier to Keras (model is provided below). Whenever I run model = pytorch_to_keras(model, orig_img, [np.shape(orig_img.cpu().numpy())], names='short')
, I get
Traceback (most recent call last):
File "main.py", line 154, in
main(args)
File "main.py", line 92, in main
model = pytorch_to_keras(model, orig_img, [np.shape(orig_img.cpu().numpy())], names='short')
File "/mnt/c/Users/Yannis/CIFAR10S/real_world_experiments/CEM/venv/lib/python3.6/site-packages/pytorch2keras/converter.py", line 325, in pytorch_to_keras
names
File "/mnt/c/Users/Yannis/CIFAR10S/real_world_experiments/CEM/venv/lib/python3.6/site-packages/pytorch2keras/convolution_layers.py", line 35, in convert_conv
if len(weights[weights_name].numpy().shape) == 5: # 3D conv
KeyError: 'module.weight'
To Reproduce
Here is my pytorch model:
class resnet_modified_small(nn.Module):
def base_size(self): return 512
def rep_size(self): return 1024
def __init__(self, n_classes):
super(resnet_modified_small, self).__init__()
self.resnet = tv.models.resnet34(pretrained=True)
# define layers
self.n_classes = n_classes
self.linear = nn.Linear(7 * 7 * self.base_size(), self.rep_size())
self.cls = nn.Linear(self.rep_size(), self.n_classes)
self.dropout2d = nn.Dropout2d(.5)
self.dropout = nn.Dropout(.5)
self.relu = nn.LeakyReLU()
initLinear(self.linear)
def forward(self, out0):
x = self.resnet.conv1(out0)
x = self.resnet.bn1(x)
x = self.resnet.relu(x)
out1 = self.resnet.maxpool(x)
out2 = self.resnet.layer1(out1)
out3 = self.resnet.layer2(out2)
out4 = self.resnet.layer3(out3)
out5 = self.resnet.layer4(out4)
x = self.dropout2d(out5)
features = self.dropout(self.relu(self.linear(x.view(-1, 7*7*self.base_size()))))
cls_scores = self.cls(features)
return [out0, out1, out2, out3, out4, out5, features, cls_scores]
Environment:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.