Comments (12)
jiaxiang-wu
commented on May 22, 2024
Hi, could you please post the full log file?
from pocketflow.
sunzhe09
commented on May 22, 2024
./scripts/run_seven.sh nets/resnet_at_cifar10_run.py
--learner channel
--batch_size_eval 64
--cp_uniform_preserve_ratio 0.5
--cp_prune_option uniform
--resnet_size 20
Python script: nets/resnet_at_cifar10_run.py
of GPUs: 1
extra arguments: --learner channel --batch_size_eval 64 --cp_preserve_ratio 0.5 --cp_prune_option auto --resnet_size 20
['data_hdfs_host', 'None']
['data_dir_local_cifar10', '/data/raid/cifar-10-batches-bin']
['data_dir_hdfs_cifar10', 'None']
['data_dir_seven_cifar10', 'None']
data_dir_docker_cifar10 = /opt/ml/data
['data_dir_docker_cifar10', '/opt/ml/data']
['data_dir_local_ilsvrc12', 'None']
['data_dir_hdfs_ilsvrc12', 'None']
['data_dir_seven_ilsvrc12', 'None']
data_dir_docker_ilsvrc12 = /opt/ml/data
['data_dir_docker_ilsvrc12', '/opt/ml/data']
['model_http_url', 'https://api.ai.tencent.com/pocketflow']
--model_http_url https://api.ai.tencent.com/pocketflow --data_dir_local /data/raid/cifar-10-batches-bin
'nets/resnet_at_cifar10_run.py' -> 'main.py'
multi-GPU training disabled
[WARNING] TF-Plus & Horovod cannot be imported; multi-GPU training is unsupported
INFO:tensorflow:FLAGS:
INFO:tensorflow:data_disk: local
INFO:tensorflow:data_hdfs_host: None
INFO:tensorflow:data_dir_local: /data/raid/cifar-10-batches-bin
INFO:tensorflow:data_dir_hdfs: None
INFO:tensorflow:cycle_length: 4
INFO:tensorflow:nb_threads: 8
INFO:tensorflow:buffer_size: 1024
INFO:tensorflow:prefetch_size: 8
INFO:tensorflow:nb_classes: 10
INFO:tensorflow:nb_smpls_train: 50000
INFO:tensorflow:nb_smpls_val: 5000
INFO:tensorflow:nb_smpls_eval: 10000
INFO:tensorflow:batch_size: 128
INFO:tensorflow:batch_size_eval: 64
INFO:tensorflow:resnet_size: 20
INFO:tensorflow:lrn_rate_init: 0.1
INFO:tensorflow:batch_size_norm: 128.0
INFO:tensorflow:momentum: 0.9
INFO:tensorflow:loss_w_dcy: 0.0002
INFO:tensorflow:model_http_url: https://api.ai.tencent.com/pocketflow
INFO:tensorflow:summ_step: 100
INFO:tensorflow:save_step: 10000
INFO:tensorflow:save_path: ./models/model.ckpt
INFO:tensorflow:save_path_eval: ./models_eval/model.ckpt
INFO:tensorflow:enbl_dst: False
INFO:tensorflow:enbl_warm_start: False
INFO:tensorflow:loss_w_dst: 4.0
INFO:tensorflow:tempr_dst: 4.0
INFO:tensorflow:save_path_dst: ./models_dst/model.ckpt
INFO:tensorflow:nb_epochs_rat: 1.0
INFO:tensorflow:ddpg_actor_depth: 2
INFO:tensorflow:ddpg_actor_width: 64
INFO:tensorflow:ddpg_critic_depth: 2
INFO:tensorflow:ddpg_critic_width: 64
INFO:tensorflow:ddpg_noise_type: param
INFO:tensorflow:ddpg_noise_prtl: tdecy
INFO:tensorflow:ddpg_noise_std_init: 1.0
INFO:tensorflow:ddpg_noise_dst_finl: 0.01
INFO:tensorflow:ddpg_noise_adpt_rat: 1.03
INFO:tensorflow:ddpg_noise_std_finl: 1e-05
INFO:tensorflow:ddpg_rms_eps: 0.0001
INFO:tensorflow:ddpg_tau: 0.01
INFO:tensorflow:ddpg_gamma: 0.9
INFO:tensorflow:ddpg_lrn_rate: 0.001
INFO:tensorflow:ddpg_loss_w_dcy: 0.0
INFO:tensorflow:ddpg_record_step: 1
INFO:tensorflow:ddpg_batch_size: 64
INFO:tensorflow:ddpg_enbl_bsln_func: True
INFO:tensorflow:ddpg_bsln_decy_rate: 0.95
INFO:tensorflow:ws_save_path: ./models_ws/model.ckpt
INFO:tensorflow:ws_prune_ratio: 0.75
INFO:tensorflow:ws_prune_ratio_prtl: optimal
INFO:tensorflow:ws_nb_rlouts: 200
INFO:tensorflow:ws_nb_rlouts_min: 50
INFO:tensorflow:ws_reward_type: single-obj
INFO:tensorflow:ws_lrn_rate_rg: 0.03
INFO:tensorflow:ws_nb_iters_rg: 20
INFO:tensorflow:ws_lrn_rate_ft: 0.0003
INFO:tensorflow:ws_nb_iters_ft: 400
INFO:tensorflow:ws_nb_iters_feval: 25
INFO:tensorflow:ws_prune_ratio_exp: 3.0
INFO:tensorflow:ws_iter_ratio_beg: 0.1
INFO:tensorflow:ws_iter_ratio_end: 0.5
INFO:tensorflow:ws_mask_update_step: 500.0
INFO:tensorflow:cp_lasso: True
INFO:tensorflow:cp_quadruple: False
INFO:tensorflow:cp_reward_policy: accuracy
INFO:tensorflow:cp_nb_points_per_layer: 10
INFO:tensorflow:cp_nb_batches: 60
INFO:tensorflow:cp_prune_option: auto
INFO:tensorflow:cp_prune_list_file: ratio.list
INFO:tensorflow:cp_channel_pruned_path: ./models/pruned_model.ckpt
INFO:tensorflow:cp_best_path: ./models/best_model.ckpt
INFO:tensorflow:cp_original_path: ./models/original_model.ckpt
INFO:tensorflow:cp_preserve_ratio: 0.5
INFO:tensorflow:cp_uniform_preserve_ratio: 0.6
INFO:tensorflow:cp_noise_tolerance: 0.15
INFO:tensorflow:cp_lrn_rate_ft: 0.0001
INFO:tensorflow:cp_nb_iters_ft_ratio: 0.2
INFO:tensorflow:cp_finetune: False
INFO:tensorflow:cp_retrain: False
INFO:tensorflow:cp_list_group: 1000
INFO:tensorflow:cp_nb_rlouts: 200
INFO:tensorflow:cp_nb_rlouts_min: 50
INFO:tensorflow:dcp_save_path: ./models_dcp/model.ckpt
INFO:tensorflow:dcp_save_path_eval: ./models_dcp_eval/model.ckpt
INFO:tensorflow:dcp_prune_ratio: 0.5
INFO:tensorflow:dcp_nb_stages: 3
INFO:tensorflow:dcp_lrn_rate_adam: 0.001
INFO:tensorflow:dcp_nb_iters_block: 10000
INFO:tensorflow:dcp_nb_iters_layer: 500
INFO:tensorflow:uql_equivalent_bits: 4
INFO:tensorflow:uql_nb_rlouts: 200
INFO:tensorflow:uql_w_bit_min: 2
INFO:tensorflow:uql_w_bit_max: 8
INFO:tensorflow:uql_tune_layerwise_steps: 100
INFO:tensorflow:uql_tune_global_steps: 2000
INFO:tensorflow:uql_tune_save_path: ./rl_tune_models/model.ckpt
INFO:tensorflow:uql_tune_disp_steps: 300
INFO:tensorflow:uql_enbl_random_layers: True
INFO:tensorflow:uql_enbl_rl_agent: False
INFO:tensorflow:uql_enbl_rl_global_tune: True
INFO:tensorflow:uql_enbl_rl_layerwise_tune: False
INFO:tensorflow:uql_weight_bits: 4
INFO:tensorflow:uql_activation_bits: 32
INFO:tensorflow:uql_use_buckets: False
INFO:tensorflow:uql_bucket_size: 256
INFO:tensorflow:uql_quant_epochs: 60
INFO:tensorflow:uql_save_quant_model_path: ./uql_quant_models/uql_quant_model.ckpt
INFO:tensorflow:uql_quantize_all_layers: False
INFO:tensorflow:uql_bucket_type: channel
INFO:tensorflow:uqtf_save_path: ./models_uqtf/model.ckpt
INFO:tensorflow:uqtf_save_path_eval: ./models_uqtf_eval/model.ckpt
INFO:tensorflow:uqtf_weight_bits: 8
INFO:tensorflow:uqtf_activation_bits: 8
INFO:tensorflow:uqtf_quant_delay: 0
INFO:tensorflow:uqtf_freeze_bn_delay: None
INFO:tensorflow:uqtf_lrn_rate_dcy: 0.01
INFO:tensorflow:nuql_equivalent_bits: 4
INFO:tensorflow:nuql_nb_rlouts: 200
INFO:tensorflow:nuql_w_bit_min: 2
INFO:tensorflow:nuql_w_bit_max: 8
INFO:tensorflow:nuql_tune_layerwise_steps: 100
INFO:tensorflow:nuql_tune_global_steps: 2101
INFO:tensorflow:nuql_tune_save_path: ./rl_tune_models/model.ckpt
INFO:tensorflow:nuql_tune_disp_steps: 300
INFO:tensorflow:nuql_enbl_random_layers: True
INFO:tensorflow:nuql_enbl_rl_agent: False
INFO:tensorflow:nuql_enbl_rl_global_tune: True
INFO:tensorflow:nuql_enbl_rl_layerwise_tune: False
INFO:tensorflow:nuql_init_style: quantile
INFO:tensorflow:nuql_opt_mode: weights
INFO:tensorflow:nuql_weight_bits: 4
INFO:tensorflow:nuql_activation_bits: 32
INFO:tensorflow:nuql_use_buckets: False
INFO:tensorflow:nuql_bucket_size: 256
INFO:tensorflow:nuql_quant_epochs: 60
INFO:tensorflow:nuql_save_quant_model_path: ./nuql_quant_models/model.ckpt
INFO:tensorflow:nuql_quantize_all_layers: False
INFO:tensorflow:nuql_bucket_type: split
INFO:tensorflow:log_dir: ./logs
INFO:tensorflow:enbl_multi_gpu: False
INFO:tensorflow:learner: channel
INFO:tensorflow:exec_mode: train
INFO:tensorflow:debug: False
INFO:tensorflow:h: False
INFO:tensorflow:help: False
INFO:tensorflow:helpfull: False
INFO:tensorflow:helpshort: False
2018-11-13 11:58:27.640676: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-11-13 11:58:28.354727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties:
name: Tesla V100-SXM2-16GB major: 7 minor: 0 memoryClockRate(GHz): 1.53
pciBusID: 0000:06:00.0
totalMemory: 15.77GiB freeMemory: 15.36GiB
2018-11-13 11:58:28.354768: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-11-13 11:58:28.689319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-13 11:58:28.689359: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2018-11-13 11:58:28.689384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2018-11-13 11:58:28.689938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14863 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:06:00.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from ./models/original_model.ckpt
INFO:tensorflow:model restored from ./models/original_model.ckpt
2018-11-13 11:58:30.072571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-11-13 11:58:30.072636: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-13 11:58:30.072646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2018-11-13 11:58:30.072669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2018-11-13 11:58:30.072964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14863 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:06:00.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from ./models/original_model.ckpt
INFO:tensorflow:The current model flops is 41074688.0
INFO:tensorflow:The original model flops is 41074688.0
INFO:tensorflow:current conv model/resnet_model/conv2d_1/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_2/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_3/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_5/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_7/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_10/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_12/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_14/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_17/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_19/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_21/Conv2D
INFO:tensorflow:current states:
layer n c H W stride maxreduce layercomp
0 0.000000 0.25 0.046875 1.00 1.00 0.5 1.0 0.187500
1 0.047619 0.25 0.250000 1.00 1.00 0.5 1.0 0.111111
2 0.095238 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
3 0.142857 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
4 0.190476 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
5 0.238095 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
6 0.285714 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
7 0.333333 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
8 0.380952 0.50 0.250000 0.50 0.50 1.0 1.0 0.055556
9 0.428571 0.50 0.250000 0.50 0.50 1.0 1.0 0.500000
10 0.476190 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
11 0.523810 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
12 0.571429 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
13 0.619048 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
14 0.666667 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
15 0.714286 1.00 0.500000 0.25 0.25 1.0 1.0 0.055556
16 0.761905 1.00 0.500000 0.25 0.25 1.0 1.0 0.500000
17 0.809524 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
18 0.857143 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
19 0.904762 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
20 0.952381 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
21 1.000000 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
INFO:tensorflow:max_strategy_dict
{'model/resnet_model/conv2d/Conv2D': [1.0, 0.26413688858352186], 'model/resnet_model/conv2d_1/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_2/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_3/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_4/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_5/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_6/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_7/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_8/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_9/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_10/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_11/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_12/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_13/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_14/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_15/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_16/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_17/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_18/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_19/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_20/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_21/Conv2D': [1.0, 1.0]}
INFO:tensorflow:Start pruning
INFO:tensorflow:preserve lower bound: 0.26413688858352186, preserve ratio: 0.5, preserve upper bound: 1.0
2018-11-13 11:58:30.744687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-11-13 11:58:30.744740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-13 11:58:30.744749: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2018-11-13 11:58:30.744756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2018-11-13 11:58:30.745000: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14863 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:06:00.0, compute capability: 7.0)
2018-11-13 11:58:32.834985: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-11-13 11:58:32.835112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-11-13 11:58:32.835123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958] 0
2018-11-13 11:58:32.835153: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: N
2018-11-13 11:58:32.835447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14863 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:06:00.0, compute capability: 7.0)
INFO:tensorflow:Restoring parameters from ./models/original_model.ckpt
INFO:tensorflow:The current model flops is 41074688.0
INFO:tensorflow:The original model flops is 41074688.0
INFO:tensorflow:current conv model/resnet_model/conv2d_1/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_2/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_3/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_5/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_7/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_10/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_12/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_14/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_17/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_19/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_21/Conv2D
INFO:tensorflow:current states:
layer n c H W stride maxreduce layercomp
0 0.000000 0.25 0.046875 1.00 1.00 0.5 1.0 0.187500
1 0.047619 0.25 0.250000 1.00 1.00 0.5 1.0 0.111111
2 0.095238 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
3 0.142857 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
4 0.190476 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
5 0.238095 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
6 0.285714 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
7 0.333333 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
8 0.380952 0.50 0.250000 0.50 0.50 1.0 1.0 0.055556
9 0.428571 0.50 0.250000 0.50 0.50 1.0 1.0 0.500000
10 0.476190 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
11 0.523810 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
12 0.571429 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
13 0.619048 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
14 0.666667 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
15 0.714286 1.00 0.500000 0.25 0.25 1.0 1.0 0.055556
16 0.761905 1.00 0.500000 0.25 0.25 1.0 1.0 0.500000
17 0.809524 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
18 0.857143 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
19 0.904762 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
20 0.952381 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
21 1.000000 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
INFO:tensorflow:max_strategy_dict
{'model/resnet_model/conv2d/Conv2D': [1.0, 0.26413688858352186], 'model/resnet_model/conv2d_1/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_2/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_3/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_4/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_5/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_6/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_7/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_8/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_9/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_10/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_11/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_12/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_13/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_14/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_15/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_16/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_17/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_18/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_19/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_20/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_21/Conv2D': [1.0, 1.0]}
INFO:tensorflow:The current model flops is 41074688.0
INFO:tensorflow:The original model flops is 41074688.0
INFO:tensorflow:current conv model/resnet_model/conv2d_1/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_2/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_3/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_5/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_7/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_10/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_12/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_14/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_17/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_19/Conv2D
INFO:tensorflow:current conv model/resnet_model/conv2d_21/Conv2D
INFO:tensorflow:current states:
layer n c H W stride maxreduce layercomp
0 0.000000 0.25 0.046875 1.00 1.00 0.5 1.0 0.187500
1 0.047619 0.25 0.250000 1.00 1.00 0.5 1.0 0.111111
2 0.095238 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
3 0.142857 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
4 0.190476 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
5 0.238095 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
6 0.285714 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
7 0.333333 0.25 0.250000 1.00 1.00 0.5 1.0 1.000000
8 0.380952 0.50 0.250000 0.50 0.50 1.0 1.0 0.055556
9 0.428571 0.50 0.250000 0.50 0.50 1.0 1.0 0.500000
10 0.476190 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
11 0.523810 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
12 0.571429 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
13 0.619048 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
14 0.666667 0.50 0.500000 0.50 0.50 0.5 1.0 1.000000
15 0.714286 1.00 0.500000 0.25 0.25 1.0 1.0 0.055556
16 0.761905 1.00 0.500000 0.25 0.25 1.0 1.0 0.500000
17 0.809524 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
18 0.857143 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
19 0.904762 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
20 0.952381 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
21 1.000000 1.00 1.000000 0.25 0.25 0.5 1.0 1.000000
INFO:tensorflow:max_strategy_dict
{'model/resnet_model/conv2d/Conv2D': [1.0, 0.26413688858352186], 'model/resnet_model/conv2d_1/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_2/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_3/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_4/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_5/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_6/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_7/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_8/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_9/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_10/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_11/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_12/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_13/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_14/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_15/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_16/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_17/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_18/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_19/Conv2D': [0.26413688858352186, 1.0], 'model/resnet_model/conv2d_20/Conv2D': [0.26413688858352186, 0.26413688858352186], 'model/resnet_model/conv2d_21/Conv2D': [1.0, 1.0]}
INFO:tensorflow:state is [[ 0. 0.25 0.046875 1. 1. 0.5 1. 0.1875 ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.29616976]]
INFO:tensorflow:loss: 0.13431529700756073
INFO:tensorflow:accuracy: 0.9957031011581421
INFO:tensorflow:preserve ratio before constraint 1.0
INFO:tensorflow:preserve ratio after constraint 1.0
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d/Conv2D layer, the pruning rate is 1.0
INFO:tensorflow:Actural preserv ratio: 1.0
INFO:tensorflow:state is [[ 0.04761905 0.25 0.25 1. 1. 0.5
0.21708929 0.11111111]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26791376]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26791376]]
INFO:tensorflow:max_reduced_flops 8919535.687778298
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 704512.0
INFO:tensorflow:recommand action 16.758489238266367
INFO:tensorflow:preserve ratio after constraint 0.2679137587547302
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.05661497265100479s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.01574210822582245s
INFO:tensorflow:Prune model/resnet_model/conv2d_1/Conv2D c_in from 16 to 4
INFO:tensorflow:father conv model/resnet_model/conv2d/Conv2D
INFO:tensorflow:father conv input Tensor("data/Placeholder:0", shape=(128, 32, 32, 3), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_1/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.0952381 0.25 0.25 1. 1. 0.5
0.21715407 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.27830169]]
INFO:tensorflow:preserve ratio before constraint [[ 0.27830169]]
INFO:tensorflow:max_reduced_flops 8928262.167497106
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 1065545.1046875487
INFO:tensorflow:recommand action 11.173271576513201
INFO:tensorflow:preserve ratio after constraint 0.2783016860485077
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.04044293984770775s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.0465877465903759s
INFO:tensorflow:Prune model/resnet_model/conv2d_2/Conv2D c_in from 16 to 4
INFO:tensorflow:father conv model/resnet_model/conv2d/Conv2D
INFO:tensorflow:father conv input Tensor("data/Placeholder:0", shape=(128, 32, 32, 3), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_2/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.14285714 0.25 0.25 1. 1. 0.5
0.21736652 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.27800375]]
INFO:tensorflow:preserve ratio before constraint [[ 0.27800375]]
INFO:tensorflow:max_reduced_flops 8939000.489395529
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2949120.0
INFO:tensorflow:recommand action 4.2108187989589165
INFO:tensorflow:preserve ratio after constraint 0.2780037522315979
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.05030923709273338s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.06443603709340096s
INFO:tensorflow:Prune model/resnet_model/conv2d_3/Conv2D c_in from 16 to 4
INFO:tensorflow:father conv model/resnet_model/conv2d_2/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu:0", shape=(128, 32, 32, 16), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_3/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.19047619 0.25 0.25 1. 1. 0.5
0.21762796 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.27775574]]
INFO:tensorflow:preserve ratio before constraint [[ 0.27775574]]
INFO:tensorflow:max_reduced_flops 8864901.018329512
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.008294606384073
INFO:tensorflow:preserve ratio after constraint 0.2777557373046875
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.061224885284900665s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.05170450732111931s
INFO:tensorflow:Prune model/resnet_model/conv2d_4/Conv2D c_in from 16 to 4
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_4/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.23809524 0.25 0.25 1. 1. 0.5
0.21582394 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.27693129]]
INFO:tensorflow:preserve ratio before constraint [[ 0.27693129]]
INFO:tensorflow:max_reduced_flops 8885336.491208073
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2949120.0
INFO:tensorflow:recommand action 4.227942946554202
INFO:tensorflow:preserve ratio after constraint 0.2769312858581543
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.048485878854990005s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.051728636026382446s
INFO:tensorflow:Prune model/resnet_model/conv2d_5/Conv2D c_in from 16 to 4
INFO:tensorflow:father conv model/resnet_model/conv2d_4/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu_2:0", shape=(128, 32, 32, 16), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_5/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.28571429 0.25 0.25 1. 1. 0.5
0.21632146 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.27605718]]
INFO:tensorflow:preserve ratio before constraint [[ 0.27605718]]
INFO:tensorflow:max_reduced_flops 8813341.35248228
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.08933283095352
INFO:tensorflow:preserve ratio after constraint 0.2760571837425232
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.06321901082992554s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.0522107258439064s
INFO:tensorflow:Prune model/resnet_model/conv2d_6/Conv2D c_in from 16 to 4
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_6/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.33333333 0.25 0.25 1. 1. 0.5
0.21456867 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.2752302]]
INFO:tensorflow:preserve ratio before constraint [[ 0.2752302]]
INFO:tensorflow:max_reduced_flops 8829818.616067491
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2949120.0
INFO:tensorflow:recommand action 4.245067094149486
INFO:tensorflow:preserve ratio after constraint 0.27523019909858704
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.05189386010169983s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.056705135852098465s
INFO:tensorflow:Prune model/resnet_model/conv2d_7/Conv2D c_in from 16 to 4
INFO:tensorflow:father conv model/resnet_model/conv2d_6/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu_4:0", shape=(128, 32, 32, 16), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_7/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.38095238 0.5 0.25 0.5 0.5 1.
0.21496983 0.05555556]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26421079]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26421079]]
INFO:tensorflow:max_reduced_flops 8755421.417760199
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 131072.0
INFO:tensorflow:recommand action 90.15314650694695
INFO:tensorflow:preserve ratio after constraint 0.2642107903957367
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.041421305388212204s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.026172716170549393s
INFO:tensorflow:Prune model/resnet_model/conv2d_8/Conv2D c_in from 16 to 4
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_8/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.42857143 0.5 0.25 0.5 0.5 1.
0.21315856 0.5 ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26434961]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26434961]]
INFO:tensorflow:max_reduced_flops 8753625.062479207
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 311588.5523437744
INFO:tensorflow:recommand action 38.08255200788337
INFO:tensorflow:preserve ratio after constraint 0.2643496096134186
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.08397142961621284s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.1697128489613533s
INFO:tensorflow:Prune model/resnet_model/conv2d_9/Conv2D c_in from 16 to 4
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_9/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.47619048 0.5 0.5 0.5 0.5 0.5
0.21311483 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26510435]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26510435]]
INFO:tensorflow:max_reduced_flops 8751721.741322696
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2654208.0
INFO:tensorflow:recommand action 4.705458032822005
INFO:tensorflow:preserve ratio after constraint 0.26510435342788696
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.09935277700424194s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.14975175634026527s
INFO:tensorflow:Prune model/resnet_model/conv2d_10/Conv2D c_in from 32 to 8
INFO:tensorflow:father conv model/resnet_model/conv2d_9/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Pad_1:0", shape=(128, 34, 34, 16), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_10/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.52380952 0.5 0.5 0.5 0.5 0.5
0.21306849 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26520222]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26520222]]
INFO:tensorflow:max_reduced_flops 8712295.538418598
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.24062409491254
INFO:tensorflow:preserve ratio after constraint 0.26520222425460815
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.1162053756415844s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.1397826075553894s
INFO:tensorflow:Prune model/resnet_model/conv2d_11/Conv2D c_in from 32 to 8
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_11/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.57142857 0.5 0.5 0.5 0.5 0.5
0.21210862 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26530716]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26530716]]
INFO:tensorflow:max_reduced_flops 8706273.12594774
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2949120.0
INFO:tensorflow:recommand action 4.277036377135089
INFO:tensorflow:preserve ratio after constraint 0.26530715823173523
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.0834873877465725s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.16417166590690613s
INFO:tensorflow:Prune model/resnet_model/conv2d_12/Conv2D c_in from 32 to 8
INFO:tensorflow:father conv model/resnet_model/conv2d_11/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu_8:0", shape=(128, 16, 16, 32), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_12/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.61904762 0.5 0.5 0.5 0.5 0.5
0.211962 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26541156]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26541156]]
INFO:tensorflow:max_reduced_flops 8661924.822868243
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.32166231948198
INFO:tensorflow:preserve ratio after constraint 0.2654115557670593
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.12243277207016945s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.1352352239191532s
INFO:tensorflow:Prune model/resnet_model/conv2d_13/Conv2D c_in from 32 to 8
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_13/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.66666667 0.5 0.5 0.5 0.5 0.5
0.2108823 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26543513]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26543513]]
INFO:tensorflow:max_reduced_flops 8656149.362135286
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2949120.0
INFO:tensorflow:recommand action 4.294160524730374
INFO:tensorflow:preserve ratio after constraint 0.265435129404068
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.08029894903302193s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.16053732484579086s
INFO:tensorflow:Prune model/resnet_model/conv2d_14/Conv2D c_in from 32 to 8
INFO:tensorflow:father conv model/resnet_model/conv2d_13/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu_10:0", shape=(128, 16, 16, 32), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_14/Conv2D layer, the pruning rate is 0.25
INFO:tensorflow:Actural preserv ratio: 0.25
INFO:tensorflow:state is [[ 0.71428571 1. 0.5 0.25 0.25 1.
0.21074169 0.05555556]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26591134]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26591134]]
INFO:tensorflow:max_reduced_flops 8610861.894296741
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 131072.0
INFO:tensorflow:recommand action 91.25774869501693
INFO:tensorflow:preserve ratio after constraint 0.265911340713501
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.07442795857787132s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.05704880878329277s
INFO:tensorflow:Prune model/resnet_model/conv2d_15/Conv2D c_in from 32 to 9
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_15/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 0.76190476 1. 0.5 0.25 0.25 1.
0.20963913 0.5 ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.26815388]]
INFO:tensorflow:preserve ratio before constraint [[ 0.26815388]]
INFO:tensorflow:max_reduced_flops 8614124.010361254
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 311588.5523437744
INFO:tensorflow:recommand action 38.534065444229036
INFO:tensorflow:preserve ratio after constraint 0.2681538760662079
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.11918899044394493s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.1909613460302353s
INFO:tensorflow:Prune model/resnet_model/conv2d_16/Conv2D c_in from 32 to 9
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_16/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 0.80952381 1. 1. 0.25 0.25 0.5
0.20971855 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.28190231]]
INFO:tensorflow:preserve ratio before constraint [[ 0.28190231]]
INFO:tensorflow:max_reduced_flops 8666012.649499863
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 2691072.0
INFO:tensorflow:recommand action 4.693278653404716
INFO:tensorflow:preserve ratio after constraint 0.2819023132324219
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.15370018780231476s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.48831892758607864s
INFO:tensorflow:Prune model/resnet_model/conv2d_17/Conv2D c_in from 64 to 18
INFO:tensorflow:father conv model/resnet_model/conv2d_16/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Pad_3:0", shape=(128, 18, 18, 32), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_17/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 0.85714286 1. 1. 0.25 0.25 0.5
0.21098183 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.28127745]]
INFO:tensorflow:preserve ratio before constraint [[ 0.28127745]]
INFO:tensorflow:max_reduced_flops 8674938.831776684
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.316644888421088
INFO:tensorflow:preserve ratio after constraint 0.28127744793891907
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.2397974207997322s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.43007807806134224s
INFO:tensorflow:Prune model/resnet_model/conv2d_18/Conv2D c_in from 64 to 18
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_18/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 0.9047619 1. 1. 0.25 0.25 0.5
0.21119914 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.28067845]]
INFO:tensorflow:preserve ratio before constraint [[ 0.28067845]]
INFO:tensorflow:max_reduced_flops 8724924.35590225
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 3022848.0
INFO:tensorflow:recommand action 4.188390530563013
INFO:tensorflow:preserve ratio after constraint 0.2806784510612488
INFO:tensorflow:pruning kernel
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:approximating residual branch diff
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.1473100669682026s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.4896675869822502s
INFO:tensorflow:Prune model/resnet_model/conv2d_19/Conv2D c_in from 64 to 18
INFO:tensorflow:father conv model/resnet_model/conv2d_18/Conv2D
INFO:tensorflow:father conv input Tensor("model/resnet_model/Relu_14:0", shape=(128, 8, 8, 64), dtype=float32)
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_19/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 0.95238095 1. 1. 0.25 0.25 0.5
0.21241608 1. ]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.28095466]]
INFO:tensorflow:preserve ratio before constraint [[ 0.28095466]]
INFO:tensorflow:max_reduced_flops 8737132.510934835
INFO:tensorflow:desired_preserce 20537344.0
INFO:tensorflow:this flops 623177.1046875488
INFO:tensorflow:recommand action 19.216521130063377
INFO:tensorflow:preserve ratio after constraint 0.28095465898513794
INFO:tensorflow:pruning kernel
INFO:tensorflow:computing pruned kernel
INFO:tensorflow:pruned channel selecting
INFO:tensorflow:Channel selection time cost: 0.1764017567038536s
INFO:tensorflow:Feature map reconstructing
INFO:tensorflow:Feature map reconstruction time cost: 0.4858154207468033s
INFO:tensorflow:Prune model/resnet_model/conv2d_20/Conv2D c_in from 64 to 18
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_20/Conv2D layer, the pruning rate is 0.28125
INFO:tensorflow:Actural preserv ratio: 0.28125
INFO:tensorflow:state is [[ 1. 1. 1. 0.25 0.25 0.5 0.2127133
-
]]
INFO:tensorflow:RL choosed preserv ratio: [[ 0.28163278]]
INFO:tensorflow:preserve ratio before constraint 1
INFO:tensorflow:preserve ratio after constraint 1
INFO:tensorflow:Channel pruning the model/resnet_model/conv2d_21/Conv2D layer, the pruning rate is 1
INFO:tensorflow:loss: 3.6351118087768555
INFO:tensorflow:accuracy: 0.3563801944255829
INFO:tensorflow:Pruning accuracy 0.3563801944255829
INFO:tensorflow:The current model flops is 9225600.0
INFO:tensorflow:Pruned flops 9225600.0
INFO:tensorflow:The accuracy is 0.3563801944255829 and the flops after pruning is 9225600.0
INFO:tensorflow:The speedup ratio is 0.2246054796569605
INFO:tensorflow:The original model flops is 41074688.0
INFO:tensorflow:The pruned flops is 9225600.0
INFO:tensorflow:The max strategy dict is {'model/resnet_model/conv2d/Conv2D': [1.0, 0.25], 'model/resnet_model/conv2d_1/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_2/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_3/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_4/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_5/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_6/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_7/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_8/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_9/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_10/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_11/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_12/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_13/Conv2D': [0.25, 0.25], 'model/resnet_model/conv2d_14/Conv2D': [0.25, 1.0], 'model/resnet_model/conv2d_15/Conv2D': [0.28125, 1.0], 'model/resnet_model/conv2d_16/Conv2D': [0.28125, 0.28125], 'model/resnet_model/conv2d_17/Conv2D': [0.28125, 1.0], 'model/resnet_model/conv2d_18/Conv2D': [0.28125, 0.28125], 'model/resnet_model/conv2d_19/Conv2D': [0.28125, 1.0], 'model/resnet_model/conv2d_20/Conv2D': [0.28125, 1], 'model/resnet_model/conv2d_21/Conv2D': [1, 1.0]}
INFO:tensorflow:Actural preserv ratio: 1
INFO:tensorflow:roll-out #0: a-loss = 0.00e+00 | c-loss = 0.00e+00 | noise std. = 1.00e+00
INFO:tensorflow:best reward updated: -inf -> 0.3564
INFO:tensorflow:The best pruned model occured with
strategy: [1.0, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.28125, 0.28125, 0.28125, 0.28125, 0.28125, 0.28125, 1.0],
accuracy: 0.3563801944255829 and
pruned ratio: 9225600.0
Traceback (most recent call last):
File "main.py", line 69, in
tf.app.run()
File "/home/trainer/anaconda3/envs/tf19_py36/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "main.py", line 55, in main
learner.train()
File "/data/raid/PocketFlow/learners/channel_pruning/learner.py", line 141, in train
self.__prune_and_finetune_auto()
File "/data/raid/PocketFlow/learners/channel_pruning/learner.py", line 597, in __prune_and_finetune_auto
self.__prune_rl()
File "/data/raid/PocketFlow/learners/channel_pruning/learner.py", line 697, in __prune_rl
self.__save_in_progress_pruned_model()
File "/data/raid/PocketFlow/learners/channel_pruning/learner.py", line 482, in __save_in_progress_pruned_model
self.max_save_path = self.saver_eval.save(self.sess_eval, FLAGS.cp_best_path)
AttributeError: 'NoneType' object has no attribute 'save'
@jiaxiang-wu
from pocketflow.
jiaxiang-wu
commented on May 22, 2024
Hi, are you running the program with scripts/run_seven.sh or scripts/run_local.sh?
from pocketflow.
sunzhe09
commented on May 22, 2024
yes I have change scripts/run_seven.sh to scripts/run_local.sh ,the first lines of the log were added by myself to say which command I have used
from pocketflow.
jiaxiang-wu
commented on May 22, 2024
Can you post the full content of your scripts/run_local.sh, and the complete command for starting the program with this script?
from pocketflow.
sunzhe09
commented on May 22, 2024
the shell scripts is：
#!/bin/bash
default arguments
nb_gpus=1
parse arguments passed from the command line
py_script="$1"
shift
for i in "$@"
do
case "$i" in
-n=|--nb_gpus=)
nb_gpus="${i#*=}"
shift
;;
*)
# unknown option
;;
esac
done
extra_args=python utils/get_path_args.py local ${py_script} path.conf
extra_args="$@ ${extra_args}"
echo "Python script: ${py_script}"
echo "# of GPUs: ${nb_gpus}"
echo "extra arguments: ${extra_args}"
obtain list of idle GPUs
idle_gpus=python utils/get_idle_gpus.py ${nb_gpus}
export CUDA_VISIBLE_DEVICES=${idle_gpus}
re-create the logging directory
rm -rf logs && mkdir logs
execute the specified Python script with one or more GPUs
cp -v ${py_script} main.py
if [ ${nb_gpus} -eq 1 ]; then
echo "multi-GPU training disabled"
python main.py ${extra_args}
elif [ ${nb_gpus} -le 8 ]; then
echo "multi-GPU training enabled"
options="-np ${nb_gpus} -H localhost:${nb_gpus} -bind-to none -map-by slot
-x NCCL_DEBUG=INFO -x NCCL_SOCKET_IFNAME=eth1 -x NCCL_IB_DISABLE=1
-x LD_LIBRARY_PATH --mca btl_tcp_if_include eth1"
mpirun ${options} python main.py --enbl_multi_gpu ${extra_args}
fi
The command is:
./scripts/run_local.sh nets/resnet_at_cifar10_run.py
--learner channel
--batch_size_eval 64
--cp_uniform_preserve_ratio 0.5
--cp_prune_option uniform
--resnet_size 20
from pocketflow.
vuiseng9
commented on May 22, 2024
hi @sunzhe09 @jiaxiang-wu, I am running into the exact issue when I tried the auto mode on the Imagenet dataset on a local machine with one gpu card.
Run command
./scripts/run_local.sh nets/resnet_at_ilsvrc12_run.py --learner channel --cp_prune_option auto --cp_preserve_ratio 0.5
Error Trace
File "/workspace/pocketflow/nets/resnet_at_ilsvrc12_run.py", line 69, in <module>
tf.app.run()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/workspace/pocketflow/nets/resnet_at_ilsvrc12_run.py", line 55, in main
learner.train()
File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 141, in train
self.__prune_and_finetune_auto()
File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 598, in __prune_and_finetune_auto
self.__prune_rl()
File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 698, in __prune_rl
self.__save_in_progress_pruned_model()
File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 482, in __save_in_progress_pruned_model
self.max_save_path = self.saver_eval.save(self.sess_eval, FLAGS.cp_best_path)
AttributeError: 'NoneType' object has no attribute 'save'
Potential Root Cause
From brief debug, the error originated here.
# lines between 695-700 of learners/channel_pruning/learner.py
# in function __prune_rl()
with self.pruner.model.g.as_default():
self.__save_in_progress_pruned_model()In the function __save_in_progress_pruned_model(), the self.saver_eval and self.sess_eval are initialized to None and have not been set so far. The error happened close to the end of first episode of the ddpg when the pruned model is intended to be saved? Just guessing, could it be typos and we
actually need self.saver_train and self.sess_train instead?
# lines between 480-485 of learners/channel_pruning/learner.py
def __save_in_progress_pruned_model(self):
""" save a in progress training model with a max evaluation result"""
self.max_save_path = self.saver_eval.save(self.sess_eval, FLAGS.cp_best_path)
tf.logging.info('model saved best model to ' + self.max_save_path)from pocketflow.
jiaxiang-wu
commented on May 22, 2024
@psyyz10 Can you take a look at this issue?
from pocketflow.
jiaxiang-wu
commented on May 22, 2024
Resolved in this pull request: #65
from pocketflow.
shellhue
commented on May 22, 2024
hi @sunzhe09 @jiaxiang-wu, I am running into the exact issue when I tried the auto mode on the Imagenet dataset on a local machine with one gpu card.
Run command
./scripts/run_local.sh nets/resnet_at_ilsvrc12_run.py --learner channel --cp_prune_option auto --cp_preserve_ratio 0.5Error Trace
File "/workspace/pocketflow/nets/resnet_at_ilsvrc12_run.py", line 69, in <module> tf.app.run() File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "/workspace/pocketflow/nets/resnet_at_ilsvrc12_run.py", line 55, in main learner.train() File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 141, in train self.__prune_and_finetune_auto() File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 598, in __prune_and_finetune_auto self.__prune_rl() File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 698, in __prune_rl self.__save_in_progress_pruned_model() File "/workspace/pocketflow/learners/channel_pruning/learner.py", line 482, in __save_in_progress_pruned_model self.max_save_path = self.saver_eval.save(self.sess_eval, FLAGS.cp_best_path) AttributeError: 'NoneType' object has no attribute 'save'Potential Root Cause
From brief debug, the error originated here.# lines between 695-700 of learners/channel_pruning/learner.py # in function __prune_rl() with self.pruner.model.g.as_default(): self.__save_in_progress_pruned_model()In the function
__save_in_progress_pruned_model(), theself.saver_evalandself.sess_evalare initialized toNoneand have not been set so far. The error happened close to the end of first episode of the ddpg when the pruned model is intended to be saved? Just guessing, could it be typos and we
actually needself.saver_trainandself.sess_traininstead?# lines between 480-485 of learners/channel_pruning/learner.py def __save_in_progress_pruned_model(self): """ save a in progress training model with a max evaluation result""" self.max_save_path = self.saver_eval.save(self.sess_eval, FLAGS.cp_best_path) tf.logging.info('model saved best model to ' + self.max_save_path)
this bug still exists! After pull request: #65 merged!
from pocketflow.
jiaxiang-wu
commented on May 22, 2024
@psyyz10 Can you keep an eye on this issue?
from pocketflow.
shellhue
commented on May 22, 2024
Actually, remove code from pull request: #65, bug fixed!
from pocketflow.
Related Issues (20)
- cifar10_channel pruned 的示例，通道剪枝(channel_pruning) 导出修改了计算图之后，速度比之前的更慢了！ HOT 1
- Can the compression method provided by pocketflow be applied to MASK R-CNN? HOT 1
- QQ group HOT 1
- 我可以只用模型压缩部分么？
- TypeError: forward_train() missing 1 required positional argument: 'objects'
- Missing 1 required positional argument in constructor : data_format
- Download Pretrain Model But Get 502 Bad Gateway Error HOT 1
- You must feed a value for placeholder tensor 'model/input_1' with dtype float and shape [?,160,240,1]
- Question about export_chn_pruned_tflite_model.py HOT 1
- TF Version compatibility HOT 2
- Failed to create session
- Is it possible to compress the keras model with Pocket Flow
- Question about UniformLearner HOT 2
- Default tensorboard log output is huge
- FRCNN with VOC: Cannot batch tensors with different shapes in component 1.
- IndexError: list index out of range HOT 3
- Other issues:
- auto 通道裁剪问题
- test
- TF-Plus for Multi-GPU Training
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pocketflow.