Giter Club home page Giter Club logo

lxztju / pytorch_classification Goto Github PK

View Code? Open in Web Editor NEW
1.3K 14.0 335.0 3.13 MB

利用pytorch实现图像分类的一个完整的代码,训练,预测,TTA,模型融合,模型部署,cnn提取特征,svm或者随机森林等进行分类,模型蒸馏,一个完整的代码

License: MIT License

Python 2.21% Jupyter Notebook 96.71% CMake 0.15% C++ 0.85% Shell 0.07%
pytorch image-classification deployment svm knn cnn label-smoothing densenet resnext resnet

pytorch_classification's People

Contributors

lxztju avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pytorch_classification's Issues

TypeError: __init__() got an unexpected keyword argument 'weights'

Traceback (most recent call last):
File "./tools/train_val.py", line 176, in
train(rank, local_rank, device, args)
File "./tools/train_val.py", line 67, in train
model = ClsModel(args.model_name, args.num_classes, args.is_pretrained)
File "/root/gpf/pytorch_classification/cls_models/model.py", line 31, in init
self.base_model = getattr(torchvision.models, self.model_name)(weights=ModelWeights[self.model_name])
File "/root/miniconda3/envs/pytorch_clas/lib/python3.7/site-packages/torchvision/models/resnet.py", line 277, in resnet18
**kwargs)
File "/root/miniconda3/envs/pytorch_clas/lib/python3.7/site-packages/torchvision/models/resnet.py", line 260, in _resnet
model = ResNet(block, layers, **kwargs)
TypeError: init() got an unexpected keyword argument 'weights'


使用的包版本:
pytorch==1.8.1+cu101
torchvision==0.9.1+cu101

为什么会报上面的错误?

关于训练的问题

求请教!
在跑代码时报错:ValueError: too many values to unpack (expected 2)
定位到时data文件夹下的dataset.py,而后尝试仅允许该文件dataset.py测试数据提取函数,发现仍然报相同的错
image

Corrupt EXIF data.

训练自己的图像会出现下面的问题:
UserWarning: Corrupt EXIF data. Expecting to read 12 bytes but only got 10.

label smoothing loss问题

Hi:
在使用您代码的过程中,我对比了nn.CrossEntropyLoss()和LabelSmoothingCrossEntropy(),并在每个step打印loss值,得到以下,左边为smoothing loss,右边为nn.CrossEntropyLoss,似乎smoothing的loss稳定且只能以微小变化(0.5~0.6),而右边则还能降得更低。
我在自己写的代码上同样遇到过,请问您有没留意到这个问题?期待您的回复

0.6104838848114014 0.13801012933254242
0.609204888343811 0.14818525314331055
0.6149349212646484 0.1491081565618515
0.6321774125099182 0.17487832903862
0.6126755475997925 0.1317736655473709
0.6100113987922668 0.1340378224849701
0.6236982345581055 0.15228210389614105
0.5979918837547302 0.11493214219808578
0.6044023036956787 0.12213070690631866
0.5954407453536987 0.1088051125407219
Epoch:3 || epochiter: 32/139|| Totel iter 310 || Loss: 0.595441||ACC: 99.219 ||LR: 0.00044660
0.6146756410598755 0.13752762973308563
0.6278829574584961 0.1602025330066681
0.6053824424743652 0.1260662078857422
0.6094704270362854 0.14142774045467377
0.6054157018661499 0.13225990533828735
0.6109800934791565 0.14464853703975677
0.6221442818641663 0.14965230226516724
0.6030154228210449 0.12700901925563812
0.6105899214744568 0.13878554105758667
0.6118856072425842 0.13709238171577454
Epoch:3 || epochiter: 42/139|| Totel iter 320 || Loss: 0.611886||ACC: 99.219 ||LR: 0.00046097
0.6141043305397034 0.15106570720672607
0.6156240701675415 0.16219794750213623
0.6021517515182495 0.1331876516342163
0.5985877513885498 0.12620455026626587
0.6136577129364014 0.1546640545129776
0.6055587530136108 0.12843844294548035
0.6039034128189087 0.13453435897827148
0.600081741809845 0.12076513469219208
0.5962940454483032 0.11982955038547516
0.6148940324783325 0.13918887078762054

数据类型问题

请教大佬,我在导入自己图像(均为png格式)之后调用resnet101尝试训练时报错,我把train.py里的第156行labels.处加了long()之后就能够正常运行,这样改的话会不会对最终结果产生比较严重的影响?还请点拨一二

预测得分为负数

作者您好,我在使用您的代码训练自己的数据时,数据一共有6类,预测结果没有问题,准确率很高,但是其他的类得分为什么会出现负数呢? 并且不同的图像得分的大小程度也差很多,请问是什么原因呢

训练没报错但是运行一半结束了

torch1.4.0 torchvision0.5.0

当运行train.py的时候,没有报错但是程序运行一半结束了

****** Training resnet101 ******
****** loading the Imagenet pretrained weights ******
****** using single gpu to training ********
...... Initialize the network done!!! .......
Process finished with exit code 0

程序输出到这里就结束了,可以指点一下吗

ValueError: too many values to unpack (expected 2)

当自己尝试使用resnet50或者moblienetv2时都会报如下错误:
Traceback (most recent call last): File "train.py", line 147, in <module> images, labels = next(batch_iterator) File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in __next__ data = self._next_data() File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/xuhaowang/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/data/xuhaow/wordload/pytorch_classification-master/data/dataset.py", line 49, in __getitem__ img_path, label = self.imgs[index] ValueError: too many values to unpack (expected 2)
请问这种情况有人遇到过么,该如何解决

LOCAL_PRETRAINED具体在哪部分代码实现的?

您好,想请教一下,LOCAL_PRETRAINED这部分的功能实现代码具体是在哪个位置,好像除了在build_model.py中看到from models import LOCAL_PRETRAINED, model_urls以外,就没有看到功能具体实现的代码了。
比如,我应该把下载下来的预训练模型存储在那个位置?

ValueError: optimizer got an empty parameter list

运行train.py的时候报错ValueError: optimizer got an empty parameter list,报错位置optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)这个是什么问题呀

ValueError: optimizer got an empty parameter list

...... Initialize the network done!!! .......
Traceback (most recent call last):
File "/home/jiannan/project/pytorch_classification-master/train.py", line 76, in
optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)
File "/home/jiannan/anaconda3/envs/pt-gpu/lib/python3.6/site-packages/torch/optim/adam.py", line 42, in init
super(Adam, self).init(params, defaults)
File "/home/jiannan/anaconda3/envs/pt-gpu/lib/python3.6/site-packages/torch/optim/optimizer.py", line 46, in init
raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list
这个问题要怎么解决???

为什么这几个代码不上传呢?

from data import get_train_transform, get_test_transform
from data import get_random_eraser

import numpy as np

def get_random_eraser(p=0.5, s_l=0.02, s_h=0.4, r_1=0.3, r_2=3.3333333333333335, v_l=0, v_h=255, pixel_level=False):

def eraser(input_img):
    img_h, img_w, img_c = input_img.shape
    p_1 = np.random.rand()
    if p_1 > p:
        return input_img
    else:
        while 1:
            s = np.random.uniform(s_l, s_h) * img_h * img_w
            r = np.random.uniform(r_1, r_2)
            w = int(np.sqrt(s / r))
            h = int(np.sqrt(s * r))
            left = np.random.randint(0, img_w)
            top = np.random.randint(0, img_h)
            if left + w <= img_w:
                if top + h <= img_h:
                    break

        if pixel_level:
            c = np.random.uniform(v_l, v_h, (h, w, img_c))
        else:
            c = np.random.uniform(v_l, v_h)
        input_img[top:top + h, left:left + w, :] = c
        return input_img

return eraser

数据集导入

1648558977(1)
这一块数据集导入,是不是有点问题呀,index是什么东西呀,label也读不出来。一直显示下面的错误
image

请问可以使用自己的模型做训练么?

你好,作为刚入门的选手,很开心能看到这么好的代码库。我有一个问题:我发现代码库里都是在调用现成的网络的预训练模型,能否加入我自己修改的网络训练呢?

server.py 在编写自己的项目遇到了问题

您好!

 # classify the input image and then initialize the list
            # of predictions to return to the client

            out = model(img)
            # print(out)
            pred_label = torch.max(out, 1)[1].item()
            # print(pred_label)
            data["predictions"] = []
            data["predictions"].append(label_id_name_dict[str(pred_label)])

            # indicate that the request was a success
            data["success"] = True
            # print(data["success"])

    # return the data dictionary as a JSON response
    return jsonify(data)

您在这一段代码里完成了对out进行数据处理并返回json的操作,我使用的是mmdetection,并参照官方样例:

            results = inference_detector(model, img)
            print (results)
            show_result(img, results, model.CLASSES,show=False,out_file= now + '_result.jpg')

我仔细查看了result的数据,但是我仍然不清楚该如何把result中的bbox category score信息存成dict,并返回json,希望得到您的帮助!

如何将自己的数据集转化

您好,我的图片全部在一个文件夹里,对应有一个csv文件,里面包含图片的名字和标签,这样如何转化为该项目数据的标准

关于多卡训练的问题

我用3个12G的卡训练,但是第一个卡显存会爆掉,但是第二和第三张卡还有很多显存,只有设置batchsize为1才不会报错,但是用单卡来跑的话batchsize可以设置为6,请问这个问题怎么解决可以用多卡且可以用较大的batchsize

多模型融合预测

你好,我想请问一下哪个文件的代码可以实现多模型融合预测呢?

关于程序的问题

你好 请问您有微信或者qq吗 我有些问题想直接问您。我现在在日本大学的研究课题和您做的这个比较有关 所以想具体问一下。

多批次推理

大佬,可以教学一下多批次推理服务吗,谢谢

cnn-ml.py 属性错误 train_kd.py model问题

请教大佬这个bug应该怎么解决呢
pybug2

另外,在train_kd.py中,定义优化器与损失函数那里,model是标红的,直接运行,显示没有定义;

当用teacher_model,student_model代替model时,显示下面这个bug。

optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=cfg.LR)
optimizer = optim.SGD(model.parameters(), lr=cfg.LR,
momentum=cfg.MOMENTUM, weight_decay=cfg.WEIGHT_DECAY)

pybug3

抱拳

RuntimeError

****** using single gpu to training ********
...... Initialize the network done!!! .......
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\ProgramData\Anaconda3\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\ProgramData\Anaconda3\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\yihen\Downloads\pytorch_classification-master\pytorch_classification-master\train.py", line 111, in <module>
    batch_iterator = iter(train_dataloader)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
    return self._get_iterator()
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\ProgramData\Anaconda3\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.


after running the train.py, on my own data set, it gives this error.

数据集请求

请问这个有什么推荐的数据集用来做分类吗

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.