Giter Club home page Giter Club logo

mim's People

Contributors

6vvv avatar anton-cherepkov avatar danielhonies avatar ginray avatar guopingpan avatar haochenye avatar hellock avatar hu-po avatar ice-tong avatar kennymckormick avatar kim3321 avatar kkieek avatar ktro2828 avatar mzr1996 avatar nijkah avatar rangeking avatar rangilyu avatar saratrajput avatar teamwong111 avatar yeliudev avatar yukke42 avatar zhouzaida avatar zwwwayne avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mim's Issues

download function in openmim.commands.download module cannot find config file

Hello @zhouzaida, @ZwwWayne ,

steps to reproduce:

  • install mmdet, mmcv-full and mim via pip:
pip install -U mmdet mmcv-full mim==0.1.2
  • or install mmdet via mim:
mim install mmdet
  • then call download function from mim:
from openmim.commands.download import download

download(package="mmdet", configs=['cascade_mask_rcnn_r50_fpn_1x_coco'])

error:

Exception has occurred: ValueError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
�[31m�[1mlib/python3.8/site-packages/mmdet/configs/cascade_rcnn/cascade_mask_rcnn_r50_fpn_1x_coco.py is not found.�[0m
  File "xxx.py", line 79, in download
    raise ValueError(highlighted_error(f"{config_path} is not found."))

reason:

this line creates a config path:

config_path = resource_filename(module_name, config_path)

directing to:

"lib/python3.8/site-packages/mmdet/configs/cascade_rcnn/cascade_mask_rcnn_r50_fpn_1x_coco.py"

but mmdet package doesn't include configs directory in the package so this is a bug that should be fixed.

solution:

add an argument local to download function and when local=False, download config files from mmdetection repo instead of searching in local mmdet package directory.

auto install gpu verison when docker build

Describe the feature

in my Dockerfile , i install mmcv-full cuda version first, by when i install other mmlab module , it would reinstall another verison mmcv-full by requirements , and could be a cpu version.

i found mim will install gpu version when torch.cuda.is_available()

so , is there a way , i can build a image install gpu verison and without gpu machines

Motivation
A clear and concise description of the motivation of the feature.
Ex1. It is inconvenient when [....].
Ex2. There is a recent paper [....], which is very helpful for [....].

CI/CD auto build docker images , maybe without gpu

Related resources
If there is an official code release or third-party implementations, please also provide the information here, which would be very helpful.

Additional context
Add any other context or screenshots about the feature request here.
If you would like to implement the feature and create a PR, please leave a comment here and that would be much appreciated.

maybe make FORCE_CUDA environment variable work
and cuda verison can get from torch.version.cuda or other environment variable

Notes of supporting installation by mim in downstream repositories

A few notes of supporting installation by mim in downstream repositories.

Add extra files (e.g. tools/, configs/, model_zoo.yml) into the package path

  1. Use the environment variable MIM_SYMLINK_FILES=1 to indicate a package is being installed via mim.
  2. Add symlinks to add extra files when installed by pip or from the source code.
  3. Copy these files when installed by mim (because the cloned repo will be deleted after installation)

Extra file path (TBD)

  • Option 1: in the package folder with hidden filenames, e.g., mmpose/.tools
  • Option 2: in a hidden folder in the package folder, e.g., mmpose/.mim/tools

mim could be crash if exist version conflict

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug
mim could be crash if exist version conflict
image

Reproduction

  1. What command or script did you run?
> pip install markdown==3.3.7
> pip install importlib-metadata==4.2
> mim list
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Bypass SSL check when installing packages

I'm working in an environment with a strict custom proxy enforcement which blocks ssl verification to many sites including package hosts. Is there a way to bypass this verification similar to how pip allows the following?

--trusted-host {host}

install mmcv-full failed

system: ubuntu 18.04
cuda: cuda 10.2
python version: python 3.7
pytorch version: torch 1.8.1
command: mim install mmcv-full

WXWorkCapture_16215783608763

`mim run mmaction2`: mmaction2 is not an OpenMMLAB codebase.

Describe the bug
I installed mmaction2. mim run mmaction2 raise not a mmlab codebase error, while mim run mmaction raise not installed error.

Reproduction

mim run mmaction extracting_features.py  # or
mim run mmaction2 extracting_features.py

Error traceback

(open-mmlab) louis@louis-3:~$ mim run mmaction extracting_features.py -h
Usage: mim run [OPTIONS] PACKAGE COMMAND [OTHER_ARGS]...

  Run arbitrary command of a codebase.

  Note if the command you call takes config files or checkpoint paths as
  ...

Codebase mmaction in not installed.
(open-mmlab) louis@louis-3:~$ mim run mmaction2 extracting_features.py -h
Usage: mim run [OPTIONS] PACKAGE COMMAND [OTHER_ARGS]...

  Run arbitrary command of a codebase.

  Note if the command you call takes config files or checkpoint paths as
  ...

mmaction2 is not an OpenMMLAB codebase. 
(open-mmlab) louis@louis-3:~$ mim list
Package    Version    Source
---------  ---------  ----------------------------------------------
mmaction2  0.23.0     https://github.com/open-mmlab/mmaction2
mmcls      0.18.0     https://github.com/open-mmlab/mmclassification
mmcv-full  1.5.0      https://github.com/open-mmlab/mmcv
mmgen      0.4.0      https://github.com/open-mmlab/mmgen

unrecognized arguments: --local_rank

When I run the example

mim train mmcls resnet101_b16x8_cifar10.py --work-dir tmp --gpus 4 --launcher pytorch

An error happened:

Training command is python -m torch.distributed.launch --nproc_per_node=4 --master_port=27803 /home/tiger/.local/lib/python3.7/site-packages/mmcls/.mim/tools/train.py /home/tiger/.local/lib/python3.7/site-packages/mmcls/.mim/configs/resnet/resnet101_b16x8_cifar10.py --launcher pytorch --work-dir tmp.

usage: train.py [-h] [--work-dir WORK_DIR] [--resume-from RESUME_FROM]
                [--no-validate]
                [--device DEVICE | --gpus GPUS | --gpu-ids GPU_IDS [GPU_IDS ...]
                | --gpu-id GPU_ID] [--ipu-replicas IPU_REPLICAS] [--seed SEED]
                [--diff-seed] [--deterministic]
                [--cfg-options CFG_OPTIONS [CFG_OPTIONS ...]]
                [--launcher {none,pytorch,slurm,mpi}]
                [--local-rank LOCAL_RANK]
                config
train.py: error: unrecognized arguments: --local_rank=3
Traceback (most recent call last):
  File "/home/tiger/.local/bin/mim", line 33, in <module>
    sys.exit(load_entry_point('openmim', 'console_scripts', 'mim')())
  File "/home/tiger/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/tiger/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/tiger/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/tiger/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tiger/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/tiger/code/mim/mim/commands/train.py", line 108, in cli
    other_args=other_args)
  File "/home/tiger/code/mim/mim/commands/train.py", line 257, in train
    cmd, env=dict(os.environ, MASTER_PORT=str(port)))
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', '-m', 'torch.distributed.launch', '--nproc_per_node=4', '--master_port=27803', '/home/tiger/.local/lib/python3.7/site-packages/mmcls/.mim/tools/train.py', '/home/tiger/.local/lib/python3.7/site-packages/mmcls/.mim/configs/resnet/resnet101_b16x8_cifar10.py', '--launcher', 'pytorch', '--work-dir', 'tmp']' returned non-zero exit status 1.

Support `mim install -e .`

Motivation
A clear and concise description of the motivation of the feature.
Ex1. It will be convenient for developers, you don't need to install again after you alter the code.

Related resources
Please refer to pip install -e .

TypeError: sequence item 2: expected str instance, CalledProcessError found

(mmdetection) zq@edu:~$ mim install mmcls==0.12.0
installing mmcls from https://github.com/open-mmlab/mmclassification.git.
Cloning into '/tmp/tmpbmq8_i4b/mmclassification'...
remote: Enumerating objects: 3155, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 3155 (delta 1), reused 1 (delta 0), pack-reused 3149
Receiving objects: 100% (3155/3155), 2.80 MiB | 464.00 KiB/s, done.
Resolving deltas: 100% (2045/2045), done.
Note: checking out '27a49a9646aed9b6c465e22b0bfda3aae1dfb779'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

Successfully installed dependencies.

Usage:
/home/zq/miniconda3/envs/mmdetection/bin/python -m pip install [options] [package-index-options] ...
/home/zq/miniconda3/envs/mmdetection/bin/python -m pip install [options] -r [package-index-options] ...
/home/zq/miniconda3/envs/mmdetection/bin/python -m pip install [options] [-e] ...
/home/zq/miniconda3/envs/mmdetection/bin/python -m pip install [options] [-e] ...
/home/zq/miniconda3/envs/mmdetection/bin/python -m pip install [options] <archive url/path> ...

option --use-feature: invalid choice: 'in-tree-build' (choose from '2020-resolver', 'fast-deps')
Traceback (most recent call last):
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/utils/utils.py", line 417, in call_command
subprocess.check_call(cmd)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', '-m', 'pip', 'install', '--use-feature=in-tree-build', '/tmp/tmpbmq8_i4b/mmclassification']' returned non-zero exit status 2.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/zq/miniconda3/envs/mmdetection/bin/mim", line 8, in
sys.exit(cli())
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/commands/install.py", line 100, in cli
install(
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/commands/install.py", line 204, in install
install_from_github(target_pkg, target_version, find_url, timeout,
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/commands/install.py", line 534, in install_from_github
install_from_repo(
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/commands/install.py", line 491, in install_from_repo
call_command(install_cmd)
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/utils/utils.py", line 419, in call_command
raise highlighted_error(e) # type: ignore
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/mim/utils/utils.py", line 391, in highlighted_error
return click.style(msg, fg='red', bold=True) # type: ignore
File "/home/zq/miniconda3/envs/mmdetection/lib/python3.8/site-packages/click/termui.py", line 519, in style
return "".join(bits)
TypeError: sequence item 2: expected str instance, CalledProcessError found

How to remove cache??

I found mim install mmcv-full gives me error package, but I can not remove cache, everytime it gives me error one without full extension, how to remove cache??

KeyError: 'Cascade Mask R-CNN'

  1. I've followed the installation steps from https://mmdetection.readthedocs.io/en/v2.25.0/get_started.html
  2. Then I try to execute the following command from the manual mim download mmdet --config yolov3_mobilenetv2_320_300e_coco --dest .
$ mim download mmdet --config yolov3_mobilenetv2_320_300e_coco --dest .
~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "~/miniconda3/envs/openmmlab/bin/mim", line 33, in <module>
    sys.exit(load_entry_point('openmim', 'console_scripts', 'mim')())
  File "~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "~/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "~/repositories/mim/mim/commands/download.py", line 44, in cli
    download(package, configs, dest_root)
  File "~/repositories/mim/mim/commands/download.py", line 75, in download
    model_info = get_model_info(
  File "~/repositories/mim/mim/commands/search.py", line 170, in get_model_info
    dataframe = convert2df(metadata)
  File "~/repositories/mim/mim/commands/search.py", line 396, in convert2df
    for key, value in name2collection[collection_name].items():
KeyError: 'Cascade Mask R-CNN'

I've also tried to install mim directly from master, but the same error appears.

'mmseg' is not installed.

Hi, when I try mim list, the following message is showed:

Package         Version    Source
--------------  ---------  ----------------------------------------------
mmcls           0.14.0     https://github.com/open-mmlab/mmclassification
mmcv            1.3.10     https://github.com/open-mmlab/mmcv
mmcv-full       1.3.9      https://github.com/open-mmlab/mmcv
mmsegmentation  0.16.0     http://github.com/open-mmlab/mmsegmentation

We can see that mmsegmentation is installed. When I use mim train mmseg xxx.config, mim will remind me to install mmseg as The codebase mmseg is not installed, do you want to install the latest release? . However, there is a 'mmseg' package in https://pypi.org/project/mmseg/, which is not mmsegmentation actually. So errors occured as someone describe here: #55.
BTW, I notice that even though PKG2MODULE and MODULE2PKG are setting in mim/mim/utils/default.py, there still are logical bugs somewhere.

mim install a package from github branch

Hello, I want to use a different version of mmdet from a github branch. For example, from jshilong/mmdetection, brach onnx_cascade. I try

    mim install mmdet -f https://github.com/jshilong/mmdetection.git@onnx_cascade

but it does not work yet. Thank you.

[Bug] KeyError when running `mim search mmyolo --remote`

Reproduction

  1. What command or script did you run?
mim search mmyolo --remote
  1. Did you make any modifications on the code or config?
No

Error traceback

KeyError: 'mmyolo'

Bug fix
add mmyolo in PKG2PROJECT

PKG2PROJECT = {
    'mmcv-full': 'mmcv',
    'mmcls': 'mmclassification',
    'mmdet': 'mmdetection',
    'mmdet3d': 'mmdetection3d',
    'mmsegmentation': 'mmsegmentation',
    'mmaction2': 'mmaction2',
    'mmtrack': 'mmtracking',
    'mmpose': 'mmpose',
    'mmedit': 'mmediting',
    'mmocr': 'mmocr',
    'mmgen': 'mmgeneration',
    'mmselfsup': 'mmselfsup',
    'mmrotate': 'mmrotate',
    'mmflow': 'mmflow',
    'mmyolo': 'mmyolo',
}

use site-packages/${project}/.mim/config as the base config

Describe the feature
I would like to be able to specify the config under .mim as the base config and run it as shown below.

_base_ = [
    'site-packages/mmcls/.mim/config/_base_/models/resnet18_cifar.py',
    'site-packages/mmcls/.mim/config/_base_/datasets/cifar10_bs16.py',
    'site-packages/mmcls/.mim/config/_base_/schedules/cifar10_bs128.py',
    'site-packages/mmcls/.mim/config/_base_/default_runtime.py'
]

Motivation
When specifying the config in the repo as the base config, I find it inconvenient that I need to either download it locally by git clone or copy the config to be used itself under .mim.

Additional context
The following part of mmcv is the load part of the base config. I think it is possible to load the config from .mim by changing this part, is this the proper way?
https://github.com/open-mmlab/mmcv/blob/76cfd77b3a88ce17508471bf335829eb0628abcf/mmcv/utils/config.py#L239-L269

Is this the proper way to do it, or is there a way to complete it with a .mim repo?

Ambiguous resolve of python executable

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

I tried to install mmcv-full on mac using mim, but face the following error.

Reproduction

  1. What command or script did you run?
pip install openmim
mim install mmcv-full
  1. Did you make any modifications on the code or config? Did you understand what you have modified?

No

  1. What dataset did you use?

No

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

wangruohui@RHs-MacBook ~/mmediting % mim install mmcv-full
installing mmcv-full from wheel.
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pip
Traceback (most recent call last):
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/utils/utils.py", line 418, in call_command
    subprocess.check_call(cmd)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', '-m', 'pip', '--default-timeout', '45', 'install', 'mmcv-full==1.4.6', '-f', 'https://download.openmmlab.com/mmcv/dist/cpu/torch1.10.0/index.html']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/wangruohui/Library/Python/3.8/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/commands/install.py", line 101, in cli
    install(
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/commands/install.py", line 211, in install
    install_from_wheel(target_pkg, target_version, find_url, timeout,
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/commands/install.py", line 574, in install_from_wheel
    call_command(install_cmd)
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/utils/utils.py", line 420, in call_command
    raise highlighted_error(e)  # type: ignore
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/mim/utils/utils.py", line 392, in highlighted_error
    return click.style(msg, fg='red', bold=True)  # type: ignore
  File "/Users/wangruohui/Library/Python/3.8/lib/python/site-packages/click/termui.py", line 519, in style
    return "".join(bits)
TypeError: sequence item 2: expected str instance, CalledProcessError found

Bug fix

MIM will call python. However, on some platform with both python and python3 installed, and with python2 as the default. python will be revolved as python2. Also if some env variables are note correctly configured, python might be resolved to python
in other enviromnets.

We can use sys.executable as the executable. I will make a simple chage for install command.

mim install mmcv-full in the wrong venv.

When install the mmcv-full package use openmim, the package is installed in the wrong venv.

Reproduction

  1. What command or script did you run?
    In the pytorch venv, running mim install mmcv-full, and then pip list command cannot find the package in this venv. After deactivating the pytorch venv use deactivate, the mmcv-full package appears.

Bug fix
It seams that the mim install funcition cannot recognize the venv environment.

Grid search YOLOX base_lr

Describe the feature

mim grid search can search optimal lr like followings.

search_args = '--optim_wrapper.optimizer.lr 0.001 0.01 0.2'

But YOLOX lr search has a problem. We should search base_lr instead of optim_wrapper.optimizer.lr because of param_scheduler. 
https://github.com/open-mmlab/mmdetection/blob/dev-3.x/configs/yolox/yolox_s_8xb8-300e_coco.py#L175

Config is like following.

base_lr = 0.01
optim_wrapper = dict(
    type='OptimWrapper',
    optimizer=dict(
        type='SGD', lr=base_lr, momentum=0.9, weight_decay=5e-4,
        nesterov=True),
    paramwise_cfg=dict(norm_decay_mult=0., bias_decay_mult=0.))

param_scheduler = [
   ...
    dict(
        type='CosineAnnealingLR',
        eta_min=base_lr * 0.05,  # This part is a problem for lr grid search.
        begin=5,
        T_max=max_epochs - num_last_epochs,
        end=max_epochs - num_last_epochs,
        by_epoch=True,
        convert_to_iter_based=True),
    ...
]

However, we currently do not have the function to properly search lr and param_scheduler by searching base_lr.

Motivation

I request the following features.

  1. mim gridsearch mmdet yolox_s_8xb8-300e_coco.py --work-dir tmp --gpus 1 --search-args '--base_lr 1e-2 1e-3'
  2. Then, along with base_lr, the appropriate optim_wrapper.optimizer.lr and param_scheduler are set and the search can be performed.

fatal: unable to access 'https://github.com/open-mmlab/mmdetection.git/': Could not resolve host: github.com

When I do mim install mmdet, I get the following error:

installing mmdet from https://github.com/open-mmlab/mmdetection.git.
Cloning into '/tmp/tmpkmwtxjzz/mmdetection'...
fatal: unable to access 'https://github.com/open-mmlab/mmdetection.git/': Could not resolve host: github.com
Traceback (most recent call last):
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/utils/utils.py", line 418, in call_command
    subprocess.check_call(cmd)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'clone', 'https://github.com/open-mmlab/mmdetection.git', '-b', 'v2.20.0', '/tmp/tmpkmwtxjzz/mmdetection']' returned non-zero exit status 128.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/muhammadmehdi/anaconda3/envs/memex/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/commands/install.py", line 107, in cli
    is_editable=is_editable)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/commands/install.py", line 206, in install
    is_yes, is_user_dir, is_install_master)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/commands/install.py", line 534, in install_from_github
    call_command(clone_cmd)
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/utils/utils.py", line 420, in call_command
    raise highlighted_error(e)  # type: ignore
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/mim/utils/utils.py", line 392, in highlighted_error
    return click.style(msg, fg='red', bold=True)  # type: ignore
  File "/home/muhammadmehdi/anaconda3/envs/memex/lib/python3.7/site-packages/click/termui.py", line 519, in style
    return "".join(bits)
TypeError: sequence item 2: expected str instance, CalledProcessError found

Cityscapes, LVIS v0.5 and v1 models are not properly registered in `mim`

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Thanks for providing a rich set of zoo models through mim's download function.

  1. cityscapes models are not in the registered models.
  2. lvis_v0.5 models are not in the registered models.
  3. Some lvis_v1 models cannot be loaded properly. Some of them are not in the registered models.

Reproduction

The following model ids are not in the registered models or cannot be loaded via mim's download.

Not available:
        "mask_rcnn_r50_fpn_1x_cityscapes"
        "mask_rcnn_r50_fpn_sample1e-3_mstrain_2x_lvis_v0.5"
        "mask_rcnn_r101_fpn_sample1e-3_mstrain_2x_lvis_v0.5"
        "mask_rcnn_x101_32x4d_fpn_sample1e-3_mstrain_2x_lvis_v0.5"
        "mask_rcnn_x101_64x4d_fpn_sample1e-3_mstrain_2x_lvis_v0.5"
        "mask_rcnn_r50_fpn_sample1e-3_mstrain_1x_lvis_v1"
        "mask_rcnn_r101_fpn_sample1e-3_mstrain_1x_lvis_v1"
        "mask_rcnn_x101_32x4d_fpn_sample1e-3_mstrain_1x_lvis_v1"
        "mask_rcnn_x101_64x4d_fpn_sample1e-3_mstrain_1x_lvis_v1"
Not loadable:
        "mask_rcnn_r50_fpn_random_seesaw_loss_mstrain_2x_lvis_v1"
        "mask_rcnn_r50_fpn_random_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
        "mask_rcnn_r101_fpn_random_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
        "mask_rcnn_r50_fpn_sample1e-3_seesaw_loss_mstrain_2x_lvis_v1"
        "mask_rcnn_r50_fpn_sample1e-3_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
        "mask_rcnn_r101_fpn_sample1e-3_seesaw_loss_mstrain_2x_lvis_v1"
        "mask_rcnn_r101_fpn_sample1e-3_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
        "cascade_mask_rcnn_r101_fpn_random_seesaw_loss_mstrain_2x_lvis_v1"
        "cascade_mask_rcnn_r101_fpn_random_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
        "cascade_mask_rcnn_r101_fpn_sample1e-3_seesaw_loss_mstrain_2x_lvis_v1"
        "cascade_mask_rcnn_r101_fpn_sample1e-3_seesaw_loss_normed_mask_mstrain_2x_lvis_v1"
  1. Did you make any modifications on the code or config? Did you understand what you have modified?

no

  1. What dataset did you use?

cityscapes, lvis_0.5, lvis_v1

Environment

Python3.8 from conda + Ubuntu 22.04

Error traceback
If applicable, paste the error trackback here.

When the model can't be loaded, this value error appeared:

File ~/miniconda3/lib/python3.8/site-packages/mim/commands/download.py:129, in download(package, configs, dest_root)
    127                 break
    128         else:
--> 129             raise ValueError(
    130                 highlighted_error(f'{config_path} is not found.'))
    132 return checkpoints

ValueError: /home/yuhuang/miniconda3/lib/python3.8/site-packages/mmdet/seesaw_loss/mask_rcnn_r50_fpn_random_seesaw_loss_mstrain_2x_lvis_v1.py is not found.

ValueError: Can't determine a unique package given abbreviation resnet18_8xb16_cifar10.py

Hi , when i run "mim train resnet18_8xb16_cifar10.py --work-dir tmp --gpus 4 --launcher pytorch", I got the following error:
Traceback (most recent call last):
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/bin/mim", line 8, in
sys.exit(cli())
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/mim/commands/train.py", line 107, in cli
other_args=other_args)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/mim/commands/train.py", line 156, in train
raise ValueError(highlighted_error(msg))
ValueError: Can't determine a unique package given abbreviation resnet18_8xb16_cifar10.py

How can i solve it?

mim(version 0.1.2) install mmocr failed

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.
mim(version 0.1.2) install mmocr failed
Traceback (most recent call last): File "/data/venv/mmlab/lib/python3.8/site-packages/mim/utils/utils.py", line 390, in call_command subprocess.check_call(cmd) File "/usr/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['git', 'clone', 'https://github.com/open-mmlab/mmocr.git', '-b', 'v0.2.1', '/tmp/tmpu7gj5bj_/mmocr']' returned non-zero exit status 128.
Reproduction

  1. What command or script did you run?
    mim install mmocr
A placeholder for the command.
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

there's no v0.2.1 in https://github.com/open-mmlab/mmocr.git

Update Click to 8.x

Describe the feature

Can we upgrade click to 8.x?

Motivation

Click 7.x is somehow old and other packages (such as black) migrates to use click 8.x, when install both there will be a conflict. So can we make a upgrade?

mim search error

Hi, when i run "mim search mmcls", i got the following error:
Traceback (most recent call last):
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/bin/mim", line 8, in
sys.exit(cli())
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/mim/commands/search.py", line 115, in cli
local=local)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/mim/commands/search.py", line 169, in get_model_info
dataframe = convert2df(metadata)
File "/home/wenli/anaconda3/envs/pytorch1.7_py37/lib/python3.7/site-packages/mim/commands/search.py", line 357, in convert2df
for result in results:
TypeError: 'NoneType' object is not iterable

How can i solve it?

Installation of rc versions does not work correctly

Description
If I run this in the command line:
pip install openmim
mim install mmdet>=3.0.0rc0
mim list
prints

Package    Version    Source
---------  ---------  -----------------------------------------
mmdet      2.25.3     https://github.com/open-mmlab/mmdetection

Expected behaviour
mim install mmdet>=3.0.0rc0 should install mmdet v3.0.0rc2 (the latest rc at the time of writing).

Probable cause
Maybe mim installs the newest package newer than 3.0.0rc0 by date. This can lead to the above behaviour.

Perhaps the version numbers need to be parsed instead of just installing by date of release?

Multiple GPU training failed

I tried to use 2 GPU for training, but it raised a error:

PYTHONPATH=$PWD:$PYTHONPATH mim train mmaction configs/localization/apn_coralrandom_r3dsony_32x4_10e_thumos14_flow.py --gpus 2 --validate

2021-06-13 16:38:48,309 - mmaction - INFO - workflow: [('train', 1)], max: 10 epochs
Traceback (most recent call last):
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmaction/tools/train.py", line 199, in <module>
    main()
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmaction/tools/train.py", line 195, in main
    meta=meta)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmaction/apis/train.py", line 163, in train_model
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs, **runner_kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
    self.run_iter(data_batch, train_mode=True, **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
    **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 55, in train_step
    ('MMDataParallel only supports single GPU training, if you need to'
AssertionError: MMDataParallel only supports single GPU training, if you need to train with multiple GPUs, please use MMDistributedDataParallelinstead.

However, I did's manually set the distributed method in my own code. It seems that mim uses the train.py instead of dist_train.sh . How to fix this?

mim train mmdet error

When I try to use mim train mmdet -h, this error happen:

python: can't open file '/home/tiger/.local/lib/python3.7/site-packages/mmdet/tools/train.py': [Errno 2] No such file or directory

However I can find mmdet by pip list or mim list

Centralized config

I wish to store all my openmmlab configs in one place, lets say ${config_project}.

However, when I ran mim mmdet train ${config_project}/my_config.py ..., it always try to find this path under ${mmdet}/.mim or ${mmdet}, resulting in a FileNotFound error. Do I have to copy-paste all configs to my mmdet installation? Or there are other ways to do this?

download function should not require local model_zoo.yml file

At the moment download function assumes there is a model_zoo.yml file in local mmdet/mmcls package dir. But when installed from wheel, these packages does not include model_zoo.yml file. Thus, download functions gives error.

There should be an option for parsing model info from remote. I have opened a PR that solves this issue: #49

Allow to install CUDA version of MMCV-full through commands.

Describe the feature
Allow to install CUDA version of MMCV-full through commands. For example, when typing MMCV_WITH_OPS=1 mim install mmcv-full, it should install a CUDA version MMCV-full no matter whether GPU is detected on the current machine. The correct CUDA version should be inferred by nvcc.

Motivation
Sometimes the training program runs on a computation cluster but the installation is done in a management cluster which does not have GPU but have nvcc.

allow for updating the packages

Describe the feature
Please allow for command like mim install --upgrade mmdet
Motivation
In order to update a package, we need to first uninstall the package, then re-install the package. This can be simplified.

Any updates in mim for mmlab major version updates?

I used mim with mmcls==1.0.0rc0, but it raised an error like following.

$mim train mmcls mmclassification/configs/resnet/resnet18_8xb16_cifar10.py
....
train.py: error: unrecognized arguments: --gpus 1
Traceback (most recent call last):
  File "/opt/conda/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/mim/commands/train.py", line 97, in cli
    is_success, msg = train(
  File "/opt/conda/lib/python3.8/site-packages/mim/commands/train.py", line 256, in train
    ret = subprocess.check_call(
  File "/opt/conda/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
....

I saw gpus was deleted from mmcls/tools/train.py, and there are no branches in mim repo for major updates.
Any prepares?

Password for Jupyter log-in

I pulled the docker image of mmdetection:latest and passed the cli as shown below to run the container. It leads to a Jupyter Notebook instance, but unable to log-in due to password credential. How do I log-in?

~ via  v14.19.2 via 🐍 v2.7.17
❯ docker --version
Docker version 20.10.15, build fd82621

~ via  v14.19.2 via 🐍 v2.7.17
❯ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
mmdetection latest 3ee366cdfedc 15 hours ago 3.19GB
docker_nvidia_tf1 latest d7d445d17804 2 days ago 15.8GB
docker_tf2_opencv_od latest b97693ec92c7 5 days ago 5.57GB
rama_vicky/openvino latest 0a2f7d183cd1 10 days ago 5.49GB
ubuntu bionic c6ad7e71ba7d 12 days ago 63.2MB
rama_vicky/od latest 6d460175bd0a 13 days ago 20.6GB
nvcr.io/nvidia/tensorflow 22.03-tf1-py3 14a3757c83fc 6 weeks ago 14.9GB
datamachines/tensorflow_opencv 2.8.0_4.5.5-20220318 2c763733665c 7 weeks ago 4.52GB
datamachines/cudnn_tensorflow_opencv 11.4.2_2.7.0_3.4.16-20220103 5a9e58a5fb76 4 months ago 17.8GB
openvino/ubuntu18_dev 2021.4.2_src 6baa7ea0da8d 6 months ago 6.35GB
openvino/ubuntu18_dev 2020.3 9c5ca9c923f3 23 months ago 5.19GB

~ via  v14.19.2 via 🐍 v2.7.17
❯ docker run -it
∙ --gpus all
∙ --shm-size=8g
∙ -v /dev:/dev
∙ -v /tmp/.X11-unix:/tmp/.X11-unix
∙ -v /home/kannan/mmdetection/:/var/lib/mmdetection/
∙ -e DISPLAY=$DISPLAY
∙ --network=host
∙ mmdetection:latest
[I 03:23:54.000 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
[I 03:23:54.132 NotebookApp] Serving notebooks from local directory: /tensorflow/models/research/object_detection
[I 03:23:54.132 NotebookApp] Jupyter Notebook 6.4.10 is running at:
[I 03:23:54.132 NotebookApp] http://Kannan:8888/
[I 03:23:54.132 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 03:24:02.620 NotebookApp] 302 GET / (127.0.0.1
Screenshot from 2022-05-12 08-54-38
) 0.420000ms
[I 03:24:02.626 NotebookApp] 302 GET /tree? (127.0.0.1) 0.690000ms
[W 03:24:08.658 NotebookApp] 401 POST /login?next=%2Ftree%3F (127.0.0.1) 1.050000ms referer=http://kannan:8888/login?next=%2Ftree%3F
Screenshot from 2022-05-12 08-54-38

[Bug] Cannot install mmcls requirements after installing mmdet.

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug

(pytorch110cu111) [root@iZuf602264ukmv152q0po9Z mmclassification]# mim install -r requirements/runtime.txt 
Looking in indexes: http://mirrors.cloud.aliyuncs.com/pypi/simple/
Looking in links: https://download.openmmlab.com/mmcv/dist/cu111/torch1.10.0/index.html
Requirement already satisfied: matplotlib in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from -r requirements/runtime.txt (line 1)) (3.5.2)
Requirement already satisfied: numpy in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from -r requirements/runtime.txt (line 2)) (1.23.1)
Requirement already satisfied: packaging in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from -r requirements/runtime.txt (line 3)) (21.3)
Requirement already satisfied: python-dateutil>=2.7 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (1.4.4)
Requirement already satisfied: pyparsing>=2.2.1 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (3.0.9)
Requirement already satisfied: pillow>=6.2.0 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (9.2.0)
Requirement already satisfied: cycler>=0.10 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from matplotlib->-r requirements/runtime.txt (line 1)) (4.34.4)
Requirement already satisfied: six>=1.5 in /root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->-r requirements/runtime.txt (line 1)) (1.16.0)
Looking in indexes: http://mirrors.cloud.aliyuncs.com/pypi/simple/
WARNING: The repository located at mirrors.cloud.aliyuncs.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirrors.cloud.aliyuncs.com'.
ERROR: Could not find a version that satisfies the requirement mmdet==2.25.0 (from versions: none)
ERROR: No matching distribution found for mmdet==2.25.0
pip download failed with args: ['mmdet==2.25.0', '-d', '/tmp/tmpe6838qar', '--no-deps', '--no-binary', ':all:']
ERROR: Exception:
Traceback (most recent call last):
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 399, in run
    conflicts = self._determine_conflicts(to_install)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 531, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/operations/check.py", line 101, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/operations/check.py", line 42, in create_package_set_from_installed
    dependencies = list(dist.iter_dependencies())
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/pip/_internal/metadata/pkg_resources.py", line 200, in iter_dependencies
    return self._dist.requires(extras)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/mim/commands/install.py", line 169, in patched_requires
    mmdeps_text = get_mmdeps_from_mmpkg(self.project_name,
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/mim/commands/install.py", line 283, in get_mmdeps_from_mmpkg
    mminstall_content = get_mminstall_from_pypi(mmpkg, index_url=index_url)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/site-packages/mim/commands/install.py", line 313, in get_mminstall_from_pypi
    exit(status_code)
  File "/root/miniconda3/envs/pytorch110cu111/lib/python3.8/_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
SystemExit: 1
(pytorch110cu111) [root@iZuf602264ukmv152q0po9Z mmclassification]# pip install -r requirements
ERROR: Could not open requirements file: [Errno 21] 是一个目录: 'requirements'

Reproduction

  1. What command or script did you run?
A placeholder for the command.
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Error when "mim install mmdet" on Windows10

On Windows 10, run "mim install mmdet":

ERROR: Failed building wheel for pycocotools
Failed to build pycocotools
ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

mim install should prompt the supported built version of mmcv-full

Describe the feature
mim install should print all of the build versions of mmcv-full supported by the current environment.

Motivation

Related resources
If there is an official code release or third-party implementations, please also provide the information here, which would be very helpful.

Additional context
Add any other context or screenshots about the feature request here.
If you would like to implement the feature and create a PR, please leave a comment here and that would be much appreciated.

mim.train number of GPUs is not update in logged config

When using mim.train with specified number of GPUs, training is done on the specified GPUs, but logged config is not updated accordingly.

from mim import train

work_dir = "/xxx"
path_config = "/yyy/my_config.py"

random_args = (
    "--work-dir",
    work_dir,
)

train(
    package="mmtrack",
    config=path_config,
    gpus=2,
    launcher="pytorch",
    other_args=random_args,
)

output:

...
work_dir = '/xxx'
gpu_ids = [0]

But when looking at batch size and GPUs usage, both GPUs are correctly used.

note: also the readme seams to be wrong on how to use mim.train. I had to use package and put the other_args in a list.

error with mmyolo

Please help me, trying to test download mmyolo config via python:

env:

openmim==0.3.2
mmyolo=0.1.1
mmengine==0.1.0
mmcv==2.0.0rc2
mmdet==3.0.0rc2

command:

from mim import download

filepath = download("mmyolo", ["yolov5_n-v61_syncbn_fast_8xb16-300e_coco"], dest_root=".")

error:

[.../lib/python3.8/site-packages/mmyolo/model-index.yml or .../lib/python3.8/site-packages/mmyolo/model_zoo.yml is not found, please upgrade your mmyolo to support search command

Error while installing mmseg

Installation commands:

$ conda create -n mim python=3.7 -y
$ conda activate mim
$ conda install pytorch=1.7.0 torchvision torchaudio cudatoolkit=11.0 -c pytorch
$ pip install openmim
$ mim install mmseg

Error message:

installing mmseg from wheel.
Collecting mmseg
  Using cached mmseg-1.3.0.tar.gz (817 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/wolf/Applications/anaconda3/envs/mim/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pzv486bm/mmseg_1a4e72c8212d4085bdb0f2778d4bd53a/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pzv486bm/mmseg_1a4e72c8212d4085bdb0f2778d4bd53a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-fymx11nb
         cwd: /tmp/pip-install-pzv486bm/mmseg_1a4e72c8212d4085bdb0f2778d4bd53a/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-pzv486bm/mmseg_1a4e72c8212d4085bdb0f2778d4bd53a/setup.py", line 6, in <module>
        reload(sys)
    NameError: name 'reload' is not defined
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/f8/31/3bc9205f39cc8ab37193a6fbb24693993b2f305aba9f35b09fad882107ee/mmseg-1.3.0.tar.gz#sha256=f8878cddde0e96b7c70ff457edf662e6741716e71723a46a08b9efdcf9e3542d (from https://pypi.org/simple/mmseg/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached mmseg-1.2.4.tar.gz (1.6 MB)
    ERROR: Command errored out with exit status 1:
     command: /home/wolf/Applications/anaconda3/envs/mim/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pzv486bm/mmseg_63ef6e91598f462b8ec783045bc3f28b/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pzv486bm/mmseg_63ef6e91598f462b8ec783045bc3f28b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-_uhoraow
         cwd: /tmp/pip-install-pzv486bm/mmseg_63ef6e91598f462b8ec783045bc3f28b/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-pzv486bm/mmseg_63ef6e91598f462b8ec783045bc3f28b/setup.py", line 3, in <module>
        reload(sys)
    NameError: name 'reload' is not defined
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/c9/3a/a38acf3f3ae401082d74cd283d29a873dd3d1b77dc0c7fb28b80f0348073/mmseg-1.2.4.tar.gz#sha256=e7ba753b7bbefe54b4a7c2e27ed6714d1349e30ad6db9d7f67ce67b80b818f4c (from https://pypi.org/simple/mmseg/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement mmseg (from versions: 1.2.4, 1.3.0)
ERROR: No matching distribution found for mmseg
Traceback (most recent call last):
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/utils/utils.py", line 390, in call_command
    subprocess.check_call(cmd)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', '-m', 'pip', '--default-timeout', '45', 'install', 'mmseg']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wolf/Applications/anaconda3/envs/mim/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/commands/install.py", line 107, in cli
    is_editable=is_editable)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/commands/install.py", line 210, in install
    is_user_dir)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/commands/install.py", line 533, in install_from_wheel
    call_command(install_cmd)
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/utils/utils.py", line 392, in call_command
    raise highlighted_error(e)  # type: ignore
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/mim/utils/utils.py", line 364, in highlighted_error
    return click.style(msg, fg='red', bold=True)  # type: ignore
  File "/home/wolf/Applications/anaconda3/envs/mim/lib/python3.7/site-packages/click/termui.py", line 519, in style
    return "".join(bits)
TypeError: sequence item 2: expected str instance, CalledProcessError found

Error when using mim list

Describe the bug
There is an error when using commad mim list.
The version of mim is 0.1.3.

Reproduction

  1. What command or script did you run?
    mim list
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
    No

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
sys.platform: linux
Python: 3.7.0 (default, Oct  9 2018, 10:31:47) [GCC 7.3.0]
CUDA available: False
GCC: gcc (GCC) 5.4.0
PyTorch: 1.8.0+cu111
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0+cu111
OpenCV: 4.5.2
MMCV: 1.3.9
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: not available
MMDetection: 2.14.0+1419e93
  1. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [pip]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback

Traceback (most recent call last):
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/commands/list.py", line 27, in cli
    table_data = list_package(all=all)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/commands/list.py", line 55, in list_package
    installed_path = get_installed_path(pkg_name)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/utils/utils.py", line 77, in wrapper
    return func(package)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/utils/utils.py", line 318, in get_installed_path
    return osp.join(pkg.location, package2module(package))
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/utils/utils.py", line 77, in wrapper
    return func(package)
  File "/mnt/lustre/linguangchen/anaconda3/envs/pt1.8/lib/python3.7/site-packages/mim/utils/utils.py", line 295, in package2module
    highlighted_error(f'can not infer the module name of {package}'))
ValueError: can not infer the module name of threadpoolctl

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.