我在国内 从你们那边跑例子 下数据下不下来怎么解决 about federatedscope HOT 22 CLOSED

Welcome :>

我已经提交了DingDing申请 这边示例代码也运行成功了 真心感谢

from federatedscope.

@rayrayraykk Hi, I think this bug is due to the different behavior of os.path.join on Windows and Linux.
In leaf_cv.py, the URL passed to the function download_url() is yield by osp.join(url, name) which will yield
https://federatedscope.oss-cn-beijing.aliyuncs.com\femnist_all_data.zip
on windows and https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip on Linux, since the default os.sep is different. ("\" for windows).
Besides URL parsing, this may also trigger issue in file = url.rpartition('/')[2] in download_url().

To fix this, I reproduce the aforementioned error:

File "C:\Users\HONGYI001\Anaconda3\envs\VFL\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>

Then I change osp.join(url, name) to url = url + '/' + name and it works:

2022-05-18 20:57:19,371 (utils:90) INFO: the output dir is exp\sub_exp_20220518205719
2022-05-18 20:57:20,797 (utils:215) INFO: Downloading https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip
https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip
Preprocess data (Please leave enough space)...

It's a small problem. Please consider fixing this. Thanks

from federatedscope.

return opener.open(url, data, timeout)

File "F:\Anaconda3\envs\Federated\lib\urllib\request.py", line 517, in open
response = self._open(req, data)
File "F:\Anaconda3\envs\Federated\lib\urllib\request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "F:\Anaconda3\envs\Federated\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "F:\Anaconda3\envs\Federated\lib\urllib\request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "F:\Anaconda3\envs\Federated\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>

from federatedscope.

Thanks for your interest :) I have assigned this thread to @rayrayraykk

from federatedscope.

Please provide the detailed command or config you used, thanks.

from federatedscope.

Please provide the detailed command or config you used, tha
conda install --file enviroment/requirements-torch1.10.txt -c pytorch -c conda-forge -c nvidia

use code in quickstart in my computer not jupyterlab
https://try.federatedscope.io/user/heliumzzz/lab/tree/01_quick_start.ipynb

from federatedscope.

Please provide the detailed command or config you used, thanks.

from federatedscope.core.auxiliaries.data_builder import get_data

cfg.data.type = 'femnist'
cfg.data.splits = [0.6, 0.2, 0.2]
cfg.data.batch_size = 10
cfg.data.subsample = 0.05
cfg.data.transform = [['ToTensor'], ['Normalize', {'mean': [0.1307], 'std': [0.3081]}]]

data, modified_cfg = get_data(cfg.clone())
cfg.merge_from_other_cfg(modified_cfg)

from federatedscope.

Please provide the detailed command or config you used, thanks.

error in get_data

from federatedscope.

Can you download the data file from
https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

from federatedscope.

Can you download the data file from https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

i can

from federatedscope.

Can you download the data file from https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

下一步我应该如何处理数据 和getdata返回值里一样

from federatedscope.

Can you download the data file from https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

下一步我应该如何处理数据 和getdata返回值里一样

Here are two possible solutions:

1. Update your urllib and ssl packages in your python env.
2. As the cfg.data.root='data', you can move femnist_all_data.zip to data/femnist/raw and try again.

Hope this will help.

from federatedscope.

Can you download the data file from https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

下一步我应该如何处理数据 和getdata返回值里一样

Here are two possible solutions:

1. Update your urllib and ssl packages in your python env.
2. As the cfg.data.root='data', you can move femnist_all_data.zip to data/femnist/raw and try again.

Hope this will help.

urllib3 1.26.9

from federatedscope.

我将数据放到了data下的 raw目录
from federatedscope.core.auxiliaries.data_builder import get_data

cfg.data.root = 'data'
cfg.data.splits = [0.6, 0.2, 0.2]
cfg.data.batch_size = 10
cfg.data.subsample = 0.05
cfg.data.transform = [['ToTensor'], ['Normalize', {'mean': [0.1307], 'std': [0.3081]}]]

data, modified_cfg = get_data(cfg.clone())

cfg.merge_from_other_cfg(modified_cfg)
这是我的代码

Traceback (most recent call last):
File "F:\Federated\trainer.py", line 47, in
Fed_runner = FedRunner(data=data,
File "F:\Anaconda3\envs\Federated\lib\site-packages\federatedscope-0.1.0-py3.9.egg\federatedscope\core\fed_runner.py", line 38, in init
self._setup_for_standalone()
File "F:\Anaconda3\envs\Federated\lib\site-packages\federatedscope-0.1.0-py3.9.egg\federatedscope\core\fed_runner.py", line 50, in _setup_for_standalone
self.server = self._setup_server()
File "F:\Anaconda3\envs\Federated\lib\site-packages\federatedscope-0.1.0-py3.9.egg\federatedscope\core\fed_runner.py", line 158, in _setup_server
model = get_model(self.cfg.model,
File "F:\Anaconda3\envs\Federated\lib\site-packages\federatedscope-0.1.0-py3.9.egg\federatedscope\core\auxiliaries\model_builder.py", line 54, in get_model
model = get_cnn(model_config, local_data)
File "F:\Anaconda3\envs\Federated\lib\site-packages\federatedscope-0.1.0-py3.9.egg\federatedscope\cv\model\model_builder.py", line 14, in get_cnn
data = next(iter(local_data['train']))
TypeError: 'NoneType' object is not iterable

Process finished with exit code 1

from federatedscope.

Can you download the data file from https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip

下一步我应该如何处理数据 和getdata返回值里一样

Here are two possible solutions:

1. Update your urllib and ssl packages in your python env.
2. As the cfg.data.root='data', you can move femnist_all_data.zip to data/femnist/raw and try again.

Hope this will help.

我的urllib3版本是1.26.9

from federatedscope.

Make sure the data directory contains:

-data
--femnist
---raw
----femnist_all_data.zip

If your data directory contains processed:

-data
--femnist
---raw
----all_data
----femnist_all_data.zip
---processed
----xxxxxxx

Please delete processed and try again.

from federatedscope.

from federatedscope.

Make sure the data directory contains:

-data
--femnist
---raw
----femnist_all_data.zip

If your data directory contains processed:

-data
--femnist
---raw
----all_data
----femnist_all_data.zip
---processed
----xxxxxxx

Please delete processed and try again.

但是还是报同样的错误

from federatedscope.

Make sure the data directory contains:

-data
--femnist
---raw
----femnist_all_data.zip

If your data directory contains processed:

-data
--femnist
---raw
----all_data
----femnist_all_data.zip
---processed
----xxxxxxx

Please delete processed and try again.

但是还是报同样的错误

Please join our DingDing group for your case, thanks. The QR code can be found in:
https://federatedscope.io/docs/community/

from federatedscope.

谢谢您的耐心解答 真心感谢

from federatedscope.

Welcome :>

from federatedscope.

@rayrayraykk Hi, I think this bug is due to the different behavior of os.path.join on Windows and Linux. In leaf_cv.py, the URL passed to the function download_url() is yield by osp.join(url, name) which will yield https://federatedscope.oss-cn-beijing.aliyuncs.com\femnist_all_data.zip on windows and https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip on Linux, since the default os.sep is different. ("" for windows). Besides URL parsing, this may also trigger issue in file = url.rpartition('/')[2] in download_url().

To fix this, I reproduce the aforementioned error:

File "C:\Users\HONGYI001\Anaconda3\envs\VFL\lib\urllib\request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>

Then I change osp.join(url, name) to url = url + '/' + name and it works:

2022-05-18 20:57:19,371 (utils:90) INFO: the output dir is exp\sub_exp_20220518205719
2022-05-18 20:57:20,797 (utils:215) INFO: Downloading https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip
https://federatedscope.oss-cn-beijing.aliyuncs.com/femnist_all_data.zip
Preprocess data (Please leave enough space)...

It's a small problem. Please consider fixing this. Thanks

Thanks! We have fixed this in #95.

from federatedscope.