fyu / lsun Goto Github PK
View Code? Open in Web Editor NEWLSUN Dataset Documentation and Demo Code
LSUN Dataset Documentation and Demo Code
I'm sorry to bother you, but I do receive this error after download about 444MB of the LSUN for bedroom category.
Does it mean that I was shutdown by the server? Or am I doing something wrong?
I just run 'python download.py -c bedroom' in my remote Ubuntu server.
I'm looking forward for your advice, thanks!
python download.py -c test
python data.py export ~/Downloads/test_lmdb --out_dir .
sequence item 0: expected str instance, int found
Any help would be appreciated.
I get a 404 error when I access the link (http://dl.yf.io/lsun/scenes/), therefore cannot download datasets. Can you please address the problem?
I cannot download anything using wget http://dl.yf.io/lsun/objects/<class>.zip
How do i extract test dataset for each category?
I have downloaded the lsun data and extracted bedroom_val_lmdb.zip into bedroom_val_lmdb, then I used the following command to view the data:
python data.py view bedroom_val_lmdb
However, following error happened:
lmdb.Error: bedroom_val_lmdb: No locks available
Could you please tell me what's wrong? Thx.
I can't find layout dataset in this project. When i export image with data.py. The data only include some .webp image.Where is the layout label? Thank you.
Hello, I'm looking for the LSUN layout dataset, do you have this dataset? I have been searching for it for a very long time, and would be deeply thankful if you can provide it.
It would be nice to add this to the README. It's not obvious to me what this acronym is after reading through the github repository, paper, or related web content.
I just try to export_data.py export tower_train_lmdb --out_dir data
Traceback (most recent call last):
File "export_data.py", line 65, in
main()
File "export_data.py", line 61, in main
export_images(lmdb_path, args.out_dir)
File "export_data.py", line 39, in export_images
image_out_dir = join(out_dir, '/'.join(key[:6]))
TypeError: sequence item 0: expected str instance, int found
How can I solve this question,btw my export fuction dont use 'flat' parameter
I downloaded data from the 20 LSUN object categories listed here: http://tigress-web.princeton.edu/~fy/lsun/public/release/ but I get the following error when I try to open the images:
OSError: cannot identify image file <_io.BytesIO object at 0x7f615045c3b8>
This surprised me because my code works just fine with the LSUN scene recognition dataset. For reference, here's the rest of the code:
from torch.utils.data import Dataset
from PIL import Image
import os
import os.path
import io
import string
import sys
import lmdb
import pickle
class LSUN_object(Dataset):
def __init__(self, root, target, transform=None, target_transform=None):
super(LSUN_object, self).__init__()
self.transform = transform
self.target = target
self.target_transform = target_transform
self.env = lmdb.open(root, max_readers=1, readonly=True, lock=False, readahead=False, meminit=False, subdir=True)
with self.env.begin(write=False) as txn:
self.length = txn.stat()['entries']
cache_file = '_cache_' + ''.join(c for c in root if c in string.ascii_letters)
if os.path.isfile(cache_file):
self.keys = pickle.load(open(cache_file, "rb"))
else:
with self.env.begin(write=False) as txn:
self.keys = [key for key, _ in txn.cursor()]
pickle.dump(self.keys, open(cache_file, "wb"))
def __len__(self):
return self.length
def __getitem__(self, index):
img, target = None, None
env = self.env
with env.begin(write=False) as txn:
imgbuf = txn.get(self.keys[index])
buf = io.BytesIO()
buf.write(imgbuf)
buf.seek(0)
# Fails here, but I don't know why? Why would it not be an image in the object dataset?
img = Image.open(buf).convert('RGB')
target = self.target
if self.transform is not None:
img = self.transform(img)
if self.target_transform is not None:
target = self.target_transform(target)
return img, target
Is there any significant difference between scenes and objects that I'm not seeing?
After multiple attempts to run python3 download.py -c church_outdoor
, I keep getting this error after few hours of downloading
Downloading church_outdoor train set
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
29 2335M 29 686M 0 0 180k 0 3:41:07 1:04:57 2:36:10 186k
curl: (18) transfer closed with 1729112674 bytes remaining to read
Downloading church_outdoor val set
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5817k 100 5817k 0 0 118k 0 0:00:48 0:00:48 --:--:-- 173k
I try to utilize this dataset to train my network.But After I export those dataset,I found those're .webp file. So could you tell me how to convert to .png?
PS: When I directly train my network with lmdb files in the dataset,an error occurred .Maybe lmdb files have somthing wrong I guess.
error:
F0326 01:27:43.210084 15934 data_transformer.cpp:734] Check failed: datum_channels > 0 (0 vs. 0)
Hi @fyu ,
I think the link for downloading the object categories dl.yf.io/lsun/objects/ is down. I wanted to know if the URL is changed?
Thank you,
Shenoy
Is there any possibility to post somewhere the mean and std of lsun dataset for each channel?
Thanks.
the LSUN download URL seems to have stopped working:
http://lsun.cs.princeton.edu/htbin/list.cgi?tag=bedroom
http://lsun.cs.princeton.edu/challenge/2015/roomlayout/data/
changing to downside website
http://dl.yf.io/lsun/scenes/
Hi,
I don't know what is the category for each test image. The file data.py only extract the information, don't create the file which you tell ('The submitted results on the testing set will be stored in a text file with one line per image'). Please, how can I extract these information?
Thanks for the dataset, its very useful.
I would like to recommend uploading the dataset to Academic Torrents, such that we can download it much faster.
Although requests
has been installed via pip, I get the error that no module is named as request.
$ pip install --user request
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting request
Downloading https://files.pythonhosted.org/packages/f1/27/7cbde262d854aedf217061a97020d66a63163c5c04e0ec02ff98c5d8f44e/request-2019.4.13.tar.gz
Collecting get (from request)
Downloading https://files.pythonhosted.org/packages/3f/ef/bb46f77f7220ac1b7edba0c76d810c89fddb24ddd8c08f337b9b4a618db7/get-2019.4.13.tar.gz
Collecting post (from request)
Downloading https://files.pythonhosted.org/packages/0f/05/bd79da5849ea6a92485ed7029ef97b1b75e55c26bc0ed3a7ec769af666f3/post-2019.4.13.tar.gz
Requirement already satisfied: setuptools in /home/mahmood/.local/lib/python2.7/site-packages (from request) (41.0.1)
Collecting query_string (from get->request)
Downloading https://files.pythonhosted.org/packages/12/3c/412a45daf5bea9b1d06d7de41787ec4168001dfa418db7ec8723356b119f/query-string-2019.4.13.tar.gz
Collecting public (from query_string->get->request)
Downloading https://files.pythonhosted.org/packages/54/4d/b40004cc6c07665e48af22cfe1e631f219bf4282e15fa76a5b6364f6885c/public-2019.4.13.tar.gz
Building wheels for collected packages: request, get, post, query-string, public
Building wheel for request (setup.py) ... done
Created wheel for request: filename=request-2019.4.13-cp27-none-any.whl size=1677 sha256=635c9a974941fd509fd7dfe5f47bf12a96489c47c67f748a8672ff858c40ff52
Stored in directory: /home/mahmood/.cache/pip/wheels/30/84/5f/484cfba678967ef58c16fce6890925d5c7172622f20111fbfd
Building wheel for get (setup.py) ... done
Created wheel for get: filename=get-2019.4.13-cp27-none-any.whl size=1691 sha256=6db08500aa1c1620b31332e0ca0c167927bd6dd2b0e4ec25039cfe13fa232e15
Stored in directory: /home/mahmood/.cache/pip/wheels/c1/e3/c1/d02c8c58538853e4c9b78cadb74f6d5c5c370b48a69a7271aa
Building wheel for post (setup.py) ... done
Created wheel for post: filename=post-2019.4.13-cp27-none-any.whl size=1661 sha256=079dee3764bce7c1f6a44eae59940c4a529c0cd3a0ab10342be4697697fb020c
Stored in directory: /home/mahmood/.cache/pip/wheels/c3/c3/24/b5c132b537ab380c02d69e6bd4dec1f5db56b5fe19030473d7
Building wheel for query-string (setup.py) ... done
Created wheel for query-string: filename=query_string-2019.4.13-cp27-none-any.whl size=2048 sha256=d91766531ceb491430edddb1fd8be7629137904c0c3b0814e1a5709618e75c91
Stored in directory: /home/mahmood/.cache/pip/wheels/d6/a4/78/01b20a9dc224dcc009fab669f7f27b943b8889c5150bd68d8a
Building wheel for public (setup.py) ... done
Created wheel for public: filename=public-2019.4.13-cp27-none-any.whl size=2537 sha256=226deaaa53d129077c1778b2902044b4efd00e67fa5d97113338abdd630ba285
Stored in directory: /home/mahmood/.cache/pip/wheels/23/7c/6e/f5b4e09d6596c8b8802b347e48f149031e2363368048f1347a
Successfully built request get post query-string public
Installing collected packages: public, query-string, get, post, request
Successfully installed get-2019.4.13 post-2019.4.13 public-2019.4.13 query-string-2019.4.13 request-2019.4.13
$ pip install --user requests
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Requirement already satisfied: requests in /home/mahmood/.local/lib/python2.7/site-packages (2.22.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (1.25.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (2019.6.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python2.7/dist-packages (from requests) (2.6)
WARNING: You are using pip version 19.2.1, however version 19.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ python download.py -c bedroom
Traceback (most recent call last):
File "download.py", line 8, in <module>
from urllib.request import Request, urlopen
ImportError: No module named request
Any thought?
The download link cannot be accessed now.
The datasets download from "http://dl.yf.io/lsun/objects/airplane.zip"
My command line input is “D:\Project\Lsun>python data.py export D:/dataset/airplane --out_dir D:/dataset/airplan
e/image --flat”
I want to use the torchvision.datasets.ImageFolder()to load the image, but prompt a error about
"PIL. UnidentifiedImageError: always identify a image file"
Datasets images display normally on windows10.
What should I do?
Thanks
When I extract the data I do not get a .txt or meta-data file that. Which is required here: https://github.com/CompVis/latent-diffusion/blob/main/configs/latent-diffusion/lsun_churches-ldm-kl-8.yaml and mentioned here: CompVis/latent-diffusion#190
At some point, the LSUN download URL seems to have stopped working: http://lsun.cs.princeton.edu/htbin/list.cgi?tag=bedroom
Hello,
in
Line 21 in 7fea09d
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:12 --:--:-- 0
91 3127M 91 2873M 0 0 2208k 0 0:24:09 0:22:12 0:01:57 2473k
curl: (18) transfer closed with 266626630 bytes remaining to read
The partially downloaded classroom zip file has caused a subsequent unzip exception.
Maybe it can be solved by adding the -C -
flag?
Thank you
Can you please guide me the way that I need to send data to subprocess.call on windows machine
I suspect that the args/parmater curl should be different format, looks like below format is not accepting and getting below mentioned errors
my machine : OS Name Microsoft Windows Server 2016 Datacenter
Tried in both Python3 and Python2, but getting the same error.
Please help me on the same.
def download(out_dir, category, set_name, tag):
url = 'http://lsun.cs.princeton.edu/htbin/download.cgi?tag={tag}'
'&category={category}&set={set_name}'.format(**locals())
if set_name == 'test':
out_name = 'test_lmdb.zip'
else:
out_name = '{category}_{set_name}_lmdb.zip'.format(**locals())
out_path = join(out_dir, out_name)
cmd = ['curl', url, '-o', out_path]
print('Downloading', category, set_name, 'set')
subprocess.call(cmd, shell=True)
But getting below error
'curl' is not recognized as an internal or external command,
operable program or batch file.
'category' is not recognized as an internal or external command,
operable program or batch file.
The syntax of the command is incorrect.
Downloading bridge val set
'curl' is not recognized as an internal or external command,
operable program or batch file.
'category' is not recognized as an internal or external command,
operable program or batch file.
The syntax of the command is incorrect.
Hi @fyu , seem like the server is down. Can you check ?
Whe I export churches_lmdb to churches by cmd:
python3 data.py export churches_lmdb --out_dir churches
The directory structure like this:
but the meta list is just only show the filename of the images:
I get the filelist from https://ommer-lab.com/files/lsun.zip
Is there any filelist that include the right subpath of images?
In my folder: bedroom_val, there are data.mdb and lock.mdb.
Then I use
export_images('./bedroom_val', './data',True)
result:
Traceback (most recent call last):
File "/Users/pegasus/Downloads/lsun-master/data.py", line 90, in <module>
export_images('./bedroom_val', './data',True)
File "/Users/pegasus/Downloads/lsun-master/data.py", line 48, in export_images
image_out_path = join(image_out_dir, key + '.webp')
TypeError: can't concat str to bytes
Then i added key.decode() where
image_out_path = join(image_out_dir, key.decode() + '.webp')
However, there still has an error:
Traceback (most recent call last):
File "/Users/pegasus/Downloads/lsun-master/data.py", line 90, in <module>
export_images('./bedroom_val', './data',True)
File "/Users/pegasus/Downloads/lsun-master/data.py", line 50, in export_images
fp.write(val)
TypeError: write() argument must be str, not bytes
I don't know how to fix this problem where
with open(image_out_path, 'w') as fp:
fp.write(val)
Would it make sense for this download utility to support Python 3 as well?
On the first glance, making lsun
work on both 2.7 and 3.x looks straightforward. Please let me know if this is something that may actually be helpful and I can work on the changes. Thanks.
Hello,
I just download the bedroom data and right now I have in lsun-folder a bedroom_train_lmdb.zip. I tried to extract jpg from it using the command "python data.py export bedroom_train_imdb.zip/bedroom_train_lmdb --out_dir bedroom" and I receive the following error: lmdb.Error: bedroom_train_lmdb.zip/bedroom_train_lmdb: The system cannot find the path specified.
Same error for command: "python data.py export bedroom_train_imdb.zip --out_dir bedroom
Could you please give me a hint what am I doing wrong?
Thanks,
Diana
Hello, teacher. I am a student and saw a paper you published in 2015 is LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. The paper gives the code to download the LSUN dataset. However, after I download it, the decompression always gives an error and the display data has been corrupted. Would you like to ask the teacher if there are other solutions? In addition, I hope that the teacher can share the data of this part of bedroom. I‘m looking forward to your reply. Thank you!
The link for the dataset is broken.
Many projects such as pytorch and researchers rely on this source to download LSUN dataset.
These files should be mirrored to high-speed hosts - on a gigabit down line, download times in excess of 12 hours seem common for only 20gb.
Is this normal?
running python3 download.py -c church_outdoor
but getting an error
Traceback (most recent call last):
File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.8/http/client.py", line 1256, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 951, in send
self.connect()
File "/usr/lib/python3.8/http/client.py", line 922, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 113] No route to host
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "download.py", line 59, in <module>
main()
File "download.py", line 41, in main
categories = list_categories()
File "download.py", line 17, in list_categories
with urlopen(Request(url)) as response:
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 113] No route to host>
Are there training.mat and validation.mat files in this dataset that i can download? im trying to use the opensource implementation for the roomnet and it requires .mat files and i can't seem to find them or download them . if anyone knows where i can find them it would be really helpful . thank you in advance
I get a 404 error when I tried downloading following the instructions., therefore cannot download datasets. I think the server is down now. Can you please address the problem?
python3 download.py -c bedroom
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 712, in create_connection
raise err
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
OSError: [Errno 51] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "download.py", line 59, in
main()
File "download.py", line 41, in main
categories = list_categories()
File "download.py", line 17, in list_categories
with urlopen(Request(url)) as response:
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1282, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1256, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 51] Network is unreachable>
TEST dataset has only label 0. Why is that and how can we get the labels of the testset?
The server seems to be down now 😞.
At website http://complexity.cecs.ucf.edu/lsun/ I notice that there is a car dataset. I followed the guidance on the website. But there is no permission for ordinary prople.
The dataset can be downloaded either from GitHub https://github.com/fyu/lsun or the categories lists on http://tigress-web.princeton.edu/~fy/lsun/public/release/. More details are available on the dataset page http://www.yf.io/p/lsun.
Hi
I haven't been able to find any information on the license attached to this dataset. Am I allowed to use this data for my professional work? It would mostly revolve around testing out data augmentation methods and / or directly training a classifier using some of the dataset ?
Thanks
Hi,
When I run the download script as in the README file, for bedroom category, I get the following error:
http.client.RemoteDisconnected: Remote end closed connection without response
I looked at http://dl.yf.io/ and seems like it's down.
How can this be fixed to download the dataset?
When i run
python download.py
I get the error
Traceback (most recent call last):
File "download.py", line 62, in <module>
main()
File "download.py", line 43, in main
categories = list_categories(args.tag)
File "download.py", line 20, in list_categories
return json.loads(f.read())
File "/Users/sorensonderby/anaconda/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/Users/sorensonderby/anaconda/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/sorensonderby/anaconda/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
How do i solve this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.