export lmdb to jpg has error about lsun HOT 11 OPEN

Hi,
I work with Ubuntu 18.04,
and i changed the function to:

def export_images(db_path, out_dir, flat=False, limit=-1):
    print('Exporting', db_path, 'to', out_dir)
    env = lmdb.open(db_path, map_size=1099511627776,
                    max_readers=100, readonly=True)
    count = 0
    with env.begin(write=False) as txn:
        cursor = txn.cursor()
        for key, val in cursor:
            if not flat:
                image_out_dir = join(out_dir, '/'.join(key[:6].decode()))
            else:
                image_out_dir = out_dir
            if not exists(image_out_dir):
                os.makedirs(image_out_dir)
            print('Current key:', key)
            image_out_path = join(image_out_dir, key.decode() + '.jpg')
            img = cv2.imdecode(
                numpy.fromstring(val, dtype=numpy.uint8), 1)
            cv2.imwrite(image_out_path, img)
            count += 1
            if count == limit:
                break
            if count % 1000 == 0:
                print('Finished', count, 'images')

and it works, all images save in image_out_dir.
but the problem is each image saved in separately,
so i recommend to add in finish --flat to your command, for example:

python data.py export ./bedroom_val_lmdb --out_dir ./bedroom_val --flat

it extracts that all images into one folder, so this is a more neat..

from lsun.

The changes below fixes this issue when running the script in a windows environment. Using .decode() for the key and 'wb' for writing in binary mode seems to work in my case. Modify these lines in the data.py script:

LINE 44 : image_out_dir = join(out_dir, '/'.join(key[:6].decode()))

LINE 50 : with open(image_out_path, 'wb') as fp:

from lsun.

@patriciaVitoria
line 49 :
image_out_path = join(image_out_dir, key.decode() + '.webp')

from lsun.

Is this still a problem? Can you print the key on that line?

from lsun.

@fyu, @ratsuki
I can confirm it is still a problem.
My environment is python 3.6 & opencv 3.4.1

from lsun.

I encounter the same problem, my env is python 3.6 . Didyou solve the problem ?
@ratsuki @fyu @shirishr

from lsun.

Same issue. They key prints fine : b'000013'. Any solutions ?

from lsun.

When i modify both lines below i get the following error:

python data.py export bedroom_val_lmdb --out_dir data
Exporting bedroom_val_lmdb to data
Traceback (most recent call last):
File "data.py", line 89, in
main()
File "data.py", line 85, in main
export_images(lmdb_path, args.out_dir, args.flat)
File "data.py", line 49, in export_images
image_out_path = join(image_out_dir, key + '.webp')
TypeError: can't concat str to bytes

Anyone know how to solve it?

from lsun.

The changes below fixes this issue when running the script in a windows environment. Using .decode() for the key and 'wb' for writing in binary mode seems to work in my case. Modify these lines in the data.py script:

LINE 44 : image_out_dir = join(out_dir, '/'.join(key[:6].decode()))

LINE 50 : with open(image_out_path, 'wb') as fp:

It works on Ubuntu16.04 with python3.6, thanks!

from lsun.

it exports the image as webp, how do you convert it as jpg?

@arthurmchr @cippall

from lsun.

Hi,
I work with Ubuntu 18.04,
and i changed the function to:

def export_images(db_path, out_dir, flat=False, limit=-1):
    print('Exporting', db_path, 'to', out_dir)
    env = lmdb.open(db_path, map_size=1099511627776,
                    max_readers=100, readonly=True)
    count = 0
    with env.begin(write=False) as txn:
        cursor = txn.cursor()
        for key, val in cursor:
            if not flat:
                image_out_dir = join(out_dir, '/'.join(key[:6].decode()))
            else:
                image_out_dir = out_dir
            if not exists(image_out_dir):
                os.makedirs(image_out_dir)
            print('Current key:', key)
            image_out_path = join(image_out_dir, key.decode() + '.jpg')
            img = cv2.imdecode(
                numpy.fromstring(val, dtype=numpy.uint8), 1)
            cv2.imwrite(image_out_path, img)
            count += 1
            if count == limit:
                break
            if count % 1000 == 0:
                print('Finished', count, 'images')

and it works, all images save in image_out_dir.
but the problem is each image saved in separately,
so i recommend to add in finish --flat to your command, for example:

python data.py export ./bedroom_val_lmdb --out_dir ./bedroom_val --flat

it extracts that all images into one folder, so this is a more neat..

Hi, thanks for the sharing. When I use this code to extract jpg images from mdb, I got the error:
Traceback (most recent call last):
File "exportmdb.py", line 88, in
main()
File "exportmdb.py", line 84, in main
export_images(lmdb_path, args.out_dir, args.flat)
File "exportmdb.py", line 49, in export_images
img = cv2.imdecode(numpy.fromstring(val, dtype=numpy.uint8), 1)
cv2.error: OpenCV(4.5.2) ../modules/imgcodecs/src/loadsave.cpp:736: error: (-215:Assertion failed) !buf.e
mpty() in function 'imdecode_'

Do you know how to fix this? Thanks so much!

from lsun.