is there any way to read all file names in the sub directory? about scandir HOT 3 CLOSED

Hi @chanduthedev - scandir should be able to do this quite easily. If you know that the depth is always 2 (one master folder and sub-folders), then you can make a function that does a scandir loop within a scandir loop: scan_2_deep() in the example below. If the depth is variable, you'd have to make some kind of recursive function: one such function that scans n deep is shown in the example as scan_n_deep(). Here's a Python 3 program showing this:

import os
import sys


def scan_2_deep(path):
    for entry in os.scandir(path):
        if not entry.is_dir():
            # Skip non-directories in master folder
            continue

        for sub_entry in os.scandir(entry.path):
            if not sub_entry.is_file():
                # Skip non-files in sub-folder
                continue

            print(sub_entry.path)


def scan_n_deep(path, n):
    for entry in os.scandir(path):
        if entry.is_dir():
            if n > 1:
                scan_n_deep(entry.path, n-1)
        elif n == 1:
            print(entry.path)


#scan_2_deep(sys.argv[1])

scan_n_deep(sys.argv[1], int(sys.argv[2]))

Does that help?

from scandir.

Thank you for the quick reply @benhoyt. This scan_2_deep method basically iterate all the folders one by one rite. Same logic I implemented before posting this question. It was taking around 13-14mins to read all ~4million files from 2million folders.

I also tried with your method scan_2_deep method, it also took around 13mins 6 secs. where as existing (paths.list_images() in python) method i used took 14mins 21sec.

There is very less much improvement of iterating through over folders than paths.list_images(). I will be using paths.list_images().
Thank you so much for your quick response.

from scandir.

Sounds good!

from scandir.