Comments (5)
martindurant
commented on June 3, 2024
Indeed, gcsfs has not yet implemented file listing with prefix/delimiter in the way that s3fs has. The method used there could be copied (they should be very similar), but of course this takes some effort.
#22 might be of interest to you: it enables you to get info and open files for reading when you already know the specific path and don't need to list the bucket.
from gcsfs.
bmabey
commented on June 3, 2024
Okay, taking a look at the implementation we would need to switch out the use of list_bucket to use this endpoint instead: https://cloud.google.com/storage/docs/json_api/v1/objects/list
Is that what you were thinking?
from gcsfs.
martindurant
commented on June 3, 2024
Yes, I think that's it. It would require changing what we store in the cache.
from gcsfs.
martindurant
commented on June 3, 2024
Similar, but not duplicate of #21
from gcsfs.
asford
commented on June 3, 2024
Resolved with #57.
from gcsfs.
Related Issues (20)
- _put_file does not allow to specify content encoding HOT 4
- __init__() got multiple values for keyword argument 'listings_expiry_time' HOT 1
- Is there already a way to list metadata attributes? HOT 2
- gcsfs: Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object HOT 3
- Inconsistent `STORAGE_EMULATOR_HOST` format HOT 2
- GCSFileSystem does not accept token, file.json or instance of service credentials HOT 13
- Errors when deleting a directory with huge number of files HOT 13
- The info object in GCSFS does not have mtime/ctime HOT 2
- `find` performance regression HOT 4
- Stale cache info leading to failing isfile check HOT 2
- _find does not support maxdepth
- GCS High Performance Parallel Listing
- Strange behavior with `HTTPError` and multiprocessing HOT 3
- isdir/info method works incorrectly HOT 20
- Clarify how to pass JSON credentials HOT 2
- Is it needed to be so strict about dependency on fsspec HOT 1
- Release 2023.10.0 and consider relaxing fsspec dependency HOT 3
- Error when listing large directory with versions=True
- Request: add chmod
- Issues when using identity_pool.Credentials for connecting GCSFileSystem HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gcsfs.