Importing blob files about sodapy HOT 9 CLOSED

imports2 seems to be fully supported by Socrata. At least in their soda-java. https://github.com/socrata/soda-java/blob/master/src/main/java/com/socrata/api/SodaImporter.java#L33

from sodapy.

Yea, that'd be rad if you want to add a file upload endpoint! Here is the backstory on the import2 api if you're interested.

from sodapy.

Hey guys, I just had a good chat with @PeterAustinMoore from Socrata yesterday. Apparently the imports2 endpoint is the way forward. It's old, as @chrismetcalf pointed out, but the code behind it has been updated and it's what will likely be supported. An important caveat: it's not officially supported, and only used internally at this point, so use at your own risk!

He walked me through some sample code that you could construct by walking through the "create geospatial dataset" wizard on the web interface. It basically uploads a file as the first call, which returns a file id. You then make another request using that file ID to turn it into a dataset and import its data as rows. This request returns the /api/views/xxxx-xxxx.json response that we've seen before.

For geospatial data, the /api/views/... response contains a childViews property. It's an array of the shapefile layers that were imported. Those layers are the actual data, and what you'd want to connect datasync or something else to. Alternatively, you can use the replace methods on the imports2 endpoint.

It sounds like this is the intended creation process for any dataset, whether tabular, geospatial, or blobular. I just made up the word blobular but I'm going to stick with it. There's a method querystring (I believe that's what it is) that has a few options.

Sounds like it's worth touching up the overall create() method of this library, though it's definitely a breaking change, as it requires a file to use it. Any thoughts so far or info you guys know that I don't?

from sodapy.

Thanks for the thoughts @timwis. Great to know that imports2 won't disappear at some point. I think you generally covered what I added with #17. My goal was not to support shapefile imports though, just non-row based data. They sound like similar processes.

from sodapy.

I'm afraid that refactoring the existing create might be squeezing too much distinct functionality into one method, and support the creation of an additional create method to upload and publish file-based datasets. I probably won't roll a new version of sodapy until the new create can handle geospatial data as well, but the functionality of the current PR looks fine for a merge.

from sodapy.

Opening a new issue for shapefiles. This is not a pressing need for me at this point. Maybe someone else could pick it up.

from sodapy.

Hey all, jumping in late to the discussion, but let me know if you need help. It looks like we're going to officially expose the imports2 API to learn more about how people will use it while we design 3.0, so let me know if you have feedback or questions! 🎉

from sodapy.

@chrismetcalf expose as in document? :D :D

from sodapy.

@timwis Correct! No guarantees that everything on your wishlist will be in there, but at least it'll be something to start a discussion around

from sodapy.