[KED-862] Implement dataset versioning on CSVBlobDataSet about kedro HOT 13 CLOSED

@921kiyo I'm happy to pick this up

from kedro.

@evanmiller29 Seems like the docs did not build properly, we have manually rebuilt it just now and the docs should be up to date for 0.15.0 now (browser might have cached the old version) :)

from kedro.

@evanmiller29 We have a documentation about how to feed credentials to a dataset in https://kedro.readthedocs.io/en/latest/04_user_guide/04_data_catalog.html#feeding-in-credentials :)

from kedro.

It looks like this hasn't been updated in a while, so I'm happy to make a PR (I have a copy in my fork mzjp2@8db124b) but equally, also happy to defer to Evan if he's back. :)

from kedro.

Here's the link for the pull request - #76

from kedro.

@921kiyo - on https://kedro.readthedocs.io/en/latest/04_user_guide/07_advanced_io.html?#versioning it mentions kedro.io.core.FilepathVersionMixin but I can't find it in kedro.io.core. Should it be AbstractDataSet?

from kedro.

@evanmiller29
I think the statement is out of date. We have just release 0.15.0 yesterday and merged FilepathVersionMixIn and S3VersionMixIn under one abstract class AbstractVersionedDataSet. See the "Breaking changes to the API" section for 0.15.0 in RELEASE.md. I will update the docs.

from kedro.

Ahhh cool. No worries - I thought I'd just raise it anyway so you know.

I'll give it a read. It's looking good to me so far

from kedro.

@921kiyo - I'm looking at the CSVBlobDataSet docs and I'm wondering how you pass in the account credentials to kedro to enable saving a CSV to Azure?

A simple code stub would be great. Once this is all done I'd be happy to add it to the docs.

Evan

EDIT: Also wondering if adding it to the example catalog.yml might be a good idea too

from kedro.

@921kiyo - just an FYI:

import pandas as pd

data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5],
'col3': [5, 6]})

data_set = CSVBlobDataSet(filepath="test.csv",
bucket_name="test_bucket",
load_args=None,
save_args={"index": False})

TypeError: __init__() got an unexpected keyword argument 'bucket_name'

I think bucket_name should be container_name

from kedro.

@evanmiller29 Thanks for letting us know. It is a typo in the docstring. We will fix it :)

from kedro.

from kedro.

Thanks @evanmiller29 ! :)

from kedro.