Blog posts for working with Kedro and Databricks on the Azure platform about kedro-devrel HOT 2 OPEN

My thoughts after working with Databricks over the last few months:

• The core platform works interchangeably on Azure, AWS, GCP.
• One of the big levers on Azure identified in the original user research is that users on this platform will also use Azure DevOps CI/CD and Azure Data Lake.
• By working with users, I have seen the need for guidance on these use cases first-hand.
• The deployment documentation covers 90% of what is needed to add a run of a Databricks project to any CI/CD.
• I am cautious about adding documentation for these features, as it greatly increases the surface area of the docs that need maintenance. Maintaining them also requires Azure specialty.
• I think the right approach would be to write two blog posts:

1. Show how to mount an Azure Data Lake on Databricks and access the data using Kedro.
2. Show how to add a package and run of a Kedro project to an Azure Devops pipeline.

This eliminates the need for ongoing maintenance, but helps our users in the way we want to. I would be in favour of closing this ticket and creating two new ones for writing these blog posts.

What are your thoughts @yetudada, @merelcht, @SajidAlamQB, @AhdraMeraliQB, @stichbury ?

from kedro-devrel.

I'm more than happy with that approach, @jmholzer if you recommend it, as it's well thought out and argued. My only concern would be in resourcing the writing of those posts. I suspect the first may be easier since that's Databricks functionality and we can use their resources, but I'll need a steer on who to ask. Something for the CoE?

from kedro-devrel.