brayanjuls / diane Goto Github PK

We should add unit tests for all the possible use cases of the function createOrReplaceHiveView.

To keep it simple we could use MIT license.

Implement a gitlab ci workflow that executes the unit test on each PR.

Should include instructions on how to get the project running locally and the different ways to contribute.

Follow this step in order to publish this library:

• Create a sonatype account.
• Create a Jira ticket in sonatype to claim your groupId for this project.
  • Ideally the groupId should be com.brayanjules
• Use the plugin sbt-pgp to sign your artifact before publication.
• Use the plugin sbt-sonatype to publish to maven central.
• Use the pugin sbt-release to automate the release process.

Use the recommended scalafmt configuration.

Many Spark users are registering tables in a Hive metastore without any idea of what the Hive metastore is, its basic functionality, and best practices for leveraging the Hive metastore.

We should brainstorm how to provide end user interfaces that demystify the Hive metastore for end users. What tables are registered, what are the paths, which tables are external/managed, etc.

Guidance on how to convert from external => managed and from not registered => external are examples of issues that baffle Spark users, myself included.

The purpose of this issue is to brainstorm the types of interfaces we can provide end users.

Issue migrated from MrPowers/jodie#16

It'd be useful to have a Hive command that provides a more elegant way to show all the Hive tables / associated metadata than what's outlined in this answer.

Perhaps HiveHelpers.allTables() that returns a list of HiveTable objects. The HiveTable object can contain the table name, HiveTableType, etc. I'm just making up abstractions that could be cool. Feel free to implement this however it's best abstracted.

This issue was migrated from MrPowers/jodie#20

I just added this section to the jodie README: https://github.com/MrPowers/jodie#accessing-the-library

I think this is a good way to present the section. It shows how to access the lib in SBT projects. It also provides the JAR file links in case someone wants to download the JARs and for some custom workflow.