data-tools / big-data-types Goto Github PK

For each type, there are two type classes, both of them have the same method name, one without arguments, the other expects an instance of A.

For example, BigQuery has bigQueryFields and bigQueryFields(value: A), both return a BigQuery type from any other type. It makes sense as the usage of them is in the format of SqlTypeToBigQuery[Something].bigQueryFields or SqlInstanceToBigQuery[SomeType].bigQueryFields(instanceOfSomeType)

But there are also extension methods that allow to extract a BigQuery Schema directly from an instance, like myInstance.bigQueryFields and it's quite confusing. A better name would be myInstance.asBigQuery or in the case of Spark, myInstance.asSpark

This would cause a major version change as it will be a breaking change.

As example code:
BigQuery code12

We should change the same for Spark and ensure that all tests are running well.

Similar to Spark and BigQuery, a CassandraTypeConversion should allow the transformation from any Cassandra CreateTable to other types.

There is a guide on how to add this kind of conversion here:

https://github.com/data-tools/case-class-to-database-types/blob/bb47f0a0f4e419360bb102801770429f683a0a46/src/main/scala/org/datatools/cc2db/types/BigQueryTypes.scala#L34