Check existing relation type when renaming to __dbt_backup about dbt-spark HOT 4 CLOSED

@jtcohen6 this is good to know! We dynamically check the relation type in the spark__drop_relation implementation -- sounds like we should do something similar in the rename too

from dbt-spark.

@drewbanin Got it, makes sense.

A related thought, though possibly specific to Databricks:

For managed tables, renaming a table moves the table location; for unmanaged (external) tables, renaming a table does not move the table location.

Renaming a table can be a very slow/expensive operation, on par with building the intermediate table.

Edit: Realizing this is a much longer conversation

from dbt-spark.

@jtcohen6 I also penned a response before also realizing that there's a lot to say here.........

Lmk the next time you're free for an espresso :)

from dbt-spark.

There was a short-term resolution for this in #17: do not alter + rename tables because it is an expensive process.

There will be a fix for the larger issue—some macros need to know current relation types, and we must adjudicate which—in #50: get all existing relation types at the start of the run via smarter metadata query.

I'm going to close this issue given both developments.

from dbt-spark.