Giter Club home page Giter Club logo

formatdataform's Introduction

Dataform formatter

Format .sqlx files in your Dataform project using sqlfluff

Version Linux macOS

To format a file or directory run

formatdataform format <path_to_file_or_directory>

Note

When ran for the first time on a dataform workspace, the formatdataform cli will setup necessary files & default .sqluff config file in .formatdataform directory to support formatting. You can alternatively manually setup the defaults by running formatdataform setup.

You can override the default sqlfluff config file by using the --config flag or -c (shorthand) as follows

formatdataform -c <path_to_sqlfluff_config_file> format <path_to_file_or_directory>

Or alternatively, you can directly edit the sqlfluff config file generated in .formatdataform/.sqlfluff

Important

Ensure that the config block has the last block which handles the regex for [sqlfluff:templater:placeholder] to handle the parsing of ${ref("TABLE_NAME")} blocks in .sqlx files

Installation

  1. Prerequisite: Install sqlfluff
pip install sqlfluff
  1. Install the Latest release of formatdataform binary
curl -sSfL https://raw.githubusercontent.com/ashish10alex/formatdataform/main/install_latest.sh | bash

OR

go install github.com/ashish10alex/formatdataform@latest

This installs the binary formatdataform to $GOBIN, which defaults to $GOPATH/bin

OR

Manually clone the repository and build the cli and add the cli to your system path

git clone https://github.com/ashish10alex/formatdataform.git
go build -o formatdataform
mv formatdataform /usr/local/bin/formatdataform

To run tests

  1. Install gotestsum for prettier test visualization
  2. Run gotestsum --format testname

Known issues

  1. sqlfluff expect the table name to have backticks i.e `gcp_project_id.dataset.table` instead of gcp_project_id.dataset.table
  2. Does not format sql in pre_operations block. To handle this we would need to identify if there is a javascript block inside the pre_operations block and then get the query out of that block and format it
  3. Does not format config block

formatdataform's People

Contributors

ashish10alex avatar

Watchers

Lucian avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.