yu-iskw / action-sqlfluff Goto Github PK

View Code? Open in Web Editor NEW

64.0 6.0 24.0 440 KB

Run sqlfluff with reviewdog to check or format styles

Home Page: https://github.com/marketplace/actions/run-sqlfluff-with-reviewdog

License: MIT License

Shell 79.55% jq 7.11% Dockerfile 9.55% Makefile 3.79%

sql dbt ci reviewdog sqlfluff github-actions lint linter

action-sqlfluff's People

Contributors

Stargazers

Watchers

action-sqlfluff's Issues

Alternate working directory configuration not respected (dbt_project.yml)

Is it possible to use this action template on dbt when the dbt_project.yml file lives somewhere other than the root of the repository? If so, how to configure it? The below describes what I have attempted.

In a git action template, I am setting the working-directory for yu-iskw/action-sqlfluff@v3 as follows:

working-directory: ${{ github.workspace }}/dbt_transform_code

In the SQL Fluff configuration file, I've also experimented with setting the project_dir to '/home/runner/work/<repo-name>/<repo-name>/dbt_transform_code'

In either case, I see the following error from the git action run:

    File "/usr/local/bin/sqlfluff", line 8, in <module>
      sys.exit(cli())
    File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
      rv = self.invoke(ctx)
    File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
      return __callback(*args, **kwargs)
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/cli/commands.py", line 830, in fix
      result = lnt.lint_paths(
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 1164, in lint_paths
      for i, linted_file in enumerate(runner.run(expanded_paths, fix), start=1):
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 142, in run
      for lint_result in self._map(
    File "/usr/local/lib/python3.9/multiprocessing/pool.py", line 870, in next
      raise value
  dbt.exceptions.DbtProjectError: Runtime Error
    no dbt_project.yml found at expected path /github/workspace/dbt_project.yml

Which appears to be telling me the process is looking for the dbt_project.yml file to exist at the root of the repository.

Parse error when running on latest version

Don't understand the error I am facing

Action.yml

name: Lint Check

on:
  pull_request:
    branches:
      - master
env:
  DBT_PROFILES_DIR: ${{ github.workspace }}/.dbt/
  TRINO_DEV_URL: trino-dev-coordinator-int.de.razorpay.com
  DBT_USER: github-actions-dbt
jobs:
  Lint_Check:
    name: Lint_Check
    runs-on: [self-hosted]

    steps:
      - name: Check out
        uses: actions/checkout@v2
      - uses: yu-iskw/action-sqlfluff@v3
        name: sql-lint-check
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          config: "${{ github.workspace }}/.sqlfluff"
          paths: "${{ github.workspace }}/models"
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"

sqlfluff config

[sqlfluff]
templater = dbt
dialect = ansi
sql_file_exts = .sql
exclude_rules = L027

[sqlfluff:rules]
max_line_length = 150

[tool.sqlfluff.templater.jinja]
apply_dbt_builtins = True

[sqlfluff:templater:dbt]
project_dir = .
profiles_dir = ./.dbt
profile = trino
target = trino-dev

ERROR screenshots

Can't get changed files.

My repository is like this.

.
├── README.md
├── other_directory
│   └── other_files
├── dbt
│   ├──.sqlfluff
│   ├──packages.yml
|   └── models

And I want to lint to dbt/models/. So my yaml for workflow write like this.

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: auth
        uses: google-github-actions/setup-gcloud@v0
        with:
          project_id: unipos-dev-diu
          service_account_key: ${{ secrets.gcp_key }}
          export_default_credentials: true
      - uses: yu-iskw/[email protected]
        id: lint-sql
        env:
          GCLOUD_PROJECT: my-project
        with:
          working-directory: ./dbt
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_version: "0.13.0"
          sqlfluff_command: "lint"
          config: "./.sqlfluff"
          paths: './models'
          extra_requirements_txt: './extra_requirements.txt'
      - name: 'Show outputs (Optional)'
        shell: bash
        run: |
          echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
          echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'

I think setting woriking-directory is required, because of running dbt deps. But, then here is not working well...
I expect models/marts/hoge.sql, but git diff --name-only --no-color sqlfluff_bot HEAD returns dbt/models/marts/hoge.sql.
There is any solutions ?

SQLFluff failing due to ModuleNotFoundError: No module named 'pytz'

Hi all,

sqlfluff has been failing for us since this morning with the following error:

ModuleNotFoundError: No module named 'pytz'

Rest of the error is below:

Traceback (most recent call last):
    File "/usr/local/bin/sqlfluff", line 5, in <module>
      from sqlfluff.cli.commands import cli
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/cli/commands.py", line 518, in <module>
      def lint(
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/cli/commands.py", line 218, in core_options
      get_plugin_manager().hook.get_templaters()
    File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/plugin/host.py", line 12, in get_plugin_manager
      pm.load_setuptools_entrypoints(project_name)
    File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line [287](https://github.com/motorway/motorway-dbt/actions/runs/4292196911/jobs/7478336098#step:4:289), in load_setuptools_entrypoints
      plugin = ep.load()
    File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
      module = import_module(match.group('module'))
    File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "/usr/local/lib/python3.9/site-packages/sqlfluff_templater_dbt/__init__.py", line 3, in <module>
      from sqlfluff_templater_dbt.templater import DbtTemplater
    File "/usr/local/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 13, in <module>
      from dbt.config import read_user_config
    File "/usr/local/lib/python3.9/site-packages/dbt/config/__init__.py", line 2, in <module>
      from .profile import Profile, read_user_config  # noqa
    File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 10, in <module>
      from dbt.contracts.connection import Credentials, HasCredentials
    File "/usr/local/lib/python3.9/site-packages/dbt/contracts/connection.py", line 28, in <module>
      from dbt.contracts.util import Replaceable
    File "/usr/local/lib/python3.9/site-packages/dbt/contracts/util.py", line 7, in <module>
      from dbt import deprecations
    File "/usr/local/lib/python3.9/site-packages/dbt/deprecations.py", line 6, in <module>
      import dbt.tracking
    File "/usr/local/lib/python3.9/site-packages/dbt/tracking.py", line 25, in <module>
      import pytz
  ModuleNotFoundError: No module named 'pytz'

Screenshot below:

SQLFluff failing due to ImportError

sqlfluff has been failing for us since yesterday around 3pm GMT, with the following error:

ImportError: cannot import name 'CompilationException' from 'dbt.exceptions' (/usr/local/lib/python3.9/site-packages/dbt/exceptions.py).

We haven't changed any configs or anything - think the issue started around the same time that dbt went down too.

Rest of the error is below:

Traceback (most recent call last): File "/usr/local/bin/sqlfluff", line 5, in <module> from sqlfluff.cli.commands import cli File "/usr/local/lib/python3.9/site-packages/sqlfluff/__init__.py", line 6, in <module> from sqlfluff.api import lint, fix, parse, list_rules, list_dialects File "/usr/local/lib/python3.9/site-packages/sqlfluff/api/__init__.py", line 4, in <module> from sqlfluff.api.simple import lint, fix, parse, APIParsingError File "/usr/local/lib/python3.9/site-packages/sqlfluff/api/simple.py", line 4, in <module> from sqlfluff.core import ( File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/__init__.py", line 9, in <module> from sqlfluff.core.linter import Linter File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/linter/__init__.py", line 6, in <module> from sqlfluff.core.linter.linter import Linter File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 23, in <module> from sqlfluff.core.rules import get_ruleset File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/rules/__init__.py", line 12, in <module> for plugin_rules in get_plugin_manager().hook.get_rules(): File "/usr/local/lib/python3.9/site-packages/sqlfluff/core/plugin/host.py", line 12, in get_plugin_manager pm.load_setuptools_entrypoints(project_name) File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints plugin = ep.load() File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load module = import_module(match.group('module')) File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/usr/local/lib/python3.9/site-packages/sqlfluff_templater_dbt/__init__.py", line 3, in <module> from sqlfluff_templater_dbt.templater import DbtTemplater File "/usr/local/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 17, in <module> from dbt.exceptions import ( ImportError: cannot import name 'CompilationException' from 'dbt.exceptions' (/usr/local/lib/python3.9/site-packages/dbt/exceptions.py)

Screenshot below:

sqlfluff v2.0

maybe this action is not compatible sqlfluff v2.0 (I test with 2.0.3)
is there any update plan?

fatal: --unshallow on a complete repository does not make sense

When using this GitHub Action, I encounter the following error message:

🐶 Get changed files
  fatal: --unshallow on a complete repository does not make sense

I suspect the error here, but do not understand why this occurs.

Maybe this condition could help solving this:

if $(git rev-parse --is-shallow-repository); then
    ... repository is shallow ...
fi

Source: StackOverflow

My workflow file:

    steps:
      - name: Checkout repo
        uses: actions/checkout@v3

      - name: Lint added and modified .sql files
        uses: yu-iskw/action-sqlfluff@v3
        id: lint-sql
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_version: "1.4.5"
          sqlfluff_command: "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: "${{ github.workspace }}/models"
          extra_requirements_txt: "${{ github.workspace }}/.github/config/requirements.txt"

extra_requirements_txt configuration fails on dbt-spark[PyHive] adapter

Since dbt-spark[PyHive] has binary dependency of libsasl2 . I work-around this problem by changing the Dockerfile for now.

SQLFluff parse error in lint mode

@yu-iskw See #31

Current version of sqlfluff is 1.4.5

I note the current release version of sqlfluff is 1.4.5.

I looked at what appeared to be the last release bump in this repository #42 but it doesn't actually seem to have updated this sqlfluff version so I am not a little confused by what the supported version means in terms of this particular action?

i.e. What is actually the version of sqlfluff installed within this action?

"Show output - optional" step shows empty results

code I'm using:

name: sqlfluff with reviewdog
on:
pull_request:
jobs:
test-check:
name: Lint-and-Parse-SQL
runs-on: ubuntu-latest

steps:
  - uses: actions/checkout@v2
  - uses: yu-iskw/[email protected]
    id: lint-sql
    #id: sqlfluff-with-reviewdog-in-composite
    with:
      github_token: ${{ secrets.github_token }}
      reporter: github-pr-review
      sqlfluff_version: "2.0.7"
      sqlfluff_command: "lint" # Or "lint"
      config: "${{ github.workspace }}/.sqlfluff"
      paths: '${{ github.workspace }}/sql'
      fail_on_error: true
  - name: 'Show outputs (Optional)'
    shell: bash
    run: |
      echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
      echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'
      echo '${{ steps.lint-sql.outputs.sqlfluff-exit-code }}' | jq -r '.'
      echo '${{ steps.lint-sql.outputs.reviewdog-return-code }}' | jq -r '.'

The check results as a success even when violations are found. I believe, Brewdog is supposed to fail the check with fail_on_error: true parameter set but I think there is a brewdog bug waiting for a fix to address this issue.

So for now, I'm trying to obtain the output value, specifically for sqlfluff-exit-code and reviewdog-return-code to allow to use these to dictate whether the check should be a success or a fail but I can't seem to get the output values to show!

Any help would be appreciated, thanks!

Not working with dbt Snowflake adapter

When I run this action with the following setup:

 name: Lint files
        id: lint-files
        uses: yu-iskw/action-sqlfluff@v3
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_command: "lint" # Or "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          templater: dbt

I get the following error:

 Running sqlfluff 🐶 ...
  19:42:09  Error importing adapter: No module named 'dbt.adapters.snowflake'
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 183, in _credentials_from_profile
      cls = load_plugin(typename)
    File "/usr/local/lib/python3.9/site-packages/dbt/adapters/factory.py", line 215, in load_plugin
      return FACTORY.load_plugin(name)
    File "/usr/local/lib/python3.9/site-packages/dbt/adapters/factory.py", line 70, in load_plugin
      raise RuntimeException(f'Could not find adapter type {name}!')
  dbt.exceptions.RuntimeException: Runtime Error
    Could not find adapter type snowflake!

I suspect this is because this action seems to install dbt-core and not the adapter specific dbt.

Action doesn't pick up space in paths

Hi,

There is an issue with including spacing in the 'paths' input for the action. I have tried several methods (ubuntu way of escaping space, env variables, repo variables) and nothing has worked out.

Thanks!

Wrong JSON output with tee and dbt templater

When running with the dbt templater the process might add more output lines that aren't actual JSON format and then it will fail trying to run jq here

e.g:

�[0m14:02:51  Unable to do partial parsing because config vars, config profile, or config target have changed
�[0m14:02:52  Unable to do partial parsing because config vars, config profile, or config target have changed
[{"filepath":  ...

I think one possible solution is to replace tee with --write-output from sqlfluff itself that was added since version 0.10.1, this way the resulting file only contains the final report.

I could try to make a PR to fix this, what do you think?

Please add build essentials (gcc) to Dockerfile

When the action attempts to install libsasl2, it failed with missing gcc error.

Fail action when there are violations

The action always shows a green check-mark even if there are violations after linting

Is there any of the inputs that need to be changed to make it show a red x when there are any violations after linting?

my yml file has this

name: Code QA
on:
  pull_request:
jobs:
  test-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: yu-iskw/action-sqlfluff@v3
        id: lint-sql
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_version: "0.13.0"
          sqlfluff_command: "lint" # Or "fix"
          fail_on_error: 'true'
          config: "${{ github.workspace }}/.github/linters/.sqlfluff"
          paths: '${{ github.workspace }}/dbt/models'
      - name: 'Show outputs (Optional)'
        shell: bash
        run: |
          echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
          echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'

I was hoping fail_on_error would do the trick but it didn't

SQLFluff not leaving comments on GitHub Pull Requests on lint mode

Wasn't sure if this was an issue or the way I've configured SQLFluff, but it looks like SQLFluff isn't leaving comments on Github for the lint mode. This has been an issue for more than a month, not sure exactly when it happened. It used to run properly beforehand and I haven't changed any of the settings since last time.

Below is a snippet from the SQLFluff run on Github actions [Run yu-iskw/[email protected]] - it looks like it's correctly running and finding formatting issues:

EDIT: Ignore the below comment, was misremembering.
However, the show outputs section looks empty, whereas in the past it had outputs:

Here's my workflow file:

Please let me know if there's any more details you need or if this isn't an issue, happy to move/delete.

reviewdog outputs raw format instead of code suggestion

When running sqlfluff fix on changes that end up being more than a few lines of sql, instead of a code suggestion on pr review it gets outputted in raw format. Wondering why that's the case or if there is something with my workflow implementation?

small change:

change that is over 20 lines long:

.sqlfluff config:

dialect = bigquery
templater = dbt
rules = core
exclude_rules = L008, L032, L031, L040

[sqlfluff:templater:dbt]
project_dir = .

[sqlfluff:templater:jinja]
apply_dbt_builtins = True

[sqlfluff:rules]
capitalisation_policy = consistent
extended_capitalisation_policy = consistent

[sqlfluff:rules:L016]
max_line_length = 200

workflow:

steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 1

      - uses: yu-iskw/[email protected]
        id: fix-sql
        env: 
          DBT_PROFILES_DIR: '${{ github.workspace }}/dbt'
        with: 
          github_token: ${{ secrets.GITHUB_TOKEN }}
          working-directory: '${{ github.workspace }}/dbt'
          level: 'warning'
          reporter: github-pr-review
          filter_mode: 'added'
          reviewdog_version: '0.14.1'
          sqlfluff_version: '1.4.5'
          sqlfluff_command: 'fix'
          paths:  "${{ github.workspace }}/dbt/models"
          config: "${{ github.workspace }}/dbt/.sqlfluff"
          extra_requirements_txt: "${{ github.workspace }}/dbt/sqlfluff_requirements.txt"
      
      - name: 'Show outputs (Optional)'
        shell: bash
        run: |
          echo '${{ steps.fix-sql.outputs.sqlfluff-results }}' | jq -r '.'
          echo '${{ steps.fix-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'

working-directory input not working

When I try to run the action, it is always running in /github/home and this is inaccessible.

I have attempted to move my profiles.yml file to /github/home/.dbt but I do not have permissions to create that directory and it does not exist.

I have attempted to change the working directory as an input but each time it runs the action runs in the /github/home directory looking for .dbt/ which again does not exist and cannot be created due to permissions.

I also checked the ${{ github.workspace }} which is supposed to be the default, but for my step setup dbt this is different to where the action is running. For my custom step it is /home/runner/work/dbt-bq-elt/dbt-bq-elt and where the action tries to run is /github/home.

Here is my configuration for the action. Note I am attempting to create a profiles.yml file in the setup dbt section in order for SQLFluff to correctly connect to our BigQuery warehouse:

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: 'setup dbt'
        shell: bash
        run: |
          echo "${{ secrets.gcloud_service_key }}" > "${{ github.workspace }}/dbt-service-account.json"
          sed s/template-project-id/${{ secrets.GOOGLE_PROJECT_ID }}/g profiles.github.yaml > profiles.github1.yaml
          sed -i s,template-json-directory,${{ github.workspace }},g profiles.github1.yaml > profiles.github2.yaml
          mkdir -p "~/.dbt"
          mkdir "${{ github.workspace }}/.dbt"
          mv profiles.github2.yaml "~/.dbt/profiles.yml"
      - uses: yu-iskw/action-sqlfluff@v2
        id: lint-sql
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_version: "0.9.1"
          working-directory: /home/runner
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          fail_on_error: true
      - name: 'Show outputs (Optional)'
        shell: bash
        run: |
          echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
          echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'

action-sqlfluff GH action outputs raw format instead of inline code suggestion

@yu-iskw The issue reported in #56 is still present when the diff produced by sqlfluff fix is bigger than a few lines (including a screenshot below). Do we have a fix/workaround for it?

The relevant config from the workflow yaml file

    steps:
      - uses: actions/checkout@v2
      - uses: yu-iskw/action-sqlfluff@v3
        id: lint-sql
        with:
          github_token: ${{ secrets.github_token }}
          reporter: github-pr-review
          sqlfluff_version: "1.4.5"
          sqlfluff_command: "fix"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: "${{ github.workspace }}/models"
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"
      - name: "Show outputs (Optional)"
        shell: bash
        run: |
          echo '${{ steps.lint-sql.outputs.sqlfluff-results }}' | jq -r '.'
          echo '${{ steps.lint-sql.outputs.sqlfluff-results-rdjson }}' | jq -r '.'

Automatically suggest to fix violations rather than just leaving comments

Motivation

GitHub now supports the "suggestion" feature on pull requests. The action only leaves comments about violations at the moment. We have to fix the violations manually. But, what if the action can leave suggestions?

What is the best way to use inside docker-compose project?

I have dbt inside docker compose with all dependencies.

Currently, I'm running check that compilation successful in PR's like this

name: dbt
on: pull_request
jobs:
  compile:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Start docker compose
        run: docker compose up -d
      - name: Compile sources
        run: docker compose exec service dbt compile
      - name: Stop compose
        if: always()
        run: |
          docker-compose down
          docker network prune -f

I'm trying to add sqlfluff linter, and it requires all database dependencies to be set manually.
However I can run command inside compose

docker compose exec service sqlfluff lint --format json --dialect postgres --templater dbt ...

Is it possible to utilize this? Or should I build workflow from scratch?

(conclusion=failure)

I'm trying to get this working and I think i'm close.
For some reason at the moment "Running reviewdog 🐶 ..." process breaks with an error:
2024/03/25 23:30:19 [sqlfluff-fix] reported: https://github.com/<my_repo>/runs/23080766658 (conclusion=failure)
Not sure what that means or where to begin looking for a fix. Any suggestions appreciated.

SQLfluff lint fails, but workflow passed

The exit code is wrong for running SQLFluff lint: it return 0 even if the SQLFluff lint raises an exception.

It's because the sqlfluff_exit_code get the wrong code from "tee" command.
https://github.com/yu-iskw/action-sqlfluff/blob/main/entrypoint.sh#:~:text=%24changed_files%20%7C-,tee%20%22%24lint_results%22,-sqlfluff_exit_code%3D%24%3F
I can give a PR for fixing this.

I get an error for my github action even after I fix all the linting issues

I commited my code, got comments on my PR to fix some linting violations. After fixing all those violations and commiting the code again, i still get an error. The image above shows the details on the github action fail.

Is there any hooks/options for running `dbt deps` before linting SQL?

I am using this action on a dbt project which uses other packages.

Check only added and changed files

Motivation

It takes a long time to check many SQL files with sqlfluff. So, the action should check only added and changed SQL files to shorten the elapsed time.

Enhancement: enable --show-lint-violations for sqlfluff fix

Currently: unfixable violations when running sqlfluff fix causes failed check with the error [n unfixable linting violations found]

Enhancement: be able to see what rules failed in the output with the sqlfluff flag --show-lint-violations so that the dev can go into the code and manually fix. In general have an option to pass additional options/flags to the action when running fix or lint

here is sqlfluff feature:
sqlfluff/sqlfluff#3788

Not planned

Security Policy violation Branch Protection

This issue was automatically created by Allstar.

Security Policy Violation
Dismiss stale reviews not configured for branch main

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

SqlFluff action causes Deprecation Warning on GitHub

This loc: https://github.com/yu-iskw/action-sqlfluff/blob/main/entrypoint.sh#L24

Throws:

Running sqlfluff 🐶 ...
  ==== finding fixable violations ====
    [1 templating/parsing errors found]
  ==== no fixable linting violations found ====
  All Finished!
  Warning: The `set-output` command is deprecated and will be disabled soon.
  Please upgrade to using Environment Files. For more information see:
  https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Thank you.

SQLFluff linting job failing due to missing CompilationException from dbt.exceptions

The SQLFluff linting was working as expected with the following versions: SQLFluff version 1.4.5 with DBT Version 1.2 in DBT Cloud. But recently it started failing with the following error:

Traceback (most recent call last):
    File "/usr/local/bin/sqlfluff", line 5, in <module>
      from sqlfluff.cli.commands import cli
    File "/usr/local/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 518, in <module>
      def lint(
    File "/usr/local/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 218, in core_options
      get_plugin_manager().hook.get_templaters()
    File "/usr/local/lib/python3.10/site-packages/sqlfluff/core/plugin/host.py", line 12, in get_plugin_manager
      pm.load_setuptools_entrypoints(project_name)
    File "/usr/local/lib/python3.10/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
      plugin = ep.load()
    File "/usr/local/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
      module = import_module(match.group('module'))
    File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "/usr/local/lib/python3.10/site-packages/sqlfluff_templater_dbt/__init__.py", line 3, in <module>
      from sqlfluff_templater_dbt.templater import DbtTemplater
    File "/usr/local/lib/python3.10/site-packages/sqlfluff_templater_dbt/templater.py", line 17, in <module>
      from dbt.exceptions import (
  ImportError: cannot import name 'CompilationException' from 'dbt.exceptions' (/usr/local/lib/python3.10/site-packages/dbt/exceptions.py)

Security Policy violation SECURITY.md

This issue was automatically created by Allstar.

Security Policy Violation
Security policy not enabled.
A SECURITY.md file can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. Examples of secure reporting methods include using an issue tracker with private issue support, or encrypted email with a published key.

To fix this, add a SECURITY.md file that explains how to handle vulnerabilities found in your repository. Go to https://github.com/yu-iskw/action-sqlfluff/security/policy to enable.

For more information, see https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

yu-iskw / action-sqlfluff Goto Github PK

action-sqlfluff's People

Contributors

Stargazers

Watchers

Forkers

action-sqlfluff's Issues

Motivation

Motivation

Recommend Projects

Recommend Topics

Recommend Org