Giter Club home page Giter Club logo

Comments (7)

kieronellis avatar kieronellis commented on June 12, 2024 1

@yu-iskw could you reopen this issue or should I create a new one

from action-sqlfluff.

Aka-shi avatar Aka-shi commented on June 12, 2024

@yu-iskw

from action-sqlfluff.

yu-iskw avatar yu-iskw commented on June 12, 2024

@Aka-shi Thank you for the feedback. I haven't seen the error. Can you share the dbt model which have the violations so that I reproduce it, if you don't mind?

from action-sqlfluff.

yu-iskw avatar yu-iskw commented on June 12, 2024

I am guessing the issue is caused by any breaking change of github action. I will look into it.

from action-sqlfluff.

yu-iskw avatar yu-iskw commented on June 12, 2024

@Aka-shi After all, I don't understand the root cause.

  1. Can you tell me how to reproduce it in detail? I am not familiar with trio with dbt.
  2. Can you share the logs?

from action-sqlfluff.

yu-iskw avatar yu-iskw commented on June 12, 2024

If you still have the issue, please feel free to re-open.

from action-sqlfluff.

kieronellis avatar kieronellis commented on June 12, 2024

I'm getting the same issue but only with "lint" mode.

test_sqlfluff.sql

select blaa1, blaa2
from blaa

test.yaml

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Decode service account credentials
        env:
          # The base64 encoded credentials.
          GOOGLE_APPLICATION_CREDENTIALS_BASE64: "${{ secrets.GOOGLE_APPLICATION_CREDENTIALS_BASE64 }}"
        run: |
          echo "...Decoding GOOGLE_APPLICATION_CREDENTIALS_BASE64..."
          python decode_base64.py || exit 1
          echo "...and validating json output..."
          python -m json.tool gcloud.json > /dev/null || exit 1

      - uses: yu-iskw/[email protected]
        env:
          DBT_PROFILES_DIR: ${{ github.workspace }}/.github/sqlfluff/.dbt/
        id: lint_sql
        with:
          github_token: ${{ secrets.TOKEN_GITHUB }}
          reporter: github-pr-review
          sqlfluff_version: "1.2.1"
          sqlfluff_command: "lint" # "fix" or "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          fail_on_error: true
          dialect: bigquery
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"
          processes: 10

.sqlfluff

# https://docs.sqlfluff.com/
[sqlfluff]
templater = dbt
dialect = bigquery
# Exclude: 
# L008 Space after comma.
# L029 Keywords should not be used as identifiers.
# L032 Prefer specifying join keys instead of using USING.
# L034 Select wildcards then simple targets before calculations and aggregates.
# L048 Quoted literals should be surrounded by a single whitespace.
# L054 Inconsistent column references in GROUP BY/ORDER BY clauses.
exclude_rules = L008, L029, L032, L034, L048, L054
# Number of passes to run before admitting defeat, default = 10
# runaway_limit = 100

[sqlfluff:rules:L003]
# Indentation not consistent with previous lines.
tab_space_size = 2

[sqlfluff:rules:L010]
# Keywords
capitalisation_policy = lower

[sqlfluff:rules:L014]
# Unquoted identifiers
extended_capitalisation_policy = lower

[sqlfluff:rules:L016]
max_line_length = 120
ignore_comment_clauses = True

[sqlfluff:rules:L019]
comma_style = leading

[sqlfluff:rules:L030]
# Function names
extended_capitalisation_policy = lower

[sqlfluff:rules:L040]
# Null & Boolean Literals
capitalisation_policy = lower

[sqlfluff:rules:L063]
# Data Types
extended_capitalisation_policy = lower

[sqlfluff:rules:L064]
# Consistent usage of preferred quotes for quoted literals
preferred_quoted_literal_style = single_quotes

output of local run sqlfluff, version 1.2.1

sqlfluff lint models/test_sqlfluff.sql
=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
== [models/test_sqlfluff.sql] FAIL                                                 
L:   1 | P:   1 | L036 | Select targets should be on a new line unless there is
                       | only one select target.
L:   2 | P:  10 | L009 | Files must end with a single trailing newline.
All Finished 📜 🎉!

output of github action in "lint" mode

Running sqlfluff 🐶 ...
  14:15:28  Partial parse save file not found. Starting full parse.
  [{"filepath": "models/test_sqlfluff.sql", "violations": [{"line_no": 1, "line_pos": 1, "code": "L036", "description": "Select targets should be on a new line unless there is only one select target."}, {"line_no": 2, "line_pos": 10, "code": "L009", "description": "Files must end with a single trailing newline."}]}]
  parse error: Expected string key before ':' at line 1, column 3
 Running reviewdog 🐶 ...
  parse error: Expected string key before ':' at line 1, column 3
  reviewdog: parse error: failed to unmarshal rdjson (DiagnosticResult): proto: syntax error (line 1:1): unexpected token 

Everything the same, but changing the test.yaml to "fix" mode, everything works as expected

name: sqlfluff with reviewdog
on:
  pull_request:
jobs:
  test-check:
    name: runner / sqlfluff (github-check)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Decode service account credentials
        env:
          # The base64 encoded credentials.
          GOOGLE_APPLICATION_CREDENTIALS_BASE64: "${{ secrets.GOOGLE_APPLICATION_CREDENTIALS_BASE64 }}"
        run: |
          echo "...Decoding GOOGLE_APPLICATION_CREDENTIALS_BASE64..."
          python decode_base64.py || exit 1
          echo "...and validating json output..."
          python -m json.tool gcloud.json > /dev/null || exit 1

      - uses: yu-iskw/[email protected]
        env:
          DBT_PROFILES_DIR: ${{ github.workspace }}/.github/sqlfluff/.dbt/
        id: lint_sql
        with:
          github_token: ${{ secrets.TOKEN_GITHUB }}
          reporter: github-pr-review
          sqlfluff_version: "1.2.1"
          sqlfluff_command: "fix" # "fix" or "lint"
          config: "${{ github.workspace }}/.sqlfluff"
          paths: '${{ github.workspace }}/models'
          fail_on_error: true
          dialect: bigquery
          extra_requirements_txt: "${{ github.workspace }}/requirements.txt"
          processes: 10

Github action output in "fix" mode

 Running sqlfluff 🐶 ...
  ==== finding fixable violations ====
  === [dbt templater] Sorting Nodes...
  14:20:24  Partial parse save file not found. Starting full parse.
  === [dbt templater] Compiling dbt project...
  === [dbt templater] Project Compiled.
  WARNING    One fix for L003 not applied, it would re-cause the same error. 
  == [models/test_sqlfluff.sql] FAIL
  L:   1 | P:   1 | L036 | Select targets should be on a new line unless there is
                         | only one select target.
  L:   2 | P:  10 | L009 | Files must end with a single trailing newline.
  ==== fixing violations ====
  2 fixable linting violations found
  FORCE MODE: Attempting fixes...
  Persisting Changes...
  == [models/test_sqlfluff.sql] PASS
  Done. Please check your files to confirm.
 Running reviewdog 🐶 ...
  diff --git a/models/test_sqlfluff.sql b/models/test_sqlfluff.sql
  index f6625431..fffca06a 100644
  --- a/models/test_sqlfluff.sql
  +++ b/models/test_sqlfluff.sql
  @@ -1,2 +1,4 @@
  -select blaa1, blaa2
  -from blaa
  \ No newline at end of file
  +select
  +  blaa1
  +, blaa2
  +from blaa
  Saved working directory and index state WIP on (no branch): 0907bae1 Merge 989b0596cddaa8f8b40d45ec4c8a4e3dc7b37552 into e6ef8ca8816c60a38cd686ed9cf8497348f73111
  models/test_sqlfluff.sql:1:-select blaa1, blaa2
  models/test_sqlfluff.sql:2:-from blaa
  models/test_sqlfluff.sql:1:+select
  models/test_sqlfluff.sql:2:+  blaa1
  models/test_sqlfluff.sql:3:+, blaa2
  models/test_sqlfluff.sql:4:+from blaa
  reviewdog: input data has violations
  Dropped refs/stash@{0} (b01474aac5a1f5dcffae1ba610c3254d1bb7b1db)

image

from action-sqlfluff.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.