Giter Club home page Giter Club logo

steampipe-mod-github-sherlock's Introduction

GitHub Sherlock Mod for Powerpipe

Important

Powerpipe is now the preferred way to run this mod! Migrating from Steampipe →

All v0.x versions of this mod will work in both Steampipe and Powerpipe, but v1.0.0 onwards will be in Powerpipe format only.

Interrogate your GitHub resources with the help of the World's greatest detectives: Powerpipe + Sherlock. GitHub Sherlock allows you to perform deep analysis of your GitHub organization and repo configuration and test them against operations & security best practices.

Run checks in a dashboard: image

Or in a terminal: image

Documentation

Getting Started

Installation

Install Powerpipe (https://powerpipe.io/downloads), or use Brew:

brew install turbot/tap/powerpipe

This mod also requires Steampipe with the Github plugin as the data source. Install Steampipe (https://steampipe.io/downloads), or use Brew:

brew install turbot/tap/steampipe
steampipe plugin install github

Finally, install the mod:

mkdir dashboards
cd dashboards
powerpipe mod init
powerpipe mod install github.com/turbot/steampipe-mod-github-sherlock

Browsing Dashboards

Start Steampipe as the data source:

steampipe service start

Start the dashboard server:

powerpipe server

Browse and view your dashboards at http://localhost:9033.

Running Checks in Your Terminal

Instead of running benchmarks in a dashboard, you can also run them within your terminal with the powerpipe benchmark command:

List available benchmarks:

powerpipe benchmark list

Run a benchmark:

powerpipe benchmark run github_sherlock.benchmark.org_best_practices

Different output formats are also available, for more information please see Output Formats.

Open Source & Contributing

This repository is published under the Apache 2.0 license. Please see our code of conduct. We look forward to collaborating with you!

Steampipe and Powerpipe are products produced from this open source software, exclusively by Turbot HQ, Inc. They are distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #powerpipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

steampipe-mod-github-sherlock's People

Contributors

bigdatasourav avatar cbruno10 avatar dboeke avatar e-gineer avatar francois2metz avatar graza-io avatar judell avatar khushboo9024 avatar madhushreeray30 avatar misraved avatar parthai avatar rajlearner17 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

steampipe-mod-github-sherlock's Issues

Deprecate mod and move applicable checks into GitHub Insights mod

This mod is older and we don't really support Sherlock type mods anymore (this is the only one actually). Non-compliance checks we add into Insights mods in general, so we should look to consolidate GitHub mods and make GitHub Insights mod more complete.

After GitHub Insights supports all checks from this mod, we should:

Update to take into account column changes in plugin

Is your feature request related to a problem? Please describe.
PR #292 on the plugin removes a column (which causes permission issues for users preventing being able to obtain repos), so we need to update the control.private_repo_no_outside_collaborators to take this into consideration.

Describe the solution you'd like
Update query.

Describe alternatives you've considered
n/a

Additional context
Plugin Issue - Plugin PR

ERROR: context deadline exceeded

Describe the bug

If you have way too many repos, eventually something will time out
with "ERROR: context deadline exceeded" after about 12 minutes.

Steampipe version (steampipe -v)

v0.8.4

Plugin version (steampipe plugin list)

turbot/github@latest: v0.6.1

To reproduce

Get lots of repos that have lots of issues in them.

Run steampipe check all from this repo.

Expected behavior

All checks complete.

Additional context

Command took 12 minutes 36 seconds to complete.

Optimize ad-hoc queries to use 'in' operator

Is your feature request related to a problem? Please describe.
The ad-hoc fix can be reverted back to use in operator

from
(b.name = 'main' or b.name = 'master') to in ('main', 'master')

Depends on SDK Update

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Error in control "No outside collaborators should have access in each private repository": pq: missing FROM-clause entry for table "r"

Describe the bug
Error when running this check: No outside collaborators should have access in each private repository

| + No outside collaborators should have access in each private repository ......................   1 /   1 [=         ]
| | |
| | ERROR: pq: missing FROM-clause entry for table "r"

Steampipe version (steampipe -v)
Example: v0.5.1

Plugin version (steampipe plugin list)
Github plugin:
hub.steampipe.io/plugins/turbot/github@latest | 0.5.0 | github
I'm at commit 911402c of this particular repo

To reproduce
I just ran all checks:
steampipe check all

Expected behavior
No error message.

Additional context
Here is the code of the control that's failing:

control "private_repo_no_outside_collaborators" {
title = "No outside collaborators should have access in each private repository"
description = "Outside collaborators should not have access to private repository content."
sql = <<-EOT
select
html_url as resource,
case
when outside_collaborator_logins = '[]' then 'ok'
else 'alarm'
end as status,
full_name || ' has ' || jsonb_array_length(outside_collaborator_logins) || ' outside collaborator(s).' as reason,
full_name
from
github_my_repository
where
visibility = 'private' and r.fork = ${local.include_forks}
EOT
}

Duplicate local variable error

Describe the bug
On running steampipe query command we get the following error -

Error: failed to load workspace: Failed to decode all mod hcl files
mod defines more that one resource named local.include_forks

Steampipe version (steampipe -v)
Example: v0.8.0-rc.1

Mod version (steampipe plugin list)
Example: v0.3

To reproduce
Steps to reproduce the behaviour (please include relevant code and/or commands).

Expected behaviour
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

enable restriction to repos matching a pattern

This mod can wind up checking a lot of repos that you might not care about, and can hit rate limits doing so.

Would be nice to be able to check this way:

steampipe check benchmark.public_repo_best_practices --var repo_pattern=turbot/steampipe-mod

E.g. by doing something like this.

variable "repo_pattern" {
  type = string
  default = ""
}

benchmark "public_repo_best_practices" {
  title = "Public Repository Best Practices"
  description = "Best practices for your public repositories."
  children = [
    control.public_repo_issues_enabled
  ]
}

control "public_repo_issues_enabled" {
  title = "Issues should be enabled in each public repository"
  description = "Issues are essential to keep track of tasks, enhancements, and bugs."
  sql = <<-EOT
    select
      html_url as resource,
      case
        when has_issues then 'ok'
        else 'alarm'
      end as status,
      full_name || ' issues are ' || case when(has_issues)::bool then 'enabled' else 'disabled' end || '.' as reason,
      full_name
    from
      github_my_repository
    where
      full_name ~ $1
      and visibility = 'public' 
      and fork = ${local.include_forks}
  EOT
  param "repo_pattern" {
    default = var.repo_pattern
  }  
}

@cbruno10 we talked about this way back when I first started, was reminded of it today when building an example of using this mod. Would this be the right approach?

Steampipe sherlock controls fail due to internal desc = 'List' call is missing

Describe the bug

turbot-macpro-raj:steampipe-mod-github-sherlock raj$ steampipe check control.private_repo_default_branch_blocks_force_push
+ Default branch should block force push in each private repository ................................................................................................................. 1 / 1 [          ]
  ERROR: rpc error: code = Internal desc = 'List' call is missing required quals: column:'repository_full_name' operator: =

A similar issue persists for the below controls

steampipe check control.private_repo_default_branch_blocks_deletion
steampipe check control.private_repo_default_branch_protections_apply_to_admins
steampipe check control.private_repo_default_branch_requires_pull_request_reviews
steampipe check control.private_repo_default_branch_blocks_force_push

steampipe check control.public_repo_default_branch_blocks_deletion
steampipe check control.public_repo_default_branch_protections_apply_to_admins
steampipe check control.public_repo_default_branch_requires_pull_request_reviews
steampipe check control.public_repo_default_branch_blocks_force_push

Steampipe version (steampipe -v)
Example: v0.9.0

Plugin version (steampipe plugin list)
Example: v0.8.1

To reproduce

Execute entire sherlock mod steampipe check all or steampipe check control.private_repo_default_branch_blocks_force_push

Expected behavior
A clear and concise description of what you expected to happen.

Additional context

Update index doc and README for Steampipe v0.14.0 release

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

control.private_repo_no_outside_collaborators evaluates <nil> in reason

Describe the bug
No outside collaborators should have access in each private repository control does evaluate outside collaborators resulting in reason while running the control.

Steampipe version (steampipe -v)
Example: v0.10.x

Plugin version (steampipe plugin list)
Example: v0.9.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Update mod.sp to address deprecation warning

Describe the bug
The current version of the mod returns a warning due to using the deprecated option version instead of min_version for the github plugin.

This issue can be trivially resolved by updating

  require {
    plugin "github" {
      version = "0.29.0"
    }
  }

to

  require {
    plugin "github" {
      min_version = "0.29.0"
    }
  }


**Steampipe version (`steampipe -v`)**
Example: v0.3.0

**Plugin version (`steampipe plugin list`)**
Example: v0.5.0

**To reproduce**
Steps to reproduce the behavior (please include relevant code and/or commands).

**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional context**
Add any other context about the problem here.

Update mod to be compatible with forth-coming GraphQL changes in GitHub plugin.

Is your feature request related to a problem? Please describe.
We're currently looking to release a drastically different version of the GitHub plugin utilising GraphQL, this means that a lot of the queries will need to be adapted to facilitate the changes.

Describe the solution you'd like
All queries/checks/etc to be updated.

Describe alternatives you've considered
n/a

Additional context
GitHub plugin changes are currently on a graphql-dev branch or within PRs waiting to be merged into it.

Organization Best Practices controls should handle reason coming as <nil> for edge cases

Describe the bug
GitHub Organization with non-matching conditions of Organization Best Practices controls fails e.g.

Organization members should not be able to create public repositories

...................................................................................................     2 /     3 [==        ]
  | ALARM: <nil> .............................................................................................................................................................................. rajcloud
  | OK   : Turbot Development users cannot create public repositories. ........................................................................................................................ turbotio
  | OK   : Turbot HQ users cannot create public repositories. ................................................................................................................................. turbothq
  | 

Organization default repository permissions should be limited

  | OK   : <nil> .............................................................................................................................................................................. rajcloud
  | OK   : <nil> ................................................................................................................................................................................ turbot
  | OK   : Turbot Development default repository permissions are none. ........................................................................................................................ turbotio
  | OK   : <nil> .............................................................................................................................................................................. turbothq

Steampipe version (steampipe -v)
Example: v0.9.1

Plugin version (steampipe plugin list)
GitHub: v0.8.1

To reproduce
You can create sample organization without all required best practice parameters and run GitHub Sherlock mod as below
steampipe check github_sherlock.benchmark.org_best_practices

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.