Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
It is difficult to troubleshoot or understand what exactly Steampipe is requesting from Turbot. A user must translate in their head from SQL to GraphQL. For new users this can be very difficult. Dumping the query to logs would give new users a way to link their steampipe query to Turbot GraphQL. (Basically, this would be like the "Developers" tab in the Turbot console.)

Describe the solution you'd like
When STEAMPIPE_LOG=TRACE, output the Turbot GraphQL query and variables to the Steampipe logs.

Describe alternatives you've considered
Dig through the API logs in Turbot Master.

Additional context
This is a further enhancement to make it easier for users to tell the difference between Turbot-side and Steampipe-side filtering.

References
https://turbot.com/v5/docs/reference/graphql/query/watches

Describe the bug
In the Turbot Plugin, turbot_tag table. To confirm my understanding of it:

• table is distinct key:value pairs -- they get assigned a unique ID id.
• resource_id field is tied to resources that have the key:value (are associated with the id)
• all ids / key:values are returned as results because they exist in the Turbot CMDB, which means they are tags that are associated to Turbot, AWS, Azure or GCP resources
• CMDB only has current resources -- not prior / in the past.

If all is true, how does the turbot_tags table return null / '[]' for the resource_id column? Wouldnt every tag have at least 1 resource id associated?

Steampipe version (steampipe -v)
v0.19.5

Plugin version (steampipe plugin list)
Turbot v0.10.0

To reproduce
Example, if you run this query in Steampipe:

select
  *
from
  turbot_tag
order by
  resource_ids,
  key,
  value;

You will see examples of resource Ids column with []. However I would expect at least one resource ID. Right?

When you look up any of the Tags with [] in Turbot specifically, you see there are active resources associated. A tag like "Bucket Name: bob-demo-4-12-2023" is active, in the CMDB, and in AWS with that tag. But the result for it is '[]'.

In Turbot a search like: tags:'Bucket Name'='bob-demo-4-12-2023' returns one bucket. But in Steampipe its []

Expected behavior
All tags have at least 1 resourceId associated.

Describe the bug
When querying {resource{data}}, Turbot will always return the most recent resource data state. When making queries in turbot_resource, this is appropriate behavior. However, for notifications, we want the previous resource state. Using {resource{object}} is the correct query to make.

Steampipe version (steampipe -v)
❯ steampipe -v
steampipe version 0.14.6

Plugin version (steampipe plugin list)
| hub.steampipe.io/plugins/turbot/turbot@latest | 0.5.0 |

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

1. Pick any cloud resource with resource_created and resource_updated notifications
2. Replace 259791392905645 in the below query with the resource ID from step 1.

select id, process_id, notification_type, create_timestamp, resource_title, resource_new_version_id, resource_old_version_id, resource_type_id, resource_type_uri, resource_data 
from turbot_notification 
where filter = 'notificationType:resource resourceId:"259791392905645"'

3. Compare the resource_data column as the resource changes.

Expected behavior
The results from the query in step 2, should reflect the resource's change over time.

Additional context
The query needs to change on line 164, line 309 and line 66

Describe the bug
no values are returned for filter column if it is not passed as a qual

select
    id, filter
from
    turbot_demo.turbot_resource
where
    resource_type_uri = 'tmod:@turbot/aws-iam#/resource/types/accessKey'
and
    filter = '$.turbot.custom.createTimestamp:<=T-30d'

returns

+-----------------+-----------------------------------------+
| id              | filter                                  |
+-----------------+-----------------------------------------+
| 209793964907564 | $.turbot.custom.createTimestamp:<=T-30d |
| 216255846230771 | $.turbot.custom.createTimestamp:<=T-30d |
| 245280856563960 | $.turbot.custom.createTimestamp:<=T-30d |
| 215084373625757 | $.turbot.custom.createTimestamp:<=T-30d |
| 203192415818086 | $.turbot.custom.createTimestamp:<=T-30d |
| 241433740146509 | $.turbot.custom.createTimestamp:<=T-30d |
| 241434343667200 | $.turbot.custom.createTimestamp:<=T-30d |
| 224758800125111 | $.turbot.custom.createTimestamp:<=T-30d |
| 241434545526718 | $.turbot.custom.createTimestamp:<=T-30d |
| 245281067298667 | $.turbot.custom.createTimestamp:<=T-30d |
+-----------------+-----------------------------------------+

but

select
    id, filter
from
    turbot_demo.turbot_resource
where
    resource_type_uri = 'tmod:@turbot/aws-iam#/resource/types/accessKey'

returns

+-----------------+--------+
| id              | filter |
+-----------------+--------+
| 209793964907564 | <null> |
| 224758800125111 | <null> |
| 216255846230771 | <null> |
| 241434343667200 | <null> |
| 215084373625757 | <null> |
| 245280856563960 | <null> |
| 203192415818086 | <null> |
| 241434545526718 | <null> |
| 241433740146509 | <null> |
| 245281067298667 | <null> |
+-----------------+--------+

Steampipe version (steampipe -v)
v0.13.0

Plugin version (steampipe plugin list)
v0.4.0

To reproduce
run queries abovve

Expected behavior
filter column should. be populated

Additional context
n/a

Describe the bug
A clear and concise description of what the bug is.

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
The existing turbot_* tables are insufficient to cover the breadth of the Turbot API. Further, queries like resource_control_policy.graphql are easy to do in raw GraphQL but painful to do in Steampipe.

There is also considerable difficulty for users new to Turbot and to Steampipe with grokking where the filtering happens, whether Turbot-side or Steampipe-side. This difficulty can lead to long running queries when a user specifies Steampipe-side filtering then unintentionally pulls over hundreds of thousands of rows from Turbot. I believe/hope that if the user specifies the GraphQL themselves, it will be clear to them where the filtering/joining is happening.

Describe the solution you'd like
Specify a path to a file or directory on the local file system that contains GraphQL files. On launch, Steampipe will parse these query files then autogenerate tables and columns to match these queries. The overal operation would be similar to how dynamic tables are created for the CSV, Terraform and GoogleSheets plugins.

Describe alternatives you've considered
There aren't any really, other than to write the GraphQL queries into a general purpose programming language.

Additional context
The Turbot GraphQL API is incredibly rich. The current approach to statically defined tables removes some of that richness and imposes additional development load on Turbot plugin developers to implement each new table.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Use Case
I would like to write a Turbot Workspace Health mod. Determining the number of policy values in tbd, invalid, and error, and their age is very important to overall workspace health. Please include a metadata and workspace columns in this table.

References
Turbot docs on Notifications: https://turbot.com/v5/docs/concepts/notifications
API Refernce for Notifications: https://turbot.com/v5/docs/reference/graphql/query/notifications
Notification Definition: https://turbot.com/v5/docs/reference/graphql/object/Notification

To get Activity information out of Turbot, you need access to the Notifications table.

Describe the bug
queries to the table turbot_mod_version fail with AccessDenied exception from GovCloud

Steampipe version (steampipe -v)
Example: v0.16.0

Plugin version (steampipe plugin list)
Example: v0.8.0

To reproduce
configure a connection to a workspace running in AWS Gov Cloud
select * from turbot_mod_version;

Expected behavior
Query returns list of installed mod versions for this workspace

Additional context

Description
Sometimes, we want to get a count of a set of controls in Turbot without having to pull over every control to do the count Steampipe-side. Turbot's resources GraphQL API queries have a metadata.stats.total section that includes count data. This ticket requests the creation of a turbot_control_count table that exclusively returns the metadata.stats.total.

  controls(filter: $filter, paging: $paging) {
    metadata {
      stats {
        total
      }
    }

References

• Resource Counts in the turbot_notification table
• Related to

Describe the bug
The filter column fails to populate with an unusual error.

Error: 2 connections failed: 
connection 'taurus': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column
connection 'canis': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column (SQLSTATE HV000)

Steampipe version

❯ steampipe --version
steampipe version 0.11.0

Plugin version (steampipe plugin list)
+--------------------------------------------------+---------+----------------------------------------------+
| Name | Version | Connections |
+--------------------------------------------------+---------+----------------------------------------------+
| hub.steampipe.io/plugins/turbot/aws@latest | 0.43.0 | aac,aaa,aws,aab,all_sandbox |
| hub.steampipe.io/plugins/turbot/azure@latest | 0.22.0 | azure |
| hub.steampipe.io/plugins/turbot/csv@latest | 0.1.0 | csv |
| hub.steampipe.io/plugins/turbot/gcp@latest | 0.19.0 | demo |
| hub.steampipe.io/plugins/turbot/github@latest | 0.10.0 | github |
| hub.steampipe.io/plugins/turbot/slack@latest | 0.3.0 | slack |
| hub.steampipe.io/plugins/turbot/steampipe@latest | 0.2.0 | steampipe |
| hub.steampipe.io/plugins/turbot/turbot@latest | 0.2.0 | corvus,taurus,astro,canis |
+--------------------------------------------------+---------+----------------------------------------------+

To reproduce

1. Run this query against any single or multiple Turbot workspaces. I'm running TE 5.37.21.

with workspace as (
    select workspace
    from turbot_resource
    where filter = 'resourceId:"tmod:@turbot/turbot#/" level:self'
),
     webhook_rotation_policy as (
         select value,
                workspace,
                filter
         from turbot_policy_setting
         where filter = 'policyTypeId:"tmod:@turbot/turbot#/policy/types/webhookSecretRotation"'
     )
select w.workspace as resource,
       e.filter,
       case
           when e.value is null then 'Skip'
           else e.value
           end     as webhook_rotation_setting,
       case
           when e.value like 'Enforce: Rotate webhook secret' then 'ok'
           else 'alarm'
           end     as status,
       case
           when e.value like 'Enforce: Rotate webhook secret'
               then 'Webhook Secrets in ' || w.workspace || ' are set to rotate.'
           else 'Webhook Secrets in ' || w.workspace || ' do not rotate.'
           end     as status
from workspace w
         left join webhook_rotation_policy e using (workspace)

2. I've tried running it against multiple workspace and against individual workspaces. There's no change in error.
3. I've tried the webhook_rotation_policy query alone and it works just fine. The workspace query returns properly too. My hunch is that filter error comes from joining the two queries.

Expected behavior
I get a list of workspaces with a column indicating whether the Webhook rotation policy has been set to Enforce or not.

Additional context
Working on a Turbot Workspace health mod. This query is a part of that effort.

References
We have tables for Turbot Controls and Turbot Notifications. Steampipe's values as diagnostic tool would be greatly enhanced with the addition of a table to grab process logs also. This may also require enhancements on the turbot_control table to include "last_process_id" information to look up process logs.

Use Case
A customer reports a problem with a given control. We can hand them a steampipe query that will take a control_id. The query then returns any resource updates along with the control changes and the debug logs for each control change. This way we get a single diagnostic package instead of piecemeal "send me this, send me something else, send me another thing....etc."

The intent is to build a query to dump the information related to https://{workspace}/apollo/controls/{control_id}/control
as well as https://{workspace}/apollo/processes/{process_id}/detail
and https://{workspace}/apollo/processes/{process_id}/logs?filter=logLevel%3A%3E%3Ddebug
and https://{workspace}/apollo/processes/{process_id}/notifications (We already have the turbot_notification table so this is covered)

Describe the bug
The column name has a typo.
It is poliy_type_trunk_title
It should be policy_type_trunk_title

Plugin version (steampipe plugin list)
latest

https://hub.steampipe.io/plugins/turbot/turbot/tables/turbot_policy_value#inspect

References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.

Describe the bug
A clear and concise description of what the bug is.

I am trying to query turbot_active_grant with limit of 10 records in my turbot v5 environment which has 20+k records . i am getting out of memory error . I have 4GB Mem. I tried using latest Steampipe v0.16.0-rc.8 version . but no luck.

Steampipe version (steampipe -v) v0.16.0-rc.8
Example: v0.3.0

Plugin version (steampipe plugin list) 0.5.0
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

run below query in amazon linux 2

select grant_id,
resource_id,
identity_profile_id,
identity_display_name,
identity_email,
identity_status,
level_uri,
resource_type_uri,
workspace,
create_timestamp
from nonprod.turbot_active_grant
where resource_trunk_title = 'turbot'
and level_trunk_title = 'superuser'
limit 10;

Expected behavior
A clear and concise description of what you expected to happen.

retrun limited rows in few seconds

Additional context
Add any other context about the problem here.

Description
Sometimes, we want to get a count of a set of resources in Turbot without having to pull over every resource to do the count steampipe-side. Turbot's resources GraphQL API queries have a metadata.stats.total section that includes count data. However, a column for metadata counts in the normal turbot_resource table has been deemed inapproriate (and I agree). This ticket requests the creation of a turbot_resource_count table that exclusively returns the metadata.stats.total.

  resources(filter: $filter, paging: $paging) {
    metadata {
      stats {
        total
      }
    }

References
Resource Counts in the turbot_notification table

If this implementation is successful, additional tables for turbot_control_count, and turbot_notification_count will be requested.

References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.

I would like to audit who has active grant permissions in my turbot workspace.

Is your feature request related to a problem? Please describe.
If we need a count of controls or notifications, without the metadata column, we have to ship over a large number of rows to Steampipe in order to get a count. Doing a count of rows makes much more sense in the Turbot DB.

Describe the solution you'd like
Introduction of the metadata column to the turbot_control and turbot_notification tables.

Describe alternatives you've considered
None. I'm doing what Cody asked me to do 😁

Additional context
Useful when aggregating data across multiple workspaces.

Describe the bug
I'm trying to build a report of permission grants in Turbot. I'm having a hard time identifying the difference between AWS/Owner and Turbot/Owner. I see we have a level_title column which provies [Owner, Admin, Metadata, etc] but no column that says [AWS, Azure, GCP, Turbot].

Steampipe version (steampipe -v)
0.12.2

Plugin version (steampipe plugin list)
0.4.0

To reproduce

         select *
         from turbot_active_grant
         where identity_profile_id like '%bob%'

Run this against the Turbot Demo environment.

Expected behavior
A column that indicates whether this is grant is a AWS/*, Azure/*, GCP/* or Turbot/*. An additional column that shows the grant as it shows in the Turbot console would be nice (AWS/Owner), though not strictly necessary.

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
When querying turbot notifications across multiple workspaces, it's difficult to keep track of where the notifications came from without the workspace column. Related to: #3

Describe the solution you'd like
Inclusion of workspace column in the turbot_notification table.

Describe alternatives you've considered
There aren't any really.

Additional context
None.

Recompipe the plugin with steampipe-plugin-sdk v1.8.2 and Go Version 1.17

Is your feature request related to a problem? Please describe.
Verifying that Installed Mods are up-to-date is an important part of verifying workspace health. A regular turbot_resource query will get the installed mods. Discovery of the current Recommended version isn't currently possible through Steampipe.

Describe the solution you'd like
A new table for available mod version, perhaps called turbot_mod_versions. Should be able to search by modName, OrgName, free text search and status.

Describe alternatives you've considered
There is no alternative, other than to go look in the Turbot console for this information.

Additional context
The table should represent the same descriptive power as this query (pulled from the Turbot console).

query modVersionSearchByName {
  modVersionSearches(search: "", modName: "aws", orgName: "turbot", status: AVAILABLE) {
    items {
      identityName
      name
      versions {
        status
        version
      }
    }
  }
}