Giter Club home page Giter Club logo

Comments (17)

nivethj avatar nivethj commented on June 20, 2024 1

It ran long and timed out.

Console

limit 10;
Error: Post "https://server/api/latest/graphql": net/http: TLS handshake timeout (SQLSTATE HV000)

Log file

2022-11-29 19:59:22.753 UTC [16105] STATEMENT: select grant_id,
resource_id,
identity_profile_id,
identity_display_name,
identity_email,
identity_status,
level_uri,
resource_type_uri,
workspace,
create_timestamp
from nonprod.turbot_active_grant
where resource_trunk_title = 'turbot'
and level_trunk_title = 'superuser'
limit 10;
2022-11-29 20:00:38.460 UTC [16105] LOG: disconnection: session time: 3:58:52.816 user=steampipe database=steampipe host=127.0.0.1 port=37158

from steampipe-plugin-turbot.

dboeke avatar dboeke commented on June 20, 2024

I find when I do this query on an environment with user mode enabled that it results in many duplicate records. I wonder of that is the underlying cause of the issue?

from steampipe-plugin-turbot.

dboeke avatar dboeke commented on June 20, 2024 
select 
  grant_id,
  resource_id,
  identity_profile_id,
  identity_display_name,
  identity_email,
  identity_status,
  level_uri,
  resource_type_uri,
  workspace,
  create_timestamp,
  resource_trunk_title
from 
  turbot_active_grant
where 
  resource_type_uri = 'tmod:@turbot/turbot#/resource/types/turbot'
  and level_uri = 'tmod:@turbot/turbot-iam#/permission/levels/superuser'

from steampipe-plugin-turbot.

dboeke avatar dboeke commented on June 20, 2024

both of the where constraints should hit indexes

from steampipe-plugin-turbot.

misraved avatar misraved commented on June 20, 2024

Hello @nivethj, I tried to replicate the issue but without much success.

To replicate the issue, could you please provide a few more details -

  • How many plugins other than turbot are you using in the Amazon Linux 2 environment?
  • Do other queries work or is it only the turbot_active_grant table queries which are failing?
  • If you restart steampipe and rerun the query, do you still see the issue?

from steampipe-plugin-turbot.

nivethj avatar nivethj commented on June 20, 2024

Below are the requested details

How many plugins other than turbot are you using in the Amazon Linux 2 environment?
I’m only using only turbot plugin. I have AWS plugin installed , but not using it.

Do other queries work or is it only the turbot_active_grant table queries which are failing?
I haven’t worked on other tables . turbot_active_grant is working in sandbox which has few rows (<20 rows)

If you restart steampipe and rerun the query, do you still see the issue?
Restart is not helping . query runs for 15 mins and end with error

from steampipe-plugin-turbot.

misraved avatar misraved commented on June 20, 2024

Hey @nivethj, sorry for the delayed response.

Sorry to see that you are still blocked with this issue. We released a new version of the turbot plugin (v0.9.0) which includes essential memory management fixes.

Could you please upgrade the plugin and check if the issue persists?

from steampipe-plugin-turbot.

cbruno10 avatar cbruno10 commented on June 20, 2024

@nivethj To add onto @misraved 's comment, can you please also ensure that you're running Steampipe CLI v0.16.1? This CLI has some updates to allow it to better handle the memory management improvements mentioned. Thanks!

from steampipe-plugin-turbot.

nivethj avatar nivethj commented on June 20, 2024

updated steampipe to (steampipe version 0.16.3). turbot plugin to 0.9.0.

row limit 10 is working with 60% Mem , when i update row limit 100 , max memory reached

from steampipe-plugin-turbot.

github-actions avatar github-actions commented on June 20, 2024

'This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.'

from steampipe-plugin-turbot.

nivethj avatar nivethj commented on June 20, 2024

im following this request

from steampipe-plugin-turbot.

misraved avatar misraved commented on June 20, 2024

Hello @nivethj, we are extremely sorry for the delay.

While running the query, could you please grab the plugin logs for us to investigate? I tried to replicate this error but without any success.

Hoping to catch something unusual in those logs which might help us in debugging this issue.

from steampipe-plugin-turbot.

misraved avatar misraved commented on June 20, 2024

Apologies for the radio silence on this issue @nivethj!!

Are you still seeing this error with the new v0.19.5 CLI and the latest turbot plugin versions?

from steampipe-plugin-turbot.

nivethj avatar nivethj commented on June 20, 2024

tried on v0.20.5 . still the same issue

Welcome to Steampipe v0.20.5
For more information, type .help
steampipe-postgres-fdw updated to 1.7.2.

=> Same query used

net/http: TLS handshake timeout (SQLSTATE HV000)

from steampipe-plugin-turbot.

bigdatasourav avatar bigdatasourav commented on June 20, 2024

Hello @nivethj, I attempted to replicate the issue, but unfortunately, I couldn't succeed. Could you kindly clarify whether by 20+k records you are referring to 20+k resources or active grants? If it pertains to 20+k resources, please provide me with the number of active grants you currently have.

from steampipe-plugin-turbot.

bigdatasourav avatar bigdatasourav commented on June 20, 2024

We have tested in a similar env with 20+K active grants, with limits 10, 100, 1000 - (Works fine)

Welcome to Steampipe v0.20.7
For more information, type .help
> select grant_id,
resource_id,
identity_status,
level_uri,
resource_type_uri,
workspace,
create_timestamp
from turbot_active_grant
limit 10;
+-----------------+-----------------------------------------------------+--------------------------------------------+----------------------+
| identity_status | level_uri                                           | resource_type_uri                          | create_timestamp     |
+-----------------+-----------------------------------------------------+--------------------------------------------+----------------------+
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-06-15T17:50:36Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/turbot#/resource/types/turbot | 2022-02-22T07:39:50Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-08-20T11:21:59Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2022-02-03T18:54:05Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-10-19T19:52:26Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2022-01-27T02:56:21Z |
+-----------------+-----------------------------------------------------+--------------------------------------------+----------------------+

 select
identity_status,
level_uri,
resource_type_uri,
create_timestamp
from turbot_active_grant
limit 100;
+-----------------+-----------------------------------------------------+--------------------------------------------+----------------------+
| identity_status | level_uri                                           | resource_type_uri                          | create_timestamp     |
+-----------------+-----------------------------------------------------+--------------------------------------------+----------------------+
| Inactive        | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account   | 2022-02-26T07:56:14Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-06-15T17:50:36Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-08-20T11:21:59Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2021-10-19T19:52:26Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2022-01-27T02:56:21Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot | 2022-02-03T18:54:05Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/turbot#/resource/types/turbot | 2022-02-22T07:39:50Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Inactive        | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder | 2022-02-22T11:01:11Z |

 select
identity_status,
level_uri,
resource_type_uri,
create_timestamp
from turbot_active_grant
limit 1000;
+-----------------+-----------------------------------------------------+-----------------------------------------------+----------------------+
| identity_status | level_uri                                           | resource_type_uri                             | create_timestamp     |
+-----------------+-----------------------------------------------------+-----------------------------------------------+----------------------+
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:14Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/user     | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:05Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot    | 2021-06-15T17:50:36Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/user     | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:06Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot    | 2021-08-20T11:21:59Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot    | 2021-10-19T19:52:26Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot    | 2022-01-27T02:56:21Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:13Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/turbot    | 2022-02-03T18:54:05Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:13Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:15Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/turbot#/resource/types/turbot    | 2022-02-22T07:39:50Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:19Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:30Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:45Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/owner    | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:47Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:51Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:09Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:58Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:00Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:03Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:04Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:13Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/user     | tmod:@turbot/aws#/resource/types/region       | 2022-02-26T07:56:13Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:56:14Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/user     | tmod:@turbot/aws#/resource/types/account      | 2022-02-26T07:55:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/admin    | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |
| Active          | tmod:@turbot/turbot-iam#/permission/levels/metadata | tmod:@turbot/turbot#/resource/types/folder    | 2022-02-22T11:01:10Z |

Could you please update the CLI version to the latest and try?

from steampipe-plugin-turbot.

bigdatasourav avatar bigdatasourav commented on June 20, 2024

Hey @nivethj, We are closing this issue because we have not heard from you. Please feel free to reopen the issue if you want to share or discuss anything.

from steampipe-plugin-turbot.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.