Hi thanks for the great plugin,

Was wondering if we can later add support GCP cloud billing, and since GCP is project based, can we separately add a tab in the entity details page, so that we can view each relavant gcp project costs?

I know this may be a big ask, please let me know if theres any way i can help.

Hi, I'm installing this new super plugins for GCP, but I'm blocked :(
I followed all the instructions, but when I pull it up inside my GKE, I get an error that my dataset cannot be found in the US, but it is not here, is in the EU

.

I don't see any config parameter that has to pass the EU location.

Can anyone help me? Thank you.

In the current implementation, InfraWallet backend checks if there are any category mapping records in table category_mappings, and inserts default mappings if the table does not contain any records. This is not convenient when we update the default category mappings (because there are records already, and probably some users already adjust the mappings).

Here is a proposal for better supporting category mappings:

• We create two database tables: category_mappings_default and category_mappings_override
• For category_mappings_default, when starting InfraWallet backend, it always cleans up the table and inserts the latest version of default category mapping
• For category_mappings_override, users can define overriding rules there and later on we will provide UI for configuring such mappings

hi @gluckzhang

Thanks for adding gcp earlier.

I do have an additional idea. now that we can catalog all the cost for all gcp projects, is there a way we can alert/notify or have a alert within the plugin page instead of resorting to a slack integration/ or directly within the component such that if the associated gcp_project of that component has a spending average higher than last month, we can see it or be notified?

I think the plugin itself currently tackles multicloud costs data visualization very well already, but i believe the next step is to take the data of what we process, alert users that if they are spending over their last monthly average.

The example above is based on gcp_project_id since i know gcp environment best, but im assuming the same logic for determining if the cost was going up for a certain project-id/service-account/resource-id logic is the same, it is just the cloud data category + resource type would be different.

Once again this is just a an idea that i think would be helpful and bring it to your radar, it doesnt even have to be a integration to slack or directly visisble in the component overview page, it could just all be shown in the plugin page after select a specific gcp-project-id. or in backstage's logic a certain group own services that are linked to their gcp-project-id.

For some of the fields in InfraWallet's configuration, and data model, we should follow the definitions in
https://focus.finops.org/#specification

For category mappings, we should use the same category names as well:
https://github.com/FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec/blob/v1.0/specification/columns/servicecategory.md

Currently InfraWallet backend outputs error messages from cloud providers as backend logs and frontend does not display such failures. To better inform the users that InfraWallet fails to fetch costs from some of the cloud accounts, these error messages should be sent to the frontend as well. We can still keep the response code as 200 since it is a partial failure, but it would be nice to tell which cloud account fails.

In the current implementation, the cost data is cached based on the query parameters sent from the frontend.

        const cacheKey = [
          client.constructor.name,
          filters,
          groups,
          granularity,
          startTime,
          endTime,
        ].join('_');
        const cachedCosts = (await cache.get(cacheKey)) as Report[] | undefined;

This will be an issue when a client like AwsClient fails to fetch costs from some of the AWS accounts. The partial results are still cached and then, the backend will not retry to fetch all cost data until the cache expires.

Hi, Can you validate the idea to add a Oracle Cloud Infrastructure to InfraWallet?
It would be more robust to be able to see all the costs in a single plugin.

It's and idea... thank you for consider it.... I'm interesting on your roadmap.

To better understand the correlations between cloud costs and business metrics, it is helpful to display these metrics such as usage, the number of monthly active users, etc. on the same chart (ColumnsChartComponent).

We may have a similar implementation as cost clients:

• an abstract class MetricProvider that defines the common logic and data format
• implementations for different metric providers, such as Datadog, Prometheus, etc.
• a new API endpoint for fetching metrics so that frontend can visualize them

An example in the ApexCharts documentation: https://apexcharts.com/react-chart-demos/mixed-charts/multiple-yaxis/

For the demo site or someone who wants to contribute to the frontend part, it is not necessary to set up different kinds of real cloud integrations. It is better to have a mock client that returns some fake cost data in this case.

Hi,
I'm working on GCP and I can view the cost of my services, So far everything is perfect.
But the plugin show me the cost of services without saving, is it possible to add the commitments?

Documentation:
https://cloud.google.com/billing/docs/how-to/bq-examples

Example:
SELECT invoice.month, SUM(cost) + SUM(IFNULL((SELECT SUM(c.amount) FROM UNNEST(credits) c), 0)) AS total, (SUM(CAST(cost AS NUMERIC)) + SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) AS c), 0))) AS total_exact FROM 'project-ID.dataset.gcp_billing_export_v1_XXXXXX-XXXXXX-XXXXXX' GROUP BY 1 ORDER BY 1 ASC ;

Thank you.

Hi guys,
Is it possible to add a default group by i.e. by default I want the graphs to display with group by Name (name of cloud accounts)?

Before grouping the cost reports, the frontend should enable users to filter the records. For example, group the records whose project tag has the value X by providers.

Hi
In case of extracting data from cost explorer, can we add the functionality to filter based on a tag(s). For example, filter cost of all cloud services having tag "x". Also things like filter cost of all Lambdas having a tag "y".

Currently if one exports the costs to CSV from the InfraWallet cost table, all the numbers have the same format and become strings. It would be better to use raw numbers to make Excel visualizations/functions easier.

Hi All,

I couldn't find anything about permissions being expose for the permissions plugin.
Would you consider adding permissions or is this topic something not considered for this project?
I could create a PR for this if it is considered.

Best Regards.

In the current version, if the credentials for one account are not set up correctly, InfraWallet backend responds 500 and does not return the cost data of other correct accounts.

Currently, there is a bunch of duplicate code in each InfraWalletApi client, and the flow of fetchCostsFromCloud is actually the same:

• read config
• loop billing accounts / sub-accounts in each cloud provider
• check if there is any cached data for a sub-account based on the query
• if there is no cache, send APIs to fetch cost data from a cloud provider
• convert the response from the cloud provider to the InfraWallet data model
• cache the results for some time
• return the full list of cost records

The code should be refactored using class inheritance instead of the current interface.

Hi
Is it possible to provide IAM roles across different accounts (configured with cross account assume role) and fetch cost explorer metrics using these IAM roles. That would make it much simpler and easy to use. For implementation around that, you can refer to https://github.com/awslabs/backstage-plugins-for-aws in which plugins like ecs make use of IAM roles.

Interested to use this plugin!

Currently, we do a full string comparison in getCategoryByServiceName (https://github.com/electrolux-oss/infrawallet/blob/main/plugins/infrawallet-backend/src/service/functions.ts#L48C17-L48C41) for category mappings. It is better to support regular expressions there to categorize similar service names easily.

For example, in AWS different LLMs are considered as different services. For now, if we would like to categorize them into Artificial Intelligence, all of the names need to be added to category_mappings_default. If there is a new model, we need to update the mappings every single time.

A better approach is to categorize all the services whose names contain Bedrock to Artificial Intelligence.

Hi guys,
I tried using the infra wallet plugin for cost explorer metrics on backstage. I have tried integrating with multiple aws accounts. On running the plugin, I am getting error code 500 from /reports api endpoint.

{
    "error": {
        "name": "Error",
        "message": "An internal error occurred logId=e0721b098d4b588b14ba"
    },
    "request": {
        "method": "GET",
        "url": "/reports?&filters=&groups=&granularity=monthly&startTime=1711909800000&endTime=1719772199999"
    },
    "response": {
        "statusCode": 500
    }
}

I have integrated 2 aws accounts with this plugin i.e. my app-config.yaml:

  infraWallet:
    integrations:
      aws:
      - name: accountA
        accountId: ${AWS_ACCOUNT_ID_A}
        roleArn: ${AWS_IAM_ROLE_ARN_A{
        region: ${AWS_REGION}
      - name: accountB
        accountId: ${AWS_ACCOUNT_ID_B}
        roleArn: ${AWS_IAM_ROLE_ARN_B}
        region: ${AWS_REGION}

Can anyone help with what's wrong or how to debug this?