📝 Scenario

As a SQL admin, I need to review my AHB usage in order to determine if I'm maximizing my existing licenses

💎 Solution

Create a new Azure Hybrid Benefit report for SQL Server licenses.

ℹ️ Additional context

The main issue here is that SQL and Windows usage data is inconsistent. Consider creating a proposal for the SQL and/or VM teams to emit usage in a consistent manner. Another possible workaround is to manipulate data in Data factory, but this is less ideal.

• Add a new data factory instance to the hub module.

⚠️ Problem

In an effort to accelerate timelines, we are merging with https://github.com/MSBrett/ccm_lite for the v0.1 release.

🛠️ Solution

Update docs based on revised plans for the 0.1 release.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a hub admin, I need to automate CM export creation in order to minimize administrative overhead

💎 Solution

Create a msexports_Setup pipeline in ADF to manage Cost Management data exports automatically. Use config\settings.json to store the list of exports that will be managed.

📗 Documentation updates

1. Remove steps 2 and 3 (create and run exports) in Create a new hub.
2. Add the following paragraph before the info message under the "Access the cost data from custom tools" bullet in Get started with hubs.

   Data is ingested into the msexports container and transformed when moved into the ingestion container. Do not use the msexports container for anything other than Cost Management exports. If manipulating data, please do that in the ingestion container after the transform pipeline completes. Don't remove or rename columns, as that can break Power BI reports.

3. Remove the comment about needing to setup Cost Management exports on the FinOps hub template description.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to enable tag inheritance in order to streamline cost allocation and chargeback

💎 Solution

Create a template that enables tag inheritance for an EA billing account, MCA billing profile, or EA/MCA subscription.

Also add this as an option to the export and FinOps hub with exports templates.

📝 Scenario

As a FinOps practitioner, I need to deploy scheduled actions as part of my templates in order to automate onboarding for new projects

💎 Solution

Publish scheduled action modules to the Bicep Registry for each supported scope.

ℹ️ Additional context

Bicep requires separate files per deployment scope. Since this would require excessive duplication of code, we'll introduce a simple templating language optimized for enabling multi-scope deployment modules that support native Bicep tooling.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

🚀 Goal

Support large datasets (up to 500 GB/mo) with up to 5 years of historical data that refreshes when changed.

📋 Tasks

🙋‍♀️ Ask for the community

We'd love to get your feedback! Please leave any questions, comments, or change requests in the release discussions linked from each task list above. We'll try to close each of those out before we finalize the release.

⚠️ Problem

Everyone is currently working out of main but that will eventually cause issues when features have interim PRs that are not ready to roll out yet.

🛠️ Solution

Document a branching strategy where devs work out of their own branches/forks, then commit back to a feature branch, which then gets merged into a dev branch. When the release is ready, we'll merge dev to main.

ℹ️ Additional context

This should be done before closing any PRs that will not be included in the first release.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments if you'd like to share any thoughts to improve the plans.

📝 Scenario

As a DevOps team/leadership, I need to understand how many resources are deployed by type in order to <TODO: What is the goal? Why is this needed?>

⭐ This is a requested feature, but we could use more context on how and why it would be used. Please leave comments.

💎 Solution

Create an Inventory page that provides a breakdown by resource type.

ℹ️ Additional context

Consider writing a proposal to offer a built-in view inside of Cost Management for the same, if applicable.

⚠️ Problem

Need automated validation to avoid missing validation steps.

💎 Solution

Investigate the following template validation options:

• https://learn.microsoft.com/azure/azure-resource-manager/bicep/deploy-github-actions
• https://learn.microsoft.com/training/modules/test-bicep-code-using-github-actions
• https://learn.microsoft.com/azure/azure-resource-manager/templates/test-toolkit
• https://github.com/Azure/PSRule.Rules.Azure

📝 Scenario

As a FinOps practitioner, I need to monitor both actual and amortized costs using a single, consolidated dataset in order to produce chargeback reports and reconcile invoices without duplicating data

💎 Solution

Change amortization to be applied as separate rows, like Cost Management cost allocation, which would allow data to be either amortized or actual based on whether a filter is applied. Amortization would be the default since that is the more common scenario across the organization. (Only the people paying the invoice require the actual, billed charges.)

ℹ️ Additional context

Consider writing a proposal to change how Cost Management provides amortized data. If the data was generated the same way cost allocation is generated, then this workaround wouldn't be needed and organizations would be able to reduce their data storage costs.

⚠️ Problem

EA, MOSA, MCA, CSP, and classic Internal subscriptions all use different schemas. FOCUS is not available yet, so we need an interim schema.

🛠️ Solution

Standardize a cost data schema that can be used until FOCUS is finalized.

The following changes have been scoped out for now to minimize changes compared to the Cost Management connector:

• Rename AccountOwnerId to ClassicAccountOwnerId
• Rename AccountName to ClassicAccountName
• Rename BenefitId/Name to CommitmentId/Name
• Rename BillingCurrencyCode to Currency
• Rename ConsumedService to ResourceProvider = ConsumedService || (parse from ResourceId)
• Rename CostInBillingCurrency to Cost
• Rename CustomerTenantId to CustomerId
• Rename CustomerName to CustomerName
• Rename MeterCategory to ServiceName
• Rename MeterSubCategory to ServiceTier
• Rename PartnerTenantId to PartnerId
• Rename ProductOrderId to OfferId
• Rename ProductOrderName to OfferName
• Rename ResellerMpnId to ResellerId
• Add ResourceGroupId = ResourceGroupName == null ? null : (parse from ResourceId)
• Add ResourceLocationId and ResourceLocationName for internal ID and English display names
• Add ResourceTypeName (display names from Azure portal)
• Add ServiceLevel = Allowed: OnPremises, Infrastructure, Platform, Software, Other
• Add Hybrid Benefit columns from Power BI reports: HybridBenefitStatus, HybridBenefitVCPUs, CPUHours, CPUs, ImageType, InstanceName, SKU, UsageType, VMProperties, vCPUs
• Add unique name columns from Power BI reports: BenefitNameUnique, ResourceGroupNameUnique, ResourceNameUnique, SubscriptionNameUnique
• Add utilization and savings columns from Power BI reports: BenefitSavings, BenefitType, BenefitUtilizationAmount, BenefitUtilizationPotential, DiscountSavings, NegotiatedSavings, OnDemandCost
• Consider merging ServiceInfo1 and ServiceInfo2 into AdditionalInfo
• Only show column PartnerEarnedCreditRate/Applied if available
• FOCUS: Rename ServiceFamily to ServiceCategory + add mapping to allowed values
• Consolidation of actual and amortized cost into a single dataset
• Consider a way to opt in to localized strings

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need a central cost optimization hub in order to drive cost optimization within my organization

💎 Solution

Create a Cost optimization workbook that summarizes potential savings opportunities.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to backfill historical cost data in order to report on past data and monthly trends in the initial release

💎 Solution

Update the ms-cm-exports_Setup pipeline in ADF to handle data ingestion for historical data. Reuse the same exports to ensure data is placed in the same folder. Refer to config\settings.json for data retention settings.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to break costs down by tag in order to identify tagging gaps and perform chargeback

💎 Solution

Create a Tags page in Power BI that helps understand what tag keys are applied, offers drilldown into the tag values, and also what resources are missing tags.

ℹ️ Additional context

Consider creating a proposal for a built-in view in Cost Management for this, given the in-portal experience would likely be a better place for DevOps teams to discover and resolve tag gaps.

🚀 Goal

Create an ARM template that configures a new Cost Management export to ADLS and generates a Power BI report that can work against that dataset. The template should also include a Data Factory pipeline to delete old files when data is exported each day.

📋 Tasks

🙋‍♀️ Ask for the community

We'd love to get your feedback! Please leave any questions, comments, or change requests in the release discussions at the bottom of each task list above. We'll try to close each of those out before we finalize the release.

📝 Scenario

As a CSP partner, I need to view partner-earned credit in order to understand how much I'm earning from my customers

💎 Solution

Create a separate Power BI report focused on partner-earned credit (PEC) scenarios.

Issues

• 1. #Triage when new issues are created -- open issue !#Information !#Spec in progress !#Spec review !#Ready !#In progress !#Code review !assigned > #Triage @flanakin
• 2. #Triage > #Info when comment includes "#needs-info" in a comment -- open issue "#needs-info" comment from @ankur-ms/@flanakin/@tsilvers-ms > #Info -#Triage @author
• 2a1. +#Attention when no activity -- @1am: open issues inactive for 14d #Info -Attention > #Attention + reply
• 2b. -#Attention when activity -- issue comment #Attention > -#Attention
• 2a2. #Attention > Close when no activity -- @1am: open issues inactive for 14d #Info #Attention > Close #NoActivity + reply
• 2b1. #Duplicate > Close when no activity -- @1am: open issues inactive for 7d #Duplicate > Close + reply
• 3. #Info > #Triage when author comments -- open issue comment from author #Info > -#Info #Triage @flanakin
• 4. -#Triage when "#approved" comment -- open issue "#approved" comment from @ankur-ms/@flanakin/@tsilvers-ms #Triage > -#Triage @none
  - [ ] No tags
  - [ ] "#ADF" > #Data factory
  - [ ] "#ARM" > #ARM
  - [ ] "#PBI" > #Power BI
• 6. #Spec in progress > #Spec review when "Spec review:" PR created -- open issues #Spec in progress +#Spec review > -#Spec in progress
• 7. #Spec review > #Ready when "Spec review:" PR closes -- open issues #Spec review +#Ready > -Spec review
• 8. #Ready > #In progress when dev assigned -- open issues #Ready @ankur-ms/tsilvers-ms > -#Ready #In progress
• 9. #In progress > #Code review when dev creates PR -- new PR #In progress > issue+#Code review +#(copy Area|Breaking|GFI|HW|Micro|Type from issue)
  -- Issues #In progress +#Code review > -#In progress
• 10. #Code review > #Pending release when PR closes linked issue -- open issues closed #Code review > -#Code review #Pending release
• 11. #Pending release > #Released when issue is included in a release -- closed issue "Released in v" comment #Pending release > -#Pending release #Released

Pull requests

• 1. (New) > #Review when opened -- new/updated PR =main > #Review @ankur-ms @tsilvers-ms @flanakin +reviewers
• 3a. #Attenton > #Review when PR updated -- author updated PR #Attention > -#Attention #Review
• 3c. #Attenton > #Review when PR comment updated -- author PR comment #Attention > -#Attention #Review
• 4. #Review > Close???

⚠️ Problem

CSV is slower to process, so we're switching to parquet for data storage.

🛠️ Solution

Switch Power BI to ingest data in parquet format.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to estimate the cost of my hub instance in order to plan for future spending

💎 Solution

Create a Power BI report that looks at hub configuration settings and available data size to report on historical and estimate future monthly spending.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to automate the deployment of Cost Management resources in order to streamline onboarding

🚀 Goal

Publish modules to the Bicep Registry for all Cost Management features to simplify template creation.

📋 Tasks

🙋‍♀️ Ask for the community

We'd love to get your feedback! Please leave any questions, comments, or change requests in the release discussions at the bottom of each task list above. We'll try to close each of those out before we finalize the release.

⚠️ Problem

Power BI isn't very clear about what currency is selected.

🛠️ Solution

Show all cost data (KPIs, charts, and tables) with a formatted currency value.

ℹ️ Additional context

Power BI allows hard-coding currency, but we need to investigate how to set it dynamically from another Currency field.

This preview feature may help: https://learn.microsoft.com/power-bi/create-reports/desktop-dynamic-format-strings

⚠️ Problem

Custom columns require more work to copy between reports for manual upgrades. Avoid using custom columns when possible.

🛠️ Solution

Move the following calculated columns to be defined in the query:

• CommitmentNameUnique
• SubscriptionNameUnique
• ResourceNameGroupUnique
• ResourceNameUnique
• CommitmentSavings
• CommitmentType
• CommitmentUtilizationAmount
• CommitmentUtilizationPotential
• DiscountSavings
• Month
• NegotiatedSavings
• OnDemandCost
• RetailCost
• RetailPrice

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

⚠️ Problem

Settings must be saved to storage in order to have exports dynamically created.

🛠️ Solution

• Update the finops-hub template and hub.bicep module to accept exportScopes.
• Pre-create the config, ms-cm-exports, and ingestion storage containers in the hub.bicep module.
• Save the config/settings.json file in the hub.bicep module.

ℹ️ Additional context

See finops-hub template and hub module specs.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

⚠️ Problem

Docs cover a few specific topics, which are good, but they don't explain:

• What FinOps toolkit is.
• Who it's built for.
• Why they'd use it.
• How the end-to-end setup process works.
• What will get deployed (in a simple visual)
• How to customize it and what not to customize.

🛠️ Solution

Add more complete documentation for consumers.

🙋‍♀️ Ask for the community

Check out the docs and let us know what you think is missing or would like to see added.

📝 Scenario

As a FinOps hub admin, I need to be notified before my exports expire in order to ensure cost management activiites are not interupted

💎 Solution

Enable an option to either notify admins or auto-renew exports.

📝 Scenario

As a FinOps practitioner, I need to normalize cost data to a standard schema in order to **manage costs from **

💎 Solution

Normalize cost data to an interim schema (since FOCUS is not available yet) that aligns to the current Cost Managment connector schema.

📗 Documentation updates

1. Change the transform step in the data processing page to:

   The msexports_Transform pipeline transforms the raw data to the normalized schema and saves it in parquet format to the ingestion container. For details about the transformation, see the dev docs.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Do you have preferences on what should be covered in the normalized schema?
3. Leave comments to help us solidify the vision.

## 🐛 Problem

prob

## 👣 Repro steps

repro

## 🤔 Expected

expected

## 📷 Screenshots

screens

## ℹ️ Additional context

add'l

⚠️ Problem

Bicep-Build.ps1 does not account for different scopes in text strings in README.md or metadata.json. These must be manually updated.

🛠️ Solution

• Move core BRM files to a template folder.
• Create a metadata file that contains details that change:
  • Name
  • Description (can change per scope)
  • Version
• Update the build script to copy template files and update them based on the metadata.
  • Validate summary is <= 120 characters.
  • Allow formatting strings with the scope type.
  • For tenant scope, use "billing account" since we don't actually support tenant scope.
  • Prefix the README.md with the name, summary, and description.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps hub admin, I need to automate assignment of the Storage Account Data Reader role in order to streamline Power BI setup

💎 Solution

Create a template that assigns the Storage Account Data Reader role. Consider making this an optional part of the finops-hub template.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps practitioner, I need to have data p ushed at specific times in order to have a complete dataset when presenting to leadership

💎 Solution

Configure a data factory pipeline to manually run exports at specific times rather than waiting for exports to be run on their schedule.

📝 Scenario

As a FinOps practitioner, I need to stop all charges when a budget is exceeded in order to control costs

💎 Solution

Create a template that can deploy an action group with script to shut down and/or delete resources.
-OR-
Add the ability to monitor cost and trigger custom actions in FinOps hubs.

📝 Scenario

As a FinOps practitioner, I need to dedupe CM exports in order to directly link Power BI to ADLS as a data source

💎 Solution

Create an ms-cm-exports_Transform pipeline to copy data from the ms-cm-exports container to the ingestion container in parquet format and overwrite old exports.
Start from https://github.com/MSBrett/ccm_lite/blob/main/ARM/azuredeploy.json and copy relevant pieces of https://github.com/MSBrett/ccm_lite/blob/main/Deploy-EA.ps1, as appropriate.

🙋‍♀️ Ask for the community

Let us know if there's anything you'd like to see changed.

📝 Scenario

As a FinOps practitioner, I need to enable exports independently from the main FinOps toolkit deployment in order to handle scenarios where different accounts require different access

💎 Solution

Create a separate export template in addition to the finops-hub-with-exports template.

📝 Scenario

As a FinOps practitioner, I need to ingest data using a standard schema in order to support multiple clouds and account types

💎 Solution

Create a data factory pipeline that transforms data to the FOCUS schema. Use a new container for data ingestion and move Cost Management exports into this container when transformed.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

⚠️ Problem

There are some common patterns that we'll use in multiple bicep files that could be error prone when repeated often or introduce maintenance burden when needing to update them.

🛠️ Solution

Add an option to include reusable files using our Bicep templating. Something like:

// @include ../../include/telemetry.bicep

Consider explicit directives for the most common scenarios, like telemetry.

// @telemetry 'unique-id'

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

⚠️ Problem

Azure Quickstart Templates is the primary repository that FinOps toolkit releases will be published to. We need to minimize the effort needed to publish updates to ensure the process is as smooth and error-free as possible.

🛠️ Solution

Automate the AQT release process as much as possible to avoid manual, error-prone steps. If we can't automate the actual publishing, at least generate a ZIP file that can be used to manually publish.

⛔ Blocked on modules other changes being committed so we can move modules into the finops-hub folder.

ℹ️ Additional context

See also: https://github.com/Azure/azure-quickstart-templates/blob/master/1-CONTRIBUTION-GUIDE/README.md#contribution-guide

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

⚠️ Problem

When a hub has multiple currencies, Power BI shows them all together, which isn't valid. People have to select a single currency before the report can accurately render.

🛠️ Solution

Have the filter preset based on what currencies are available. Prefer the currency that is used the most.

⚠️ Problem

The main readme must include an explicit note about telemetry collected and how to disable it.

🛠️ Solution

Add the data collection section to this repo.

🙋‍♀️ Ask for the community

Let us know if there's anything specific you'd like to see in this space.

⚠️ Problem

Most people won't discover the toolkit from GitHub alone.

🛠️ Solution

Publish the toolkit templates to Azure Quickstart Templates.

ℹ️ Additional context

See also: https://github.com/Azure/azure-quickstart-templates/blob/master/1-CONTRIBUTION-GUIDE/README.md#contribution-guide

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

📝 Scenario

As a FinOps hubs admin, I need to deploy and auto-configure PBIX files in order to streamline setup after each release

💎 Solution

Add an option to templates that allow an admin to decide if they want files deployed or not. When files are deployed, they should be auto-configured to point to the correct storage account. Auth credentials should be provided on first-run. Update the pbix template output to point to the new files.

⚠️ Problem

As a FinOps practitioner, I need to build custom reports outside of Power BI in order to meet my organizational reporting needs

🛠️ Solution

Document tools and services that can be connected with Data Lake Storage.

📝 Scenario

As a hub admin, I need to save my deployment parameters file in order to update my deployment in the future.

💎 Solution

Save the deployment parameters file to the export storage account.

📝 Scenario

As a FinOps practitioner, I need to enable tag inheritance in order to facilitate chargeback reporting

💎 Solution

Create templates to enable tag inheritance for EA billing accounts, MCA billing profiles, and EA/MCA subscriptions.

Consider blueprints as an option to automate application for all subscriptions.

📝 Scenario

As a FinOps practitioner, I need to view cost reports in order to understand my current cost details

💎 Solution

Create cost summary and commitment discount Power BI reports to get a more complete picture of current costs.

ℹ️ Additional context

See reports docs for initial spec.

⚠️ Problem

PBIX files are ZIP files, which contain text files. As binary files, there's no transparency into what changed in each version.

🛠️ Solution

Investigate options that can version control PBIX (or possibly generic ZIP) files. Some options:

• https://bitbucket.org/sippey/zippey
• https://github.com/synchrotech/powerbi-source-control
• https://github.com/ckrf/xlsx-git
• https://github.com/pbi-tools/pbi-tools

📝 Scenario

As a user in a global organization, I need to convert my billing currency to my personal local currency in order to understand and report on costs within my local geography

💎 Solution

Cost details includes a separate column for cost in USD, but all the reports are using the Cost column. Add the ability to convert this USD column to any currency using a free or open-source currency conversion service.

ℹ️ Additional context

Microsoft has currency conversion rates available internally. Consider writing a proposal for how we'd like to see that exposed via Cost Management APIs.

🐛 Problem

problem goes here

👣 Repro steps

1. 2. 3. ...

🤔 Expected

expected this

📷 Screenshots

[ o ]

ℹ️ Additional context

More to say...

📝 Scenario

As a FinOps practitioner, I need to report on many datasets in order to show a complete picture of my costs

💎 Solution

Add support for the following datasets based on what's currently available in the Cost Management connector for Power BI:

• Balance summary (EA only)
• Budgets
• Actual cost
• Pricesheets
• Reservation charges
• Shared reservation recommendations
• Scoped reservatios recommendations

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.