Comments (10)
Related to the yml editor, I was dreaming about a preview entity page while writing new catalog files.
Is a bit edgy and maybe only useful while adopting/learning but having feedback while learning to add new kinds would be nice.
from community.
During March 16th Adotion SIG, I showed my proposal for a North Star of a future ingestion system. I am describing the user experience of using the ingestion system to help align the community on where we could be going. It may take a long time to realize this experience, but I'm hoping it'll help us talk about how we want things to be, especially as we move to event-driven ingestion.
from community.
Just popping in a quick comment that we should have a .Net ingestor - this could pulling in data based on .csproj
or .vbproj
(sorry but there is still a lot of legacy out there) files and creating entities from them
@fcorti this could to beside the What Else on your diagram 😉
from community.
We met Feb 28, 2023
Attendees: Francesco Corti, Mihai Tabara, minkimcello, Ravikumar Tadikonda, Taras Mankovski, Waldir Montoya, Phil K.
We discussed the idea and proposal described by the picture below.
The general feedback is positive on the proposal.
Something to discuss further is the way to "extend" the metadata collected through the various "bundled ingestors".
This is something that we would like to discuss into a future session.
Action: @fcorti shares a doodle to setup another working session.
Please raise comments here if you have any thoughts, suggestions, concern.
from community.
Cloud Ingestor
@adamdmharvey, the idea of a cloud ingestor is an interesting one. Historically, the catalog wasn't designed to ingest lots of frequently changing data, but it does unlock some interesting possibilities. For example, we could create a shared plugin to compute DORA metrics. The move to event-driven ingestion might nudge the community in this direction.
from community.
Wanted to potentially throw out the idea of a Cloud Ingestor. For example, we could find it very useful to ingest Kuberentes clusters as resources in the catalog, which we can then tie elements to. (e.g., AWS EKS clusters) And that's not intended to mean duplicating the Kubernetes plugin, necessarily. (though there's good overlap)
Eliminating the manual stitching for this would be powerful, and being able to throw away spreadsheets (which we don't want to maintain), wikis, or scripts to query against Terraform files or even just AWS CLI, since that will all lack the Backstage ownership level data.
- AWS
- AWS Accounts Ingestor (e.g., if we have an AWS org, being able to ingest all of the accounts attached to it)
- AWS EKS Ingestor
- GCP Projects Ingestor
etc...
from community.
@fcorti this could to beside the What Else on your diagram 😉
Yep, I expect this list of ingestors to be prioritised (once agreed) so that we can start from the "most requested" and then leave the community the space to implement one, few or many of them.
I personally like the idea of providing the "base" for the implementation of all the ingestors that the adopters will want and need. Ideally in some time only the "legacy" ones should remain as the ones to develop.
from community.
Wanted to potentially throw out the idea of a Cloud Ingestor. For example, we could find it very useful to ingest Kuberentes clusters as resources in the catalog, which we can then tie elements to. (e.g., AWS EKS clusters) And that's not intended to mean duplicating the Kubernetes plugin, necessarily. (though there's good overlap)
Eliminating the manual stitching for this would be powerful, and being able to throw away spreadsheets (which we don't want to maintain), wikis, or scripts to query against Terraform files or even just AWS CLI, since that will all lack the Backstage ownership level data.
AWS
- AWS Accounts Ingestor (e.g., if we have an AWS org, being able to ingest all of the accounts attached to it)
- AWS EKS Ingestor
GCP Projects Ingestor
etc...
We'd use this type of ingestor.
from community.
I'm at AWS and we've been seeing customers ingesting infrastructure in to the catalog as resources but each implementation is pretty ad-hoc. Has there been any developments on how we think this can be approached in a healthy way?
from community.
People use infrastructure in a wide range of ways, so it might be harder to have a single representation of that data.
For us at least, we don't want a single resource for each record, but for buckets of record. Right now we bucket based on type and an entityRef of the component that generated the infrastructure. This allows for a kind of "environment agnostic" view of the resources. We're looking to find ways to associate what components use the resource buckets in the case of shared infrastructure (like anything from a database or an s3 bucket or a ECS cluster) utilizing tracing data, but we haven't started that work yet.
from community.
Related Issues (20)
- Agenda for Backstage Community Sessions (August 23, 2023) HOT 2
- Org Member: dlochrie HOT 1
- Org Member: sankaritan HOT 1
- Adoption SIG: Meeting Notes
- Agenda for Backstage Community Sessions (Sep 27, 2023) HOT 3
- Org Member: jmezach HOT 2
- Backstage Adoption Use Cases Working Group
- Agenda for Backstage Community Sessions (November 22, 2023) HOT 2
- Improving visibility of SIGs HOT 4
- Agenda for Backstage Community Session (December 20, 2023) HOT 1
- Agenda for Backstage Community Session (December 20, 2023) HOT 3
- Agenda for Backstage Community Session (January 24, 2024) HOT 1
- Agenda for Backstage Community Session (Feburary 21st, 2024) HOT 4
- Org Member: jorgelainfiesta HOT 1
- Org Member: karlhaworth HOT 2
- Org Member: sennyeya HOT 1
- Agenda for Backstage Community Sessions (May 17, 2023) HOT 1
- Agenda for Backstage Community Sessions (June 28, 2023)
- Org Member: afscrome HOT 1
- Agenda for Backstage Community Sessions (July 26, 2023) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community.