meltano / .github Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 0.0 3.67 MB

.github's Introduction

The declarative code-first data integration engine

Say goodbye to writing, maintaining, and scaling your own API integrations.
Unlock 600+ APIs and DBs and realize your wildest data and ML-powered product ideas.

Libraries.io dependency status for latest release

Integrations

Meltano Hub is the single source of truth to find any Meltano plugins as well as Singer taps and targets. Users are also able to add more plugins to the Hub and have them immediately discoverable and usable within Meltano. The Hub is lovingly curated by Meltano and the wider Meltano community.

Installation

If you're ready to build your ideal data platform and start running data workflows across multiple tools, start by following the Installation guide to have Meltano up and running in your device.

Documentation

Check out the "Getting Started" guide or find the full documentation at https://docs.meltano.com.

Contributing

Meltano is a truly open-source project, built for and by its community. We happily welcome and encourage your contributions. Start by browsing through our issue tracker to add your ideas to the roadmap. If you're still unsure on what to contribute at the moment, you can always check out the list of open issues labeled as "Accepting Merge Requests".

For more information on how to contribute to Meltano, refer to our contribution guidelines.

Community

We host weekly online events where you can engage with us directly. Check out more information in our Community page.

If you have any questions, want sneak peeks of features or would just like to say hello and network, join our community of over +2,500 data professionals!

👋 Join us on Slack!

Responsible Disclosure Policy

Please refer to the responsible disclosure policy on our website.

License

This code is distributed under the MIT license, see the LICENSE file.

.github's People

Contributors

Watchers

.github's Issues

feat: auto-generate title-case, uppercase, and lowercase aliases

Context:

#1 (comment)

bug: label sync does not handle archived repositories

https://github.com/meltano/.github/actions/runs/4297661938/jobs/7490903913


Run ./.github/actions/sync_labels
[HttpError: Repository was archived so is read-only.] {
  statusCode: 403,
  headers: {
    server: 'GitHub.com',
    date: 'Tue, 28 Feb 2023 21:44:33 GMT',
    'content-type': 'application/json; charset=utf-8',
    'content-length': '137',
    'x-oauth-scopes': 'project, repo, write:org',
    'x-accepted-oauth-scopes': '',
    'x-github-media-type': 'github.v3; param=symmetra-preview; format=json',
    'x-github-api-version-selected': '2022-11-28',
    'x-ratelimit-limit': '5000',
    'x-ratelimit-remaining': '4939',
    'x-ratelimit-reset': '1677624103',
    'x-ratelimit-used': '61',
    'x-ratelimit-resource': 'core',
    'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
    'access-control-allow-origin': '*',
    'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
    'x-frame-options': 'deny',
    'x-content-type-options': 'nosniff',
    'x-xss-protection': '0',
    'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
    'content-security-policy': "default-src 'none'",
    vary: 'Accept-Encoding, Accept, X-Requested-With',
    'x-github-request-id': 'E40C:3827:9450226:130FDBB7:63FE75C1',
    connection: 'close'
  },
  body: {
    message: 'Repository was archived so is read-only.',
    documentation_url: 'https://docs.github.com/rest/reference/issues#create-a-label'
  },
  method: 'POST',
  endpoint: '/repos/meltano/target-postgres/labels'
}

guild-se: Label Strategy

Inspired by the labels specified as Meltano Area Labels in the Handbook, it would be helpful to come up with a set of issue categories to create as labels and apply to issues in the Singer Ecosystem.

The areas that come to mind are:

Configuration
Performance Improvement
REST/Graph/SQL/Soap (Stream Subclass Types)
Meltano Cloud
SDK Migration
SDK Feature Candidate (for features pioneered in Taps/Targets to inform a generic SDK implementation)

feat: Dry run label sync workflow when a PR updates label config

Use Pulumi to manage label syncing

The idea of using Pulumi to manage our GitHub repositories has come up several times before. Label syncing seems like a good first step to take with Pulumi.

We would use the following Pulumi package: https://www.pulumi.com/registry/packages/github/

Initially, as this issue proposed, it may only provide label syncing across all repositories in the meltano and MeltanoLabs GitHub orgs. Using Pulumi for this may be simpler than fixing the bugs that currently exist in the label syncer, while providing some other benefits (e.g. nice diffs).

Once we're got Pulumi set up for this, it would be easy to expand it to also enforce various configuration settings across the repositories within the meltano and MeltanoLabs GitHub orgs. For instance, branch protection, and permissions.

We'll have to be careful with this, since a bad deploy could affect all of our GitHub orgs/repositories. We should at least ensure all repository resources are considered protected so that Pulumi can't delete them.

CC @tayloramurphy @meltano/engineering

Manage Labels and Label Assignment using Groups

Until we are willing to invest in adopting Pulumi to manage GitHub labels (and more), it may be easier to further augment the label-sync action in this repo to support label groups and repo selection.

label-groups:
  core:
  - name: "label name"
    description: "label description"
    color: "0075CA" #hex color
    aliases:
    - "other labels to rename"
  singer-ecosystem:
  - name: "SDK Feature Candidate"
    description: "Applied to PR's and issues that may be precursors to SDK functionality."
    color: "0075CA" #hex color
    aliases:
    - "other labels to rename"

repo-labels:
- name: Core Labels
  label-groups:
  - core
  patterns:
  - "meltano/*"
  - "MeltanoLabs/*"
- name: Singer Ecosystem Labels
  label-groups:
  - singer-ecosystem
  patterns:
  - "meltano/sdk"
  - "MeltanoLabs/*"

bug: hitting rate limits on dry runs

https://github.com/meltano/.github/actions/runs/4298099799/jobs/7491835935

[HttpError: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.] {
  statusCode: 403,
  headers: {
    server: 'GitHub.com',
    date: 'Tue, 28 Feb 2023 22:47:58 GMT',
    'content-type': 'application/json; charset=utf-8',
    'transfer-encoding': 'chunked',
    'retry-after': '1',
    'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
    'access-control-allow-origin': '*',
    'x-github-media-type': 'github.v3; format=json',
    'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
    'x-frame-options': 'deny',
    'x-content-type-options': 'nosniff',
    'x-xss-protection': '0',
    'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
    'content-security-policy': "default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com objects-origin.githubusercontent.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events; font-src github.githubassets.com; form-action 'self' github.com gist.github.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com objects-origin.githubusercontent.com secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/assets-cdn/worker/ gist.github.com/assets-cdn/worker/",
    vary: 'Accept-Encoding, Accept, X-Requested-With',
    'x-github-request-id': 'F138:14C8:A20CA80:14D614EA:63FE849E',
    connection: 'close'
  },
  body: {
    documentation_url: 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits',
    message: 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'
  },
  method: 'POST',
  endpoint: '/repos/meltano/tap-stripe/labels'
}

bug: connection refused error

branch dry run PR gives this https://github.com/meltano/.github/actions/runs/4346140721/jobs/7591811220

Error: connect ECONNREFUSED 192.30.255.117:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '192.30.255.117',
  port: 443,
  method: 'POST',
  endpoint: '/repos/MeltanoLabs/meltano-map-transform/labels'
}