Giter Club home page Giter Club logo

charted's People

Contributors

auguwu avatar dependabot[bot] avatar iceemc avatar noelbot avatar renovate-bot avatar renovate[bot] avatar spotlightishere avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

cyberflamego

charted's Issues

[`charted_cache_worker`] Do we collect "pointers" of names -> ids?

For the REST API, we allow requesting by their snowflake or their Name, but in caching, a cache key can be created from an immutable value and can't be tampered with.

Do we keep a collection of "pointers" ("pointers" are a loose term; I'm essentially saying point an name to an ID) and using that to resolve IDs from user/repo/org names?

Gradle Dependency Management

This issue tracks how we can improve the dependency declaration since charted-server's subprojects are brought into smaller chunks of projects.

This should implement the versioning catalog, so we can declare libraries as lib.name.subname and it'll resolve to the Maven coordinates. Makes it way easier instead of writing the coordinates over and over again.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

cargo
Cargo.toml
  • argon2 0.5.3
  • axum-extra 0.9.3
  • axum-server 0.6.0
  • clap_complete 4.5.2
  • dotenvy 0.15.7
  • eyre 0.6.12
  • futures-util 0.3.30
  • hex 0.4.3
  • jsonwebtoken 9.3.0
  • ldap3 0.11.3
  • md-5 0.10.6
  • mimalloc 0.1.41
  • moka 0.12.7
  • num_cpus 1.16.0
  • opentelemetry_sdk 0.22.1
  • paste 1.0.14
  • prost 0.12.4
  • prost-types 0.12.4
  • rand 0.8.5
  • redis 0.25.3
  • sentry-eyre 0.1.0
  • sentry-tower 0.32.3
  • sentry-tracing 0.32.3
  • serde_repr 0.1.19
  • toml 0.8.12
  • tower 0.4.13
  • tower-http 0.5.2
  • tracing-error 0.2.0
  • tracing-opentelemetry 0.23.0
  • uuid 1.8.0
  • rustc_version 0.4.0
  • tonic-build 0.11.0
  • axum 0.7.5
  • async-trait 0.1.80
  • base64 0.22.1
  • bollard 0.16.1
  • chrono 0.4.38
  • clap 4.5.4
  • cli-table 0.4.7
  • clap_complete 4.5.2
  • color-eyre 0.6.3
  • dotenvy 0.15.7
  • erased-serde 0.4.4
  • eyre 0.6.12
  • mime 0.3.17
  • multer 3.0.0
  • once_cell 1.19.0
  • opentelemetry 0.22.0
  • owo-colors 4.0.0
  • prometheus-client 0.22.2
  • regex 1.10.4
  • remi 0.7.1
  • remi-azure 0.7.1
  • remi-fs 0.7.1
  • remi-s3 0.7.1
  • reqwest 0.12.4
  • semver 1.0.22
  • sentry 0.32.3
  • serde 1.0.200
  • serde_json 1.0.116
  • serde_yaml 0.9.34
  • sqlx 0.7.4
  • tempfile 3.10.1
  • testcontainers 0.16.6
  • tracing 0.1.40
  • tracing-subscriber 0.3.18
  • tokio 1.37.0
  • url 2.5.0
  • utoipa 4.2.0
  • validator 0.18.1
  • which 6.0.1
crates/charts/Cargo.toml
  • flate2 1.0.30
  • itertools 0.12.1
  • tar 0.4.40
crates/common/Cargo.toml
  • humantime 2.1.0
  • mac_address 1.1.6
  • rand 0.8.5
  • rustc_version 0.4.0
crates/config/Cargo.toml
  • azure_storage 0.20.0
  • aws-sdk-s3 1.25.0
  • toml 0.8.12
  • ubyte 0.10.4
crates/entities/Cargo.toml
crates/metrics/Cargo.toml
crates/search/Cargo.toml
crates/search/meilisearch/Cargo.toml
  • meilisearch-sdk 0.25.0
crates/server/Cargo.toml
  • paste 1.0.14
  • serde_path_to_error 0.1.16
  • serde_repr 0.1.19
crates/tatsuki/Cargo.toml
  • async-std 1.12.0
  • cron 0.12.1
  • log 0.4.21
  • pin-project-lite 0.2.14
  • tokio 1.37.0
  • tokio-util 0.7.10
  • tracing 0.1.40
features/gc/Cargo.toml
  • logos 0.14.0
  • sentry 0.32.3
  • tracing 0.1.40
proc-macros/Cargo.toml
  • heck 0.5.0
  • proc-macro2 1.0.81
  • quote 1.0.36
  • syn 2.0.60
  • utoipa 4.2.0
tools/devtools/Cargo.toml
  • bollard 0.16.1
  • itertools 0.12.1
  • tracing-subscriber 0.3.18
tools/helm-plugin/Cargo.toml
  • dirs 5.0.1
  • hcl-rs 0.16.9
  • inquire 0.7.5
  • is_ci 1.2.0
  • once_cell 1.19.0
  • similar 2.5.0
  • tracing 0.1.40
  • tracing-log 0.2.0
  • tracing-subscriber 0.3.18
  • url 2.5.0
docker-compose
.devcontainer/docker-compose.yml
  • bitnami/redis 7.2.4
  • bitnami/postgresql 15.6.0
docker-compose.yml
  • bitnami/postgresql 15.6.0
  • bitnami/redis 7.2.4
tools/devtools/docker-compose.yml
  • bitnami/redis 7.2.4
  • bitnami/postgresql 15.6.0
  • docker.elastic.co/elasticsearch/elasticsearch 8.13.3
  • getmeili/meilisearch v1.7
dockerfile
distribution/docker/alpine.Dockerfile
  • rust 1.78-alpine3.19
  • alpine 3.19
distribution/docker/debian.Dockerfile
  • rust 1.78-slim-bullseye
tools/helm-plugin/Dockerfile
github-actions
.github/actions/BuildBinary.yaml
  • actions/checkout v4
  • taiki-e/cache-cargo-install-action v2
  • rui314/setup-mold v1
  • Swatinem/rust-cache v2
  • actions/cache v4
  • Noelware/setup-protoc 1.2.0
.github/workflows/CI.yaml
  • actions/checkout v4
  • Swatinem/rust-cache v2
  • actions/cache v4
  • Noelware/setup-protoc 1.2.0
  • actions/checkout v4
  • EmbarkStudios/cargo-deny-action v1
  • actions/checkout v4
  • actions/checkout v4
  • cachix/install-nix-action v26
  • cachix/cachix-action v14
  • actions/checkout v4
  • Swatinem/rust-cache v2
  • Noelware/setup-protoc 1.2.0
  • auguwu/clippy-action 1.3.0
.github/workflows/OpenAPI.yaml
  • actions/checkout v4
  • Swatinem/rust-cache v2
  • oven-sh/setup-bun v1
  • Noelware/setup-protoc 1.2.0
  • EndBug/add-and-commit v9
.github/workflows/Release.yaml
  • actions/checkout v4
.github/workflows/ReleaseBeta.yaml
  • actions/checkout v4
  • softprops/action-gh-release v2
  • Noelware/s3-action 2.2.6
  • Noelware/s3-action 2.2.6
  • actions/checkout v4
  • softprops/action-gh-release v2
  • Noelware/s3-action 2.2.6
  • Noelware/s3-action 2.2.6
  • actions/checkout v4
  • cachix/install-nix-action v26
  • cachix/cachix-action v14
  • actions/checkout v4
  • docker/login-action v3
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • docker/build-push-action v5
  • actions/checkout v4
  • docker/login-action v3
  • docker/setup-qemu-action v3
  • docker/setup-buildx-action v3
  • docker/build-push-action v5
.github/workflows/ReleaseNightly.yaml
.github/workflows/UpdateProtos.yaml
  • actions/checkout v4
  • EndBug/add-and-commit v9

feat: add more configuration to Prometheus metrics

This issue tracks how we can manually configure the Prometheus exporter that can be enabled via metrics: true in the charted.yml file.

Since most of the metrics are configured via a Collector, which can be seen from the following sources:

Proposed Idea

The proposed idea is to make a MetricsConfig configuration type with the following:

metrics:
  redis:
    - charted_redis_total_net_input
    - charted_redis_total_net_output
    - charted_redis_total_commands_processed
    - [...]

Each component is represented as a key and it can be the following:

  • redis
  • postgresql
  • elasticsearch
  • meilisearch
  • elastic_apm
  • sentry
  • cassandra
  • ktor
  • jvm

Worker Queue without seperate services

I wanted to add a worker-queue system to do actions like index data in Elasticsearch when a user is created, and more.

The API server will act as a Kafka producer, in which, the server can spawn in worker queues to do separate actions it needs and it can interact with Koin, the dependency injection library the server uses to manage components like Cassandra, Redis, and such. But, how can the server spawn worker queues?

I could think it could be based off a @WorkerQueue annotation in a lib/workers-managed subproject with the following annotations:

annotation class WorkerQueue(
   val triggers: Array<Trigger> = [],
   val name: String
)
  • triggers are a list of triggers that the worker queue can check.
  • name is the name of the worker queue.

A worker queue can spawn more workers to do that specific worker's actions, like indexing data in Elasticsearch. We can setup a trigger that is linked to the worker queue (in the workers:{name}:triggers:{trigger} hash in Redis) to do certain triggers, a trigger can be setup with the @Trigger annotation:

annotation class Trigger(
   val arguments: Array<KClass<*>> = [],
   val name: String
)

The arguments array is the arguments (order does matter!) that the trigger(...args): Unit can run. The name is pretty self-explanatory.

Example with Elasticsearch

// create the worker
@WorkerQueue(name = "elasticsearch", triggers = [Trigger(name = "indexDataOnCreate", arguments = [User::class])])
class IndexElasticsearchWorker(private val elasticsearch: ElasticsearchService) {
    suspend fun indexDataOnCreate(user: User) {
       // do stuff here :>
    }
}

⚠️ !! THIS IS NOT DONE !! ⚠️

[Feature] Multiple authenticators for sessions

This will enable having multiple authenticators when authenticating with charted-server. At the moment of this issue, only local JWT sessions are working at the moment. This issue brings in support for:

  • LDAP (OpenLDAP, Active Directory)

Roadmap

  • Implement LDAP authenticator (OpenLDAP, Active Directory)

End to End + Integration Tests, Benchmarks

This issue exists to track the progress of adding e2e and integration tests, plus benchmarking with charted-server. Integration and end to end testing will allow us to track issues that might interfere with the lifecycle of charted-server's development and production lifecycle. Benchmarking can help how charted-server performs with heavy load with the components used.

Benchmarking

Benchmarking would be using JMH with the :benchmarking subproject. It should help benchmark:

  • REST APIs (users, repositories, organizations, api keys)
  • Search APIs (elastic/meilisearch)
  • Database Usage (postgres/cassandra)
    ...more?

Integration Tests

Integration tests already exist in charted-server (i.e, lib/cassandra tests) but it should test how it performs via the subproject.

Some integration tests might require the use of Docker, so we will need to exclude tests if the Docker engine doesn't exist on the user's system. At the moment, it'll error out since Docker isn't present.

End to End Testing

End to end testing is not present in the source code of charted-server, but it should help test the server and tools. This would also require Docker since the server expects us to have a Postgres and Redis installation running.

Validate NameOrSnowflake in `Path<NameOrSnowflake>` declarations

Since Names are user input, they should be validated:

/workspaces/Noelware/Products/charted/charted (feat/repo-endpoints*) » curl http://localhost:3651/users/@Me                                                             noel@floofbox
{"success":false,"errors":[{"code":"ENTITY_NOT_FOUND","message":"User with ID or name [@Me] was not found."}]}% 

Server Ratelimits

This will allow the server to configure ratelimits but allow configuration over convention. This goes in hand with charted-dev/charted#1050, which will be implemented after that is done.

Cache database objects

This seems like a good idea to reduce latency so that we don’t have to hit PostgreSQL each time, but a new configuration key would be present:

database.caching

Allows to configure the TTL for objects and which strategy (Redis, in-memory) to use.

Hyper v1

Since the release of hyper v1, most stuff is broken and we need to refractor the server to use the new v1 features.

Figure out what to do with `serde_yaml`

As of March 24th, 2024 (was also my 20th birthday), serde_yaml was deprecated and charted-server heavily relies on serde_yaml for the Helm-related operations it needs to do.

While it's not at the upmost priority, should we replace it with a different crate and experiment with it?

Single-user mode

This proposal serves as a point to provide a single-user/single-organization version of charted-server if wished, which will remove:

  • Repository and Organization members
  • Repository Releases
  • Session management (local, LDAP)
  • Session Integrations (GitHub, Noelware, etc)
  • Passwordless authentication

Roadmap

  • Introduce single-user configuration option
  • Create server/single-mode Gradle module (which will be the same thing as server but with features that are not enabled)

Configure and run Logback (not) automatically

Since migrating to Logback 1.4, there is issues with logging, i.e, not printing to stdout, so this should aim to run Logback BEFORE anything else (i.e, when hooks (uncaught thread exceptions/runtime hooks) are applied) in the Bootstrap.kt file.

And, it causes more issues when we want to add more appenders and such. So, this will deprecate the config/logback.properties configuration file and Logback will start programmatically using SPI.

Configuration

Since we have pieces that are configurable to Logback, we will have to load the configuration before the runtime/thread exception hooks, which will present a LogbackConfig data class:

Kotlin DSL

logback {
   // Enables the file appender for appending logs to files.
   enable<FileAppender>()

   // Enables Logstash for the Elastic Stack
   enable<LogstashAppender> {
       type = LogstashAppender.Connectors.TCP
       endpoint = "localhost:4040"
    }

    // Enables the appender for Sentry, which will use the DSN configured from `sentry_dsn` by default
    // you can add the DSN with the DSN parameter in `enable<SentryAppender>`
    enable<SentryAppender>()

    // Uses the JSON layout instead of the default console formatter
    json = true

    // The timezone for the time when printed to console
    timezone = "Etc/Utc".parseIntoTimezone()
}

YAML

logback:
  json: true

  # This will be overridden by the `TZ` environment variable, defaults to UTC.
  timezone: Etc/Utc
  appenders:
    - type: file
    - type: logstash
      options:
        type: TCP
        endpoint: localhost:4040

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

Renovate tried to run on this repository, but found these problems.

  • WARN: Package lookup failures

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • Update Rust crate md-5 to 0.10.6
  • Update Rust crate meilisearch-sdk to 0.24.2
  • Update Rust crate mimalloc to 0.1.39
  • Update Rust crate paste to 1.0.14
  • Update Rust crate proc-macro2 to 1.0.69
  • Update Rust crate prost to 0.12.1
  • Update Rust crate redis to 0.23.3
  • Update Rust crate reqwest to 0.11.22
  • Update Rust crate semver to 1.0.20
  • Update Rust crate sentry to 0.31.7
  • Update Rust crate sentry-tower to 0.31.7
  • Update Rust crate sentry-tracing to 0.31.7
  • Update Rust crate serde to 1.0.189
  • Update Rust crate serde_json to 1.0.107
  • Update Rust crate serde_yaml to 0.9.25
  • Update Rust crate sqlx to 0.7.2
  • Update Rust crate syn to 2.0.38
  • Update Rust crate sysinfo to 0.29.10
  • Update Rust crate tar to 0.4.40
  • Update Rust crate thiserror to 1.0.49
  • Update Rust crate tokio-util to 0.7.9
  • Update Rust crate tonic to 0.10.2
  • Update Rust crate tonic-build to 0.10.2
  • Update Rust crate tower-http to 0.4.4
  • Update Rust crate tracing to 0.1.39
  • Update Rust crate url to 2.4.1
  • Update Rust crate which to 4.4.2
  • Update dependency @tailwindcss/forms to v0.5.6
  • Update dependency @tailwindcss/typography to v0.5.10
  • Update dependency @vitejs/plugin-vue-jsx to v3.0.2
  • Update dependency @vue/test-utils to v2.4.1
  • Update dependency autoprefixer to v10.4.16
  • Update dependency markdown-it to v13.0.2
  • Update dependency markdown-it-shikiji to v0.6.10
  • Update dependency pinia to v2.1.7
  • Update dependency prettier to v3.0.3
  • Update dependency shikiji to v0.6.10
  • Update dependency unplugin-vue-components to v0.25.2
  • Update dependency vite-plugin-vue-devtools to v1.0.0-rc.5
  • Update dependency vue-eslint-parser to v9.3.2
  • Update dependency vue-router to v4.2.5
  • Update dependency vue-tsc to v1.8.19
  • Update dependency zod to v3.22.4
  • Update fontsource monorepo (@fontsource/cantarell, @fontsource/inter, @fontsource/jetbrains-mono)
  • Update postgresql-ha Docker tag to v11.9.8
  • Update Rust crate aws-sdk-s3 to 0.33.0
  • Update Rust crate bytes to 1.5.0
  • Update Rust crate clap to 4.4.6
  • Update Rust crate clap_complete to 4.4.3
  • Update Rust crate regex to 1.10.2
  • Update Rust crate tokio to 1.33.0
  • Update Rust crate toml to 0.8.2
  • Update Rust crate uuid to 1.5.0
  • Update bitnami/postgresql Docker tag to v15.4.0
  • Update bitnami/redis Docker tag to v7.2.2
  • Update dependency @formkit/vue to v1.2.2
  • Update dependency @formkit/zod to v1.2.2
  • Update dependency @tanstack/vue-query to v4.37.0
  • Update dependency @types/node to v20.8.7
  • Update dependency @vitejs/plugin-vue to v4.4.0
  • Update dependency @vueuse/core to v10.5.0
  • Update dependency @vueuse/head to v1.3.1
  • Update dependency aspect_rules_js to v1.33.0
  • Update dependency eslint to v8.51.0
  • Update dependency eslint-config-prettier to v8.10.0
  • Update dependency eslint-plugin-vue to v9.17.0
  • Update dependency luxon to v3.4.3 (luxon, @types/luxon)
  • Update dependency openapi-typescript to v6.7.0
  • Update dependency radix-vue to v0.4.1
  • Update dependency tsx to v3.14.0
  • Update dependency typescript to v5.2.2
  • Update dependency unplugin-vue-router to v0.7.0
  • Update dependency vite to v4.5.0
  • Update dependency vitest to v0.34.6
  • Update docker.elastic.co/elasticsearch/elasticsearch Docker tag to v8.10.4
  • Update getmeili/meilisearch Docker tag to v1.4
  • Update redis Docker tag to v18.1.5
  • Update typescript-eslint monorepo to v6.8.0 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
  • Update Rust crate jsonwebtoken to v9
  • Update Rust crate utoipa to v4
  • Update Rust crate which to v5
  • Update dependency @intlify/unplugin-vue-i18n to v1
  • Update dependency @tanstack/vue-query to v5
  • Update dependency @vueuse/head to v2
  • Update dependency eslint-config-prettier to v9
  • 🔐 Create all rate-limited PRs at once 🔐

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

  • Update Rust crate erased-serde to 0.3.31

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

  • Update Rust crate dyn-clone to 1.0.14

Warning

Renovate failed to look up the following dependencies: Failed to look up helm package emails, Failed to look up helm package server, Failed to look up helm package logstash, Failed to look up helm package petal, Failed to look up helm package redpanda.

Files affected: .noelware/deployment/main.tf


Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

bazel
build/tools/cc.bzl
  • rules_cc 0.0.9
build/tools/nixpkgs.bzl
build/tools/nodejs.bzl
  • aspect_rules_js v1.32.2
  • aspect_rules_ts v2.0.0
build/tools/pkg.bzl
  • rules_pkg 0.9.1
build/tools/protobuf.bzl
  • rules_proto 5.3.0-21.7
build/tools/rust.bzl
  • rules_rust 0.29.0
bazelisk
.bazelversion
  • bazel 6.3.2
cargo
cli/Cargo.toml
  • ansi_term 0.12.1
  • async-trait 0.1.70
  • chrono 0.4.26
  • clap 4.3.10
  • clap_complete 4.3.1
  • cli-table 0.4.7
  • color-eyre 0.6.2
  • dotenv 0.15.0
  • eyre 0.6.8
  • mimalloc 0.1.37
  • num_cpus 1.16.0
  • promptly 0.3.1
  • serde_json 1.0.104
  • serde_yaml 0.9.22
  • sqlx 0.7.1
  • sysinfo 0.29.4
  • tokio 1.29.1
  • tracing 0.1.37
  • tracing-subscriber 0.3.17
  • utoipa 3.5.0
crates/avatars/Cargo.toml
  • bytes 1.4.0
  • eyre 0.6.8
  • remi-core 0.4.0
  • reqwest 0.11.18
  • tracing 0.1.37
crates/common/Cargo.toml
  • argon2 0.5.1
  • async-trait 0.1.72
  • axum 0.6.19
  • chrono 0.4.26
  • eyre 0.6.8
  • hex 0.4.3
  • humantime 2.1.0
  • md-5 0.10.5
  • once_cell 1.18.0
  • paste 1.0.14
  • rand 0.8.5
  • regex 1.9.0
  • semver 1.0.17
  • serde 1.0.185
  • serde_json 1.0.104
  • sqlx 0.7.0
  • thiserror 1.0.41
  • tracing 0.1.37
  • utoipa 3.5.0
  • validator 0.16.1
  • chrono 0.4.26
  • rustc_version 0.4.0
  • which 4.4.2
crates/config/Cargo.toml
  • aws-sdk-s3 0.28.0
  • base64 0.21.2
  • clap 4.3.11
  • eyre 0.6.8
  • lazy_static 1.4.0
  • merge-struct 0.1.0
  • once_cell 1.18.0
  • paste 1.0.13
  • regex 1.9.1
  • remi-fs 0.4.0
  • remi-s3 0.4.0
  • serde 1.0.185
  • serde_yaml 0.9.22
  • tracing 0.1.37
  • thiserror 1.0.43
crates/database/Cargo.toml
  • async-trait 0.1.73
  • eyre 0.6.8
  • sentry 0.31.5
  • serde 1.0.186
  • sqlx 0.7.0
  • tracing 0.1.37
crates/emails/Cargo.toml
  • prost 0.12.0
  • tonic 0.10.0
  • tracing 0.1.37
  • tonic-build 0.10.0
  • which 4.4.2
crates/helm-charts/Cargo.toml
  • async-recursion 1.0.4
  • bytes 1.4.0
  • eyre 0.6.8
  • flate2 1.0.26
  • itertools 0.11.0
  • multer 2.1.0
  • once_cell 1.18.0
  • regex 1.9.1
  • remi-core 0.4.0
  • semver 1.0.18
  • sentry 0.31.5
  • serde_yaml 0.9.25
  • tar 0.4.39
  • tracing 0.1.37
  • remi-fs 0.4.0
  • tempfile 3.8.0
  • tokio 1.32.0
crates/logging/Cargo.toml
  • ansi_term 0.12.1
  • chrono 0.4.26
  • sentry-tracing 0.31.5
  • serde_json 1.0.100
  • tracing 0.1.37
  • tracing-log 0.1.3
  • tracing-subscriber 0.3.17
crates/metrics/Cargo.toml
  • dyn-clone 1.0.12
  • erased-serde 0.3.28
  • prometheus-client 0.21.2
  • serde 1.0.185
  • tokio 1.29.1
crates/openapi/Cargo.toml
  • once_cell 1.18.0
  • utoipa 3.5.0
crates/openapi/proc-macro/Cargo.toml
  • proc-macro2 1.0.67
  • quote 1.0.33
  • syn 2.0.37
  • utoipa 3.5.0
crates/proc-macros/Cargo.toml
  • axum 0.6.20
  • proc-macro2 1.0.67
  • quote 1.0.33
  • serde_json 1.0.104
  • syn 2.0.37
  • utoipa 3.5.0
crates/redis/Cargo.toml
  • async-recursion 1.0.4
  • eyre 0.6.8
  • redis 0.23.1
  • tokio 1.29.1
  • tracing 0.1.37
crates/search/Cargo.toml
  • async-trait 0.1.72
  • eyre 0.6.8
  • once_cell 1.18.0
  • serde 1.0.185
crates/search/elasticsearch/Cargo.toml
  • async-trait 0.1.72
  • elasticsearch 8.5.0-alpha.1
  • eyre 0.6.8
  • once_cell 1.18.0
  • serde 1.0.185
  • serde_json 1.0.104
  • tokio 1.29.1
  • tracing 0.1.37
  • url 2.4.0
crates/search/meilisearch/Cargo.toml
  • async-trait 0.1.73
  • eyre 0.6.8
  • meilisearch-sdk 0.24.1
  • serde 1.0.185
  • tracing 0.1.37
crates/sessions/Cargo.toml
  • async-trait 0.1.72
  • chrono 0.4.30
  • eyre 0.6.8
  • jsonwebtoken 8.3.0
  • redis 0.23.1
  • sentry 0.31.6
  • serde 1.0.185
  • serde_json 1.0.104
  • sqlx 0.7.1
  • tokio 1.29.1
  • tokio-util 0.7.8
  • tracing 0.1.37
  • utoipa 3.5.0
  • uuid 1.4.1
crates/sessions/integrations/Cargo.toml
  • async-trait 0.1.73
  • eyre 0.6.8
  • serde 1.0.188
crates/sessions/integrations/github/Cargo.toml
  • reqwest 0.11.20
  • sentry 0.31.7
  • serde 1.0.188
  • tracing 0.1.37
crates/sessions/ldap/Cargo.toml
  • eyre 0.6.8
  • ldap3 0.11.3
  • sentry 0.31.7
  • tokio 1.32.0
  • tracing 0.1.37
crates/sessions/local/Cargo.toml
  • argon2 0.5.1
  • async-trait 0.1.72
  • chrono 0.4.26
  • eyre 0.6.8
  • jsonwebtoken 8.3.0
  • once_cell 1.18.0
  • sentry 0.31.5
  • serde_json 1.0.104
  • sqlx 0.7.1
  • tracing 0.1.37
  • uuid 1.4.1
crates/sessions/passwordless/Cargo.toml
  • sentry 0.31.6
  • tracing 0.1.37
crates/storage/Cargo.toml
  • bytes 1.4.0
  • remi-core 0.4.0
  • remi-fs 0.4.0
  • remi-s3 0.4.0
  • serde 1.0.185
features/gc/Cargo.toml
  • logos 0.13.0
  • sentry 0.31.6
  • tracing 0.1.37
server/Cargo.toml
  • ansi_term 0.12.1
  • argon2 0.5.1
  • async-trait 0.1.71
  • axum 0.6.18
  • base64 0.21.2
  • chrono 0.4.24
  • erased-serde 0.3.31
  • eyre 0.6.8
  • futures-util 0.3.28
  • jsonwebtoken 8.3.0
  • mime 0.3.17
  • multer 2.1.0
  • once_cell 1.18.0
  • paste 1.0.14
  • prometheus-client 0.21.2
  • remi-core 0.4.0
  • rust-embed 8.0.0
  • semver 1.0.19
  • sentry 0.31.1
  • sentry-eyre 0.1.0
  • sentry-tower 0.31.1
  • sentry-tracing 0.31.1
  • serde 1.0.185
  • serde_json 1.0.96
  • serde_path_to_error 0.1.14
  • serde_yaml 0.9.21
  • sqlx 0.7.0
  • tokio 1.28.1
  • tower 0.4.13
  • tower-http 0.4.1
  • tower-layer 0.3.2
  • tracing 0.1.37
  • tracing-log 0.1.3
  • tracing-subscriber 0.3.17
  • utoipa 3.5.0
  • validator 0.16.1
server/proc-macro/Cargo.toml
  • heck 0.4.1
  • proc-macro2 1.0.67
  • quote 1.0.33
  • syn 2.0.37
  • utoipa 3.5.0
tools/devtools/Cargo.toml
  • async-trait 0.1.72
  • clap 4.3.19
  • clap_complete 4.3.2
  • color-eyre 0.6.2
  • eyre 0.6.8
  • itertools 0.11.0
  • simple_tables 0.3.0
  • serde 1.0.185
  • tokio 1.29.1
  • tracing 0.1.37
  • tracing-subscriber 0.3.17
  • which 4.4.0
  • promptly 0.3.1
  • toml 0.7.6
  • serde_starlark 0.1.14
tools/helm-plugin/Cargo.toml
  • ansi_term 0.12.1
  • async-trait 0.1.70
  • chrono 0.4.26
  • clap 4.3.10
  • clap_complete 4.3.1
  • color-eyre 0.6.2
  • dirs 5.0.1
  • dotenv 0.15.0
  • eyre 0.6.8
  • reqwest 0.11.18
  • serde 1.0.185
  • serde_json 1.0.104
  • serde_yaml 0.9.25
  • sysinfo 0.29.7
  • tokio 1.29.1
  • tracing-subscriber 0.3.17
  • tracing 0.1.37
  • which 4.4.0
  • url 2.4.0
  • once_cell 1.18.0
docker-compose
.devcontainer/docker-compose.yml
  • bitnami/redis 7.2.1
  • bitnami/postgresql 15.4.0
docker-compose.yml
  • bitnami/postgresql 15.4.0
  • bitnami/redis 7.2.1
tools/devtools/docker-compose.yml
  • bitnami/redis 7.0.11
  • bitnami/postgresql 15.3.0
  • docker.elastic.co/elasticsearch/elasticsearch 8.9.0
  • getmeili/meilisearch v1.2
dockerfile
distribution/docker/alpine.Dockerfile
  • alpine 3.18
distribution/docker/debian.Dockerfile
tools/helm-plugin/distribution/docker/Dockerfile
github-actions
.github/workflows/CI.yaml
  • actions/checkout v3
  • actions/cache v3
  • actions/checkout v3
  • Swatinem/rust-cache v2
  • auguwu/clippy-action 1.3.0
.github/workflows/OpenAPI.yaml
  • actions/checkout v4
  • actions/setup-node v3
  • actions/cache v3
  • EndBug/add-and-commit v9
.github/workflows/Release.yaml
.github/workflows/ReleaseBeta.yaml
.github/workflows/ReleaseNightly.yaml
.github/workflows/UpdateProtos.yaml
  • actions/checkout v4
  • EndBug/add-and-commit v9
helm-values
distribution/helm/values.yaml
helmv3
distribution/helm/Chart.yaml
  • common ~2.13.0
  • postgresql ~13.1.0
  • redis ~18.1.0
npm
types/js/package.json
  • @augu/eslint-config 4.0.1
  • @augu/tsconfig 1.1.1
  • @noelware/utils 2.4.0
  • @types/node 20.7.0
  • @typescript-eslint/eslint-plugin 6.8.0
  • @typescript-eslint/parser 6.8.0
  • eslint 8.50.0
  • eslint-config-prettier 9.0.0
  • eslint-formatter-codeframe 7.32.1
  • openapi-typescript 6.6.2
  • prettier 3.0.3
  • tsx 3.13.0
  • typescript 5.2.2
web/package.json
  • @fontsource/cantarell 5.0.5
  • @fontsource/inter 5.0.5
  • @fontsource/jetbrains-mono 5.0.5
  • @formkit/vue 1.1.0
  • @formkit/zod 1.1.0
  • @iconify/vue 4.1.1
  • @ncharts/types 0.1.0-unstable
  • @noelware/utils 2.4.0
  • @tailwindcss/forms 0.5.4
  • @tailwindcss/typography 0.5.9
  • @tanstack/vue-query 4.29.25
  • @vue/compiler-sfc 3.3.4
  • @vueuse/core 10.2.1
  • @vueuse/head 1.1.26
  • autoprefixer 10.4.14
  • luxon 3.3.0
  • markdown-it 13.0.1
  • markdown-it-emoji 2.0.2
  • markdown-it-shikiji 0.6.6
  • ofetch 1.3.3
  • pinia 2.1.4
  • postcss 8.4.31
  • postcss-import 15.1.0
  • radix-vue 0.2.3
  • shikiji 0.6.6
  • tailwindcss 3.3.3
  • vue 3.3.4
  • vue-router 4.2.4
  • zod 3.22.3
  • @augu/eslint-config 4.0.1
  • @augu/tsconfig 1.1.1
  • @intlify/unplugin-vue-i18n 0.13.0
  • @types/luxon 3.3.0
  • @types/node 20.4.2
  • @typescript-eslint/eslint-plugin 6.1.0
  • @typescript-eslint/parser 6.1.0
  • @vitejs/plugin-vue 4.2.3
  • @vitejs/plugin-vue-jsx 3.0.1
  • @vue/test-utils 2.4.0
  • eslint 8.45.0
  • eslint-config-prettier 8.8.0
  • eslint-formatter-codeframe 7.32.1
  • eslint-plugin-vue 9.15.1
  • jsdom 22.1.0
  • prettier 3.0.0
  • typescript 5.1.6
  • unplugin-auto-import 0.16.6
  • unplugin-fluent-vue 1.1.4
  • unplugin-vue-components 0.25.1
  • unplugin-vue-router 0.6.4
  • vite 4.4.4
  • vite-plugin-vue-devtools 1.0.0-alpha.2
  • vite-plugin-vue-layouts 0.8.0
  • vitest 0.33.0
  • vue-eslint-parser 9.3.1
  • vue-tsc 1.8.5
terraform
.noelware/deployment/main.tf
  • emails 0.2.0-beta
  • server 0.1.0-beta
  • logstash 8.9.1
  • petal 0.1.0-beta
  • postgresql-ha 11.9.2
  • redis 18.0.1
  • redpanda 23.2.8
  • helm 2.11.0

  • Check this box to trigger a request for Renovate to run again on this repository

[Issue] 400 on Put /v2/<name>/<chart>/manifests/<version>

This bug might encounter frequently when you use the OCI registry feature within charted-server. If anyone can try to debug and fix, submit a PR! :3

Reproduce

$ helm create <my-chart>
$ helm package <my-chart>
$ helm push <my-chart> oci://<charted-server>:<port>
# failed commit on ref "manifest-sha256:<blob>"
$ helm push <my-chart> oci://<charted-server>:<port>
# Pushed: <charted-server>:<port>/<name>/<my-chart>:<version>
# Digest: sha256:4130b573cec900b12215a5c165a0ead33e3f2faf305fcf2e4fe6b306bb305c92

Allow uppercase letters in `Name`s.

Currently, the Name struct doesn't allow uppercase letters:

/workspaces/Noelware/Products/charted/charted (feat/repo-endpoints*) » curl http://localhost:3651/users/@Me                                                             noel@floofbox
{
  "errors": [
    {
      "code": "INVALID_ID_OR_NAME",
      "message": "invalid name (@Me): invalid input [@Me]: character '@' in index 0 was not a valid character"
    }
  ],
  "success": false
}

feat: docker registry v2 features

I just learned this last night but I wanted to discuss how and why we need to implement the Docker Registry v2 Specification into charted-server

Known Limitations

  • helm push requires the scheme to be oci://, so it is required to do oci://<charted-server-api>/repos/:id
  • helm push adds /v2 as if we're using a Docker registry rather than charted-server itself and charted-server is only in API version v1 (as of this issue is created).

Why?

I just learned out that helm registry exists and uses the Docker Registry v2 specification for pushing charts, which are tightly packed into a .tar.gz file (used when doing helm package).

How?

One of the things we can do is closely look at the specification in the documentation and check out the registry code on GitHub and checking what endpoints for push/pull requests (charted-server logs requests by default (might be a config option to configure logging (i.e, --log.level / --log.colors / --log.formatter=<json|default>)))

Redesign modules that require generics

The design for some modules (like charted_search) would want a trait with methods that accept generic types, but generic types aren't object-safe, so what do we do to keep it easy and maintainable?

use charted_search::SearchService;

// some options to use to modify the request body
pub struct Options {}

// a service that extends `SearchService`
pub struct MyService {}

impl SearchService for MyService {
    type Options = Options;

    fn index() { /* do some indexing */ }
    fn search() { /* do some search */}
}

Even though hacks do exist where we would have to use an enum to differentiate without getting the dreaded we cannot turn this object-safe as we would want to use generics for:

  • Index Name: Into<String>
  • Index Type: serde::DeserializedOwned/serde::Serialize

I'm wondering if there is any way we could do this where we can keep a dyn SearchService without doing more enum hacks and making it harder to maintain multiple objects that implement a trait that require dynamic dispatch.

I was thinking of doing a "hacky vtable" but I'm not too sure if we can do:

struct Vtable {
  do_index: unsafe fn(a: impl Trait<Of, Some, Sorts>) -> /* result */,
}

and calling it since Rust treats fn as a function pointer rather than a type (like Fn, which function pointers do implement!) and only generate what implementation of the Trait<T, U, V> is used, and it is a design chosen by the Rust language, and I think we need to think for now on how to keep this maintainable without dreading hacky solutions.

Using enums and generating an implementation of that is still too hacky for me since it's more code that is only there to patch existing designs by Rust.

Integration Testing Framework

We would like our tests to expand on the API server, but there is some issues:

  • We need a running Postgres and Redis instances with Docker
  • Allow Elasticsearch and Meilisearch as optional services (if we need them to test full-text search)

How will this be implemented?

A custom HCL-based format for tests will be allowed to be executed on the server test runtime, which will look something like:

test "expect that x = y" {
  http {
    get = "/"
  }

  # enable elasticsearch to be ran for this test
  services = ["elasticsearch"]

  assert {
    "statusCode" = 200
    "headers[Content-Type]" = "application/json; charset=utf-8"
  }
}

Roadmap

  • Implement HCL-based test runtime
  • Implement test context for HCL-based test runtime

Task Scheduler

At the moment, we do not do background task processing for components that might want to run jobs in the background. So, this is a blueprint of how we can do this.

We will start with a WorkerThreadPool, which will inherit a pool of N threads workers (can be configured with CHARTED_JAVA_OPTS="-Porg.noelware.charted.modules.tasks.maxThreadWorkerPoolSize=32" environment variable), N being from the org.noelware.charted.modules.tasks.maxThreadWorkerPoolSize system property or how many CPU processors exist on the current host.

private const val MAX_THREAD_WORKER_POOL_SIZE: String = "org.noelware.charted.modules.tasks.maxThreadWorkerPoolSize"

class WorkerThreadPool(threads: Int = System.getProperty(MAX_THREAD_WORKER_POOL_SIZE)?.ifNotNull { this.toInt() } ?: Runtime.getRuntime().availableProcessors()) {
   private val channel = Channel()
   private val jobs = mutableListOf<Worker>()
}

A WorkerThreadPool will launch all the jobs from the org.noelware.charted.modules.tasks.Worker interface as a service loader (so we do not have to programmatically queue it up with WorkerThreadPool.queueWorker)

Search Index Worker

A worker for indexing Elasticsearch or Meilisearch data is probably the most needed to update data every hour, just in case if network connections dropped and any entity hasn't been inserted or updated into the search module.

feat: tracing via OpenTelemetry/Elastic APM

This issue is the tracker for enabling tracing with charted's components (Docker Registry Proxy, Helm Charts Engine, Ktor), which will natively support:

Tasks

  • Add lib/tracing, lib/tracing/apm, and lib/tracing/opentelemetry subprojects
  • Finish and tested Elastic APM tracing
  • Finish and tested OpenTelemetry tracing

[Feature] Garbage Collection

As you use charted-server as your primary Helm Chart registry, there might be repositories, users, or organizations that are inactive and want to be pruned by some constraints (i.e, delete all users that have been last signed in (via POST /users/login) 30 days ago).

As there is no real query language that is extendable, charted-server will rely on a custom SQL-like language called Isuki, or IQL for short.

Isuki (IQL) is Noelware's SQL querying language that is also being built with Noelware Analytics to provide a human and SQL-like language, so:

  • LastPublished >= 30d - Queries the Isuki object with the property LastPublished that is greater or equal than 30 days.

This will be configured with the GarbageCollectionConfig DSL class, which an example in YAML:

gc:
  inactive-repos:
     $object: Repository
     constraint: last_published >= 30d
     actions: [delete]
  inactive-users:
    $object: User
    constraint: last_logged_in >= 30d
    actions:
    - deactivate
    - email:deactivation
  registry-layers:
    $object: Features/DockerRegistry
    constraint: image_size >= 1.5gb
    actions: [delete]
    schedule:
      cron: "* 0 0 * *"

Kotlin Script:

gc {
  action<Repo>("inactive-repos") {
     constraint {
        lastPublished >= "30d".toTimeSpan()
     }

     actions = listOf(GCActions.Delete)
  }

  action<User>("inactive-users") {
    constraint {
      lastLoggedIn >= "30d".toTimeSpan()
    }

    actions = listOf(GCActions.Deactivate, GCActions.Email("deactivate"))
  }

  action<DockerRegistry>("registry-layers") {
    constraint { imageSize >= "1.5gb".toByteSizeValue() }
    actions = listOf(GCActions.Delete)
    schedule {
      cron("* 0 0 * *")
    }
  }
}

The feature is only enabled if gc is not null, so if it is not present or if gc: null was specified, then it will not be enabled.

Note: This feature won't be added until Isuki has been developed and we have a refine structure that most people are happy with.

Roadmap

  • Wait until JNI bindings for Isuki are created
  • Implement garbage-collector module

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: The following managers configured in enabledManagers are not supported: "docker, helm, js, node, rust"

Unable to build via Bazel on Darwin arm64

This was mentioned elsewhere, but using Bazel in any shape/form under Darwin on arm64 results in a failure to discover the local toolchain:

$ ./dev sync-deps
===> Syncing dependencies...
===> $ bazel run //thirdparty:crate_index -- --repin
ERROR: /private/var/tmp/_bazel_spot/c3971260ce50c880b6c55138442b37f7/external/local_config_cc/BUILD:48:19: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'darwin_arm64'
ERROR: /private/var/tmp/_bazel_spot/c3971260ce50c880b6c55138442b37f7/external/local_config_cc/BUILD:48:19: Analysis of target '@local_config_cc//:toolchain' failed
ERROR: Analysis of target '//thirdparty:crate_index' failed; build aborted:
INFO: Elapsed time: 0.086s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
ERROR: Build failed. Not running target

This affects both native and provided-via-Nix Bazel, who should be using clang as provided via nixpkgs. Planning to debug this further at a later date, but filing this issue as a reminder for now.

Finalize Logstash pipeline

Since Logstash is the core of Noelware's Observability pipeline, we need to make sure that charted-server actually sends logs to Logstash.

Even though that it should send via TCP requests from Logstash's tcp input plugin, that hasn't been implemented yet.

Finalize all deployments

Roadmap

  • Finalize Docker deployment (cr.noelware.cloud/charted/server) ~ Finished in commit ed0b4eed
  • Finalize Helm deployment (charts.noelware.org/~/charted/server) Development has moved to charted-dev/helm-charts
  • Finalize Debian deployment (to Noelware's Artifacts Registry) Not needed (yet)
  • Finalize RPM deployment (to Noelware's Artifacts Registry) Not needed (yet)
  • Finalize Tar/Zip deployments (to Noelware's Artifacts Registry) It'll be a flat binary to install

Abstract API responses

Currently, in the source code for API responses, we just use the generic buildJsonObject function from kotlinx.serialization which can be tedious to write out, why not implement a serializer?

At the time, I didn't really dive into the documentation for kotlinx.serialization, so it's safe to say we can implement our own serializer for it: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/json.md#under-the-hood-experimental

Since we send out responses like:

// successful (200..204)
{
   "success": true,
   "data": // data here
}

// unsuccessful
{
   "success": false,
   "errors": [/* error objects */]
}

So, we can build out a serializer for the following types:

@Serializable
sealed class Response<out T>(val success: Boolean) {
    data class Success<out T>(override val success: Boolean, val data: T): Response<T>(true)
    data class Exception(val errors: List<APIError>): Response<Nothing>(false)
}

// functions to aid development
fun <T> ok(data: T): Response.Success<T> = Response.Success(data)
fun err(errors: List<APIError>): Response.Exception = Response.Exception(errors)
fun err(error: APIError): Response.Exception = err(listOf(error))
fun <E: Throwable> err(throwable: E): Response.Exception = err(throwable.toAPIError())

Server tests

This issue aims to fix the issue that is brought up in most cases in all server tests

SessionsPluginsTest > generate fake user, bail if invalid password, accept if valid password STANDARD_ERROR
    [Test worker] INFO org.noelware.charted.common.ReflectionUtils - Finding field [_database] in class [Builder]!
    [Test worker] INFO org.noelware.charted.common.ReflectionUtils - Finding field [_redis] in class [Builder]!
    [Test worker @coroutine#53] INFO org.noelware.charted.server.testing.TestBootstrapPhase - Initializing bootstrapping phase for tests...
    [Test worker @coroutine#53] INFO com.zaxxer.hikari.HikariDataSource - Postgres-HikariPool - Starting...
    [Test worker @coroutine#53] INFO com.zaxxer.hikari.pool.HikariPool - Postgres-HikariPool - Added connection org.postgresql.jdbc.PgConnection@52bb3dcd
    [Test worker @coroutine#53] INFO com.zaxxer.hikari.HikariDataSource - Postgres-HikariPool - Start completed.
    [Test worker @coroutine#53] INFO org.noelware.charted.server.testing.TestBootstrapPhase - Connected to Postgres test container, running all migrations!
    [Test worker @coroutine#53] INFO Exposed - Preparing create tables statements took 9ms
    [Test worker @coroutine#53] INFO Exposed - Executing create tables statements took 27ms
    [Test worker @coroutine#53] INFO Exposed - Extracting table columns took 8ms
    [Test worker @coroutine#53] INFO Exposed - Extracting column constraints took 82ms
    [Test worker @coroutine#53] INFO Exposed - Preparing alter table statements took 92ms
    [Test worker @coroutine#53] INFO Exposed - Executing alter table statements took 34ms
    [Test worker @coroutine#53] INFO Exposed - Checking mapping consistence took 40ms
    [Test worker @coroutine#53] INFO org.noelware.charted.server.testing.TestBootstrapPhase - Ran all pending migrations! Connecting to Redis...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.redis.DefaultRedisClient - Connecting to the Redis server...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.redis.DefaultRedisClient - Checking connection...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.redis.DefaultRedisClient - Connected to Redis in [3ms]
    [Test worker @coroutine#53] INFO org.noelware.charted.server.testing.TestBootstrapPhase - Connected to Redis! Initializing storage driver...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.storage.DefaultStorageHandler - Determining which storage service to use!
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.storage.DefaultStorageHandler - Configured to use storage trailer remi:filesystem
    [Test worker @coroutine#53] INFO org.noelware.remi.support.filesystem.FilesystemStorageService - Initialized filesystem storage service on directory [./.data] with drive [/dev/sdb2 (fuseblk)]
    [Test worker @coroutine#53] INFO org.noelware.charted.server.testing.TestBootstrapPhase - Initialized storage driver, now starting up Koin...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.apikeys.DefaultApiKeyManager - Collecting all API key expirations from Redis...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.apikeys.DefaultApiKeyManager - Took 637098µs to collect 0 sessions from Redis!
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.sessions.SessionManager - Collecting all sessions from Redis...
    [Test worker @coroutine#53] INFO org.noelware.charted.modules.sessions.SessionManager - Took 367411µs to collect 0 sessions from Redis!
    [Test worker @coroutine#53] WARN org.koin.core.Koin - (+) override index 'org.noelware.ktor.endpoints.AbstractEndpoint::_root_' -> '[Singleton:'org.noelware.charted.server.endpoints.v1.api.repositories.RepositoriesEndpoints',binds:org.noelware.ktor.endpoints.AbstractEndpoint]'
    [Test worker @coroutine#53] WARN org.koin.core.Koin - (+) override index 'org.noelware.ktor.endpoints.AbstractEndpoint::_root_' -> '[Singleton:'org.noelware.charted.server.endpoints.v1.api.admin.AdminEndpoints',binds:org.noelware.ktor.endpoints.AbstractEndpoint]'
    [Test worker @coroutine#53] WARN org.koin.core.Koin - (+) override index 'org.noelware.ktor.endpoints.AbstractEndpoint::_root_' -> '[Singleton:'org.noelware.charted.server.endpoints.v1.api.OrganizationsEndpoint',binds:org.noelware.ktor.endpoints.AbstractEndpoint]'
    [Test worker @coroutine#53] WARN org.koin.core.Koin - (+) override index 'org.noelware.ktor.endpoints.AbstractEndpoint::_root_' -> '[Singleton:'org.noelware.charted.server.endpoints.v1.InfoEndpoint',binds:org.noelware.ktor.endpoints.AbstractEndpoint]'
    [Test worker @coroutine#53] INFO org.koin.core.Koin - loaded 28 definitions in 0.241333 ms
    [Test worker] INFO org.noelware.charted.server.testing.TestChartedServer - Starting test server!
    [Test worker @coroutine#56] INFO org.noelware.charted.modules.helm.charts.DefaultHelmChartModule - Creating missing directories if any...
    [Test worker @coroutine#56] INFO org.noelware.charted.modules.helm.charts.DefaultHelmChartModule - Creating index.yaml for user [21242647142203392]
    [DefaultDispatcher-worker-3 @call-context#57] INFO ktor.test - No ktor.deployment.watch patterns specified, automatic reload is not active.
    [DefaultDispatcher-worker-3 @call-context#57] INFO org.noelware.ktor.plugin.NoelKtorRouting - Installing NoelKtorRouting into the base application!
    [DefaultDispatcher-worker-3 @call-context#57] ERROR org.koin.core.Koin - * No instance found for org.noelware.charted.modules.elasticsearch.ElasticsearchModule on scope ['_root_']
    [DefaultDispatcher-worker-3 @call-context#57] INFO org.noelware.ktor.plugin.NoelKtorRouting - Now registering 10 endpoints to Ktor...
    [DefaultDispatcher-worker-3 @call-context#57] INFO ktor.test - Application started in 0.008 seconds.
    [DefaultDispatcher-worker-3 @call-context#57] INFO org.noelware.charted.server.plugins.LogPluginKt - API server has started [30s]
    [Charted-CoroutineExecutor[0] @coroutine#61] WARN Exposed - Transaction attempt #1 failed: java.sql.SQLException: HikariDataSource HikariDataSource (Postgres-HikariPool) has been closed.. Statement(s): SELECT users.id, users.gravatar_email, users.description, users.avatar_hash, users.created_at, users.updated_at, users.username, users."password", users.flags, users.email, users."name" FROM users WHERE users.username = ?
    org.jetbrains.exposed.exceptions.ExposedSQLException: java.sql.SQLException: HikariDataSource HikariDataSource (Postgres-HikariPool) has been closed.
    SQL: [SELECT users.id, users.gravatar_email, users.description, users.avatar_hash, users.created_at, users.updated_at, users.username, users."password", users.flags, users.email, users."name" FROM users WHERE users.username = ?]
        at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:49)
        at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:141)
        at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:127)
        at org.jetbrains.exposed.sql.AbstractQuery.iterator(AbstractQuery.kt:61)
        at org.jetbrains.exposed.sql.IterableExKt$mapLazy$1.iterator(IterableEx.kt:136)
        at kotlin.collections.CollectionsKt___CollectionsKt.firstOrNull(_Collections.kt:269)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invokeSuspend(SessionsPlugin.kt:323)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invoke(SessionsPlugin.kt)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invoke(SessionsPlugin.kt)
        at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt$suspendedTransactionAsyncInternal$1.invokeSuspend(Suspended.kt:129)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
    Caused by: java.sql.SQLException: HikariDataSource HikariDataSource (Postgres-HikariPool) has been closed.
        at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:96)
        at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:141)
        at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:138)
        at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:126)
        at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:127)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:76)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:75)
        at kotlin.UnsafeLazyImpl.getValue(Lazy.kt:81)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction.getConnection(ThreadLocalTransactionManager.kt:89)
        at org.jetbrains.exposed.sql.Transaction.getConnection(Transaction.kt)
        at org.jetbrains.exposed.sql.statements.Statement.prepared(Statement.kt:24)
        at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:47)
        ... 14 more
    [DefaultDispatcher-worker-3 @request#57] ERROR org.noelware.charted.server.internal.DefaultChartedServer - Unknown exception had occurred while handling request [GET /users/@me]
    org.jetbrains.exposed.exceptions.ExposedSQLException: java.sql.SQLException: HikariDataSource HikariDataSource (Postgres-HikariPool) has been closed.
    SQL: [Failed on expanding args for SELECT: org.jetbrains.exposed.sql.Query@6ef00534]
        at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:49)
        at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:141)
        at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:127)
        at org.jetbrains.exposed.sql.AbstractQuery.iterator(AbstractQuery.kt:61)
        at org.jetbrains.exposed.sql.IterableExKt$mapLazy$1.iterator(IterableEx.kt:136)
        at kotlin.collections.CollectionsKt___CollectionsKt.firstOrNull(_Collections.kt:269)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invokeSuspend(SessionsPlugin.kt:323)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invoke(SessionsPlugin.kt)
        at org.noelware.charted.server.plugins.SessionsPlugin$doBasicAuth$user$1.invoke(SessionsPlugin.kt)
        at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt$suspendedTransactionAsyncInternal$1.invokeSuspend(Suspended.kt:129)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
    Caused by: java.sql.SQLException: HikariDataSource HikariDataSource (Postgres-HikariPool) has been closed.
        at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:96)
        at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:141)
        at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:138)
        at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:126)
        at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:127)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:76)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:75)
        at kotlin.UnsafeLazyImpl.getValue(Lazy.kt:81)
        at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction.getConnection(ThreadLocalTransactionManager.kt:89)
        at org.jetbrains.exposed.sql.Transaction.getConnection(Transaction.kt)
        at org.jetbrains.exposed.sql.statements.Statement.prepared(Statement.kt:24)
        at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:47)
        ... 14 more
    [DefaultDispatcher-worker-3 @request#57] INFO org.noelware.charted.server.plugins.LogPluginKt - GET HTTP/1.1 /users/@me :: 500 Internal Server Error [Ktor client] [21ms]

SessionsPluginsTest > generate fake user, bail if invalid password, accept if valid password FAILED
    org.opentest4j.AssertionFailedError: expected: <500 Internal Server Error> but was: <401 Unauthorized>
        at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
        at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
        at app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
        at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
        at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1153)
        at app//kotlin.test.junit5.JUnit5Asserter.assertEquals(JUnitSupport.kt:32)
        at app//kotlin.test.AssertionsKt__AssertionsKt.assertEquals(Assertions.kt:63)
        at app//kotlin.test.AssertionsKt.assertEquals(Unknown Source)
        at app//kotlin.test.AssertionsKt__AssertionsKt.assertEquals$default(Assertions.kt:62)
        at app//kotlin.test.AssertionsKt.assertEquals$default(Unknown Source)
        at app//org.noelware.charted.server.testing.plugins.SessionsPluginsTest$test4$1.invokeSuspend(SessionsPluginsTest.kt:131)
        at app//kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at app//kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at app//kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
        at app//kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at app//kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at app//kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at app//kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at app//kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at app//io.ktor.server.testing.TestApplicationKt.testApplication(TestApplication.kt:288)
        at app//org.noelware.charted.server.testing.TestChartedServer.start(TestChartedServer.kt:116)
        at app//org.noelware.charted.server.testing.AbstractChartedServerTest.withChartedServer(AbstractChartedServerTest.kt:116)
        at app//org.noelware.charted.server.testing.plugins.SessionsPluginsTest.test4(SessionsPluginsTest.kt:119)

feat: add gpg signing for publishing

This feature is the ability to implement GPG signing when parcel publish is executed; if the repository enables "Force GPG Signing" to force publishers under the Repository settings to use GPG signing when publishing. :)

v0.1.0-beta Release

  • Finalize sessions crate with LDAP and GitHub integration for logging in (#554) | LDAP support is simple and can break at any time, GitHub integration will come in a later release.
  • Finalize database crate with caching (#1050)
  • Get Docker and Helm deployments working (#1098) | Docker deployment is finished, Helm charts are being moved to charted-dev/helm-charts
  • Finalize Noelware infra for charted (in .noelware/deployment) This will be moved into its own repository
  • Finalize Logstash logging pipeline (as Logstash is a core part of Noelware's Observability pipeline) (#1097)
  • Server Ratelimits (#1083) | Implemented in a later release
  • Allow uppercase letters in Names (#1216)
  • Validate Names in Path<NameOrSnowflake> usages (#1217)

Endpoints

API Keys

  • GET /apikeys/{idOrName}
  • DELETE /apikeys/{id}
  • PATCH /apikeys/{id}
  • PUT /apikeys
  • GET /apikeys

Organizations

  • GET /organizations/{idOrName}/icon/{hash}
  • GET /organizations/{idOrName}/repositories
  • PUT /organizations/{idOrName}/repositories
  • POST /organizations/{idOrName}/icon
  • GET /organizations/{idOrName}
  • DELETE /organizations
  • PATCH /organizations
  • PUT /organizations
  • GET /organizations

Repositories

  • GET /repositories/{id}/releases/{version}/tarball
  • GET /repositories/{id}/releases/{version}/provenance
  • DELETE /repositories/{id}/releases/{version}
  • PUT /repositories/{id}/releases/{version}
  • GET /repositories/{id}/releases/{version}
  • PUT /repositories/{id}/releases
  • GET /repositories/{id}/releases
  • DELETE /repositories/{idOrName}
  • GET /repositories/{id}
  • PATCH /repositories
  • GET /repositories

Users

  • GET /users/{idOrName}/repositories
  • GET /users/{idOrName}/avatar/{hash}
  • GET /users/{idOrName}/avatar
  • GET /users/@me/avatar/{hash}
  • PUT /users/@me/repositories
  • GET /users/{idOrName}
  • POST /users/@me/avatar
  • GET /users/@me/avatar
  • GET /users/@me
  • DELETE /users
  • PATCH /users
  • PUT /users
  • GET /users

Main

  • GET /indexes/{idOrName}
  • GET /openapi.yaml
  • GET /openapi.json
  • GET /cdn/{...path}
  • GET /features
  • GET /metrics
  • GET /info
  • GET /

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.