hermesgermany / galapagos Goto Github PK

Some of the texts on the "Applications" page are missing their german translation. This should be fixed.

The dialogue, which gets opend when deleting a Topic, is not correctly tranlated and should be fixed.

Similar to developer certificates, it should also be possible to obtain developer API keys for Kafka clusters based on Confluent Cloud. However, an artificial expiry date must be implemented here, after which Galapagos will delete this API key again.

When a user generates new API keys and copies them, the copied message remains even if the environment is changed.

As tslint is now deprecated, we should switch to typescript-eslint instead.

Currently, the Last Changes Display in the Dashboard is capped to 10 Entries. As a user, I want to change the value to fit my needs.

Currently, naming patterns for topics, consumer groups etc. can only be configured in an inconsistent and not very flexible way. Additionally, in the frontend, not everything configured in the backend for the naming patterns is considered correctly (especially when deviating from the Hermes default values).

As a Galapagos administrator, I want to be able to centrally configure patterns and naming rules for

• Topic names
• Internal Topic prefixes
• Consumer Group prefixes
• Transactional ID prefixes

As a Galapagos user, I want to easily get an overview over the prefixes which are valid for my application. Also, I want to see the applicable naming convention for Topic names directly in the UI, as far as feasible.

Naming persistence

Galapagos should persist the prefixes associated with an application, although this would not be necessary from a "naming logic" point of view. The relevant information what prefixes are valid for use by an application are determined by

• Galapagos Configuration
• Name of the (known) Application
• Potential aliases of the (known) Application

KnownApplications are entities which are provided by external means, e.g. via JSON Import or via external tooling writing them directly to galapagos.internal.known-applications topic. These may change at any time, but an application should not (immediately) lose the right to write to its internal topics only because its name changed in an external (e.g. Enterprise Architecture) system. So we must decouple this a bit.

Additionally, there is an Admin Job for updating application ACLs from stored metadata. As this may - in principle - also be called at any time, this should also not change application rights "only" because its metadata in an external system changed.

For this reason, even the current implementation already stores valid prefixes for an application in ApplicationMetadata and thus in the internal topic galapagos.internal.application-metadata. This is stored when the certificate for the application is created on a given environment.

When streamlining the naming logic, the metadata should include all valid prefixes at the time of certificate creation on the first environment. When creating certificates for later stages, the prefixes of the first stage shall be copied automatically.

Update process

When e.g. the name of the KnownApplication changes for an already "registered" application and a user wants to make use of the changed prefix, they will have to recreate (or "extend") the certificate on the first environment, and then on all following stages. This is not completely intuitive, but still logical, as the first environment usually is where the new prefix will be used first.

As this may also cause previous rights to be revoked, the user shall get an explicit notification about changes in rights when these are detected, every time a certificate is issued / extended.

Open Questions

Even during certificate extension, rights on the affected stage may be revoked immediately also for the running production application (as the ACLs are assigned to the "user names" in the certificates, not to the certificates themselves). This is not following the Principle of least surprise, and should be mitigated somehow. Potential solutions are:

• Offer user the option to keep previous prefixes
• Issue a new identity even when creating extension certificates, with changed rights only assigned to new identity. Would require some "housekeeping" to remove previous identities after certificate expiry.

Developer certificate rights are not cleaned up after expiration of Certificates. This should be implemented.

When generating a certificate on the Prod, right after doing the same on dev stage, The radio button for "Generate private key" is now selected, but disabled because we are on a production environment. Nevertheless, the button "Generate certificate" is enabled because the radio button is selected.

Currently it is not possible to change the description of a topic. This should be fixed.

As Application Topic Owner I want to be able to delete a published Schema Version even if there are subscribers on the respective Topic. This way I am not bound to the Developer Teams and can just delete a Json-Schema If i want to .
Deleting a schema of a Topic with subscribers should only be possible on the DEV-Stage, on all other stages the feature toggle will not be used. Please Note that the schema still must not be deletable if it is already staged to the next environment.
The feature toggle has to be defined in the application.properties file.

These cases can't happen in Production and can get deleted.

The current Version of ng-bootstrap uses bootstrap 5 but galapagos works with bootstrap 4.6

The html template for the single-topic component has become very large (+400 lines of code), so that the overview is quickly lost here. Thus, in the context of this ticket, the single-topic component should be divided into several individual components. These components then have their own html templates, which in turn leads to better readability.

Currently Topic Types are in English only and it would be nice to translate them in different languages

In many Topics Galapagos is not showing the latest data written to a Topic. This should be fixed with this issue.

Currently, Galapagos creates its Metadata Topics with a Partition Count of 3. This has no advantage, as even with multiple instances of Galapagos running, each instance fully reads each metadata topic. But in environments like Confluent Cloud, where you pay for each Topic Partition, this causes unnecessary costs. So, Galapagos should create Metadata Topics with a Partition Count of 1.

When opening the certificate dialog on the PROD-Stage, then both radio buttons are disabled, but actually only the second one should be disabled, the CSR radio button should still be selectable.
On the DEV-Stage it works fine, both button can be choosen, but generating a certificate on PROD using the CSR mode is not possible and should be fixed in this issue.

If I enter a JSON schema directly when creating a topic, create the topic and then create another topic, the JSON schema can still be seen in the corresponding text field. The other text fields on the "Create topic" page are emptied after creating a topic, but this one is not.

Basically, each topic requires a number of technical settings that are necessary for proper operation or for working with the topic by applications. Users should be given maximum support in setting the correct topic settings. For that, there shpuld be a new Section on the "Create Topic" Page, where the following Data for a Topic should be captured from the user:

1. Contains the the Topic personal data?
2. Which Cleanup-strategy should be used (Log Compaction, Deletion or both)?
3. Criticality (how critical is the data on the topic?)
4. Estimation of the size of the topic

Currently the staging texts are only available in german.

Even if i change the Language Settings and the whole site is on english, the toast will stay in german. That makes it hard to understand for non-german user.

Currently to exit the JSON schema mode, is to reload the whole website.

As a user I want to use the website in darkmode and save my preferred settings

Since adding support for Confluent Cloud, it was not possible to easily get the Version number of the managed Kafka Cluster. As a workaround, the UI should on the dashboard page show the String "Confluent Cloud", if the current environment is in the "ccloud" mode.

Currently, creating a GitHub Release (via GitHub UI) does not update the pom.xml with the non-SNAPSHOT version.

The pom.xml should be updated by the release.yml script, and the updated file should then be commited and form the "real" commit for the performed release. The tag of the release should also be updated to point at this commit.

Currently, the existing UI-Tests are not executed, because they are not integrated in den CI Process. The Tests should be executed when ever merging into the next_minor branch.

Currently if you delete your ownership of a app, the app will stay in the ui as long as you didn't refresh the page. This UI update should happen immediately after deleting ownership.

Data Topics is misleading and should get renamed to Master Data

We are still at Angular 8. To use latest features like the Angular plugin for ESLint (also see #9), we should move to at least 10.2. Later Angular versions are still not fully supported by all Angular libraries we use, so we should be fine with 10.2 for now.

With 1.8.0, the JSON Schema Syntax Highlighting stopped working. The developer console reports that the Highlight.js library was not imported.

This seems to be related to the Angular 10.2 update. Now, the import must seemingly be placed in the application's main module, see MurhafSousli/ngx-highlightjs#137 (comment)

Galapagos still uses JUnit 4 for backend testing. As a Galapagos developer I would like to write tests in JUnit 5 style in order to be able to use new features that allow me to write simpler tests.

In the dashboard the message for subscription doesn't change for pending or accepting. And it doesn't get displayed at all, if a request got accepted to declined.

The property galapagos.kafka.readonly is currently not working. If this Property is set to "true", Galapagos will not perform any modification operations in Kafka. Currently this property has no effects and should be fixed.

As part of the support of Confluent Cloud, we will have to generate API keys from Galapagos. Since there is currently no officially available API for creating and deleting API keys, we have to use a command line client and "wrap" with Java - or use the "unofficial" API that the command line client uses.

The Button "Delete Schema Version" is visible even if I am not the owner of the currently viewed Topic. Luckily, it does not work in this case, as the backend detects my missing authorization for that topic.

There seems to be just a small check for isOwnerOfTopic missing since the 1.8.0 release.

In both application-dev.properties and application-ccloud.properties, the property galapagos.kafka.certificates-workdir=file:${java.io.tmpdir}/galapagos is defined, although it is no longer in use since Galapagos 2.1.0. Instead, this property should be defined on each certificate-based environment, e.g.

galapagos.kafka.environments[0].certificates.certificatesWorkdir=file:${java.io.tmpdir}/galapagos

application-dev.properties should be adjusted accordingly, and the old property should just be removed from application-ccloud.properties.

If you first select an application which has no API-Key, and then switch to another application, it doesn't generate a name for this topic and the Button to create a Topic is still deactivated.

Analogous to #9, we should also include a validation of Java Formatting Rules in the build process.

As a Galapagos Maintainer, I want to make sure that every contribution adheres to our Java Formatting Rules. I do not want to have to check formatting rules when reviewing changes.

As a Galapagos Contributor, I want easy access to the Galapagos Java Formatting Rules. They should be available in a format which I can use in most IDEs (maybe after installing a plugin). Also, auto-formatting Java Code should be available via a Maven Goal.

As a Galapagos administrator, I would like to be able to automatically create a backup of the productive metadata. This should be possible via a new Admin job. The JSON output should then be written to standard output. Using this JSON i can see all the metadata currently active within Galapagos or i can also import this JSON using the already present admin job "ImportBackupJob".

The username of the API key only shows, if a new api key gets generated. But if I refresh the site, the username disappears.

The Galapagos logo should also be found in the UI. Preferably in a place where it is permanently visible.

The Show Topic Data Pup Up shows only a preview of the data and not the whole data. A simply text that explains this would help to remove the confusion.

Currently Galapagos is logging the event but isn't showing it in the Recent Changes Tab on the Dashboard. Galapagos should also show these events in the Dashboard.

As a Galapagos Admin i want to be able to delete the tooling API Key, which can be generated using the already present Adminjob "GenerateToolingApiKeyJob". To do so, there should be a new Adminjob "DeleteApiKeyJob". This Job should also delete all the corresponding ALCs that are associated with thta key.

As Application Topic Owner I want to be able to Add a change description for a published Schema Version. This way the subscribers of a specific topic can see what was added since the last published Schema.
For this purpose the UI should contain a new input field under the textarea, where a new Schema gets published. Note: The input field should only appear for Schemas with Version two or above! For a initial Schema this makes no sense to enter a change Description. This change description should of course also be shown on the single topic page and in the email which gets sent to the subscribers, such that interested Teams can see immediately what was changed. The Change Description has to be a required field.