testground/celestia-app: create basic validator steps creation about test-infra HOT 6 CLOSED

Introduction/Motivation 🧐

Currently, we are using pre-generated data manually for app in testground through copying it from the test-infra/core-configs repo into the Dockerfile

COPY --from=builder ${PLAN_DIR}/core-configs /core-configs

This is not an optimal solution as it requires the test user to pre-gen and upload the keys/validators info into the docker container per each of the instance that fires up during test-execution.

Implementation 🧱🛠️

For all this to work properly, it is necessary to have

1. Orc -> Orchestrator(needs a dedicated role in testground plan manifest/code-base) role
2. RoV -> The rest of validators (no role assignment in testground manifest/code-base)

Topics that needs to be created in order for both of roles (Orc/RoV) to communicate between each other:

1. Account address - passing accounts info between testable instances
2. Genesis - passing genesis.json around
3. Gentx - passing gentx-.json around

Orchestrator 🪄

Here are the step-by-step guide on what this app-role should do

1. Initializing the chain
2. Create key
3. Subscribe to the Account address topic (RoV will publish those)
4. Add genesis account for each of received address through subscription
5. Publish a genesis.json to the genesis topic
6. GenTxs
7. Subscribe to Gentx topic (RoV publishes their gentx-.json)
8. Collect-gentxs
9. Publish a final genesis.json to the topic
10. Starts the app

RoV - Rest Of Validators 🤹‍♀️

1. Create key
2. Publish their account address to the account address topic
3. Subscribe to genesis topic(Orc will publish this)
4. Saves the received genesis.json to local config
5. GenTxs
6. Publish the local gentx.json to the gentx topic
7. Subscribes to the genesis topic again to get the final version
8. Starts the app

Hints

Roles

We use compositions to assign roles. Please take a look at test-infra/composition.toml to have an idea how roles are defined and test-infra/main.go in runsync() to see how we treat each of the roles

Every app cli command should be wrapped into a dedicated func that triggers cobra cmds

More info can be found from the func StartNode() located at test-infra/testkit/appkit

func WrapCliCommand(cmd *Cobra.cmd, home string) error {
    cmd.SetArgs([]string{}) //what command is needed
    // Use server cmd to execute the cmd
}

Syncronizing data between apps

More info can be found in official docs.
For our purposes, the Orc and RoV are publishing and subscribing to differrent topics. An example can be found for existing PoC, where the Bridge Celestia Node is subscribing to the Validator App Node, which publishes it's IP and Hash of the first block

topic := sync.NewTopic("name", &Foo{})
client.Publish(ctx, topic, &Foo{foo: 1, bar: "snack"})

fooCh := make(chan *Foo)
client.Subscribe(ctx, topic, fooCh) 

Start small - go big later

Testground eats a lot of resources at the start to fire up the instances. Setting up a 2 or 3 app instances in the beginning is making a lot of sense for the dev purposes
Keep in mind that creating a separate composition.toml is also ok ✅

from test-infra.

What will be the approach here? I think @musalbas was advocating to write (somewhat parametrizable e.g. bash) scripts that that do all the above and make them part of the code that stand up the experiments. Alternatively, this can happen in golang but from what I understood this is like using some code as a library that is rather written with the CLI in mind and not really use-as-library friendly?

Or is this issue just to gather requirements first?

from test-infra.

It is also worth noting that there are two somewhat orthogonal goals we have in mind:

1. have reproducible rather academic experiments with measurements/diagrams. Ideally, in realistic public network conditions.
2. have an extendable infrastructure to implement several scenarios to test/experiment with, potentially even on a somewhat automated basis.

I think the 1. is somewhat more urgent as it proofs certain properties of our architecture and e.g. the way we use bitswap for block reconstruction and such. But my understanding is that you are planning to achieve both goals with the current approach (test-ground).

from test-infra.

What will be the approach here? I think @musalbas was advocating to write (somewhat parametrizable e.g. bash) scripts that that do all the above and make them part of the code that stand up the experiments. Alternatively, this can happen in golang but from what I understood this is like using some code as a library that is rather written with the CLI in mind and not really use-as-library friendly?

The approach is to use the cmd that the app populates to the end user and not through bash-scripts.
And this approach is the same as the end user is triggering celestia-appd binary.
Example: test-infra#23-L127-132

You can parametrise whatever you want to do with the appd through cmd string args.

Or is this issue just to gather requirements first?

I don't think so, as my intention with this issue is to only expose what is left to be done in cmd for app to have full functionality in testplan design

from test-infra.

To clarify, we'd like the number of validators to be parameterized and generated automatically by a script, but it seems that currently 10 validators are manually hard coded. Is that temporary?

from test-infra.

To clarify, we'd like the number of validators to be parameterized and generated automatically by a script, but it seems that currently 10 validators are manually hard coded. Is that temporary?

Yes. It's a temporary solution to go further with adding celestia DA network(bridge/full/light) on top for a PoC.
Further down the road, creating keys/gentxing/collect-gentxs and passing genesis.json around should be done by the same fashion as is done already for starting celestia-app

func StartNode(cmd *cobra.Command, home string) error {
	cmd.ResetFlags()
	cmd.Flags().Set(flags.FlagHome, "")

	cmd.SetErr(os.Stdout)
	cmd.SetArgs([]string{"start", "--home", home, "--log_level", "info"})

	if err := svrcmd.Execute(cmd, EnvPrefix, app.DefaultNodeHome); err != nil {
		return err
	}

	return nil
}

from test-infra.