Orca setup script is broken about kanidm HOT 20 CLOSED

Well, mostly I was thinking about the way I approached orca in the past was "benchmark" and I modelled a lot of things on ldclt from 389-ds which is their benchmark tool.

But recently @jinnatar pointed out she wanted to see an "enterprise simulator". Where the benchmark isn't just "sterile" kind of do-searches like ldclt was doing, but should be about simulating user actions.

Then that got me thinking about a way to potentially use that as a basis to not only do an enterprise simulator, but also how to potentially model a more controlled test mode.

Currently the design has workers that take a task queue and then execute the tasks and report on the stats. That ends up with a thread-per-worker and each worker just shoves out tasks etc. But it's hard to see how the tasks the correspond to a user, or different things, and it's hard to do other kinds of interactions.

Instead, I think we could do it like erlang. Instead of workers, we spawn a task for each "person". Then each of these tasks has a little schedule, like "login, click about, idle for some time, then do some more etc". It models how a person would interact. At the start of the test, we can have each "task" create it's own user/password etc, rather than the crazy batch thing we have now. It also allows easier interleaving of things naturally based on user interactions.

We still need a centralised stats-collector but that model already exists, so we can time operations for each task.

This also then makes it easier to do different kinds of tests later like the high-search load tests (direct each "person" to just search) etc.

I think for now we drop the LDAP support too, we can add that back later,

from kanidm.

Did some poking around this and from what I can infer, dropping the add to idm_admins & just switching orca to use idm_admin instead of admin should be the right way to go. At least it now succeeds conntest and from what I can read from the sources, all actions performed are in the domain of idm_admins and not system_admins.

So the problems in order are:

1. The setup script should infer the password for idm_admin
2. Orca needs to be patched for using idm_admin instead of admin and to no longer attempt adding admin into idm_admins.
3. Orca has plenty of other compatibility issues as well, but those should be new separate issues. I'd say when conntest passes, this one is done.

from kanidm.

(I just realized I linked the wrong commit, I'm gonna fix when I get home)

from kanidm.

We might want to consider converting this to ansible rather than a shell script so we could coordinate vms for benchmarking?

from kanidm.

Yeah I removed admin_pass from setup_dev_environment because it wasn't needed after idm_admin permissions changed. I've never really gotten orca to work so getting it some love would be handy.

from kanidm.

Yeah I removed admin_pass from setup_dev_environment because it wasn't needed after idm_admin permissions changed. I've never really gotten orca to work so getting it some love would be handy.

So I shouldn't do any harm if I just add it back right?

from kanidm.

Add it specifically to the orca script because it's needed there.

from kanidm.

Add it specifically to the orca script because it's needed there.

Sounds like a plan :) I already got that patched somehow but now it won’t let orca make admin an idm_admins, and I really don’t know how to override that

from kanidm.

Don't do that then, it's not necessary (or shouldn't be, use the right account for the right actions)

from kanidm.

from kanidm.

I'd say look at all the actions that things are doing and go from there, rather than just one-off troubleshooting. Orca hasn't really been maintained/tested as often as the rest of the tooling.

from kanidm.

Uh I'm noticing it now, thanks!

from kanidm.

I probably need to look into a cleanup of this with @Seba-T first then :)

from kanidm.

I actually just started poking around it again, what a perfect timing :D

from kanidm.

@Seba-T I'm suspecting we could re-write it as I have some new and better ideas this time ....

from kanidm.

Yeah I mean I'd gladly redo it from scratch rather than endlessly struggle against something that hasn't been maintained in ages (which has been my experience so far :p)

from kanidm.

So thinking about it, I think actually #1131 has overlap to this - both this issue and the mentioned one "need a way to setup a db of expected content" so I think we can roll these up together.

from kanidm.

So thinking about it, I think actually #1131 has overlap to this - both this issue and the mentioned one "need a way to setup a db of expected content" so I think we can roll these up together.

In this instance wouldn't we just need to create some users account and then take it from there? Or is #1131 about implementing a setup tool that allows to specify what you want to fill the db with, i.e. "generate 10k users and put them in x group"

from kanidm.

Well it turns out they're the same problem.

We need a way to setup a db full of expected content.

We need a way to setup a db full of expected content AND then we do a load test against that content.

from kanidm.

Mhh makes sense. We'll have a way to setup a db full of expected content then :p

from kanidm.