Adding thread_runner to take care of Test Case runs.

All the files have to fixed for lint and flake errors. This issue can be referred for all the PRs fixing lint and flake.

Adding function for non-standard commands

Modify the CONTRIBUTION file as per the progress.
For now, the CONTRIBUTION file looks great and helpful.

the remote_exec class variable in the test_case can be easily misinterpreted as remote executioner whereas in the actual sense it the reference of the mixin object passed to the test case be the runner_thread. So the variable has to be changed accordingly.

Currently re-connect is not done in paramiko if there is a node reboot. But this will cause issues if ops is desired after node reboot also.

https://github.com/aujjwal-redhat/RedAnt_Test/blob/8f420dd6d4fab291c824cbac50d03b728dd70516/redant_libs/peer_ops.py#L15

The remote command execution object shouldn't be created in the ops library. It should be rather created by the Test runner thread. Once created in the test runner thread, we can pass it as an import value. Creating the rexe object in the ops libraries will defeat the very idea of keeping things modular.

The parsing module consists of 3 components:

1. The detailed config file
2. The Parser class which parses the config file and generates config hashmap given the path of config file
3. The ParamsHandler which parses the configuration parameters from config hashmap

The gluster_test_main file makes call to parsing module whose function is mentioned in the Gluster-test-design document.

The usual way of formatting string in python had been
("sample %s" %(val) ) or ("sample %s", val)

But the cleaner way of doing it is using the formatted strings.
(f"sample {val}"). I propose we use this format across the project rather than having multiple different ways of formatting our string.

For reference : https://docs.python.org/3.6/reference/lexical_analysis.html#formatted-string-literals

Adding a maintainers' markdown file.

Logging is important for debugging the carcass of the TCs which fail. Hence it would be annoying to go through a data dump containing logs for all the TC to check why it failed. The better way would be to mimic the tests director structure and create a per volume type run per TC log file. So, no more going over a big file and using grep. Just navigate to that particular log file belonging to a TC.

The idea is to document for posterity. Now, python has sphinx and other ways of directly converting the code to a documented website ( though we have to do some configurations ). But, I believe that will be more of a "what's the API and class the module has", rather than the "why" and "how".

Let's keep this issue as long running. We can give PRs and actually add the Updates: # keyword during a commit. So do not close this issue.

The readme is to be updated with the latest changes wrt addition of redant.

The current logging is done by the following manner RR.rlogger.<log_level>(<log_message>) now, instead of this, we can have one method which is of the form, RR.rlogger.log(<log_message>, log_level>

The IPs are taken from the config file and and are accessed through Parent Test. The test case should be running successfully along with other components.

Cleaning up of the files which are redundant.

Create parent_test to be integrated with the rest of the components.

https://github.com/aujjwal-redhat/RedAnt_Test/blob/8f420dd6d4fab291c824cbac50d03b728dd70516/redant_libs/peer_ops.py#L13

Instead of printing, we should create a logging class and use it across. Console print must be limited to only showcasing success and failure details and not return values.

Basic test cases have to be added which uses the ops library.

NA

Add new requirements in the requirements.txt file as per the progress.

It is confusing to refer to thread_runner.py and test_runner.py. Also design doc specifies runner_thread.py.

There should be validation for the value being returned by the remote command executor. The value being returned will be a dictionary and hence has to be parsed accordingly for success or failure. If there is a failure, an exception must be thrown. Acc to design, we will be relying on the try-except flow rather than the return values being checked on the TC.

Add linters to check for the typos, code styling and other errors in the code without actually executing it.

The TC will contain a comment line at the very beginning stating the TCs type, whether it is disruptive or non-disruptive and along with that as to what all volume types, this TC will have to be run for.

For example,

#disruptive;replicated,distributed,distributed-replicated

Now this parsing has to be part of the test_list_builder.

redant_test_main => gluster_test_main
redant_test_runner => test_runner

The gluster Ops library will mainly contain operations for starting and stopping gluster service on the server.

Change the nomenclature of args.
Adding the directory path in ls func

The test runner is deemed to be the script which would be actually invoked by the user to run the test cases.

The idea is that the test runner will parse through the existing test cases and decide their order of execution and also if they can be parallelized. For executing the TCs, the test runner will spin up the thread_runner which would inturn be responsible for that one TC ( for a given volume type ).

This issue is to track as to what all features will come under the test runner.

Errors & Messages:

E1101:

'%s %r has no %r member%s'
https://pycodequ.al/docs/pylint-messages/e1101-no-member.html

In our code

E1101: ❌

Instance of 'VolumeOps' has no 'rlog' member (no-member)

E0401: ❌

Unable to import 'redant_libs.support_libs.rexe' (import-error)

R0801: ❌

Similar lines in 5 files

R0903: ❌

Too few public methods (0/2) (too-few-public-methods)

C0103: ❌

Variable name "f" doesn't conform to snake_case naming style (invalid-name)

W0703: ❌

Catching too general exception Exception (broad-except)

R0902: ❌

Too many instance attributes (12/7) (too-many-instance-attributes)

Regerating the errors locally :

1. Remove the specific errors or all of them from disable in .pylintrc file.
2. Then run the command:

pylint -j 4 --rcfile=.pylintrc redant_libs/

The redant_test_main has the following functionalities under it,

• Parsing of the config file
• Invocation of the test list builder and forming the order of test execution
• Using the above two data to invoke the test_runner.

Now to achieve this, following parameters have to be provided to the redant_test_main during invocation as command line arguments,

1. Config file path
2. Test directory ( which contains the test cases or tree of test cases within )
3. Log file directory ( within which the log files for each TC will be created. )

Fixing the docstring of the ops libraries

Pylint is showing multiple issues especially snake_case naming style.

Useful links:
Snake case naming style
What is snake case

We should start exploring as to how to spin up a sphinx document for the code base and also as to how to host it in read-the-docs. ( or any other hosting site which is free and open source ).

Log Message format

===================
All log messages MUST include:

• The cmd that is/was executed
• The machine(s) on which the command is/was executed

Ex: peer_ops

On starting the execution of cmd on a node:

 self.rlog("Running %s on node %s" % (cmd,node), 'I')

On completing the execution:

 self.rlog("Successfully ran %s on %s " % (cmd, node),'I')

Log levels

===========

• Everything in the ops library has to be in info mode.
• Everything in the test cases has to be in error mode.
• Everything other than ops to be in debug mode.

The whole framework is divided into the following parts

1. Core - Contains the core test framework which includes : Parsing + Test List Builder + Test Runner + Runner Thread + Test Main
2. Config - Contains only the config file
3. Tests - Contains : Parent Test + Performance tests + Functional Tests
4. Support - Contains: Ops libs

Currently the Volume Mount operation exists in io_ops library but the volume mount operation should exist in volume_ops library.

Add test_list_builder to create a list of tests-to-run to be used by the redant_test_runner component.

-> Added Try-Except blocks
-> Removed the extra spaces and the imports
-> Created the class and defined all ops func inside it.
-> Used format specifiers

The current dependency flow is a little circuitous and can lead to issues. I'm proposing we combine all the base support libraries and ops into a framework_mixin which can be used across the test functions instead of individual calls to the classes and modules.

If there are multiple functions existing in a test case the way the those test functions will be called through runner thread has to be handled.

The current flow in rexe doesn't handle all exceptions and hence changing it to handle all the exceptions.

We all know the python is slick in handling types, for instance we can create a function called

def sample_func(sample_var):
   print(sample_var)

and this function can take in any data type and print it ( even functions as functions are first class objects in python ). So this can be disastrous for us who are designing the framework as this can lead to issues when somebody invokes and uses the function
in a way which isn't intended. The question is, do we have a salvation ?

Yes,

PEP-3107 : https://www.python.org/dev/peps/pep-3107/

The very least we can do is annotate the parameters so that it is seen in the function definitions when the user if going through them. If anyone uses IDEs, these annotations would help in correcting a user if they're going to commit a mistake in the type of data they're going to pass.

The relog module already uses function annotations, and can be used as a reference for further changes across the framework.

With the addition of the linting in actions. fixing the code coming under my purview.

Add a PR template file to help in making a PULL request.

Addition of io_ops library to the redant_mixin

APIs which return servers and clients info which consists of ip,hostname,user,passwd have to be added

Log path in non-disruptive condition is wrong.

The readme is not complete without the addition of the design doc.

Making small changes like correcting grammatical mistakes to the readme to be linked to this.