Error in pipeline in GithubActions
14:25:43.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDOUT: Starting Mlflow UI on port 5000 14:25:46.430 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 2020/12/19 14:25:46 INFO mlflow.store.db.utils: Creating initial MLflow database tables... 14:25:46.453 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 2020/12/19 14:25:46 INFO mlflow.store.db.utils: Creating initial MLflow database tables... 14:25:46.468 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 2020/12/19 14:25:46 ERROR mlflow.cli: Error initializing backend store 14:25:46.480 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: ] 14:25:46.483 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.484 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: ) 14:25:46.485 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: FOREIGN KEY(experiment_id) REFERENCES experiments (experiment_id) 14:25:46.487 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT runs_lifecycle_stage CHECK (lifecycle_stage IN ('active', 'deleted')), 14:25:46.489 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT status CHECK (status IN ('SCHEDULED', 'FAILED', 'FINISHED', 'RUNNING')), 14:25:46.491 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT source_type CHECK (source_type IN ('NOTEBOOK', 'JOB', 'LOCAL', 'UNKNOWN', 'PROJECT')), 14:25:46.493 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT run_pk PRIMARY KEY (run_uuid), 14:25:46.495 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: experiment_id INTEGER, 14:25:46.496 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: artifact_uri VARCHAR(200), 14:25:46.497 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: lifecycle_stage VARCHAR(20), 14:25:46.500 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_version VARCHAR(50), 14:25:46.505 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: end_time BIGINT, 14:25:46.509 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: start_time BIGINT, 14:25:46.509 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: status VARCHAR(20), 14:25:46.509 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: user_id VARCHAR(256), 14:25:46.510 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: entry_point_name VARCHAR(50), 14:25:46.510 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_name VARCHAR(500), 14:25:46.510 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_type VARCHAR(20), 14:25:46.510 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: name VARCHAR(250), 14:25:46.511 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: run_uuid VARCHAR(32) NOT NULL, 14:25:46.511 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: (Background on this error at: http://sqlalche.me/e/gkpj) 14:25:46.511 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.516 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: The above exception was the direct cause of the following exception: 14:25:46.516 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: _tracking_store = _tracking_store_registry.get_store(store_uri, artifact_root) 14:25:46.517 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1618, in _run_visitor 14:25:46.517 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: conn._run_visitor(visitorcallable, element, **kwargs) 14:25:46.518 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: visitorcallable(self.dialect, self, **kwargs).traverse_single(element) 14:25:46.541 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: (Background on this error at: http://sqlalche.me/e/gkpj) 14:25:46.541 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: ] 14:25:46.541 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.541 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: ) 14:25:46.542 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: FOREIGN KEY(experiment_id) REFERENCES experiments (experiment_id) 14:25:46.542 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT runs_lifecycle_stage CHECK (lifecycle_stage IN ('active', 'deleted')), 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 2020/12/19 14:25:46 INFO mlflow.store.db.utils: Updating database tables 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: INFO [alembic.runtime.migration] Running upgrade -> 451aebb31d03, add metric step 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: INFO [alembic.runtime.migration] Will assume transactional DDL. 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT status CHECK (status IN ('SCHEDULED', 'FAILED', 'FINISHED', 'RUNNING')), 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT source_type CHECK (source_type IN ('NOTEBOOK', 'JOB', 'LOCAL', 'UNKNOWN', 'PROJECT')), 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CONSTRAINT run_pk PRIMARY KEY (run_uuid), 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: experiment_id INTEGER, 14:25:46.543 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: artifact_uri VARCHAR(200), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: INFO [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: lifecycle_stage VARCHAR(20), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_version VARCHAR(50), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: end_time BIGINT, 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: start_time BIGINT, 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: status VARCHAR(20), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: user_id VARCHAR(256), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: entry_point_name VARCHAR(50), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_name VARCHAR(500), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: source_type VARCHAR(20), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: name VARCHAR(250), 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: run_uuid VARCHAR(32) NOT NULL, 14:25:46.548 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CREATE TABLE runs ( 14:25:46.549 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: [SQL: 14:25:46.549 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.549 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: DETAIL: Key (typname, typnamespace)=(runs, 2200) already exists. 14:25:46.549 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "pg_type_typname_nsp_index" 14:25:46.549 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: cursor.execute(statement, parameters) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: self.dialect.do_execute( 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: raise value.with_traceback(tb) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: reraise(type(exception), exception, tb=exc_tb, cause=cause) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: util.raise_from_cause(sqlalchemy_exception, exc_info) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: self._handle_dbapi_exception( 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: ret = self._execute_context( 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1039, in _execute_ddl 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return connection._execute_ddl(self, multiparams, params) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return meth(self, multiparams, params) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 982, in execute 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: self.connection.execute( 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 821, in visit_table 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return meth(obj, **kw) 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 138, in traverse_single 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: self.traverse_single( 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 777, in visit_metadata 14:25:46.550 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return meth(obj, **kw) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: INFO [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 138, in traverse_single 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2049, in _run_visitor 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: bind._run_visitor( 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4315, in create_all 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: InitialBase.metadata.create_all(engine) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/store/db/utils.py", line 30, in _initialize_tables 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: mlflow.store.db.utils._initialize_tables(self.engine) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/store/tracking/sqlalchemy_store.py", line 99, in __init__ 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return SqlAlchemyStore(store_uri, artifact_uri) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/server/handlers.py", line 64, in _get_sqlalchemy_store 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: return builder(store_uri=store_uri, artifact_uri=artifact_uri) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/registry.py", line 37, in get_store 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/server/handlers.py", line 91, in _get_tracking_store 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: _get_tracking_store(backend_store_uri, default_artifact_root) 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/server/handlers.py", line 105, in initialize_backend_stores 14:25:46.564 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: initialize_backend_stores(backend_store_uri, default_artifact_root) 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/mlflow/cli.py", line 291, in server 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: Traceback (most recent call last): 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: DETAIL: Key (typname, typnamespace)=(runs, 2200) already exists. 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "pg_type_typname_nsp_index" 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: cursor.execute(statement, parameters) 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: self.dialect.do_execute( 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: Traceback (most recent call last): 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: CREATE TABLE runs ( 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: [SQL: 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: DETAIL: Key (typname, typnamespace)=(runs, 2200) already exists. 14:25:46.565 [ducttape-1] DEBUG org.testcontainers.containers.output.WaitingConsumer - STDERR: 2020/12/19 14:25:46 ERROR mlflow.cli: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
which causes test to not pass

Here we mean all the classes which would wrap the model data (probably from model part of jpmml) without signature of model

Integrate testing model to repository to give developers ability to test their code as a port of the workflow with automatic runs on pull requests.

Scala files should be placed in some folders, maybe sbt should be used as project manager. Try to reproduce given example of scala project

Try to integrate https://www.scalatest.org/ with repository workflow.

• crete some project with needed dependencies etc and try to communicate with local instance of mlflow
• prepare scripts for mlflow setup for dev-environment (should be docker or something else independent from platform)

This work could be duplicated so we will choose the most proper solutions to use in project

Here all important things about making your nussnacker instance working should be placed as a source of hints for other developers

Reading about format of data returned by models served by mlflow we would need the the MultiMap and MultiSet collections I think (as we can see at https://www.mlflow.org/docs/latest/models.html#local-model-deployment). There is a need of our own implementation as for scala 2.12 I don't see any usable one.

Use application.conf file to define the environment of application which includes maily mlflwo configurtion

We need to change the api a little and create some kind of SignatureInterpreter interface which will be able to create the typed signeture based on nussknacker classes having the signature definition from some framework (so a big part of MLFModelInstance methods will be moved to such interpreter)

• crete some project with needed dependencies etc and try to communicate with local instance of mlflow
• prepare scripts for mlflow setup for dev-environment (should be docker or something else independent from platform)

This work could be duplicated so we will choose the most proper solutions to use in project

Let's see how we can use https://github.com/testcontainers/testcontainers-scala for integration tests of our module

A request to the served wine model using model API should return n predicted values for n rows of data. Right now it only returns a single value

For now, there is Double used as the type of the inputs and outputs of the model. Wee need to change it to be connected with the signature. Maybe at the top of the interface it should be just any ref, which would have been parsed and checked in more specific (but rather abstract prinz) classes.

Add scalafix or something better which can enforce some concrete coding rules in scala

Try to add our plugin to nussknacker environment and see it in the UI as some flow block that would allow to receive and send data in the flow of nussknacker data processing workflow. Try to maybe add it manually by using docker bash console and in the next step try to integrate the build process with injection of built plugin to nussknacker environment automagically.

Current configuration of MLflow environment servers single model as we can see in this file. But MLflow after starting serving this model (or even earlier when running the model) add model to registered models database so it can be listed using the endpoint specified here. When you read this documentation there is only specification of something named "run_link" here. We should get to know and configure MLflow to serve the information about deployed model location (I think so but I'm not 100% sure) or at least try to figure out what information we should get from MLflow and from where (which endpoints) to give nussknacker the ability to send data to model and receive some results.

Implement insertion ordered Multimap and add .map method for Multimap to simplify data conversion.

for now, there is an modelInstance.getSignature.getOutputType.head.typingResult returned as the typed result of the. prinz output. We need to modify it to provide all. the outputs as the object with fields "output_1", "output_2", ... which would be connected with all values provided by mlflow model

May be helpful:

• pl.touk.nussknacker.engine.flink.util.sink.EmptySink
• pl.touk.nussknacker.engine.flink.util.transformer.PeriodicSourceFactory

Team name and projects names need to be updated everywhere

Add definition of input and output data of served model by manual definition in model creation process

We can do this by adding some "data reset procedure" before environment startup by removing the whole database of mlflow or something similar

When many classes extend from UnitIntegrationTest the first called tests not clear the docker env properly so the following tests cannot be done properly

We wanted to create single fat jar not to include every dependency as a jar from specified build path

Experiment id is needed to get the MLmodel file with signature which is located on S3 server on specified path by experiment id

Implement conversion between collections and data formats returned by served MLflow models

Create README docs with some explanation of project purpose and design graph

Add ngnix with base (a few conf files) which redirects all the trafiic of data in docker environment which goes to mlflow port i.e. 5000. This task includes disabling the exposed port of mlflow (which is exposed to host by the 5000:5000 definition), creating separete docker network for mlflow<->ngnix communication and exposing 5000 portby ngnix. After this task done user should be able to do requests to localhost:5000 and when the request is recognized as "invocation request" (i.e. requests to path which looks like for example http://localhost:5000/model///invocations) should be redirected to localhost:1234/invocations where the model is located (so for now we ignore the model version and model name here as we have single model served). Other requests to localhost:5000 (so to ngnix) should be redirected to mlflow:5000

• crete some project with needed dependencies etc and try to communicate with local instance of mlflow
• prepare scripts for mlflow setup for dev-environment (should be docker or something else independent from platform)

This work could be duplicated so we will choose the most proper solutions to use in project

Get to know how data is managed in nussknacker and try to send it to mlflow model available in prinz API

Try to abstract the main responsibilities of Prinz into interfaces (something like ModelRepository, ModelInstance, DataFlow etc.) and try to create their implementations for MLflow. This task is divided into two parts:

• gathering data about model from model repository or something similar so there is a need to implement REST client that can get all needed data about served models and their configuration - you can see some example of base for such client on separate branch here which was used to test MLflow during first configurations so it cover only a few endpoints that were tested
• sending data to model served by repository and receiving predictions from model. For now there is single model in dev-environment that is served under static URL http://localhost:1234 so you can test it like in this example via curl request. There is an issue #20 which will try to unify method of getting the information about model location but for now we can stick to this static URL

We need to try to get visible MLmodel signature like described here https://www.mlflow.org/docs/latest/models.html#fields-in-the-mlmodel-format

We need to use async client with async backend to call run method in MLFModelInstance asynchronously (and return here some Future or ComputableFuture etc.)