dataiku / dataiku-api-client-python Goto Github PK

Python client for the DSS public API

Home Page: https://doc.dataiku.com/dss/latest/api/public/

License: Other

Python 100.00%

dataiku-api-client-python's Introduction

API client for Dataiku Data Science Studio

For more information, see:
https://doc.dataiku.com/dss/latest/python-api/rest-api-client/index.html

dataiku-api-client-python's People

Contributors

Stargazers

Watchers

dataiku-api-client-python's Issues

Could not find imported module support code for ansible.modules.dss_general_settings after release/11.4 update

The Could not find imported module support code for ansible.modules.dss_general_settings. Looked for (['ansible.module_utils.dataikuapi.dss.webapp.DSSWebApp', 'ansible.module_utils.dataikuapi.dss.webapp']) error is encountered when using release/11.4 version in an ansible module

- name: Configure K8S settings
  when: configure_k8s
  dss_general_settings:
    connect_to: "{{ dss_connection_info }}"
    settings:
      containerSettings:
        executionConfigs: "{{ k8s_executionconfigs }}"

Everything work fine with release/11.1 and previous versions.

~/ansible-dss-k8sconfig$ ansible-galaxy install git+https://github.com/dataiku/dataiku-api-client-python,release/11.1 --force
Starting galaxy role install process
- extracting dataiku-api-client-python to /home/matthieu/ansible-dss-k8sconfig/roles/dataiku-api-client-python
- dataiku-api-client-python (release/11.1) was installed successfully

TASK [Configure K8S settings] *********************************************************************************************************************************
task path: /home/matthieu/ansible-dss-k8sconfig/config_dss.yml:8
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: matthieu
<localhost> EXEC /bin/sh -c 'echo ~matthieu && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/matthieu/.ansible/tmp `"&& mkdir "` echo /home/matthieu/.ansible/tmp/ansible-tmp-1680157531.0316403-2491-43528231463604 `" && echo ansible-tmp-1680157531.0316403-2491-43528231463604="` echo /home/matthieu/.ansible/tmp/ansible-tmp-1680157531.0316403-2491-43528231463604 `" ) && sleep 0'
Including module_utils file ansible/__init__.py
Including module_utils file ansible/module_utils/__init__.py
Including module_utils file ansible/module_utils/basic.py
Including module_utils file ansible/module_utils/_text.py
Including module_utils file ansible/module_utils/common/_collections_compat.py
Including module_utils file ansible/module_utils/common/__init__.py
Including module_utils file ansible/module_utils/common/_json_compat.py
Including module_utils file ansible/module_utils/common/_utils.py
Including module_utils file ansible/module_utils/common/arg_spec.py
Including module_utils file ansible/module_utils/common/file.py
Including module_utils file ansible/module_utils/common/locale.py
Including module_utils file ansible/module_utils/common/parameters.py
Including module_utils file ansible/module_utils/common/collections.py
Including module_utils file ansible/module_utils/common/process.py
Including module_utils file ansible/module_utils/common/sys_info.py
Including module_utils file ansible/module_utils/common/text/converters.py
Including module_utils file ansible/module_utils/common/text/__init__.py
Including module_utils file ansible/module_utils/common/text/formatters.py
Including module_utils file ansible/module_utils/common/validation.py
Including module_utils file ansible/module_utils/common/warnings.py
Including module_utils file ansible/module_utils/compat/selectors.py
Including module_utils file ansible/module_utils/compat/__init__.py
Including module_utils file ansible/module_utils/compat/_selectors2.py
Including module_utils file ansible/module_utils/compat/selinux.py
Including module_utils file ansible/module_utils/dataiku_api_preload_imports.py
Including module_utils file ansible/module_utils/dataiku_utils.py
Including module_utils file ansible/module_utils/dataikuapi/apinode_admin/auth.py
Including module_utils file ansible/module_utils/dataikuapi/__init__.py
Including module_utils file ansible/module_utils/dataikuapi/apinode_admin/__init__.py
Including module_utils file ansible/module_utils/dataikuapi/apinode_admin/service.py
Including module_utils file ansible/module_utils/dataikuapi/apinode_admin_client.py
Including module_utils file ansible/module_utils/dataikuapi/apinode_client.py
Including module_utils file ansible/module_utils/dataikuapi/base_client.py
Including module_utils file ansible/module_utils/dataikuapi/dss/admin.py
Including module_utils file ansible/module_utils/dataikuapi/dss/__init__.py
Including module_utils file ansible/module_utils/dataikuapi/dss/analysis.py
Including module_utils file ansible/module_utils/dataikuapi/dss/apideployer.py
Including module_utils file ansible/module_utils/dataikuapi/dss/apiservice.py
Including module_utils file ansible/module_utils/dataikuapi/dss/dataset.py
Including module_utils file ansible/module_utils/dataikuapi/dss/discussion.py
Including module_utils file ansible/module_utils/dataikuapi/dss/future.py
Including module_utils file ansible/module_utils/dataikuapi/dss/job.py
Including module_utils file ansible/module_utils/dataikuapi/dss/macro.py
Including module_utils file ansible/module_utils/dataikuapi/dss/managedfolder.py
Including module_utils file ansible/module_utils/dataikuapi/dss/meaning.py
Including module_utils file ansible/module_utils/dataikuapi/dss/metrics.py
Including module_utils file ansible/module_utils/dataikuapi/dss/ml.py
Including module_utils file ansible/module_utils/dataikuapi/dss/notebook.py
Including module_utils file ansible/module_utils/dataikuapi/dss/plugin.py
Including module_utils file ansible/module_utils/dataikuapi/dss/project.py
Including module_utils file ansible/module_utils/dataikuapi/dss/app.py
Including module_utils file ansible/module_utils/dataikuapi/dss/codestudio.py
Including module_utils file ansible/module_utils/dataikuapi/dss/continuousactivity.py
Including module_utils file ansible/module_utils/dataikuapi/dss/flow.py
Including module_utils file ansible/module_utils/dataikuapi/dss/jupyternotebook.py
Including module_utils file ansible/module_utils/dataikuapi/dss/mlflow.py
Including module_utils file ansible/module_utils/dataikuapi/dss/modelcomparison.py
Including module_utils file ansible/module_utils/dataikuapi/dss/modelevaluationstore.py
Including module_utils file ansible/module_utils/dataikuapi/dss/projectfolder.py
Including module_utils file ansible/module_utils/dataikuapi/dss/projectlibrary.py
Including module_utils file ansible/module_utils/dataikuapi/dss/recipe.py
Including module_utils file ansible/module_utils/dataikuapi/dss/savedmodel.py
Including module_utils file ansible/module_utils/dataikuapi/dss/scenario.py
Including module_utils file ansible/module_utils/dataikuapi/dss/sqlquery.py
Including module_utils file ansible/module_utils/dataikuapi/dss/statistics.py
Including module_utils file ansible/module_utils/dataikuapi/dss/streaming_endpoint.py
Including module_utils file ansible/module_utils/dataikuapi/dss/utils.py
Including module_utils file ansible/module_utils/dataikuapi/dss/wiki.py
Including module_utils file ansible/module_utils/dataikuapi/dss_plugin_mlflow/__init__.py
Including module_utils file ansible/module_utils/dataikuapi/dss_plugin_mlflow/utils.py
Including module_utils file ansible/module_utils/dataikuapi/dssclient.py
Including module_utils file ansible/module_utils/dataikuapi/dss/feature_store.py
Including module_utils file ansible/module_utils/dataikuapi/dss/projectdeployer.py
Including module_utils file ansible/module_utils/dataikuapi/fmclient.py
Including module_utils file ansible/module_utils/dataikuapi/fm/instances.py
Including module_utils file ansible/module_utils/dataikuapi/fm/__init__.py
Including module_utils file ansible/module_utils/dataikuapi/fm/future.py
Including module_utils file ansible/module_utils/dataikuapi/fm/instancesettingstemplates.py
Including module_utils file ansible/module_utils/dataikuapi/fm/tenant.py
Including module_utils file ansible/module_utils/dataikuapi/fm/virtualnetworks.py
Including module_utils file ansible/module_utils/dataikuapi/utils.py
Including module_utils file ansible/module_utils/distro/__init__.py
Including module_utils file ansible/module_utils/distro/_distro.py
Including module_utils file ansible/module_utils/errors.py
Including module_utils file ansible/module_utils/parsing/convert_bool.py
Including module_utils file ansible/module_utils/parsing/__init__.py
Including module_utils file ansible/module_utils/pycompat24.py
Including module_utils file ansible/module_utils/six/__init__.py

Error encountered after an uptating to release/11.4

:~/ansible-dss-k8sconfig$ ansible-galaxy install git+https://github.com/dataiku/dataiku-api-client-python,release/11.4 --force
Starting galaxy role install process
- changing role dataiku-api-client-python from release/11.1 to release/11.4
- extracting dataiku-api-client-python to /home/matthieu/ansible-dss-k8sconfig/roles/dataiku-api-client-python
- dataiku-api-client-python (release/11.4) was installed successfully

TASK [Configure K8S settings] *********************************************************************************************************************************
task path: /home/matthieu/ansible-dss-k8sconfig/config_dss.yml:8
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: matthieu
<localhost> EXEC /bin/sh -c 'echo ~matthieu && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/matthieu/.ansible/tmp `"&& mkdir "` echo /home/matthieu/.ansible/tmp/ansible-tmp-1680157621.6201165-2590-237838050100479 `" && echo ansible-tmp-1680157621.6201165-2590-237838050100479="` echo /home/matthieu/.ansible/tmp/ansible-tmp-1680157621.6201165-2590-237838050100479 `" ) && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/matthieu/.ansible/tmp/ansible-tmp-1680157621.6201165-2590-237838050100479/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "msg": "Could not find imported module support code for ansible.modules.dss_general_settings.  Looked for (['ansible.module_utils.dataikuapi.dss.webapp.DSSWebApp', 'ansible.module_utils.dataikuapi.dss.webapp'])"
}

set recipe params

It would be useful to be able to set the recipe params on a recipe object

python3 compatibility

at least iter_rows() on sql_query and dataset fail

Is there python apikey for push action to version control

I connected my dataikue server to remote bitbucket repository.

Now, I want to build jenkins pipeline, which using python apikey for push action to the version control. is there such a thing?

'NoneType' object has no attribute 'startswith'

Hi guys

May I found help with this issue. It occurs when I try to install plugin throught playbook (dss_plugin).

Thanks in advance.

`The full traceback is:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 144, in run
res = self._execute()
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 648, in _execute
result = self._handler.run(task_vars=variables)
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 46, in run
result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/init.py", line 809, in _execute_module
(module_style, shebang, module_data, module_path) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/init.py", line 203, in _configure_module
environment=final_environment)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 1023, in modify_module
environment=environment)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 859, in _find_module_utils
recursive_finder(module_name, b_module_data, py_module_names, py_module_cache, zf)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 740, in recursive_finder
recursive_finder(py_module_file, py_module_cache[py_module_file][0], py_module_names, py_module_cache, zf)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 740, in recursive_finder
recursive_finder(py_module_file, py_module_cache[py_module_file][0], py_module_names, py_module_cache, zf)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 558, in recursive_finder
finder.visit(tree)
File "/usr/lib64/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/usr/lib64/python2.7/ast.py", line 249, in generic_visit
self.visit(item)
File "/usr/lib64/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/usr/lib/python2.7/site-packages/ansible/executor/module_common.py", line 460, in visit_ImportFrom
elif node.module.startswith('ansible.module_utils'):
AttributeError: 'NoneType' object has no attribute 'startswith'

fatal: [localhost]: FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}`

project.list_dashboards() doesn't exist

As a Dataiku developer I expect to retrieve a list of dashboards and their details using the dataikuapi, but when I call the project.list_dashboards() method I get the following error:
AttributeError: 'DSSProject' object has no attribute 'list_dashboards'

The Developer Guide for Projects calls for a .list_dashboards method
https://developer.dataiku.com/latest/api-reference/python/projects.html#dataikuapi.dss.project.DSSProject.list_dashboards

dataikuapi.dss.project.DSSProject (and the entire repo) doesn't contain a .list_dashboards() method.
https://github.com/search?q=repo%3Adataiku%2Fdataiku-api-client-python%20list_dashboards&type=code

Delete project: drop managed dataset option is not exposed.

v 4.0.2
Some data remains after project deletion.

recipe definition and payload

These functions are missing the return statement.

def set_payload(self, payload):
    """
    Set the raw payload of this recipe
    :param str payload: the payload, as a string
    """
    self.data['payload'] = payload

def set_json_payload(self, payload):
    """
    Set the raw payload of this recipe
    :param dict payload: the payload, as a dict. The payload will be converted to a JSON string internally
    """
    self.data['payload'] = json.dumps(payload)

ModuleNotFoundError: No module named 'dataikuapi'

Hello,
there is wrong import path causing error:
ModuleNotFoundError: No module named 'dataikuapi'

dataiku-api-client-python/dataikuapi/dssclient.py

Line 7 in b242c63

from dataikuapi.dss.notebook import DSSNotebook

Thanks for fixing it,
Milan

Apinode api incompatible with new version of requests package

For kennethreitz/requests > 2.12.1

Something regarding initialization of Auth object with an apikey set to None

Pinned requests dependency

Please resolve whatever issues are preventing you from using the most recent version of requests and unpin this dependency. Pinned dependencies in library packages introduce issues for users downstream, as shown in the output from pip included below.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dataiku-api-client 11.0.0 requires requests<2.22,>=2, but you have requests 2.28.1 which is incompatible.
Successfully installed adlfs-2022.11.2 aiohttp-3.8.3 art-5.8 azure-storage-blob-12.14.1 backoff-2.2.1 contourpy-1.0.6 cycler-0.11.0 exceptiongroup-1.0.4 fonttools-4.38.0 importlib-metadata-5.1.0 ipywidgets-8.0.2 jsonschema-4.17.3 jupyter-1.0.0 jupyter-console-6.4.4 jupyterlab-widgets-3.0.3 kiwisolver-1.4.4 llvmlite-0.39.1 markdown-3.4.1 matplotlib-3.6.2 mlflow-2.0.1 nest_asyncio-1.5.6 networkx-2.8.8 numba-0.56.4 numpy-1.22.4 pandas-1.5.2 papermill-2.4.0 pika-1.3.1 pyarrow-10.0.1 pydantic-1.10.2 pytest-7.2.0 pytest-asyncio-0.20.2 qtconsole-5.4.0 qtpy-2.3.0 redis-4.3.5 requests-2.28.1 scikit-learn-1.1.3 shap-0.41.0 slicer-0.0.7 treelite-3.0.1 treelite_runtime-3.0.1 widgetsnbextension-4.0.3

API-Client calling GET instead of POST or PUT

The client is calling a GET request instead of a POST or PUT request, even when selecting the POST or PUT methods in the GUI.

Please release 2.2.0 on Pypi

Also would be great to use tags :)

Client list users method have attribute wrongly spell

See I wanted to know all the users which are currently active.
Problem with the current code is, list_users() returns list of dictionary. And each dictionary has attribute activeWebSocketSesssions, ideally this should be activeWebSocketSessions.

Note - I wanted to make that change using the pull request, but I did not find this attribute in the current codebase, may be because you guys did not revel it, may be not. Aways this is my finding, that will help make this product a bit better.

    def list_active_users(self):
        """
        Get all the active users who currently login to the DSS

        This method takes nothing and returns name of the users who are actively working now

        Returns:
        string: calculate all currently active users
        """
        user_list = []
        dss_users = self.client.list_users()
        for user in dss_users:
            if user["activeWebSocketSesssions"] != 0:
                user_list.append(user["displayName"])
                
        return user_list

Incorrect documentation on `get_scoring_jar_stream` and `get_scoring_pmml_stream`

I'm using DSS version 9.0.3. The documentation for ML model export functions get_scoring_jar_stream and get_scoring_pmml_stream says that they should return file-like objects that need to be closed after use.

In fact they return requests.Response objects, so correct usage is along the lines of

response = model_details.get_scoring_pmml_stream()
pmml_xml = response.content

The documentation should be updated to reflect this: see here and here.

missing import in scenario.py

datetime is not imported => ScenarioRun.get_start_time() fails...

create_user fails

There is a mismatch in the "profile" vs "userProfile" variable.

ImportError: No module named 'dssclient'

getting the below import error

from dssclient import DSSClient
ImportError: No module named 'dssclient'

Version:
dataiku-api-client==3.1.4

even the previous versions of the client gives the same error.

Dataikuapi is not correctly honoring "DKU_NO_CHECK_CERTIFICATE" in requests

In theory, Dataiku supports using self-signed certificates by setting e.g. using an environmental variable DKU_NO_CHECK_CERTIFICATE or by setting no_check_certificate=True in the set_remote function call of the non-open source dataiku package.

However, this parameter is not passed to the api_client(), which causes requests to fail which are made to Dataiku instances using self-signed certificates. It is possible to mitigate this by setting the verify value of the internal Session object to false:

client = dataiku.api_client()
client._session.verify = False

Which is not obvious and requires looking through the internals of the package.

A fix at the central call site of the dataikuapi package, _perform_http:

dataiku-api-client-python/dataikuapi/dssclient.py

Lines 1228 to 1233 in fc51e29

 http_res = self._session.request( 

 method, "%s/dip/publicapi%s" % (self.host, path), 

 params=params, data=body, 

 files=files, 

 stream=stream, 

 headers=headers)

would only be possible in parts by duplicating the logic to check the "DKU_NO_CHECK_CERTIFICATE" variable or config files, but would still fail for Dataiku session set up by using the set_remote_dss function call.

The best option would be to add a "no_check_certificate" parameter to the DSSClient class that can be set when creating an instance of the class. But as the DSSClient is also used by the closed-source dataiku package, it seems to me that this can't be solved by contributing to the open source dataikuapi package.

dataikuapi/dss/recipe.py : set_code_env fix

Hi,

I don't know if the the post I've made on the Community forum has been reviewed yet.
I haven't seen a commit on it (not in the "master" nor the "dss80-set-code-env-mode") so I post an issue here.

The function set_code_env (from the class CodeRecipeSettings) doesn't set the code env but a string :

if code_env is not None: rp["envSelection"] = {"envMode": "EXPLICIT_ENV", "envName": "code_env"}

Should be :

if code_env is not None: rp["envSelection"] = {"envMode": "EXPLICIT_ENV", "envName": code_env}

Hoping I won't trouble your workspace.

Greetings,
Steven

Limitations of relative imports

Relative imports can have various impacts when the package is used from an external script, since said script may execute in the right virtual env but maybe not the right python context module wise.

There is two solutions to this:

Expose symbols between modules in the main init file
Use fully qualified imports

Unfortunately, the fully qualified break the possibility to "relocate" the module, ie, making it a submodule of another, which is required here for it to be usable as Ansible module utils.

No action is specifically required, this issue is merely a reminder of previous work in case the question arise later on.

Pipeline fails with import error for dataiku

I'm trying to run the below example on my jenkins node:

https://community.dataiku.com/t5/Knowledge-Base/Building-a-Jenkins-pipeline-for-Dataiku-DSS/ta-p/10201
In one of the stages, I created python venv which install the requirments.txt file. I can see in the console log that dataiku-api-client was installed with version 8.0.0, but still, it fails later with the first command of import dataiku:

ModuleNotFoundError: No module named 'dataiku'
My python version is 3.7.4
any idea what's wrong with that?

	http_res = self._session.request(
	method, "%s/dip/publicapi%s" % (self.host, path),
	params=params, data=body,
	files=files,
	stream=stream,
	headers=headers)

dataiku / dataiku-api-client-python Goto Github PK

dataiku-api-client-python's Introduction

dataiku-api-client-python's People

Contributors

Stargazers

Watchers

Forkers

dataiku-api-client-python's Issues

Recommend Projects

Recommend Topics

Recommend Org