sensein / etelemetry-client Goto Github PK

While "I'm using this!" is a nice flag for libraries, one feature that would make this more useful for end-user tools would be to be able to have the following statuses:

• STARTED
• SUCCESS
• FAILURE

I would imagine the API roughly as:

import etelemetry

run_uuid = etelemetry.check_available_version("my/project", "1.0.0", status="START")

...

etelemetry.update_status(run_uuid, status="FAILURE" if exit_code else "SUCCESS")
sys.exit(exit_code)

WDYT?

for etelemetry to be more "adoptable" it should not introduce "heavy" dependencies, in particular at run time. requests is relatively fast to import but still not as "fast" (and I did not check memory footprint), and might not actually be used by the code which would use etelemetry.

(git)lena:~/proj/repronim/etelemetry-client[master]git
$> multitime python3 -c 'import requests'         
===> multitime results
1: python3 -c "import requests"
            Mean        Std.Dev.    Min         Median      Max
real        0.118       0.000       0.118       0.118       0.118       
user        0.113       0.000       0.113       0.113       0.113       
sys         0.004       0.000       0.004       0.004       0.004       

$> multitime python3 -c 'import http'    
===> multitime results
1: python3 -c "import http"
            Mean        Std.Dev.    Min         Median      Max
real        0.058       0.000       0.058       0.058       0.058       
user        0.050       0.000       0.050       0.050       0.050       
sys         0.008       0.000       0.008       0.008       0.008 

similar to check_available_versions(raise_exception=False)

Tag was pushed but releases page wasn't "updated" for that to be a release (by just open/edit/save cycle)

might be a great tool to remind people about software citation and provide the specific doi

I am obviously aware that the package's usage is internet-focused, nevertheless, would it be possible to have more tests providing hopefully a more accurate predictor for the package's ability to function which work inside a network sandbox?

As of 0.3.0 the package has 4 tests, and 3 of them fail with urllib or socket errrors: https://ppb.chymera.eu/e46a99.log

see https://pypi.org/project/usagestats/ . I wonder if it is feasible to collect similar stats (not just on invocation of top level command but for subcommands) with etelemetry?

If the DO_NOT_TRACK environment variable is set, etelemetry should do nothing. See https://consoledonottrack.com for more information.

spotted heudiconv reporting

WARNING: Could not check for version updates: Connection to server could not be made

and checked in dandi-cli (etelemetry 0.2.2)

2021-08-10 11:24:36,513 [   DEBUG] Could not check dandi/dandi-cli for version updates: Connection to server could not be made

expected (e.g. server is under maintenance ATM)?

but also it results in a delay on every invocation since I guess it is not giving up for a few seconds. I wonder if such status ("can't reach server") should be cached locally and not reattempted for some time. Could be critical for use of etelemetry in tools which have many invocations etc

Similar to #1, a lot of development tools may end up importing a library that uses etelemetry. For instance, in nipype, running make specs will call a script that imports nipype. It seems plausible that plenty of IDEs will also load the library.

For PyPI stats, developers won't tilt the stats so much, but if I'm using a tool that results in a lot of pings, I might be shifting stats by noticeable differences, at least in smaller projects. A development mode could either tag submissions so the server can distinguish, or set bounds on the client to avoid excessive reporting (e.g., suppress outputs for 10 minutes after calling).

Perhaps this doesn't matter for etelemetry purposes, and you want to get those reports as much as any other, but figured I'd bring it up.

The way multiprocessing works, each subprocess will end up importing a library that uses etelemetry, and so my usage will often get multiplied by the number of cores available. So, for instance, because fmriprep uses nipype with the MultiProc plugin, on an 8 core system I ping the server 8 times. Possibly more as workers are expired and restarted...

Hi all,

I'm trying to use the testing function pytest -v --doctest-modules nipype on my system for my nipype 1.5.0 build. I am receiving the error:

______________ ERROR collecting interfaces/spm/tests/test_base.py ______________
/usr/lib/python3.8/site-packages/etelemetry/client.py:5: in <module>
    import ci_info
E   ModuleNotFoundError: No module named 'ci_info'

Is there a dependency I am missing? My google searches are drowned out in general information about CI + python and I haven't been able to identify a specific python module.

System information:

Python 3.8.3
python-etelemetry 0.2.1

Thank you!

Came up in the case of dandi-cli, where we would like to announce all previous versions as "bad". So instead of listing them all (as I believe is required ATM), it would have been more convenient if I could just either declare "minimal_nonbad_version" of some kind, or just have bad_versions: ["<0.4.6"] or bad_versions: [("", "0.4.6")] to explicitly list "(from, to)" with empty string signaling that there is no lower limit.

Running mypy on code that uses the latest version of etelemetry fails with the error message "module is installed, but missing library stubs or py.typed marker". Please add type annotations to this project and include a py.typed file in the package to indicate typing support to mypy.

I guess current counts might counting also invocations on CIs unless developers explicitly disable ET by exporting NO_ET=1 env var. People will forget etc, so current counts might be artificially blown up.
Most of the CIs make themselves detectable by exposing environment variables specific to the CI. I think etelemetry-client should check for most common (travis, circle ci, github workflows), issue a log message that no checking on CI, and do not report to server

sometimes it may be useful to embed the check functionality as a cli tool or as a set of CURL commands, so that ET can be embedded into containers as well.

something similar to https://github.com/watson/is-ci, in order to avoid excessive entries

noted that dandi client is too slow, thought that it would be some added recent imports. nope:

❯ dandi -l 1 validate
2022-12-14 16:32:28,296 [   DEBUG] Starting new HTTPS connection (1): rig.mit.edu:443
2022-12-14 16:32:33,305 [   DEBUG] Could not check dandi/dandi-cli for version updates: Connection to server could not be made

so it took 5 seconds to silently timeout while checking for updates. I feel like we should add such checks to happen out of band - e.g. that function to start a thread which would be joined/waited at the end of the process (atexit) or smth like that.

For user: it might be valuable to know that newer version drops support for the python he uses, and what would be the latest newer (if any) which supports user's python

For developers: would be useful to know % of users at this point in time who still uses the version they are considering to drop. Would have been very valuable e.g. in python 2 to 3 migration times.

we should add something like this to the etelemetry client.

Telemetry
---------
vcpkg collects usage data in order to help us improve your experience.
The data collected by Microsoft is anonymous.
You can opt-out of telemetry by re-running the bootstrap-vcpkg script with -disableMetrics,
passing --disable-metrics to vcpkg on the command line,
or by setting the VCPKG_DISABLE_METRICS environment variable.

Read more about vcpkg telemetry at docs/about/privacy.md

All uses of etelemetry are currently failing on Python 3.9 and 3.10. The code was fixed in PR #26, yet no release has yet been made with this PR.

there is https://datashader.org/ -- probably overkill for just "draw the map with users", but who knows -- might be nice one to use

many tools in which we are embedding etelemetry has very similar boilerplate. perhaps could be provided by the client itself.

cc: @yarikoptic

right now it's going to the mit server, we may want to support other servers. in this case the server should be able to send back a version as well, so that as the client is upgraded, it can rely on a specific api.

I thought I'd have permissions here but I don't. No big plans, but it seems Python 3.12 no longer bundles setuptools, so some dependencies need to be fixed up a bit to enable 3.12 downstream.

The conda package does not set a version:

>>> import etelemetry
>>> etelemetry.__version__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'etelemetry' has no attribute '__version__'

or

$ pip show etelemetry
Name: etelemetry
Version: 0+unknown
Summary: Etelemetry python client API
Home-page: https://github.com/sensein/etelemetry-client
Author: Senseable Intelligence Group
Author-email:
License: Apache License, 2.0
Location: /opt/conda/lib/python3.8/site-packages
Requires: requests, ci-info
Required-by: nipype

When installed through pip, although the dunder __version__ remains undefined, at least pip knows how to retrieve the version metadata:

$ pip show etelemetry
Name: etelemetry
Version: 0.2.2
Summary: Etelemetry python client API
Home-page: https://github.com/sensein/etelemetry-client
Author: Senseable Intelligence Group
Author-email:
License: Apache License, 2.0
Location: /opt/conda/lib/python3.8/site-packages
Requires: ci-info, requests
Required-by: nipype

when http calls misbehave because of dns or routing, it would be good to return a response within a specific period of time.