musicfox / pycmc Goto Github PK

View Code? Open in Web Editor NEW

21.0 4.0 2.0 913 KB

A Python interface for the chartmetric.com API

License: MIT License

Python 99.16% Dockerfile 0.81% Shell 0.03%

python chartmetric api-client musicfox

pycmc's People

Contributors

Stargazers

Watchers

Forkers

romeodespres djdamie

pycmc's Issues

[BUG]: credentials process occasionally throws a 504 HTTPError

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

pycmc module called
pycmc function utilized
Function parameters input
Python stacktrace
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem. Please only post stacktraces as text.

Step #1 - "Run Unit Tests": FAILED tests/test_artist.py::test_urls - requests.exceptions.HTTPError: 504 S...
Step #1 - "Run Unit Tests": =========================== short test summary info ============================
Step #1 - "Run Unit Tests": /usr/local/lib/python3.8/site-packages/requests/models.py:941: HTTPError
Step #1 - "Run Unit Tests": 
Step #1 - "Run Unit Tests": E           requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://api.chartmetric.com/api/token
Step #1 - "Run Unit Tests": >           raise HTTPError(http_error_msg, response=self)
Step #1 - "Run Unit Tests":         if http_error_msg:
Step #1 - "Run Unit Tests":     
Step #1 - "Run Unit Tests":             http_error_msg = u'%s Server Error: %s for url: %s' % (self.status_code, reason, self.url)
Step #1 - "Run Unit Tests":         elif 500 <= self.status_code < 600:
Step #1 - "Run Unit Tests":     
Step #1 - "Run Unit Tests":             http_error_msg = u'%s Client Error: %s for url: %s' % (self.status_code, reason, self.url)
Step #1 - "Run Unit Tests":         if 400 <= self.status_code < 500:
Step #1 - "Run Unit Tests":     
Step #1 - "Run Unit Tests":             reason = self.reason
Step #1 - "Run Unit Tests":         else:
Step #1 - "Run Unit Tests":                 reason = self.reason.decode('iso-8859-1')
Step #1 - "Run Unit Tests":             except UnicodeDecodeError:
Step #1 - "Run Unit Tests":                 reason = self.reason.decode('utf-8')
Step #1 - "Run Unit Tests":             try:
Step #1 - "Run Unit Tests":             # encodings. (See PR #3538)
Step #1 - "Run Unit Tests":             # isn't utf-8, we fall back to iso-8859-1 for all other
Step #1 - "Run Unit Tests":             # choose to localize their reason strings. If the string
Step #1 - "Run Unit Tests":             # We attempt to decode utf-8 first because some servers
Step #1 - "Run Unit Tests":         if isinstance(self.reason, bytes):
Step #1 - "Run Unit Tests":         http_error_msg = ''
Step #1 - "Run Unit Tests":     
Step #1 - "Run Unit Tests":         """Raises :class:`HTTPError`, if one occurred."""
Step #1 - "Run Unit Tests":     def raise_for_status(self):
Step #1 - "Run Unit Tests": 
Step #1 - "Run Unit Tests": self = <Response [504]>
Step #1 - "Run Unit Tests": 
Step #1 - "Run Unit Tests": _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Step #1 - "Run Unit Tests":     response.raise_for_status()
Step #1 - "Run Unit Tests": pycmc/credentials.py:182: in FetchAccessToken
Step #1 - "Run Unit Tests":     fetched = FetchAccessToken()
Step #1 - "Run Unit Tests": pycmc/credentials.py:103: in Update
Step #1 - "Run Unit Tests":     credentials.Update()  # first refresh our credentials
Step #1 - "Run Unit Tests": pycmc/credentials_manager.py:52: in <module>
Step #1 - "Run Unit Tests":     ???
Step #1 - "Run Unit Tests": <frozen importlib._bootstrap>:219: in _call_with_frames_removed
Step #1 - "Run Unit Tests":     ???
Step #1 - "Run Unit Tests": <frozen importlib._bootstrap_external>:783: in exec_module
Step #1 - "Run Unit Tests":     ???
Step #1 - "Run Unit Tests": <frozen importlib._bootstrap>:604: in _exec
Step #1 - "Run Unit Tests":     _bootstrap._exec(spec, module)
Step #1 - "Run Unit Tests": /usr/local/lib/python3.8/importlib/__init__.py:169: in reload
Step #1 - "Run Unit Tests":     reload(
Step #1 - "Run Unit Tests": pycmc/utilities.py:95: in RequestData
Step #1 - "Run Unit Tests":     data = utilities.RequestData(urlhandle, params)
Step #1 - "Run Unit Tests": pycmc/artist.py:308: in urls
Step #1 - "Run Unit Tests": _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
Step #1 - "Run Unit Tests": tests/test_artist.py:118: 
Step #1 - "Run Unit Tests": 
Step #1 - "Run Unit Tests": >       test = pycmc.artist.urls("3380",)
Step #1 - "Run Unit Tests":     def test_urls():
Step #1 - "Run Unit Tests": 
Step #1 - "Run Unit Tests": __________________________________ test_urls

Additional context
Add any other context about the problem here.

Fix artist.fanmetrics endpoint

Our fanmetrics endpoint is semi-broken, as you can see from the lack of valueCol usage, below.

def fanmetrics(cmid, start_date, dsrc="instagram", valueCol="followers"):
    """
    Query the Chartmetric API for artist fan metrics.
    https://api.chartmetric.com/api/artist/:id/stat/:source
    :param cmid:        string or int Chartmetric artist ID
    :param start_date:  string ISO date %Y-%m-%d
    :param dsrc:        string data source, choose from
                        'spotify', 'facebook', 'twitter', 'instagram',
                        'youtube', 'wikipedia', 'bandsintown', 'soundcloud',
			            'facebook_fans_by_country',
			            'facebook_storytellers_by_country' 
    :param valueCol:    string specific data field returned, choose from
                        'followers', 'popularity', 'listeners',
                        'talks', 'subscribers'
    :return:            nested dict, {valueCol: [fanmetrics]},
                        fanmetrics are dictionaries of time-series stats
    """
    urlhandle = f"/artist/{cmid}/stat/{dsrc}"
    params = {"since": start_date}
    data = utilities.RequestData(urlhandle, params)
    return utilities.RequestGet(data)

Furthermore, we should update the docs to reflect changes in upstream documentation.

Cleanup and fix failing tests

Some of our tests have been failing for a while and there were a few deprecations upstream since I've worked on this client.

Update deprecated tests

ig audience analysis
playlist evolution stats

Fix failing tests

Or comment & explain failing tests & remove functionality

  1 tests/test_album.py::test_charts FAILED                                  [  1%]
  2 tests/test_artist.py::test_charts FAILED                                 [ 13%]
  3 tests/test_charts.py::test_amazon_tracks FAILED                          [ 31%]
  4 tests/test_charts.py::test_amazon_albums FAILED                          [ 32%]
  5 tests/test_charts.py::test_cm_score_tracks FAILED                        [ 39%]
  6 tests/test_charts.py::test_cm_score_artists FAILED                       [ 40%]
  7 tests/test_charts.py::test_cm_score_albums FAILED                        [ 42%]
  8 tests/test_charts.py::test_youtube_artists FAILED                        [ 59%]
  9 tests/test_playlist.py::test_evolution FAILED                            [ 80%]
 10 tests/test_playlist.py::test_lists FAILED                                [ 81%]
 11 tests/test_track.py::test_charts FAILED                                  [ 88%]
 12 tests/test_track.py::test_get_track_ids FAILED                           [ 89%]

Cleanup tests

Our tests need a number of new things:

better test-wide confest.py w/reusable fixtures
class-based organization for some tests to minimize boilerplate and upgrade readability

In addition, our current tests need updating:
~~- [ ] add modern type hints~~

more cohesive testing of api combinations
removal of all magic numbers

Add Youtube Views and Market Coverage API Upstream Endpoint

Chart metric has added a few endpoints which need coverage, the youtube views endpoint being one of them.

https://api.chartmetric.com/apidoc/#api-Artist-GetArtistYoutubeMarketCoverageViews

Add rate limiting api and refactor credentials

The current credentials.py and other auth code don't consider rate limiting, which it should. We should implement a means of taking into account:

Header Name	Description
X-RateLimit-Limit	Number of requests allowed per minute on current plan
X-RateLimit-Remaining	Number of requests remaining in current time period (1 minute)
X-RateLimit-Reset	Time at which rate limit will reset

as from the docs

DEPRECATION/ARCHIVAL NOTICE -- 📞 for maintainer(s)

To the community:

As you've probably noticed, this repo hasn't been updated or given much love for a very long time.

As we at Tincre, formerly Musicfox, aren't using this library in production we can no longer support (or appear to support, let's be real) the pycmc api wrapper.

We would love to hand over the keys if anyone's interested; otherwise expect this repo to remain archived with no future updates. If that someone is you, please reach out via [email protected].

Cover new 2.0 API endpoints

We need to ensure full coverage of all endpoints. As Chartmetric has continued to update its API over time, this client lib needs an update!

Documentation links

radio endpoints
~~api-search~~
Add charts Shazam parameters
Add fan metrics weekly_diff, weekly_diff_percent, monthly_diff, monthly_diff_percent
Auth changes
Search cities
YouTube Audience w/date
Instagram audience w/date
TikTok endpoints
See more below!

Todos

All radio endpoints covered in client
All radio endpoints tested under tests/
Search covered in client
Search endpoint fully unit tested
Regenerate documentation
Add new functionality to README.md

Notes

We should make sure to test all options for these new endpoints, as some in the past were known to return atypical HTTP responses when data is not available versus other endpoints which return typical responses.

[Feature]: Query Builder, Query, and Query Exception Classes

Is your feature request related to a problem? Please describe.
Each endpoint uses a chain of functions, which, after months of scrappy use and rapid prototyping, throw some side effects and occasionally spit errors.

This is sub-optimal in production environments, as consistency and reasonable error propagation are necessities.

Describe the solution you'd like
I'd like to see a class for each query request with proper error encapsulation. This will likely need a query exceptions class, in tandem.

Objectives:

Every API call is self-contained
Every API call is identical, other than query and updated header authorization, if necessary
Every query is built by exactly the same process
Every client function (e.g. artist.charts) instantiates a Query via the constructor params, which create a QueryBuilder and call the API

Describe alternatives you've considered
The semi-"functional" approach we take now is frustrating to use.

[BUG]: psutil is not used and breaks some macintosh builds

Describe the bug

Building deps for me via Pipenv failed on a new M1 macbook air (as of Jan 2021) in x86 emulation mode with clang and gcc available.

We currently rely on the excellent, but third party psutil library. Though tests exist for its use in the utilities.FindProcess method, we are not using this and will definitely not be using this for the 1.0.0 release.

remove psutil

To Reproduce
Steps to reproduce the behavior:

Get a fresh copy of pycmc via git clone [email protected]:musicfox/pycmc && cd pycmc
Install fresh dependencies: pipenv shell then pipenv install.
Receive error:

Installing initially failed dependencies...
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]:       retcode = do_install(
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]:       do_init(
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]:       do_install_dependencies(
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]:       batch_install(
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]:       _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]:   File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting psutil==5.7.3
[pipenv.exceptions.InstallError]:   Using cached psutil-5.7.3.tar.gz (465 kB)
[pipenv.exceptions.InstallError]: Building wheels for collected packages: psutil
[pipenv.exceptions.InstallError]:   Building wheel for psutil (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for psutil (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   ERROR: Command errored out with exit status 1:
[pipenv.exceptions.InstallError]:    command: /Users/jason/.local/share/virtualenvs/pycmc-694t731m/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"'; __file__='"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-wheel-6rr07mlp
[pipenv.exceptions.InstallError]:        cwd: /private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/
[pipenv.exceptions.InstallError]:   Complete output (46 lines):
[pipenv.exceptions.InstallError]:   running bdist_wheel
[pipenv.exceptions.InstallError]:   running build
[pipenv.exceptions.InstallError]:   running build_py
[pipenv.exceptions.InstallError]:   creating build
[pipenv.exceptions.InstallError]:   creating build/lib.macosx-11.1-x86_64-3.8
[pipenv.exceptions.InstallError]:   creating build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_pswindows.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_common.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/__init__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_psosx.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_psbsd.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_psaix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_pslinux.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_compat.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_psposix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   copying psutil/_pssunos.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   creating build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_contracts.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_connections.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/runner.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_unicode.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_misc.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_posix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_linux.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_sunos.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/__init__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_aix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_process.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_bsd.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_system.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_osx.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_memleaks.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_windows.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/__main__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   copying psutil/tests/test_testutils.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:   warning: build_py: byte-compiling is disabled, skipping.
[pipenv.exceptions.InstallError]:   
[pipenv.exceptions.InstallError]:   running build_ext
[pipenv.exceptions.InstallError]:   building 'psutil._psutil_osx' extension
[pipenv.exceptions.InstallError]:   creating build/temp.macosx-11.1-x86_64-3.8
[pipenv.exceptions.InstallError]:   creating build/temp.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:   creating build/temp.macosx-11.1-x86_64-3.8/psutil/arch
[pipenv.exceptions.InstallError]:   creating build/temp.macosx-11.1-x86_64-3.8/psutil/arch/osx
[pipenv.exceptions.InstallError]:   clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=573 -DPSUTIL_OSX=1 -I/Users/jason/.local/share/virtualenvs/pycmc-694t731m/include -I/Users/jason/.pyenv/versions/3.8.7/include/python3.8 -c psutil/_psutil_common.c -o build/temp.macosx-11.1-x86_64-3.8/psutil/_psutil_common.o
[pipenv.exceptions.InstallError]:   xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
[pipenv.exceptions.InstallError]:   error: command 'clang' failed with exit status 1
[pipenv.exceptions.InstallError]:   ----------------------------------------
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for psutil
[pipenv.exceptions.InstallError]:   Running setup.py clean for psutil
[pipenv.exceptions.InstallError]: Failed to build psutil
[pipenv.exceptions.InstallError]: Installing collected packages: psutil
[pipenv.exceptions.InstallError]:     Running setup.py install for psutil: started
[pipenv.exceptions.InstallError]:     Running setup.py install for psutil: finished with status 'error'
[pipenv.exceptions.InstallError]:     ERROR: Command errored out with exit status 1:
[pipenv.exceptions.InstallError]:      command: /Users/jason/.local/share/virtualenvs/pycmc-694t731m/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"'; __file__='"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-record-ig83ij_3/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jason/.local/share/virtualenvs/pycmc-694t731m/include/site/python3.8/psutil
[pipenv.exceptions.InstallError]:          cwd: /private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/
[pipenv.exceptions.InstallError]:     Complete output (46 lines):
[pipenv.exceptions.InstallError]:     running install
[pipenv.exceptions.InstallError]:     running build
[pipenv.exceptions.InstallError]:     running build_py
[pipenv.exceptions.InstallError]:     creating build
[pipenv.exceptions.InstallError]:     creating build/lib.macosx-11.1-x86_64-3.8
[pipenv.exceptions.InstallError]:     creating build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_pswindows.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_common.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/__init__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_psosx.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_psbsd.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_psaix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_pslinux.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_compat.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_psposix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     copying psutil/_pssunos.py -> build/lib.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     creating build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_contracts.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_connections.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/runner.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_unicode.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_misc.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_posix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_linux.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_sunos.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/__init__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_aix.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_process.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_bsd.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_system.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_osx.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_memleaks.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_windows.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/__main__.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     copying psutil/tests/test_testutils.py -> build/lib.macosx-11.1-x86_64-3.8/psutil/tests
[pipenv.exceptions.InstallError]:     warning: build_py: byte-compiling is disabled, skipping.
[pipenv.exceptions.InstallError]:     
[pipenv.exceptions.InstallError]:     running build_ext
[pipenv.exceptions.InstallError]:     building 'psutil._psutil_osx' extension
[pipenv.exceptions.InstallError]:     creating build/temp.macosx-11.1-x86_64-3.8
[pipenv.exceptions.InstallError]:     creating build/temp.macosx-11.1-x86_64-3.8/psutil
[pipenv.exceptions.InstallError]:     creating build/temp.macosx-11.1-x86_64-3.8/psutil/arch
[pipenv.exceptions.InstallError]:     creating build/temp.macosx-11.1-x86_64-3.8/psutil/arch/osx
[pipenv.exceptions.InstallError]:     clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=573 -DPSUTIL_OSX=1 -I/Users/jason/.local/share/virtualenvs/pycmc-694t731m/include -I/Users/jason/.pyenv/versions/3.8.7/include/python3.8 -c psutil/_psutil_common.c -o build/temp.macosx-11.1-x86_64-3.8/psutil/_psutil_common.o
[pipenv.exceptions.InstallError]:     xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
[pipenv.exceptions.InstallError]:     error: command 'clang' failed with exit status 1
[pipenv.exceptions.InstallError]:     ----------------------------------------
[pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 1: /Users/jason/.local/share/virtualenvs/pycmc-694t731m/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"'; __file__='"'"'/private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-install-kxq14ar7/psutil_268e8fa3367f4dfd905fb045327f6430/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/pd/l3x_zl3d1_jgw0t0hrx6kv9r0000gn/T/pip-record-ig83ij_3/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jason/.local/share/virtualenvs/pycmc-694t731m/include/site/python3.8/psutil Check the logs for full command output.
ERROR: Couldn't install package: psutil

Expected behavior
I expect to be able to install the environment dependencies via any standard method (virtualenv, Pipenv, or even Poetry using requirements.txt).

[BUG]: 502 Bad Gateway during credentials process

Describe the bug

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
    super().init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/application/app.py", line 27, in <module>
    import utilities
  File "/application/utilities.py", line 6, in <module>
    import pycmc
  File "/usr/local/lib/python3.8/site-packages/pycmc/__init__.py", line 7, in <module>
    from . import credentials_manager
  File "/usr/local/lib/python3.8/site-packages/pycmc/credentials_manager.py", line 52, in <module>
    credentials.Update()  # first refresh our credentials
  File "/usr/local/lib/python3.8/site-packages/pycmc/credentials.py", line 103, in Update
    fetched = FetchAccessToken()
  File "/usr/local/lib/python3.8/site-packages/pycmc/credentials.py", line 182, in FetchAccessToken
    response.raise_for_status()
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://api.chartmetric.com/api/token

To Reproduce
See above stacktrace, however, it's obvious that something in the update process caused the error. ⚠️ The error only occurred for two minutes on November 25th between 9:15 and 9:16 CST.

Expected behavior
No error.

Additional context
The credentials update process is byzantine and could use an update. Possibly related to #25 !

Upgrade Python dependencies, rename, and release on PyPi

We need a routine dependency upgrade and complementary updated release on PyPi.

migrate project to pipenv
update project dependencies
release on PyPi
new name

Library cleanup

Lots and lots of general cleanup. Structural, styling, inline comments. All. Of. It. Clean this awful mess!

[Feature]: Credentials handling improvement suggestions

Hi, I think there are a couple issues with the current credential management system. For most of them I'd be happy to implement fixes but I think it is better to first discuss them before coding :)

1. Superfluous token refreshes

The current pattern for API calls is the following:

api_call (e.g. pycmc.artist.metadata())
    utilities.RequestData
        importlib.reload(credentials_manager)
            credentials.Update
                credentials.FetchAccessToken
    utilities.RequestGet

Authentication is reperformed before each API call. Hence all functions in the library would be twice as fast if the token was only refreshed when needed (i.e. Error 401).

2. Rate limit handling

I know #15 already references this but I still mention it here because I think it should happen at the same place than token refreshing. In other words, the automatic handling of error 401 (unauthorized) could also handle error 429 (too many requests).

3. Rigid environment variable flow

I think the current mandatory JSON environment variable is not very user-friendly. First, the only needed input from user is the refresh token; the empty dict structure could automatically be infered. Second, I'd say it would be better if users had an alternative way to connect, cause maybe they don't want to use an environment variable.

To solve this, I would imagine a lazy design that does not authenticate at import time but only on first API call, coupled with a new pycmc.authenticate(refresh_token) function. Authentication would use CMCREDENTIALS by default until the user calls authenticate at some point.

4. Impossibility to manage several Chartmetric accounts

I'd like to develop applications that have several users, each with their own API token (think of a web server doing Chartmetric requests in place of the end-user). The current global variable system does not allow this. The way I would redesign the API would be an object-oriented interface similar to for instance what they do at spotipy:

import pycmc

client = pycmc.ChartmetricClient(refresh_token)
client.artists.metadata(...)
client.search_engine.search(...)

That way the application could handle several clients at a time.

I think number 1-3 are not breaking changes and are not that hard to implement. Number 4 is a breaking change; even though the API spirit is the same, everything must refactored into this object-oriented interface. I guess you have the right to do such a change when switching to 1.0 but maybe you don't want to since you must already have a bunch of apps running on the current API...

However, if the amount of required coding frightens you, please note that:

I already have similar solutions implemented on my side that I could easily transpose to pycmc.
I'd be very happy to spend time on this project because I use the Chartmetric API a lot at my company and I'd appreciate having a stable public library for this rather than coding my custom solutions all the time.

Add CI/CD Pipeline

We need a CI/CD pipeline added here. Likely need to use google cloud build but checkout jenkins etc since this is open source.

Migrate docstrings to Markdown format

Our docstrings get an upgrade

What's a docstring you say?

"""
I am a docstring.
"""
"""
# I am
## therefore 
### a markdown
`docstring`
"""

The docstring is just the portion between three quotation marks """. They're used to generate documentation and for inline help in IDEs and other things.

For example...

Here's a nice markdown docstring rendering of the docstring in pycm/utilities for the function strDateToday.

Files that need looking at

Ignore the other items but here's the list:

Filename	Hit	Miss	%
pycm/init.py	13	0	100%
pycm/album.py	35	25	29%
pycm/artist.py	70	55	21%
pycm/background.py	26	5	81%
pycm/chart_cleaners.py	212	202	5%
pycm/charts/init.py	0	0	100%
pycm/charts/amazon.py	12	8	33%
pycm/charts/applemusic.py	17	12	29%
pycm/charts/beatport.py	7	4	43%
pycm/charts/cm_score.py	16	12	25%
pycm/charts/deezer.py	7	4	43%
pycm/charts/itunes.py	17	12	29%
pycm/charts/qq.py	7	4	43%
pycm/charts/shazam.py	13	9	31%
pycm/charts/soundcloud.py	7	4	43%
pycm/charts/spotify.py	14	8	43%
pycm/charts/youtube.py	24	16	33%
pycm/credentials.py	66	23	65%
pycm/credentials_manager.py	30	11	63%
pycm/curator.py	23	17	26%
pycm/playlist.py	27	21	22%
pycm/recommendation.py	12	10	17%
pycm/search_engine.py	10	8	20%
pycm/track.py	39	31	21%
pycm/utilities.py	36	13	64%

Doing one together

Clone the repo: git clone [email protected]:musicfox/pycm.
Update your repo and start a new branch (from inside the pycm repo project directory): git pull origin develop && git flow feature start markdown-docstrings.
Open up album.py under the pycm source directory (pycm/pycm).
Let's fix the docstring in the first charts function, which looks like:

def charts(stype, cmid, start_date, end_date=None):
    """
    Query the charts for the given album of a selected streamer type. 

    https://api.chartmetric.com/api/album/:id/:type/charts
    
    :param stype:           string streaming platform, choose from
                            'applemusic', 'itunes' or 'amazon'
    :param cmid:            string or int chartmetric album ID
    :param start_date:      string start data in ISO format
    :param end_date:        string end date in ISO format

    :return:                list of dictionaries of the chart for
                            the given album
    """
    logging.info(
        f"This is known to have authentication issues when "
...

Now alter the doctring to look like:

    """
    # `charts`
    Query the charts for the given album of a selected streamer type. 

    https://api.chartmetric.com/api/album/:id/:type/charts
    
    ## Parameters
    - `stype`: string streaming platform, choose from
                   'applemusic', 'itunes' or 'amazon'
    - `cmid`: string or int chartmetric album ID
    - `start_date`: string start data in ISO format
    - `end_date`: string end date in ISO format

   ## Returns
    A list of dictionaries of the chart for the given album.
    """

Save your work and move to the next function. Repeat. Loop. Almost.Forever. Until you're done with one file.
Stage + commit the result: git add path/to/my.file && git commit -m ":pencil: update docstrings #11"
Open up a new file and repeat the entire process until finished.
After your last one, commit + push: git push origin feature/markdown-docstrings