floscha / tinycards-python-api Goto Github PK

The Deck class has two properties front_language and back_language which aren't used anywhere. Not even in the JSON converter itself. They also appear nowhere in the messages used by the Tinycards web application. Therefore, they can safely be deleted.

With the current TravisCI configuration, pytest is run on the top-level folder. This includes test scripts and build artifacts in the coverage calculation, making the reported coverage smaller than it actually is.

Originally, we used pytest --cov tinycards-python-api, to only include the actual source code. Since this caused build problems, it was changed to the root path. However, pytest should again only be run on the source code while still maintaining a successful build.

Currently, all functionality that requires authentification fails because Tinycards appears to no longer work with classical sessions but with JSON Web Tokens instead. Consequently, the rest_api.py module needs to be refactored so that authentification can be done through JWT.

The unit tests of the build break because of some errors happening while logging in. When adding some sleep time between tests, the chance that all tests pass increases but still does not always work. Figure out where the problem lies and fix accordingly.

Add a method to search for decks on Tinycards (https://tinycards.duolingo.com/api/1/searchables) with the following parameters:

1. query - The used search term(s).
2. useFuzzySearch - Whether or not to use fuzzy search. Can be true or false.
3. types - What entity to search for. Can be DECK, DECK_GROUP or USER.
4. limit - What number of results to should be returned.
5. page - The page to return when returning more than limit results (zero-indexed).

Monitor coverage through https://coveralls.io/

Currently JSON marshalling involves loads of boilerplate code both for the constructor of the model classes themselves and even worse for converting JSON to objects and back which has been implemented for each model class individually. This is bloating the code base and makes it hard to maintain. Instead, the following two improvements shall be made:

1. Use Python's data classes to remove trivial constructor methods.
2. Implement a more general solution to convert between JSON and Python objects.

I've dumped a deck to JSON using the deck_to_json function, however the resulting cards in the deck only have the text for front/back included with no image information. Is there a way to source the images of each card as well? I know the Deck class contains this information when you print a deck from a users collection.

Add code quality metrics with https://landscape.io.

Why?

This would be useful to configure the user experience when decks are used to learn & test one-self. In particular,

• decks to learn languages using special characters may be painful to use if typing is enabled ;
• decks with "secondary" facts (e.g. sample sentences) on the back of the cards may want to only test using the front values (since Tinycards doesn't allow one to label these "secondary" facts as "not used for tests" yet).

What? / How?

Default:

• Request:

  {
    "description": "Test description",
    "blacklistedSideIndices": "[]",
    "blacklistedQuestionTypes": "[]",
    "gradingModes": "[]",
    "fromLanguage": "en",
    "ttsLanguages": "[]",
    "private": false,
    "shareable": false
  }

  N.B.: complex JSON values are stringified.

• Response:

  {
    "description": "Test description",
    "blacklistedSideIndices": [],
    "blacklistedQuestionTypes": [],
    "gradingModes": [],
    "fromLanguage": "en",
    "ttsLanguages": [],
    "private": false,
    "shareable": false
  }

Shareable only via link

• Request:

  {
    "private": true,
    "shareable": true
  }

• Response:

  {
    "compactId": "LEbBQJFU",
    "slug": "test",
    "private": true,
    "shareable": true
  }

  The compactId and slug fields can then be used to generate an URL of this form: https://tiny.cards/decks/LEbBQJFU/test, i.e.: https://tiny.cards/decks/<compactId>/<slug>.

Test knowledge with "front" values only

• Request:

  {
    "blacklistedSideIndices": "[1]"
  }

• Response:

  {
    "blacklistedSideIndices": [
      1
    ]
  }

Test knowledge with "back" values only

• Request:

  {
    "blacklistedSideIndices": "[0]"
  }

• Response:

  {
    "blacklistedSideIndices": [
      0
    ]
  }

Test without typing answers

• Request:

  {
    "blacklistedQuestionTypes": "[[\"ASSISTED_PRODUCTION\",\"PRODUCTION\"],[\"ASSISTED_PRODUCTION\",\"PRODUCTION\"]]"
  }

• Response:

  {
    "blacklistedQuestionTypes": [
      [
        "ASSISTED_PRODUCTION",
        "PRODUCTION"
      ],
      [
        "ASSISTED_PRODUCTION",
        "PRODUCTION"
      ]
    ]
  }

Do not accept typos

• Request:

  {
    "blacklistedQuestionTypes": "[]",
    "gradingModes": "[\"NO_TYPOS\",\"NO_TYPOS\"]"
  }

• Response:

  {
    "blacklistedQuestionTypes": [],
    "gradingModes": [
      "NO_TYPOS",
      "NO_TYPOS"
    ]
  }

Text-to-speech

• Request:

  {
    "ttsLanguages": "[\"en\",\"ja\"]"
  }

• Response:

  {
    "ttsLanguages": [
      "en",
      "ja"
    ]
  }

N.B.: first language is the front of the cards, second language in the back of the cards.

The "fullname" field is optional on tinycards. If not set it will be empty in the API response.

For example this line inrest_api.py fails if no fullname is set.

Calling client.update_deck(deck) results in a 401 Unauthorized response from the API.

Exists some function to create a new deck? If exists how to use this?

I'm trying to authenticate and I get the following error when creating the Tinycards object:

In [14]: client = Tinycards(username, password)

KeyError Traceback (most recent call last)
<ipython-input-14-bec297a8003b> in <module>()
----> 1 client = Tinycards(username, password)

/usr/lib/python3.6/site-packages/tinycards/client/tinycards.py in __init__(self, identifier, password)
     24         """Initialize a new instance of the Tinycards class."""
     25         self.data_source = RestApi()
---> 26         self.user_id = self.data_source.login(identifier, password)
     27 
     28     # --- Read user info.

/usr/lib/python3.6/site-packages/retrying.py in wrapped_f(*args, **kw)
     47             @six.wraps(f)
     48             def wrapped_f(*args, **kw):
---> 49                 return Retrying(*dargs, **dkw).call(f, *args, **kw)
     50 
     51             return wrapped_f

/usr/lib/python3.6/site-packages/retrying.py in call(self, fn, *args, **kwargs)
    204 
    205             if not self.should_reject(attempt):
--> 206                 return attempt.get(self._wrap_exception)
    207 
    208             delay_since_first_attempt_ms = int(round(time.time() * 1000)) - start_time

/usr/lib/python3.6/site-packages/retrying.py in get(self, wrap_exception)
    245                 raise RetryError(self)
    246             else:
--> 247                 six.reraise(self.value[0], self.value[1], self.value[2])
    248         else:
    249             return self.value

/usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    691             if value.__traceback__ is not tb:
    692                 raise value.with_traceback(tb)
--> 693             raise value
    694         finally:
    695             value = None

/usr/lib/python3.6/site-packages/retrying.py in call(self, fn, *args, **kwargs)
    198         while True:
    199             try:
--> 200                 attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
    201             except:
    202                 tb = sys.exc_info()

/usr/lib/python3.6/site-packages/tinycards/networking/rest_api.py in login(self, identifier, password)
     67         set_cookie_headers = {
     68             k: v for (k, v) in
---> 69             [c.split('=') for c in r.headers['set-cookie'].split('; ')]
     70         }
     71         self.jwt = set_cookie_headers.get('jwt_token')

/usr/lib/python3.6/site-packages/requests/structures.py in __getitem__(self, key)
     50 
     51     def __getitem__(self, key):
---> 52         return self._store[key.lower()][1]
     53 
     54     def __delitem__(self, key):

KeyError: 'set-cookie'

Add a method to get a user's favorite decks like used by https://tinycards.duolingo.com/favorites.

Also, add methods to add and remove favorites.

Does the API provide an interface for accessing Private or Draft decks?

As hinted in #57, I believe some properties of the Deck class which are taken straight from Tinycards' internal JSON schema, are rather unintuitive for users not familiar with this schema. Instead, the properties could be based on the Deck Settings dialog of Tinycards' web UI, shown in the screenshot below:

To start with, I suggest the following properties to be replaced (the intended bullets are the suggested replacements):

• blacklisted_side_indices can be set to the indices 0 or 1.
  • quiz_learners_on would take one of the following values: both sides (default), front only, back only. Optionally the string could be lowercased to also enable inputs exactly like they appear in the UI, e.g. Both sides.
• blacklisted_question_types is currently a rather over-complicated list of lists of string constants (ASSISTED_PRODUCTION or PRODUCTION) that are quite hard to make sense of by themselves.
  • enable typing questions could simply be a boolean. The downside of this simplification is that it is global and cannot have different values for front and back sides. However this feature is also not available in the Tinycards UI (to my knowledge at least), so it's debatable wether or not this option should actually exist.
• grading_modes is a list of string constants ('NO_TYPOS or nothing).
  • allow typos in responses could also just be a boolean.

Additionally, the tts_languages property could accept not only the language codes (which many I believe have to look up), but instead also the full name of the language, again internally lowercased to e.g. accept both English and english.

In the end, I believe this issue comes down to a design choice whether the Python API should be closer to the web UI or the internal JSON representations. While the tts_languages property could accept both languages codes as well as the full names, the other properties seem more mutual exclusive. @marccarre do you have an opinion on this?

Hi, great work and thanks for creating tinycards api!

I've realised that users may not have 'learningLanguage' key set in their user data response, if they have never signed in to duolingo, in my case i had created my account on tinycards app so this account was never used on duolingo app. Once signed in to duolingo and enrolled any language course, then KeyError stopped happening since my data had 'learningLanguage:sv'

Steps to reproduce;

• Create a new account from tinycards.duolingo.com ( not duolingo.com )
• call get_user_info()
• KeyError below happens...
• log in to duolingo.com with the new account, enroll a course
• call get_user_info(), KeyError won't happen anymore

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/eralp/Documents/markdown-tinycards-sync/sample/__main__.py", line 9, in <module>
    markdown_sync(user_name, password)
  File "/Users/eralp/Documents/markdown-tinycards-sync/sample/core.py", line 19, in markdown_sync
    user = client.get_user_info()
  File "/Users/eralp/Documents/markdown-tinycards-sync/env/lib/python3.6/site-packages/tinycards/client/tinycards.py", line 37, in get_user_info
    user_info = self.data_source.get_user_info(self.user_id)
  File "/Users/eralp/Documents/markdown-tinycards-sync/env/lib/python3.6/site-packages/tinycards/networking/rest_api.py", line 86, in get_user_info
    user_info = json_converter.json_to_user(json_response)
  File "/Users/eralp/Documents/markdown-tinycards-sync/env/lib/python3.6/site-packages/tinycards/networking/json_converter.py", line 16, in json_to_user
    learning_language=json_data['learningLanguage'],
KeyError: 'learningLanguage'

Remove the pytest package from the project's dependencies and only include for CI.

Just install it with pip and try the import tinycards command.
Failed with "ImportError: No module named networking.rest_api"

Log below:

Looking in indexes: https://pypi.python.org/simple, https://pypi.apple.com/simple
Collecting tinycards
Downloading https://files.pythonhosted.org/packages/53/c3/f6be435a81618a19a5f6427fc2d4849dabac497412a510b834444ffc52f0/tinycards-0.23.tar.gz
Collecting requests (from tinycards)
Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB)
100% |████████████████████████████████| 61kB 3.5MB/s
Collecting retrying==1.3.3 (from tinycards)
Downloading https://files.pythonhosted.org/packages/44/ef/beae4b4ef80902f22e3af073397f079c96969c69b2c7d52a57ea9ae61c9d/retrying-1.3.3.tar.gz
Collecting idna<2.9,>=2.5 (from requests->tinycards)
Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 17.3MB/s
Collecting chardet<3.1.0,>=3.0.2 (from requests->tinycards)
Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
100% |████████████████████████████████| 143kB 6.3MB/s
Collecting certifi>=2017.4.17 (from requests->tinycards)
Downloading https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl (154kB)
100% |████████████████████████████████| 163kB 7.0MB/s
Collecting urllib3<1.25,>=1.21.1 (from requests->tinycards)
Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
100% |████████████████████████████████| 122kB 19.5MB/s
Requirement already satisfied: six>=1.7.0 in /Library/Python/2.7/site-packages (from retrying==1.3.3->tinycards) (1.11.0)
Installing collected packages: idna, chardet, certifi, urllib3, requests, retrying, tinycards
Running setup.py install for retrying ... done
Running setup.py install for tinycards ... done
Successfully installed certifi-2018.11.29 chardet-3.0.4 idna-2.8 requests-2.21.0 retrying-1.3.3 tinycards-0.23 urllib3-1.24.1
$ python
Python 2.7.10
Type "help", "copyright", "credits" or "license" for more information.

import tinycards
Traceback (most recent call last):
File "", line 1, in
File "/Library/Python/2.7/site-packages/tinycards/init.py", line 1, in
from tinycards.client import Tinycards
File "/Library/Python/2.7/site-packages/tinycards/client/init.py", line 1, in
from .tinycards import Tinycards
File "/Library/Python/2.7/site-packages/tinycards/client/tinycards.py", line 1, in
from tinycards.networking.rest_api import RestApi
ImportError: No module named networking.rest_api

Add unit tests to validate JSON conversion.

What? Why?

#49 added support for JPEG deck covers loaded from local files.
However, it would be nice to

• be able to set covers from existing image URLs (as suggested here by @floscha),
• handle other formats (e.g. PNG).

In order to do this, the coverImageUrl field would have to be supported.

How?

Below is an extract of requests/responses with the fields relevant to covers, when manually interacting with decks on https://tinycards.duolingo.com.

Observations

• Invariant 1.: before a custom cover is set, coverImageUrl is always null.
• Invariant 2.: in deck updates, even for deck cover updates, coverImageUrl is always sent back, with the latest value received from the server-side, i.e. null if no custom cover, or the last known URL to server-side otherwise.
• Invariant 3.: responses from POSTs/PATCHs requests and subsequent GET requests are always equivalent w.r.t. to imageUrl and coverImageUrl.

1. Deck creation w/o cover:

POST https://tinycards.duolingo.com/api/1/decks

• no imageFile field
• no imageUrl field
• no imageCoverUrl field
  Response:

{
  "coverImageUrl": null,
  "imageUrl": "https://s3.amazonaws.com/tinycards/image/16cb6cbcb086ae0f622d1cfb7553a096"
}

2. Subsequent deck loading:

GET https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": null,
  "imageUrl": "https://s3.amazonaws.com/tinycards/image/16cb6cbcb086ae0f622d1cfb7553a096",
}

3. Deck cards update:

PATCH https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": null
}

Response:

{
"coverImageUrl": null,
"imageUrl": "https://s3.amazonaws.com/tinycards/image/16cb6cbcb086ae0f622d1cfb7553a096"
}

4. Subsequent deck loading:

GET https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": null,
  "imageUrl": "https://s3.amazonaws.com/tinycards/image/16cb6cbcb086ae0f622d1cfb7553a096",
}

5. Deck cover update from local file:

PATCH https://tinycards.duolingo.com/api/1/decks/{deckID}

------<separator>
Content-Disposition: form-data; name="imageFile"; filename="cover.jpg"
Content-Type: image/jpeg
[...bytes...]
------<separator>
Content-Disposition: form-data; name="coverImageUrl"

null

Response:

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353"
}

6. Subsequent deck loading:

GET https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353"
}

7. Deck cards update

PATCH https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353"
}

Response:

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353"
}

8. Subsequent deck loading:

GET https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353"
}

9. Deck cover update from Tinycards' image library:

PATCH https://tinycards.duolingo.com/api/1/decks/{deckID}

------<separator>
Content-Disposition: form-data; name="imageAttribution"

<source URL of the image>
------<separator>
Content-Disposition: form-data; name="imageFile"; filename="cover.jpg"
Content-Type: image/png
[...bytes...]
------<separator>
Content-Disposition: form-data; name="coverImageUrl"

https://d9np3dj86nsu2.cloudfront.net/image/9f6a54aef25a5184620e5762c8430353

Response:

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/ada59f16c4e4d8b09f7eee9104ce6bc7",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/ada59f16c4e4d8b09f7eee9104ce6bc7"
}

10. Subsequent deck loading:

GET https://tinycards.duolingo.com/api/1/decks/{deckID}

{
  "coverImageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/ada59f16c4e4d8b09f7eee9104ce6bc7",
  "imageUrl": "https://d9np3dj86nsu2.cloudfront.net/image/ada59f16c4e4d8b09f7eee9104ce6bc7"
}

Why?

One may want to create cards with more than two facts, i.e. not just "front" and "back", but several "back" facts, like possible under the Tinycards UI. Currently, there isn't any "high-level" API covering this use-case, which forces one to do something along the lines of the below:

cards = [...]  # source cards, containing multiple "fact" strings.
fields = set(['question', 'answer', 'sentence', '...'])  # the various fields for the multiple "facts" to support.
username, password = get_credentials()
tinycards = Tinycards(username, password)
user_id = tinycards.user_id
deck = Deck('Complex deck', user_id)
deck = tinycards.create_deck(deck)
for card in cards:
    front = Concept(fact=Fact(card.facts.pop('question')), user_id=user_id)
    front_side = Side(concepts=front, user_id=user_id)
    back = [Concept(fact=Fact(card.facts.pop('answer')), user_id=user_id)]
    back.extend([Concept(fact=card.facts[field], user_id=user_id) for field in fields if field in card.facts])
    back_side = Side(concepts=back, user_id=user_id)
    deck.cards.append(Card(front_side, back_side, user_id=user_id))
tinycards.update_deck(deck)

N.B.: this also happens not to be documented anywhere at the moment, unlike the simple case of two facts per card documented here:

How?

Either relax the validation done in:

I'm trying to edit a tinycards deck. But cards object in deck is empty. Is this expected?

`
import tinycards

client = tinycards.Tinycards('[email protected]', 'my-password')

all_decks = client.get_decks()

for this_deck in all_decks:
print(this_deck.title)
print(this_deck.cards)
for this_card in this_deck.cards:
print(this_card.front)
`

Hey, really nice API !

I'm trying to do a simple test by getting the user info, but it was throwing me:
--> 'TypeError: 'dict' object is not callable' in 'json_to_user'.
Then I realized that you trying to get "learning_languages" from JSON with ( ) instead [ ].

I replaced here and had no error, but no data is shown

Add a .from_tuple() method to the Card class, instead of the Deck classes add_card() method.

Add methods to add and remove subscriptions (https://tinycards.duolingo.com/api/1/users/{userName}/subscriptions).

Use HTTP POST to add a subscription and DELETE to remove an existing subscription.

Description

It is currently not possible to create decks with cover images.

Supporting this feature would allow covers to be set programmatically, instead of having to set these manually once the decks are created via tinycards-python-api.

Current state

Currently, Deck's cover argument is simply ignored:

Publish the project on PyPI once #1, #2, #3, and #4 are implemented.

Improve the usage example in the README.md.

Instead of listing various random functions, better give one more complete example.

Setup Travis CI to enable automated builds and tests.

Hello everybody!

Thank you very much for this API, it is really useful.
However I have one question: I have found the average_freshness argument in the SearchableData class and if I'm right this refers to the freshness of each set inside a deck.

I have marked these on the screenshot.

If I am correct, how can this average_freshness be accessed using the API?

Add a command line interface to enable interacting with Tinycards without the need to actually write code.

Commands could include

• tinycards login
• tinycards import file-name
• tinycards export file-name
• tinycards decks {list, create}

Add a method to get the current trends from Tinycards (https://tinycards.duolingo.com/api/1/trendables) with the following parameters:

1. types - What entity to search for. Can be DECK, DECK_GROUP or USER.
2. limit - What number of results to should be returned.
3. page - The page to return when returning more than limit results (zero-indexed).

Make all model classes available from the module level, to e.g. allow the follwing:

import tinycards

test_deck = tinycards.Deck('Test Deck')

A recent CI run results in KeyError: 'blacklistedQuestionTypes'. Check if the Tinycards REST API has changed and adapt the code accordingly.

It would be nice to be able to import and export decks using JSON. This would allow easy editing in any text editor we like, and also allow cross-compability with other tools when exporting.

Automatically deploy changes to PyPI as described here: https://docs.travis-ci.com/user/deployment/pypi/

Currently, one large part of the test suite consists of integrations tests which involve the real Tinycards backend. However, this imposes several limitations:

• Tests cannot be run offline.
• Tests cannot be run in parallel and are thus pretty slow.
• Only one Python version can be tested with Travis CI.

Consequently, those HTTP requests shall be mocked, which would solve all problems listed above, but unfortunately also means that actual Tinycards API changes remain undetected. We will however assume that the Tinycards API is stable enough, so that this issue can be considered at a later point in time.

#49 and #53 clashed and master is broken as-of a24eb63.
Among others, integration tests fail with:

>       updated_deck = self.get_deck(deck.id, deck.user_id)
E       AttributeError: 'Deck' object has no attribute 'user_id'

tinycards/networking/rest_api.py:277: AttributeError