Giter Club home page Giter Club logo

joppy's Introduction

joppy

Python interface for the Joplin data API.

build lint tests codecov

https://img.shields.io/badge/Joplin-2.14.17-blueviolet Python version

💻 Installation

From pypi:

pip install joppy

From source:

git clone https://github.com/marph91/joppy.git
cd joppy
pip install .

🔧 Usage

General function description

  • add_<type>(): Create a new element.
  • delete_<type>(): Delete an element by ID.
  • get_<type>(): Get an element by ID.
  • get_all_<type>(): Get all elements of a kind.
  • modify_<type>(): Modify an elements property by ID.
  • search_all(): Search elements using joplins search engine.

For details, consult the implementation, joplin documentation or create an issue.

💡 Example snippets

Start joplin and get your API token. Click to expand the examples.

Get all notes
from joppy.api import Api

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

# Get all notes. Note that this method calls get_notes() multiple times to assemble the unpaginated result.
notes = api.get_all_notes()
Add a tag to a note
from joppy.api import Api

# Create a new Api instance.

api = Api(token=YOUR_TOKEN)

# Add a notebook.

notebook_id = api.add_notebook(title="My first notebook")

# Add a note in the previously created notebook.

note_id = api.add_note(title="My first note", body="With some content", parent_id=notebook_id)

# Add a tag, that is not yet attached to a note.

tag_id = api.add_tag(title="introduction")

# Link the tag to the note.

api.add_tag_to_note(tag_id=tag_id, note_id=note_id)
Add a resource to a note
from joppy.api import Api
from joppy import tools

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

# Add a notebook.
notebook_id = api.add_notebook(title="My first notebook")

# Option 1: Add a note with an image data URL. This works only for images.
image_data = tools.encode_base64("path/to/image.png")
api.add_note(
    title="My first note",
    image_data_url=f"data:image/png;base64,{image_data}",
)

# Option 2: Create note and resource separately. Link them later. This works for arbitrary attachments.
note_id = api.add_note(title="My second note")
resource_id = api.add_resource(filename="path/to/image.png", title="My first resource")
api.add_resource_to_note(resource_id=resource_id, note_id=note_id)
Bulk remove tags

Inspired by https://discourse.joplinapp.org/t/bulk-tag-delete-python-script/5497/1.

import re

from joppy.api import Api

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

# Iterate through all tags.
for tag in api.get_all_tags():

    # Delete all tags that match the regex. I. e. start with "!".
    if re.search("^!", tag.title) is not None:
        api.delete_tag(tag.id)
Remove unused tags

Reference: https://discourse.joplinapp.org/t/prune-empty-tags-from-web-clipper/36194

from joppy.api import Api

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

for tag in api.get_all_tags():
    notes_for_tag = api.get_all_notes(tag_id=tag.id)
    if len(notes_for_tag) == 0:
        print("Deleting tag:", tag.title)
        api.delete_tag(tag.id)
Remove spaces from tags

Reference: https://www.reddit.com/r/joplinapp/comments/pozric/batch_remove_spaces_from_all_tags/

import re

from joppy.api import Api

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

# Define the conversion function.
def to_camel_case(name: str) -> str:
    name = re.sub(r"(_|-)+", " ", name).title().replace(" ", "")
    return "".join([name[0].lower(), name[1:]])

# Iterate through all tags and apply the conversion.
for tag in api.get_all_tags():
    api.modify_tag(id_=tag.id, title=to_camel_case(tag.title))
Remove orphaned resources

Inspired by https://discourse.joplinapp.org/t/joplin-vacuum-a-python-script-to-remove-orphaned-resources/19742. Note: The note history is not considered. See: https://discourse.joplinapp.org/t/joplin-vacuum-a-python-script-to-remove-orphaned-resources/19742/13.

import re

from joppy.api import Api

# Create a new Api instance.
api = Api(token=YOUR_TOKEN)

# Getting the referenced resource directly doesn't work:
# https://github.com/laurent22/joplin/issues/4535
# So we have to find the referenced resources by regex.

# Iterate through all notes and find the referenced resources.
referenced_resources = set()
for note in api.get_all_notes(fields="id,body"):
    matches = re.findall(r"\[.*\]\(:.*\/([A-Za-z0-9]{32})\)", note.body)
    referenced_resources.update(matches)

assert len(referenced_resources) > 0, "sanity check"

for resource in api.get_all_resources():
    if resource.id not in referenced_resources:
        print("Deleting resource:", resource.title)
        api.delete_resource(resource.id)

For more usage examples, check the example scripts or tests.

📰 Examples

Before using joppy, you should check the Joplin plugins. They are probably more convenient. However, if you need a new feature or just want to code in python, you can use joppy.

Apps

App Description
jimmy A tool to import your notes to Joplin
joplin-sticky-notes Stick your Joplin notes to the desktop
joplin-vieweb A simple web viewer for Joplin

Scripts

Script Description
custom_export.py Export resources next to notes, instead of a separate folder.
note_export.py Export notes to any format supported by pandoc.
note_stats.py Get some simple statistics about your notes, based on nltk.
note_tree_export.py Joplin only supports PDF export of a single note. This script allows to export one, multiple or all notebooks to PDF or TXT.
visualize_note_locations.py Visualize the locations of your notes.
joplin-ui-tests System tests for the joplin desktop app. Based on selenium.

☀️ Tests

To run the tests, some additional system packages and python modules are needed. After installing them, just run:

python -m unittest

It's possible to configure the test run via some environment variables:

  • SLOW_TESTS: Set this variable to run the slow tests. Default not set.
  • API_TOKEN: Set this variable if there is already a joplin instance running. Don't use your default joplin profile! By default, a joplin instance is started inside xvfb. This takes some time, but works for CI.

📖 Changelog

0.2.3

  • Don't use the root logger for logging.
  • Add support for revisions.

0.2.2

  • Fix adding non-image ressources (#24).
  • Cast markup_language to an appropriate enum type.
  • Add changelog.

0.2.1

  • Fix PDF output example (#19).
  • ⚠️ Drop tests for python 3.6, since it's EOL. It may still work.
  • Fix the type of todo_completed and todo_due. They are a unix timestamp, not a bool.

0.1.1

  • Add typing support to the pypi module.

0.1.0

  • Use a requests session for speedup (#15).
  • ⚠️ Convert the API responses to data objects (#17). Main difference is to use note.id instead of note["id"] for example.

0.0.7

  • Fix getting the binary resource file (#13).

0.0.6

  • Add convenience method for deleting all notes.
  • Add example scripts.

0.0.5

  • Fix package publishing workflow.

0.0.4

  • Add support for python 3.6 and 3.7.

0.0.3

  • Fix search with special characters (#5).
  • Remove arbitrary arguments from the internal base requests, since they aren't needed and may cause bugs.

0.0.2

  • CI and test improvements.
  • Move complete setup to setup.cfg.

0.0.1

  • Initial release.

joppy's People

Contributors

marph91 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

joppy's Issues

add_resource_to_note assumes it is an image

Please see here
https://github.com/marph91/joppy/blob/cc3d1f4d0c3b6fea08b8e8ed76c44eac2c512a45/joppy/api.py#L338C9-L338C9

If the resource is an image, this line is fine.
body_with_attachment = f"{note.body}\n![{resource.title}](:/{resource_id})"
Otherwise, we should not include the exclamation mark, i.e. it should be
body_with_attachment = f"{note.body}\n[{resource.title}](:/{resource_id})"
as in the case of adding a PDF resource.

The inclusion exclamation mark for non-images makes the resource unable to render.

When exporting notes to pdf, the image resource fails to load.

Problem Description

I encountered the following error when running note_export.py to export notes to PDF

WARNING:pypandoc:This document format requires a nonempty <title> element.
  Defaulting to '-' as the title.
  To specify a title, use 'title' in metadata or --metadata title="...".
ERROR: Failed to load image at 'file:///D:/PycharmProjects/pythonProject/:/8cbb74f887a64d579b65
9d164f1bdebd': OSError: Bad URL: /D|/PycharmProjects/pythonProject/|/8cbb74f887a64d579b659d164f
1bdebd
ERROR: Failed to load image at 'file:///D:/PycharmProjects/pythonProject/:/d13e4a8f680740299aaf
161958dac938': OSError: Bad URL: /D|/PycharmProjects/pythonProject/|/d13e4a8f680740299aaf161958
dac938
ERROR: Failed to load image at 'file:///D:/PycharmProjects/pythonProject/:/44eac417e2c74aacafa6
a6d8973f77e4': OSError: Bad URL: /D|/PycharmProjects/pythonProject/|/44eac417e2c74aacafa6a6d897
3f77e4

image

The code block does not render well and the image fails to load. How can I solve this problem?

Creating a new note based on previous note

Hi, thank you for creating this Python wrapper.

I have a certain use case and was hoping you could advise if this is doable or not.

From a high level, I want to create a new note each from a template note (just a regular note that I'm going to use as a template).

From looking at api.py I was planning to use get_note() to obtain the template note's body, and then pass that into add_note(). Is this the right idea? Thank you in advance.

slow in my environment

hi,

I tried this api, its very convenient but relatively slow.
So I modify it to use "session":

  1. add one line under import requests as
    import requests
    re_session = requests.Session()
    
  2. modify another line
    response: requests.models.Response = getattr(requests, method)( 
    => response: requests.models.Response = getattr(re_session, method)(
    

It's much faster now, but still slower than using subprocess.Popen to call curl for the same thing.

Is this the limitation of the python requests module or maybe I did something wrong?

get_notes just get 100 notes, not all notes of a given notebook

When using api.get_notes, I just get 100 notes, not the whole note. I used this line:

notes_notebook2=api.get_notes(notebook_id='dfa6f637732f4d318b4fcf21b05aa6af',fields='id,title,body')['items']
len(notes_notebook2)
I get a length of 100 for a notebook that has about 5000 notes.

In order to reproduce, one would just need to test with a notebook with more than 100 notes and see that he would obtain only 100 notes.

get_resource: does not work for filedata

It seems like get_resource with the argument get_file=True will fail since the Joplin API is sending a binary blob in this case and not json data.

Excerpt:

Traceback (most recent call last):
  File "C:\devel\joplin\.venv\lib\site-packages\requests\models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Python39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
    pprint(api.get_resource(id_, get_file=True))
  File "C:\devel\joplin\.venv\lib\site-packages\joppy\api.py", line 265, in get_resource
    response: JoplinItem = self.get(f"/resources/{id_}/file", query=query).json()
  File "C:\devel\joplin\.venv\lib\site-packages\requests\models.py", line 917, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] �PNG
→
IHDR☻g♠��D� IDATx���l[������N+�
...

version info:
joppy: 0.0.6
Joplin 2.7.15 (prod, win32)
Sync Version: 3
Profile Version: 41
Revision: 8352e23

search api常常不灵

seach api 时灵时不灵。不知道是什么原因。您这边测试一直都正常吗。

just thanks

Just wanted to say thanks for breathing life into this project.
there is a huge need of joplin Python API and I think the project will get popular soon.
Best
Shahin

Extend the test jobs

Currently the tests are only run against a hardcoded version of joplin and python. To ensure compatibility, the tests should be ran against multiple versions. Ideally there should be distinct test jobs. Possible options:

  • joplin: latest release, latest pre-release and oldest supported version (2.X?) -> The latest release should be sufficient, since the API doesn't change much.
  • python: 3.6, ... -> can be done via build matrix
  • os: linux (ubuntu), windows, mac -> Difficult, since the tests use linux-specific packets. Ubuntu should be sufficient, since joppy doesn't have any os-specific dependencies.

body is not stored in NoteData

When getting the body of a note with get_note. The body is none

note = joplinApi.get_note("123456")
print(note.body) # this is none 

But if you do:

note = joplinApi.get_note("123456", fields="body")
print(note.body) # this return the body string

The second note still is NoteData type, but with no id, no title, only body

The method to access the body of a note seems inconsistent, and I don't understand why those two NoteData are separated and if that behavior is expected or if it has something to do with the dataclasses merge (#17)

Search query can lead to a 403 Client Error: Forbidden for url

Hi,

The following query search lead to a 403 Client Error: Forbidden for url whereas searching in joplin gives a correct result

Step to reproduce
`from joppy.api import Api

api = Api(token='xxx')

q='"https://books.google.com.br/books?id=vaZFBgAAQBAJ&pg=PA83&dq=lakatos+copernicus&hl=pt-BR&sa=X&ved=0ahUKEwjewoWZ6q7hAhUNJ7kGHdy5CZUQ6AEIUDAF#v=onepage&q=lakatos%20copernicus&f=false"'

api.search(query=q)`

I may miss some specific knowledge or should have read a documentation that may help me solve this issue.

The context of the problem is the following. I was using an alternative solution for my program that helps me remove semantically identical notes which was to split body at each special character and search for the longuest line in the obtain list of string but it leads small search query and more than 2 notes in many cases so that I cannot finalize the elimination of doublons I have in joplin. Here what I was doing to remove special characters:

`
import re

notelines=re.split(r'[`-=~!@#$%^&*()_+[]{};'\:"|<,./<>?]', note['body'])

q='"'+max(notelines,key=len)+'"'

identicalnotes=api.search(query=q)
`

I reverted to:

`
notelines=note['body'].replace('(',' ').replace(')',' ').replace('[',' ').replace(']',' ').replace('"',' ').split('\n')

q='"'+max(notelines,key=len)+'"'

identicalnotes=api.search(query=q)
`

The error reported here is based on this "filtering" out of '(', ')', '[', ']' and '"'. If remove other special characters found in http link like '%' or '#', I would loose to much "informations".

(inserting code mode seems not to work properly. I had to include two line breaks in order to make it readable.)

Release notes

Hello.
This is not really an issue, more a disussion.

As you know I'm using your super API for Joplin.
I'm currently using 0.0.5, and I would like to update, but I don't know what the new releases bring and if ot worth the time. And if it remains compatible (I guess yes since no major version update?).

Could you explain your versionning logic: is X.Y.Z: X major = compatibility, and Y = new endpoints?

A changelog at the project root may help about that.

Check for projects that would benefit from the API

API.add_note(..) silently fails

I have a script that has been sending me updated todos for the past few weeks. A few days ago the api object's 'add_note' stopped posting to what I can see on my side.

Is this just me?

Thanks in advanced

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.