arkalim / kindle-to-notion Goto Github PK

Hello @arkalim. A really great work 👍
I'm wondering are you going to support HTML export format https://www.easeus.com/knowledge-center/export-kindle-highlights.html#kindle_mobile_app ?

HTML format benefits comparing to Clippings.txt

• Heading sections
• Highlight color

Try to run the setup command as mentioned in the README.md and I get the below error.

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

Can you publish an image for arm platform too?

Hi, nothing is synchronized. I noticed that I get this error on github: "Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files."

Got the following error after the command "npm start". I guess, it's because I have more than 100 highlights in the book "The Brothers Karamazov". Is there a way to fix it? Also I see another error: body.parent.page_id should be defined. I don't know how to fix this. HELP please.

Hi @arkalim ,

I just setup the kindle-to-notion sync but everytime I run the workflow, nothing gets synced to Notion and I see the following in the log:
📋 Parsing Clippings

➕ Grouping Clippings

💹 Stats for Clippings

🟢 Every book is already synced!

But nothing really gets synced to Notion DB. Could you please help me debug this issue?

Thanks!

I have an older kindle (d01200) and the current regex was unable to capture the data from the clippings file. Below are the changes one could do to sync their data to Notion.

// src/models/parser.ts

private regex =
    /(.+) \((.+)\)\r*\n- Your Highlight (location|Location|page)( |(.+))([0-9]+)-([0-9]+) \| Added on ([a-zA-Z]+), ([a-zA-Z]+|[0-9]+) ([0-9]+|[a-zA-Z]+),? ([0-9]{4}), ([0-9]+):([0-9]+\s? A?P?M?)\r*\n\r*\n(.+)/gm;
    

// src/models/parser.ts line 38-39

const time = `${match[12]}:${match[13]}`;
const highlight = match[14];    

Input data looks something like below:

No Rules Rules (Reed Hastings;Erin Meyer)
- Your Highlight Location 2620-2620 | Added on Sunday, January 22, 2023, 08:20 PM

farm for dissent, socialize the idea, and for big bets, test it out. 
==========
No Rules Rules (Reed Hastings;Erin Meyer)
- Your Highlight Location 3725-3725 | Added on Saturday, January 28, 2023, 05:10 AM

Erin’s book, The Culture Map. 
==========

Thanks to @arkalim for making this project. Solves the most crucial piece for me, just had to fix the regex for my device.

In the end of the process, where it says

Connect your Kindle to your computer. Navigate to Kindle arrow_right documents and copy My Clippings.txt. Replace my My Clippings.txt in resources folder with yours.

After doing it, running npm start, I get:

No books and no clipping have been imported as a result. I've tested with the Clippings that came in the resources folder, and it worked. So, I have no ideia what is happening...

Maybe it has something to do with this:

Apparently, the clippings from my kindle it's not a plain/text file and maybe thats whats happening.

My Clippings.txt

Hi, after following your Yt tutorial I sadly still have issues...
my Workspace key looks a bit different, but I took the 32char code between library- and the ?

still I get an error : Failed to sync highlights APIResponseError: Could not find database with ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

https://www.notion.so/[workspacename]/Library-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx?pvs=x

(ofc instead of xxx there is the "real" key)

Do you have any ideas why I get this issue?
Best regards

I have a problem and I can't resolve it. Apparently it says that the books are already synchronized, but they do not appear at all on the page created in Notion.

Everything was configured exactly as in the video, but I still couldn't get it to work.

Set Up job
Current runner version: '2.309.0'
Operating System
Runner Image
Runner Image Provisioner
GITHUB_TOKEN Permissions
Secret source: Actions
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'actions/checkout@v3' (SHA:f43a0e5ff2bd294095638e18286ca9a3d1956744)
Download action repository 'addnab/docker-run-action@v3' (SHA:4f65fabd2431ebc8d299f8e5a018d79a769ae185)
Download action repository 'EndBug/add-and-commit@v7' (SHA:8c12ff729a98cfbcd3fe38b49f55eceb98a5ec02)
Download action repository 'ad-m/github-push-action@master' (SHA:29f05e01bb17e6f28228b47437e03a7b69e1f9ef)
Complete job name: sync-highlights

Build addnab/docker-run-action@v318s

Build container for action use: '/home/runner/work/_actions/addnab/docker-run-action/v3/Dockerfile'.

Checkout repository

Run actions/checkout@v3
Syncing repository: mripol/kilde-highlight
Getting Git version info
Temporarily overriding HOME='/home/runner/work/_temp/1a55afff-9978-4de3-b32c-f2af882031c1' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --global --add safe.directory /home/runner/work/kilde-highlight/kilde-highlight
Deleting the contents of '/home/runner/work/kilde-highlight/kilde-highlight'
Initializing the repository
Disabling automatic garbage collection
Setting up auth
Fetching the repository
Determining the checkout info
Checking out the ref
/usr/bin/git log -1 --format='%H'
'4cd602ea8e4c1463ef5cf30fc9561d07c4b49270'

Sync highlights

Run addnab/docker-run-action@v3
/usr/bin/docker run --name b0ae1e1492cb74cdbaf9ff359f9094dcf_a4279e --label 94351b --workdir /github/workspace --rm -e "INPUT_IMAGE" -e "INPUT_RUN" -e "INPUT_OPTIONS" -e "INPUT_SHELL" -e "INPUT_REGISTRY" -e "INPUT_USERNAME" -e "INPUT_PASSWORD" -e "INPUT_DOCKER_NETWORK" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/kilde-highlight/kilde-highlight":"/github/workspace" 94351b:0ae1e1492cb74cdbaf9ff359f9094dcf
Unable to find image 'ghcr.io/arkalim/kindle-to-notion:master' locally
master: Pulling from arkalim/kindle-to-notion
31e352740f53: Pulling fs layer
2629b68d4311: Pulling fs layer
ddb7cc70f260: Pulling fs layer
18afe6373474: Pulling fs layer
edf0b2b9bb48: Pulling fs layer
745762490991: Pulling fs layer
a84e6dbb39e0: Pulling fs layer
ad5591fb15e2: Pulling fs layer
e14969b0211a: Pulling fs layer
18afe6373474: Waiting
edf0b2b9bb48: Waiting
745762490991: Waiting
a84e6dbb39e0: Waiting
ad5591fb15e2: Waiting
e14969b0211a: Waiting
ddb7cc70f260: Verifying Checksum
ddb7cc70f260: Download complete
31e352740f53: Verifying Checksum
31e352740f53: Download complete
edf0b2b9bb48: Verifying Checksum
edf0b2b9bb48: Download complete
18afe6373474: Verifying Checksum
18afe6373474: Download complete
31e352740f53: Pull complete
745762490991: Verifying Checksum
745762490991: Download complete
a84e6dbb39e0: Verifying Checksum
a84e6dbb39e0: Download complete
2629b68d4311: Verifying Checksum
2629b68d4311: Download complete
e14969b0211a: Verifying Checksum
e14969b0211a: Download complete
ad5591fb15e2: Verifying Checksum
ad5591fb15e2: Download complete
2629b68d4311: Pull complete
ddb7cc70f260: Pull complete
18afe6373474: Pull complete
edf0b2b9bb48: Pull complete
745762490991: Pull complete
a84e6dbb39e0: Pull complete
ad5591fb15e2: Pull complete
e14969b0211a: Pull complete
Digest: sha256:e619f20321eb17d192d20c77045ed3a9ad1e08c508941279b309b8db189a5875
Status: Downloaded newer image for ghcr.io/arkalim/kindle-to-notion:master
📋 Parsing Clippings

➕ Grouping Clippings

💹 Stats for Clippings

🟢 Every book is already synced! "-> apears that the books are already sync"

Commit cache changes

Run EndBug/add-and-commit@v7

Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Warning: The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Running in /home/runner/work/kilde-highlight/kilde-highlight
Add input parsed as single string, running 1 git add command.

Using 'mripol [email protected]' as author.
Using "Commit cache changes" as commit message.
Internal logs
Working tree clean. Nothing to commit.
Outputs

Push cache changes

Run ad-m/github-push-action@master
Push to branch main
Everything up-to-date

Post Checkout repository

Post job cleanup.
/usr/bin/git version
git version 2.42.0
Temporarily overriding HOME='/home/runner/work/_temp/67449942-d286-45bc-a0c0-6ad70a92ba12' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --global --add safe.directory /home/runner/work/kilde-highlight/kilde-highlight
/usr/bin/git config --local --name-only --get-regexp core.sshCommand
/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
/usr/bin/git config --local --name-only --get-regexp http.https://github.com/.extraheader
http.https://github.com/.extraheader
/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http.https://github.com/.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"

Complete job

Cleaning up orphan processes

Hi,

Thanks for this, it works great in my experience so far!

Less an issue, more a question... any plans to include notes, along with highlights?

I tested this out today and found that, whilst the notes are present in the MyClippings.txt file, they didn't appear in Notion after sync. I couldn't see any reference to notes in the codebase when I had a brief look, so assuming this isn't part of the functionality, but wondered if it might be in future?

Thanks!

It tells me my API token is invalid.

The integration is added to the database and I provided the correct IDs for both of them

Hello
i have a pb when running sync.yaml. here is the error message. could you help me ? It is in Sync Higlights part :

Unable to find image '3425285a4d654d7ebb8dd6c24903a639:latest' locally
docker: Error response from daemon: pull access denied for 3425285a4d654d7ebb8dd6c24903a639, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

I've checked variable and secret : there both OK. 3425285a4d654d7ebb8dd6c24903a639 is my BOOK_DB_ID.

I am getting all sorts of errors when running the commands specified in the readme. I am running Windows 10 with Docker installed. Does anyone have tips for how to enter the docker commands in powershell without throwing errors?

Hi!
I was wondering if there is a way to automate the syncing process where every time you plug in your kindle and type a single command the clippings file gets replaced and it syncs.

Good morning,

what a great project. Thanks for your work :)

Unfortunately the updating of highlights does not work (Looks like it does not read the updated My Clippings file properly)

Do you have an idea :)?

Cheers, Phil

Hi,

Thanks a lot for this tool! It's super cool.

Found a small issue (may be due to my Kindle Paperwhite version) and resolution.

The clippings are in a different format than the one used in this code.

Eg.
'=======
Kindle Paperwhite User's Guide, 3rd Edition (Amazon)

• Your Highlight at location 347-348 | Added on Wednesday, 28 June 2017 22:45:15

tap the sides of the screen. To exit Kindle
'=======

Key differences in the format are:

• on Location vs at location
• Month dd, yyy HH:mm:ss AM/PM vs dd Month yyyy HH:mm:ss (24Hr format)
• CR and line-breaks after date and time

I have replaced the regex code with the below, and it worked.

/(.+) \((.+)\)\r*\n- Your Highlight (at|on) (location|Location|page)( |(.+))([0-9]+)-([0-9]+) \| Added on ([a-zA-Z]+), ([0-9]+) ([a-zA-Z]+) ([0-9]{4}) ([0-9]+):([0-9]+):([0-9]+)\r*\n\r*\n(.+)/gm

Maybe if you can modify the Regex to match the above example too, it would help.

Thanks

Goal: in Notion Database, automatically sync:

• Date Started = date of earliest note
• Date Finished = date of latest note

It seems like I'll need to alter the /interfaces/notion.ts file and /interfaces/clipping.ts file, but I'm not sure about that. Can you point me in the right direction?

The code only works with books that were purchased through Amazon, it doesn't sync PDFs.

I followed your video on YouTube, but the synchronization keeps failing? Does this version support Chinese? There is something wrong with my operation. T T, can you help me find out?

My Clippings.txt

Hey, since the code already uses highlightCount in business logic, could you update it to add it as a property to the book page? And update it whenever one syncs new highlights.

Hi there,

I am interested in using this software and potentially making some changes to it (perhaps in a fork), however, there is license associated with it. Could you please supply a license so users interested in forking know what requirements are in place for them?

Thanks 😊

(Really cool idea btw)

I tried with the "My Clippings.txt" in https://github.com/arkalim/kindle-highlights, and it works. But when I use the "My Clippings.txt" copied from my device, nothing shows up in the database and the workflow shows the sync has done. Attachment is my clippings. Please see if you can solve this problem, thanks.
My Clippings.txt

Hey Abdur, great work and a great idea. However it seems there's a problem when I run npm start, the highlights are not syncing, I think the problem is on my side, but I'm not very good at coding so I can't figure it out. Could you help me out?

Attaching the terminal log.

Thanks, Marek

terminal.txt