ncarlier / readflow Goto Github PK

Hello @ncarlier!

I'm trying to setup readflow with docker-compose on arm64 instance with Traefik as a global reverse proxy. There was a lot of non-clear moments in a process of building the docker-compose file, but now it is working properly (almost) with mock auth, but when I try to login, even with right credentials, I'm getting an auth loop: basic auth pop-up just creating again:

If I click "Cancel", I get that:

I'm using the latest UI code from repo (commit eaca909), and the latest API Docker image (https://hub.docker.com/layers/ncarlier/readflow/edge/images/sha256-88b58d7081da8dc8ffa1ed6024fac98c472c3e59cb555599a22794e508e54160?context=explore)
I've read issue #18, and added necessary labels to UI container, but it not helped:(

Here is my setup config files:

docker-compose.yml

version: '3.1'

services:

  readflow_api:
    image: ncarlier/readflow:edge
    restart: unless-stopped
    container_name: readflow_api
    expose:
      - 8080:8080
    environment:
      READFLOW_DB: postgres://readflow_usr:******@readflow_db/readflow_db?sslmode=disable
      READFLOW_CONFIG: /var/readflow.toml
    volumes:
      - ./data/readflow.toml:/var/readflow.toml
    networks:
      readflow:
        ipv4_address: 172.16.8.2
      traefik-public:
    labels:
      traefik.enable: true
      traefik.http.services.readflow-api.loadbalancer.server.port: 8080
      traefik.http.routers.readflow-api.rule: Host(`links.domain.com`) && PathPrefix(`/api`)
      traefik.http.routers.readflow-api.entrypoints: websecure
      traefik.http.routers.readflow-api.tls: true
      traefik.http.routers.readflow-api.tls.certresolver: le_resolver
      traefik.http.routers.readflow-api.priority: 2
      traefik.http.routers.readflow-api.middlewares: readflow-api-strip
      traefik.http.middlewares.readflow-api-strip.stripprefix.prefixes: /api
      logging: promtail
      logging_jobname: containerlogs

  readflow_db:
    image: postgres
    restart: unless-stopped
    container_name: readflow_db
    environment:
      POSTGRES_DB: readflow_db
      POSTGRES_USER: readflow_usr
      POSTGRES_PASSWORD: ******
      PGDATA: /data/postgres
    volumes:
       - ./db:/data/postgres
    networks:
      readflow:
        ipv4_address: 172.16.8.3
    labels:
      logging: promtail
      logging_jobname: containerlogs

  readflow_ui:
    build: ./readflow/ui/
    image: "ncarlier/readflow-ui:latest"
    container_name: readflow_ui
    restart: always
    expose:
      - 80:80
    labels:
      traefik.enable: true
      traefik.http.services.readflow-ui.loadbalancer.server.port: 80
      traefik.http.routers.readflow-ui.rule: Host(`links.domain.com`)
      traefik.http.routers.readflow-ui.entrypoints: websecure
      traefik.http.routers.readflow-ui.tls: true
      traefik.http.routers.readflow-ui.tls.certresolver: le_resolver
      traefik.http.routers.readflow-ui.priority: 1
      traefik.http.routers.readflow-ui.middlewares: auth-readflow
      # admin:12345
      traefik.http.middlewares.auth-readflow.basicauth.users: "admin:$$apr1$$IIsc.lGe$$BGDkLEBFMsu06W1a7coB.."
      traefik.http.middlewares.auth-readflow.basicauth.headerField: "X-WebAuth-User"
      logging: promtail
      logging_jobname: containerlogs
    networks:
      readflow:
        ipv4_address: 172.16.8.5
      traefik-public:

networks:
  readflow:
    driver: bridge
    ipam:
      config:
        - subnet: 172.16.8.0/24
  traefik-public:
    external: true

readflow.toml

[global]
db = "postgres://readflow_usr:******@readflow_db/readflow_db?sslmode=disable"
listen_addr = ":8080"
metrics_listen_addr = ""
authn = "proxy"
public_url = "https://links.domain.com/api"
secret_salt = "******"
block_list = ""

[integration]
external_event_broker_uri = ""
external_web_scraper_url = ""
image_proxy_url = ""
avatar_provider = ""

[integration.sentry]
dsn_url = ""

[integration.pocket]
consumer_key = ""

Part of UI Dockerfile:

ENV REACT_APP_API_ROOT https://links.domain.com/api
ENV REACT_APP_REDIRECT_URL /login
ENV REACT_APP_AUTHORITY mock

Could you please help me to work out that auth problem?

I haven't started getting Readflow running up yet, however I noticed that it uses 3rd party OAuth which is something I'd like to avoid. What's the easiest way to replace it with something local?

"mock", "proxy" or OIDC if URL - not quite sure what they mean...

Here is my use case :

1. Open readflow : I have 182 articles to read
2. Invert order to get newest articles first : I still have 182 articles to read
3. Open the any article (Here I take the first one)
4. Mark as read

I'm redirected to the list of articles (which is fine) but :

• the ordering condition is not kept (I should have newest first)
• the unread count is not decreased, I still have 182 articles to read (I should have 181)

By the way, if I invert the ordering condition, I get the correct unread count (181) and the article is gone !

Some OIDC issuers require client_secret to be sent when logging in, but Readflow does not allow to specify it.

Trying to run readflow with docker but I am getting the following error:

{"level":"warn","version":1,"time":1641044388,"message":"Migrating Database..."}
{"level":"warn","version":2,"time":1641044391,"message":"Migrating Database..."}
{"level":"warn","version":3,"time":1641044392,"message":"Migrating Database..."}
{"level":"warn","version":4,"time":1641044393,"message":"Migrating Database..."}
{"level":"warn","version":5,"time":1641044393,"message":"Migrating Database..."}
{"level":"warn","version":6,"time":1641044393,"message":"Migrating Database..."}
{"level":"warn","version":7,"time":1641044393,"message":"Migrating Database..."}
{"level":"warn","version":8,"time":1641044394,"message":"Migrating Database..."}
{"level":"warn","version":9,"time":1641044394,"message":"Migrating Database..."}
{"level":"warn","version":10,"time":1641044394,"message":"Migrating Database..."}
{"level":"fatal","error":"pq: syntax error at or near "VALUE"","time":1641044394,"message":"Unable to apply migration"}

I use "Enter" to open an article, "M" to mark it as read.

I don't think there's a shortcul to go back to the list I if don't want the article to be marked as read

Just want to try the app.
curl -sf https://gobinaries.com/ncarlier/readflow | sh gives

Error: Repository has no tags

go get -v github.com/ncarlier/readflow gives error

github.com/ncarlier/readflow (download)
github.com/ncarlier/readflow/pkg/model
# github.com/ncarlier/readflow/pkg/model
../../go/src/github.com/ncarlier/readflow/pkg/model/category.go:55:42: undefined: gofakeit.RandString

go version
go version go1.18.3 linux/amd64 and go env -w GO111MODULE=auto in ~/.profile if it matters

All suggested installation methods fail on Ubuntu 20.04 server edition.

Curl: Error: Failed to extract resource URL (the same happens if I run install.sh from my terminal after cloning the repository).

gobinaries.com: Error: The repository has no tags.

docker: The container exits on run with error code 1.

Haven't tried build from source yet, as I didn't really plan to install Golang on my server...

Hello,

I'm using the docker-compose file.

I setup the OIDC for readflow, but when I try to access the public (even the local) address, it will redirect to the /login page, in which will return a 404.

I checked my NGINX config, but it seems to be redirecting properly because I set the authentication method as "mock" and I could access it.

Also I can see the 404 error inside the container:

2:15PM INF GET /login duration=0 remote-addr=<REDACTED> req-id=<REDACTED_?> size=19 status=404 user-agent="Mozilla/5.0 (X11; Linux x86_64; rv:122.0) Gecko/20100101 Firefox/122.0"

By the logs, it seems that the OIDC was properly identified:

2:14PM INF using authentication issuer=https://<KEYCLOAK_REALM> method=oidc

And well, it doesn't even try to redirect to keycloak.

Thanks so far :)

Beside each category (and even the main menu entries) it could be useful to see how many articles are inside (unread count for most ot them, or total)

Hello,

There seems to be a problem,

I've put the container from docker hub onto my kubernetes and when i want to access url, it always redirect me to the info endpoint and so i can't access the ui.

Can you fix this please ?

The link in the readme to the documentation points to https://about.readflow.app/docs which is a 404 page.

with latest code:

cd ui
npm install

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @graphiql/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/graphql
npm WARN   graphql@"^15.5.1" from the root project
npm WARN   5 more (@apollo/client, @graphql-typed-document-node/core, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer graphql@">= v14.5.0 <= 15.5.0" from @graphiql/[email protected]
npm WARN node_modules/@graphiql/toolkit
npm WARN   @graphiql/toolkit@"^0.2.0" from [email protected]
npm WARN   node_modules/graphiql
npm WARN 
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/graphql
npm WARN   peer graphql@">= v14.5.0 <= 15.5.0" from @graphiql/[email protected]
npm WARN   node_modules/@graphiql/toolkit
npm WARN     @graphiql/toolkit@"^0.2.0" from [email protected]
npm WARN     node_modules/graphiql
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: [email protected]
npm WARN Found: [email protected]
npm WARN node_modules/graphql
npm WARN   graphql@"^15.5.1" from the root project
npm WARN   5 more (@apollo/client, @graphql-typed-document-node/core, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer graphql@">= v14.5.0 <= 15.5.0" from [email protected]
npm WARN node_modules/codemirror-graphql
npm WARN   codemirror-graphql@"^1.0.0" from [email protected]
npm WARN   node_modules/graphiql
npm WARN 
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/graphql
npm WARN   peer graphql@">= v14.5.0 <= 15.5.0" from [email protected]
npm WARN   node_modules/codemirror-graphql
npm WARN     codemirror-graphql@"^1.0.0" from [email protected]
npm WARN     node_modules/graphiql
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/graphql
npm ERR!   graphql@"^15.5.1" from the root project
npm ERR!   peer graphql@"^14.0.0 || ^15.0.0" from @apollo/[email protected]
npm ERR!   node_modules/@apollo/client
npm ERR!     @apollo/client@"^3.4.5" from the root project
npm ERR!   4 more (@graphql-typed-document-node/core, graphql-tag, ...)
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer graphql@">= v14.5.0 <= 15.5.0" from [email protected]
npm ERR! node_modules/graphiql
npm ERR!   graphiql@"^1.4.2" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/graphql
npm ERR!   peer graphql@">= v14.5.0 <= 15.5.0" from [email protected]
npm ERR!   node_modules/graphiql
npm ERR!     graphiql@"^1.4.2" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /Users/xxx/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/xxx/.npm/_logs/2022-06-16T02_46_56_804Z-debug-0.log

Hi, I'm trying to have a selfhosted readflow instance. I have little experience with docker, I created a docker compose file but when I start it I don't have the user interface.

I used docker-compose.yml in the root folder as an example.

This is my version:

version: "3"

services:
  ########################################
  # PostgreSQL
  ########################################
  postgres:
    image: postgres:11
    container_name: postgres
    restart: always
    environment:
      - POSTGRES_DB=${POSTGRES_DB:-readflow}
      - POSTGRES_USER=${POSTGRES_USER:-postgres}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    networks:
      - internal
    volumes:
      - postgresdb_volume:/var/lib/postgresql/data
    labels:
      - "traefik.enable=false"

  #######################################
  # Imaginary: Image proxy
  #######################################
  imaginary:
    image: h2non/imaginary:latest
    container_name: imaginary
    command: -enable-url-source -http-cache-ttl 172800
    environment:
      - MALLOC_ARENA_MAX=2
    logging:
      driver: "json-file"
      options:
        max-size: "1m"
    restart: always
    networks:
      - internal
    labels:
      - "traefik.enable=false"

  ########################################
  # API
  ########################################
  api:
    image: "ncarlier/readflow:latest"
    container_name: readflow
    restart: always
    depends_on:
      - postgres
    networks:
      - web
      - internal
    environment:
      - READFLOW_AUTHN=mock
      - READFLOW_DB=postgres://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD}@postgres/${POSTGRES_DB:-readflow}?sslmode=disable
      - READFLOW_LISTEN_METRICS=:9090
      - READFLOW_IMAGE_PROXY=http://imaginary:9000
      - READFLOW_PUBLIC_URL="https://mysite.site"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.readflow.rule=Host(`mysite.site`)"
      - "traefik.http.services.readflow.loadbalancer.server.port=8080"
      - "traefik.http.routers.readflow.service=readflow"
      - "traefik.http.routers.readflow.tls=true"
      - "traefik.http.routers.readflow.tls.certresolver=lets-encrypt"

volumes:
    postgresdb_volume:
        external:
            name: postgresdb_volume

networks:
    web:
        external:
            name: web
    internal:
        external: false

I don't understand if the UI is present in the docker image released or there is another docker image that actually missing from hub.

Thank a lot for your work

Hi,

first of all great work! :-)

I wanted to make readflow work out of a Docker-Compose with traefik. After Setting the env vars - if I go to my domain. I get redirected to about.readflow.app.

Could you please give me a hint?

regards,

docker build .
docker compose up (no ncarlier/readflow:latest)
modifying docker-compose.yml to have the build . command,
Also used latest brach, and release
how can so much be so wrong.

Due to the auto-generation of the UI config, it is impossible to run Readflow in a read-only environment, which is necessary for it to work with NixOS. Please add the ability to disable auto-generation and return to the previous behavior.

Most of the time, I let opened readflow on a list of articles. I could be interesting to be notified when new articles are available and I shall refresh

[sevos]:<~>$ readflow -l MyIP:MyPort -db postgres://Login:Pass@Host/reader
{"level":"fatal","error":"could not validate PostgreSQL connection: pq: SSL is not enabled on the server","time":1580559460,"message":"could not configure database"}

Readflow by default is trying to use SSL for PostgreSQL connection which seems to be an overkill, since it's done locally anyways. Any way to disable it?

Thanks for make this amazing project, I have been searching a self-hosted read it later project which was made by golang for a long time.

I found readflow yesterday and tried to deploy it to my server for several hours, but still not working, I'm not a professional programmer, I don't know much about the instructions in the configuration file, I tried my best to understand what the config file was saying and google what I didn't understand, but no luck.

In fact, it took me almost 2 hours just to google how to successfully compile binaries when go get fails. Maybe a layman like me is not the target user of this project, but I still hope to try this project, I will try to explain the problems I have encountered and hope it can be solved, thank you again for developing this project.

config.toml

[global]
db = "postgres://username:password@localhost/databasename?sslmode=disable"
listen_addr = ":9021"
metrics_listen_addr = ""
authn = "mock"
public_url = ""
secret_salt = "${READFLOW_SECRET_SALT}"
block_list = "${READFLOW_BLOCK_LIST}"
[integration]
external_event_broker_uri = "${READFLOW_BROKER_URI}"
external_web_scraper_url = "${READFLOW_WEB_SCRAPER_URL}"
image_proxy_url = "${READFLOW_IMAGE_PROXY_URL}"
avatar_provider = "${READFLOW_AVATAR_PROVIDER}"
[integration.sentry]
dsn_url = "${READFLOW_SENTRY_DSN}"
[integration.pocket]
consumer_key = "${READFLOW_POCKET_CONSUMER_KEY}"

run it

readflow --config config.toml

the output

{"level":"info","component":"database","uri":"postgres://readflow:xxxxx@localhost/readflow?sslmode=disable","time":1658973321,"message":"using PostgreSQL database"}
{"level":"info","component":"cache","uri":"boltdb:///tmp/readflow.cache","time":1658973321,"message":"using BoltDB cache"}
{"level":"info","component":"service","rev":1,"time":1658973321,"message":"properties loaded"}
{"level":"info","time":1658973321,"message":"using Mock as authentication backend"}
{"level":"info","component":"api","provider":"https://robohash.org/{seed}?set=set4&size=48x48","time":1658973321,"message":"using Avatar provider"}
{"level":"info","listen":":9021","time":1658973321,"message":"server started"}

I can't get the web page in the browser when visit http://ip:9021, I can only see a json file

{"version":"snapshot","authority":"mock","vapid":"BASakRma0VQ2znpuOtnaSVsLg34PAatW6dUMLi1t55vdU2Awxnulnb5UmPR4yEiOq5xamXz6YVVoR8QNhq8rDQc"}

I cannot read the bottom if my article because of the "mark as read" button.

NB : of course, I cannot scroll down anymore

OIDC requires a ClientID and in some cases a Client Secret to be set. The example config does not make clear how these can be set. A glance at the code seems to indicate that these have to be set at compile time. Is that correct? If so, could this be changed in order to allow for easier container deployment?

Let's compare both images :

I think it's no big deal if the font style/size is not the same but on image, it's unpleasant that ratio aren't kept

Sample article link is : https://www.frandroid.com/bons-plans/bons-plans-smartphone/593100_bon-plan-le-xiaomi-mi-9-se-est-deja-disponible-a-282-euros-au-lieu-de-369-euros

I use the down arrow to scroll down while parsing the list of articles.

But the bottom arrow doesn't scroll when reading a single article.

I need to click inside the article panel to make scroll work

If various web page styles(or markdown styles) are supported, the reading experience will be improved. Now these styles are not obvious and beautiful. Such as

1. Blockquotes

   quotes

2. Inline code
   code
3. Code Blocks

   code
   code
   

Hi!
Nice project! I've had a similar idea since I've read about 1flow a few years ago.
It's very interesting as many webpages that I fav'ed back in the day don't exist anymore. I lost all reference and content. web.archive.org exists of course, but it's not failsafe.
Anyway, in the settings, can we have some bookmarklet button, so we could add anything while we're browsing?

Hi!

Fantastic app, love it, about to install it. :)

I was wondering if you would be willing to add something like define a weight on an article, how important it is to be read, and then have a priority view where it would show you the list of articles in order of weight / priority.

Cheers,
Gergely.

Hi, this app looks amazing but when I try to install it via Go or using the curl methods, it gives errors such as "Unable to extract artefact URL". I'm also planning to try out feedpushr and noticed that the instructions where slightly different by supplying a version and that repo has releases/packages.

Hi @ncarlier,

Hope you are all well ! And happy new year !

I really beg you to consider to use dart and the flutter framework for creating, from unique code base, a mobile and desktop version or readflow.

Really, it s a good technical choice and a no brainer; golang + dart = awesomeness :-)

Take care my friend :-)

Cheers,
Luc Michalski

curl -sf https://gobinaries.com/ncarlier/readflow | sh

results in

Error: Repository has no tags

and

curl -s https://raw.githubusercontent.com/ncarlier/readflow/master/install.sh | bash

results in

error: Unable to extract artefact URL

The repository does indeed have no tags, and looking at install.sh, it seems to assume there will be GH releases, which I'm not seeing either.

hey, I am looking into this project for the first time, and I was wondering if you consider nested tags and RSS features that is interesting to explore at some time?

• An article might be for #work, but also about #learning/programming and #learning/ai
  

• RSS: I know it's less and less popular, but would lend itself perfectly to the readflow model of "incoming webhook". of course it could also be added externally, like "check this RSS and then call the readflow incoming webhook"

This is not a feature request. I don't expect you to add features that random people would like.
My question is more "Do these features fit your idea for readflow?"
Best

I was reading some articles and marking them as read using the "M" key.

One article came on top of the list (I'm using the "newset first" order).

As soon as this article appeared, it became impossible to mark the first article as read using the "M" key.
NB : I can still mark articles as read using the button.

Once this new article as been marked as read (using the button), I can use again the "M" key

Well I managed to make some progress in the compilation but the ui with npm I can't build it in Windows, should I do it with linux?

I have a category when N articles.
When I use the "Mark all as read" button I have the following error : « User Error: expected iterable, but did not find one for field Mutation.markAllArticlesAsRead. » in the modal.
Please find attached a screenshot :

By the way, the N articles are actually marked as read.

github.com/ncarlier/readflow/pkg/rule-engine
# github.com/ncarlier/readflow/pkg/rule-engine
go/src/github.com/ncarlier/readflow/pkg/rule-engine/processor.go:17:13: undefined: expr.Node

FreeBSD 11, amd64. Any idea as to what's the issue?

I use to read my articles in order from most recent to oldest.

But the default order is from oldest to most recent.

Each time I access readflow, I have to invert the ordering condition. I would by great if this setup could be saved.

Cherry on the cake : keep a different ordering condition per category ;)

I tried to run readflow locally on my windows host and it's failing due to windows filesystem.

I am looking for docker-compose file without taefik. Ports 80 and 443 are also busy.

I tried to remove the links from taefik but it doesn't work without it.

https://pastebin.com/Lq4WnScR

The icons do not show up on my mobile phone, the labels are used instead, which is quite unusable 😞

Firefox 68.1, no additional modules, Android 7.1.1

When I try to build the UI with npm run build, the following error appears:

> [email protected] build
> npm run lint && react-scripts build


> [email protected] lint
> eslint src/**/*.{js,jsx,ts,tsx,json}


Oops! Something went wrong! :(

ESLint: 8.25.0

No files matching the pattern "src/**/*.jsx" were found.
Please check for typing mistakes in the pattern.