Giter Club home page Giter Club logo

reddit-top-rss's Introduction

Reddit Top RSS

Reddit Top RSS is a set of scripts for Reddit's API that generates RSS feeds for specified subreddits with score thresholds. To preview your outputted feed items there is a front end that utilizes the Bootstrap v4 framework.

Preview

Reddit Top RSS screenshot

Motivation

I prefer to interact with Reddit in a low-volume way, so I let Reddit Top RSS surface the most popular posts per subreddit in my RSS reader of choice. I usually use the averagePostsPerDay filter so I can expect a certain amount of posts in my feeds per day.

Installation and usage

Reddit app setup

Due to Reddit's 2023 updated API policies it is now required to first set up an app in your Reddit account that Reddit Top RSS will authenticate through.

  1. First, log into your Reddit account.
  2. Navigate to the Reddit app preferences page: https://www.reddit.com/prefs/apps
  3. Click the create app or create another app button, depending on whether you’ve already created an app before.
  4. Choose any name for the app. I've chosen Top RSS. Reddit will not allow you to use Reddit in the name.
  5. Set the type of app to web app.
  6. You can leave description and about url fields blank.
  7. Enter in any valid URI in the redirect uri field. I've used http://reddit-top-rss.test.
  8. Click the create app button when you’re done.
  9. Your app’s client ID and client secret will be displayed. You'll need to add these to either your config.php or Docker environment variables.

Manual

To install this repository manually:

  1. Clone this repository somewhere with PHP >= 5.6 installed.
  2. Copy config-default.php to config.php.
  3. Enter your Reddit user, app ID, and secret into lines 49, 56, and 63, respectively.
  4. Open index.php in a browser to view the front end.
  5. Enter your parameters into the fields to get a preview of the posts that the filters will output. Click the RSS button at top to open a new tab with the rendered RSS XML output of the specified filters. This is the URL you subscribe to in your RSS aggregator.

Docker

A docker image is available at https://hub.docker.com/r/johnny5w/reddit-top-rss.

Command line

docker run -p 80:8080 -e REDDIT_USER=your_reddit_user -e REDDIT_CLIENT_ID=your_app_id -e REDDIT_CLIENT_SECRET=your_app_secret johnny5w/reddit-top-rss:latest

docker-compose

version: '3'
services:
  reddit-top-rss:
    image: johnny5w/reddit-top-rss
    container_name: reddit-top-rss
    restart: unless-stopped
    ports:
      - 80:8080
    environment:
      - REDDIT_USER=your_reddit_user
      - REDDIT_CLIENT_ID=your_app_id
      - REDDIT_CLIENT_SECRET=your_app_secret
      - DEFAULT_SUBREDDIT=news

Docker environment variables

The following required environment variables must be set, or you will not be authorized with Reddit's API:

Parameter Function
REDDIT_USER Your Reddit user account with which you've created an app
REDDIT_CLIENT_ID The client ID of the app you've created
REDDIT_CLIENT_SECRET The secret for the app you've created

The following optional environment variables can be used to override the application defaults:

Parameter Function
DEFAULT_SUBREDDIT This sets the initial subreddit on the first page load.
Default: pics
MERCURY_URL URL of your Mercury parser instance. See below.
MERCURY_API_KEY API key for your Mercury parser instance.
CACHE_REDDIT_JSON Whether to cache the JSON responses from Reddit.
Default: true
CACHE_MERCURY_CONTENT Whether to cache the responses from your Mercury instance.
Default: true
CACHE_RSS_FEEDS Whether to cache the outputted XML from Reddit Top RSS.
Default: true

Supported URL parameters

There are five URL paramenters supported:

subreddit

The exact string of the subreddit as it appears in the Reddit URL. Only one subreddit may be chosen.

score

Items below the desired score will be filtered out.

threshold

This parameter will get the average score for the past month's hot posts and will filter out items that fall below this percentage. This is helpful for volatile subreddits — and subreddits in general — since more people are using the service and causing posts to be scored higher and higher. Since this is a percentage, the number of items in the outputted feed should be more consistent than when using the score parameter.

averagePostsPerDay

Reddit Top RSS will attempt to output an average number of posts per day by looking at a subreddit's recent history to determine the score below which posts will be filtered out. This is the filter I find most useful.

view

Accepted values are html and rss:

  • html shows the front end preview.
  • rss shows the rendered RSS XML feed. Use this for the URL to subscribe to in your RSS aggregator.
  • If the view parameter is left blank or omitted, the front end is shown.

URI examples

  • https://www.example.com?subreddit=funny&threshold=10000
  • https://www.example.com?subreddit=worldnews&score=1000&view=rss
  • https://www.example.com?subreddit=coolgithubprojects&averagePostsPerDay=3

Configuration

Reddit Top RSS comes with a default configuration. If you'd like to turn caching off, set a different default subreddit, or use a self-hosted Mercury parser, just enter your desired values in config.php.

Mercury parser

If you'd like to include parsed article content in your outputted feed items, set a self-hosted Mercury parser URL and optional API key in config.php. An easy to install, Dockerized version of the Mercury parser can be found here: https://www.github.com/HenryQW/mercury-parser-api.

Caching

By default Reddit Top RSS will cache Reddit JSON files, rendered RSS XML files, and Mercury parsed content to speed up the application. To clear the cache, click the Clear cached results link in the footer. To disable caching for any of the above items, set the appropriate values to false in config.php.

License

This project is released under the MIT License.

Buy me a coffee

BuyMeCoffee


reddit-top-rss's People

Contributors

johnwarne avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

reddit-top-rss's Issues

Feature request: Multiple subreddits and custom feed URL's

Hello,

Thank you for this.

It would be more amazing if we could:

  1. Update it so that you can add multiple feeds, as many as you want, and it will merge them into one feed to monitor all your fav subs.
  2. Add custom feed URL's like search feeds with different sorting parameters for example adding this to the combined feed:

https://www.reddit.com/r/startups/search.rss?q=saas&restrict_sr=1&sr_nsfw=&sort=new

Considering updating a fork with this functions, would this be complex, how many hours for a dev? (I am not a dev).

Thanks!

Found XSS Reflected Injection vulnerabilities

Hello, I was searching for simple vulnerabilities in code and I came across your repository. I have noticed that you do not perform any kind of Input sanitization in your code, and while there isn't any kind of risk because the purpose of this webpage is just generating RSS feeds without any kind of data being persisted, it would be a good idea to adopt the practice of sanitizing your inputs. Every developer should know that user-manipulated input is always dangerous.

The following XSS injection was found on your webpage:

image

The picture above shows you the result of injecting code, and the code causing this is the following line
echo "<div class='alert alert-warning' role='alert'>/r/" . $_GET["subreddit"] . " is not a valid subreddit</div>"; in your postlist.php file.

I would suggest the use of Regular expression or the following PHP functions
strip_tags
htmlentities
htmlspecialchars

Goodbye!

Support Docker IPv4

I don't have a ton of time atm to dig into this but I believe if I disabled IPv6 and try to startup a docker container it will fail. This is an issue with Nginx where it needs some kind of configuration change.

This is the error message I see when I disabled IPv6 on my Ubuntu server and ran your image

2024/02/20 15:54:29 [emerg] 7#7: socket() [::]:8080 failed (97: Address family not supported by protocol)
2024-02-20 15:54:29,760 INFO exited: nginx (exit status 1; not expected)
2024-02-20 15:54:29,761 INFO gave up: nginx entered FATAL state, too many start retries too quickly

I ran into a similar issue when running the nginx-proxy-manager image. But they had an environment param I could pass. I think a similar param might be needed and passed to nginx to support something similar (their codebase).

I did have this image working fine before I disabled IPv6, so that should eliminate some possible other issues.

Error with new oauth

Thanks for improving this app.

I provided my docker the credentials for the new app via env
It seems to work but outputs this error.

 [11-May-2023 09:36:42] NOTICE: fpm is running, pid 9
 [11-May-2023 09:36:42] NOTICE: ready to handle connections
 NOTICE: PHP message: PHP Warning:  include(auth.php): Failed to open stream: No such file or directory in /var/www/html/index.php on line 11
 NOTICE: PHP message: PHP Warning:  include(): Failed opening 'auth.php' for inclusion (include_path='.:/usr/share/php8') in /var/www/html/index.php on line 11
 2023/05/11 09:36:52 [error] 12#12: *1 FastCGI sent in stderr: "PHP message: PHP Warning:  include(auth.php): Failed to open stream: No such file or directory in /var/www/html/index.php on line 11PHP message: PHP Warning:  include(): Failed opening 'auth.php' for inclusion (include_path='.:/usr/share/php8') in /var/www/html/index.php on line 11" while reading response header from upstream, client: 192.168.16.28, server: _, request: "GET /?subreddit=worldnews&averagePostsPerDay=4 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "reddit-top-rss.mysite.org"
 NOTICE: PHP message: PHP Warning:  include(auth.php): Failed to open stream: No such file or directory in /var/www/html/postlist.php on line 11
 NOTICE: PHP message: PHP Warning:  include(): Failed opening 'auth.php' for inclusion (include_path='.:/usr/share/php8') in /var/www/html/postlist.php on line 11
 NOTICE: PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 23
 NOTICE: PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 26
 NOTICE: PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 32
 NOTICE: PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 41
 NOTICE: PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/postlist.php on line 41
 2023/05/11 09:36:54 [error] 12#12: *3 FastCGI sent in stderr: "PHP message: PHP Warning:  include(auth.php): Failed to open stream: No such file or directory in /var/www/html/postlist.php on line 11PHP message: PHP Warning:  include(): Failed opening 'auth.php' for inclusion (include_path='.:/usr/share/php8') in /var/www/html/postlist.php on line 11PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 23PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 26PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 32PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/sort-and-filter.php on line 41PHP message: PHP Warning:  Undefined variable $accessToken in /var/www/html/postlist.php on line 41" while reading response header from upstream, client: 192.168.16.28, server: _, request: "GET /postlist.php?subreddit=worldnews&score=&threshold=&averagePostsPerDay=4 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "reddit-top-rss.mysite.org", referrer: "https://reddit-top-rss.mysite.org/?subreddit=worldnews&averagePostsPerDay=4"

its a new container and manually built as per the git master dockerfile

edit: checked your dockerhub image same thing

Found XSS Reflected Injection vulnerabilities

Hello,
I was searching for vulnerabilities in code and I came across your repository. I have noticed that you do not perform any kind of Input sanitization in your code when selecting the "score", and while there isn't any kind of risk because the purpose of this webpage is just generating RSS feeds without any kind of data being persisted, it would be a good idea to adopt the practice of sanitizing your inputs. Every developer should know that user-manipulated input is always dangerous.

The following XSS injection was found on your webpage:
xss

and this is the poc url that you also can access:

http://80.211.128.73:5000/?subreddit=news&score=2134"><script>alert("This happens when you do not perform any kind of input sanitization :)")</script>

https://reddit-top-rss.herokuapp.com - Application Error

It seems like the application has an error on heroku.

Do you plan to fix the error, or is it just no longer being maintained?

As much as I enjoy the free instance, I can understand if the latter is the case, just wanted to check before thinking of a solution.

Thank you, this is great

Found your service a short while ago, and it finally enabled me to subscribe to many subreddits using my favorite rss reader and filter out all the noise.

So this is my way of saying that I care for the work you did and I appreciate it (currently daily 😁 ) very much!

Big thanks!

Best,
Markus

request: Reddit vote count to the feed title

Could be useful, I'm in general more likely to open links to posts with a higher number of upvotes and may ignore (i.e. mark as read in an RSS reader) submissions with less success, when not particularly relevant to me.

Filter NSFW

Is it possible to filter out NSFW posts so that it only shows SFW posts

User profile

Hi,

Sorry if this has been asked before. Can we also have rss for users profile? For example u/user (Reddit.com/u/user)

feature request, comments count?

it would be good idea to have comments count filter?
for example any technical sub, I want to read feed only when it has been commented

thanks again

Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 56

I am running reddit-top-rss as a local docker container. A couple days ago, it stopped working and now feeds showed up in my RSS reader.

I read about the authorization using the client_id and client_secret and added that to my container as described in the README.

However, it does still not work.

The docker logs show the following:

172.17.0.1 - - [06/Jun/2023:06:12:39 +0000] "GET /?subreddit=fantasy&threshold=50&view=rss HTTP/1.1" 500 5 "-" "Newsboat/2.31.0 (Linux x86_64)" "-" 0.393 0.393 . -
2023/06/06 06:12:39 [error] 9#9: *13170 FastCGI sent in stderr: "PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 56PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 56PHP message: PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/html/sort-and-filter.php:56
Stack trace:
#0 /var/www/html/rss.php(17): include()
#1 /var/www/html/index.php(25): include('...')
#2 {main}
  thrown in /var/www/html/sort-and-filter.php on line 56" while reading response header from upstream, client: 172.17.0.1, server: _, request: "GET /?subreddit=fantasy&threshold=50&view=rss HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "localhost:9999"

Visiting the website shows this:

image

Did I do something wrong with the credentials, or has reddit-to-rss stopped working?

Error sort-and-filter.php

Hi,
i have an issue and hope you still maintain this ;)
i don't know if there were some changes on reddit side or if my server had a hiccup

Here are the logs from docker:

2023-05-04 19:57:24,863 INFO supervisord started with pid 1
 2023-05-04 19:57:25,869 INFO spawned: 'nginx' with pid 7
 2023-05-04 19:57:25,876 INFO spawned: 'php-fpm' with pid 8
 [04-May-2023 19:57:25] NOTICE: fpm is running, pid 8
 [04-May-2023 19:57:25] NOTICE: ready to handle connections
 2023-05-04 19:57:26,957 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
 2023-05-04 19:57:26,958 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
 192.168.16.28 - - [04/May/2023:19:58:00 +0000] "GET / HTTP/1.1" 200 1177 "-" "Mozilla/5.0 (Windows NT 10.0; rv:112.0) Gecko/20100101 Firefox/112.0" "172.20.0.1" 0.009 0.008 . -
 192.168.16.28 - - [04/May/2023:19:58:01 +0000] "GET /postlist.php?subreddit=worldnews&score=&threshold=&averagePostsPerDay= HTTP/1.1" 200 105 "https://reddit-top-rss.mysite.org/?subreddit=worldnews" "Mozilla/5.0 (Windows NT 10.0; rv:112.0) Gecko/20100101 Firefox/112.0" "172.20.0.1" 0.126 0.124 . -
 192.168.16.28 - - [04/May/2023:19:58:10 +0000] "GET /postlist.php?subreddit=worldnews&score=1&threshold=&averagePostsPerDay= HTTP/1.1" 200 105 "https://reddit-top-rss.mysite.org/?subreddit=worldnews&score=1" "Mozilla/5.0 (Windows NT 10.0; rv:112.0) Gecko/20100101 Firefox/112.0" "172.20.0.1" 0.074 0.072 . -
 NOTICE: PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25
 NOTICE: PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25
 NOTICE: PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25
 NOTICE: PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25
 NOTICE: PHP message: PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/html/sort-and-filter.php:25
 Stack trace:
 #0 /var/www/html/rss.php(17): include()
 #1 /var/www/html/index.php(25): include('...')
 #2 {main}
   thrown in /var/www/html/sort-and-filter.php on line 25
 192.168.16.22 - - [04/May/2023:19:58:25 +0000] "GET /?subreddit=technology&averagePostsPerDay=2&view=rss HTTP/1.1" 500 5 "-" "Mozilla/5.0 (compatible; Miniflux/2.0.43; +https://miniflux.app)" "-" 0.096 0.096 . -
 2023/05/04 19:58:25 [error] 9#9: *11 FastCGI sent in stderr: "PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25PHP message: PHP Warning:  Trying to access array offset on value of type null in /var/www/html/sort-and-filter.php on line 25PHP message: PHP Warning:  Trying to access array offset
on value of type null in /var/www/html/sort-and-filter.php on line 25PHP message: PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/html/sort-and-filter.php:25
 Stack trace:
 #0 /var/www/html/rss.php(17): include()
 #1 /var/www/html/index.php(25): include('...')
 #2 {main}
   thrown in /var/www/html/sort-and-filter.php on line 25" while reading response header from upstream, client: 192.168.16.22, server: _, request: "GET /?subreddit=technology&averagePostsPerDay=2&view=rss HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "reddit-top-rss:8080"

i tried to make some sense of it and look if i could fix it myself but i dont know if it's a backend change of the json or something else?

hope you see this and thanks for providing this tool as long as you did

Link element may contain relative URL

I came across an item where the link element contains a relative URL instead of an absolute URL:

<item>
  <title>Stadia has officially gone 40 days without a new game announcement/release, feature update, or real community update. It has been out for 69 days. (self.Stadia)</title>
  <link>/r/Stadia/comments/eusxgc/stadia_has_officially_gone_40_days_without_a_new/</link>
  <guid isPermaLink="false">https://www.reddit.com/r/Games/comments/euum86/stadia_has_officially_gone_40_days_without_a_new/</guid>
  <description><![CDATA[<p><a href='https://www.reddit.com/r/Games/comments/euum86/stadia_has_officially_gone_40_days_without_a_new/'>Post permalink</a> </p>]]></description>
  <pubDate>Mon, 27 Jan 2020 21:19:27 +0000</pubDate>
</item>

Feedly seems to be able to deal with this, but I don't think it's valid. Or it could just be a bug in my own reader.

Thanks for the great project!

stop redirecting the frontpage to /r/pics

Requests to http://localhost/8080 are redirected to http://localhost/8080?subreddit=pics. Can you consider loading a blank page instead?

Currently I load the desired subreddit directly (see the second URL). A non-existing subreddit can also be used (e.g. http://localhost/3000?subreddit=name), but reddit-top-rss may keep making connection attempts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.