dgtlmoon / changedetection.io Goto Github PK

Looking to use this instead of visualping but I need email notifications.

Whats the best way to hack that together with this application? (if at all possible) #

Hi guys,

Is there a way to get notified only when a certain "text" appears on the website ?

For example to monitor the content of certain websites for specific keywords ? And only when those keywords appear on the website to get notified ?

I tried the "tag" option, but it doesn't seem to work. For example I'm trying to watch certain websites for "FDA approvals", is there a way for this tool to include, notifications for keywords like "FDA"

Trying to remember how it looked previously but is the cog for changing the overall settings now missing?

Edit, it used to look like this

Better would be a way to click on the diff text (green/red insert/delete) and have some mechanism that lets us tell the machine that we want to ignore that block

can be based on context of the above 2 and below 2 lines

run before generating the checksum

maybe using the text highlight JS to get the lines/text selected, then we know where to set the context

and/or click on an individual line to add that ignore text to a config

[feature] A chrome+FF plugin for sending the current URL to your private changedetection install

Would be nice to have a list of tags to choose from.

Ability to disable a site in the UI without deleting it.

        "5f3b8505-0a97-4b3c-8ba3-d570eb8f1d41": {
            "url": "http://deadmetalclothing.bigcartel.com",
            "tag": "",
            "last_checked": 1614755165,
            "last_changed": 1614755165,
            "last_viewed": 1613930169,
            "newest_history_key": "1614755165",
            "title": null,
            "previous_md5": "6c7060bf62b5c90e298b641f2affab20",
            "uuid": "5f3b8505-0a97-4b3c-8ba3-d570eb8f1d41",
            "headers": {},
            "history": {
                "1612559229": "/datastore/5f3b8505-0a97-4b3c-8ba3-d570eb8f1d41/ea6644dcd0070f732884711f3de49e2b.stripped.txt",
                "1613930169": "/datastore/5f3b8505-0a97-4b3c-8ba3-d570eb8f1d41/ea6644dcd0070f732884711f3de49e2b-1613930170.0731645.stripped.txt",
                "1614755165": "/datastore/5f3b8505-0a97-4b3c-8ba3-d570eb8f1d41/6c7060bf62b5c90e298b641f2affab20-1614755173.1128836.stripped.txt"
            },
            "ignore_text": [],
            "last_error": false,
            "last_check_status": 503,
            "viewed": false
        },

I think nearly always the content could be ignored?

Should be configurable tho, incase someone is actually wanting to see that content

expanding on #4

when a page hasnt been updated in x amount to make a notification.

example i have cron job do action. if no file changed in 1 day there is problem. send alert!

Thank you for the great project! I have been looking for something like this for a long time.

Would it be possible to add screenshot function to take screenshot of the full page when change occurs?

The Diff page seems to be mixed up. I tried to find any logic behind it and it seems that it compares the newest updated page with the last added page. I have attached the backup archive but the files themselves seem to be correct.
changedetection-backup-1612340948.zip

services like www.wachete.com can notify based on if X percent of change has happened.

So things like, "updated 2 days ago" from "updated 1 day ago" on a series of posts don't trigger notifications if you have a 5% threshold specified.

Currently:
The URL is used as the title, which makes finding pages in a long list quite difficult - especially if there's multiple URLs with only a numeric ID in the querystring.

Feature request:
Ability to add a custom page title, instead of using the URL. When fetching the page, grab the <title/> and automatically use that, and allow the user to override the <title/> with their own if they prefer (similar to versionista).

In the list of monitored pages, use the page <title/> by default, or custom title set by the user.

I have most of my code hidden and waits for js to show it,
it will be great if i can just directly compare the html changes in diff,

currently its only comparing the text that is displayed.,

please help.

Would be nice to have a password/auth system enabled, better for people self hosting on a public web server

[feature] add proxy setting, handy for people running on tor etc

Hi there, thanks for your tool.
I can't get the test notifications working.

Steps to reproduce

1. Add new URL
2. Add apprise URLs to "Notification URLs"
3. Check checkbox "Send test notification on save."
4. Click on "Save"

Expected behavior

The tool sends out a test notification to the given URLs

Actual behavior

No notification is sent.

It would be really useful to be able to set the polling frequency individually per monitored site.

Some sites I need hourly checks and some is enough with monthly. Then it makes sense to spare the server 719 unnecessary requests every month.

Would be great to have options to update the check once/day or once/week, etc.

I've watched a website for about a month that changes twice a week and haven't gotten anything added to my RSS feed. Here's what the feed looks like:

<?xml version='1.0' encoding='UTF-8'?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0"><channel><title>changedetection.io</title><link>https://changedetection.io</link><description>Feed description</description><docs>http://www.rssboard.org/rss-specification</docs><generator>python-feedgen</generator><lastBuildDate>Wed, 02 Jun 2021 20:04:28 +0000</lastBuildDate></channel></rss>****

Did I forget to set something up? Looks like RSS works for others

I can't access the GUI after using the docker run command provided. Any ideas?

lots of docker has docker-compose.yml

this repo no have. help?

First of all: thank you for your work!

I've seen you recently added "BASE URL" to notifications if it is set in the docker-compose-file (#65 ). Is it possible to also append the diff-URL for the specific website which has changed so that one can click this URL in the notification to directly navigate to [BASE-URL]/diff/[uuid]?

Thanks for your efforts and kind regards.

This is a follow-up idea to #2 where the change can be highlighted, and whenever such a change on the website occurs, an RSS feed entry could be made. This might exchange for some the need for notification on the webpage...

A nice idea, could be to have a checkbox which enables browser push notifications when that page changes.

This way you can know when a very important page has changed..

It'd be cool to be able to restrict the parts of a website to be watched using XPath queries, e.g. to only watch /html/body/main/div[@id='content']. You might find inspiration at muety/website-watcher.

how to set up
You should add this by typing.
I've been working for an hour and still haven't set it up. (Windows)

your tool seems to be working pretty well for a persons who don't need refreshes very often...Is there a way I can customize your tool to detect changes to webpages every 1 minute or 30 seconds ?

I'm trying to watch some press releases website for some hot stocks, on specific tags and 5 minutes is a lot of time on the stock market...It would be great if you could give the possibility for the user to customize the refresh rate for a minute or 30 seconds...

I'm sure a lot of traders will use it if you can drop under 1 minute...information is power and finding out first about your stock can make the difference between winning or loosing...

If you don't have time to upgrade the tool to my request, I have some limited python skills myself, I could try to modify your tool myself if you can share some insight on where to check the time interval inside your code...

thanks and waiting to hear from you

Adding in XPath or CSS selection would give the ability to only monitor a portion of the website for changes.

My changedetection.io server is behind a reverse proxy and I use individual folders per site. I see a new URL base path option for just notifications but no way to specify subfolders for the site.

Ex: domain.com/changedetection.io/

First of all, thank you for this project.

I'm currently using a lot of notifications, these are working pretty fine.
It would be great, if you can add an URL for the webpage/UI to the notification. So you can easily navigate to the webui after you receive anotification.
Maybe it would be good to set these URL on the settingspage or as a docker variable.

Thank you in advance.

It would be nice to have the ability to send a notification when a web page change.
But it would be a hassle to implement all notification channel. (Email, Slack, Telegram, Pushover, Gotify,.....)
Maybe a simple bash script can be called with v-env that indicates the web page that changed.

I've deleted your sample entries from my dashboard, but that didn't clean up the data folder as I would have expected it. The old directories are still available and have not been deleted with the entry.

Even a timestamp change displayed on the web page will cause a false positive changedetection. All of the pages that I'd like to monitor have minor updates like this, which means that I can't utilise changedetection.io currently.

It would be great if you could target specific areas of the page, using CSS Selectors.

Add an example how to use changedetection.io to check for content changes inside of basic auth htpasswd protected pages (maybe add an automated test too)

Is it possible to make an image for arm architecture?

Hi

I've jumped updated my docker container with the latest version and now it won't start showing the error message above. Will try creating from scratch to see if the same issue exists

Edit: same problem seen when starting afresh

Some libraries to check out

• https://github.com/rtfpessoa/diff2html

UX ideas

• Maybe limit the red/green changes to within +/- n number of lines of context (don't show the whole text)
• Add a jump/scroll-to button to get to the diff
• View all context's of all diff's in all websites as one page (for example, nice to check all the sites in the morning and see them all as one 'stream')

Is there a way to specify a port different from 5000?

Hi, thanks for the great project!

I have just installed your latest version of the project on my Ubuntu bionic machine and when I click on Backup in the webview, I get the following error:

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/eventlet/wsgi.py", line 573, in handle_one_response result = self.application(self.environ, start_response) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__ return self.wsgi_app(environ, start_response) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app response = self.handle_exception(e) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python3.8/site-packages/flask_login/utils.py", line 269, in decorated_view return func(*args, **kwargs) File "/app/backend/__init__.py", line 582, in get_backup backup_filepath = os.path.join(app.config['datastore_path'], backupname) KeyError: 'datastore_path'

Best regards,
st4hli

Some sites, especially behind Cloudflare, will not work properly because the only response is the CF captcha, e.g. https://www.maimmunizations.org/

This is difficult to diagnose currently because it's not possible/easy to see the current state of a page being checked.

Hi! I don't know how to install it.

`admin@myNode:~/changedetection.io $ docker run -d --restart always -p "192.168.1.62:5000:5000" -v datastore-volume:/datastore dgtlmoon/changedetection.io
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested

admin@myNode:~/changedetection.io $ uname -an
Linux myNode 5.4.83-v7l+ #1379 SMP Mon Dec 14 13:11:54 GMT 2020 armv7l GNU/Linux
`

I've been having issues updating settings for sites. I'm using the 0.31 container and these are the things I've noticed

1. minutes_between_check Settings are only applied when restarting container even though I'm using the webpage to update them and I can see the url-watches.json file being updated. Personally not a problem for me, but I didn't see it in the documentation and think it could be helpful to know.
2. Some sites share the same individual settings. changing one specific one changes the other.

The settings page doesn't work properly on an android phone on Chrome, even with the 'desktop site' option. Can't access the left side of the page even with zooming/scrolling. Other pages seem fine.

Is there a way to restore a backup besides manually extracting the zipped backup file to the new location on the filesystem?

Hi,

Thanks for this project.

It would be nice if we could bind to a vpn adapter.

That way, if a VPN is off or is being reconnected, the IP address won't leak.

qBittorrent offers to bind to either a adapter or IP address.

I think adapter is better because VPN ip addresses tend to change.

Thank you again.

Hello,
thank you for this awesome tool.

I noticed that it doesn't work on some pages, I suppose it is for Cloudflare ddos protection.

Please` turn JavaScript on and reload the page.
                                               
 DDoS protection by Cloudflare                 
                                               
 Ray ID: xxxxxxxxxxxxxxxx

Do you know if it's possible to bypass this error?
Thank you and have a good day,
Alessandro

when i press button i get

Sponsor dgtlmoon/changedetection.io
Some errors were encountered when parsing the FUNDING.yml file:

Some users provided are not enrolled in GitHub Sponsors. Apply to Sponsors.
Learn more about formatting FUNDING.yml.

have u icon for homer?

Currently all that appears in the RSS feed is the name of the page that has changed, clicking the link also just takes you to the page in question. Would it be possible to either include the details of what has changed in the feed or link through to the changedetection.io page that shows the differences?

I currently have the interval set to 12 minutes however the last checked is 2 hours, isn't it supposed to be 12 minutes maximum ?