nekromoff / eia-scraper Goto Github PK

Go through past projects up to certain date (e.g. -10 days).

toto suvisi so #40, ale zatial to nemam premyslene, davam to sem ako zaklad, neskor rozsirim

81101 = Bratislava I
etc.

https://documentation.mailgun.com/en/latest/user_manual.html#webhooks

1. On event type: permanent failure
2. Send webhook notification to URL endpoint that will remove that email from watchers (any existing record with that email).

Use signed webhook to verify Mailgun: https://documentation.mailgun.com/en/latest/user_manual.html#webhooks

Report by email:

Reason yet unknown.

Currently, one email notification is sent per project

to allow preset payments for the service

+implement Mailgun API to send emails, if configured

Na koniec mailu pridat linku "unsubscribe all", ktora danu mailovu adresu unsubscribne zo vsetkych jej subscriptions/locations. Aj z tych, ktore v danom maile neboli obsiahnute.

some users enter quotes around locality, remove them

to automatically create template documents

Notification email is currently sent out as many times as watcher searches match project region/district/locality.

OSM + location point per each EIA project

Zacala vyskakovat tato chyba, viete to pls fixnut? Este predtym si premazte fingerprint v systemstate, lebo inak nezbehne crawl a chybu neuvidite.

zrusme u opakujucu sa sekciu: "Tento email dostavate...." pri linku na odhlasenie zo vsetkeho.

novy stlpec type do watchers

Tento vyuzijeme na odlisenie rozne typy sledovatelov - nekomercnych, komercnych, s limitom na pocet notifikacii mesacne, bez a pod. Zaroven potrebujeme evidovat pocet odoslanych notifikacii na jednotlivych watchers za urcite obdobie - nateraz dajme 3 mesiace (90 dni) retention, starsie sa mozu zmazat - tu stacia timestamps a pocet poslanych EIA notifikacii (pri viacerych spojenych do jedneho emailu sa rata kazdy EIA zaznam zvlast, napr 2 EIA projekty = pocet 2 v tabulke o odoslanych emailoch).

Nova tabulka watchers_log (?) moze mat takuto strukturu:

Typy uzivatelov nateraz:

• 9999: unlimited (db migracia na zaciatku prideli kazdemu tento typ)
• 1: basic/trial (5 notifikacii mesacne, nasledne sa uz emaily neposielaju)
• 10: commercial, 10 notifikacii mesacne
• ...dalsie urovne, zatial nedefinovane...
• 100: commercial, bez obmedzenia poctu zaznamov

Pred kazdym rozoslanim notifikacii sa najprv skontroluje type vo watchers pre danu emailovu adresu a overi sa v novej tabulke so zaznamom pocet odoslanych notifikacii za poslednych 30 dni.

• Ak je pocet mensi, ako limituje typ, posle sa notifikacia standardne
• Ak je pocet mensi, ale zaroven je sucasnym zaslanim dosiahnuty (presiahnuty, ak dochadza k spojeniu viacerych), posle sa notifikacia standardne + upozornenie, ze pre dostavanie dalsich notifikacii si je potrebne zmenit program na vyssi
• Ak je pocet vacsi ako limituje typ, notifikacia sa neposle

When sending the notification email, add date or name of the EIA project to subject (to prevent Gmail / email client from grouping all EIA notifications).