svandragt / fafi Goto Github PK
View Code? Open in Web Editor NEWWeb content indexing and search tool
Web content indexing and search tool
Links containing errors should be skipped for a number of hours as they slow down the indexing.
https://developers.google.com/safe-browsing/v4/update-api
Avoid pulling in malware and help users discover malicious links.
Must preserve privacy of the user.
Dependabot couldn't authenticate with https://pypi.python.org/simple/.
You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.
default to application data (using appdirs).
We can always add a cli option to set our own.
I deleted my firefox profile and fafi is trying to use a profile that no longer exists.
Handle profile deletion by asking the user to pick a new sqlite.
This should not affect already indexed bookmarks.
Spike:
Currently the default is 7 results, but it should be customisable through a commandline parameter.
404s should be skipped.
5xx should be errored
Not sure about following redirects.
blocks further dev
if we use direnv to set the vaiable to the poetry environment, then have the spec file read the env var, then the spec file can be included into version control
blocked by #9
Firefox supports multiple profiles. Therefore fafi should be able to index bookmarks relating to any of the available profiles (no parameter), or a specific profile when one is specified.
spin up a basic web server and a search/results UI
task runner, need to first publish my taskfile fork.
would be nice to install fafi
using pipx install fafi
or something like it which would abstract away all the environmental hoops.
poetry build && poetry publish
pyinstaller --onedir src/fafi/main.py --name fafi
cd dist
mv fafi app
ln -s app/fafi fafi
zip -r ../fafi.zip
mv ../fafi.zip .
Should be created in central location (already have appdirs).
This is an issue when installing fafi via pipx.
INdexing is currently done one link at a time, we can speed it up using multiple indexers.
root@unanimous-staghound:~/.local/bin# ./fafi
Traceback (most recent call last):
File "./fafi", line 5, in <module>
from fafi.fafi import cli
File "/root/.local/pipx/venvs/fafi/lib/python3.8/site-packages/fafi/fafi.py", line 7, in <module>
import appdata
ModuleNotFoundError: No module named 'appdata'
need to keep the url to avoid reimporting, but the title, text can be cleared.
might require a new field (isDeleted) to avoid indexing.
Index bookmarks old to new, and save the last indexed bookmark created date as state.
When indexing, index only bookmarks newer than that date.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.