A natural language search engine for your personal notes, transactions and images

• Features
• Demos
  • Khoj in Obsidian
  • Khoj in Emacs, Browser
  • Interfaces
• Architecture
• Setup
  • Install
  • Configure
  • Run
• Use
  • Interfaces
  • Query Filters
• Upgrade
  • Khoj Server
  • Khoj.el
  • Khoj Obsidian
• Troubleshoot
• Advanced Usage
  • Access Khoj on Mobile
• Miscellaneous
• Performance
  • Query Performance
  • Indexing Performance
  • Miscellaneous
• Development
  • Visualize Codebase
  • Setup
    • Using Pip
    • Using Docker
    • Using Conda
  • Test
• Credits

• Natural: Advanced natural language understanding using Transformer based ML Models
• Local: Your personal data stays local. All search, indexing is done on your machine*
• Incremental: Incremental search for a fast, search-as-you-type experience
• Pluggable: Modular architecture makes it easy to plug in new data sources, frontends and ML models
• Multiple Sources: Search your Org-mode and Markdown notes, Beancount transactions and Photos
• Multiple Interfaces: Search using a Web Browser, Emacs or the API

These are the general setup instructions for Khoj.

Check the Khoj Obsidian Readme to setup Khoj with the Obsidian Plugin. Its simpler as it can skip the configure step below.

pip install khoj-assistant

khoj

1. Enable content types and point to files to search in the First Run Screen that pops up on app start
2. Click Configure and wait. The app will download ML models and index the content for search

• Khoj via Obsidian
  • Install the Khoj Obsidian plugin
  • Click the Khoj search icon 🔎 on the Ribbon or Search for Khoj: Search in the Command Palette
• Khoj via Emacs
  • Install khoj.el
  • Run M-x khoj <user-query>
• Khoj via Web
  • Open http://localhost:8000/ via desktop interface or directly
• Khoj via API
  • See the Khoj FastAPI Swagger Docs, ReDocs

Use structured query syntax to filter the natural language search results

• Word Filter: Get entries that include/exclude a specified term
  • Entries that contain term_to_include: +"term_to_include"
  • Entries that contain term_to_exclude: -"term_to_exclude"
• Date Filter: Get entries containing dates in YYYY-MM-DD format from specified date (range)
  • Entries from April 1st 1984: dt:"1984-04-01"
  • Entries after March 31st 1984: dt>="1984-04-01"
  • Entries before April 2nd 1984 : dt<="1984-04-01"
• File Filter: Get entries from a specified file
  • Entries from incoming.org file: file:"incoming.org"
• Combined Example
  • what is the meaning of life? file:"1984.org" dt>="1984-01-01" dt<="1985-01-01" -"big" -"brother"
  • Adds all filters to the natural language query. It should return entries
    • from the file 1984.org
    • containing dates from the year 1984
    • excluding words "big" and "brother"
    • that best match the natural language query "what is the meaning of life?"

pip install --upgrade khoj-assistant

• Use your Emacs Package Manager to Upgrade
• See khoj.el readme for details

• Upgrade via the Community plugins tab on the settings pane in the Obsidian app
• See the khoj plugin readme for details

• Symptom: Errors out complaining about Tensors mismatch, null etc
  • Mitigation: Disable image search using the desktop GUI
• Symptom: Errors out with "Killed" in error message in Docker
  • Fix: Increase RAM available to Docker Containers in Docker Settings
  • Refer: StackOverflow Solution, Configure Resources on Docker for Mac
• Symptom: pip install khoj-assistant fails while building the tokenizers dependency. Complains about Rust.
  • Fix: Install Rust to build the tokenizers package. For example on Mac run:

    brew install rustup
    rustup-init
    source ~/.cargo/env

  • Refer: Issue with Fix for more details

1. Setup Khoj on your personal server. This can be any always-on machine, i.e an old computer, RaspberryPi(?) etc
2. Install Tailscale on your personal server and phone
3. Open the Khoj web interface of the server from your phone browser. It should be http://tailscale-url-of-server:8000 or http://name-of-server:8000 if you've setup MagicDNS
4. Click the Install/Add to Homescreen button
5. Enjoy exploring your notes, transactions and images from your phone!

• The beta chat and search API endpoints use OpenAI API
  • It is disabled by default
  • To use it add your openai-api-key via the app configure screen
  • Warning: If you use the above beta APIs, your query and top result(s) will be sent to OpenAI for processing

• Semantic search using the bi-encoder is fairly fast at <50 ms
• Reranking using the cross-encoder is slower at <2s on 15 results. Tweak top_k to tradeoff speed for accuracy of results
• Filters in query (e.g by file, word or date) usually add <20ms to query latency

• Indexing is more strongly impacted by the size of the source data
• Indexing 100K+ line corpus of notes takes about 10 minutes
• Indexing 4000+ images takes about 15 minutes and more than 8Gb of RAM
• Note: It should only take this long on the first run as the index is incrementally updated

• Testing done on a Mac M1 and a >100K line corpus of notes
• Search, indexing on a GPU has not been tested yet

Interactive Visualization

git clone https://github.com/debanjum/khoj && cd khoj
python3 -m venv .venv && source .venv/bin/activate
pip install -e .

• Copy the config/khoj_sample.yml to ~/.khoj/khoj.yml
• Set input-files or input-filter in each relevant content-type section of ~/.khoj/khoj.yml
  • Set input-directories field in image content-type section
• Delete content-type and processor sub-section(s) irrelevant for your use-case

khoj -vv

Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML

# To Upgrade To Latest Stable Release
# Maps to the latest tagged version of khoj on master branch
pip install --upgrade khoj-assistant

# To Upgrade To Latest Pre-Release
# Maps to the latest commit on the master branch
pip install --upgrade --pre khoj-assistant

# To Upgrade To Specific Development Release.
# Useful to test, review a PR.
# Note: khoj-assistant is published to test PyPi on creating a PR
pip install -i https://test.pypi.org/simple/ khoj-assistant==0.1.5.dev57166025766

git clone https://github.com/debanjum/khoj && cd khoj

• Required: Update docker-compose.yml to mount your images, (org-mode or markdown) notes and beancount directories
• Optional: Edit application configuration in khoj_docker.yml

docker-compose up -d

Note: The first run will take time. Let it run, it's mostly not hung, just generating embeddings

docker-compose build --pull

• Install Conda

git clone https://github.com/debanjum/khoj && cd khoj
conda env create -f config/environment.yml
conda activate khoj
python3 -m pip install pyqt6  # As conda does not support pyqt6 yet

• Copy the config/khoj_sample.yml to ~/.khoj/khoj.yml
• Set input-files or input-filter in each relevant content-type section of ~/.khoj/khoj.yml
  • Set input-directories field in image content-type section
• Delete content-type, processor sub-sections irrelevant for your use-case

python3 -m src.main -vv

Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML

cd khoj
git pull origin master
conda deactivate khoj
conda env update -f config/environment.yml
conda activate khoj

pytest

• Multi-QA MiniLM Model, All MiniLM Model for Text Search. See SBert Documentation
• OpenAI CLIP Model for Image Search. See SBert Documentation
• Charles Cave for OrgNode Parser
• Org.js to render Org-mode results on the Web interface
• Markdown-it to render Markdown results on the Web interface