Giter Club home page Giter Club logo

ingestai / embedditor Goto Github PK

View Code? Open in Web Editor NEW
212.0 3.0 14.0 1.78 MB

⚡ GUI for editing LLM vector embeddings. No more blind chunking. Upload content in any file extension, join and split chunks, edit metadata and embedding tokens + remove stop-words and punctuation with one click, add images, and download in .veml to share it with your team.

Home Page: https://embedditor.ai

License: GNU Affero General Public License v3.0

Dockerfile 0.66% PHP 81.44% Shell 0.15% Blade 17.75%
embeddings llm vector-database vector-search vectorization datascience ml nlp nltk markup-language

embedditor's Introduction

Embedditor is the open-source MS Word equivalent for embedding that helps you get the most out of your vector search.

PHP version Laravel version

WebsiteDiscordTwitterDocumentationTry demo on IngestAI

Get the most out of your vector search

Embedditor is an open source embedding pre-reprocessing editor, that helps you edit GPT / LLM embeddings just as if it's a Microsoft Word document, so you can get the most out of your vector search, while significanty reducing costs of embedding and vector storage.

Join Our Community

Stargazers repo roster for @embedditor/embedditor

Features

Rich editor Interface

  • ⚡ Join and split one or multiple chunks with a few clicks
  • ⚡ Edit embedding metadata and tokens
  • ⚡ Exclude words, sentences, or even parts of chunks from embedding
  • ⚡ Select the parts of chunk you want to be embedded
  • ⚡ Add additional information to your mebeddings, like url links or images
  • ⚡ Get a nice looking HTML-markup for your AI search results
  • ⚡ Save your pre-processed embedding files in .veml or .jason formats

Pre-processing automation

  • ⚡ Filteer our from vectorization most of the 'noise', like punctuations or stop-words
  • ⚡ Remove from embedidng unsignificant, requently used words with TF-IDF algorithm
  • ⚡ Normalize your embedding tokens before vectorization

Benefits

Rich Spreadsheet Interface

  • ⚡ Optimized relevance of the content retrieved from a vector database
  • ⚡ Improved efficiency and accuracy in your AI / LLM-related applications
  • ⚡ Visually better looking search results with images, url links, etc
  • ⚡ Increased cost-efficiency with up to 30% cost-reduction on embedding and vector storage
  • ⚡ Full control over your data, effortlessly deploying Embedditor locally on your PC or dedicated envirement
  • ⚡ Save your pre-processed or ready embeddings in .json or .veml format to use it in LangChain, Chromat or any other Vector DB

Quick try

Sign up for free and try it in IngestAI.

GUI

Access Dashboard using: http://localhost:8080/

Screenshots

1 2 3 4

Installation

  1. Copy .env.example into .env

  2. Set the following settings in the .env

    OPENAI_API_KEY=

  3. Setup the project

  • php artisan migrate
  • php artisan db:seed
  • php artisan storage:link

embedditor's People

Contributors

andrey-vorobev-av avatar buzzillio avatar evgeniy-tropin avatar igor-shaev avatar vladimirzhukov avatar yevhentropin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

embedditor's Issues

Demo on IngestAI

Hi. Readme says that embedditor should be available as demo on https://ingestai.io/ but I could not find it. Is it still available?

I tried running the app myself but I'm pretty inexperienced with docker and backend devolopment, and those instructions are not detailed enough.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.