Description

Github action workflow to publish the redisvl package to GitHub Actions.

Acceptance Criteria

• Github action to publish to pypi
• Manually test with test.pypi

Description

Implement the index management command for creating, reading, and deleting an index.

Interface

usage: redisvl index <command> [<args>]

Index creation and manipulation commands

Commands:
        list        List available indices
        create      Create a new index
        delete      delete an existing index


positional arguments:
  command     Subcommand to run

optional arguments:
  -h, --help  show this help message and exit
  -s SCHEMA, --schema SCHEMA
                        Path to schema file
  --host HOST           Redis host
  -p PORT, --port PORT  Redis port
  -a PASSWORD, --password PASSWORD

Will have to wait until SearchIndex class in the library is completed in #7

Acceptance Criteria

• redisvl index list
• redisvl index create
• redisvl index delete

RedisVL currently only supports the OpenAI API for versions < 1.0.0. This prevents the use of features in more current versions of Langchain, e.g. langchain_openai.

Here is how i am using semantic cache

SemanticCacheObject = SemanticCache(
    name=cache_index,                               # underlying search index name
    prefix=cache_prefix,                            # redis key prefix for hash entries
    redis_url=redis_url,                            # redis connection url string
    distance_threshold=distance_threshold,          # semantic cache distance threshold
    ttl=ttl                                         # time to live each key in cache
)
query_embedding = <embedding of my query>
SemanticCacheObject.check(vector=query_embedding, return_fields=["prompt", "response", "metadata"]):

Now I want to filter the responses based on a specific column in the index, before applying the vector search.
Then how to apply filter condition on this semantic cache index in check() function.
Eg: I am expecting something like this

SemanticCacheObject.check(vector=query_embedding, return_fields=["prompt", "response", "metadata"], 
filter = <column name> )

Is this supported now in redis vl?

1. There is no direct way today to use another service than the OpenAI.
   It would be nice to be able to use any service with an API compatible with the OpenAI one.
   The target host seems to be hardcoded towards the Open AI service, I may want to use an on-prem/local LLM service, but as for today this is not possible.
2. Add support for mistral.ai equivalent

Currently, using redisvl there is a pattern emerging from the FilterExpression usage. Functions like the following are commonly being written by end users.

def build_filter_expression(years: List[int], categories: List[str]) -> FilterExpression:
    """
    Construct a filter expression based on the provided years and categories.
    Args:
        years (list): A list of years (integers or strings) to be included in the filter
                      expression. An empty list means there's no filter applied based on years.
        categories (list): A list of category strings to be included in the filter
                           expression. An empty list means there's no filter applied based
                           on categories.
    Returns:
        FilterExpression: A FilterExpression object representing the combined filter for both years and categories.
    """
    # Build filters
    year_filter = Tag("year") == [str(year) for year in years] if years else None
    category_filter = Tag("categories") == [str(category) for category in categories] if categories else None
    # Parse and create filter expression
    if not year_filter and not category_filter:
        return FilterExpression("*")
    if year_filter and category_filter:
        return year_filter & category_filter
    return year_filter or category_filter

This function is verbose and there are two specific problems

the presence of conditional values i.e. None
Chaining given the presence of conditionals

Deliverable:
Design and implement and approach that allows for a conditional value to be passed
Ensure that if None is passed, the following chaining operation is not effected.

example, the function should be able to look like.

def build_filter_expression(years: List[int], categories: List[str]) -> FilterExpression:
    """
    Construct a filter expression based on the provided years and categories.
    Args:
        years (list): A list of years (integers or strings) to be included in the filter
                      expression. An empty list means there's no filter applied based on years.
        categories (list): A list of category strings to be included in the filter
                           expression. An empty list means there's no filter applied based
                           on categories.
    Returns:
        FilterExpression: A FilterExpression object representing the combined filter for both years and categories.
    """
    # Build filters
    year_filter = Tag("year") == [str(year) for year in years]
    category_filter = Tag("categories") == [str(category) for category in categories]
    return year_filter & category_filter

AC:
Implement this capability with tests and examples in the notebooks.

https://github.com/RedisVentures/redisvl/blob/e3a97e9ed3c4adb6f766ed475cf40ae8cdb98336/redisvl/index.py#L88

Can we use redisvl for vectorization operations using open source redis server only without an additional redis stack server?

When loading an existing index, get_redis_connection method in connection.py should allow passing of an existing redis client instance as well

for example :
index = SearchIndex.from_dict(schema)
index.set_client(redis)

works

But, this won’t work for SerachIndex.from_existing

This prevents users from passing redis connection objects created outside to be passed into while loading an existing index

I love this package so far, we're adding it to https://github.com/BerriAI/litellm. I want to make sure this package will be maintained for the next year?

(asking because it's not part of the redis python lib)

Description

Jupyter Notebook that takes one of the current simple test examples and walks through using redisvl as a python library and a cli.

Acceptance Criteria

• Well explained notebook with example usage for simple data in a jupyter notebook.

I see the index data getting cleared after about 5-10 mins, does it clear the Index after X minutes ?

If i set num_results = 1, will it return the top match ?

query = VectorQuery(
            vector=[0.1, 0.1, 0.5],
            vector_field_name="litellm_embedding",
            return_fields=["response", "prompt", "vector_distance"],
            num_results=1
        )

Hello, I'm trying to use async mode - seeing this bug, can I get help ? cc @tylerhutcherson

Code to repro

Create Index

def __init__(
        self,
        host=None,
        port=None,
        password=None,
        redis_url=None,
        similarity_threshold=None,
        use_async = False,
        **kwargs,
    ):
        from redisvl.index import SearchIndex
        from redisvl.query import VectorQuery

        print_verbose(
            "redis semantic-cache initializing INDEX - litellm_semantic_cache_index"
        )
        if similarity_threshold is None:
            raise Exception("similarity_threshold must be provided, passed None")
        self.similarity_threshold = similarity_threshold
        schema = {
            "index": {
                "name": "litellm_semantic_cache_index",
                "prefix": "litellm",
                "storage_type": "hash",
            },
            "fields": {
                "text": [{"name": "response"}],
                "text": [{"name": "prompt"}],
                "vector": [
                    {
                        "name": "litellm_embedding",
                        "dims": 1536,
                        "distance_metric": "cosine",
                        "algorithm": "flat",
                        "datatype": "float32",
                    }
                ],
            },
        }
        if redis_url is None:
            # if no url passed, check if host, port and password are passed, if not raise an Exception
            if host is None or port is None or password is None:
                raise Exception(f"Redis host, port, and password must be provided")
            redis_url = "redis://:" + password + "@" + host + ":" + port
        
        print_verbose(f"redis semantic-cache redis_url: {redis_url}")
        if use_async == False:
            self.index = SearchIndex.from_dict(schema)
            self.index.connect(redis_url=redis_url)
            try:
                self.index.create(overwrite=False)  # don't overwrite existing index
            except Exception as e:
                print_verbose(f"Got exception creating semantic cache index: {str(e)}")
        elif use_async == True:
            print("creating async semantic cache index")
            schema["index"]["name"] = "litellm_semantic_cache_index_async"
            self.index = SearchIndex.from_dict(schema, use_async=True)
            self.index.connect(redis_url=redis_url)
            try:
                self.index.create(overwrite=False)  # don't overwrite existing index
            except Exception as e:
                print_verbose(f"Got exception creating semantic cache index: {str(e)}")

Query index

        from redisvl.query import VectorQuery
        import numpy as np

        # query

        # get the messages
        messages = kwargs["messages"]
        prompt = ""
        for message in messages:
            prompt += message["content"]

        # convert to embedding
        embedding_response = await litellm.aembedding(
            model="text-embedding-ada-002",
            input=prompt,
            cache={"no-store": True, "no-cache": True},
        )

        print("async got embedding_response", embedding_response)

        # get the embedding
        embedding = embedding_response["data"][0]["embedding"]

        query = VectorQuery(
            vector=embedding,
            vector_field_name="litellm_embedding",
            return_fields=["response", "prompt", "vector_distance"],
        )
        print("async querying semantic cache", query)
        results = await self.index.aquery(query)
        print("async got results", results)

stacktrace

An exception occurred: Traceback (most recent call last):
  File "/Users/ishaanjaffer/Github/litellm/litellm/caching.py", line 972, in async_get_cache
    cached_result = await self.cache.async_get_cache(cache_key, messages=messages)
  File "/Users/ishaanjaffer/Github/litellm/litellm/caching.py", line 472, in async_get_cache
    results = await self.index.aquery(query)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/redisvl/index.py", line 106, in wrapper
    await check_async_redis_modules_exist(_redis_conn.client)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/redisvl/utils/utils.py", line 60, in check_async_redis_modules_exist
    installed_modules = await client.module_list()
TypeError: object list can't be used in 'await' expression

Hi, is there a way to hold a pre defined redis cache instance somewhere? I currently define everytime a request comes in a new one what takes way too loong.
So there must be the apprich to hold it somehow? Everything i can think about is pickling it and save it but that sounds not right. What are your approaches?

After initializing an index() I need to know if it was created successfully, Can you expose something like this ?

Description

redisvl.readers will be a module of classes named after the method used to read data from specific formats. For example, the pandas reader will be implemented first with support with csv, and pickled dataframe formats.

Interface

class Reader:

    support_map = {
        "pickle": self._from_pickle,
        "json": self._from_json,
        "parquet": self._from_parquet,
        "csv": self._from_csv
    }

    def __init__(data_format="pickle"):
        self._data_format = data_format

    @staticmethod
    def _from_pickle() -> t.Iterable[t.Dict[str, t.Any]]:

    @staticmethod
    def _from_json() -> t.Iterable[t.Dict[str, t.Any]]:

    @staticmethod
    def _from_parquet() -> t.Iterable[t.Dict[str, t.Any]]:
    
    @staticmethod
    def _from_csv() -> t.Iterable[t.Dict[str, t.Any]]:

    def __iter__():
        self.support_map[self._data_format]()

The Reader will be imported by the user and passed to the SearchIndex class. The SearchIndex will use the Reader to get data prior to loading it into redis.

The interface isn't complete stable, but should be more flushed out after the pandas reader is adapted to it.

Out of scope here

out of scope items, but worth thinking about while developing.

• multifile reader
• distributed reader (i.e. Dask)
• larger than memory reader
• conversion of vectors to bytes

Acceptance Criteria

• Implemented the redisvl.readers.pandas module adhering to the above interface.
• tests confirming support for
  • Pickled dataframe
  • csv
• Use in CLI load command

An easy fix... but this is something we should address to prevent any footguns:

index = SearchIndex.from_dict({"index": {"name": "tyler"}, "fields": {"text": [{"name": "test"}], "vector": [{"name": "test", "dims": 3, "algorithm": "flat"}]}})

for example yields an error from Redis because there is a duplicate field name.

One possibility for fixing this is to flip the schema spec on it's head and make the field name the "key" by which we organize the schema:

index:
    name: hash-test
    prefix: hash
    storage_type: hash

fields:
    sentence:
        - type: text
    embedding:
        - type: vector
          dims: 768
          algorithm: flat
          distance_metric: cosine

I want to use Amazon bedrock models as vectorizer but there is no such vectorizer. How can i use such embedding models?

I have created a custom vectorizer:

`import os
import boto3
from typing import Callable, Dict, List, Optional

from tenacity import retry, stop_after_attempt, wait_random_exponential
from langchain_community.embeddings.bedrock import BedrockEmbeddings
from tenacity.retry import retry_if_not_exception_type

from redisvl.utils.vectorize.base import BaseVectorizer

class AmazonBedrockTextVectorizer(BaseVectorizer):

def __init__(
    self, model_id: str = "amazon.titan-embed-text-v1", api_config: Optional[Dict] = None
):
    # Set up AWS credentials
    aws_access_key_id = api_config.get("aws_access_key_id") if api_config else os.getenv("AWS_ACCESS_KEY_ID")
    aws_secret_access_key = api_config.get("aws_secret_access_key") if api_config else os.getenv("AWS_SECRET_ACCESS_KEY")
    region_name = api_config.get("region_name", "us-east-1") if api_config else os.getenv("AWS_REGION", "us-east-1")
    
    if not aws_access_key_id or not aws_secret_access_key:
        raise ValueError("AWS access key and secret key are required.")
    
    # Initialize Bedrock client
    client = boto3.client(
        'bedrock-runtime',
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
        region_name=region_name
    )

    # print(f"Client --> {client}")

    dims = self._set_model_dims(model=model_id, client=client)
    super().__init__(model=model_id, dims=dims, client=client)

@staticmethod
def _set_model_dims(client, model) -> int:
    try:
        bedrock_em_model = BedrockEmbeddings(model_id=model,client=client)
        embedding = bedrock_em_model.embed_query("dimension test")
    except (KeyError, IndexError) as ke:
        raise ValueError(f"Unexpected response from the Cohere API: {str(ke)}")
    except Exception as e:  # pylint: disable=broad-except
        # fall back (TODO get more specific)
        raise ValueError(f"Error setting embedding model dimensions: {str(e)}")
    return len(embedding)

@retry(
    wait=wait_random_exponential(min=1, max=60),
    stop=stop_after_attempt(6),
    retry=retry_if_not_exception_type(TypeError),
)
def embed(
    self,
    text: str,
    preprocess: Optional[Callable] = None,
    as_buffer: bool = False,
    **kwargs,
) -> List[float]:
    if not isinstance(text, str):
        raise TypeError("Must pass in a str value to embed.")

    if preprocess:
        text = preprocess(text)
    
    bedrock_em_model = BedrockEmbeddings(model_id=self.model,client=self.client)

    embedding = bedrock_em_model.embed_query(text)

    # print(f"\n\n **** Embedding: {embedding} ****")

    return self._process_embedding(embedding, as_buffer)

@retry(
    wait=wait_random_exponential(min=1, max=60),
    stop=stop_after_attempt(6),
    retry=retry_if_not_exception_type(TypeError),
)
def embed_many(
    self,
    texts: List[str],
    preprocess: Optional[Callable] = None,
    batch_size: int = 10,
    as_buffer: bool = False,
    **kwargs,
) -> List[List[float]]:
    input_type = kwargs.get("input_type")

    if not isinstance(texts, list):
        raise TypeError("Must pass in a list of str values to embed.")
    if len(texts) > 0 and not isinstance(texts[0], str):
        raise TypeError("Must pass in a list of str values to embed.")
    if not isinstance(input_type, str):
        raise TypeError(
            "Must pass in a str value for cohere embedding input_type.\
                See AWS" 
        )

    embeddings: List = []

    bedrock_em_model = BedrockEmbeddings(model_id=self.model,client=self.client)

    for batch in self.batchify(texts, batch_size, preprocess):


        response = bedrock_em_model.embed_query(texts=batch)

        embeddings += [
            self._process_embedding(embedding, as_buffer)
            for embedding in response.embeddings
        ]

    # print(f"\n\n **** Embeddings: {embeddings} ****")
    return embeddings`

It is working perfectly and creating vectors.

But when I check the vectors dimensions for Huggingface, OpenAI, AmazonTitan Models

Hf : 768 AZ : 1536 OpenAI : 1536

Here I found one similarity that amazon titan and OpenAI models have same dimensions. and its greater than HFace.

I guess that why it giving this error:

ResponseError: Error parsing vector similarity query: query vector blob size (6144) does not match index's expected size (3072).

While using Hugging face default model its working great.

For example...

OpenAITextVectorizer(model="text-embedding-ada-002", api_config={"api_key": <openai_key>})

Should be:

OpenAITextVectorizer(model="text-embedding-ada-002")

....assuming that OPENAI_API_KEY is set in the current env.

In some cases we need to perform multiple queries to be able to extract the document we need.

First case:
Query1 without return fields: returns the id of keys (with with vector_distance or other) => The SubsetA

=> If I need to perform a second query, Query2 taking SubsetA as input to perform a new VectorQuery limited to this subset:
Search query language permits me to use INKEYS keyword to limit the scope of a query to a subset of keys

Second case:
Query1 without return an indexed id field (with with vector_distance or other) => The SubsetB
=> If I need to perform a second query, Query2 taking SubsetB as input to perform a new VectorQuery limited to this subset:
Search query language permits me to use a TAG filter on the id field to limit the scope of a query to a subset of id

in both case we may imagine N steps.

It would be nice to provide helper method , enrich query object to be able to chain the Queries.

Tried the given code

from redisvl.extensions.llmcache import SemanticCache
llmcache = SemanticCache(
    name="llmcache",                     # underlying search index name
    prefix="llmcache",                   # redis key prefix for hash entries
    redis_url="redis://localhost:6379",  # redis connection url string
    distance_threshold=0.1               # semantic cache distance threshold
)

and getting given error

using python 3.9 and redisvl==0.2.1
ValueError: Required Redis database module searchlight with version >= 20600 not installed. Refer to Redis Stack documentation:

Executing:

from redisvl.query.filter import Text, Tag, Num, FilterExpression
from redisvl.query import VectorQuery

unionTag = Tag("id_texte") == {"VA2", "VA4", "VA1"}

query1 = VectorQuery(
vector = vector1f,
vector_field_name = "texte_vec",
return_fields = ["titre", "questions", "keyword", "texte", "id_texte"],
num_results = 5
)
queryf = VectorQuery(
vector = vector1f,
vector_field_name = "texte_vec",
return_fields = ["titre", "questions", "keyword", "texte", "id_texte"],
num_results = 5,
filter_expression=unionTag
)

print(query1)
print(unionTag)
print(queryf)
filter_query_2 = query1.set_filter(unionTag)
print(filter_query_2)
=====================Results=======
*=>[KNN 5 @texte_vec $vector AS vector_distance] RETURN 6 titre questions keyword texte id_texte vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 5
@id_texte:{A2|A4|A1}
@id_texte:{A2|A4|A1}=>[KNN 5 @texte_vec $vector AS vector_distance] RETURN 6 titre questions keyword texte id_texte vector_distance SORTBY vector_distance ASC DIALECT 2 LIMIT 0 5
None

I would expect to be able to update a VectorQuery with a filter_expression

Limitations

Current supported types for HNSW are FLOAT32 and FLOAT64.

Feature Request

Support for binary and scalar embedding quantization(https://huggingface.co/blog/embedding-quantization) using HNSW.

Description

Github action to run the pytest suite while a Redis container is running with multiple versions of RediSearch

Acceptance Criteria

• GA workflow to run pytest suite
• add running redis instance in container to action
• enable tests on push and pull request
• require tests to pass to merge PRs

Update to Pydantic 2.0+

I took a look at patching it.
This project's @validator is easily updated to @field_validator.
There is a breaking unit test: tests/integration/test_llmcache.py that I was not able to resolve.

I am following the official documentation scrips for semantic cache. In the following code

from redisvl.extensions.llmcache import SemanticCache
llmcache = SemanticCache(
    name="llmcache",                     # underlying search index name
    redis_url="redis://localhost:6379",  # redis connection url string
    distance_threshold=0.2               # semantic cache distance threshold
)
llmcache.store(
    prompt="What is the capital city of France?",
    response="Paris",
    metadata={"city": "Paris", "country": "france"}
)
llmcache.check(prompt=question)[0]['response']

I have these questions,

1 - In llmcache.store() hope I can store the custom vector of the prompt directly rather than prompt. That custom vector can be generated by any embedding model like sentence transformer, minilm-l12-v2, openai embeddings, huggingface embeddings...etc
2 - Is there any embedding length limitation to store using llmcache.store()? Shall I use a vector with any length?
3 - In llmcache.check() hope we can pass the vector( from sentence transformer, minilm-l12-v2, openai embeddings, huggingface embeddings,..etc) directly for the semantic matching purpose,rather than the query
4 - inside llmcache.check() which distance measure using for finding the semantic similarity ? is it cosine similarity or any other? do we have the privilege to configure that?

Has progress begun on llm session management? It would be an awesome feature.

Currently, our apps pull past conversation messages straight from our db upon each request, identified by a conversation id.

However, saving on-going conversations in-cache would be much faster and off-load some work from the db.

Is there an idea on how this would be implemented? Or should I pr something from scratch?

For instance Dialect 2 does not permit to retrieve multi-value data stored in json
Dialect 3 and 4 do permit.

See https://github.com/RedisVentures/redisvl/blob/8d388011c22fea168c9ed3f40c2543614d655304/redisvl/schema/schema.py#L117

        # From Dict
        schema = IndexSchema.from_dict({
            "index": {
                "name": "user-index",
                "prefix": "user",
                "storage_type": "json",
            },
            "fields": [
                {"name": "user", "type": "tag"},
                {"name": "credit_score", "type": "tag"},
                {
                    "name": "embedding",
                    "type": "vector",
                    "attrs": {
                        "algorithm": "flat",
                        "dims": 3,
                        **"distance_metrics": "cosine",**
                        "datatype": "float32"
                    }
                }
            ]
        })

Description

Implement the SearchIndex class in the redisvl.index module.

SearchIndex will contain most of the methods required to interact with the VSS capabilities of Redis.

There will be three types of intializers that can be used:

1. from_yaml for file-based schema
2. from_dict for python dictionary based schema (library usage)
3. from_existing for communicating with existing schema.

Interface

class SearchIndex:

    def __init__(self, schema, sync=False):
        # initialize redis client

    @classmethod
    def from_yaml():
        pass

    @classmethod
    def from_dict():
        pass

    @classmethod
    def from_existing():
        pass

    # create index
    def create(self):
        pass

    def delete(self):
        pass

    def info(self):
        # if init, pull from redis, otherwise, return schema
        pass

    # load vectors
    async def load_concurrent(self, reader, concurrency=5):
        pass

    def load(self, reader):
        pass

Acceptance Criteria

• SearchIndex class
• Pytests confirming functionality
• Update library examples

It would be nice to have an optional mode permitting to monitor the number of token the Semantic Cache permits to save.
To do so, today we can store the token as metadata item.

from langchain_community.callbacks import get_openai_callback

with get_openai_callback() as cb:
        response = agent.run(input=user_query, callbacks=[retrieval_handler])
        red.set("query_token_total",cb.total_tokens)
        red.incr("session_token",cb.total_tokens)
    return response

(...)

metadata=dict()
                if int(red.get("query_token_total").decode('utf-8'))!=0:
                    metadata["token"]= int(red.get("query_token_total").decode('utf-8'))
                    llmcache.store(user_query, response, metadata=metadata)
                else:
                    llmcache.store(user_query, response)

I am wondering if it would make sense to make such easier as it permits to highlight the Semantic Cache value.

The Alias capability of Indexes with ft.aliasadd, ft.aliasdel, FT.ALIASUPDATE permits agility in making index and apps evolving.
Would be nice to be able to handle this at SearchIndex level directly.

• Support for both Sync and Async client connections
• Proper checks and handling
• Ability to pass in external redis connections to use instead of creating new ones

Also mentioned here: #79

Description

Interface

The theme of this interface is simplicity. We don't want to cover every possible use case, but we want to simplify the string construction methods that are currently used to build query strings in Redis.

To accomplish this, SearchEngine will be primarily geared towards vector search. We will allow the query to be returned and then passed to the vector_search command such that redis-py arguments and fluent modifiers can be chained to the result of create_vector_query.

class SearchEngine:

    def __init__(self, index):
        self._index = index
        pass

    def create_vector_query(self, topk, return_fields, sort_by, ) -> Query:
        pass

    def vector_search(self, query, query_vector, **kwargs):
        pass

Acceptance Criteria

• Implement the above interface in the query module
• Update existing tests and examples
• Add new tests to confirm functionality

The dialect 3 api changes the outputs for JSON:

q = VectorQuery(
    vector=[0.23, 0.12, -0.03, 0.98],
    vector_field_name="embedding",
    filter_expression=filter_expression,
    return_fields=["name", "description", "price"],
    dialect=3
)
index.query(q)

returns:

[{'id': 'product:36edbfd1372144759975f01fe6968bbf',
  'vector_distance': '1.08839428425',
  'name': '["Wireless earbuds"]',
  'description': '["Wireless Bluetooth in-ear headphones"]',
  'price': '[64.99]'}]

BUT dialect 2 returns:

[{'id': 'product:36edbfd1372144759975f01fe6968bbf',
  'vector_distance': '1.08839428425',
  'name': 'Wireless earbuds',
  'description': 'Wireless Bluetooth in-ear headphones',
  'price': '64.99'}]

(which is the correct and user expected format.

Additionally FilterQuery types break when using dialect=3...

Need to fix the parsing layer in RedisVL.

Description

API documentation for the python library for redisvl. Autogenerate this using sphinx or similar tooling.

Acceptance Criteria

• Method to create barebones API documentation using a tool like sphinx
• Add documentation dependencies to [dev] extras require
• add instructions for generating documentation to README.