Document bot and usage in README for FOSS release

Current Page

Proposed Changes

• User clicks telegram > channel
• User sees 2 buttons (maybe another row of 2 buttons like the others)
  1. Add Abbot - Telegram App
  2. Add Abbot - Manually

Button 1

• On click, reveal a set of instructions / steps and a final button that links to https://t.me/atl_bitlab_bot?startgroup=true
• Instructions / Steps
  1. If you have Telegram installed, click the Add Abbot Link button
  2. That will open Telegram on the device from which you clicked
  3. You will see a drop down list of all the group chats you're in
  4. Select the desired group and add Abbot to it
  5. Abbot will automatically send a message with further instructions

Button 2

• <insert manual steps of opening TG app, searching for abbot and adding him to group>

Right now, Strike is hard coded into the fn: main.py/abbot()

Goal:

• abstract the Strike class into a Payment class such that it can handle any number of possible payment integrations
• examples:
  • Strike
  • OpenNode
  • LNBits
  • Voltage
  • Home node
  • Cloud node
  • What else?

• file org
• utils
• debugging
• error handling
• etc

• Research paid / managed Vector Databases
• Research FOSS Vector Databases
  • https://milvus.io/
  • https://weaviate.io/
  • https://www.marqo.ai/
• Decide on one to use
• Setup integration in src/lib/db/vector.py
• Abstract setup enough to allow for toggling between different providers

In UI Telegram button should look like Nostr
When the user clicks telegram button the UI will show 2 buttons:

• DM Abbot
• Telegram group

When the user clicks on DM Abbot it will go straight to Telegram message
When the user clicks on Telegram group UI will show a Text box for the user to copy their telegram groupID or link (TBD)

Is your feature request related to a problem? Please describe.
Right now, messages are stored in a .jsonl file on the server which doesn't scale long term.
Need to migrate to a DB solution of some kind.

Describe the solution you'd like
Design decisions for DB

1. local or hosted or combo
2. DB type: sql or nosql or vector

=> Use a hosted pinecone DB storing embeddings for ease and speed

Describe alternatives you've considered
Functional Req:

• Setup hosted pinecone DB
• Connect it to app
• Turn text into embeddings
• Store embeddings in hosted pinecone DB
  Nonfunc Req (stretch):
• Store some amount of embeddings locally and redundantly in hosted DB

Additional context
Figure out how to use embeddings (i.e. vector data embeddings)

• https://platform.openai.com/docs/api-reference/embeddings/object
• https://platform.openai.com/docs/guides/embeddings/use-cases
• https://docs.pinecone.io/docs/overview
• https://docs.pinecone.io/docs/python-client

• Add payments page to front end
  • Something like /purchase or /subscribe
  • This should be a checkout page with a form a 2 inputs:
    • Chat link or chat name or chat ID (see #58)
• Add new routes to the /pages/api folder for calling the Strike API
  • Checkout https://atlbitlab.com/payments for an example implementation
  • Checkout https://github.com/ATLBitLab/atlbitlab.com/blob/master/pages/payments.tsx for example frontend code
  • Checkout https://github.com/ATLBitLab/atlbitlab.com/blob/master/pages/api/strike/invoice for example api route code

Stretch (nostr):

• Add paywall to frontend that requires LN invoice payment before adding Abbot to nostr channel
• Paywall gates the signing and sending of the BOT_CHANNEL_INVITE event

Right now, messages are stored in a .jsonl file on the server which doesn't scale long term. Need to migrate to a local noSQL DB.

Chat GPT API is throwing this warning message:

  "warning": "This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations"

Need to migrate to Chat GPT 4 using the chat/completions route

The wall we throw spaghetti on to see what sticks. Put any and all ideas you have for abbot: new functionality, feature requests, product ideas, business models, revenue streams, etc. this should be a stream of consciousness where we review and vote adhoc. All ideas here become backlog tickets. Top 3 most voted become to to-dos.

• Add a new section to help.tsx called "Why Admin?"
• Explain the need for giving Abbot admin privileges

Migrate bot from current server to own server

Continue development of the Processor class in src/lib/payments.py

• Build classes for various payment methods
  • Strike
  • OpenNode
  • LNBits
  • Voltage
• Upon Abbot being added to a new chat, create new mongo channel object in channels using default
• Add logic to Abbot class to calculate number of tokens per completion and cost per completion
• Add new telegram slash command /fund that takes an int arg in sats (e.g. 1000000 sats)

Stretch Goals:

• Update src/payments.py adding OpenNode, LNBits and Voltage processors

See ATLBitLab/abbot-backup#6

• Integrate the bot to nostr relays
• Define way to toggle filters based on what is desired
• Pull & store notes from relays on loop

• Change bash script shebang line from #!/usr/bin/bash to #!/usr/bin/env bash.
• Create flake.nix file.
  • Package nostr for Nix.
  • Package nostr-sdk for Nix.
• Update README file.

See #56 for context

Payment Methods

• Voltage (cloud node)
• Alby
• Self-sovereign LN Node
  • LND
  • CLN
  • LDK

• Depends on https://github.com/ATLBitLab/bot/issues/8
• Create new command /unleash
• Command is admin only
• Allows Abbot to talk to users directly in DMs without the need for using a command or tagging
• Restrict chat-like interactions to DMs, group chat must tag abbot directly or use a command
• Ways to have Abbot reply in group chat in a measured way
  • Only if tagged
  • After 5 messages sent

Describe the bug
Website is washed out and turns grey/white when light-mode is used in MacOS or iOS.

To Reproduce
Steps to reproduce the behavior:

1. Change appearance in MacOS or iOS

Either use Telegram or use the backend to design a better, more integrated /help menu

We should have a page for our team in the abbot/frontend folder. Add a team page and include team member profile pictures and bios.

Func Requirements

• New team.tsx file in frontend/pages
• React code to build a grid like structure: 2 columns x 3 rows with 2 members in each row
• Something like this
  

Code Design Suggestions

• These are not requirements, so feel free to ignore and code however you want
• The way I would approach this is:

1. Copy/paste the code in the index.tsx page into a team.tsx file
2. Rename the component export default function Team()
3. Delete all code in-between the <main></main> tags
4. Create new file in components/ folder called Member.tsx
5. Build a simple Member component with divs and an img tag passing in the image url and bios as Props
6. Use .map in index.tsx to iterate over objects for each team member that look like so

const members = [
    {
        "name": "bryan",
        "image": "@/public/team/bryan.jpg",
        "bio": "Bitcoin ipsum dolor sit amet..."
    },
    {
        "name": "brandon",
        "image": "@/public/team/brandon.jpg",
        "bio": "Bitcoin ipsum dolor sit amet...."
    }
]

Then down below in the <main> tag:

{
    members.map(member => <Member {...member} />)
}

See ATLBitLab/abbot-backup#7

See ATLBitLab/abbot-backup#3

We are throwing a pre TAB party on Sept 5 or 6 at ATL BitLab and want to demo Abbot. The following issues are vital to the demo success:

Functional Requirements

• https://github.com/ATLBitLab/bot/issues/3
• https://github.com/ATLBitLab/bot/issues/8
• https://github.com/ATLBitLab/bot/issues/1
• https://github.com/ATLBitLab/bot/issues/12

Non-Functional Requirements

• https://github.com/ATLBitLab/bot/issues/2

Remove any data from abbot repo
Make public
Update README.md with Hacktoberfest content

See ATLBitLab/abbot-backup#3

Requirements

• Telegram: need 2 new slash commands: unleash and leash

  1. /unleash
     • takes an optional argument that must be an integer
     • default is 5 if no arg passed
     • sets DM keys "config.unleashed" = true and "count" = arg | 5 to turn on the feature
     • arg = how many messages abbot waits before chiming in
  2. /leash
     • takes an optional argument that must be an integer
     • default is None if no arg passed, don't change the count
     • sets DB keys "config.unleashed" = false to turn off the feature

• Nostr

See ATLBitLab/abbot-backup#5

• Build a class for the Abbot bot to manage and update state
• note for open-abbot: Abbot meaning goes from Atl BitLab Bot => A Bitcoin Bot

class GPT:
    def __init__(self, api_key, model, name, personality):
        openai.api_key = api_key
        self.model = model
        self.name = name
        self.personality = personality
        self.messages = [dict(role="system", content=personality)]

    def __str__(self):
        return f"GPT(model={self.model}, name={self.name})"

    def __repr__(self):
        return f"GPT(model={self.model}, name={self.name}, personality={self.personality}, messages={self.messages})"

    def update_messages(self, telegram_message: Message | str | dict):
        prompt = try_get(telegram_message, "text") if type(telegram_message) == Message else telegram_message
        message_dict = dict(role="user", content=prompt)
        self.messages.append(message_dict)

    def chat_completion(self):
        response = openai.ChatCompletion.create(
            model=self.model,
            messages=self.messages,
        )
        return try_get(response, "choices", 0, "message", "content")


class ABBot(GPT):
    def __init__(
        self,
        token: str,
        name: str,
        handle: str,
        started: bool = True,
        unleashed: bool = False,
    ) -> object:
        self.token = token
        self.name = name
        self.handle = handle
        self.started = started
        self.unleashed = unleashed

    def __str__(self):
        return f"ABBot({self.name})"

    def __repr__(self):
        return f"ABBot(name='{self.name}', handle={self.handle}, started={self.started}, unleashed={self.unleashed})"

    def start(self):
        self.started = True

    def stop(self):
        self.started = False

    def unleash(self):
        self.unleashed = True

    def leash(self):
        self.unleashed = False

• Currently passing the entire "history" list to API model and using OpenAI latest gpt 4 185k
• As token count approaches 185k, the model has less and less tokens for the output response
• Also, by sending all messages to AI model, the cost to the group grows with each message since input & output tokens cost $
• Need a scalable approach for determining how many messages to send to AI model to balance response quality and cost

MVP

• Look at the token_count being updated with each message
• If its >= 92500, send half the history to AI model

        messages_history = self.history[self.history_len :] if self.history_tokens >= 90000 else self.history

Put together an architecture diagram of the whole system

Consider new data structure for storing messages

1. Dictionary with key=date and value=list({message}, {message}, {message} ...)
2. New .jsonl file for each day (i.e. 2023-08-23.jsonl)

• Integrate the bot to nostr relays
• Pull & store notes from relays on loop
• Define a list of relevant keywords
  (e.g. atlanta, bitcoin, lightning, and any permutation of these words together: Atlanta bitcoin, bitcoin Atlanta)
• Run parallel bg process to do following;
  • Clean up notes
  • Look for keywords
  • Look for questions / question marks
  • Once found, feed to GPT to get answer
  • Reply to note with answer
  • If user replies or DMs bot a follow up question, reply with an invoice (?)
  • If reply is not question, feed to GPT to reply

Related to #9.

Need to rotate debug logs and figure out a strategy for pushing / pulling logs from remote an local. Should probably gitignore all except debug.log

Rethink and redesign the commands and their arguments.

• Code should be written in src/lib/db/mongo.py
• Need 3 new classes: TelegramMessage, MongoTelegramMessage and MongoTelegram
• MongoTelegram
  • wrapper class for interacting with the telegram DB in mongo
  • should be similar or identical to MongoNostr
  • consider merging MongoNostr and MongoTelegram toggling db, collections, datatypes by using the __init__ function
  • something like below
  • doing this would allow us to write 1 class for interacting with 2 dbs

@to_dict
class MongoAbbot:
    def __init__(self, db_name):
        self.db_name = db_name
        if db_name == "telegram":
            self.channels = telegram_channels
            self.dms = telegram_dms
        elif db_name == "nostr":
            self.channels = nostr_channels
            self.dms = nostr_dms

• see below for updates to DB Schema defined initially in ATLBitLab/abbot-backup#12

MongoDB

• Databases: nostr, telegram

Nostr

• Name: nostr
• Collections: channels, dms

channel

{
  "id": "06cff0a67cc40475de5b9b16e39d70f1f242a9cb4772fd84ba5ed1851420e4ab",
  "pubkey": "c41652c2f9c58c3ffe77285cf2c77f87bd45b4c7798a5d81e5d06df52ca32c1c",
  "created_at": 1697820145,
  "kind": 40,
  "tags": [],
  "content": "{\"name\":\"ATL BitLab\",\"about\":\"Atlanta's #bitcoin hackerspace. Est. block 738919. Participant in Bitcoin Hackerspace Network.\",\"picture\":\"https://pbs.twimg.com/profile_images/1640759486305431552/cavNb8x1_400x400.jpg\"}",
  "sig": "0747b51a039df61274d9e276e77c87afedaf894f0daaddd910d90fbc5f9d29a2a7a99a46a94e7812d65c75684bb0db927514bcf28d8ece888cc571ed20932a50",
  "messages": [
    {
      "id": "634b3f1a6e2e064dca17bc34f5c2a4777fe266e36a556c11c5622b61a1545c0c",
      "pubkey": "9ddf6fe3a194d330a6c6e278a432ae1309e52cc08587254b337d0f491f7ff642",
      "created_at": 1697839314,
      "kind": 42,
      "tags": [
        [
          "e",
          "06cff0a67cc40475de5b9b16e39d70f1f242a9cb4772fd84ba5ed1851420e4ab",
          "wss://nos.lol/",
          "root"
        ]
      ],
      "content": "Wubba lubba dub dubbbbbbbb!",
      "sig": "b2dbbefe56e5cddf1a0ed8df11ac188b5c30ad24d82feb4804c72b756a51d1df600b96f216ef85b024b20028279e74a63148d25c2103821b29b419086966ddc3"
    },
    {
      "id": "12c48607d3c6b177e02456e973efdd064bdfcdc374c3f240d358a21d111e4066",
      "pubkey": "ea0110bcc29b5fecf70b9898aff07e9b74ae764f673a38a8f95c78fb0a41188c",
      "created_at": 1697915513,
      "kind": 42,
      "tags": [
        [
          "e",
          "06cff0a67cc40475de5b9b16e39d70f1f242a9cb4772fd84ba5ed1851420e4ab",
          "wss://nos.lol/",
          "root"
        ]
      ],
      "content": "Abbot in the house!! \ud83d\ude4c  ",
      "sig": "e7e7370587b31caf61549419af6f14b50a26f204629379a88b64122e008d611b7a5f79deb45fd8c2f5148a176d9d9fccb61d1f66e75795df0dddb2631e85835a"
    }
  ],
  "history": [
    {
      "role": "user",
      "content": "Wubba lubba dub dubbbbbbbb!"
    },
    {
      "role": "assistant",
      "content": "Abbot in the house!! \ud83d\ude4c  "
    }
  ]
}

dm

{
  "id": "6039795807c6f86628ccb8195214e6971929ab50a861dd7af85e40ce69fa3d0a",
  "pubkey": "9ddf6fe3a194d330a6c6e278a432ae1309e52cc08587254b337d0f491f7ff642",
  "created_at": 1697395393,
  "receiver": "ea0110bcc29b5fecf70b9898aff07e9b74ae764f673a38a8f95c78fb0a41188c",
  "messages": [
    {
      "id": "6039795807c6f86628ccb8195214e6971929ab50a861dd7af85e40ce69fa3d0a",
      "pubkey": "9ddf6fe3a194d330a6c6e278a432ae1309e52cc08587254b337d0f491f7ff642",
      "created_at": 1697395393,
      "kind": 4,
      "tags": [
        [
          "p",
          "ea0110bcc29b5fecf70b9898aff07e9b74ae764f673a38a8f95c78fb0a41188c"
        ]
      ],
      "content": "WuNx/mLdkjhas7A3JI59Eg==?iv=LfVClj/9AS8hLQ8+PY0MgA==",
      "sig": "e72c4df42aefd93c81a1632697be5d7a9b8d46f84bf1ce855018b04c2c74fe11b08d0e02d24abbb9d6d6df91edab86e54bac88769f7035e4c5418b28d7434ffd"
    },
    {
      "id": "4d4a390d49b44e02afb0b5241c22fec85ad4fd15aa56c161b0a7fc64c5585a1a",
      "pubkey": "ea0110bcc29b5fecf70b9898aff07e9b74ae764f673a38a8f95c78fb0a41188c",
      "created_at": 1697396103,
      "kind": 4,
      "tags": [
        [
          "p",
          "9ddf6fe3a194d330a6c6e278a432ae1309e52cc08587254b337d0f491f7ff642"
        ]
      ],
      "content": "8qyjvmgIdfuUo5hxPjk+aQ==?iv=HecDrNIJXVs4Cr2Ibp3BtA==",
      "sig": "95936c8e8ae21cf88dd32af1b07dc1a663e5ffe9c8eb3db45c71f917cd94b520c7d1ebf7157a78921aec9417a74048b861d45d825e780536189591e8863aee3a"
    }
  ],
  "history": [
    {
      "role": "user",
      "content": "Hey buddy"
    },
    {
      "role": "assistant",
      "content": "Hey!"
    }
  ]
}

Comments:

• id: channel id
• "messages": nostr events kind 42
• "history": message history for GPT alternating the "role" between "user" (nostr users) and "assistant" (Abbot) as users interact with abbot; this will be fed to OpenAI API chatCompletion

Telegram

• Name: telegram
• Collections: channel, dm

channel

{
  "id": -1001204119993,
  "title": "ATL BitLab",
  "created_at": 1697821151,
  "type": "supergroup",
  "admins": [{ "username": "nonni_io", "user_id": 1711738045 }, {}],
  "balance": 50000,
  "config": { "started": false, "introduced": false, "unleashed": false, "count": null },
  "messages": [
    {
      "message": {
        "date": 1697821151,
        "...": "..."
      },
      "user": {
        "...": "..."
      },
      "chat": {
        "...": "..."
      }
    },
    {
      "message": {
        "date": 1697821151,
        "...": "..."
      },
      "user": {
        "...": "..."
      },
      "chat": {
        "...": "..."
      }
    }
  ],
  "history": [
    {
      "role": "user",
      "content": "Hey everyone!"
    },
    {
      "role": "assistant",
      "content": "Hi there! How can I help?"
    }
  ]
}

dm

{
  "id": 1711738045,
  "username": "nonni_io",
  "created_at": 1697821151,
  "messages": [
    {
      "message": {
        "date": 1697821151,
        "...": "..."
      },
      "user": {
        "...": "..."
      },
      "chat": {
        "...": "..."
      }
    },
    {
      "message": {
        "date": 1697821167,
        "...": "..."
      },
      "user": {
        "...": "..."
      },
      "chat": {
        "...": "..."
      }
    }
  ],
  "history": [
    {
      "role": "user",
      "content": "Hey Abbot!"
    },
    {
      "role": "assistant",
      "content": "Hey! How can I help?"
    }
  ]
}

Comments:

• id: chat id
• "channel_messages": chat messages
• "history": chat history alternating the "role" between "user" (telegram users) and "assistant" (Abbot) as users interact with abbot; this will be fed to OpenAI API chatCompletion

Generally speaking, this restriction seems like a good idea, but specifically, I’d like to prevent the bot from replying if it restarts but no one runs the /start command. Need some brainstorming on ways to prevent the bot from replying every time it reboots and hasn’t been started.

Something to this effect ...

import asyncio

class AppBuilder:
    def __init__(self):
        self.host = '127.0.0.1'
        self.port = 8888
        self.group1_handlers = []
        self.group2_handlers = []
        self.group3_handlers = []

    def add_group1_handler(self, handler):
        self.group1_handlers.append(handler)
        return self

    def add_group2_handler(self, handler):
        self.group2_handlers.append(handler)
        return self

    def add_group3_handler(self, handler):
        self.group3_handlers.append(handler)
        return self

    async def handle_client(self, reader, writer):
        while True:
            data = await reader.read(100)
            if not data:
                break

            # Replace the following line with your actual message parsing logic
            parsed_message = ... # Your message parsing logic here
            
            kind = parsed_message.get('kind', None)

            if kind == 4:
                for handler in self.group1_handlers:
                    await handler(data, reader, writer)

            elif kind in [40, 41, 42, 43, 44]:
                for handler in self.group2_handlers:
                    await handler(data, reader, writer)

            elif kind == 21021:
                for handler in self.group3_handlers:
                    await handler(data, reader, writer)

    async def run(self):
        server = await asyncio.start_server(
            self.handle_client, self.host, self.port
        )
        addr = server.sockets[0].getsockname()
        print(f'Serving on {addr}')

        async with server:
            await server.serve_forever()

# Example handlers
async def group1_handler(data, reader, writer):
    print(f"Group 1 received: {data}")

async def group2_handler(data, reader, writer):
    print(f"Group 2 received: {data}")

async def group3_handler(data, reader, writer):
    print(f"Group 3 received: {data}")

# Example usage
app = (AppBuilder()
       .add_group1_handler(group1_handler)
       .add_group2_handler(group2_handler)
       .add_group3_handler(group3_handler))
asyncio.run(app.run())

Related to #3.

Build out an API interface over top of the bot once migrated to server.

• Define routes
• Move current logic into routes

Is your feature request related to a problem? Please describe.

• Telegram identifies group chats with IDs and usernames
• Could be a chat link or chat name or chat id, etc.

Describe the solution you'd like

• We need to figure out the best way for a participant in a Telegram channel to identify their chat via our website and request an invoice

Alternative solutions

• We could assign a globally unique user name to new Abbot users
• We store this in the database object and show it to them when they signup