donbarbos / telegram-bot-template Goto Github PK

🤖 Template for telegram bot using postgres, pgbouncer, redis, docker, amplitude, prometheus, grafana, CI

License: GNU Lesser General Public License v3.0

Dockerfile 1.23% Python 67.25% CSS 0.81% JavaScript 5.55% HTML 24.53% Mako 0.63%

telegram-bot python aiogram-bot-template docker-compose postgresql aiogram docker poetry redis telegram telegram-bot-template telegrambot template telegram-bot-example database pydantic ruff sqlalchemy analytics metrics

telegram-bot-template's Introduction

Telegram bot template

Best way to create a scalable telegram bot with analytics

✨ Features

🚀 How to Use

🐳 Running in Docker (recommended method)

configure environment variables in .env file
start services
```
docker compose up -d --build
```

💻 Running on Local Machine

install dependencies using Poetry
```
poetry install
```
start the necessary services (at least the database and redis)
configure environment variables in .env file
start telegram bot
```
poetry run python -m bot
```

start admin panel

poetry run gunicorn -c admin/gunicorn_conf.py

make migrations
```
poetry run alembic upgrade head
```

🌍 Environment variables

to launch the bot you only need a token bot, database and redis settings, everything else can be left out

name	description
`BOT_TOKEN`	Telegram bot API token
`RATE_LIMIT`	Maximum number of requests allowed per minute for rate limiting
`DEBUG`	Enable or disable debugging mode (e.g., `True` or `False`)
`USE_WEBHOOK`	Flag to indicate whether the bot should use a webhook for updates (e.g., `True` or `False`)
`WEBHOOK_BASE_URL`	Base URL for the webhook
`WEBHOOK_PATH`	Path to receive updates from Telegram
`WEBHOOK_SECRET`	Secret key for securing the webhook communication
`WEBHOOK_HOST`	Hostname or IP address for the main application
`WEBHOOK_PORT`	Port number for the main application
`ADMIN_HOST`	Hostname or IP address for the admin panel
`ADMIN_PORT`	Port number for the admin panel
`DEFAULT_ADMIN_EMAIL`	Default email for the admin user
`DEFAULT_ADMIN_PASSWORD`	Default password for the admin user
`SECURITY_PASSWORD_HASH`	Hashing algorithm for user passwords (e.g., `bcrypt`)
`SECURITY_PASSWORD_SALT`	Salt value for user password hashing
`DB_HOST`	Hostname or IP address of the PostgreSQL database
`DB_PORT`	Port number for the PostgreSQL database
`DB_USER`	Username for authenticating with the PostgreSQL database
`DB_PASS`	Password for authenticating with the PostgreSQL database
`DB_NAME`	Name of the PostgreSQL database
`REDIS_HOST`	Hostname or IP address of the Redis database
`REDIS_PORT`	Port number for the Redis database
`REDIS_PASS`	Password for authenticating with the Redis database
`SENTRY_DSN`	Sentry DSN (Data Source Name) for error tracking
`AMPLITUDE_API_KEY`	API key for Amplitude analytics
`POSTHOG_API_KEY`	API key for PostHog analytics
`PROMETHEUS_PORT`	Port number for the Prometheus monitoring system
`GRAFANA_PORT`	Port number for the Grafana monitoring and visualization platform
`GRAFANA_ADMIN_USER`	Admin username for accessing Grafana
`GRAFANA_ADMIN_PASSWORD`	Admin password for accessing Grafana

📂 Project Folder Structure

.
├── admin # Source code for admin panel
│   ├── __init__.py
│   ├── app.py # Main application module for the admin panel
│   ├── config.py # Configuration module for the admin panel
│   ├── Dockerfile # Dockerfile for admin panel
│   ├── gunicorn_conf.py # Gunicorn configuration file for serving admin panel
│   ├── static # Folder for static assets
│   │   ├── css/
│   │   ├── fonts/
│   │   ├── img/
│   │   ├── js/
│   │   └── plugins/
│   ├── templates # HTML templates for the admin panel
│   │   ├── admin/
│   │   ├── index.html
│   │   ├── my_master.html
│   │   └── security/
│   └── views # Custom View modules for handling web requests
│       ├── __init__.py
│       └── users.py
│
├── bot # Source code for Telegram Bot
│   ├── __init__.py
│   ├── __main__.py # Main entry point to launch the bot
│   ├── analytics/ # Interaction with analytics services (e.g., Amplitude or Google Analytics)
│   ├── cache/ # Logic for using Redis cache
│   ├── core/ # Settings for application and other core components
│   ├── database/ # Database functions and SQLAlchemy Models
│   ├── filters/ # Filters for processing incoming messages or updates
│   ├── handlers/ # Handlers for processing user commands and interactions
│   ├── keyboards # Modules for creating custom keyboards
│   │   ├── default_commands.py # Default command keyboards
│   │   ├── __init__.py
│   │   ├── inline/ # Inline keyboards
│   │   └── reply/ # Reply keyboards
│   ├── locales/ # Localization files for supporting multiple languages
│   ├── middlewares/ # Middleware modules for processing incoming updates
│   ├── services/ # Business logic for application
│   └── utils/ # Utility functions and helper modules
│
├── migrations # Database Migrations managed by Alembic
│   ├── env.py # Environment setup for Alembic
│   ├── __init__.py
│   ├── README
│   ├── script.py.mako # Script template for generating migrations
│   └── versions/ # Folder containing individual migration scripts
│
├── configs # Config folder for Monitoring (Prometheus, Node-exporter and Grafana)
│   ├── grafana # Configuration files for Grafana
│   │   └── datasource.yml
│   └── prometheus # Configuration files for Prometheus
│       └── prometheus.yml
│
├── alembic.ini # Configuration file for migrations
├── docker-compose.yml # Docker Compose configuration file for orchestrating containers
├── Dockerfile # Dockerfile for Telegram Bot
├── LICENSE.md # License file for the project
├── poetry.lock # Lock file for Poetry dependency management
├── pyproject.toml # Configuration file for Python projects, including build tools, dependencies, and metadata
└── README.md # Documentation

🔧 Tech Stack

sqlalchemy — object-relational mapping (ORM) library that provides a set of high-level API for interacting with relational databases
asyncpg — asynchronous PostgreSQL database client library
aiogram — asynchronous framework for Telegram Bot API
flask-admin — simple and extensible administrative interface framework
loguru — third party library for logging in Python
poetry — development workflow
docker — to automate deployment
postgres — powerful, open source object-relational database system
pgbouncer — connection pooler for PostgreSQL database
redis — in-memory data structure store used as a cache and FSM
prometheus — time series database for collecting metrics from various systems
grafana — visualization and analysis from various sources, including Prometheus

👷 Contributing

First off, thanks for taking the time to contribute! Contributions are what makes the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

Fork this repository
Create a branch
Commit your changes
Push your commits to the branch
Submit a pull request

📝 License

Distributed under the LGPL-3.0 license. See LICENSE for more information.

📢 Contact

donbarbos: [email protected]

telegram-bot-template's People

Contributors

Stargazers

Watchers

telegram-bot-template's Issues

Feature: Automatic database backups

add to docker-compose.yml


  pgbackup:
    container_name: pgbackup
    image: prodrigestivill/postgres-backup-local
    restart: always
    volumes:
      - ./backup:/backups
    networks:
      - app
    depends_on:
      - postgres
    environment:
      - POSTGRES_HOST=postgres
      - POSTGRES_DB=${DB_NAME}
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASS}
      - POSTGRES_EXTRA_OPTS=-Z9 --schema=public --blobs
      - SCHEDULE=@every 0h30m00s
      - BACKUP_KEEP_DAYS=7
      - BACKUP_KEEP_WEEKS=4
      - BACKUP_KEEP_MONTHS=6
      - HEALTHCHECK_PORT=81

start

docker-compose up -d pgbackup

output

docker logs 1fc25aaab5d9
2024/08/09 07:11:35 Running version: v0.0.10
2024/08/09 07:11:35 new cron: @every 0h30m00s
2024/08/09 09:11:35 272 cmd: /backup.sh
2024/08/09 09:11:35 272: Creating dump of bot_db database from postgres...
2024/08/09 09:11:35 272: Replacing daily backup /backups/daily/bot_db-20240809.sql.gz file this last backup...
2024/08/09 09:11:35 272: '/backups/daily/bot_db-20240809.sql.gz' => '/backups/last/bot_db-20240809-091135.sql.gz'
2024/08/09 09:11:35 272: Replacing weekly backup /backups/weekly/bot_db-202432.sql.gz file this last backup...
2024/08/09 09:11:35 272: '/backups/weekly/bot_db-202432.sql.gz' => '/backups/last/bot_db-20240809-091135.sql.gz'
2024/08/09 09:11:35 272: Replacing monthly backup /backups/monthly/bot_db-202408.sql.gz file this last backup...
2024/08/09 09:11:35 272: '/backups/monthly/bot_db-202408.sql.gz' => '/backups/last/bot_db-20240809-091135.sql.gz'
2024/08/09 09:11:35 272: Point last backup file to this last backup...
2024/08/09 09:11:35 272: '/backups/last/bot_db-latest.sql.gz' -> 'bot_db-20240809-091135.sql.gz'
2024/08/09 09:11:35 272: Point latest daily backup to this last backup...
2024/08/09 09:11:35 272: '/backups/daily/bot_db-latest.sql.gz' -> 'bot_db-20240809.sql.gz'
2024/08/09 09:11:35 272: Point latest weekly backup to this last backup...
2024/08/09 09:11:35 272: '/backups/weekly/bot_db-latest.sql.gz' -> 'bot_db-202432.sql.gz'
2024/08/09 09:11:35 272: Point latest monthly backup to this last backup...
2024/08/09 09:11:35 272: '/backups/monthly/bot_db-latest.sql.gz' -> 'bot_db-202408.sql.gz'
2024/08/09 09:11:35 272: Cleaning older files for bot_db database from postgres...
2024/08/09 09:11:35 272: SQL backup created successfully

Feature: add support of Posthog

Bot restaring error

1 Field required [type=missing, input_value={'REDIS_HOST': 'redis', '... 'GRAFANA_PORT': '3000'}, input_type=dict]
2024-01-23 11:58:41 For further information visit https://errors.pydantic.dev/2.5/v/missing
2024-01-23 11:58:41 POSTHOG_API_KEY
2024-01-23 11:58:41 Field required [type=missing, input_value={'REDIS_HOST': 'redis', '... 'GRAFANA_PORT': '3000'}, input_type=dict]
2024-01-23 11:58:41 For further information visit https://errors.pydantic.dev/2.5/v/missing

repository does not exist or may require 'docker login'

docker compose up -d --build

migrator Error pull access denied for bot, repository does not exist or may require 'docker login': denied: requested access to the res...

Feature: Admin panel (improvement proposal)

Proposal for adding Admin panel, questions:

Main question: Do you need an admin panel as a feature or is it already overhead?
Which panel is better? The Django admin site, FastAPI Admin, Flask-Admin, ... or offer your options.

Update docker-compose (or something)

It would be nice to add something like this to the dockerfile (docker-compose): alembic upgrade head
Or use may be 'prestart.sh' that could be mount.

Docker Image Build Failure: Seeking Assistance to Resolve the Issue

Greetings to the development community! I hope this message finds you well. Unfortunately, I am encountering a persistent problem while attempting to build a Docker image for our project. Despite my efforts and troubleshooting attempts, the image build process fails consistently, impeding further progress.
Check: Action workflows
And how to contribute

Feature: add support of Webhooks for telegram bot

Localization error in docker (container: bot)

2024-04-06 22:01:45 bot            | Traceback (most recent call last):
2024-04-06 22:01:45 bot            |   File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
2024-04-06 22:01:45 bot            |     return _run_code(code, main_globals, None,
2024-04-06 22:01:45 bot            |   File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
2024-04-06 22:01:45 bot            |     exec(code, run_globals)
2024-04-06 22:01:45 bot            |   File "/usr/src/app/bot/__main__.py", line 10, in <module>
2024-04-06 22:01:45 bot            |     from bot.core.loader import app, bot, dp
2024-04-06 22:01:45 bot            |   File "/usr/src/app/bot/core/loader.py", line 32, in <module>
2024-04-06 22:01:45 bot            |     i18n: I18n = I18n(path=LOCALES_DIR, default_locale="en", domain=I18N_DOMAIN)
2024-04-06 22:01:45 bot            |   File "/usr/local/lib/python3.10/site-packages/aiogram/utils/i18n/core.py", line 24, in __init__
2024-04-06 22:01:45 bot            |     self.locales = self.find_locales()
2024-04-06 22:01:45 bot            |   File "/usr/local/lib/python3.10/site-packages/aiogram/utils/i18n/core.py", line 73, in find_locales
2024-04-06 22:01:45 bot            |     raise RuntimeError(f"Found locale '{name}' but this language is not compiled!")
2024-04-06 22:01:45 bot            | RuntimeError: Found locale 'en' but this language is not compiled!
bot exited with code 1

System Information:

Operating System: Windows 11 version 23H2 (OS Build 22631.3374)
Docker Desktop version: 4.28.0 (139021)
Docker version: 25.0.3, build 4debf41

Feature: add middleware for Prometheus

i need middleware to send events to the Prometheus.

you can use aiogram-prometheus

Error in creating migration

Since telegram is now growing in popularity, the Integer type used in the users table in id will no longer work.
As the record of this format: 10 digits in user id in telegram will cause an error in the database.

migrations\versions\2024-01-23_initial.py
def upgrade() -> None:\n
    # ### commands auto generated by Alembic - please adjust! ###
    op.create_table(
        "users",
        sa.Column("id", sa.BigInteger(), autoincrement=False, nullable=False),
        sa.Column("first_name", sa.String(), nullable=False),
        sa.Column("last_name", sa.String(), nullable=True),
        sa.Column("username", sa.String(), nullable=True),
        sa.Column("language_code", sa.String(), nullable=True),
        sa.Column("created_at", sa.DateTime(), server_default=sa.text("TIMEZONE('utc', now())"), nullable=False),
        sa.Column("is_admin", sa.Boolean(), nullable=False),
        sa.Column("is_suspicious", sa.Boolean(), nullable=False),
        sa.Column("is_block", sa.Boolean(), nullable=False),
        sa.Column("is_premium", sa.Boolean(), nullable=False),
        sa.PrimaryKeyConstraint("id"),
        sa.UniqueConstraint("id"),
    )

bot\database\models\user.py
# ruff: noqa: TCH001, TCH003, A003, F821
from __future__ import annotations

from sqlalchemy.orm import Mapped, mapped_column

from bot.database.models.base import Base, created_at, int_pk, big_int_pk


class UserModel(Base):
    __tablename__ = "users"

    id: Mapped[big_int_pk]
    first_name: Mapped[str]
    last_name: Mapped[str | None]
    username: Mapped[str | None]
    language_code: Mapped[str | None]
    referrer: Mapped[str | None]
    created_at: Mapped[created_at]

    is_admin: Mapped[bool] = mapped_column(default=False)
    is_suspicious: Mapped[bool] = mapped_column(default=False)
    is_block: Mapped[bool] = mapped_column(default=False)
    is_premium: Mapped[bool] = mapped_column(default=False)

bot\database\models\base.py
import datetime
from typing import Annotated

from sqlalchemy import text, BigInteger
from sqlalchemy.orm import DeclarativeBase, mapped_column

int_pk = Annotated[int, mapped_column(primary_key=True, unique=True, autoincrement=False)]
big_int_pk = Annotated[int, mapped_column(primary_key=True, unique=True, autoincrement=False, type_=BigInteger)]
created_at = Annotated[datetime.datetime, mapped_column(server_default=text("TIMEZONE('utc', now())"))]


class Base(DeclarativeBase):
    repr_cols_num: int = 3  # print first columns
    repr_cols: tuple = ()  # extra printed columns

    def __repr__(self) -> str:
        cols = [
            f"{col}={getattr(self, col)}"
            for idx, col in enumerate(self.__table__.columns.keys())
            if col in self.repr_cols or idx < self.repr_cols_num
        ]
        return f"<{self.__class__.__name__} {', '.join(cols)}>"

Feature: add migrations (alembic)

inline keyboard button problem

I have a problem with all the commands except the "/menu" and "/help" (see picture)
a full text description looks like this:
["

2024-07-03 22:23:07 2024-07-03 17:23:07.396 | INFO | bot.middlewares.logging:call:113 - received message | chat_type: private | user_id: ??? | text: /supports
2024-07-03 22:23:07 Cause exception while process update id=??? by bot id=???
2024-07-03 22:23:07 TelegramBadRequest: Telegram server says - Bad Request: can't parse inline keyboard button: Text buttons are unallowed in the inline keyboard
2024-07-03 22:23:07 File "/usr/local/lib/python3.10/site-packages/aiogram/fsm/middleware.py", line 42, in call
2024-07-03 22:23:07 return await handler(event, data)
2024-07-03 22:23:07 File "/usr/src/app/bot/middlewares/logging.py", line 165, in call
2024-07-03 22:23:07 return await handler(event, data)
2024-07-03 22:23:07 File "/usr/src/app/bot/middlewares/database.py", line 23, in call
2024-07-03 22:23:07 return await handler(event, data)
2024-07-03 22:23:07 File "/usr/local/lib/python3.10/site-packages/aiogram/dispatcher/event/telegram.py", line 121, in trigger
2024-07-03 22:23:07 return await wrapped_inner(event, kwargs)
2024-07-03 22:23:07 aiogram.exceptions.TelegramBadRequest: Telegram server says - Bad Request: can't parse inline keyboard button: Text buttons are unallowed in the inline keyboard
"]
how i make to fix it?