Giter Club home page Giter Club logo

obervinov / users-package Goto Github PK

View Code? Open in Web Editor NEW
1.0 1.0 0.0 223 KB

This Python module is designed to simplify user management in Telegram bots, providing the necessary functionality for tasks such as user authentication, authorization and compliance with speed limits, ensuring effective management of user attributes and access rights.

License: MIT License

Python 98.27% HCL 1.73%
authentication python3 permissions users telegram telegram-bot authorization rate-limits roles limits module poetry poetry-python python-module

users-package's Introduction

Users Package

Release CodeQL Tests and Checks

GitHub release (latest SemVer) GitHub last commit GitHub Release Date GitHub issues GitHub repo size

About this project

Project Description

This Python module is designed to simplify user management in Telegram bots, providing the necessary functionality for tasks such as user authentication, authorization and compliance with speed limits, ensuring effective management of user attributes and access rights.

Key Features

  • This module is designed primarily for Telegram bots but can be adapted for various projects that require user management, role-based access control, and request rate limiting.

  • This module requires certain dependencies related to Vault

Description of module Constants

This module contains constant values

Constant Name Description Default Value
VAULT_CONFIG_PATH Path for configuration data in Vault. "configuration/users"
VAULT_DATA_PATH Path for historical data in Vault. "data/users"
USER_STATUS_ALLOW User access status for allowed access. "allowed"
USER_STATUS_DENY User access status for denied access. "denied"

Description of module Exceptions

Exception Describe Usage Example Tips
WrongUserConfiguration Raised when user configuration is wrong. python<br>try:<br> raise WrongUserConfiguration("Incorrect user configuration")<br>except WrongUserConfiguration as e:<br> print(e)<br> Please, see the configuration example: https://github.com/obervinov/users-package?tab=readme-ov-file#-structure-of-configuration-and-statistics-data-in-vault
VaultInstanceNotSet Raised when the Vault instance is not set. python<br>try:<br> raise VaultInstanceNotSet("Vault instance not set")<br>except VaultInstanceNotSet as e:<br> print(e)<br> Please, see documentation for the initialization of the class: https://github.com/obervinov/users-package?tab=readme-ov-file#class-initialization
FailedDeterminateRateLimit Raised when the rate limit cannot be determined. python<br>try:<br> raise FailedDeterminateRateLimit("Failed to determine the rate limit")<br>except FailedDeterminateRateLimit as e:<br> print(e)<br> Please, check misconfiguration between configuration and historical data in Vault.

Users class

Class Initialization

The Users class provides authentication, authorization and management of user attributes for Telegram bots. You can initialize it using various options

  • vault (any): Configuration for initializing the Vault client.

    • (VaultClient): an already initialized instance for interacting with the Vault API.
    • (dict): configuration for initializing a VaultClient instance in this class.
  • rate_limits (bool): Enable rate limit functionality.

  • Examples:

    • Initialize with VaultClient and without rate_limits:

      users_without_ratelimits = Users(vault=vault_client, rate_limits=False)
    • Initialize with VaultClient and with rate_limits:

      users_with_ratelimits = Users(vault=vault_client)
    • Initialize with Vault configuration dictionary:

      vault_config = {
        "name": "my_project",
        "url": "https://vault.example.com",
        "approle": {
            "id": "my_approle",
            "secret-id": "my_secret"
        }
      }
      users_with_dict_vault = Users(vault=vault_config)

For more details and examples, see the class docstring.

User Access Check

The user_access_check method is the main entry point for authentication, authorization, and request rate limit verification. It is used to control the request rate (limits) for a specific user.

  • Arguments:

    • user_id (str): Required user ID.
    • role_id (str): Required role ID for the specified user ID.
  • Examples:

    user_access_check(user_id='user1', role_id='admin_role')
  • Returns:

    • A dictionary with access status, permissions, and rate limit information.
      {
        'access': self.user_status_allow / self.user_status_deny,
        'permissions': self.user_status_allow / self.user_status_deny,
        'rate_limits': {
            'end_time': '2023-08-06 11:47:09.440933'
        }
      }

For more details, see the method docstring.

Authentication

The authentication method checks if the specified user ID has access to the bot.

  • Arguments:

    • user_id (str): Required user ID.
  • Example:

    authentication(user_id='user1')
  • Returns:

    • self.user_status_allow if access is granted.
    • self.user_status_deny if access is denied.

Authorization

The authorization method checks whether the user has the specified role.

  • Arguments:

    • user_id (str): Required user ID.
    • role_id (str): Required role ID for the specified user ID.
  • Example:

    authorization(user_id='user1', role_id='admin_role')
  • Returns:

    • self.user_status_allow if the user has the role.
    • self.user_status_deny if the user does not have the role.

Description of class attributes

Data Type Attribute Purpose Default Value
object vault Vault instance for interacting with the Vault API. None
bool rate_limits Enable request rate limit functionality. True
str user_status_allow User access status: allowed. "allowed"
str user_status_deny User access status: denied. "denied"
str vault_config_path The prefix of the configuration path in the Vault. "configuration/users"
str vault_data_path The prefix of the path of historical data in the Vault. "data/users"

Description of class methods

Method Name Description Arguments Usage Examples Returns Examples Configuration Path History Path
__init__ Creates a new Users instance. vault (any): Configuration for initializing the Vault client. rate_limits (bool): Enable rate limit functionality. Users(vault=vault_client) N/A N/A N/A
user_access_check Main entry point for authentication, authorization, and rate limit verification. user_id (str): Required user ID. role_id (str): Required role ID for the specified user ID. user_access_check(user_id='user1', role_id='admin_role') {'access': self.user_status_allow, 'permissions': self.user_status_allow, 'rate_limits': {'end_time': '2023-08-06 11:47:09.440933'}} N/A N/A
authentication Checks if the specified user ID has access to the bot. user_id (str): Required user ID. authentication(user_id='user1') self.user_status_allow or self.user_status_deny {self.vault_config_path}/{user_id}:status reads configuration in Vault to determine access status. {self.vault_data_path}/{user_id}:authentication writes authentication data to Vault.
authorization Checks whether the user ID has the specified role ID. user_id (str): Required user ID. role_id (str): Required role ID for the specified user ID. authorization(user_id='user1', role_id='admin_role') self.user_status_allow or self.user_status_deny {self.vault_config_path}/{user_id}:roles reads configuration in Vault to determine role ID status. {self.vault_data_path/{user_id}:authorization writes authorization data to Vault.
vault Getter for the 'vault' attribute. N/A vault = users_instance.vault VaultClient instance or None N/A N/A
vault Setter for the 'vault' attribute. vault (any): Configuration for initializing the Vault client. users_instance.vault = vault_client N/A N/A N/A
user_status_allow Getter for the 'user_status_allow' attribute. N/A status = users_instance.user_status_allow str (User status for allowed access) N/A N/A
user_status_allow Setter for the 'user_status_allow' attribute. user_status_allow (str): User status for allowed access. users_instance.user_status_allow = 'custom_status' N/A N/A N/A
user_status_deny Getter for the 'user_status_deny' attribute. N/A status = users_instance.user_status_deny str (User status for denied access) N/A N/A
user_status_deny Setter for the 'user_status_deny' attribute. user_status_deny (str): User status for denied access. users_instance.user_status_deny = 'custom_status' N/A N/A N/A
vault_config_path Getter for the 'vault_config_path' attribute. N/A path = users_instance.vault_config_path str (Path to the configuration data in Vault) N/A N/A
vault_config_path Setter for the 'vault_config_path' attribute. vault_config_path (str): Path to the configuration data in Vault. users_instance.vault_config_path = 'custom_path' N/A N/A N/A
vault_data_path Getter for the 'vault_data_path' attribute. N/A path = users_instance.vault_data_path str (Path to the data in Vault) N/A N/A
vault_data_path Setter for the 'vault_data_path' attribute. vault_data_path (str): Path to the data in Vault. users_instance.vault_data_path = 'custom_path' N/A N/A N/A

RateLimiter class

Class Initialization

The RateLimiter class provides the speed limit functionality for requests to the Telegram bot in the context of a specific user.

  • vault (any): Configuration for initializing the Vault client.

    • (VaultClient): an already initialized instance for interacting with the Vault API.
    • (dict): configuration for initializing a VaultClient instance in this class.
  • user_id (str): User ID for checking speed limits.

  • Examples:

    • Initialize with a VaultClient instance:

      limiter = RateLimiter(vault=vault_client, user_id='User1')
    • Initialize with a Vault configuration dictionary:

      vault_config = {
        "name": "my_project",
        "url": "https://vault.example.com",
        "approle": {
            "id": "my_approle",
            "secret-id": "my_secret"
        }
      }
      limiter = RateLimiter(vault=vault_config, user_id='User1')

For more details and examples, see the class docstring.

Rate Limit Determination

The determine_rate_limit method is the main entry point for checking restrictions on requests to the bot for the specified user. It returns information about whether the request rate limits are active.

  • Arguments:

    • None
  • Examples:

    determine_rate_limit()
  • Returns:

    • Dictionary with a timestamp of the end of restrictions on requests or None if rate limit is not applied.
      (dict | None)
      {"end_time": "2023-08-07 10:39:00.000000"}

For more details, see the method docstring.

Active Rate Limit

The _active_rate_limit method is used for a situation when restrictions on requests are already applied and the user ID has a timestamp of the end of restrictions. The method checks whether it is time to reset the request rate limit.

  • Arguments:

    • None
  • Example:

    _active_rate_limit()
  • Returns:

    • A dictionary with a timestamp - if it has not expired yet and the restrictions on requests continue, or None - if the time for limiting requests has expired and you need to reset the restrictions.
        (dict | None)
        {"end_time": "2023-08-07 10:39:00.000000"}

Apply Rate Limit

The _apply_rate_limit method is used if the request limit counters is full and it is necessary to apply request limits for the specified user.

  • Arguments:

    • None
  • Example:

    _apply_rate_limit()
  • Returns:

    • Dictionary with the end time of restrictions on requests for the specified user.
        (dict | None)
        {"end_time": "2023-08-07 10:39:00.000000"}

Recalculating request counters from the requests_history

The _update_requests_counters method recalculates the request counters based on the historical user data (per_hour and per_day).

  • Arguments:

    • None
  • Example:

    _update_requests_counters()
  • Returns:

    • None

Consideration of the users requests time

The _update_requests_history method added current request timestamp to the requests history list in the Vault.

  • Arguments:

    • None
  • Example:

    _update_requests_history()
  • Returns:

    • None

Description of Class Attributes

Data Type Attribute Purpose Default Value
any _vault The initialized VaultClient instance or None if initialization failed. N/A
str user_id The user ID for which rate limits are applied. N/A
str vault_config_path Path to the configuration data in Vault. "configuration/users"
str vault_data_path Path to the data in Vault. "data/users"
dict requests_configuration Configuration for rate limits from Vault. Value from Vault Secret
dict requests_counters Counters for user's requests. Value from Vault Secret
dict requests_ratelimits Rate limit information for the user. Value from Vault Secret
list requests_history Historical data with timestamps of user requests. Value from Vault Secret

Description of Class Methods

Method Name Description Arguments Usage Examples Returns Examples Configuration Path History Path
__init__ Creates a new RateLimiter instance. vault (any): Configuration for initializing the Vault client. user_id (str): The user ID for which rate limits are applied. RateLimiter(vault=vault_client, user_id='12345') N/A {self.vault_config_path}/{user_id}:requests reads requests configuration in Vault to determine rate limit parameters. {self.vault_data_path}/{user_id}:requests_counters and {self.vault_data_path}/{user_id}:requests_ratelimits reads requests historical data in Vault to determine rate limits state.
vault Getter for the 'vault' attribute. N/A vault = limiter.vault VaultClient instance or None N/A N/A
vault Setter for the 'vault' attribute. vault (any): Configuration for initializing the Vault client. limiter.vault = vault_client N/A N/A N/A
vault_config_path Getter for the 'vault_config_path' attribute. N/A path = limiter.vault_config_path str (Path to the configuration data in Vault) N/A N/A
vault_config_path Setter for the 'vault_config_path' attribute. vault_config_path (str): Path to the configuration data in Vault. limiter.vault_config_path = 'custom_path' N/A N/A N/A
vault_data_path Getter for the 'vault_data_path' attribute. N/A path = limiter.vault_data_path str (Path to the data in Vault) N/A N/A
vault_data_path Setter for the 'vault_data_path' attribute. vault_data_path (str): Path to the data in Vault. limiter.vault_data_path = 'custom_path' N/A N/A N/A
determine_rate_limit Determine the rate limit status for the specified user ID. N/A rate_limits = limiter.determine_rate_limit() dict (Rate limit timestamp for the user ID) or None N/A N/A
_active_rate_limit Check and handle active rate limits for the user. N/A rate_limits = limiter._active_rate_limit() dict (Rate limit timestamp for the user ID) or None (if rate limit has been reset) N/A {self.vault_data_path}/{user_id}:requests_ratelimits writes or delete rate limit timestamp in Vault.
_apply_rate_limit Apply rate limits to the user ID and reset request counters. N/A rate_limits = limiter._apply_rate_limit() dict (Rate limit timestamp for the user ID) or None N/A {self.vault_data_path}/{user_id}:requests_ratelimits writes rate limit timestamp in Vault.
_update_requests_counters Update the request counters based on the historical user data. N/A limiter._update_requests_counters() None N/A Read {self.vault_data_path}/{user_id}:requests_history and updates {self.vault_data_path}/{user_id}:requests_counters in the Vault.
_update_requests_history Update the request history based on the user's requests in the Vault. N/A limiter._update_requests_history() None N/A {self.vault_data_path}/{user_id}:requests_history writes current request timestamp to requests list in the Vault.

Structure of configuration and statistics data in vault

This project uses a Vault server with the KV2 engine to store and retrieve configuration data. It supports user configurations to define system access rights, roles, and request restrictions.

Users Configuration

  • path to the secret: configuration/users/{user_id}

  • keys and Values:

    • status: The status of user access, which can be either

      • self.user_status_allow
      • self.user_status_deny
    • roles: A list of roles associated with the user ID, e.g., ["role1", "role2"].

    • requests: Limits on the number of requests

      • requests_per_day
      • requests_per_hour
      • random_shift_time (additional, random shift in minutes from 0 to the specified number) in minutes
      {
        "requests_per_day": 10,
        "requests_per_hour": 1,
        "random_shift_minutes": 15
      }
  • example of a secret with configuration:

{
  "status": "allowed",
  "roles": ["admin_role", "additional_role"],
  "requests": {
    "requests_per_day": 10,
    "requests_per_hour": 1,
    "random_shift_minutes": 15
  }
}

Users Data and Historical Records

  • path to the secret: data/users/{user_id}
  • keys and values:
    • requests_counters: Historical data with statistics on user requests. It includes counters for the number of requests

      • requests_per_day
      • requests_per_hour
      {
        "requests_per_day": 9,
        "requests_per_hour": 1
      }
    • requests_ratelimits: Information about rate limits, including the

      • end_time of the rate limit
      {
        "end_time": "2023-08-07 10:39:00.000000"
      }

      or

      {
        "end_time": None
      }
    • requests_history: Historical data with timestamps of user requests

      • list of timestamps of user requests
      [
        "2023-08-07 10:39:00.000000",
        "2023-08-07 10:40:00.000000",
        "2023-08-06 10:00:00.000000"
      ]
    • authorization: Details about the authorization process, including the time, status

      • timestamp
      • self.user_status_allow or self.user_status_deny
      • role ID
      {
        "time": "2023-08-07 10:39:00.000000",
        "status": "allowed",
        "role": "role1"
      }
    • authentication: Records of the authentication process, indicating the time and status

      • timestamp
      • self.user_status_allow or self.user_status_deny
      {
        "time": "2023-08-07 10:39:00.000000",
        "status": "allowed"
      }
  • example of a secret with historical data:
    "requests_counters": {
      "requests_per_day": 9,
      "requests_per_hour": 1
    }
    "requests_ratelimits": {"end_time": "2023-08-07 10:39:00.000000"}
    "requests_history": [
      "2023-08-07 10:39:00.000000",
      "2023-08-07 10:40:00.000000",
      "2023-08-06 10:00:00.000000"
    ]
    "authorization": {
      "time": "2023-08-07 10:39:00.000000",
      "status": "allowed",
      "role": "role1"
    }
    "authentication": {
      "time": "2023-08-07 10:39:00.000000",
      "status": "allowed"
    }

Additional usage example

Interaction Model 1: Using a Unified Entrypoint (Method: user_access_check())

sequenceDiagram
    participant User
    participant Users
    User->>Users: Create Users instance with rate limits
    User->>Users: Call user_access_check(user_id, role_id)
    Users-->>User: Return access, permissions, and rate limits

Interaction Model 2: Using Separate Methods for Authentication, Authorization, and Rate Limits

sequenceDiagram
    participant User
    participant Users
    User->>Users: Create Users instance with rate limits
    User->>Users: Call authentication(user_id)
    Users-->>User: Return access status
    User->>Users: Call authorization(user_id, role_id)
    Users-->>User: Return permissions status
sequenceDiagram
    participant User
    participant RateLimiter
    User->>RateLimiter: Create RateLimiter instance with user_id
    User->>RateLimiter: Call determine_rate_limit()
    RateLimiter-->>User: Return rate limit timestamp for user_id (or None)

Example 1 - With Entrypoint and Rate Limits:

# import modules
from vault import VaultClient
from users import Users

# create the vault client
vault_client = VaultClient(
  url='http://0.0.0.0:8200',
  name='mybot1',
  approle={
      'id': id,
      'secret-id': secret-id
  }
)

# create the Users instance of the class with rate limits
users = Users(vault=vault_client)

# use the main entrypoint
user_info = users.user_access_check(
  user_id=message.chat.id,
  role_id="admin_role"
)
# check permissions, roles, and rate limits
if user_info["access"] == users.user_status_allow:
    print("Hi, you can use the bot!")
    if user_info["permissions"] == users.user_status_allow:
        if user_info["rate_limits"]["end_time"]:
            print(f"You have sent too many requests, the limit is applied until {user_info['rate_limits']['end_time']}")
        else:
            print("You have admin's rights")
    else:
        print("You do not have access rights to this function")
else:
    print("Access denied, goodbye!")

Example 2 - With Entrypoint and Without Rate Limits:

# import modules
from vault import VaultClient
from users import Users

# create the vault client
vault_client = VaultClient(
  url='http://0.0.0.0:8200',
  name='mybot1',
  approle={
      'id': id,
      'secret-id': secret-id
  }
)

# create the Users instance of the class without rate limits
users = Users(
  vault=vault_client,
  rate_limits=False
)

# use the main entrypoint
user_info = users.user_access_check(
  user_id=message.chat.id,
  role_id="admin_role"
)
# check permissions, roles, and rate limits
if user_info["access"] == users.user_status_allow:
    print("Hi, you can use the bot!")

    if user_info["permissions"] == users.user_status_allow:
        print("You have admin's rights")
    else:
        print("You do not have access rights to this function")

else:
    print("Access denied, goodbye!")

Example 3 - Without Entrypoint:

# import modules
from vault import VaultClient
from users import Users

# create the vault client
vault_client = VaultClient(
  url='http://0.0.0.0:8200',
  name='mybot1',
  approle={
      'id': id,
      'secret-id': secret-id
  }
)

# create the Users instance of the class
users = Users(vault=vault_client)

# check access to the bot
if users.authentication(user_id='user1') == users.user_status_allow:
    print("You can use this bot")

# check user role
if users.authorization(
  user_id='user1',
  role_id='admin_role'
) == users.user_status_allow:
    print("You have admin's permissions")

# check rate limit for user_id
limiter = RateLimiter(
  vault=vault_client,
  user_id='user1'
)
if limiter.determine_rate_limit()['end_time']:
    print(f"You have sent too many requests, the limit is applied until {user_info['rate_limits']['end_time']}")

Installing

tee -a pyproject.toml <<EOF
[tool.poetry]
name = myproject"
version = "1.0.0"
description = ""

[tool.poetry.dependencies]
python = "^3.10"
users = { git = "https://github.com/obervinov/users-package.git", tag = "v2.0.4" }

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
EOF

poetry install

GitHub Actions

Name Version
GitHub Actions Templates v1.2.2

users-package's People

Contributors

github-actions[bot] avatar obervinov avatar

Stargazers

 avatar

Watchers

 avatar

users-package's Issues

Users:v2 expand access rights and user attributes

Context
For more flexible management of telegram bot users and their rights to the bot, it is necessary to expand user attributes, add additional methods for working with these attributes and rewrite existing class methods for a new model.

Data structure requirements

  • role attributes (configuration)
{'roles': ['role_admin', 'role_guest'']}
  • rate limit attributes (configuration)
{'requests': {'requests_per_day': 10, 'requests_per_hour': 1, 'random_shift_minutes': 15}}
  • rate limit history (data)
{'requests_ counters': {'requests_per_day': 9, 'requests_per_hour': 1}}
  • rate limit applied (data)
{'rate_limits': {'end_time': '2023-08-07 10:39:00.000000' }}

Requirements

  • add a rate limit check method for an account: if rl is applied, return the end time, if not applied, none
  • the rights check method should accept user_id and role_name as input: return allowed if user_id contains role_name and denied if not
  • for the convenience of using the module by bots, select an entry point in a separate method that will immediately call all three methods: authentication(), authorization(), rl_controller() - aggregate the responses of these methods into a single dictionary and give in the response
  • if the authorization of the role and user was successful, write +1 request to the request counter
  • if the incoming role is startup, the rate limit counter should not be taken into account
  • if the hourly rate_limit + random_shift_minutes has expired, reset the hourly rate_limit counter

Data structure in vault
For configuration:

  • configuration/users/{user_id}:status -> authentication()
  • configuration/users/{user_id}:roles -> authorization()
  • configuration/users/{user_id}:requests -> rl_controller()

For dynamic data:

  • data/users/{user_id}:requests_counters -> rl_controller()
  • data/users/{user_id}:rate_limits -> rl_controller()
  • data/users/{user_id}:authorization -> authorization()
  • data/users/{user_id}:authentication -> authentication()

New methods:

  • user_access_check() - an entry point that aggregates function calls and responses for unification and ease of use in bots.
  • authentication() - method for checking user access rights to the bot
  • authorization() - method for checking access rights to a specific bot functionality (assigning roles)
  • rl_controller() - method for implementing request accounting and request rate limiting in the context of users

Documentation:

  • Assigning methods
  • General concept of the module
  • Configuration Structure
  • Persistent data structure
  • Usage examples
  • Diagram with the logic of the module
  • More information about vault client and module

Deprecate outdated class and methods users:v1

The old class and methods are deprecated and will be removed in the module version 2:

  • check_permissions()
  • record_event()

Module concept and functionality revised and rewritten in version 2, more #21

Error when call `rl_controller.determine_rate_limit()`

[07/Feb/2024 11:16:11] INFO [logger.logger.counters_watching:494] [class.RateLimiter] Counters updated for user ID 174335103: {'per_hour': 0, 'per_day': 0, 'first_request_time': '2024-02-07 11:16:11.295304'}
Traceback (most recent call last):
  File "/home/pyinstabot-downloader/app/src/bot.py", line 521, in <module>
    main()
  File "/home/pyinstabot-downloader/app/src/bot.py", line 517, in main
    telegram.launch_bot()
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telegram/telegram.py", line 157, in launch_bot
    self.telegram_bot.polling(
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/__init__.py", line 1104, in polling
    self.__threaded_polling(non_stop=non_stop, interval=interval, timeout=timeout, long_polling_timeout=long_polling_timeout,
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/__init__.py", line 1179, in __threaded_polling
    raise e
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/__init__.py", line 1141, in __threaded_polling
    self.worker_pool.raise_exceptions()
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/util.py", line 149, in raise_exceptions
    raise self.exception_info
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/util.py", line 92, in run
    task(*args, **kwargs)
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/telebot/__init__.py", line 7860, in _run_middlewares_and_handler
    result = handler['function'](message)
  File "/home/pyinstabot-downloader/app/src/bot.py", line 115, in bot_callback_query_handler
    button_post(
  File "/home/pyinstabot-downloader/app/src/bot.py", line 161, in button_post
    user = users_rl.user_access_check(call.message.chat.id, constants.ROLES_MAP['Post'])
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/users/users.py", line 261, in user_access_check
    user_info['rate_limits'] = rl_controller.determine_rate_limit()
  File "/home/pyinstabot-downloader/app/.venv/lib/python3.9/site-packages/users/ratelimits.py", line 217, in determine_rate_limit
    self.requests_configuration['requests_per_day'] <= self.requests_counters['requests_per_day'] or
TypeError: string indices must be integers

Configuration `per_day` in RateLimit not working as expected

Describe the bug
Configuration per_day in RateLimit not working as expected.
According to my observations, this parameter does not work as expected and when the number of requests is exceeded, the query execution time accounting breaks down.

To Reproduce
Steps to reproduce the behavior:

  1. Set configuration {"requests_per_day": 5, "requests_per_hour": 1, "random_shift_minutes": 360}
  2. Generate 11-12 queries to apply daily ratelimit
  3. The ratelimit timer will set the same processing time for all received requests for some time and will not add +1 day to it.

Expected behavior
If the daily ratelimit is exceeded, the processing time +1 day (+1 hour from the previous request) should be returned to all requests.

Screenshots
Снимок экрана 2024-05-11 в 18 21 38

Additional context
Users data in Vault

{
  "authentication": "{\"time\": \"2024-05-10 10:37:42.206036\", \"status\": \"allowed\"}",
  "authorization": "{\"time\": \"2024-05-10 10:37:54.057882\", \"status\": \"allowed\", \"role\": \"post\"}",
  "requests_counters": "{\"requests_per_hour\": 3, \"requests_per_day\": 7}",
  "requests_history": "[\"2024-04-28 18:03:46.689374\", \"2024-04-28 20:17:29.715123\", \"2024-04-29 18:31:12.010274\", \"2024-04-29 18:34:00.162449\", \"2024-04-29 18:44:46.783133\", \"2024-04-29 18:44:47.614515\", \"2024-04-30 12:56:04.160023\", \"2024-04-30 19:28:25.244414\", \"2024-04-30 20:21:53.361840\", \"2024-05-01 06:14:55.136715\", \"2024-05-01 06:14:56.162464\", \"2024-05-01 06:28:04.770559\", \"2024-05-01 06:28:05.759283\", \"2024-05-01 06:28:06.929947\", \"2024-05-01 06:28:07.893657\", \"2024-05-01 06:49:41.293213\", \"2024-05-01 06:49:42.123203\", \"2024-05-01 09:30:40.433108\", \"2024-05-01 09:30:42.427200\", \"2024-05-01 09:30:43.070135\", \"2024-05-01 09:30:43.892978\", \"2024-05-01 09:30:44.626797\", \"2024-05-01 09:30:45.444172\", \"2024-05-01 09:30:46.327353\", \"2024-05-01 09:30:47.213644\", \"2024-05-01 09:30:48.253656\", \"2024-05-01 09:30:49.121649\", \"2024-05-01 09:30:49.964373\", \"2024-05-01 09:30:50.770601\", \"2024-05-02 10:05:52.731315\", \"2024-05-02 10:05:53.510574\", \"2024-05-02 10:05:54.548760\", \"2024-05-02 10:05:55.440846\", \"2024-05-02 10:05:56.354842\", \"2024-05-02 10:05:57.198197\", \"2024-05-02 10:05:57.998544\", \"2024-05-02 10:05:58.864686\", \"2024-05-04 07:36:58.402686\", \"2024-05-04 07:36:59.409894\", \"2024-05-04 07:37:00.548504\", \"2024-05-04 07:37:01.473791\", \"2024-05-04 07:37:02.530700\", \"2024-05-09 11:49:03.184859\", \"2024-05-09 11:55:44.834973\", \"2024-05-09 11:55:45.902370\", \"2024-05-09 11:55:47.169803\", \"2024-05-10 10:35:40.252446\", \"2024-05-10 10:36:11.728285\", \"2024-05-10 10:37:12.231424\", \"2024-05-10 10:38:47.389254\"]",
  "requests_ratelimits": "{\"end_time\": \"2024-05-11 15:18:04.473465\"}"
}

Application logs

[2024-05-11 14:13:09,436] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:09,651] INFO [logger.logger:authorization:363] [class.Users] Check role `role_list` for user `123456789`: allowed
[2024-05-11 14:13:09,862] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:10,070] INFO [logger.logger:authorization:363] [class.Users] Check role `role_list` for user `123456789`: allowed
[2024-05-11 14:13:12,631] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:12,849] INFO [logger.logger:authorization:363] [class.Users] Check role `role_list` for user `123456789`: allowed
[2024-05-11 14:13:13,048] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:13,279] INFO [logger.logger:authorization:363] [class.Users] Check role `role` for user `123456789`: allowed
[2024-05-11 14:13:13,707] INFO [logger.logger:_update_requests_counters:520] [class.RateLimiter] Current request counters: {'requests_per_hour': 0, 'requests_per_day': 0}
[2024-05-11 14:13:13,708] WARNING [logger.logger:_active_rate_limit:378] [class.RateLimiter] A rate limit has been detected for user ID 123456789 that has already been applied and has not expired yet
[2024-05-11 14:13:14,162] INFO [logger.logger:update_status_message:307] [Bot]: Message with type `status_message` for user 123456789 has been updated
[2024-05-11 14:13:14,163] INFO [logger.logger:process_one_post:451] [Bot]: Message for user 123456789 added in queue
[2024-05-11 14:13:14,172] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:14,331] INFO [logger.logger:authorization:363] [class.Users] Check role `role` for user `123456789`: allowed
[2024-05-11 14:13:14,646] INFO [logger.logger:_update_requests_counters:520] [class.RateLimiter] Current request counters: {'requests_per_hour': 1, 'requests_per_day': 1}
[2024-05-11 14:13:15,095] INFO [logger.logger:process_one_post:451] [Bot]: Message for user 123456789 added in queue
[2024-05-11 14:13:15,104] INFO [logger.logger:authentication:304] [class.Users] Access from user ID 123456789: allowed
[2024-05-11 14:13:15,250] INFO [logger.logger:authorization:363] [class.Users] Check role `role` for user `123456789`: allowed
[2024-05-11 14:13:15,530] INFO [logger.logger:_update_requests_counters:520] [class.RateLimiter] Current request counters: {'requests_per_hour': 2, 'requests_per_day': 2}
[2024-05-11 14:13:15,530] WARNING [logger.logger:_active_rate_limit:378] [class.RateLimiter] A rate limit has been detected for user ID 123456789 that has already been applied and has not expired yet
[2024-05-11 14:13:15,871] INFO [logger.logger:process_one_post:451] [Bot]: Message for user 123456789 added in queue

Check list

  • Add timestamp with end_time in all messages of logger about exist the rate limit A rate limit has been detected for user ID 123456789 that has already been applied and has not expired yet

Incorrect calculation of rate_limit if it is already applied and you need to calculate the timer for additional messages

Context
In a situation when a message exceeding the quota for requests is received, rate_limit is applied with the time stamp now +1 hour'. But all subsequent messages received after the restrictions are applied receive a timestamp according to the same formula now + 1 hour'.
The result is a situation where after 1 hour 3 will be processed at once (for example) messages, not 1, as specified in the configuration.

Image

Image

requests configuration

        "requests": {
            "requests_per_day": 10,
            "requests_per_hour": 1,
            "random_shift_minutes": 15
        }

To-Do

  • with an already existing limit, depending on the per_hour setting, you need to add +1 hour to the already existing rate limit end time

  • the request counter should reset to zero only during the end of the timer rate_limits in end_time

  • when calculating the limit end timer for an incoming request (if rate limits is already applied), you need to proceed from the per_hour limit on requests if >1 request per hour is allowed

      math.ceil("requests_counters": { "requests_per_hour": 1} / "requests": {"requests_per_hour": 1})

Tests

  • add tests to check the recalculation of request counters
  • add tests to check the end_time timestamp, namely whether the time shift in end_time is calculated correctly when the request counter limit is exceeded multiple times

Change permissions attribute to list with exist roles

Context:
Now the user_access_check() and authorization() methods return the verdict allowed or denied after checking the role. This is an inconvenient situation when you need to check for example several roles.
https://github.com/obervinov/users-package/blob/v2.0.0/users/users.py#L333C9-L333C22
https://github.com/obervinov/users-package/blob/v2.0.0/users/users.py#L194

To-Do:
Instead of the verdict string, you need to return a dictionary with a list of all existing user roles
{'permissions': ['role1', 'role2']}

Fix work with transit dependencies in `setup.py`

  • fix install_requires
    install_requires=[
        'logger @ git+https://github.com/obervinov/[email protected]',
        'vault @ git+https://github.com/obervinov/[email protected]',
    ]
  • remove dependency_links

  • fix requirements.txt

logger @ git+https://github.com/obervinov/[email protected]
vault @ git+https://github.com/obervinov/[email protected]
  • update topics in classifiers
    classifiers=[
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3.10",
        "Operating System :: OS Independent",
        "Intended Audience :: Developers",
        "Topic :: Software Development"
    ]

Feature request: Add an additional backend - `Postgres` to store historical user data

Is your feature request related to a problem? Please describe.
Add an additional PostgreSQL backend to store historical user data (in addition to Vault).

Describe the solution you'd like
All user configuration should still be stored in the Vault.
Postgres will serve as a repository of historical data.
Adding the functionality of this backend will be minor and should require explicit inclusion when initializing the Users() class.
Postgresql should store:

  • user authentication and authorization records
  • user requests in the container
  • end time rate limits

Describe alternatives you've considered
None

Additional context
Adding postgresql, in the future will help to greatly expand and simplify the work.

Fix wrong links format by release/v2.0.0 in CHANGELOG.md

https://github.com/obervinov/users-package/blob/main/CHANGELOG.md?plain=1#L10-L18

## v2.0.0 - 2023-11-07
### What's Changed
**Full Changelog**: https://github.com/obervinov/users-package/compare/v1.0.5...v2.0.0 by @obervinov in https://github.com/obervinov/users-package/pull/24
#### 📚 Documentation
* [Users:v2 expand access rights and user attributes](https://github.com/obervinov/users-package/issues/21)
* [Improvements to the Vault dependency](https://github.com/obervinov/users-package/issues/27)
#### 💥 Breaking Changes
* [Users:v2 expand access rights and user attributes](https://github.com/obervinov/users-package/issues/21)
* [Deprecate outdated class and methods users:v1](https://github.com/obervinov/users-package/issues/26)
#### 🚀 Features
* [Update template workflow to v1.0.5](https://github.com/obervinov/users-package/issues/23)
* [Users:v2 expand access rights and user attributes](https://github.com/obervinov/users-package/issues/21)
* [Improvements to the Vault dependency](https://github.com/obervinov/users-package/issues/27)

Documentation updates: pr template

Add in PR template **full changelog**: https://github.com/obervinov/_templates/compare/1...2 by @ obervinov https://github.com/obervinov/_templates/pull/1

GitHub Actions workflow updates: 2023.05.22

  • Add a filter to the release creation workflow to exclude the launch if the module files have not changed
name: Create GitHub Release

on:
  push:
    branches: [main]
    paths: ['users/**']

jobs:
  create-release:
    uses: obervinov/_templates/.github/workflows/[email protected]
  • Add workflow with package installation verification (by tag, branch, default)
  verify-package:
    uses: obervinov/_templates/.github/workflows/[email protected]
  • Tests and checks workflow - run on the main branch for the show badge
name: Tests and Checks

on:
  push:
    branches:    
      - '*'
      - '*/*'
      - '**'

Improvements to the Vault dependency

  • add to the initialization of the Users class creating vault_client using vault dependencies
  • change input argument of class Users vault(object) -> vault(dict)= {'name': None, 'approle': {}, 'url': None}
  • add to README.md information about vault environment variables

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.