Implement an Asynchronous Task Management system using OOP principles about legacypygamecommunitybot HOT 12 CLOSED

Implementation

In order to implement this module, we will need at least two things:

1. A submodule that implements the backbone of this task system using base classes (e.g. ClientEventJob and IntervalJob), from which a set of utility job subclasses can be made. This also includes making a job manager class for managing those.
2. A small module that implements wrapper classes for Gateway events received by a bot client.

from legacypygamecommunitybot.

1. Job Submodule

This module defines the basis for all jobs. The pattern of inheritance for jobs is as follows:

BotJob    // Base class of all jobs that defines essential job attributes and methods
┣ IntervalJob    // A job that runs at a certain interval
┃ ┗ OneTimeJob    // A job that only runs once
┃   ┗ DelayJob    // A job that schedules the jobs given to it after a certain period of time
┃
┗ ClientEventJob  // A job that runs in reaction to an event on Discord received by the bot's client

BotJob simply defines everything needed for all jobs to work.

IntervalJob is a base class for interval based jobs, by having class attributes that subclasses can override to control at which interval they run, how often they run and more. These attributes get passed to the discord.ext.tasks.Loop() instance given to every job object to handle their code.

ClientEventJob is a base class for jobs which run in reaction to an event on Discord received by the bot's client (Gateway Events). A subclass can override the EVENT_TYPES tuple class variable to only contain the ClientEvent classes whose instances they want to receive.

Various methods inherited from BotJob can be overloaded to achieve extra functionality, like job initialisation, error handling, cleanup, etc. The .DATA attribute of a job object is a namespace which they can use to store arbitrary data and control their state while running.

As of now, every BotJob can use its .manager attribute, an instance of a BotJobManagerProxy, to get access to methods which can be used to listen for events from the gateway, manipulate other jobs using methods for instantiation, initialization, stopping, killing and restarting other jobs. This allows for interception of job to job manipulations for logging purposes.

BotJobManager is a class that manages all job objects at runtime. When a job object is added to it, that job receives the manager in its .manager attribute and can then use it to register other jobs, or to listen for a specific ClientEvent type. Access to the BotJobManager instance used at runtime is needed for cases where jobs want to register or schedule other jobs at runtime, such as the DelayedRegisterJob job instances.

Job objects are unable to run properly if they are not added to a BotJobManager instance, and will be removed from it when killed.

This is the core of how the job system is structured.

from legacypygamecommunitybot.

2. ClientEvent classes: Gateway Event wrappers

ClientEvent is a base class for all classes whose instances act as wrappers for Gateway Events passed to the bot client by the Discord API. Events like e.g. on_message are wrapped in a OnMessage subclass instance which are then dispatched to ClientEventJob objects that define them in their EVENT_TYPES tuple class variable, or to jobs which request them from the BotJobManager directly.

Some ClientEvent subclasses have -Base at the end of their name, which indicates that they are the superclass of a selection of ClientEvent types. This is useful for ClientEventJob subclasses, which can use them to request several ClientEvent types at once. For example, a ClientEventJob that requests for OnMessageBase would get all ClientEvent types relating to manipulating messages in a test channel on Discord dispatched to them. In this case, those would be OnMessage, OnMessageEdit and OnMessageDelete.
This means that EVENT_TYPES = (OnMessageBase, )
would be the same as
EVENT_TYPES = ( OnMessage, OnMessageEdit, OnMessageDelete )
when defined as a class variable in a ClientEventJob subclass. Other useful superclasses are OnReactionBase, OnGuildBase, OnGuildChannelBase, OnMemberBase and OnGuildRoleBase.

from legacypygamecommunitybot.

Examples

class GreetingTest(core.ClientEventJob):
    """
   A job that waits for a user to type a message starting with 'hi', before responding with 'Hi, what's your name?'.
   This job will then wait until it receives another `OnMessage` event, before saying 'Hi, {event_content}'
   """
    EVENT_TYPES = (events.OnMessage,)

    def __init__(self, target_channel: discord.TextChannel):
        super().__init__()
        self.DATA.target_channel = target_channel

    async def on_job_init(self):
        if "target_channel" not in self.DATA:
            self.DATA.target_channel = common.guild.get_channel(822650791303053342)

    def check_event(self, event: events.ClientEvent):
        return event.message.channel.id == self.DATA.target_channel.id

    async def on_job_run(self, event: events.OnMessage, *args, **kwargs):
        if event.message.content.lower().startswith("hi"):
            await self.DATA.target_channel.send("Hi, what's your name?")

            author = event.message.author
            user_name = None

            check = (
                lambda x: x.message.author == author
                and x.message.channel == self.DATA.target_channel
                and x.message.content
            )

            name_event = await self.manager.wait_for_event(
                events.OnMessage, check=check
            )
            user_name = name_event.message.content

            await self.DATA.target_channel.send(f"Hi, {user_name}")
            
            
class Main(core.OneTimeJob):
    async def on_job_run(self):
        await self.manager.create_and_register_job(GreetingTest)

__all__ = [
    "Main",
]

from legacypygamecommunitybot.

All progress on this addition are in the async_task_system branch.

from legacypygamecommunitybot.

I've made some large revisions on the async_task_system branch on the local side, therefore I'll hide and rewrite some of the previous comments.

from legacypygamecommunitybot.

Implementation

In order to implement this module, we will need these things:

1. A submodule that implements the backbone of this task system using job base classes, from which a set of utility job subclasses can be made for any functionality to be implemented within the bot.

2. Another submodule that implements a class for managing instances of those job classes at runtime, involving their creation, introspection, modification and termination.

3. A small module defining base classes for event objects, which the job manager class can dispatch to all jobs upon request.

4. Another small module that implements wrapper classes for Gateway events received by the bot client using those event object base classes.

from legacypygamecommunitybot.

1. Job Submodule

This module defines the basis for all jobs. The pattern of inheritance for jobs is as follows:

jobs/base_jobs.py
┣ JobBase    // Base class of all jobs that defines essential job attributes and methods
┃ ┣ IntervalJobBase    // A job that runs at a certain interval
┃ ┗ EventJobBase    // A job that runs in reaction to event objects dispatched by the job manager
┃   ┗ ClientEventJobBase  // A job that runs in reaction to gateway events on Discord received by the bot's client by default, which are wrapped in event objects
┃
┗ JobProxy // A proxy job class that is returned to a caller when jobs are instantiated, that only exposes the required functions other jobs should get access to
...
jobs/utils/__init__.py
┣ ClientEventJobBase  // A subclass of  job that runs in reaction to gateway events on Discord received by the bot's client by default
┣ SingleRunJob    // A job that only runs once before completing
┃ ┗ RegisterDelayedJob    // A job that registers the jobs given to it to the job manager after a certain period of time
┃ 
┗ MethodCallJob // a job that can be used to schedule a method call
...

JobBase is an internal class that simply defines everything needed for all jobs to work.

IntervalJobBase is a base class for interval based jobs, and subclasses can be configured by overloading class attributes to control at which interval they run, how often they run and more. These attributes get passed to the discord.ext.tasks.Loop() instance given to every job object to handle their code. Configuration should also be possible at the instance level while being instantiated.

EventJobBase is a base class for jobs which run in reaction to event objects dispatched by the job manager. A subclass can override the EVENT_TYPES tuple class variable to only contain the event classes whose instances they want to receive.

There are multiple utility jobs based upon these base job classes to ease things like Discord messaging, scheduling a method call, and more.

Various methods inherited from JobBase can be overloaded to achieve extra functionality, like job initialisation, error handling, cleanup, etc. The .data attribute of a job object is a namespace which they can use to store arbitrary data and control their state while running.

Every job object can use its .manager attribute to get access to methods which can be used to e.g. listen for events from Discord or manipulate other jobs using methods for instantiation, initialization, stopping, killing and restarting. This allows for interception of job-to-job manipulations for logging purposes. This is achieved using a proxy object to the main job manager.

JobProxy is a proxy object that limits external access to a job for encapsulation purposes, thereby only exposing the required functions other jobs should get access to.

JobBase subclasses are meant to overload special methods like on_init(), on_start(), on_run(), on_run_error(), on_stop(), etc. They are used to run all of the code of a job. on_start(), on_run(), on_run_error() and on_stop() rely on discord.ext.tasks.Loop() for calling them from inside its task loop. When a Loop instance is created for a job object, Loop.before_loop() receives on_start, Loop.after_loop() receives on_stop, Loop(coro, ...) receives on_run(), and so on.

from legacypygamecommunitybot.

2. Job Manager Submodule

This module defines the job manager.

manager.py
┣ JobManager   // A class for managing all jobs created at runtime
┗ JobManagerProxy   // A proxy job class that is instantiated with every job object created, that only exposes the required functions jobs should get access to

JobManager is a class that manages all job objects at runtime. When a job object is added to it, that job receives a JobManagerProxy object to the manager in its .manager attribute and can then use that to access methods for registering other jobs, or to wait for a specific event type to be dispatched, or even to dispatch custom event types.

Job objects are unable to run properly if they are not added to a JobManager instance, and will be removed from it when killed, or when they have completed.

from legacypygamecommunitybot.

3 & 4. Event Classes Submodule

These modules define base classes for event objects, which are used to propagate event occurences to all jobs that are listening for them. They can also store data in attributes to pass on to listening jobs.

events/
┣ base_events.py   // Module for base classes for event objects 
┃  ┣ BaseEvent      // Base class for all event objects
┃  ┗ CustomEvent    // Base class for any custom event object that can be dispatched.
┃
┗ client_events.py   // Module for BaseEvent subclasses that are used for propagating Discord gateway events
      ┗ ClientEvent      // A subclass of BaseEvent used for propagating Discord gateway events
            ┣ OnMessageBase  // Base class for all Discord message-related gateway events
            ┃ ┣ OnMessage    // Class for propagating the event of a message being sent
            ┃ ┣ OnMessageEdit  // Class for propagating the event of a message being edited
            ┃ ...
            ┣ OnRawMessageBase // Base class for all raw Discord message-related gateway events
    ...

BaseEvent is the base class for the entire event class hierarchy.

ClientEvent is a subclass of BaseEvent used for propagating Discord gateway events.

EventJobBase subclasses can specify which events they should recieve at runtime using an overloadable EVENT_TYPES class variable, which holds a tuple containing all the class objects for the events they would like to recieve. This system makes use of the event class hierarchy, and therefore also support subclasses. By default, this tuple only contains BaseEvent in EventJobBase, meaning that any event object that gets dispatched will be registered into the event queue of the EventJobBase instance. In ClientEventJobBase, this tuple holds ClientEvent instead.

from legacypygamecommunitybot.

Examples

This sample code shows how the main file for job-class-based program is meant to be structured. Here, the Main job class is used as entry point into the code, by being imported and registered into a running job manager. Only one single job instance of the class Main should be registered at runtime.

job_main.py

class GreetingTest(core.ClientEventJobBase, permission_level=JobPermissionLevels.MEDIUM): # MEDIUM is the default
    """
   A job that waits for a user to type a message starting with 'hi', before responding with 'Hi, what's your name?'.
   This job will then wait until it receives another `OnMessage` event, before saying 'Hi, {event_content}'
   """
    EVENT_TYPES = (events.OnMessage,)

    def __init__(self, target_channel: Optional[discord.TextChannel] = None):
        super().__init__() # very important
        self.data.target_channel = target_channel

    async def on_init(self):
        if self.data.target_channel is None:
            self.data.target_channel = common.guild.get_channel(822650791303053342)

    def check_event(self, event: events.OnMessage):   # additionally validate any dispatched events
        return event.message.channel.id == self.data.target_channel.id

    async def on_run(self, event: events.OnMessage):
        if event.message.content.lower().startswith("hi"):
            with self.queue_blocker():    # block the event queue of this job while talking to a user, so that other events are ignored if intended
                await self.data.target_channel.send("Hi, what's your name?")

                author = event.message.author

                check = (
                    lambda x: x.message.author == author
                    and x.message.channel == self.data.target_channel
                    and x.message.content
                )

                name_event = await self.wait_for(self.manager.wait_for_event(    # self.wait_for signals that the job is awaiting something to other jobs and the job manager
                    events.OnMessage, check=check
                ))
                user_name = name_event.message.content

                await self.data.target_channel.send(f"Hi, {user_name}")
            
            
class Main(core.SingleRunJob, permission_level=JobPermissionLevel.HIGHEST): # prevent most jobs from killing the `Main` job
    """The main job class that serves as an entry into a program that uses jobs.
    """
    async def on_run(self):
        await self.manager.create_and_register_job(GreetingTest)

__all__ = [
    "Main",
]

from legacypygamecommunitybot.

This issue will now be closed, as the code for implementing this functionality has been made bot-agnostic and is awaiting addition to the early snakecore module. A new issue has been created at pygame-community/snakecore#9.

from legacypygamecommunitybot.