versx / chuckdevicecontroller Goto Github PK

ChuckDeviceController is a .NET based frontend and backend written in C# 11.0 using ASP.NET Core, EntityFramework Core, and Dapper.NET to control real devices and parse received protobuff proto data from iOS and Android mobile devices running Pokemon Go.

Home Page: https://versx.github.io/ChuckDeviceController

License: Other

C# 42.50% HTML 6.04% CSS 9.24% JavaScript 34.90% Batchfile 0.03% SCSS 3.75% Less 3.51% Smalltalk 0.01% Shell 0.02%

android grpc ios iphone mysql pogo pokemon pokemon-go pokemongo webhooks

chuckdevicecontroller's Introduction

Welcome to ChuckDeviceController

ChuckDeviceConfigurator
Controls devices that request jobs as well as includes a dashboard interface to configure job controllers and other required entity types.

Dashboard management UI
Device controller
Plugin system

ChuckDeviceController
Parses raw proto data received and inserts/upserts data entities into a MySQL compatible database.

ChuckDeviceCommunicator
Relays new and changed data entities to outgoing webhook endpoints that are received from the ChuckDeviceController via gRPC.

ChuckDeviceProxy
Splits and proxies requests from Atlas devices to separate endpoints in order to add support for ChuckDeviceController until Atlas is updated.

Features

Plugin system
- Create new job controller instances
- Fetch database entries
- Add WebAPI routes and page views to existing dashboard
- Assign devices to job controller instances
- Create instances
- Create geofences
- Load/save configuration files
- Route generator and optimizer
- Event bus service for communication between plugins and host application
- Add custom settings to the dashboard UI
- Much more planned...
Job controller instance types
- Bootstrap
- Dynamic Routes
- Circle Plots (Pokemon and Raids)
- Leveling
- Pokemon IV
- Quests
- Smart Raid
- Spawnpoint TTH Finder
- More planned
User and access management system
- 2FA capability
- 3rd party authentication for Discord, GitHub, and Google accounts as well as local accounts
MAD proto data parsing support
Separate device controller, proto parser & data upsert service as well as a webhook relay service to load balance across multiple machines if needed or desired
Reusable Geofence and Circle point lists
Reusable IV lists for Pokemon IV job controller instances
Quality of life utilities
- Clear Quests (by instance, geofence, or all)
- Upgraded/downgraded fort converter
- Stale Pokestop clearing
- Instance reloader
- Truncate expired Pokemon and Incident (Invasions) data
- Assignments/Assignment groups re-quester
and more...

Documentation

Overview

https://cdc.rtfd.io
or
https://versx.github.io/ChuckDeviceController

Getting Started

https://cdc.rtfd.io/en/latest/getting-started

Plugins

Getting Started

Overview
Requirements
- Git
- .NET 7 SDK
- MySQL or MariaDB
  Supported Databases
  - MySQL 5.7 and MySQL 8.0
  - MariaDB 10.3-10.10
Preparation
ChuckDeviceConfigurator
ChuckDeviceController
ChuckDeviceCommunicator
- Description
- Configuration
ChuckDeviceProxy
- Description
- Configuration

Plugins Included

BitbucketAuthProviderPlugin:
Adds Bitbucket.org user authentication support
DeviceAuthPlugin:
Adds device token and IP based device authentication support
Example.DotNetCorePlugin:
Very basic 'Clock' plugin example
FindyJumpyPlugin:
Adds new Pokemon spawnpoint job controllers
GitLabAuthProviderPlugin:
Adds GitLab.com user authentication support
HealthChecksPlugin:
Adds health checks endpoint and UI
MemoryBenchmarkPlugin:
Displays basic memory usage information and chart
MicrosoftAuthProviderPlugin:
Adds Microsoft.com account authentication support
MiniProfilerPlugin:
Adds basic profiling options and data.
PogoEventsPlugin:
Provides current and upcoming Pokemon Go events.
RazorTestPlugin:
Very basic Razor Mvc pages plugin example
RedditAuthProviderPlugin:
Adds Reddit.com user authentication support
RequestBenchmarkPlugin:
Displays web request benchmark times for routes used
RobotsPlugin:
Adds web crawler robots management based on specified UserAgent strings and routes which creates a dynamic robots.txt file
TestPlugin:
In-depth example plugin demonstrating all, if not most, possible functionality of the plugin system
TodoPlugin:
Basic TODO list plugin that adds support for keeping track of things to do
VisualStudioAuthProviderPlugin:
Adds VisualStudioOnline.com user authentication support

Screenshots

TODO

Finish account management page localization
Finish TTH finder job controller
Finish implementing permissions provided by API keys
Finish documentation
Finish plugin service event callbacks
Add more helper methods to ChuckDeviceController.Plugin.Helpers library
Move most ChuckDeviceConfigurator services to plugins

Dedication

❤️ In loving memory of Chuckleslove, rest in peace brother

chuckdevicecontroller's People

Contributors

Stargazers

Watchers

Forkers

bschultz thunder123456 sabregreen kkpan11

chuckdevicecontroller's Issues

Error getting account

The following error occurs when device tries to get account:

[Device] [xxxxxx] Received control request: get_account
fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'Chuck.Data.Contexts.DeviceControllerContext'.
      System.InvalidOperationException: A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
         at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection()
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
      System.InvalidOperationException: A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
         at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection()
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.InvalidOperationException: A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
         at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection()
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
         at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
         at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
         at Z.EntityFramework.Plus.QueryCacheExtensions.FromCache[T](IQueryable`1 query, MemoryCacheEntryOptions options, String[] tags)
         at Z.EntityFramework.Plus.QueryCacheExtensions.FromCache[T](IQueryable`1 query, String[] tags)
         at Chuck.Data.Repositories.AccountRepository.GetAllAsync(Boolean fromCache) in /Users/bschultz/development/workspaces/other/ChuckDeviceController/src/Chuck.Data/Repositories/AccountRepository.cs:line 74
         at Chuck.Data.Repositories.AccountRepository.GetNewAccountAsync(Int32 minLevel, Int32 maxLevel, List`1 inuseAccounts) in /Users/bschultz/development/workspaces/other/ChuckDeviceController/src/Chuck.Data/Repositories/AccountRepository.cs:line 82
         at ChuckDeviceController.Controllers.DeviceController.HandleGetAccount(Device device) in /Users/bschultz/development/workspaces/other/ChuckDeviceController/src/ChuckDeviceController/Controllers/DeviceController.cs:line 217
         at ChuckDeviceController.Controllers.DeviceController.HandleControllerRequest(DevicePayload payload) in /Users/bschultz/development/workspaces/other/ChuckDeviceController/src/ChuckDeviceController/Controllers/DeviceController.cs:line 111
         at ChuckDeviceController.Controllers.DeviceController.PostAsync(DevicePayload payload) in /Users/bschultz/development/workspaces/other/ChuckDeviceController/src/ChuckDeviceController/Controllers/DeviceController.cs:line 59
         at lambda_method536(Closure , Object )
         at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
         at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 121
         at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
         at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Feature Request: Multiselect for scan area in instance

Can we turn the scan area into a multiselect and assign multiple scan areas to an instance?

Catch rate is not processed

I don't know if it is a part of the re-implement from the pvp stats but the catch rate is not processing.

Use Sidebar instead of Navbar

Fix mustache on /dashboard/utilities page

Discord auth setup is confusing

Describe the bug
Build any commit of Chuck after #43 has been enabled. Discord auth is now forced and there is no easy way to allow users to submit the Discord settings.

To Reproduce

Build latest commit with a clean database
Run Chuck dotnet ChuckDeviceController.dll
Load Chuck in browser http://localhost:5000
Page will not load since the required Metadata Discord keys will not exist

Expected behavior
Chuck should create the required database keys. Additionally if Discord Auth is not setup users should be able to access the site.

INSERT INTO `metadata` (`key`, `value`) VALUES
('DISCORD_CLIENT_ID', NULL),
('DISCORD_CLIENT_SECRET', NULL),
('DISCORD_ENABLED', 'False'),
('DISCORD_REDIRECT_URI', NULL),
('DISCORD_USER_IDS', NULL);

Screenshots
n/a

Desktop (please complete the following information):

OS: all
Browser: all
Version v0.12.0 and later

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
I believe this function is where logic should be changed to allow non-authenticated requests? 1e27e61#diff-47743c7a548540a7f5ca5f042a2b2c79cb438b361a13fd06c5744a002acc6bdcR211-R217

info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://0.0.0.0:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /Users/clayton/src/ChuckDeviceController/bin
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.ArgumentNullException: Value cannot be null. (Parameter 's')
         at System.UInt64.Parse(String s)
         at ChuckDeviceController.Controllers.DiscordController.LoadSettings() in /Users/clayton/src/ChuckDeviceController/src/ChuckDeviceController/Controllers/DiscordController.cs:line 58
         at ChuckDeviceController.Controllers.DiscordController..ctor(DeviceControllerContext context, ILogger`1 logger) in /Users/clayton/src/ChuckDeviceController/src/ChuckDeviceController/Controllers/DiscordController.cs:line 47
         at lambda_method70(Closure , IServiceProvider , Object[] )
         at Microsoft.AspNetCore.Mvc.Controllers.ControllerActivatorProvider.<>c__DisplayClass4_0.<CreateActivator>b__0(ControllerContext controllerContext)
         at Microsoft.AspNetCore.Mvc.Controllers.ControllerFactoryProvider.<>c__DisplayClass5_0.<CreateControllerFactory>g__CreateController|0(ControllerContext controllerContext)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
         at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 103
         at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
         at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Add a Tile for good accounts on the Account dashboard

The Accounts Dashboard does not show any stats for "good" accounts that can be reused.

The Get stats query here gets clean accounts.

Accounts in this "SELECT * FROM account WHERE failed IS NULL AND last_encounter_lat IS NOT NULL;"
status are not currently displayed on the dashboard besides in the total accounts count. it would be nice to have a quick visual of the current count of good reusable accounts in the DB.

IV List error when adding quest instance

The following error occurs when trying to add an quest instance:

An invalid form control with name='iv_list' is not focusable.

Refactor

Queue everything

.net core 6 support to allow easy local build and test on Apple-M1 arm64

.net 5 does not support arm64 and is a pain on new apple hardware; .net 6 has full arm64 support

Change instance level defaults

Is your feature request related to a problem? Please describe.
This is a quality of life enhancement.

Describe the solution you'd like
Change the default Min level & Max level for all instance types from: 0-30 to 30-40.

Describe alternatives you've considered
We can keep the system as it is. This will require more clicking during setup but isn't the end of the world as it happens at initial setup time and not super often after the project is up.

Additional context
Most of have been mapping for a long time and have a large stock of lvl 30 accounts. Many scanning features require the higher level accounts. Realistically people should only be using accounts under level 30 for questing or leveling up purposes.

(feel free to close if not interested in the change)

Failed db migration

This only happened while initializing a scratch db using MariaDB 10.3

mariadb 10.4 and mariadb 10.5 both worked properly. I'm unable to test with mysql at this time.

Dropping 10.3 support might be the easiest route?

Chuck Device Controller v0.11.0.0
Starting...
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Current: 0, Latest: 7
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      MIGRATION IS ABOUT TO START IN 30 SECONDS, PLEASE MAKE SURE YOU HAVE A BACKUP!!!
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 1
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 2
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 3
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 4
info: [RawSql] Result -> OK
warn: ChuckDeviceController.Data.DatabaseMigrator[0]
      Failed to execute migration: ALTER TABLE `instance` CHANGE COLUMN `geofence` `geofences` longtext NOT NULL
warn: ChuckDeviceController.Data.DatabaseMigrator[0]
      Failed to execute migration: UPDATE `instance` SET geofences = CONCAT("[\"", geofences, "\"]")
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 5
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 6
info: [RawSql] Result -> OK
warn: ChuckDeviceController.Data.DatabaseMigrator[0]
      Failed to execute migration: ALTER TABLE `webhook` CHANGE COLUMN `geofence` `geofences` longtext DEFAULT NULL
warn: ChuckDeviceController.Data.DatabaseMigrator[0]
      Failed to execute migration: UPDATE `webhook` SET geofences = CONCAT("[\"", geofences, "\"]")
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migrating database to version 7
info: [RawSql] Result -> OK
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration successful
info: ChuckDeviceController.Data.DatabaseMigrator[0]
      Migration done
info: ChuckDeviceController.JobControllers.InstanceController[0]

Add accounts is not working

Accounts are not saved to database but I also didn't get any errors in log

Rewrite frontend using React

Eventually

[Feature] Instance Stats

Save stats for instances based on completion:

Circle instances: all circles visited
Quest instances: all quests found
Bootstrap instances: all bootstrap locations visited
IV instances: hourly rate
Find TTH instances: hourly rate

New tables:

instance_hourly_stats
instance_completion_stats

Rename of geofence is not working

When i try to rename an existing geofence there is an error.
Error! Geofence with name 'test2' does not exist
The same error is also when i try to rename an instance.

Speed up /dashboard/accounts query

MySQL Triggers

https://github.com/koenbeuk/EntityFrameworkCore.Triggered

https://onthedrift.com/posts/efcore-triggered-part1/

IV Priority List in Quest instance

When editing a quest instance the IV Priority List is shown in the form and saved.

It should be hidden and not saved like it is when you add a new quest instance.

Bootstrap logic currently uses a boxed version of the geofence

Describe the bug
The current logic for bootstrap uses a boxed version of the geofence, this results in areas being scanned that don't need to be if the provided geofence already excludes areas that don't have stops/gyms ie farm land or forest areas on the edges of towns.

To Reproduce
Steps to reproduce the behavior:
Create a geofence that is not a square and avoids an area that would not have stops/gyms.
Create a bootstrap instance using the geofence.
assign device to the instance
watch device location on the map or watch spawn point that are discovered, you will see the device is bootstrapping areas outside of the geofence.

Expected behavior
bootstrap areas only in the geofence.

versx / chuckdevicecontroller Goto Github PK

chuckdevicecontroller's Introduction

Welcome to ChuckDeviceController

Features

Documentation

Overview

Getting Started

Plugins

Getting Started

Screenshots

TODO

Dedication

chuckdevicecontroller's People

Contributors

Stargazers

Watchers

Forkers

chuckdevicecontroller's Issues

Recommend Projects

Recommend Topics

Recommend Org