Giter Club home page Giter Club logo

sldb's Introduction

SLDB

SLDB (Spring Lobby DataBase) is a data warehouse and real time ranking application for all games based on SpringRTS engine. It is designed to offer all the required functionalities to manage the player base in the context of FOSS (Free Open Source Software) games, where players can create as many accounts as they want and games can be hosted by anyone. For this purpose, following functionalities have notably been implemented:

  • advanced automatic multi-accounts ("smurfs") detection
  • automatic and manual (by admins) accounts join/split
  • by-game and by-game-type TrueSkill ranking system to optimize players balance on all types of games
  • retro-active account management regarding ranking (automatic re-rating of all required matches when new accounts are joined/split)
  • protection against fake data sent by thrid parties
  • configurable privacy levels to limit account information disclosure

SLDB is also designed to allow fast access to historical data concerning all games and players using main Spring lobby since July 2012 (matches results, ranking data, statistics...).

Components

Please see the file called COMPONENTS for a more detailed description of each component.

The SLDB lobby inteface bot is based on the templates provided by following project:

Dependencies

The SLDB application is based on a partitionned database which requires MySQL 5.5 or later.

The SLDB application depends on following projects:

The SLDB backup script requires following dependencies, which are only needed if you plan to use this script for your backups:

Additionally, some SLDB components require some standard but non-core Perl modules to be available on the system (easily installable through CPAN):

  • all components require the "DBI" Perl module
  • sldbSetup requires the "Term::UI" Perl module
  • ratingEngine requires the "Inline::Python" Perl module
  • xmlRpc requires the "RPC::XML::Server" Perl module ("Net::Server::PreFork" is also highly recommended)
  • zkMonitor (deprecated) requires the "HTML::TreeBuilder" and "WWW::Mechanize" Perl modules

SLDB also depends on following project (hosted remotely) for additional functionalities:

Installation

  • Ensure MySQL 5.5 or later is installed on the system

  • Copy following dependencies into SLDB directory: SimpleConf.pm, SimpleLog.pm and SpringLobbyInterface.pm

  • Install the TrueSkill python module as "trueskill" subdirectory of SLDB (version "0.2.1" is known to be compatible with SLDB)

  • Use your favorite Perl package manager to install following standard Perl modules (available on CPAN) and their dependencies: Inline::Python, RPC::XML::Server, Net::Server::PreFork (and also HTML::TreeBuilder and WWW::Mechanize if you plan to use the deprecated zkMonitor component)

  • run the sldbSetup.pl script from SLDB directory and execute all steps, as selectionned by default by the script

  • Insert rows in the "gamesNames" table of the SLDB database to declare the list of games for which TrueSkill should be enabled (the regex fields are in standard MySQL/MariaDB regular expressions format, as described here)

  • Edit the etc/sldbLi.conf file to set following parameters:

    • lobbyPassword (password of the lobby account used by sldbLi)
    • sldb (replace <dbLogin>, <dbPwd> and <dbName> by the corresponding values for SLDB)
    • etcDir (directory containing sldbLi config files, should be the "etc" subdirectory of SLDB)
    • varDir (directory containing sldbLi dynamic data, should be the "var" subdirectory of SLDB)
    • logDir (directory containing sldbLi log files, should be the "var/log" subdirectory of SLDB)
  • Edit the etc/users.conf file and update the <> placeholders with the desired privileged lobby user names and account IDs for SLDB

  • You are now ready to launch all SLDB components, using recommended order:

      ./slMonitor.pl
      ./ratingEngine.pl
      ./sldbLi.pl
      ./xmlRpc.pl
    

Backups

SLDB includes a basic backup script: sldbBackup.pl

The "Configuration" section of this script must be edited before first use, to match host environment (in particular, a LFTP bookmark must be created first to store the remote FTP server connection information).

This script requires an account with sufficient privileges to run mysqlhotcopy. It can be launched manually once with verbose option to check that all works as expected:

    ./sldbBackup.pl --verbose

Then it is recommended to schedule this script to be executed weekly in off peak periods, using crontab for example.

Documentation

Please see the file called COMPONENTS for a description of each SLDB component.

Please see the file called TABLES for a basic description of SLDB data model.

Please see the file called XMLRPC for a description of the XML-RPC interface.

Licensing

Please see the file called LICENSE.

Author

Yann Riou [email protected]

sldb's People

Contributors

yaribz avatar badosu avatar

Stargazers

 avatar Alexander Lipatov avatar Fabrice avatar Daniel avatar

Watchers

Daniel avatar James Cloos avatar  avatar  avatar

sldb's Issues

Table has no partition for value

I'm getting the error Unable to insert rating for new user 7, period 202001 and type teamFfa in tsT
see version for mysql below and full error details

[root@road-flag ~]# mysql --version
mysql  Ver 15.1 Distrib 10.1.43-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Jan 21 10:32:53 road-flag systemd[1]: Started ratingEngine.
Jan 21 10:32:53 road-flag perl[1393]: DBD::mysql::db do failed: Table has no partition for value 202001 at /home/uberserver/builds/SLDB/Sldb.pm line 164.     Jan 21 10:32:53 road-flag perl[1393]: 20200121103253 - NOTICE   - [ratingEngine] Connecting to database DBI:mysql:database=sldb;host=localhost as user sldb   Jan 21 10:32:53 road-flag perl[1393]: 20200121103253 - NOTICE   - [ratingEngine] Initializing trueskill rating environment
Jan 21 10:32:53 road-flag perl[1393]: 20200121103253 - NOTICE   - [ratingEngine] Checking rating database state
Jan 21 10:32:53 road-flag perl[1393]: 20200121103253 - NOTICE   - [ratingEngine] Starting rating queues polling...
Jan 21 10:32:53 road-flag perl[1393]: 20200121103253 - CRITICAL - [ratingEngine] Unable to insert rating for new user 7, period 202001 and type teamFfa in tsTJan 21 10:32:53 road-flag systemd[1]: ratingEngine.service: Main process exited, code=exited, status=1/FAILURE
Jan 21 10:32:53 road-flag systemd[1]: ratingEngine.service: Failed with result 'exit-code'.

Convert three variable assignments to the usage of combined operators

๐Ÿ‘€ Some source code analysis tools can help to find opportunities for improving software components.
๐Ÿ’ญ I propose to increase the usage of combined operators accordingly.

diff --git a/Sldb.pm b/Sldb.pm
index 32fa3d9..872bec1 100644
--- a/Sldb.pm
+++ b/Sldb.pm
@@ -1502,7 +1502,7 @@ sub getFirstRangeAddr {
 # Called by computeAllUserIps()
 sub getLastRangeAddr {
   my $ip=shift;
-  $ip=$ip-($ip%256)+255;
+  $ip -= ($ip%256) - 255;
   return $ip;
 }
 
diff --git a/slMonitor.pl b/slMonitor.pl
index 3186ec3..13e81ed 100755
--- a/slMonitor.pl
+++ b/slMonitor.pl
@@ -555,7 +555,7 @@ sub getFirstRangeAddr {
 
 sub getLastRangeAddr {
   my $ip=shift;
-  $ip=$ip-($ip%256)+255;
+  $ip -= ($ip%256) - 255;
   return $ip;
 }
 
diff --git a/sldbLi.pl b/sldbLi.pl
index d9a761e..613f5dc 100755
--- a/sldbLi.pl
+++ b/sldbLi.pl
@@ -450,7 +450,7 @@ sub formatArray {
       $title="[$title]";
       $title=(' ' x int(($rowLength-realLength($title))/2)).$title.(' ' x ceil(($rowLength-realLength($title))/2));
     }else{
-      $title=$title.':';
+      $title .= ':';
     }
     unshift(@rows,$title);
   }

Variable dbName needs to be rewritten after assignment to dbDs

Migrating system from a dbName=<dbName> to a dbName=DBI: setting I received the following:

20220203123545 - INFO     - [ratingEngine] Creating new partition p202202 in table tsDuelPlayers
20220203123545 - CRITICAL - [ratingEngine] Unable to add partition "p202202" to table tsDuelPlayers (Duplicate partition name p202202)

I'm pretty sure this is caused by this query returning 0, even if the value already exists, due to dbName being set in DBI format: https://github.com/badosu/SLDB/blob/5504e2af76061acbcee40a93f2074da58792a471/ratingEngine.pl#L255

I suspect other components might present the same issue

Default dir configs for sldbLi don't work as expected

Received:

sldbLi_1        | 20220203153446 - ERROR    - [SldbLi] Configuration file "etc/sldbLi.conf" contains inconsistent values for following global parameter(s): etcDir,varDir,logDir
sldbLi_1        | 20220203153446 - ERROR    - [SldbLi] Unable to load main configuration parameters
sldbLi_1        | 20220203153446 - CRITICAL - [SldbLi] Unable to load SldbLi configuration at startup

The default values are empty, replacing with etc, var and log would be helpful: https://github.com/Yaribz/SLDB/blob/master/etc/sldbLi.conf#L19-L21

Missing columns in game table

my games table is

+----------------+---------------------+------+-----+---------------------+-------+
| Field          | Type                | Null | Key | Default             | Extra |
+----------------+---------------------+------+-----+---------------------+-------+
| hostAccountId  | int(10) unsigned    | NO   | PRI | NULL                |       |
| startTimestamp | timestamp           | NO   | PRI | 0000-00-00 00:00:00 |       |
| endTimestamp   | timestamp           | NO   | MUL | 0000-00-00 00:00:00 |       |
| endCause       | tinyint(1) unsigned | YES  |     | NULL                |       |
| hostName       | char(20)            | YES  |     | NULL                |       |
| modName        | varchar(255)        | YES  | MUL | NULL                |       |
| mapName        | varchar(255)        | YES  | MUL | NULL                |       |
| nbSpec         | tinyint(3) unsigned | YES  |     | NULL                |       |
| nbPlayer       | tinyint(3) unsigned | YES  |     | NULL                |       |
| description    | varchar(255)        | YES  |     | NULL                |       |
| passworded     | tinyint(1) unsigned | YES  |     | NULL                |       |
| gameId         | char(32)            | YES  | UNI | NULL                |       |
+----------------+---------------------+------+-----+---------------------+-------+

And yet line 442 of slMonitor.pl has a insert statement with extra columns

I hope doing these manual alters tables fixes the situation but how did I get out of sync with the correct schema?

alter table games add column engineName varchar (30);
alter table games add column engineVersion varchar (100);
alter table games add index (engineVersion);

Add mod to SLDB

Can we please have BAR added to SLDB?
modName:~Beyond\ All\ Reason\ test-\d+-.......

Insert games into local SLDB instance

Hi
Adding allowed games to local instances appears to be undocumented, If the games are to be added with sql can you please advise what format the regex should be? Or, if there is another interface I should be using?
sldbLi.log
20200114105456 - WARNING - [SldbLi] Unable to find ratable mod matching host mod, getSkill call from parity_host
I have tried Beyond All Reason test-d+-....... and ~Beyond All Reason test-d+-....... and /Beyond All Reason test-d+-......./ and Beyond\ All\ Reason\ test-d+-.......

modName:~Beyond\ All\ Reason\ test-\d+-.......

thank you

CRITICAL - [slMonitor] Unable to find rank of user x in accounts table!

slMonitor is crashing with the following msg

20200426031532 - CRITICAL - [slMonitor] Unable to find rank of user "356" in accounts table!

There is no id 356 in the accounts table, max id in accounts table is currently 355
How can this get out of sync? there are no logs of failed inserts
sldbLi log

20200426064300 - NOTICE   - [SLDB] getSkills called for non-rated user ID "356"
20200426064300 - ERROR    - [SLDB] Unable to find rank of unrated player (id:356, userId:356)
20200426064300 - WARNING  - [SldbLi] Unable to find skill of ID "356" / IP <IP ADDRESS REMOVED> getSkill call from parity
20200426064440 - NOTICE   - [SLDB] getSkills called for non-rated user ID "356"
20200426064440 - ERROR    - [SLDB] Unable to find rank of unrated player (id:356, userId:356)
20200426064440 - WARNING  - [SldbLi] Unable to find skill of ID "356" / IP <IP ADDRESS REMOVED>, getSkill call from parity
20200426081318 - NOTICE   - [SLDB] getSkills called for non-rated user ID "6"
20200426082848 - NOTICE   - [SLDB] getSkills called for non-rated user ID "332"
20200426082848 - ERROR    - [SLDB] Unable to find rank of unrated player (id:332, userId:332)
20200426082848 - WARNING  - [SldbLi] Unable to find skill of ID "332" / IP <IP ADDRESS REMOVED>, getSkill call from parity
20200426111240 - NOTICE   - [SLDB] getSkills called for non-rated user ID "337"
20200426112251 - NOTICE   - [SLDB] getSkills called for non-rated user ID "337"
20200426112346 - NOTICE   - [SLDB] getSkills called for non-rated user ID "337"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.