CleanSys is a cleaning schedule management system written in Python (using the Django framework) which is built for large households with many cleaning schedules and time-variant relationships between Cleaners and Schedules.

• Screenshots
• Installation (local)
• Installation on Ubuntu Server with Apache
• Update on Ubuntu Server

CleanSys strives to distribute cleaning duties equally among Cleaners and manages to do so even when there is a high turnover of Cleaners, with people moving in, out or within the household.

It makes sure that:

• Every Cleaner cleans his/her fair share in each cleaning schedule he/she is assigned to (not cleaning too much or too little).
• A Cleaner's work load is spread out over time as much as possible.

A simplified example of a use-case:

CleanSys was built for a 15-person household with 4 floors. Each floor has its own weekly repeating cleaning schedules, two kitchens which some floors share, and several schedules which apply to the entire household.

This cleaning-schedule management system takes over the tedious task of creating schedules on paper and provides additional features, such as a quick and easy way for Cleaners to switch duties with each other.

CleanSys comes with powerful editing capabilities for the administrator, an intuitive interface for the Cleaners, simple click-on-your-name login, and a strong focus on transparency with a granularity down to the sub-task level of each schedule.

This project was made with german users in mind, so the interface language is german as well. If you would like to have a translation (and are willing to put some effort into it yourself), please open an Issue for it.

Here is a selection of CleanSys' pages:

Login page	Cleaner's main page	Duty tasks page	Schedule overview (current week is highlighted)

Schedule print view	Analytics page showing assignment count of all cleaners over time

Administration main page	Cleaner creation form	Schedule creation form

Schedule group creation form	Affiliate a Cleaner with a schedule group	Tasks for a specific schedule	Add a new task to a schedule

Instructions are verified on Mac OSX

Clone the project into your workspace and create the logs and media directories.

cd /path-to-workspace/
git clone https://github.com/monoclecat/cleansys.git
cd cleansys
mkdir logs
mkdir media

A virtual environment isolates Python environment of CleanSys from your system's Python environment.

pip3 install virtualenv  # If you haven't installed the virtualenv package
virtualenv -p python3 .  # Create virtualenv inside the newly cloned repository
source bin/activate  # activate the virtualenv

The required pip packages and their versions are listed in requirements.txt. Install them into your virtualenv's site-packages:

pip3 install -r requirements.txt

Copy the directory cleansys/setting_templates to cleansys/settings. Upon start-up, Django will auto-detect the new settings directory and will run the import statements in __init__.py. Only change files in cleansys/settings, as this directory is mentioned in
.gitignore and won't cause issues when updating CleanSys.

It is very important that you give a new value to the SECRET_KEY's in both dev_settings.py and prod_settings.py (the SECRET_KEY in both files may not be the same!).

common_settings.py contains settings not in need of modification.

This Git ships without a database and any migrations. Create them with:

python manage.py makemigrations
python manage.py migrate

This will set up an empty database with all the required tables and columns.

The admin area of CleanSys uses the login of the Django superuser. Create one with:

python manage.py createsuperuser

The best place to start is to set up the demonstration database. To set it up, run:

python manage.py create_demo_database

Finally, start the Django server:

python manage.py runserver

If you are using a Python IDE such as PyCharm (I very much recommend), you might come across this error when opening the Python Console or running tests from the test parent directory (webinterface/tests):

django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. 
You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

In this case, open PyCharm > Preferences > Build, Execution, Deployment > Console and add DJANGO_SETTINGS_MODULE=cleansys.settings to the Environment Variables field of the Django Console.

If you are getting this error when running tests, do the same to Run/Debug Configurations > Edit Configurations... > Templates > Django Tests and to your existing tests configurations.

Installing CleanSys on an Ubuntu server is very similar to the installation on a Unix system, such as a Mac.

Follow steps 1 through 6 of the previous instructions, substituting path-to-workspace with /var/www/. Also, you will run into 'Permission denied' errors if you don't run some of the command as root (prepend sudo to the command).

I first had to install pip3 sudo apt install python3-pip and virtualenv: pip3 install virtualenv.

Change the ownership of the directory cleansys to your username, create the virtualenv in it without using sudo and install the pip packages inside requirements.txt without using sudo (installing pip packages using sudo is a major security risk).

Having problems creating a virtualenv without sudo? Read this

cd /var/www
sudo git clone https://github.com/monoclecat/cleansys.git
sudo chown -R "$USER":"$USER" cleansys  # Your user needs ownership of cleansys/

virtualenv -p python3 cleansys/
cd cleansys

source bin/activate  # Activate the virtual environment
python3 -m pip install -r requirements.txt

Create the settings directory from settings_template:

cd /var/www/cleansys/cleansys
cp setting_templates/ settings -R  # Don't run with sudo or you'll have to chown again

Adapt dev_settings.py and prod_settings.py, and make sure __init__.py imports the settings you want (edit files using the vim editor with sudo vim filename).

When adapting the production settings, please follow the Django deployment checklist. Necessary edits include:

• Setting ALLOWED_HOSTS
• Setting a new SECRET_KEY
• Setting ADMINS
• Setting the SMTP login credentials (if no email fault reporting is wished, remove the line which sets LOGGING['loggers']['django.request'])

Create the logs and media directories:

mkdir /var/www/cleansys/logs
mkdir /var/www/cleansys/media

Almost there! Set up the empty database and set up the static files for deployment:

python3 manage.py makemigrations
python3 manage.py migrate
python3 manage.py collectstatic

deactivate  # Deactivate virtualenv

Now, we will follow a recommended way of deploying Django: How to use Django with Apache and mod_wsgi

Install requirements:

sudo apt-get install libapache2-mod-wsgi-py3 apache2

When running CleanSys with Apache, server errors and access logs won't be printed into the console but can be found in the logs under /var/log/apache2.

By installing Apache, a new user has been created on your system: www-data, along with a new group with the same name. Following this informative AskUbuntu answer, we will now add our user to the www-data-group. Then, the ownership of all files and directories will be transferred to your user and the www-data-group.

This allows us to split up the permissions for the files and directories. Running chmod 640 on a file will give the owning user read+write permissions (6), all users in the owning group read permissions (4) and any other user no permissions (0). Read more about permissions in the chmod command help.

Add your user to the www-data group and log out and back in to the server to make the group change take effect.

sudo gpasswd -a "$USER" www-data

Now we will set the permissions. We will only give the www-data-group write access where necessary.

sudo chown -R "$USER":www-data /var/www  # Set ownership recursively

find /var/www/ -type f -exec chmod 0640 {} \;
sudo find /var/www -type d -exec chmod 2750 {} \;  # The '2' sets the setgid bit so that all new files inherit the same group

chmod g+w /var/www/cleansys  # Add write permissions to group
chmod g+w /var/www/cleansys/db.sqlite3
chmod g+w /var/www/cleansys/logs
chmod g+w /var/www/cleansys/media

At any time, you can check if the www-data user has sufficient privileges to successfully start the server by running sudo -u www-data bash -c "source bin/activate; python3 manage.py runserver"

Create an Apache site-configuration file:

sudo vim /etc/apache2/sites-available/cleansys.conf

This opens the Vim editor. Press i and paste the following:

WSGIDaemonProcess cleansys python-home=/var/www/cleansys python-path=/var/www/cleansys
WSGIProcessGroup cleansys

<VirtualHost *:80>
    Alias /static/ /var/www/cleansys/webinterface/static/
    <Directory /static>
        Require all granted
    </Directory>
    WSGIScriptAlias / /var/www/cleansys/cleansys/wsgi.py process-group=cleansys
    <Directory /var/www/cleansys/cleansys>
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>
</VirtualHost>

Save the file by first pressing esc to leave insert mode and then pressing :wq (:: "Command", wq: "Write and quit").

When the server has a domain name, make sure to add ServerName www.yourdomain.com in the line after the VirtualHost opening tag.

Now, disable apache2 default site that uses port 80, enable cleansys site and restart apache2:

sudo a2dissite 000-default
sudo a2ensite cleansys
sudo systemctl reload apache2

Keep in mind that CleanSys has no built-in protection against access from outside of your local network, in case your server is accessible from the internet.

When changing any file or ownership, you will have to reload Apache for the changes to take effect.

sudo systemctl reload apache2

Cronjobs are great for automating things. For CleanSys, a Cronjob which runs cronscripts/create_assignments.sh once a week will make sure there are always Assignments for the next WARN_WEEKS_IN_ADVANCE__ASSIGNMENTS_RUNNING_OUT + 4 weeks (see webinterface/management/commands/create_assignments.py). A good documentation on Cron can be found here.

We will be putting our Cronjobs into the system Crontab under /etc/crontab. Just open the file with your favorite terminal editor using sudo (ex. sudo vim /etc/crontab) and append the examples below to the file.

To test a Cronjob you can initially set its interval to */1 * * * * to run it every minute.

The following job will run cronscripts/create_assignments.sh (mentioned above) every Monday at 3:00 in the morning:

0 3 * * 0 www-data bash /var/www/cleansys/cronscripts/create_assignments.sh >> /var/www/cleansys/logs/cron.log

The DutySwitch model enables the trading of Assignments between Cleaners and requires a daily-run cron script to work properly. The following job will run cronscripts/process_dutyswitch_proposals.sh every day at 1:00 in the morning:

0 1 * * * www-data bash /var/www/cleansys/cronscripts/process_dutyswitch_proposals.sh >> /var/www/cleansys/logs/cron.log

The following job will run cronscripts/create_plots.sh every Monday at 3:15 in the morning. These plots will be shown in the Cleaner and Schedule analytics views. It makes sense to run this Cronjob after the create_assignments Cronjob Creating these plots once and just loading their html when the page is called saves a lot of resources.

15 3 * * 0 www-data bash /var/www/cleansys/cronscripts/create_plots.sh >> /var/www/cleansys/logs/cron.log

I also recommend calling python3 manage.py create_plots once after setting up the database and
creating the Assignments for the next weeks.

The following job will create a gzipped backup of db.sqlite3 and put it in /var/www/cleansys/backups every day at 4 in the morning:

0 4 * * * www-data bash /var/www/cleansys/cronscripts/create_backup.sh >> /var/www/cleansys/logs/cron.log

The following job will send the database file to all admins mentioned in the ADMINS setting every Monday at 3:30 in the morning:

30 3 * * 0 www-data bash /var/www/cleansys/cronscripts/send_database_backup.sh >> /var/www/cleansys/logs/cron.log

The following job will call functions which will send these emails every day at 12 o'clock noon:

• send_email__assignment_coming_up(): For each Assignment whose assignment_date() is exactly today+5 days in the future, a notification email is sent to that Cleaner. The Cleaner can turn these notifications on or off in his/her Email preferences.
• send_email__warn_admin_tasks_forgotten(): If the Task of a CleaningWeek with the latest do-until date has just passed (that is, its do-until date ended yesterday) and there are Tasks which have not been completed, an email is sent to the Admin.

An "email to the Admin" means an email to the address of the Django superuser, not to the address specified in Django's ADMINS setting. The address in Django's ADMINS setting is only for server fault tracebacks if these are enabled.

0 12 * * * www-data bash /var/www/cleansys/cronscripts/send_daily_emails.sh >> /var/www/cleansys/logs/cron.log

The following job will call functions which send emails on the Monday of every week at 12 o'clock noon when specific conditions are met:

• send_email__warn_admin_assignments_running_out(): Sends a notification email to the Admin if Schedule.assignments_are_running_out() returns True for any enabled Schedule.
• send_email__warn_admin_cleaner_soon_homeless(): Sends a notification email to the Admin if Cleaner.is_homeless_soon(less_than_equal=False) returns True for any Cleaner.

Again, an "email to the Admin" means an email to the address of the Django superuser, not to the address specified in Django's ADMINS setting.

0 12 * * 0 www-data bash /var/www/cleansys/cronscripts/send_weekly_emails.sh >> /var/www/cleansys/logs/cron.log

Debugging Cronjobs is a bit tricky. The log output given by tail /var/log/syslog will not give you any information useful for debugging. Instead, Cronjobs will send errors by email - not via the email server set up in your Django settings, but over your server's email server.
If you have no email server set up, you will find messages such as "No MTA installed" in /var/log/syslog.

Setting up an email server and client is actually very simple. There are enough good tutorials when searching for ubuntu setting up postfix with mutt. Basically, you need to install postfix, set it to Local only, and install the email client mutt.

Cron will be sending emails to postbox of user www-data, so be sure to run the email client with sudo -H -u www-data mutt, otherwise you will access your logged-in user's postbox.

Before updating, make sure to create a backup of your current installation. Then, check to see if there have been any changes since the last pull.

sudo systemctl stop apache2
cd /var/www
sudo cp -a cleansys /var/backups/<current_date>-cleansys  # Create a backup
cd cleansys
git diff

If you haven't worked on the code, git diff shouldn't return anything. Take note on any changes you see and decide if they are relevant, as in the update process all these changes will be thrown away and must be re-done afterwards.

Next, show differences between the settings you have modified from the templates, and the templates themselves. Screenshot this! You will need it soon.

cd /var/www/cleansys/cleansys
diff setting_templates/ settings/

Now comes the actual update: We pull the newest commit from this repository on Github.

git add -A .  # Track untracked files that could get in the way
git stash  # Any changes to files are 'stashed' to be thrown away later
git pull
git stash pop  # Throw away changes

It is possible that the settings templates have changed and that settings have been added. cleansys/setting_templates is not read by the Django server, only cleansys/settings. But the update may contain changes to the settings, which are now only in cleansys/setting_templates and must be transferred to cleansys/settings.

To see if any action is necessary, check the differences again between the files of both directories:

cd /var/www/cleansys/cleansys
diff setting_templates/ settings/

Compare the output to the screenshot you did before. Anything new? Implement any new changes in the files of the settings directory, so that the output of diff setting_templates/ settings/ matches your screenshot again.

In case you are using the vim editor (in constrast to the more intuitive nano editor) to open files in setting_templates, make sure to close the editor with :q (quit) rather than :wq (write, quit) to ensure you aren't modifying the files there.

Next, update pip packages with an active virtualenv.

source bin/activate  # activate the virtual environment
python3 -m pip install -r requirements.txt  # update packages

python3 manage.py makemigrations  # update database structure
python3 manage.py migrate
deactivate

Make sure all files and directories have the correct permissions set, as stated here.

Finally, restart your server:

sudo systemctl start apache2

Make sure there is no system-wide installed version of virtualenv. virtualenv is meant to be run as a non-root user, and installing virtualenv as root prevents this (see this and this) In my case, after trying around, I had installed virtualenv once via pip3 with sudo and once via apt with sudo, so I had to uninstall it twice with:

sudo python3 -m pip uninstall virtualenv
sudo apt remove virtualenv

And then install it for my local user without root (notice the --user flag):

sudo python3 -m pip install virtualenv --user

Upon which I also received this important warning (I obfuscated by username with <username>):

WARNING: The script virtualenv is installed in '/home/<username>/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location

This told me that the path that the local pip3 package was installed to wasn't in my PATH variable. The PATH variable (see its value with echo $PATH) stores the directory of executables and is set in ~/.profiles. Checking ~/.profiles revealed an if-statement which adds the exact directory mentioned above to $PATH if it exists. But here is the catch: ~/.profiles is only run when you login to your server. So the solution is to simply restart your session (CTRL+D and log in again).

Thanks to @nspo for his mentorship along the way! I recommend his pybarsys in case your large household or organization has a bar or a snack cupboard you would like to digitize!