Comments (15)
Just say if busy, in the plan, not coming or whatever.
I might work on one but prefer to avoid duplicate.
Thanks
from zentral.
Sorry for the late reply.
We don't have a Zentral ansible role.
from zentral.
Pushed a first review https://github.com/juju4/ansible-zentral
but architecture is a bit more complicated than I expected with image. lot of dependencies :)
I would appreciate a bit of guidance, especially as it seems, I need to setup a specific daemon for simplesamlphp. right?
for now, all services fails on configuration but I think it's because zentral.conf module is not available.
Thanks
from zentral.
Quick tips and review.
-
set the ZENTRAL_CONF_DIR variable to point to the dir containing
base.json
. -
Try to start with a simplier configuration.
- Postgres
- Rabbitmq
- Elasticsearch
- Nginx
- Zentral gunicorn
- Zentral workers
-
Spin-up a zentral-all-in-one image on GCE or AWS. This is a good example of a simple deployment on Ubuntu 16.04, with some management scripts to update zentral, get the certificates, configure the admin user…
from zentral.
It's not that simple having many dependencies and daemons.
Was already playing a lot with the vagrant all in one but as image, they hide complexity and partly the understanding of the innerwork of zentral.
Documented post-install is done properly I think. That's more what is in image and less documented which is harder
On the setup, I installed all daemons except promotheus but at this point zentral gunicorn or workers are not launching correctly. I also added redis because I thought it was needed but doesn't seem so.
$ /usr/bin/python server/manage.py runserver runworkers
Traceback (most recent call last):
File "server/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 316, in execute
settings.INSTALLED_APPS
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/var/_zentral/zentral/server/server/settings.py", line 19, in <module>
from zentral.conf import saml2_idp_metadata_file, settings as zentral_settings
File "/var/_zentral/zentral/zentral/conf/__init__.py", line 3, in <module>
from .utils import find_conf_file, load_config_file
File "/var/_zentral/zentral/zentral/conf/utils.py", line 37
raise ImproperlyConfigured("{} error in file {}".format(filetype, filepath)) from None
^
SyntaxError: invalid syntax
$ python -mjson.tool ../conf/start/zentral/base.json
[OK]
$ /usr/local/bin/gunicorn --chdir /var/_zentral/zentral/server -b 0.0.0.0:8000 -w 4 --access-logfile /var/log/zentral/gunicorn.log --error-logfile /var/log/zentral/gunicorn.err server.wsgi
Traceback (most recent call last):
File "/usr/local/bin/gunicorn", line 11, in <module>
sys.exit(run())
File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 74, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 203, in run
super(Application, self).run()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 231, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 344, in halt
self.stop()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 393, in stop
time.sleep(0.1)
File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 244, in handle_chld
self.reap_workers()
File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 524, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
If I launch manually from zentral all-in-one vagrant the zentral_worker
vagrant@vagrant:/home/zentral/app/releases/current$ sudo -H -u zentral /home/zentral/app/venv/bin/python server/manage.py runworkers
Traceback (most recent call last):
File "server/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/zentral/app/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/zentral/app/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 316, in execute
settings.INSTALLED_APPS
File "/home/zentral/app/venv/lib/python3.5/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/home/zentral/app/venv/lib/python3.5/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/home/zentral/app/venv/lib/python3.5/site-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 665, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/zentral/app/releases/2017.03.02-19.36.11-master-b98066/server/server/settings.py", line 19, in <module>
from zentral.conf import saml2_idp_metadata_file, settings as zentral_settings
File "/home/zentral/app/releases/2017.03.02-19.36.11-master-b98066/zentral/conf/__init__.py", line 38, in <module>
settings = load_config_file(find_conf_file(conf_dir, "base"))
File "/home/zentral/app/releases/2017.03.02-19.36.11-master-b98066/zentral/conf/utils.py", line 12, in find_conf_file
raise ImproperlyConfigured('{} is required'.format(' or '.join(filepaths)))
zentral.core.exceptions.ImproperlyConfigured: ('/home/zentral/app/releases/2017.03.02-19.36.11-master-b98066/conf/base.json or /home/zentral/app/releases/2017.03.02-19.36.11-master-b98066/conf/base.yml is required', None)
from zentral.
from zentral.
from zentral.
I added ZENTRAL_CONF_DIR in systemd conf files and in /etc/profile.
Python3 was probably the main issue. would be a good addition to the doc. don't remember seeing it.
vagrant@vmyrole:~$ more /lib/systemd/system/zentral-workers.service
[Unit]
Description=Zentral workers
#Requires=redis.service
#Wants=mysqld.service postgresql.service
#After=redis.service
[Service]
## no daemon mode
#Type=simple
## daemon mode
#Type=forking
User=_zentral
Group=_zentral
Environment="ZENTRAL_CONF_DIR=/var/_zentral/zentral/conf/start/zentral"
WorkingDirectory=/var/_zentral/zentral
ExecStart=/usr/bin/python3 server/manage.py runserver runworkers
#ExecStart=/usr/bin/python server/manage.py runserver runworkers --prometheus-sd-file /var/_zentral/zentral/conf/prometheus_sd/workers.yml --
promotheus-base-port 8100
UMask=0022
TimeoutSec=30
RestartSec=15s
#Restart=always
Restart=on-failure
SyslogIdentifier=zentral
PrivateTmp=true
[Install]
WantedBy=default.target
vagrant@vmyrole:~$ sudo -H -u _zentral -s
_zentral@vmyrole:/home/vagrant$ export ZENTRAL_CONF_DIR=/var/_zentral/zentral/conf/start/zentral
_zentral@vmyrole:/home/vagrant$ /usr/bin/python3 server/manage.py runserver runworkers
/usr/bin/python3: can't open file 'server/manage.py': [Errno 2] No such file or directory
_zentral@vmyrole:/home/vagrant$ cd /var/_zentral/zentral
_zentral@vmyrole:~/zentral$ /usr/bin/python3 server/manage.py runserver runworkers
CommandError: "runworkers" is not a valid port number or address:port pair.
_zentral@vmyrole:~/zentral$ /usr/bin/python3 server/manage.py runserver runworkers --prometheus-sd-file /var/_zentral/zentral/conf/prometheus_sd/workers.yml --promotheus-base-port 8100
usage: manage.py runserver [-h] [--version] [-v {0,1,2,3}]
[--settings SETTINGS] [--pythonpath PYTHONPATH]
[--traceback] [--no-color] [--ipv6] [--nothreading]
[--noreload] [--nostatic] [--insecure]
[addrport]
manage.py runserver: error: unrecognized arguments: --prometheus-sd-file /var/_zentral/zentral/conf/prometheus_sd/workers.yml --promotheus-base-port 8100
_zentral@vmyrole:~/zentral$ /usr/bin/python3 server/manage.py runserver runworkers 8100
usage: manage.py runserver [-h] [--version] [-v {0,1,2,3}]
[--settings SETTINGS] [--pythonpath PYTHONPATH]
[--traceback] [--no-color] [--ipv6] [--nothreading]
[--noreload] [--nostatic] [--insecure]
[addrport]
manage.py runserver: error: unrecognized arguments: 8100
gunicorn seems fine with python3 but web interface gets
FATAL: Peer authentication failed for user "zentral"
most probably because shell user is _zentral...
from zentral.
manage.py
, the management script for Django, accepts only one command at a time. runserver
and runworkers
are two different commands, and runworkers
is interpreted as an argument for runserver
in your examples. That's why it doesn't work.
Look in the docker-entrypoint.sh script for examples how to use the different management commands.
from zentral.
I looked at docker-entrypoint when starting.
At this point, base is here and building fine in travis https://travis-ci.org/juju4/ansible-zentral
Just need to review those services
ah runserver runworkers
is a bad copy/paste most probably... Good catch!
once fixed, just a relationship warning and web interface is up
$ sudo systemctl status zentral-workers
● zentral-workers.service - Zentral workers
Loaded: loaded (/usr/lib/systemd/system/zentral-workers.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2017-04-23 02:48:42 UTC; 9s ago
Main PID: 30665 (python3)
CGroup: /system.slice/zentral-workers.service
├─30665 /usr/bin/python3 server/manage.py runworkers
├─30670 /usr/bin/python3 server/manage.py runworkers
├─30671 /usr/bin/python3 server/manage.py runworkers
├─30672 /usr/bin/python3 server/manage.py runworkers
├─30673 /usr/bin/python3 server/manage.py runworkers
└─30674 /usr/bin/python3 server/manage.py runworkers
Apr 23 02:48:44 vmyrole zentral[30665]: File "/usr/lib64/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
Apr 23 02:48:44 vmyrole zentral[30665]: six.reraise(dj_exc_type, dj_exc_value, traceback)
Apr 23 02:48:44 vmyrole zentral[30665]: File "/usr/lib64/python3.4/site-packages/django/utils/six.py", line 685, in reraise
Apr 23 02:48:44 vmyrole zentral[30665]: raise value.with_traceback(tb)
Apr 23 02:48:44 vmyrole zentral[30665]: File "/usr/lib64/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
Apr 23 02:48:44 vmyrole zentral[30665]: return self.cursor.execute(sql, params)
Apr 23 02:48:44 vmyrole zentral[30665]: django.db.utils.ProgrammingError: relation "inventory_machinesnapshot" does not exist
Apr 23 02:48:44 vmyrole zentral[30665]: LINE 1: ...ventory_machinesnapshot"."public_ip_address" FROM "inventory...
Apr 23 02:48:44 vmyrole zentral[30665]: ^
Apr 23 02:48:44 vmyrole zentral[30665]: 2017-04-23 02:48:44,002 PID30670 workers INFO inventory worker dummy - sleep 30 seconds
from zentral.
You need to run the Django migrate
command the first time and after every Zentral code update to synchronize the table structure in Postgres with the models described in the code. This will create update and remove the necessary tables.
This is done too in the docker-entrypoint.py script.
from zentral.
ok. I put it aside because I was not sure it mapped manage.py migrate
but it is, so good.
Centos7 fun because postgresql < 7.4 ... using postgres repo for 9.5. fine on Xenial.
I also change server/settings.py to match customized pgsql db/user/pass.
Environment storage is not the best from security
https://news.ycombinator.com/item?id=8826024
and more recently https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/
Any option to set user password in an unattended way?
One thing to clarify. the gunicorn and manage.py runserver are the same?
because in existing images, there is only zentral_web and zentral_workers while I added at the beginning a zentral systemd conf mapping manage.py runserver.
Thanks
from zentral.
runserver
is a standard Django management command. It will start a simple server that is absolutely not recommended for production use. That's why we use gunicorn
in the zentral_web_app.service
of the zentral-all-in-one images.
from zentral.
perfect. removing it. will review the rest this weekend.
from zentral.
Following this thread :)
from zentral.
Related Issues (20)
- docker deployment
- Enrollments (santa/osquery) can't be edited/removed once created HOT 1
- Manifest-side, munki/osquery enrollments can't use quota's/serials/udid's for filtering/restriction
- Minor, results view search field does not work when supplied bare integers, requires quoting
- http_post probe action fails with "NoneType object is not callable" HOT 2
- Restrict email invitation domain
- Adding TOTP to a local user repeatedly fails HOT 7
- Can't get syslog output working HOT 15
- deploy.py fails on running migrations while deploying v2021.2-100-g760f7d81 HOT 5
- export in zentral targets not working HOT 4
- Support for token auth in jamf
- Release notes date 2021 should be 2022
- AWS all in one HOT 1
- Docker deployment on cloud vm HOT 2
- Best way to retrieve osquery query run results from external app? HOT 3
- Feature request: metadata linkable to/inline with service accounts/api keys in users view
- Add identifier patterns for signing ID rules to API HOT 3
- Accept unusual Google identifiers for Signing ID Santa rules via ruleset API and rules/create in web interface HOT 1
- Ruleset API endpoint rejects signing IDs containing underscore and minus characters HOT 1
- backend not found HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zentral.