Comments (24)
@lenards Thanks for your help. Adding the 0 to the LOGSTASH_PORT
fixed the error and we continued on.
We now stuck on filling up the parameters in secrets.py
. The most current error is also a syntax error indicating that IRODs
is not defined and we notice that it needs up to provide OpenStack credential afterwards in secrets.py
. Any dummy values you would recommend us to try to assign and it might fix the error?
Thanks so much!
from clank.
Hi @xuhang57,
Empty variables for IRODS_*
will allow you to move forward. I think the "null" YAML values will create an invalid Python syntax situation like you've encountered.
So - valid, empty/default values like this:
IRODS_HOST: ''
IRODS_PORT: 0
IRODS_ZONE: ''
IRODS_ADMIN_USER: ''
IRODS_ADMIN_PASS: ''
Note: I believe that the secrets.py.j2
encloses these in single quotes, so in regression question previous - I believed "null" YAML values did produce valid Python. I'll get that re-evaluated
We have provided some context to the variables in [secrets.py]
within the distribution file in the Atmosphere repository, perhaps that helps with the OpenStack credentials necessary. This aspect is an area we'll be documenting further.
Thank you for your persistence & patience.
from clank.
Hello @lokI8! Thank you for being patient as we flesh out these issues. Like @lenards said above, we have yet to test this on 16.04. In regards to the the DB_VERSION
, it defaults to 9.3. If you wish to change to 9.5 without having to change the role vars, you can modify this variable in the variables.yml. This should propagate downwards into the role. Please let us know if this does not work. I will add an Ubuntu 16.04 var file to the setup-postgres
role to automatically do this for us.
from clank.
Okay Jonathan - again, sorry about this. We have logstash configured in our environments, so the template doesn't handle this gracefully.
If you just give LOGSTASH_PORT: 0
then the local settings file generate will be valid Python:
# LOGSTASH
LOGSTASH_HOST = 'redact.make-redaction-cool-again.org'
LOGSTASH_PORT = 0
try:
import logstash
import threepio
has_logstash = True
except ImportError:
has_logstash = False
from clank.
Adding to that, Jon and I tried to switch from Ubuntu 16.04 to 14.04 LTS and we passed those above errors. During the installation, we add a global variable in dist_files\variables.yml
to fix an error that `TASK [app-generate-ini-config : create local variables.ini.ini from dict] ******
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'MOCK_USERNAME' is undefined"}``
However after that, we failed at the stage **: TASK [app-generate-ini-config : run generate script for app] *********************
The specific errors we are getting are
Undeclared variables found in /opt/dev/atmosphere/extras/init.d/celeryd.default.dist.j2: \n\t- FLOWER_EMAIL_ACCESS_LIST\n\t- FLOWER_OAUTH2_REDIRECT_URI\n\t- FLOWER_OAUTH2_CLIENT_ID\n\t- FLOWER_AUTH_SCHEME\n\t- FLOWER_BASIC_PASSWORD\n\t- FLOWER_OAUTH2_CLIENT_SECRET\n\t- FLOWER_BASIC_USERNAME
And some other errors like Unused variables found in section local.py
on auth parameters even though we set MOC_AUTH as True in the variables.yml
file.
Apologize for throwing so many installation errors at one time. It would be much appreciated if any of you can point us the direction of fix them. Thanks for your help!
from clank.
@xuhang57 @lokI8 - I can get you basic values for the FLOWER_*
variables - that lacking from dist_files/variables.yml is an oversight.
from clank.
@xuhang57 no apologies necessary - thank you for providing details for us.
from clank.
ATMO:
# ....
celeryd.default:
# ...
FLOWER_AUTH_SCHEME: "basic"
# Required if auth_scheme is 'basic'
FLOWER_BASIC_USERNAME: "<username-redacted>"
FLOWER_BASIC_PASSWORD: "<password-redacted>"
# Required if auth_scheme is 'google'
FLOWER_EMAIL_ACCESS_LIST:
FLOWER_OAUTH2_CLIENT_ID:
FLOWER_OAUTH2_CLIENT_SECRET:
FLOWER_OAUTH2_REDIRECT_URI:
from clank.
To Lucas' feedback:
-
MOCK_USERNAME
missing from variables.yml -
FLOWER_*
missing from variables.yml
commit f14a9a7 resolves this as much as it communicates the need for the variables in the distribution file.
from clank.
To Jonathan's (@lokl8) feeback:
As you noted, we are using 14.04.3 so we haven't hit this issue with uWSGI yet. Thank you for sharing this, as it'll be of great assistance as we shift to 16.04.
For context/reference:
The atmosphere repo cloned to /opt/dev/atmosphere has uWSGI version 2.0.9 in it's requirements.txt. We are hitting a compilation error on this package described here: unbit/uwsgi#1262. We've tried changing the version number to 2.0.13, but then clank won't let us pass the cloning of the atmosphere repo, as we've made modifications and it can't pull. We're trying on an older version of Ubuntu now.
I will open an issue in the Atmosphere Repository on GitHub so we can investigate updating to 2.0.13 as a resolution.
@lokI8 - within the variables.yml
provided to Clank, what are is used for ATMOSPHERE_BRANCH
?
That way I can target a potential fix to that branch.
- We had to hard code the version number of postgresql to 9.5 in the main.yml of setup-postgres/tasks, as changing the variable in any of the files in var did not propagate to the ansible role, and the version of postgres automatically installed from clank was 9.5.
Okay - this is great feedback. Our intent is to provide DB_VERSION
as a sensible default within role/setup-postgres
. Could you confirm the version of Ansible you are using?
- We had to manually install ruby 1.9.1 and comment out the line in the requirements file. There is no package matching that exact name on this version of Ubuntu, and apt-get does a reg-ex match to install the correct packages, so when the installer was unable to find it, it error'd out.
Did you get this apt-get
error in 16.04?
Some background, the ruby
dependency is a base (or transitive) dependency for Redis, I believe.
from clank.
Hi Andrew (@lenards), thanks for the quick response! Greatly appreciated.
Our ATMOSPHERE_BRANCH is just set to "" in our variables.yml, I believe this means we're just working off master.
After looking through the variables.yml file again, it appears we missed the "DB_VERSION: 9.3" variable. I'd imagine this is why changing the defaults.yml and Ubuntu.yml in setup-postgres/vars didn't impact anything. Oops! We are using Ansible version 2.1.0.0.
Yes, that apt-get error for ruby was found in 16.04. There is no package for 16.04 ubuntu that is just flat "Ruby$version". Instead, it appears apt-get installs a set of packages that it reg-ex matches to "Ruby1.9.1". We didn't experience this error once we switched over to 14.04.
Our latest error, once adding the variables you provided, is: "Testing for preconditions...", "", "Local variables.ini missing section: celeryd.default", "", "", "Configuration files were not generated.", "", "FAILED. "]. This is bizarre, because we clearly have the celeryd.default section in our variables.yml file, under the ATMO header... we just added the few variables to the end of that section, as you specified.
from clank.
@lokI8 - you are correct, ATMOSPHERE_BRANCH: ""
will default to master.
from clank.
@lokI8 - that error is unexpected regarding celeryd.default
... hmm, let me ponder some possible ways that might happen
from clank.
For the error - you'll want to look at variables.ini
(it might be variables.ini.ini
, which is not intended name) under $CLANK_HOME/roles/app-generate-ini-config/files
to see if that file is missing celeryd.default
under ATMO
.
Also, I gave you a slight misleading outline for those variables. But having more than one celeryd.default
might generate an error as well.
from clank.
For Clank - we have only verified and tested it with Ansible 2.0. I have experienced problems with Ansible 2.1 and passing along variables to parameterized roles (see Ansible Issues 16320). So that may (unconfirmed) lead to the issue with DB_VERSION
- it's on me, of course, to confirm (have not yet).
from clank.
The file is indeed named variables.ini.ini. Should I just mv the file, or should I fix a typo somewhere else?
Also, there is no ATMO header before all of the sections, but there is a "cleryd.default" section. Should I change this to "celeryd.default" as well? Is this file generated, or static?
from clank.
@lokI8 I added a quick fix to the setup-postgres
role to account for db version difference. This fix should prevent you having to modify that role by hand going forward. Thanks again.
from clank.
@lokI8 The ATMO
header will not be present within variables.ini(.ini). That's only a convention for variables.yml to handle communicating the component the sections are associated with (in short, we don't expect it there).
"cleryd.default"
is definitely a typo. That is likely the cause of the "missing section" error. I'd check the variables.yml in use to see if it has the typo as well.
from clank.
I just want to assess where this is: @lokI8 @xuhang57
I believe the remaining issue is to address:
Yes, that apt-get error for ruby was found in 16.04. There is no package for 16.04 ubuntu that is just flat "Ruby$version". Instead, it appears apt-get installs a set of packages that it reg-ex matches to "Ruby1.9.1". We didn't experience this error once we switched over to 14.04.
Which I am guessing that 16.04 doesn't have a 1.9 dependency because Ruby 2.0+ is what projects likely should be using. Perhaps our defined Redis PPA is contributing to that. We will investigate.
Was the following issue with variables.ini
and the ./configure
script for Atmosphere repo resolved?
Our latest error, once adding the variables you provided, is: "Testing for preconditions...", "", "Local variables.ini missing section: celeryd.default", "", "", "Configuration files were not generated.", "", "FAILED. "]. This is bizarre, because we clearly have the celeryd.default section in our variables.yml file, under the ATMO header... we just added the few variables to the end of that section, as you specified.
from clank.
@lokI8 Regarding:
The file is indeed named variables.ini.ini. Should I just mv the file, or should I fix a typo somewhere else?
The variables.ini.ini
can be left as is - the Ansible role will copy it (with the extra .ini
) into the correct location in ATMOSPHERE_LOCATION
(likely set to /opt/dev/atmosphere
by default).
What I would encourage you to do is verify the spelling of celeryd.default
within your variables.yml file. I think that is the source of the typo because roles/app-generate-ini-config
is iterating over the sections of ATMO
to produce the variables.ini(.ini) (sans ATMO
root heading).
from clank.
We did indeed have the same misspelling in our variables.yml as well. We've fixed that and have continued on.
The most recent error is during the app-django-manage-collectstatic role. We are getting a syntax error on line 192 of /opt/dev/atmosphere/atmosphere/settings/local.py
, on the variable LOGSTASH_PORT =
. I believe we need a value afterwords, if just the empty string, so it does not error. However, changing the code in /opt/dev/atmosphere
makes ratchet unhappy at the git cloning stage. Should we create our own branch and fix this?
Thanks for all the help =)
from clank.
@lokI8 - I was hoping not to give you the "you could clone Atmosphere & set the ATMOSPHERE_LOCATION
to that" workaround. Before doing that, let me investigate a bit. I think if you are not using a logstash instance, you might be able to set it to LOGSTASH_PORT: 0
just to give it a valid integer to make Python happy.
from clank.
@lokI8 @lenards We are having a debug session with Andy, Andre and Steve tomorrow. So I think we could close this issue now.
Thanks for all the help and we really appreciate it
from clank.
Thank you Lance. I will close this. We appreciate all the feedback and patience.
from clank.
Related Issues (20)
- Revisit "List of Files Needed" in README
- ImportError: No module named backports.ssl_match_hostname HOT 1
- Run Clank twice in Travis test
- Use `pip-tools` for Clank's requirements HOT 1
- Problem: Troposphere pull requests are created without any build verification
- Clean up travis.yml's postgres install section
- Issues of running 'pip install -r clank/requirements.txt' HOT 3
- Cannot restart atmosphere on Ubuntu 16.04 HOT 2
- Make finer grained backups
- Problem: Atmosphere Django cache directory not writable by www-data HOT 1
- Problem: Configure system `ulimit` (open files) HOT 1
- Run clank with previous release of Atmosphere/Troposphere HOT 7
- Clank on CentOS7 HOT 1
- Create New Relic deployment
- Warn/Error when New Relic is not installed/configured/working HOT 1
- Not picking up the correct repos to clone HOT 1
- Cannot uninstall new relic HOT 7
- app-install-instance-deploy-automation role fails on remote deployment when creating hosts file
- CI test in Different Linux Distros HOT 3
- Default NGINX config breaks http->https redirect
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clank.