Giter Club home page Giter Club logo

openedx-docker's Introduction

Open edX Docker

France Université Numérique introduces an alternative Docker approach to install a complete and customized version of Open edX.

The idea is to handcraft a Dockerfile, in order to make the project simpler, more flexible and fully operable by developers.

Quick preview

If you're looking for a quick preview of OpenEdX Docker, you can take a look at our dedicated demo site.

It is connected back-to-back with a demo of Richie, a CMS for Open Education based on Django.

Two users are available for testing:

Richie's admin is available at https://demo.richie.education/admin/ and can be accessed via the following user account: admin/admin.

The demonstration databases are regularly flushed.

Approach

This project builds a docker image that is ready for production.

At France Université Numérique, we are deploying Open edX Docker to OpenShift, for many customers and in multiple environments using Arnold:

  • The Open edX settings were polished to unlock powerful configuration management features: sensible defaults, flexible overrides using YAML files and/or environment variables, secure credentials using Ansible Vault and PGP keys,
  • We focused on a best practice installation of edxapp, the core Open edX application. You can build you own image by adding specific Xblocks or Django apps in a Dockerfile inheriting from this one (see https://github.com/openfun/fonzie for an example).

Docker compose is only used for development purposes so that we can code locally and see our changes immediately:

  • sources and configuration files are mounted from the host,
  • the Docker CMD launches Django's development server instead of gunicorn,
  • ports are opened on the application containers to allow bypassing nginx.

Docker compose also allows us to run a complete project in development, including database services which in production are not run on Docker. See the docker-compose file for details on each service:

  • mysql: the SQL database used to structure and persist the application data,
  • mongodb: the no-SQL database used to store course content,
  • memcached: the cache engine,
  • lms: the Django web application used by learners,
  • cms: the Django web application used by teachers,
  • nginx: the front end web server configured to serve static/media files and proxy other requests to Django.
  • mailcatcher the email backend

Concerning Redis, it is possible to run a single redis instance (the default choice) or to run redis with sentinel to simulate a HA instance. To use Redis sentinel you have to set the REDIS_SERVICE environment variable to redis-sentinel:

$ export REDIS_SERVICE=redis-sentinel

To switch back to the single redis instance, unset this environment variable:

$ unset REDIS_SERVICE

Prerequisite

Make sure you have a recent version of Docker and Docker Compose installed on your laptop:

$ docker -v
  Docker version 26.0.0, build 2ae903e

$ docker compose version
  docker compose version v2.24.5

Getting started

First, you need to choose a release/flavor of OpenEdx versions we support. You can list them and get instructions about how to select/activate a target release using the bin/activate script. An example output follows:

$ bin/activate
Select an available release to activate:
[1] master/0/bare (default)
[2] hawthorn/1/bare
[3] hawthorn/1/oee
Your choice: 3

# Copy/paste hawthorn/1/oee environment:
export EDX_RELEASE="hawthorn.1"
export FLAVOR="oee"
export EDX_RELEASE_REF="open-release/hawthorn.1"
export EDX_DEMO_RELEASE_REF="open-release/hawthorn.1"

# Or run the following command:
. ${HOME}/Work/openedx-docker/releases/hawthorn/1/oee/activate

# Check your environment with:
make info

Once your environment is set, start the full project by running:

$ make bootstrap

You should now be able to view the web applications:

See other available commands by running:

$ make --help

Developer guide

If you intend to work on edx-platform or its configuration, you'll need to compile static files in local directories that are mounted as docker volumes in the target container:

$ make dev-assets

Now you can start services development server via:

$ make dev

You should be able to view the web applications:

Hacking with themes

To work on a particular theme, we invite you to use the paver watch_assets command; e.g.:

$ make dev-watch

Troubleshooting: if the command above raises the following error:

OSError: inotify watch limit reached

Then you will need to increase the host's fs.inotify.max_user_watches kernel setting (for reference, see https://unix.stackexchange.com/a/13757):

# /etc/sysctl.conf (debian based)
fs.inotify.max_user_watches=524288

Available Docker images

The aim of this project is to prepare several flavors of images, the docker files and settings of which are living in their own directory (see releases/)

Release paths on the current repository are of the form:

{release name}/{release number}/{flavor}

With:

  • release name: OpenEdx release name (e.g. hawthorn)
  • release number: OpenEdx release number (e.g. 1)
  • flavor: the release flavor (e.g. bare for standard OpenEdx release and oee for OpenEdx Extended release).

We are pushing to DockerHub only images that are the result of a tag respecting the following pattern:

{release}(-{flavor})-x.y.z

The release (e.g. hawthorn.1) typically includes the release name (e.g. hawthorn) and the release number (e.g. 1). The flavor is optional.

Here are some valid examples:

  • dogwood.3-1.0.3
  • hawthorn.1-oee-2.0.1

Nginx

This project also provides an nginx companion image that can be used alongside the edxapp image for faster deployments and better performance.

The classical way to handle and serve static files in a Django application is to collect them (using the collectstatic management command) and post-process them using an appropriate storage back-end that uses cache-busting techniques to avoid old versus new static files collisions (e.g. ManifestStaticFilesStorage).

Depending on the edx-platform release, some files may not benefit from a cache busting md5 hash, leading to unexpected side effects during deployment.

To prevent such behavior, for each new openedx-docker release, we will also release an edxapp-nginx image with the same Docker tag. This image is an OpenShift-ready nginx image with embedded edxapp's static files. You are supposed to use this image to serve your static files, media and reverse proxy to the version-matching edxapp instance.

Alternative projects

If what you're looking for is a quick 1-click installation of the complete Open edX stack, you may take a look at Tutor.

License

The code in this repository is licensed under the GNU AGPL-3.0 terms unless otherwise noted.

Please see LICENSE for details.

openedx-docker's People

Contributors

jbpenrath avatar jmaupetit avatar kernicpanel avatar lunika avatar madmatah avatar regisb avatar rlecellier avatar rmoch avatar sampaccoud avatar slandrault avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openedx-docker's Issues

Improve Sentry configuration & installation consistency

Purpose

development settings are broken for all branches if Sentry is not activated, and it cannot be activated for non-oee branches where it is not installed.

Proposal

  • remove Sentry settings in non-oee branches *
  • fix Sentry's logger configuration in development settings to make it optional for oee branches

* Alternatively, we can choose to install Sentry even for non oee branches.

@sampaccoud WDYT?

Investigate anonymous user id discrepancy

Our Sentry is getting spammed by the same repeating error:

Stored anonymous user id <SimpleLazyObject: <User: insider>> for user None in course u'rrrssstttuuuvvvwwwxxxyyyzzz' doesn't match computed id u'aaabbbcccdddeeefffggghhhiii'

We had to silence it fast in order to avoid bursting our events quota on Sentry, so we commented out the piece of code that logs the error: see PR openfun/edx-platform#56.

However, eventhough these errors do not seem to affect our users, we should take the time to understand it and fix it at its root.

Course import is not working

Expected Behavior

We should be able to import a complete course as a tar.gz in the CMS

Actual Behavior

When doing so, the import never go further "Updating course" step and stay there forever

capture d ecran 2018-06-19 a 09 30 15

Steps to Reproduce

go to fun-mooc.fr to export an actual course, CMS: tools>export
in your fun-platform instance CMS: Create new course then: tools>import

Specifications

dogwood branches

Set X_FRAME_OPTIONS Django setting to an allowed value

Expected Behavior

No JS error related to the X-Frame-Options header should occur.

Actual Behavior

The X-Frame-Options header value is set to ALLOW, which is not allowed (for reference, see https://developer.mozilla.org/fr/docs/Web/HTTP/Headers/X-Frame-Options), leading to the following Require.js dynamic loading error:

Error: Dynamic load not allowed: common/templates/components/system-feedback.underscore base.js:83:8563

Steps to Reproduce

  1. Go to the Studio (CMS) course details view
  2. The following error message should appear in the browser console:
Invalid X-Frame-Options: “ALLOW” header from “https://cms.staging.foo.fr/settings/details/course-v1:Musicality+CS101+2019_T4” loaded into “https://cms.staging.foo.fr/course/course-v1:Musicality+CS101+2019_T4”.

Specifications

  • Version: at least dogwood.3-fun-1.3.4 (I think all releases and flavors are impacted)
  • Platform: Firefox 71 (Ubuntu GNU/Linux)

script `bin/activate` fails to find out its current directory on macosx

Expected Behavior

bin/activate script should retrieve available flavors from /releases folder

Actual Behavior

On Macosx system, realpath command does not exist

Steps to Reproduce

⇒  bin/activate
bin/activate: line 8: realpath: command not found
bin/activate: line 8: realpath: command not found
find: /releases/: No such file or directory
Select an available flavored release to activate:
Your choice:

Proposal

could we use $PWD to get current directory ?

PROJECT_DIRECTORY="$(PWD)"

devstack does not load static files

After installing hawthorn bare, I am able to start the production version.
However, both, the lms and the studio on the production side does not load the static files at all. I get a 404 for javascript files., csss etc.

Any ideas?

Thanks

[Errno 2] No such file or directory: Path(u'/edx/app/edxapp/edx-demo-course/course.xml')

Hi Openfun,
Can you help me fix it? I don't know how to solve this problem.

Thank you very much!

Importing...
    data_dir=/edx/var/edxapp/data, source_dirs=['/edx/app/edxapp/edx-demo-course']
    Importing static content? True
    Importing python lib? True
2019-09-16 05:27:22,212 ERROR 1 [xmodule.modulestore.xml] xml.py:407 - ERROR: Failed to load courselike '/edx/app/edxapp/edx-demo-course': [Errno 2] No such file or directory: Path(u'/edx/app/edxapp/edx-demo-course/course.xml')
Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml.py", line 402, in try_load_course
    course_descriptor = self.load_course(course_dir, course_ids, errorlog.tracker, target_course_id)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml.py", line 463, in load_course
    with open(self.data_dir / course_dir / self.parent_xml) as course_file:
IOError: [Errno 2] No such file or directory: Path(u'/edx/app/edxapp/edx-demo-course/course.xml')
[variant:cms][xmodule.modulestore.xml][env:sandbox] ERROR [b5371ce2c564  1] [xml.py:407] - ERROR: Failed to load courselike '/edx/app/edxapp/edx-demo-course': [Errno 2] No such file or directory: Path(u'/edx/app/edxapp/edx-demo-course/course.xml')
Traceback (most recent call last):
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml.py", line 402, in try_load_course
    course_descriptor = self.load_course(course_dir, course_ids, errorlog.tracker, target_course_id)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/xml.py", line 463, in load_course
    with open(self.data_dir / course_dir / self.parent_xml) as course_file:
IOError: [Errno 2] No such file or directory: Path(u'/edx/app/edxapp/edx-demo-course/course.xml')


Error while composing docker

Expected Behavior

Description...

Actual Behavior

Step 42/50 : RUN groupadd --gid ${DOCKER_GID} edx || echo "Group with ID ${DOCKER_GID} already exists." && useradd --create-home --home-dir /home/edx --uid ${DOCKER_UID} --gid ${DOCKER_GID} edx
---> Running in 22f9fa762ae2
Group with ID 0 already exists.
groupadd: GID '0' already exists
useradd: UID 0 is not unique
ERROR: Service 'lms-dev' failed to build: The command '/bin/sh -c groupadd --gid ${DOCKER_GID} edx || echo "Group with ID ${DOCKER_GID} already exists." && useradd --create-home --home-dir /home/edx --uid ${DOCKER_UID} --gid ${DOCKER_GID} edx' returned a non-zero code: 4
Makefile:207: recipe for target 'dev-build' failed
make: *** [dev-build] Error 1

Steps to Reproduce

make bootstrap

Specifications

  • Version: releases/hawthorn/1/oee
  • Platform: ubuntu 16.04

Add documentation about system dependencies (what and why)

Purpose

The list of apt dependencies in the DockerFile is obscure and may become difficult to maintain.

Proposal

Add documentation in the repository to list all system dependencies with an explanation of why they are required (edX dependency, developer use for debugging, etc.)

[dogwood/3/fun] Allow customizing the CELERY_RESULT_BACKEND setting

Expected Behavior

In dogwood/3/fun, we are trying to set the CELERY_RESULT_BACKEND setting to use the database as a result backend : https://github.com/openfun/openedx-docker/blob/master/releases/dogwood/3/fun/config/lms/settings.yml#L81

Actual Behavior

Actually, in the lms & cms docker_run_production.py file, CELERY_RESULT_BACKEND is not made configurable.

Solution

Use our config helper to set a default value for the CELERY_RESULT_BACKEND setting and make it configurable.

Development images improvement proposal

When developing in openedx-docker we use specifically built images.
Typical process when working on edx-platform, fun-apps or external library is to clone repository one want to work on on host filesystem, manually modify docker-compose to mount it in container, then log in container with bin/compose run --rm --service-ports lms-dev bash. Then install code, modify project settings and run Django dev server. All that ending in passing much time building images, and typing commands in a very poor container bash.

Here are some ideas to ease this process.

  • Add unix packages to container like git, text editor, network related commands...
  • Allow .bash_history persistence between container run
  • Ease the process of mounting repositories from host to container
  • install Mysql client and/or allow external connection to mysql server

The development on opendex-docker is also done by frequently changing flavors, which leads to many image builds. We could improve the situation by allowing images to be retrieved from Docker Hub instead of building them.

Speed-up CI by building only what has changed

Expected Behavior

CI builds should be kept as fast as possible.

Actual Behavior

The CI builds all releases all the time. Now that we are adding 2 releases for ironwood, it will slow it down even more.

Proposal

We could generate the CI configuration file, on the basis of what has changed, in order to rebuild only the impacted releases :

  • If what has changed is in the directory of a release : rebuild only this release
  • If what has changed impacts all releases (e.g. in the docker directory? anything else?) : rebuild all releases

Something similar was done in richie site factory. See the "bin/ci update" command and CircleCI's documentation about packing a configuration with the CLI.

ERROR: Service 'lms' failed to build : The command '/bin/sh -c python get-pip.py' returned a non-zero code: 1

When i run "$ make bootstrap"
This problem:
Hi there!

The URL you are using to fetch this script has changed, and this one will no
longer work. Please use get-pip.py from the following URL instead:

https://bootstrap.pypa.io/pip/2.7/get-pip.py

Sorry if this change causes any inconvenience for you!

We don't have a good mechanism to make more gradual changes here, and this
renaming is a part of an effort to make it easier to us to update these
scripts, when there's a pip release. It's also essential for improving how we
handle the get-pip.py scripts, when pip drops support for a Python minor
version.

There are no more renames/URL changes planned, and we don't expect that a need
would arise to do this again in the near future.

Thanks for understanding!

  • Pradyun, on behalf of the volunteers who maintain pip.

ERROR: Service 'lms' failed to build : The command '/bin/sh -c python get-pip.py' returned a non-zero code: 1
Makefile:159: recipe for target 'build' failed
make: *** [build] Error 1

make bootstrap(get-pip) error

Expected Behavior

make bootstrap

Actual Behavior

=> CACHED [production 1/4] RUN apt-get update && apt-get upgrade -y 0.0s
=> ERROR [builder 4/9] RUN python get-pip.py 0.3s

[builder 4/9] RUN python get-pip.py:
#23 0.255
#23 0.255 Hi there!
#23 0.255
#23 0.255 The URL you are using to fetch this script has changed, and this one will no
#23 0.255 longer work. Please use get-pip.py from the following URL instead:
#23 0.255
#23 0.255 https://bootstrap.pypa.io/pip/2.7/get-pip.py
#23 0.255
#23 0.255 Sorry if this change causes any inconvenience for you!
#23 0.255
#23 0.255 We don't have a good mechanism to make more gradual changes here, and this
#23 0.255 renaming is a part of an effort to make it easier to us to update these
#23 0.255 scripts, when there's a pip release. It's also essential for improving how we
#23 0.255 handle the get-pip.py scripts, when pip drops support for a Python minor
#23 0.255 version.
#23 0.255
#23 0.255 There are no more renames/URL changes planned, and we don't expect that a need
#23 0.255 would arise to do this again in the near future.
#23 0.255
#23 0.255 Thanks for understanding!
#23 0.255
#23 0.255 - Pradyun, on behalf of the volunteers who maintain pip.
#23 0.255


executor failed running [/bin/sh -c python get-pip.py]: exit code: 1
ERROR: Service 'lms' failed to build

Steps to Reproduce

Description...

Specifications

:: OPENEDX-DOCKER ::.

== Active configuration ==

Failed to solve: failed to compute cache key: failed to calculate checksum of ref

Expected Behavior

Can not run completed bootstrap command.

Actual Behavior

We got issues with the download library.

Steps to Reproduce

  1. make bootstrap

Specifications

[edxapp 5/8] COPY --from=downloads /downloads/edx-platform-* .:


failed to solve: failed to compute cache key: failed to calculate checksum of ref zp2wh8029gbqt1357kbr6hdhz::i1p0qru1gizriupvnxy4bzah1: "/downloads/edx-platform-open-release-ironwood.2/node_modules/edx-pattern-library/pattern-library/fonts": not found
make: *** [build] Error 17

Screenshot 2023-06-16 at 3.13.04 PM

Control access to instructor CSV files

Expected Behavior

Instructor CSV export files should only be accessible to instructors of the given course

Actual Behavior

edX default configuration allows any user knowing the url to download files from filesystem or s3.
Also the url is not difficult to guess. This is a major security flaw.

Proposal

Add to Fonzie an API endpoint to control user authorization to access requested file then serve it with Nginx using X-Accel-Redirect header.

We will then configure edx-platform to download files from API endpoint instead of Nginx location

GRADES_DOWNLOAD = {
   "STORAGE_CLASS": "django.core.files.storage.FileSystemStorage",
   "STORAGE_KWARGS": {
       "location": "/edx/var/edxapp/media/export",  # mountpoint where files will be stored in openedx-docker container
       "base_url": "/api/v1.0/protected/instructor_file",  # URL to access them
   }
}

And use Nginx to serve export internally.

location ~ ^/protected/(?P<file>.*) {
  root /data/media/export;
  try_files /$file =404;
  internal;
}

Unexpected end of JSON input while parsing near '...,"coffeescript":"^1.1'

Expected Behavior

Success

Actual Behavior

On 847f8fd, with an empty Docker cache and:

$ make info
.:: OPENEDX-DOCKER ::.

== Active configuration ==

* EDX_RELEASE                : eucalyptus.3
* FLAVOR                     : bare
* FLAVORED_EDX_RELEASE_PATH  : releases/eucalyptus/3/bare
* EDX_RELEASE_REF            : open-release/eucalyptus.3
* EDX_ARCHIVE_URL            : https://github.com/edx/edx-platform/archive/open-release/eucalyptus.3.tar.gz
* EDX_DEMO_RELEASE_REF       : open-release/eucalyptus.3
* EDX_DEMO_ARCHIVE_URL       : https://github.com/edx/edx-demo-course/archive/open-release/eucalyptus.3.tar.gz
* REDIS_SERVICE              : redis
* EDXAPP_IMAGE_NAME          : edxapp
* EDXAPP_IMAGE_TAG           : eucalyptus.3-bare
* EDXAPP_NGINX_IMAGE_NAME    : edxapp-nginx
$ docker --version ; docker-compose --version
Docker version 19.03.13, build 4484c46d9d
docker-compose version 1.27.3, build 4092ae5d
Step 43/80 : RUN npm install
 ---> Running in 9d9a12a108e9
npm WARN deprecated [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated [email protected]: This reporter is now deprecated and will no longer be supported by the Pa11y team. Please refer to the Pa11y migration guide or README to update your project.
npm WARN deprecated [email protected]: This module is no longer maintained, try this instead:
npm WARN deprecated   npm i nyc
npm WARN deprecated Visit https://istanbul.js.org/integrations for other alternatives.
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: This package is unmaintained. Use @sinonjs/formatio instead
npm WARN deprecated [email protected]: This package has been deprecated in favour of @sinonjs/samsam
npm WARN deprecated [email protected]: This package has been deprecated in favour of @sinonjs/samsam
npm ERR! Unexpected end of JSON input while parsing near '...,"coffeescript":"^1.1'

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-09-30T03_48_24_981Z-debug.log
ERROR: Service 'lms' failed to build : The command '/bin/sh -c npm install' returned a non-zero code: 1
Makefile:143: recipe for target 'build' failed
make: *** [build] Error 1

Docs are unclear for - how to change a theme?

Thanks for this repo!
I'm on the process of developing a theme and I'm not sure how to add it to the docker images so it will be hot reloaded.

after running
$ make clone $ make dev-assets
I need to activate the theme and run make dev-watch.
The process of activation is not clear, as in devstack I would ssh into the docker machine, clone the repo and change the lms.env.json file etc.
From digging into the repo I've found a script bin/watch which is looking for a variable called THEME but it's unclear if this script is running and when + where should I clone my repo into?

Error on devstack hawthorn.master

When execute docker-compose up

ERROR:
edx.devstack.studio | python: can't open file '/edx/app/edxapp/edx-platform/manage.py': [Errno 2] No such file or directory
edx.devstack.lms | python: can't open file '/edx/app/edxapp/edx-platform/manage.py': [Errno 2] No such file or directory

docker-compose isn't setting up right

Expected Behavior

Description...

Can't docker-compose up

make info

.:: OPENEDX-DOCKER ::.

== Active configuration ==

* EDX_RELEASE                : hawthorn.1
* FLAVOR                     : oee
* FLAVORED_EDX_RELEASE_PATH  : releases/hawthorn/1/oee
* EDX_RELEASE_REF            : hawthorn.1
* EDX_ARCHIVE_URL            : https://github.com/edx/edx-platform/archive/open-release/hawthorn.1.tar.gz
* EDX_DEMO_RELEASE_REF       : open-release/hawthorn.1
* EDX_DEMO_ARCHIVE_URL       : https://github.com/edx/edx-demo-course/archive/open-release/hawthorn.1.tar.gz
* REDIS_SERVICE              : redis
* EDXAPP_IMAGE_NAME          : edxapp
* EDXAPP_IMAGE_TAG           : hawthorn.1-oee
* EDXAPP_NGINX_IMAGE_NAME    : edxapp-nginx

Actual Behavior

ERROR: build path /Users/toor/workspace/openedx-docker/releases/master/0/bare either does not exist, is not accessible, or is not a valid URL.

Steps to Reproduce

Description...

toor@galileo:~/workspace/openedx-docker$ bin/activate
Select an available flavored release to activate:
[1] /ironwood/2/oee (default)
[2] /hawthorn/1/oee
[3] /eucalyptus/3/wb
[4] /dogwood/3/fun
Your choice: 1

# Copy/paste /ironwood/2/oee environment:
export EDX_RELEASE="ironwood.2"
export FLAVOR="oee"
export EDX_RELEASE_REF="ironwood.2"
export EDX_DEMO_RELEASE_REF="open-release/ironwood.2"

# Or run the following command:
. bin/../releases//ironwood/2/oee/activate

# You can check your environment with:
make info
toor@galileo:~/workspace/openedx-docker$ export EDX_RELEASE="ironwood.2"
toor@galileo:~/workspace/openedx-docker$ export FLAVOR="oee"
toor@galileo:~/workspace/openedx-docker$ export EDX_RELEASE_REF="ironwood.2"
toor@galileo:~/workspace/openedx-docker$ export EDX_DEMO_RELEASE_REF="open-release/ironwood.2"
toor@galileo:~/workspace/openedx-docker$ docker-compose up
WARNING: The DOCKER_UID variable is not set. Defaulting to a blank string.
WARNING: The DOCKER_GID variable is not set. Defaulting to a blank string.
ERROR: build path /Users/toor/workspace/openedx-docker/releases/master/0/bare either does not exist, is not accessible, or is not a valid URL.

Specifications

  • Version: master
  • Platform: Mac OS Docker version 20.10.13, build a224086

Fix docker-compose file image tags

Purpose

Project images should be tagged with Open edX release name (e.g. ginkgo.1) and flagged with-dev for development containers: edxapp:ginkgo.1 or edxapp:ginkgo.1-dev. But this tagging schema can occur while publishing a container.

For now, those tags are hard coded for the ginkgo.1 release in our docker-compose.yml file. I think, it can be more generic for a local usage (until we develop our flavors generator).

Proposal

Let's switch to edxapp:latest and edxapp:dev for now.

Fix dependencies issues on `dogwood/3/bare` and `eucalyptus/3/bare`

Expected Behavior

We should have fully functionnal version of dogwood/3/bare and eucalyptus/3/bare images.

Actual Behavior

Recent builds of dogwood/3/bare and eucalyptus/3/bare flavors are not working anymore.
The installed version of the django-classy-tags dependency is 2.0.0, and this version has dropped support for python 2.x.

In the current state of the project, there is no way to specify custom dependencies for bare flavors to fix this kind of issues.

Proposal

  • Add custom requirements.txt to bare flavors and use it in the Dockerfile
  • Pin django-classy-tags to 0.8.0 in dogwood/3/bare and eucalyptus/3/bare

Service 'lms' failed to build : The command '/bin/sh -c python get-pip.py' returned a non-zero code: 1

Expected Behavior

Run docker success

Actual Behavior

Run fail

Steps to Reproduce

Hi there!

The URL you are using to fetch this script has changed, and this one will no
longer work. Please use get-pip.py from the following URL instead:

https://bootstrap.pypa.io/pip/2.7/get-pip.py

Sorry if this change causes any inconvenience for you!

We don't have a good mechanism to make more gradual changes here, and this
renaming is a part of an effort to make it easier to us to update these
scripts, when there's a pip release. It's also essential for improving how we
handle the get-pip.py scripts, when pip drops support for a Python minor
version.

There are no more renames/URL changes planned, and we don't expect that a need
would arise to do this again in the near future.

Thanks for understanding!

  • Pradyun, on behalf of the volunteers who maintain pip.

ERROR: Service 'lms' failed to build : The command '/bin/sh -c python get-pip.py' returned a non-zero code: 1

Specifications

  • run with command:
    docker-compose up -d

Cannot build dogwood/3/fun image: "ImportError: cannot import name resolve_url"

Expected Behavior

I expect the dogwood/3/fun image to build correctly.

Actual Behavior

Image build fails during asset collection.

Steps to Reproduce

I ran the following commands:

$ ./bin/activate
# (select dogwood/3/fun)
$ . ./bin/../releases/dogwood/3/fun/activate
$ make bootstrap
...
Step 51/51 : RUN NO_PREREQ_INSTALL=1     paver update_assets --settings=fun.docker_build_production --skip-collect
 ---> Running in 6f92e3749cad
Warning: could not find environment JSON file at '/edx/app/lms.env.json'
/usr/local/lib/python2.7/dist-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support 
for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography import utils, x509
---> pavelib.assets.update_assets
---> pavelib.prereqs.install_ruby_prereqs
NO_PREREQ_INSTALL is set, not installing prereqs
---> pavelib.prereqs.install_node_prereqs
NO_PREREQ_INSTALL is set, not installing prereqs
python manage.py lms --settings=fun.docker_build_production preprocess_assets lms/static/sass/*.scss lms/static/themed_sass
Traceback (most recent call last):
  File "manage.py", line 112, in <module>
    startup.run()
  File "/edx/app/edxapp/edx-platform/lms/startup.py", line 34, in run
    django.setup()
  File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/local/lib/python2.7/dist-packages/django/apps/config.py", line 119, in create
    import_module(entry)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/src/django-oauth2-provider/provider/oauth2/__init__.py", line 5, in <module>
    import urls
  File "/usr/local/src/django-oauth2-provider/provider/oauth2/urls.py", line 36, in <module>
    from django.contrib.auth.decorators import login_required
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 6, in <module>
    from django.shortcuts import resolve_url
ImportError: cannot import name resolve_url

Specifications

  • Version: cf545ea
  • Platform: Ubuntu 18.04
➭ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

I expect to investigate the root cause, but I am creating this Github issue in the hope that someone has already resolved it. Please check my notes below while I investigate the problem.

[CI] Evaluate jobs parallelization

Purpose

Since the CI refactoring in #112, one must add a new job for each new release. This is great to run build in parallel, but we should test the CI-builtin parallelization mechanisms [1].

[1] https://circleci.com/docs/2.0/parallelism-faster-jobs/#using-environment-variables-to-split-tests

Proposal

  • Define a generic build job with parallelism: 4 (or any other relevant value)
  • Use the circleci tests split command to split releases and build one release per container

Refactor utility script

Purpose

For every release handled in the project, we need to copy the same utils.py script to handle configurable variables. We think there is room for improvement.

Proposal

@sampaccoud suggested to have a single location where it is versioned and copy it while building the Docker image.

Improve PROFILE_IMAGE_SECRET_KEY default value

Expected Behavior

User should be able to upload profile image on its profile page http://site/u/username

Actual Behavior

upload fails because setting PROFILE_IMAGE_SECRET_KEY is not set

File "/edx/app/edxapp/edxplatform/openedx/core/djangoapps/user_api/accounts/image_helpers.py", line 42, in _make_profile_image_name
return hashlib.md5(settings.PROFILE_IMAGE_SECRET_KEY + username).hexdigest()
TypeError: unsupported operand type(s) for +: 'NoneType' and 'unicode'

[CI] remove unused edx-platform sources clone

Purpose

At the time of writing, during the build job in the CI, we clone and save to cache the edx-platform sources. This is no longer required as they are not used for the docker image build (which uses only release archives).

Proposal

  • remove the clone and save_cache steps related to edx-platform sources

`bin/clone_repositories` fails to checkout `edx-platform` on certain platforms

Expected Behavior

bin/clone_repositories script and related Makefile commands should checkout edx-platform in ./src at a given release tag

Actual Behavior

on MacOSX, git 2.19

git checkout -f release-2018-08-29-14.14

fails to checkout revision and returns:

error: pathspec 'release-2018-08-29-14.14' did not match any file(s) known to git.

This is a serious issue because all development commands rely on this script

Remove default args in Dockerfile

Purpose

Every Dockerfile define a default EDX_RELEASE_REF and EDX_ARCHIVE_URL argument. Every time we want to use a new release, we have to update the corresponding Dockerfile.

Proposal

We should remove the default value and explicitly use build arguments in our Makefile.

wiki.0005_remove_attachments_and_images fails

Expected Behavior

Start the container using make bootstrap.

Actual Behavior

The starting process is failing with the following error:

Running migrations:
  Applying wiki.0005_remove_attachments_and_images...Traceback (most recent call last):
  File "manage.py", line 121, in <module>
    execute_from_command_line([sys.argv[0]] + django_args)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 156, in database_forwards
    schema_editor.remove_field(from_model, from_model._meta.get_field(self.name))
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 487, in remove_field
    self.execute(sql)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 136, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 101, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue

django.db.utils.OperationalError: (1091, "Can't DROP 'current_revision_id'; check that column/key exists")
Makefile:80: recipe for target 'migrate' failed
make: *** [migrate] Error 1

Steps to Reproduce

Just starting the project with:

  1. Clone the repo.
  2. Run sudo make bootstrap

Specifications

  • Version: Commit 6feedeb
  • Platform: Ubuntu 18.04 LTS Server (2GB RAM, 1 CPU Unit)

Add Celery workers to our Docker Compose set-up

Purpose

Edxapp is now configured to use Redis for the Celery workers by default. We should now add the Celery workers to our docker compose set-up.

Proposal

  • Add a Redis database with default configuration
  • Add celery workers for lms and cms (one for lms and one for cms running all the queues should be enough)

hawthorn.1-2.6.0: Javascript errors on instructor dashboard

Expected Behavior

LMS Instructor dashboard should allow asynchronous generation of CSV files and download

Actual Behavior

The page crashes, with javascript errors probably related to incomplete build

Steps to Reproduce

  1. Set LMS settings to
    FEATURES["ENABLE_GRADE_DOWNLOADS"] = True
    FEATURES["ALLOW_COURSE_STAFF_GRADE_DOWNLOADS"] = True  
    FEATURES["ENABLE_ASYNC_ANSWER_DISTRIBUTION"] = False
    FEATURES["ENABLE_INSTRUCTOR_BACKGROUND_TASKS"] = False
  1. import demo course
  2. login as instructor to LMS
  3. go to /courses/course-v1:edX+DemoX+Demo_Course/instructor#view-data_download
  4. click on Download profile inforamtion as CSV
  5. a request to /instructor/api/get_students_features/csv should be POSTed

Specifications

  • Version: hawthorn.1-2.6.0

oee image works fine (despite they are some JS errors as well)

Environment Dev doesn't work

Environment Dev doesn't work with ironwood.2

.:: OPENEDX-DOCKER ::.

== Active configuration ==
$ make info
* EDX_RELEASE                : ironwood.2
* FLAVOR                     : oee
* FLAVORED_EDX_RELEASE_PATH  : build
* EDX_RELEASE_REF            : open-release/ironwood.2
* EDX_ARCHIVE_URL            : https://github.com/edx/edx-platform/archive/open-release/ironwood.2.tar.gz
* EDX_DEMO_RELEASE_REF       : open-release/ironwood.2
* EDX_DEMO_ARCHIVE_URL       : https://github.com/edx/edx-demo-course/archive/open-release/ironwood.2.tar.gz
* REDIS_SERVICE              : redis
* EDXAPP_IMAGE_NAME          : edxapp
* EDXAPP_IMAGE_TAG           : ironwood.2-oee
* EDXAPP_NGINX_IMAGE_NAME    : edxapp-nginx

Actual Behavior

Should work with

$ make dev
or
$ ./bin/compose up lms-dev

Steps to Reproduce

$ make dev

DOCKER_UID=1000 DOCKER_GID=1000 FLAVORED_EDX_RELEASE_PATH="releases/ironwood/2/oee" EDXAPP_IMAGE_TAG=ironwood.2-oee docker-compose run --rm -e HOME="/tmp" --no-deps lms-dev bash -c "\
  rm -f /edx/app/edxapp/edx-platform/lms/envs/fun && \
  rm -f /edx/app/edxapp/edx-platform/cms/envs/fun && \
  ln -sf /config/lms /edx/app/edxapp/edx-platform/lms/envs/fun && \
  ln -sf /config/cms /edx/app/edxapp/edx-platform/cms/envs/fun && \
  ln -sf /config/lms/root_urls.py /edx/app/edxapp/edx-platform/lms/ && \
  ln -sf /config/cms/root_urls.py /edx/app/edxapp/edx-platform/cms/"
Creating openedx-docker_lms-dev_run ... done
# starts lms-dev as well via docker-compose dependency
DOCKER_UID=1000 DOCKER_GID=1000 FLAVORED_EDX_RELEASE_PATH="releases/ironwood/2/oee" EDXAPP_IMAGE_TAG=ironwood.2-oee docker-compose up -d cms-dev
openedx-docker_mongodb_1 is up-to-date
openedx-docker_mailcatcher_1 is up-to-date
openedx-docker_redis_1 is up-to-date
openedx-docker_memcached_1 is up-to-date
openedx-docker_mysql_1 is up-to-date
Creating openedx-docker_lms-dev_1 ... done
Creating openedx-docker_cms-dev_1 ... done
Wait for services to be up...
DOCKER_UID=1000 DOCKER_GID=1000 FLAVORED_EDX_RELEASE_PATH="releases/ironwood/2/oee" EDXAPP_IMAGE_TAG=ironwood.2-oee docker-compose run --rm -e HOME="/tmp" dockerize -wait tcp://mysql:3306 -timeout 60s
Creating openedx-docker_dockerize_run ... done
2021/07/06 05:37:01 Waiting for: tcp://mysql:3306
2021/07/06 05:37:01 Connected to tcp://mysql:3306
DOCKER_UID=1000 DOCKER_GID=1000 FLAVORED_EDX_RELEASE_PATH="releases/ironwood/2/oee" EDXAPP_IMAGE_TAG=ironwood.2-oee docker-compose run --rm -e HOME="/tmp" dockerize -wait tcp://cms-dev:8000 -timeout 60s
Creating openedx-docker_dockerize_run ... done
2021/07/06 05:37:02 Waiting for: tcp://cms-dev:8000
2021/07/06 05:37:02 Problem with dial: dial tcp 172.20.0.11:8000: getsockopt: connection refused. Sleeping 1s
2021/07/06 05:37:03 Problem with dial: dial tcp 172.20.0.11:8000: getsockopt: connection refused. Sleeping 1s
2021/07/06 05:37:04 Problem with dial: dial tcp 172.20.0.11:8000: getsockopt: connection refused. Sleeping 1s

OR

./bin/compose up lms-dev
openedx-docker_mailcatcher_1 is up-to-date
openedx-docker_mongodb_1 is up-to-date
openedx-docker_mysql_1 is up-to-date
openedx-docker_memcached_1 is up-to-date
openedx-docker_redis_1 is up-to-date
Starting openedx-docker_lms-dev_1 ... done
Attaching to openedx-docker_lms-dev_1
lms-dev_1         | WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/__init__.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported
lms-dev_1         |   warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)
lms-dev_1         | 
lms-dev_1         | Traceback (most recent call last):
lms-dev_1         |   File "manage.py", line 120, in <module>
lms-dev_1         |     startup.run()
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run
lms-dev_1         |     django.setup()
lms-dev_1         |   File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 27, in setup
lms-dev_1         |     apps.populate(settings.INSTALLED_APPS)
lms-dev_1         |   File "/usr/local/lib/python2.7/dist-packages/django/apps/registry.py", line 108, in populate
lms-dev_1         |     app_config.import_models()
lms-dev_1         |   File "/usr/local/lib/python2.7/dist-packages/django/apps/config.py", line 202, in import_models
lms-dev_1         |     self.models_module = import_module(models_module_name)
lms-dev_1         |   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
lms-dev_1         |     __import__(name)
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/lms/djangoapps/bulk_email/models.py", line 16, in <module>
lms-dev_1         |     from openedx.core.djangoapps.course_groups.cohorts import get_cohort_by_name
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/course_groups/cohorts.py", line 9, in <module>
lms-dev_1         |     from courseware import courses
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/courses.py", line 25, in <module>
lms-dev_1         |     from courseware.module_render import get_module
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/module_render.py", line 60, in <module>
lms-dev_1         |     from openedx.core.djangoapps.bookmarks.services import BookmarksService
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/bookmarks/services.py", line 12, in <module>
lms-dev_1         |     from . import DEFAULT_FIELDS, api
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/bookmarks/api.py", line 11, in <module>
lms-dev_1         |     from .models import Bookmark
lms-dev_1         |   File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/bookmarks/models.py", line 41, in <module>
lms-dev_1         |     class Bookmark(TimeStampedModel):
lms-dev_1         |   File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 118, in __new__
lms-dev_1         |     "INSTALLED_APPS." % (module, name)
lms-dev_1         | RuntimeError: Model class openedx.core.djangoapps.bookmarks.models.Bookmark doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
openedx-docker_lms-dev_1 exited with code 1

Specifications

  • Version: master
  • Platform: ubuntu 20.04

[create-symlinks] Error 1

Expected Behavior

Description...

Actual Behavior

dbritos@HP:~/openedx-docker$ make dev-assets
bash -c "mkdir -p data/{static/production,static/development,media,store}"
UID=1000 GID=1000 docker-compose run --rm -e HOME="/tmp" --no-deps lms-dev bash -c "
rm -f /edx/app/edxapp/edx-platform/lms/envs/fun &&
rm -f /edx/app/edxapp/edx-platform/cms/envs/fun &&
ln -sf /config/lms /edx/app/edxapp/edx-platform/lms/envs/fun &&
ln -sf /config/cms /edx/app/edxapp/edx-platform/cms/envs/fun"
ln: failed to create symbolic link '/edx/app/edxapp/edx-platform/lms/envs/fun': No such file or directory
Makefile:42: recipe for target 'create-symlinks' failed
make: *** [create-symlinks] Error 1
Description...

When i follow the steps in "https://github.com/openfun/openedx-docker" and type "make dev-assets" i receive "make: *** [create-symlinks] Error 1"

I run Ubuntu 18.04 and docker
dbritos@HP:~/openedx-docker$ docker -v
Docker version 18.09.5, build e8ff056

Steps to Reproduce

Description...

  1. item 1...
  2. item 2...

Specifications

Ubuntu 18.04
Docker version 18.09.5, build e8ff056

  • Version: ...
  • Platform: ...

How to configure the production environemnt

I need your help in making the environment for production ready.
How do i configure the MYSQL and mongoDB environemnt for production version?
I am using RDS-Mysql and MongoDB Atlas for the prodution work load.

ImportError: No module named proj.settings

Working with releases/eucalyptus/3/bare. Can you send through the entire folder it expects? - I'm sure I can figure out the settings it needs from there, but I'd rather not need to reverse-engineer the entire thing…

Improve user experience when trying to build/run the project with a root user

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
The make bootstrap command fails when launched via a root user or with sudo. We need to fail early, nicely and explain what is the problem of running Docker with a root user instead of failing with a difficult error after screwing up the user's file system.

See this discussion for details: #203

Describe the solution you'd like
Any make command should start by checking that the user does not have the 0 id with:

id -u

Describe alternatives you've considered
An alternative could be to modify the Dockerfile to allow building it with a root user.
In this case we can still display a warning but let the user continue if s.he knows what s.he is doing.

mongodb read_preference value is not allowed in development

Expected Behavior

We should be able to use the cms-dev service at: http://localhost:8082

Actual Behavior

A 500 error is raised due to an apparently mis-configuration. The stack trace follows:

Environment:


Request Method: GET
Request URL: http://localhost:8082/home/

Django Version: 1.8.12
Python Version: 2.7.3
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'djcelery',
 'method_override',
 'openedx.core.djangoapps.common_views',
 'simple_history',
 'config_models',
 'service_status',
 'django_nose',
 'contentstore',
 'course_creators',
 'external_auth',
 'student',
 'openedx.core.djangoapps.course_groups',
 'xblock_config',
 'track',
 'eventtracking.django.apps.EventTrackingConfig',
 'datadog',
 'edxmako',
 'pipeline',
 'static_replace',
 'require',
 'openedx.core.djangoapps.theming',
 'django_comment_common',
 'django.contrib.admin',
 'course_modes',
 'dark_lang',
 'reverification',
 'openedx.core.djangoapps.user_api',
 'django_openid_auth',
 'embargo',
 'monitoring',
 'course_action_state',
 'edx_jsme',
 'openedx.core.djangoapps.content.course_overviews',
 'openedx.core.djangoapps.content.course_structures',
 'openedx.core.djangoapps.credit',
 'xblock_django',
 'edx_proctoring',
 'openedx.core.djangoapps.programs',
 'openedx.core.djangoapps.self_paced',
 'provider',
 'provider.oauth2',
 'oauth2_provider',
 'lms.djangoapps.verify_student',
 'edx_sga',
 'submissions',
 'openassessment',
 'openassessment.assessment',
 'openassessment.fileupload',
 'openassessment.workflow',
 'openassessment.xblock',
 'edxval',
 'milestones',
 'organizations',
 'ckeditor',
 'courses',
 'easy_thumbnails',
 'edx_gea',
 'fun',
 'haystack',
 'password_container',
 'raven.contrib.django.raven_compat',
 'selftest',
 'teachers',
 'universities',
 'videoproviders')
Installed Middleware:
(u'raven.contrib.django.middleware.DjangoRestFrameworkCompatMiddleware',
 u'raven.contrib.django.middleware.SentryMiddleware',
 'request_cache.middleware.RequestCache',
 'django.middleware.cache.UpdateCacheMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'method_override.middleware.MethodOverrideMiddleware',
 'cache_toolbox.middleware.CacheBackedAuthenticationMiddleware',
 'student.middleware.UserStandingMiddleware',
 'contentserver.middleware.StaticContentServer',
 'crum.CurrentRequestUserMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'track.middleware.TrackMiddleware',
 'dark_lang.middleware.DarkLangMiddleware',
 'embargo.middleware.EmbargoMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'edxmako.middleware.MakoMiddleware',
 'ratelimitbackend.middleware.RateLimitMiddleware',
 'session_inactivity_timeout.middleware.SessionInactivityTimeout',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')


Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in inner
  145.                     return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
  22.                 return view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
  110.                     response = view_func(request, *args, **kwargs)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/course.py" in course_listing
  429.     courses, in_process_course_actions = get_courses_accessible_to_user(request)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/course.py" in get_courses_accessible_to_user
  596.         courses, in_process_course_actions = _accessible_courses_list(request)
File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/course.py" in _accessible_courses_list
  366.     courses = filter(course_filter, modulestore().get_courses())
File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py" in modulestore
  206.             contentstore(),
File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/contentstore/django.py" in contentstore
  28.         _CONTENTSTORE[name] = class_(**options)
File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/contentstore/mongo.py" in __init__
  42.             port=port, tz_aware=tz_aware, user=user, password=password, proxy=proxy, **kwargs
File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/mongo_connection.py" in connect_to_mongodb
  38.             **kwargs
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py" in __init__
  311.             option, value = common.validate(option, value)
File "/usr/local/lib/python2.7/dist-packages/pymongo/common.py" in validate
  446.     value = validator(option, value)
File "/usr/local/lib/python2.7/dist-packages/pymongo/common.py" in validate_read_preference
  243.         raise ConfigurationError("Not a valid read preference")

Exception Type: ConfigurationError at /home/
Exception Value: Not a valid read preference

Steps to Reproduce

Start the development server:

$ make dev

Then browse the development CMS: http://localhost:8082

Specifications

  • Version: 88b9444
  • Tested release: dogwood.3-fun
  • Platform: Firefox 71 // Ubuntu GNU/Linux

Add static files manifest to edxapp image

Expected Behavior

In PR #184, static files were moved to a custom nginx image to be served more efficiently.

Actual Behavior

To make it work we had to keep the static volume and collectstatic as explained in Arnold PR openfun/arnold#477.

Proposal

In order to be able to remove the static volume and collectstatic in edxapp, we need to include, in the edxapp image, a Manifest listing all collected files, with their hashed file name.

While building the edxapp image, run collectstatic using ManifestStaticFilesStorage as backend in order to generate the manifest. Make sure to keep only the manifest and not the files in the edxapp image.

We should also consider not keeping intermediate files: https://code.djangoproject.com/ticket/28604

Add sample yaml files for custom configuration

Purpose

Although the edxapp Docker image includes default settings that work out of the box, we need some information about how to customize such settings via the yaml files.

Proposal

We don't want to add example yaml files in the /config directories because we would think they are necessary and they would uselessly override the default values that have already set in the .py files.

We propose to include an example of yaml file configuration as a snippet in documentation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.