openedx / xqueue Goto Github PK

XQueue defines an interface for the LMS to communicate with external grader services.

License: GNU Affero General Public License v3.0

Shell 0.20% Python 95.89% HTML 0.03% Makefile 2.53% Dockerfile 1.34%

xqueue's Introduction

⛔️ WARNING

This repository is under-maintained. We are not fixing bugs or developing new features for it. We hope to deprecate it officially in October of 2024. For updates, follow along on the DEPR ticket.

Although we have stopped integrating new contributions, we always appreciate security disclosures and patches sent to [email protected]

xqueue

XQueue defines an interface for the LMS to communicate with external grader services. For example, when a student submits a problem in the LMS, it gets sent to the XQueue. The XQueue then has the problem graded by an external service and sends the response back to the LMS.

How the LMS Interacts with the XQueue

1. The LMS pushes student submissions to the XQueue with an HTTP POST request to the URL /xqueue/submit. The submission contains a callback URL indicating where the graded response should be sent.

2. When the submission has been graded, the XQueue pushes a response back to the LMS with an HTTP POST request to the callback URL.

How External Graders Interact with the XQueue

There are two ways kinds of grading services: passive and active. These interact with the XQueue in different ways.

Passive Graders (aka Push Graders)

Passive graders wait for the XQueue to send them submissions. They then respond synchronously with a graded response.

The LMS sends messages to a particular queue.

2. XQueue checks its settings and finds that the queue has a URL associated with it. XQueue forwards the message to that URL.

3. The passive grader receives a POST request from the XQueue and responds synchronously with the graded response.

4. XQueue forwards the graded response to the callback URL the LMS provided in its original message.

Active Graders (aka Pull Graders)

Active graders pull messages off the XQueue and push responses back to the XQueue.

The test client sends messages to a particular queue.

2. The active grader polls the XQueue using a REST-like interface. When it receives a submission, it pushes a response back to the XQueue, also using a REST-like interface.

If a submission is retrieved but not graded, it will be available in the queue later following the SUBMISSION_PROCESSING_DELAY when it is requeued.

XQueue pushes the response back to the LMS.

Local Testing

XQueue is now available as a docker image used as part of Docker Devstack. You can start and provision it by using xqueue specific commands configured in your devstack directory

make dev.provision.xqueue make dev.up.xqueue

XQueue will use the shared MySQL image.

Tests

You can run the unit/integration test suite using:

make test

after connecting to your container on Docker Devstack

make xqueue-shell make test

License

The code in this repository is licensed under version 3 of the AGPL unless otherwise noted.

Please see LICENSE.txt for details.

How to Contribute

Contributions are very welcome. The easiest way is to fork this repo, and then make a pull request from your fork. The first time you make a pull request, you may be asked to sign a Contributor Agreement.

Reporting Security Issues

Please do not report security issues in public. Please email [email protected]

Mailing List and other support options

https://open.edx.org/getting-help

xqueue's People

Contributors

Stargazers

Watchers

Forkers

avontd2868 admin-cm himangshuj kern3020 hikari-boulders antoviaque sokratik hanoibanhcuon agamdua smartdec name1984 mitocw nttks wutt77 andreibosco mshabdiz bdacode jeancfo iitbmooc bharatmooc sinsixx cmscom slacrey xiandiancloud xdmooc moocedx edulib maximst beacloudgenius pombreda knehez credoeducation xuyeli2002 wkoha miptliot open-craft defance hastexo pobrejuanito vimfun tony90s prembasumatary mitodl liukangxu cloudswyft-iac edificeedtech kkux atnartur idqweb armbox travlr-archived simonclouds leeyoungcheol leollon edraak learningtribes fikriamri grupocato proversity-org ed-hec eduzone-ca deepa-vn dipsims edunext digitallearn lttc-eduhk danialmalik anant-singh osll stvstnfrd aszykm ucsd-ets hircumg regisb akirasankun vteamlabs mitol-devops titlethanason mumarkhan999 followjust samnanglaor mtyaka magajh brytemorio cmltawt0 mrdot29 huniafatima-arbi

xqueue's Issues

Unusual JSON schemas

I wonder if there is any documentation for the JSON entities returned by the XQueue endpoints?

So far the best I can do is reverse engineering. I was surprised to see JSON objects embedded in JSON strings. What is the motivation for doing that? The drawbacks of this solution are that:

it makes it harder to define a proper JSON schema for these entities (consequently, it makes it impossible to decode these entities with automated solutions based on such schemas),
entities have to be parsed several times,
entities take more space because they are escaped several times,
pretty printing an entity is hopeless.

Why not use plain old JSON objects instead?

To be clearer, I propose that we replace the following schema:

{
  "foo": "{ \"bar\": \"baz\" }"
}

With this one:

{
  "foo": {
    "bar": "baz"
  }
}

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that satisfies the requirement futures==3.2.0 (from versions: 0.2.python3, 0.1, 0.2, 1.0, 2.0, 2.1, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6, 2.2.0, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.1.0, 3.1.1)
Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.6.7/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/scripts/compile.py", line 191, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 198, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 284, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/local.py", line 65, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 217, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(download_dir, ireq, wheel_cache)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 183, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 218, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 164, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/index.py", line 621, in find_requirement
    'No matching distribution found for %s' % req
pip._internal.exceptions.DistributionNotFound: No matching distribution found for futures==3.2.0

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

AttributeError: 'module' object has no attribute 'Queue'

Hi, I just install devstack (Hawthorn) on my Mac with docker. When I run the xqueue service, I met the following error:

  File "/edx/app/xqueue/xqueue/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 206, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 40, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/edx/app/xqueue/xqueue/queue/management/commands/run_consumer.py", line 3, in <module>
    from queue.consumer import Worker
  File "/edx/app/xqueue/xqueue/queue/consumer.py", line 8, in <module>
    import requests
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/urllib3/__init__.py", line 8, in <module>
    from .connectionpool import (
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 44, in <module>
    from .util.queue import LifoQueue
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/urllib3/util/queue.py", line 10, in <module>
    class LifoQueue(queue.Queue):
AttributeError: 'module' object has no attribute 'Queue'```

Any idea how to solve this issue? Thanks.

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches pluggy==0.6.0,==0.8.0
Tried: 0.3.0, 0.3.0, 0.3.1, 0.3.1, 0.4.0, 0.4.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.6.0, 0.6.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches pluggy==0.6.0,==0.8.0
Tried: 0.3.0, 0.3.0, 0.3.1, 0.3.1, 0.4.0, 0.4.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.6.0, 0.6.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Django 4.2 Upgrade

Description

Under the effort of Django 4.2 Upgrade, complete all of the following steps to complete the upgrade.

Update tox & Github action workflow using modernisers to add support for Django 4.2
Remove any versions of Python earlier than 3.8 from tox.ini, and GitHub Actions workflows.
Update the pinned Django version in the requirements to Django==4.2
Run make upgrade to update all dependencies for Django 4.2.
Run and verify all tests are passing in the CI for both Django 4.2 and Django 3.2 to contain backward compatibility.
Run available code-mods to fix the failing tests.
Add conditional checks wherever needed to still have support for Django 3.2.
Update the repo support field in the IDA Upgrade Sheet.

Switch to Ansible free Docker Image

In order to advance implementation of both OEP-45: Configuring and Operating Open edX and deprecation of the configuration repository, we would like to switch devstack from using primarily Docker images built with Ansible code from the configuration repository to images built from Dockerfiles in each service's repository. We hope this will also help simplify Tutor by providing better base images to derive from, with fewer workarounds and duplicated code blocks required in Tutor.

A/C

Ensure that the repo's Dockerfile can create a base image which is appropriate for small production installations
Ensure that the repo's Dockerfile also defines an additional image derived from the base which adds dependencies and configuration changes needed for development environments
Check with the Tutor developers during code review to see if any further changes to the Dockerfile would help simplify Tutor
Automate uploads of both images when code changes are merged
Switch devstack to use the new Ansible-free development image, and do some basic testing to ensure it works before merging

Setting the urls and keys objects don't seem to work

I've been trying to get xqueue pull code to work, but never received submitted files. After quite some hours I've found out that setting the 'urls' and 'keys' property in the creation of the Submission object don't work. the s3_urls and s3_keys remain empty.

line 81 & 82 in lms_interface.py

How does Amazon S3 work?

Hi,

How is it possible that the current S3 code works? As far as I can tell the code for the S3 Upload
is using "create_bucket", which A) does not work B) creates a bucket when it shouldn't C) should be get_bucket. Is this just a problem that I am having or is this a feature that no one is using?

Cheers,

Woutifier

Add Django 4.2 in CI in xqueue

In order to have support for Django 4.2, we first need to run tests on Django 4.2 in the xqueue service. For updating CI, we should run tox and GitHub actions modernizers.

Tox Modernizer: tox_moderniser_django42.py

Github Action Modernizer: github_actions_modernizer_django42.py

A/C

Beta Give feedback

Fix Django 4.0 and 4.1 removal warnings
Run tests with Django 4.2
Fix all test failures with Django 4.2 (if any)
After merging, ask the owning squad to add Django 4.2 tests in branch protection rules
Options

Dependabot can't evaluate your Python dependency files

Dependabot can't evaluate your Python dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

InstallationError("Invalid requirement: 'GNU AFFERO GENERAL PUBLIC LICENSE'\n",)

You can mention @dependabot in the comments below to contact the Dependabot team.

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that satisfies the requirement futures==3.2.0 (from versions: 0.2.python3, 0.1, 0.2, 1.0, 2.0, 2.1, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6, 2.2.0, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.1.0, 3.1.1)
Traceback (most recent call last):
  File "/usr/local/.pyenv/versions/3.6.7/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/scripts/compile.py", line 191, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 198, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/resolver.py", line 284, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/local.py", line 65, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 217, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(download_dir, ireq, wheel_cache)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 183, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 218, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 164, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/local/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pip/_internal/index.py", line 621, in find_requirement
    'No matching distribution found for %s' % req
pip._internal.exceptions.DistributionNotFound: No matching distribution found for futures==3.2.0

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Restore coverage checks of xqueue

xqueue has been using the deprecated codecov package, which finally was yanked from PyPI. In order to unblock CI, we've removed the broken coverage step (#886) but this repo should have coverage restored at some point.

Most repos are using the https://github.com/codecov/codecov-action v3 action, but it's not immediately obvious how to get that to work with xqueue, which runs its tests in a docker container. (Do we need to copy a coverage.xml file out of the container? Or perhaps use a different approach entirely for getting MySQL in the testing environment?)

Test xqueue on Ubuntu 24.04

This repository is using Ubuntu 20.04 for testing. That version of Ubuntu will be out of support before Teak. Therefore this repo needs to be updated to testing with Ubuntu 24.04 before Sumac is cut to allow everyone sufficient time to switch to the new version.

Update this repository to test with Ubuntu 24.04 so that we can make the switch.

Tests are run and passing on Ubuntu 20.04 and 24.04
If major changes were needed and this is a library, a new version is published to PyPI or NPM

Note: In some cases, it may not make sense to test with both the old and the new version. For example, if the workflow is running linting or publishing to a package manager. In these cases, simply update the workflow to run on the newer version or opt to set it to ubuntu-latest instead if it doesn't matter what version it's running on. If you're unsure, reach out to the maintenance working group in #wg-maintenance in slack for guidance.

Known affected workflow files:

.github/workflows/mysql8-migrations-check.yml
.github/workflows/ci.yml

SSLv3 handshake failure

I have an activated ami in AWS (one of Bitnami's ficus 3.1) and also created an external grader using AWS Lambda function. The problem comes when I try to submit a file for grading. This error shows up:
Could not connect to server at _Lambda-URL_ in timeout=30.000000 ERROR 21427 [queue.consumer] consumer.py:440 - Submission 528 to grader _Lambda-URL_ failure: Reply: cannot connect to server
Now I changed the consumer.py file a bit to log the actual error and this is what i got:
ERROR 21427 [queue.consumer] consumer.py:196 - [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Not Found (newrelic/1.8.0.13)

https://pypi.python.org/simple/newrelic/1.8.0.13
Not Found (newrelic/1.8.0.13)

you get error:
Could not find a version that satisfies the requirement newrelic==1.8.0.13

problem file:
xqueue / requirements.txt

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches pluggy==0.6.0,==0.8.0
Tried: 0.3.0, 0.3.0, 0.3.1, 0.3.1, 0.4.0, 0.4.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.6.0, 0.6.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

[xqueue] Test Python 3.11 and 3.12

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches pluggy==0.6.0,==0.8.0
Tried: 0.3.0, 0.3.0, 0.3.1, 0.3.1, 0.4.0, 0.4.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.6.0, 0.6.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Using the XQueue interface

While it is clear to me how Xqueue works, I still do not understand how to use this Stanford example which was listed in the documentation.
I have created a "code response problem". Then i added my queue name with its url to the "XQUEUES" key in /edx/app/xqueue/xqueue.env.json. Using this i was able to run a simple passive grader.(note this is different from the above mention stanford example).
Now i wanted to know how to use the above example, which i think is an active grader.I have again added the url to /edx/app/xqueue/xqueue.env.json. now what is the login function in xqueue_util.py.
Is the url same as the queue url?
Please point me to a working example, or documentation.

openedx / xqueue Goto Github PK

xqueue's Introduction

⛔️ WARNING

xqueue

How the LMS Interacts with the XQueue

How External Graders Interact with the XQueue

Passive Graders (aka Push Graders)

Active Graders (aka Pull Graders)

Local Testing

Tests

License

How to Contribute

Reporting Security Issues

Mailing List and other support options

xqueue's People

Contributors

Stargazers

Watchers

Forkers

xqueue's Issues

Description

A/C

Recommend Projects

Recommend Topics

Recommend Org