cms-dev / cms Goto Github PK

Newlines in announcements, questions and answers to questions are getting replaced with spaces. This might not be desired when it is required to, for example, quote a part of a task statement or test data.

When I submit the same file (on the same task, using the same user) two times in a row I get a 404 error on CWS, and the following message in the console:

2012/07/16 12:33:14 - ERROR    [ContestWebServer,0] Submission local copy failed - Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/cms-0.1-py2.7.egg/cms/server/ContestWebServer.py", line 794, in post
    self.files), file_)
  File "/usr/lib64/python2.7/pickle.py", line 1370, in dump
    Pickler(file, protocol).dump(obj)
  File "/usr/lib64/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.7/pickle.py", line 562, in save_tuple
    save(element)
  File "/usr/lib64/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib64/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib64/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib64/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib64/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib64/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib64/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib64/python2.7/pickle.py", line 405, in save_reduce
    self.memoize(obj)
  File "/usr/lib64/python2.7/pickle.py", line 244, in memoize
    assert id(obj) not in self.memo
AssertionError

The problem seems to be caused by pickle, invoked here.

The format imported by YamlImporter is a blend of old conventions used in the Italian old contest system and a lot of different untidy updates. It requires a complete redefinition, possibly made in agreement with other people so that it can become a sort of standard for defining IOI-style tasks.

Unfortunately, apparently when I wrote UserTest I forgot to modify the ContestExporter machine to export it. This has to be fixed. Also ContestImporter must be fixed, of course.

The file Test.py introduced in 05504c7 and modified in the following commits is just a bad hack to support compilation and testing of solutions. It is bad in a lot of way:

• It is taken from the old Italian contest system, so it doesn't know anything of the CMS infrastructure, particularly TaskTypes and Job;
• It supports only a few TaskTypes and in particular usage cases;
• It is vastly written in Italian and very untidy.

The idea is to rewrite it in terms of TaskTypes and Jobs. Unfortunately, this will require some time and I don't think this is likely to happen in time for CMS 1.0. Therefore, this issue is planned for 1.x at the time being.

The truncation length set in 664bf00 isn't configurable and it's set only for Batch. It should be made uniform and configurable. Probably it should be passed as variable in the Job.

It could happen that, during task preparation, a testcase is incorrectly generated, i.e. it doesn't follow the requirements imposed by the task (or subtask) statement, and it needs to be generated again and substituted during the contest. Strangely, AWS makes this operation incredibly hard, because it doesn't allow to set the num field of Testcase objects, forcing admins to append this testcase at the bottom of the list (instead of inserting it in the middle). This is particularly bad when subtasks are used.

At the moment we compute the "valid phase" for the user (i.e. the phase in which he/she can compete: read statements, send submissions, etc.) without taking the extra_time field of Users into account. Then we extend it by adding extra_time to its end, even if this would make the valid phase finish after the contest end time.

This may have some odd consequences: for example, if a token is be generated exactly when the contest ends, users with a non-zero extra_time are able to use it, while all others don't.

We might want to change the meaning of extra_time to, for example, "shift" the contest start time of the user instead of extending it. This issue has been created to discuss this problem and make a decision.

The format imported by YamlImporter is a blend of old conventions used in the Italian old contest system and a lot of different untidy updates. It requires a complete redefinition, possibly made in agreement with other people so that it can become a sort of standard for defining IOI-style tasks.

In /cms/grading/TaskType.py:460 it returns a tuple of 4 values, but in /cms/grading/TaskType.py:543 it returns a tuple of only 3 values.

/cms/grading/tasktypes/Batch.py:116 expects 3 values, and thus may fail with "ValueError: too many values to unpack".

Yet, the docstring of TaskType.compilation_step reports that the method returns a tuple of 4 values.

Which one is correct? Someone more familiar with that code could have a look at it and fix it?

The documentation of AES.new states that the IV parameter "is optional and when not present it will be given a default value of all zeroes". At the moment we rely on this behavior, since we don't pass any parameter to AES.new other than the key and the mode (see /cmscommon/Cryptographics.py:90 and :102).

Yet, this commit changed this behavior, making the IV mandatory. The commit is fairly recent, so the change is only present in pycrypto-2.6 (for now...). Luckily Ubuntu 12.04 (our main target) ships with pycrypto-2.4.1.

I already reported that bug on pycrypto (since I consider it a regression) and I'm waiting for an answer. I opened this issue to keep track of this and, in case, fix our code accordingly (by passing an IV to AES.new).

Here's something I noticed while using a simple Batch task on cms rev 9f62482: when a solution finishes its miserable existence with a non-zero exit code, this is reported to the contestant as “Output isn't correct”, i.e., a Wrong Answer verdict, instead of the more appropriate “Non-Zero Exit Code” or “Runtime Error”.

I've made a simple contest (log in as test:test) which includes a simple task that requires the solution to output the smallest number out of the two given on stdin (ignore the statement in the system, I just uploaded a random one). It has two test cases: 1 10 and 42 85.

Now, have a look at some solutions:

Uses Math;
// Solution A:
// should get 2/2 correct answers if there're no problems with Math module being used
Var a, b: LongInt;
Begin
    ReadLn(a, b);
    WriteLn(min(a, b));
End.

Expected result:

• Test case 0: Output is correct
• Test case 1: Output is correct

Actual result:

• Test case 0: Output is correct
• Test case 1: Output is correct

Uses Math;
// Solution B:
// should get OK on 1st test and NZEC on 2nd
Var a, b: LongInt;
Begin
    ReadLn(a, b);
    if min(a, b) = 42 then
        Halt(201)
    else
        WriteLn(min(a, b));
End.

Expected result:

• Test case 0: Output is correct
• Test case 1: Non-Zero Exit Code / Runtime Error

Actual result:

• Test case 0: Output is correct
• Test case 1: Output isn't correct

#include <iostream>
using namespace std;

// Solution C:
// should get OK on 1st test and NZEC on 2nd

int main() {
    int a = 0;
    int b = 0;

    cin >> a >> b;

    int min = a;
    if (b < a) {
        min = b;
    }

    if (min == 42) {
        return 201;
    }

    cout << min << endl;

    return 0;
}

Expected result:

• Test case 0: Output is correct
• Test case 1: Non-Zero Exit Code / Runtime Error

Actual result:

• Test case 0: Output is correct
• Test case 1: Output isn't correct

If you use virtualenvs both for developing or delpoying is extremely useful having a requirements file to use with pip.
pip is distro independent, offers almost every python module and working with virtualenvs gives the possibility of using fixed versions of the modules.

I suggest to write a requirements file for easily setting up dependencies for CMS by just launching

pip install -r requirements/base.txt

or (for tests requirements)

pip install -r requirements/tests.txt

The ranking server should reject with a 400 status code (Bad Request) all requests that would cause its data to become inconsistent.

Inconsistencies include:

• A user with a non-existing team (fixed in 3db00d4)
• A task with a non-existing contest (fixed in 3db00d4)
• A submission with a non-existing user or task (fixed in 3db00d4)
• A subchange with a non-existing submission (fixed in 3db00d4)
• A submission outside the time-frame of its contest
• A contest with begin >= end (fixed in 52b8af7)
• Two tasks with the same order for the same contest

Since a70124c SS spawns a thread to send HTTP request to the rankings. Because logger isn't thread-safe, bee26ec makes this thread write its log to a file. This is an ugly solution. We should change it.

There are quite a lot of issues with the CMS's stylesheet:

• #global and all his children has not height: 100%
• almost every value is absolute (in pixels)
• the sidebar is not scrollable (if you have more than 15 tesks on a 13" it's impossible to click on the tasks at the bottom)

Have you considered about moving to twitter bootstrap? I think it would simplify your life a lot! For instance you could use span3 for the sidebar and span9 for the rest and you'll have a totally liquid css in a few lines of code.

At the moment the ScoreTypes are rounding the score and the max_score (both the public and the private ones) at two decimal places and, where applicable, the subtask score isn't rounded at all (we're using the %g output format).

At the IOI we were required by the rules to round the score to the nearest integer and patched our code to do so. Since we had many variable-score subtasks we also decided to round their score at two decimal places.

I think it's now time to decide what to do: keep the current rounding behavior, use the IOI one or provide some kind of configuration options for it.

The -c command line argument (-a in RS) is used to specify the contest ID. Yet, many services (all the ones I tested) don't check if a contest with this ID actually exists and fail in various ways: SS and ES fail at start, CWS starts correctly but fails on requests, etc.

I think it would be better to check the ID at start and, if no contest is found, print an appropriate error message and exit cleanly. Sure, if the contest is deleted when there are active services for it there will still be problems, but I think it's unavoidable...

Hi,

After having some problems with UpdateDB.py (add_statements failed all the time), I decided that my database is too old (it was created on 28.06.2012) and that simply dropping it and importing the contest again will be much easier. So, here's what happened:

mfag@cms-latvia:~$ cmsYamlImporter -d testcontest
Using configuration file /usr/local/etc/cms.conf.
2012/08/07 11:06:53 - INFO     [None] Creating database structure.
Traceback (most recent call last):
  File "/usr/local/bin/cmsYamlImporter", line 9, in <module>
    load_entry_point('cms==0.1', 'console_scripts', 'cmsYamlImporter')()
  File "/usr/local/lib/python2.7/dist-packages/cms-0.1-py2.7.egg/cmscontrib/YamlImporter.py", line 402, in main
    user_number=args.user_number).run()
  File "/usr/local/lib/python2.7/dist-packages/cms-0.1-py2.7.egg/cmscontrib/YamlImporter.py", line 331, in run
    self.do_import()
  File "/usr/local/lib/python2.7/dist-packages/cms-0.1-py2.7.egg/cmscontrib/YamlImporter.py", line 345, in do_import
    metadata.drop_all()
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 2542, in drop_all
    tables=tables)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2234, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1904, in _run_visitor
    **kwargs).traverse_single(element)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/visitors.py", line 86, in traverse_single
    return meth(obj, **kw)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/ddl.py", line 130, in visit_metadata
    self.traverse_single(table, drop_ok=True)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/visitors.py", line 86, in traverse_single
    return meth(obj, **kw)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/ddl.py", line 173, in visit_table
    self.connection.execute(schema.DropTable(table))
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1405, in execute
    params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1496, in _execute_ddl
    compiled
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1646, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1639, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 330, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (InternalError) cannot drop table users because other objects depend on it
DETAIL:  constraint scores_user_id_fkey on table scores depends on table users
HINT:  Use DROP ... CASCADE to drop the dependent objects too.
 '\nDROP TABLE users' {}
mfag@cms-latvia:~$

When the sandbox is invoked with system call filtering (-f), if the child process tries to call raise(), kill() or tgkill() to kill itself, the sandbox reports the signal argument ("Committed suicide by signal #") and dies, regardless of the value of the signal (which is a 32-bit integer). The signal is then displayed in the submission outcome.
A contestant can exploit this issue to reliably get pieces of the input data by using tokens, with little effort.

During IOI 2012 a private branch (named "ioi2012") was made and lots of code was committed there. Part of this code is IOI specific, part is very dirty.

The branch should be polished and pushed to the public master branch. The IOI 2012 specific code should be dealt putting it in a side branch, kept for historical reference.

This task, of course, can only be performed by HSC 2012 people.

CMS is severely lacking documentation. What is available is usually out-of-date. Particularly, TaskTypes and ScoreTypes are not documented at all.

If a student submits a question containing unicode in the subject, then when the contest administrators respond, an exception is thrown in AdminWebServer, and the administrator sees a 500 Internal Server Error. The exception is thrown only when writing a log message, so it is mostly harmless as the response still goes through, and the student is unaffected.

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/tornado/web.py", line 988, in _execute
    getattr(self, self.request.method.lower())(*args, **kwargs)
  File "./cms/server/AdminWebServer.py", line 1084, in post
    (question.user.username, question.subject))
  File "/home/cmsuser/fario12/cms/cms/__init__.py", line 438, in new_method
    return self.log(msg, operation, severities[method], timestamp)
  File "/home/cmsuser/fario12/cms/cms/__init__.py", line 410, in log
    colors=config.color_shell_log)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 114: ordinal not in range(128)

This also raises the broader question of where else unicode might leak into non-unicode-safe paths too!

See the description of commit e2358b7, quoted here for convenience:

Ignore IntegrityError when pushing a file to the database.

Such IntegrityError is probably due to the concurrent access to
the database by more than one process: by default, PostgreSQL
enforces only a partial isolation level between concurrent
transactions, so it can happen that two get_from_digest() doesn't
detect a file, but the file exists at the moment of actually
committing the transaction.

This commit requires further intervention, since we should at least
check that the IntegrityError actually comes from such a condition.

For more information about isolation levels:

http://www.postgresql.org/docs/9.1/static/transaction-iso.html

When running ContestExporter with the --drop (-d) option I get an output like this:

Using configuration file /usr/local/etc/cms.conf.
2012/10/13 18:06:29 - INFO     [None/exporting contest 3] Starting export.
2012/10/13 18:06:29 - INFO     [None/exporting contest 3] Creating dir structure.
2012/10/13 18:06:29 - INFO     [None/exporting contest 3] Exporting files.
2012/10/13 18:06:29 - INFO     [None/exporting contest 3] Exporting the contest in JSON format.
2012/10/13 18:06:29 - INFO     [None/exporting contest 3] Dumping SQL database.
2012/10/13 18:06:29 - CRITICAL [None/exporting contest 3] Database engine not supported. :-(

We could either fix this or remove this option altogether, since it's deprecated. I understand why it has been deprecated, but on the other hand I think that restoring it would act as a double security measure and would allow us to prevent issues like #38.

With respect to what you say in the box README:
https://github.com/giomasce/cms/blob/master/cms/box/README#L13

Have considered importing it as a git submodule? You can mirror it on github for reliability, but this would make easier to work on the repository and send the patches upstream.

Good work BTW :)
ot

TwoSteps and Communication tasktype uses mkfifo.
But there is a limit(maybe 64k).
plz fix this.

At the moment each RWS we want to send data to is specified in cms.conf using

• a "tuple" (well, an array) of three strings: protocol, host and port
• a username
• a password

I'm proposing a simplification: we could use just one string to specify all these parameters, in the standard format

<protocol>://<username>:<password>@<host>:<port>/

For example https://usern4me:passw0rd@localhost:8890/. This seems a nicer solution to me than what we have now.

It also allows us to specify a path prefix for the URLs we're sending data to (for example "localhost:8890/some/thing/contests/day1"). This would be useful in case RWS is hidden behind a proxy that does some URL mapping.

Anyone against it?

Computing the score, the score type assigns some numerical value also to subtasks, testcases, or whatever it uses to obtain the score. The information on how the score got computed must be returned to the user, admins and the public (maybe with different flavors) as score details for each submission.

This issue is a place to discuss where we want to stand between the two opposites: let the score type build the table of score details with all the liberty it wants, or rigidly encoding the possible information that a score type can show to the users involved.

At IOI, the former approach was used, saving into the database a simple HTML table that was manipulated and stylized by the js, but apart being an ugly solution, it weighted down the database a lot, hence it is not a sustainable option for the future.

Submissions whose compilation failed count towards the score too. Yet, SS doesn't receive notifications about them. Fix this.

The title says everything. See also #34 (comment).

In case you encounter some problem (for example #53) that forces you to reimport the contest when it's already running, you may notice that YamlReimporter will delete all the questions made by users and (I guess) some other things too (announcements, messages, etc.) but it'll keep the submissions and everything that depends on them (tokens, files, evaluations, etc.).

We need to check what exactly gets deleted and fix that. One fix could be to have YamlReimporter reassign announcements, questions, etc. to the new contest it creates, as it already does for submissions. Another possible fix would be to have it update the current DB entries, instead of deleting them and creating them again (updating what depends on them, e.g. submissions).

Note: this issue doesn't apply only to the current (re)importers but also to the ones that will come.

Decide where to get managers when requesting a UserTest. Even better, make it configurable. See bbb6bd1 and next one (behavior at IOI was changed many times).

Commit 59eb5cd introduces reading of file gen/GEN to decide the values of the subtasks. In case gen/GEN it fails with and exception. Instead, it should just fall back to Sum, giving score_type (or 1.0) to each testcase.

(particularly, it shouldn't give 0.0, like it does now if gen/GEN exists, but defines no subtasks).

When the program tries to use too much memory, the sandbox dies with message "Execution killed because of forbidden syscall time" on my system, which is not really clear for the contestant.

This is the strace output of a program trying to allocate 2gb of memory when the memory limit is set to 50 mb via ulimit:

execve("./dummy", ["./dummy"], [/* 68 vars */]) = 0
uname({sys="Linux", node="localhost", ...}) = 0
brk(0)                                  = 0x23af000
brk(0x23b01c0)                          = 0x23b01c0
arch_prctl(ARCH_SET_FS, 0x23af880)      = 0
brk(0x23d11c0)                          = 0x23d11c0
brk(0x23d2000)                          = 0x23d2000
mmap(NULL, 2147487744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
brk(0x823d1000)                         = 0x23d2000
mmap(NULL, 2147618816, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
time(NULL)                              = 1346590504
open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
read(3, "0-3\n", 8192)                  = 4
close(3)                                = 0
mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot allocate memory)
mmap(NULL, 67108864, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot allocate memory)
exit_group(0)                           = ?
+++ exited with 0 +++

Martin Mares and Bernard Blackham wrote a new sandbox, that uses more advanced Linux features instead of ptrace to control the running program. This results into lower overhead and less painful configuration (particularly, no need to maintain a syscall list that varies between CPU and distribution).

The interface of the new sandbox isn't much different from the old one, although some changes are required anyway both in Sandbox and in the TaskTypes. These changes should actually be mostly simplifications, but they can be somewhat invasive and probably need a few care.

CMS should move to this new sandbox.

This is what I got in the logs when I started SS on the day1 data of the IOI:

2012/10/06 18:07:12 - INFO     [ScoringService,0] Old submission yet to score/token: 785/3401.
2012/10/06 18:07:12 - INFO     [ScoringService,0] Old submission yet to score/token: 781/3401.
2012/10/06 18:07:13 - INFO     [ScoringService,0] Old submission yet to score/token: 777/3401.
2012/10/06 18:07:13 - INFO     [ScoringService,0] Old submission yet to score/token: 773/3401.
2012/10/06 18:07:18 - INFO     [ScoringService,0] Submissions found to score/token: 769, 3401.
2012/10/06 18:07:18 - INFO     [ScoringService,0] Old submission yet to score/token: 1538/6802.
2012/10/06 18:07:18 - INFO     [ScoringService,0] Old submission yet to score/token: 1534/6802.
2012/10/06 18:07:19 - INFO     [ScoringService,0] Old submission yet to score/token: 1530/6802.
2012/10/06 18:07:19 - INFO     [ScoringService,0] Old submission yet to score/token: 1526/6802.

It seems that SS at some point checked if there were submissions in the DB that weren't scored/tokened, found some (i.e. all those that were in the queue waiting to be scored/tokened) and put them in the queue, again!

Could someone (for example @stefano-maggiolo) check this?

When SS is asked to invalidate the scores of some submissions (through the invalidate_submission RPC method, accessible on the AWS interface) it doesn't check if the submissions it's putting in the queue have actually been compiled and evaluated. So, if one of these submissions is currently being compiled/executed or, more importantly, if its compilation failed (!), a subsequent call to ScoreType.compute_score will raise an exception and SS will die. Fix that.

See c4a4ca4.

Just upgraded to latest CMS today (rev b34dbc8) and here's what I'm experiencing: after I launch CMS with cmsResourceService 0 -a 1 it keeps restarting services, even though they work OK. It looks like this: http://f.nn.lv/ma/c1/7l/cms-keeps-restarting.png

I tried reinstalling CMS, and now it only restarts ScoringService all the time, but the other services work fine.

Is it my fault or is it a bug? I didn't really do anything except git pull'ing CMS, then building & installing it, importing a new contest and launching it.

The TYPE_EVALUATION jobs queued after a successful compilation are getting a PRIORITY_LOW assigned, while all other jobs of that type are getting a PRIORITY_MEDIUM. See here. Is it intentional or just a mistake? The one who wrote the code (i.e. @stefano-maggiolo) is the only one who can tell us... :)

YamlImporter requires a big general cleanup. It is based on long methods with lots of untidy things inside. The same tasks can be performed in a far more ordered manner.

At the moment there's a field called max_submission_length in cms.conf that globally determines the maximum size for submitted files. That is, the limit applies to all files for all tasks of all contests. Also, for example, to the files of output-only tasks, that are actually outputs and could be really large.

I suggest to remove this field and to think of (and implement) a system (i.e. a protocol of communication between TaskTypes, ES, CWS, etc... that somehow extends submission_format) that allows the TaskTypes to specify these limits on a per-task and even per-file basis. They could even be made configurable through task_type_parameters.

A special note for inputs for UserTests: TaskTypes could specify some kind of formats for these files too, to allow more flexibility (setting a different number, with different names, custom sizes, etc...).

Although better than nothing, commit c5543df is just a hack. We need to check into more detail why Unicode encoded descriptions make ContestExporter fail and fix the real bug.

Currently, cmstestsuite.ReplayContest is intended to read events from an "events file" in a custom format. There is an adapter (cmstestsuite.AdaptContest) to create this events file, but its input format is even more custom.

ReplayContest must be able to read its data (directly or indirectly) from an easily retrievable format (for example, the one created by cmsContestExporter.

We used to support translations of the task statements by storing, for each user, a list of languages he/she "understands". Then we would highlight (i.e. show on large buttons) the translations into these languages (if they existed). This approach had (mainly) two issues:

• We were using the PostgreSQL ARRAY type, making CMS PostgreSQL-specific. We don't want this.
• The language choice was global among all tasks (i.e. it was impossible to choose en_US in one task and en_UK in another).

At IOI we didn't want to have the second issue (to give more freedom to team leaders in the selection of translations) so we decided to change approach and to store, for each user, the IDs of the Statement objects that will be highlighted, as a list of integers (again with an ARRAY type).

(Note that both solutions didn't provide a way to select the translations in AWS)

For the sake of generality I think the latter approach has to be preferred. Yet, storing the Statement IDs is a very ugly solution and I think we should use a Many-to-Many relationship between User and Statement objects instead.

What do you think?

Also, again for generality, we may want to allow more than one "official" statement...

Apparently AWS messes up with the task_type_parameters when it doesn't know a task, because it tries to interpret task_type_parameters, fails and instead of leaving the same string, it writes another (probably empty). I believe that Giovanni Campagna ran into this problem during IOI 2012, maybe he can comment better.

After adding a user, ScoringService will fail to score subsequent submissions. It throws this exception instead when handling new_evaluation():

Traceback: Traceback (most recent call last):
  File "./cms/async/AsyncLibrary.py", line 586, in found_terminator
    method_response = self.service.handle_message(message)
  File "./cms/async/AsyncLibrary.py", line 413, in handle_message
    result = method(**message["__data"])
  File "./cms/service/ScoringService.py", line 425, in new_evaluation
    task=submission.task)
  File "./cms/db/SQLAlchemyAll.py", line 91, in update_ranking_view
    self.ranking_view.scores[(user.username, task.num)].score = score
KeyError: (u'testuser', 1)

Restarting ScoringService after adding users seems to be a workaround.

Currently, cmstestsuite.web.CWSRequest.SubmitRequest handles only strict programming tasks, meaning that it request only one filename, that gets posted as <task_short_name>.<language_extension>.

Apart from limiting the test capabilities of cmstestsuite, this makes impossible to replay a contest with task types different from the most boring one.

README says the following:

Once configured, we can proceed to install it using the commands:
cd $REPO/cms/ sudo pip install ./

However, there's no setup.py in $REPO/cms (I guess that's a result of the recent reorganization of the repo). I believe that cd $REPO is the correct command now.