Autograder should support different grading schemes. At least three grading schemes will be supported by default, but teachers (or admins) can create custom grading schemes.

• Default grading schemes:
  1. Pass/Fail (60 % passing level)
  2. C bias ECTS scheme (University of Stavanger)
  3. No bias ECTS scheme (NTNU)
• Each assignment can use a different grading scheme.
• A grading scheme is specified as a mapping from a set of non-overlapping intervals in the range 0-100 to a specific grade for an interval.

type GradingScheme struct {
  Name        string   // name of grading scheme
  GradePoints []uint8  // grade boundaries; from highest grade's level to 0 for the failing grade
  GradeNames  []string // names for the different grades; corresponds to the GradePoints
}

Make the following repository names per-course configurable through an Advanced course configuration panel.

// Default repository names.
const (
	InfoRepo       = "course-info"
	AssignmentRepo = "assignments"
	TestsRepo      = "tests"
	SolutionsRepo  = "solutions"
)

I don't think we need it here.

Should we do this from web.NewCourse() ? If so, should we augment the /courses endpoint currently used with a user id? That is, /courses/:uid where the uid is the teacher creating the course, who will automatically be enrolled as teacher for the course.

Does that sound reasonable?

The frontend silently ignores the fact that the server has gone down or rather the tunnel is down.

POST /courses creates a new course.
The endpoint expects a JSON payload:

{
	"name": "Course A 123",
        "code": "A123",
	"year": "2017",
	"tag": "Spring",

	"provider": "github",
	"directoryid": 5555
}

The response is TBD but will at least include the course id.

When courses are requested in a relation to a user, or users in relation to a course, the state between the user and the course should also be included in that request.

It should also be possible to change this relating through an endpoint

Trying to delete repos:

scm delete repository --all --namespace autograder-test
Are you sure you want to delete all repositories in autograder-test? (y/N): y
DELETE https://api.github.com/repos/autograder-test/course-info: 403 Must have admin rights to Repository. []

Does not work with simple access token obtained from the DB remote identities. (Not sure what is necessary to fix this. @s111 I can fix, if you give me a hint.)

As we have stopped using ?user=x and ?course=x, some XHR's on the frontend is broken, these need to be updated. The frontend also requests courses?user=0 which is incorrect.

The current format looks like this:

AssignmentID: 2
Name: "Lab1"
Language: "Java"
Deadline: "27-08-2018 12:00"
AutoApprove: false

Question: Why do we need Language ? Perhaps what we actually need is a Docker image name.

The course configuration panel needs to have some way to specify the passing level for each assignment, e.g. 85%. Maybe all assignments should have the same passing level? Do we need to distinguish this per assignment?

This table should also be temporarily populate with fake test data.

{user id | group id}, assignment id, commit hash, []score, build info, build log, 

(we should simply store all submissions)

The scm tool should be able to take an autograder database file and an id corresponding to a remote_identities table row. The access token can thus be retrivied from the database, and the access token argument isn't needed.

The <UserView> could handle all variations of user listings, with various conditionals to display needed buttons and so forth, instead of having the TeacherPage.tsx handle pendingUsers and so on.

Also the ICourseStudent should be renamed to ICourseUser and reflect also teacher state in the CourseStudentState. This will allow using a switch to display different buttons easily.

I also want to rename Admin to Course Manager to more clearly distinguish it from a system admin.

Authenticating with another provider, while logged in, should associate the remote identity with the account. Implementing #10 will probably resolve this.

I always forget to run the agctl command to set up the admin user. I would like to make that happen automatically when starting the server for the first time with an empty database. Are there any concerns with doing this s111?

If you authenticate with an already used remote identity, you should not be able to "overwrite/update" another users remote identity. Implementing #10 will probably resolve this.

Autograder should clone the tests repository to disk, and recursively scan the folder structure and extract the ag.yml files with information about the assignments. This information should be stored in the database, so that it is available to be requested from the frontend.

The frontend should use /courses/:id/assignments to request information about all assignments. See issue #2 for details on the endpoint path.

Response format for one assignment:

{
    "id": 1,
    "courseid": 1,
    "name": "Lab 1",
    "deadline": "2018-04-23T18:25:43.511Z",
    "approve": "manual | automatic",
    "programminglanguage": "Go",
}

(more information may be added to this format later.)

Currently: Make the frontend talk to the backend and basic database functions plus basic model.

Frontend issue. When more than two rows are needed, it doesn't look nice.

We've started doing /courses?user=123.
I think this should be /users/123/course, while /courses/1/users should return the list of course users.
We should do further filtering with /courses/1/users?status=[accepted|rejected|pending].
Enrolling a user should be done with PUT /courses/1/users/123, i.e. enroll user 123 in course 1.
Removing a user from a course DELETE /courses/1/users/123, etc..

I will start by updating the current /courses endpoint, and yield the remaining assigment then.

Update join course page to expect a list of courses, see: 8483d11.

POST /directories lists the organizations/groups from GitHub/GitLab that can be used to create courses.
The endpoint expects a JSON payload:

{"provider": "PROVIDER"}

PROVIDER is either github or gitlab depending on what the teacher has selected.

The response is on the format:

[
  {
    "id": 123,
    "path": "some-org",
    "avatar": "http://some-url.com/image.png"
  },
  {
    "id": 456,
    "path": "anotherorg",
    // Note: avatar URL is omitted if no avatar exists.
  }
]

Implement the /courses/:cid/users endpoint. This should be easy as the GetEnrollmentsByCourse database call already exists. You should be able to filter on status, i.e., ?status=pending. We could probably also allow multiple statuses separated by comma, i.e., ?status=rejected,accepted.

• Teacher
  • Full access to create/delete/modifiy/... repositories within some form of organization or group.
• Student
  • Only need to authenticate the user.

The new autograder will assume the following repository structure. These will be created automatically when a course is created.

PS: These repository names should be defined as variables in the frontend and backend, as appropriate, so that they can be changed for the different courses rather than being hardcoded into the code at various places. Other names that people may use include: templates, labs, homework, exercises etc.

PPS: In autograder, Teacher means any teaching staff, including teaching assistants and professors alike.

The assignments folder has a separate folder for each assignment. The short name for each assignment can be provided in the folder name, for example single-paxos or state-machine-replication. Typically, the deadline gleaned from the ag.yml file will determine the ordering of the assignments as they appear in lists in the frontend. We may provide an alternative way to order the assignments. Some courses may simply use short names, such as lab1, lab2, and so on. These will be sorted by the frontend as expected.

The username is actually the github or gitlab user name. This repository will initially be empty, and the student will need to set up a remote label called assignments pointing to the assignments repository, and pull from it to get any template code provided by the teaching staff.

The tests folder is used by autograder to run the tests for each of the assignments. The folder structure inside tests must correspond to the structure in the assignments repo. Each assignment folder in the tests repository contains one or more test file and a ag.yml configuration file that will be picked up by autograder test runner. The format of this file will be specified elsewhere, but it will describe various aspects of an assignment, such as submission deadline, approve: manual or automatic, programming language, test commands, etc.

The solutions folder should never be shared with anyone except teachers. This folder is not used by autograder, but is created as a placeholder for the teaching staff to prepare and test the assignments locally. This folder will typically be used as the source for creating the assignments folder and tests folder.

Currently, teaching staff needs to populate these repositories manually for the course. This is important so as to prevent revealing commit history from old instances. That is, these repositories should not be cloned or forked from an old version of the course.

We currently have not specified a separate repository for group assignments, as we hope to find a way to avoid a separate group repository. Once this has been decided, we will update this spec.

Create an endpoint for enrolling a student to a course

POST /enrolluser

{
    "userid": 1,
    "courseid": 1
}

UPDATE:

PUT /courses/:id/users/:userid

Opening this issue for tracking the status and discussion around the backend API.

• /logout (#5)
  • GET: Logs the user out and redirects to /.
  • Query: redirect=url.
    • Change redirect to url.
• /auth/:provider (#5)
  • /callback
    The provider will redirect to this URL after authentication.
    • GET: Finish authentication, write cookie to client and redirects to /.
    • Query: redirect=url.
      • Change redirect to url.
• /api/v1
  Accessing /api/v1 requires authentication through /auth/:provider.
  • /directories
    • POST (#9): List provider directories which the user is a member of.
  • /courses
    • POST: Create new course.
      • Add entry in database.
    • GET: List courses
      • List entries in database.
    • /:id
      • GET: List course
      • PUT: Update course
    • /:id/assignments
      • GET: List assignments for course
    • /:id/assignments/:aid
      • GET: List assignment aid for course
      • PUT: Update assignment aid for course

The user struct needs a method for retriving a ProviderAPIClient for communicating with github/gitlab/etc. The client should be created on the first method call and simply returned on further calls.

The frontend needs to know the Course.ID for the course it is working with, and so we don't actually need to include the course code in the assignment.yml file and the corresponding database record. This Course.ID should be part of the URL when working with assignments.

If we did include the course code, it could cause problems if we copy lab1 in distributed systems to lab1 in operating systems, which I have done at some point, and the forgetting to change the assignment.yml file.

When clicking back and forth between Help and New Course on the frontend, the Help text is appended at the end.

The frontend is not showing/updating the list of organizations when doing a second entry into the Create New course page. Also it doesn't update when switching between the gitlab vs github toggle buttons. Not totally sure about the exact behaviors that cause this, but it looks pretty easy to reproduce. Again I'm in Chrome on macOS.

Implement the /users/:uid/courses endpoint. This should be easy as the GetEnrollmentsByUser database call already exists. You should be able to filter on status, i.e., ?status=pending. We could probably also allow multiple statuses separated by comma, i.e., ?status=rejected,accepted.

We need to provide a mechanism to archive courses to avoid cluttering the user interface with too many courses.

Issues to be decided:

• Should a course be archived based on some pre-specified archive date for the course, or manually with an Archive button in the UI?
• Should the course be accessible from the Autograder interface after it has been archived?
• Should the SCM data remain available for the students (and teachers), assuming that we keep the metadata about the course and the students taking the course in the database itself?
• There should be a way to access the metadata about an archived course, even if the actual SCM data is unavailable.
• Should we allow reusing the same github organization across multiple years, by resetting the organization to a clean state without any repositories, teams, and users (except the original owner(s))?

There needs to be an option to log in and out somewhere in the frontend. For now you should be able to log in with either GitHub or GitLab.

• The links should be to /auth/github and /auth/gitlab, respectively.
  • On a successful login:
    • The user is redirected back to /.
    • The user is redirected to url if the ?redirect=url query parameter is set.
• User information should then be retrievable from something like /user or /me.
  • (This is not decided on or implemented. I will open a separate issue for this at a later point.)
• The user should be able to log out using /logout.
  • On logout:
    • The user is redirected back to /.
    • The user is redirected to url if the ?redirect=url query paramter is set.

If you are logged in with GitHub and try to authenticate against GitLab, GitLab should be added as an additional remote identity. As it works now, Autograder detects that you are logged in and aborts the process.

GET /courses/:id get a course by id
return a JSON object if found by id

{ 
  "id": 16,
  "name": "asd",
  "code": "asd",
  "year": 2222,
  "tag": "Spring",
  "provider": "github",
  "directoryid": 23650610
}

PUT /courses/:id updates an existing course
Expected JSON payload:

{
    "name": "C++",
    "code": "DAT100",
    "year": 2017,
    "tag": "Spring",
    "provider": "github",
    "directoryid": 23650610
  }

Endpoint returns status code 200 if updated

To make IDs easier to distinguish, we should use

type UserID uint64
type CourseID uint64
type GroupID uint64

and so on as needed.

This will require touching lots of files, so should be done in coordination with others to avoid merge conflicts.

It would be nice to have a single command in the scm tool to

• remove all repositories
• remove all teams
• remove all users
• remove all authorizations (not sure about this??)

The idea would be to clean out an organization to ready it for a new year.

@s111 Your input/opinion is requested.

The frontend should not hard code the supported providers. Instead the frontend can query the /api/v1/providers endpoint to get a list of providers.

This is for testing only, so that we all see the same data when testing the system.

I prefer it to be done in Go, e.g. reading from sql file.

• Validate provider
  • Check that provider string is valid by checking against goth.GetProvider.
• Validate user input
  • Database models could be self-validating.

This will need adding assignment.yml file to the various folder with tests.

Set up Docker image with openjdk and pull in junit using maven or something similar.

When I try to login with github it redirects me back to the front page again. Next time I try, it redirects correctly to github to ask for pw. Not sure if this is a frontend or backend redirection issue?

When not logged in, clicking New Course, then Users from the Admin Menu, the frontend generates requests to the server that are unauthorized in an infinite loop. Here is an excerpt from the server's log:

INFO[0005] Handled request                               host=meling.itest.run method=GET path=/ remote_ip=92.220.248.44 status=200 time_rfc3339="2017-08-02T21:38:09+02:00" uri=/ user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
INFO[0005] Handled request                               host=meling.itest.run method=GET path=/dist/bundle.js remote_ip=92.220.248.44 status=200 time_rfc3339="2017-08-02T21:38:09+02:00" uri=/dist/bundle.js user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
ERRO[0006] code=401, message=Unauthorized
INFO[0006] Handled request                               host=meling.itest.run method=GET path=/api/v1/user remote_ip=92.220.248.44 status=401 time_rfc3339="2017-08-02T21:38:09+02:00" uri=/api/v1/user user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
ERRO[0018] code=401, message=Unauthorized
INFO[0018] Handled request                               host=meling.itest.run method=GET path=/api/v1/providers remote_ip=92.220.248.44 status=401 time_rfc3339="2017-08-02T21:38:21+02:00" uri=/api/v1/providers user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
ERRO[0021] code=401, message=Unauthorized
INFO[0021] Handled request                               host=meling.itest.run method=GET path=/api/v1/providers remote_ip=92.220.248.44 status=401 time_rfc3339="2017-08-02T21:38:25+02:00" uri=/api/v1/providers user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
ERRO[0022] code=401, message=Unauthorized
INFO[0022] Handled request                               host=meling.itest.run method=GET path=/api/v1/users remote_ip=92.220.248.44 status=401 time_rfc3339="2017-08-02T21:38:26+02:00" uri=/api/v1/users user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
ERRO[0022] code=401, message=Unauthorized