Giter Club home page Giter Club logo

doubtfire-api's Introduction

Doubtfire Logo

Doubtfire API test-doubtfire-api CodeQL RuboCop

Doubtfire is a feedback-driven learning support system.

Table of Contents

Getting started

See Doubtfire Deploy for instructions on deploying, and contributing, to the Doubtfire project.

Environment variables

Doubtfire requires multiple environment variables that help define settings about the Doubtfire instance running. Whilst these will default to other values, you may want to override them in production.

Key Description Default
DF_AUTH_METHOD The authentication method you would like Doubtfire to use. Possible values are database for standard authentication with the database, ldap for LDAP, aaf for AAF Rapid Connect, or SAML2 for SAML2.0 auth. database
DF_STUDENT_WORK_DIR The directory to store uploaded student work for processing. student_work
DF_INSTITUTION_NAME The name of your institution running Doubtfire. University of Foo
DF_INSTITUTION_EMAIL_DOMAIN The email domain from which emails are sent to and from in your institution. doubtfire.com
DF_INSTITUTION_HOST The host running the Doubtfire instance. localhost:3000
DF_INSTITUTION_PRODUCT_NAME The name of the product (i.e. Doubtfire) at your institution. Doubtfire
DF_SECRET_KEY_BASE The Rails secret key. Default key provided.
DF_SECRET_KEY_ATTR The secret key to encrypt certain database fields. Default key provided.
DF_SECRET_KEY_DEVISE The secret key provided to Devise. Default key provided.
DF_SECRET_KEY_MOSS The secret key provided to Moss for plagiarism detection. This value will need to be set to run rake submission:check_plagiarism (otherwise you won't need it). You will need to register for a Moss account to use this. No default.
DF_INSTITUTION_PRIVACY A statement related to the need for students to submit their own work, and that this work may be uploaded to 3rd parties for the purpose of plagiarism detection. Default statement provided
DF_INSTITUTION_PLAGIARISM A statement clarifying the terms plagiarism and collusion. Default statement provided
DF_INSTITUTION_SETTINGS_RB The path of the institution specific settings rb code - used to map student imports from institutional exports to a format understood by Doubtfire. No default
DF_FFMPEG_PATH The path of to the ffmpeg binary for audio processing. ffmpeg
DF_REDIS_CACHE_URL The redis URL for rails used for development and production, ignored in the test env. redis://localhost:6379/0
DF_REDIS_SIDEKIQ_URL The redis URL for sidekiq. A working redis server is mandatory for sidekiq in all environments. redis://localhost:6379/1

If you have chosen to use AAF Rapid Connect authentication, then you will also need to provide the following:

Key Description Default
DF_AAF_ISSUER_URL The URL of the AFF issuer, either https://rapid.test.aaf.edu.au for testing or https://rapid.aaf.edu.au for production. https://rapid.test.aaf.edu.au
DF_AAF_AUDIENCE_URL The URL of the AAF registered application. No default - required
DF_AAF_CALLBACK_URL The secure endpoint within your application that AAF Rapid Connect should POST responses to. It must end with /api/auth/jwt to access the Doubtfire JWT authentication endpoint. No default - required
DF_AAF_UNIQUE_URL The unique URL provided by AAF Rapid Connect used for redirection out of Doubtfire. No default - required
DF_AAF_IDENTITY_PROVIDER_URL The URL of the AAF-registered identity provider. No default - required
DF_AAF_AUTH_SIGNOUT_URL The URL to redirect to on sign out in order to log out of AAF Rapid Connect. No default - required
DF_SECRET_KEY_AAF The secret used to register your application with AAF. secretsecret12345

You may choose to keep your environment variables inside a .env file using key-value pairs:

DF_INSTITUTION_HOST=doubtfire.unifoo.edu.au
DF_INSTITUTION_NAME="University of Foo"

You can also keep multiple .env files for different environments, e.g.: .env.production is different to .env.develoment. Doubtfire uses the dotenv gem to make this happen.

Get it up and running!

Once you've installed using either in install script or the manual install steps.

$ bundle exec rails s

You should see all the Doubtfire endpoints at http://localhost:3000/api/docs/, which means the API is running.

Running Rake Tasks

You can perform developer-specific tasks using rake. For a list of all tasks, execute in the root directory:

rake --tasks

Testing

Our aim with testing Doubtfire is to migrate to a Test-Driven Development strategy, testing all new models and API endpoints (although we plan on writing more tests for existing models and API endpoints). If you are writing a new API endpoint or model, we strongly suggest you include unit tests in the appropriate folders (see below).

To run unit tests, execute:

$ rake test

Unit tests are located in the test directory, where model tests are under the model subdirectory and API tests are under the api subdirectory.

Any helpers should be included in the helpers subdirectory and helper modules should be written under the TestHelpers module.

Contributing

Refer to CONTRIBUTING.md

License

Licensed under GNU Affero General Public License (AGPL) v3

doubtfire-api's People

Contributors

akashagarwal7 avatar akshatbajaj avatar alexcu avatar angusmorton avatar ankansubedi avatar apj avatar areejasiri avatar avdongle avatar conorblencowe avatar dependabot[bot] avatar guoheng-liao avatar hellola avatar hltr avatar jakerenzella avatar justfeedme avatar kkatarya1 avatar lachlanwest avatar lindenhutchinson avatar llyyer avatar macite avatar maddernd avatar ohtake avatar pavbhat avatar rayguo-ergou avatar samanast avatar satikaj avatar tancnle avatar thisaraw avatar tounyan avatar ublefo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

doubtfire-api's Issues

Failed to install Doubtfire-api

I tried both the manual method and "setup.sh" to install this api. Both give the same error as follows:

rake aborted!
NoMethodError: undefined method call' for BigDecimal:Class /home/yuyan/doubtfire-api/config/boot.rb:2:in require'
/home/yuyan/doubtfire-api/config/boot.rb:2:in <top (required)>' /home/yuyan/doubtfire-api/config/application.rb:1:in require'
/home/yuyan/doubtfire-api/config/application.rb:1:in <top (required)>' /home/yuyan/doubtfire-api/Rakefile:5:in require'
/home/yuyan/doubtfire-api/Rakefile:5:in <top (required)>' /home/yuyan/.rbenv/versions/2.6.7/bin/bundle:23:in load'
/home/yuyan/.rbenv/versions/2.6.7/bin/bundle:23:in `

'
(See full trace by running task with --trace)

I wonder if anyone knows how to solve this problem?

Update Documentation for Docker Installation

There doesn't seem to be a docker.sh script in the development or master branch needed in steps 4, 5 and 6 of docker installation documentation. I'm happy to create a pull request once I figure out the alternative.

CORS Issues

Hi, strictly this isn't an issue but I'm having so much trouble trying to get this to work and thought this was the best place to post.

I have the doubtfire-api up and running on a cloud hosted server. I can curl the ../api/docs and get a 200 response from the external computer. However, my issue is with CORS I believe and trying to connect to the website from a computer external to the server. I've tried with Chrome, Mozilla, and Edge browsers.

I added this to the /doubtfire-api/config/application.rb as an alternative to what was already there:
config.middleware.insert_before 0, "Rack::Cors" do allow do origins '*' resource '*', :headers => :any, :methods => [:get, :post, :options] end end

Neither what was there or what I changed it to above worked.

So when I try to connect to the /api/docs externally, swagger pops up and gives me this output only:

Can't read from server. It may not have the appropriate access-control-origin settings.

Any idea what I'm missing or suggestions?

Reduce messages logged to cron tasks

PDF generation logs messages to cron that result in a large number of emails with little valuable or important information - making it harder to see messages of value.

eg:

2022-03-23 08:10:04 +1100,unknown,INFO: Starting generate pdf - 8560
2022-03-23 08:10:04 +1100,unknown,INFO: creating pdf for task 511470
2022-03-23 08:10:05 +1100,unknown,INFO:   Rendered task/task_pdf.pdf.erb within layouts/application (Duration: 91.9ms | Allocations: 29321)
2022-03-23 08:10:05 +1100,unknown,INFO: Running 'lualatex -halt-on-error -shell-escape -interaction=batchmode input' in /doubtfire/tmp/rails-latex/8560--3968591390589504356 1 times...
tput: No value for $TERM and no -T specified
2022-03-23 08:10:07 +1100,unknown,INFO:   Rendered layout layouts/application.pdf.erbtex (Duration: 2102.4ms | Allocations: 30023)
2022-03-23 08:10:07 +1100,unknown,INFO: creating pdf for task 498504
2022-03-23 08:10:07 +1100,unknown,INFO:   Rendered task/task_pdf.pdf.erb within layouts/application (Duration: 63.1ms | Allocations: 10799)
2022-03-23 08:10:07 +1100,unknown,INFO: Running 'lualatex -halt-on-error -shell-escape -interaction=batchmode input' in /doubtfire/tmp/rails-latex/8560--3968591390589504356 1 times...
tput: No value for $TERM and no -T specified
2022-03-23 08:10:11 +1100,unknown,INFO:   Rendered layout layouts/application.pdf.erbtex (Duration: 3815.8ms | Allocations: 11074)
2022-03-23 08:10:11 +1100,unknown,INFO: creating pdf for task 505280
2022-03-23 08:10:12 +1100,unknown,INFO:   Rendered task/task_pdf.pdf.erb within layouts/application (Duration: 26.6ms | Allocations: 3204)
2022-03-23 08:10:12 +1100,unknown,INFO: Running 'lualatex -halt-on-error -shell-escape -interaction=batchmode input' in /doubtfire/tmp/rails-latex/8560--3968591390589504356 1 times...
tput: No value for $TERM and no -T specified
2022-03-23 08:10:15 +1100,unknown,INFO:   Rendered layout layouts/application.pdf.erbtex (Duration: 3507.4ms | Allocations: 3411)
2022-03-23 08:10:17 +1100,unknown,INFO: Ending generate pdf - 8560

It would be good to keep error and warnings being logged, but turn off info by default in the crown tasks.

Notification improvements

I've mentioned this before in doubtfire-lms/doubtfire-web#479 before I realized this was not a frontend issue.

First of all, we really need some sort of active notification mechanism for doubtfire. Having to check it all the time actively becomes very tedious if I got a lot of units on there.

The first channel should be email (doubtfire-lms/doubtfire-web#312) since students are obligated to check it and we already have functional mailer stuff set up already. I think we should:

  1. Rename the current weekly digest emails to "weekly digest" instead of "notification" in the UI. Most people don't even know what those options are for in the frontend.
  2. Change the sender email address to something like [email protected] instead of impersonating unit chairs, especially given they have no control over these emails whatsoever and it's a setting on the recipients' (students) side. See: doubtfire-lms/doubtfire-web#284
  3. Implement email notifications for comments and task status changes, perhaps do the checks every hour or so, and send a notification email of the status changes and new comments to the student if there is any. Don't tie it to status changes or comment submits, it can be spammy. An additional property of a task comment should be added (something like "notified"), and we need some logic to make sure the changes we are notifying originates from a convenor, not the student.

Other optional channels to consider:

  • Develop an MS Teams bot to allow students to opt in and have the task changes sent to them as messages on Teams (if available from the university).
  • Web Push (available in most mainstream browsers, Safari support is coming soon:tm:), it's fairly straightforward and how it works is very similar to FCM and APNs. We don't need API keys from 3rd party providers to send notifications. There is already a gem that handles all the heavy-lifting as well: https://github.com/zaru/webpush

@jakerenzella @macite What do you think?

Task submission PDFs strip out student annotations once uploaded

It appears as though uploading a PDF with annotations (from a tool such as preview on macOS) strips those annotations once a task has been submitted.

I've had a couple of tasks requested for resubmission as it appears on the tutors end that I have submitted nothing.

Figured out that this may be due to OnTrack/Doubtfire removing PDF annotations and not flattening them. I don't have Doubtfire setup locally, but after receiving a couple of requests to resubmit work, I gather this is the issue.

Take for instance the attached PDFs. A teacher may ask students to fill information in on a PDF, and upload the task.
If a student annotates on the PDF (say using the built in PDF tool "Preview" on macOS, and possibly with other tools such as Adobe, although untested), and uploads this PDF file, OnTrack/Doubtfire will remove the annotations when generating the submission PDF.

Workaround: If the student flattens/exports the annotated PDF beforehand, this issue does not persist.

I imagine this would be quite confusing though for students uploading tasks where they're editing a PDF provided by their teacher, as I would expect uploading a PDF (annotated or not) provides my tutor with a copy of the PDF that is visually similar/identical to what I see at upload time.

image

As you can see, both are PDF files. Both are visually identical when opened up in a PDF viewer. One has been flattened by the PDF viewer and works for submission purposes (_Exported.pdf), whilst the other, will have the question showing up with no answer in the submission pdf (_Annotated.pdf) causing a lot of confusion for both tutors and students alike :)

Hope this helps. If you'd like some other PDFs to try this out with, please reach out directly.

OnTrackSubmissionIssue_Annotated.pdf

OnTrackSubmissionIssue_Exported.pdf

Completed Discussion Comments don't always appear in the tutor's inbox

Once a student completed a discussion comment (DC) (the purple box), the current design is that the DC will be marked as complete, and should appear in the tutor's inbox. Once they read it, however, it leaves (once it's read it won't appear in the inbox again).

There may be issues with the current implementation not marking the comment as unread, and the tutor never seeing the response, so the task remains on to discuss (because the tutor never marks it).

The change would be to have a completed discussion comment (a student responded) to always appear in the tutor's inbox regardless if it's read or not because it's an action the tutor needs to take.

Bug when Marking as Complete with a Grade

When marking a task that requires to also assign a letter Grade, if you click 'Mark as Completed' but then click the 'X' (instead of assigning a letter Grade), it then doesn't allow you to click 'Mark as Completed' again unless you do a browser refresh, or select a different option and then go back to Mark as Completed.

Extension request in the chat

I am getting a prompt that student has asked for extension for undefined weeks, and when I hit deny it gives already assessed error
Screen Shot 2021-03-16 at 8 07 35 pm

Assessment Tasksheets do not Generate or Download as Accessible PDFs

Not sure whether this is a subjective, individual academic issue, or an actual OnTrack / Doubtfire issue. Also not sure if it's a frontend web app or backend / API issue but assuming the latter for now. FYI @jakerenzella.

Currently, the main issue appears to be that most tasksheets produce one of the following two problems: either the words in the document are rendered as a single string of characters with no whitespace, or otherwise each individual letter is rendered on its own line in a screen-reader's virtual viewer / buffer. makes for very difficult reading. The only way around this is to perform OCR on the text, which works in general but not specifically very well for things like extrapolating programming language syntax. The OCR compromise also doesn't work if documents are password protected, unless one has the password.

Options to set a per-target deadline

I have a use case where I would like to set different deadlines for a given task depending on the target grades. To maintain backwards compatibility, there could still be a "global" deadline - but then it would be nice to be able to override the deadline depending on the target grade.

Would that be an implementable feature?

Webcals don't include events when reminders are configured

@macite,

Looking at the ontrack log... I happened to notice "#<NameError: undefined local variable or method `ev_summary' for #Webcal:...>"

2a9a5e6 adds a test case that exposes this issue,

$ bundle exec rake TEST=test/models/webcal_test.rb
...

  1) Error:
WebcalTest#test_Includes_webcal_reminders_correctly:
NameError: undefined local variable or method `ev_summary' for #<Webcal:0x00005597cffeaaf8>
    app/models/webcal.rb:120:in `block (3 levels) in to_ical'
    app/models/webcal.rb:118:in `block (2 levels) in to_ical'
    app/models/webcal.rb:109:in `block in to_ical'
    app/models/webcal.rb:78:in `to_ical'
    test/models/webcal_test.rb:161:in `block (2 levels) in <class:WebcalTest>'
    test/models/webcal_test.rb:159:in `each'
    test/models/webcal_test.rb:159:in `block in <class:WebcalTest>'

7 runs, 207 assertions, 0 failures, 1 errors, 0 skips

...

UG-Ontrack Proposal - Integrating Speaker Verification

We are looking to integrate our Speaker Verification tool within the doubtfire-api repository. In our research, we have found that we will need to make changes to the discussion_comment_api.rb, discussion_comment.rb and file_helper.rb in order to be able to evaluate the audio files with speaker-verification.

This will only be applied to the replies from students, as verifying the tutor’s discussion prompt is not necessary. The reply_attachment_path can be gathered and passed through to our Speaker Verification tool.

Implementation in discussion_comment_api.rb would look similar to this:

# do we need to specify this environment variable somewhere? 
# or is there a os.environ(“SPEAKER_VERIF_PATH”) func we could use? 

if SPEAKER_VERIF_PATH	!= ”” # make sure it is unset... else 
   result  =  discussion_comment.add_reply(attached_file) 
else  
    result, audio_path = discussion_comment.add_speaker_verif_reply(attached_file) 
    # bespoke method to be added to discussion_comment.py utilizing optionParser, 
    # to send python command to command line. 

    result = discussion_comment.process_speaker_verification(current_user.id, audio_path) 

Calendar feed returns blank data

Hi, I've noticed that the calendar feed has always been returning blank data for me containing no events whatsoever. Refreshing, re-selecting, messing with the other options have absolutely no effect. This is all I get from the API:

"BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:OnTrack\r\nCALSCALE:GREGORIAN\r\nMETHOD:PUBLISH\r\nX-PUBLISHED-TTL;VALUE=DURATION:P1D\r\nREFRESH-INTERVAL;VALUE=DURATION:P1D\r\nEND:VCALENDAR\r\n"

I have tried emailing the support email within the university back in 2021 but the communication has stopped altogether shortly after and I did not hear back from them ever since. Adding the due dates in a calendar manually is extremely cumbersome, please fix this asap so I can manage my workload properly.

Extensions granted on group work don't get granted to all group members

Students cliff and astudent are enrolled in a certain unit, and are both in the same group, "A8" is a group task assigned to the relevant group set.

  • astudent requests an extension on A8, and their due date gets updated,

  • cliff however, still sees the previous deadline for A8—the extension is granted only to the student that requests it; not to all members of the group,

  • The comments for A8 lists the extension for both students,

    cliff sees,

    astudent sees,

BUG: User email regex accepting invalid emails

The email address regex for users API doesn’t seem to be working properly. I’m able to create accounts with these emails: qwertyuiop@qwe [email protected] [email protected]. I haven't tried more combinations at the moment.
The bug can be reproduced using Swagger or the follow minitest:

  def test_post_create_user_invalid_email
    pre_count = User.all.length
    user = {
        first_name: 'Akash',
        last_name: 'Agarwal',
        email: '[email protected]',
        username: 'akash',
        nickname: 'akash',
        system_role: 'Admin'
    }

    invalid_emails = %w(qwertyuiop qwertyuiop@qwe [email protected] [email protected])

    invalid_emails.each do |email|
      # Assign invalid email
      user[:email] = email

      data_to_post = {
          user: user,
          auth_token: auth_token
      }

      post_json '/api/users', data_to_post
      # Successful assertion of same length again means no record was created
      assert_equal pre_count, User.all.length
      assert_equal 500, last_response.status
    end
  end

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.