gasparl / citapp_pc Goto Github PK

It would be useful to have a figure that shows the user interface in the paper.

Review: openjournals/joss-reviews#1179

My first reaction is that I don't know what a "Concealed Information Test" is. I do have a gist of what a reaction time is. Is there a way to better describe the work, and simplify the title to be stupidly obvious in layman's terms? I would also want to see this title in the repository description (if you are new to Github, it's the little spot of text next to the repository name that you can click to edit.)
Application is also very general. Is it a desktop application? A mobile application? A web application? Something else?

Review: openjournals/joss-reviews#1179

The installation, based on the README, is to "drag and drop the file in a browser."
As a stupid Github user I could try grabbing the link to the file and dropping it in a new
tab, and this obviously wouldn't work! I would suggest a more direct instruction that
can first direct the user to a static url to preview the experiment, and then
if the user wants to deploy on his / her own, provide complete instructions to:

1. Fork the repository to their Github account
2. Clone the forked repository
3. Customize (in some way) if necessary?
4. Either put on a local web server, or give instructions to deploy to Github pages.

Also, you could provide the user with a Dockerized version, which would let them
run a Docker container bound to port 80 to run it immediately without needing to do
cloning, etc. This would also be reproducible because you would have all versions
of the software build into containers. let me know if you need help with this, glad
to help with creating a Dockerfile and building the container with some CI (e.g., circleci)
and pushing the container to Docker Hub.

Review: openjournals/joss-reviews#1179

Hi,
well done writing this. Hard to achieve, this sort of thing.
Some comments:
-in the readme, the pic links are broken and are not shown.
-software engineers view 'automated tests' quite differently from your 'simulated human' tests (although your simulated human tests were very worthwhile developing). The engineer would use software (e.g. travis?) to run a batch of very specific tests to run upon each git push, to ensure things are not broken. These specific tests typically would test their code on a function by function basis (unit testing) and in bigger chunks of functionality (many functions). I personally use Tape when testing JS (there a many testing frameworks though; https://medium.com/javascript-scene/why-i-use-tape-instead-of-mocha-so-should-you-6aa105d8eaf4). I would advise creating some tests for the main functions in your code for peace of mind, and for future proofing.

Verbose Review

I've created issues for each comment, each referenced in openjournals/joss-reviews#1179. Here I will write more general feedback in response to the review checklist.

Repository

The source code is available on Github at the repository url. As a suggestion, I would
name the license file just LICENSE (which I see more commonly than LICENSE.txt or similar)
however if you have compelling reason to leave as is, this is probably ok.

The README is very rich with information, especially the Introduction serves well
to (better) introduce the reason to have the task than the paper itself. Since it's an
actual web based task, I would suggest to add some screen shots to the README as you
are describing the task. Currently, it's very dense in terms of text and the user
is forced to mentally visualize what it might look like.

The release version is correct, v1.0.0.

@gasparl is the only contributor, so he definitely has made reasonable contribution
to the software :) If you ever have additional contributors, you should add an AUTHORS.md file,
and a CONTRIBUTING.md file.

Functionality

My main comment here is that there needed to be better instructions for how the
user can actually deploy the file. It's fairly obvious to put the html file on a server,
but it's not obvious that the user would need to clone, put somewhere, etc. As another
suggestion, you might deploy a container so it's immediately usable (I created an issue
for this here

For the functional claims, I'll be glad to test when a preview / deployment is available.
Let's leave this (and performance evaluation) undone for now.

Documentation

I created this issue to suggest that the
statement of need could be better explained in the paper. It's not clear who the target
audience is, or why this was created in the first place. It would also be good to have
a real world example of how/when it's used. Why/when would I care to use this?

It looks like all dependencies (css and js) are provided in the repository, so we are good!

For documentation, it looks like the same content of the README is provided as a PDF but I'm not drawn to it unless I look at the files in the repository. I would add a note (and link) in the README
that says "hey! If you need to download the documentation, here it is!"

I'm not sure if this application warrants any kind of testing, but I can give
this feedback when the preview is available on Github pages.

Finally, community guildelines (how to contribute, ask for help, and the code of conduct) are all missing, I've opened an issue for that as well.

I'll now update the checklist, and we can continue conversation on each of the issues here. Looking forward to previewing / testing the web page!

In the future, it could be that someone wants to contribute to your code! The repository is missing instructions in the README.md to do this, and it's also commonly seen to have a .github folder with a CONTRIBUTING.md file. You can even find templates if you do a Google search. The example that I linked also includes a Code of Conduct, and this is also important to add. You can also add it as another file in .github, CODE_OF_CONDUCT.md. You always want to have a standard for interaction in the case that someone is behaving inappropriately in an issue or PR (it happens, if you can't believe it!)

It's also likely that someone will want to ask you a question or get help. For this, you can just have a section in the readme (Support) and then link them to your issues board here.

Review: openjournals/joss-reviews#1179

You reference an "online version" on osf, but this isn't easy enough to use or interact
with. I'd recommend you deploy the static files to Github pages for a preview
that lives with (and thus is versioned with) the codebase. I would recommend having
the demo in a "docs" folder on the master branch so you don't need to worry about updating
a separate branch (gh-pages). If you need any help / have questions please ask and I will
help you!

Review: openjournals/joss-reviews#1179

You open with:

This application implements two versions of the response time-based Concealed Informa-
tion Test (RT-CIT) deception detection method: a standard version that has been often
used in the recent years (Seymour, Seifert, Shafto, & Mosmann, 2000), and a recently
introduced enhanced version (Lukács, Kleinberg, & Verschuere, 2017)

referencing "the response time-based Concealed Information Test" as if I know what you
are talking about. I have no idea. I would suggest to rewrite the summary for the user
that doesn't know what you are talking about. For example, something along an outline like:

1. It's important to study how people recognize items (because...)
2. Thus they developed this task called the Concealed.... (references)
3. The task is great, but there are still pressing problems that (it's not reproducible, hard to run, etc.)
4. Thus, to solve these issues, we created this! :)

I would rewrite the summary in that context - you can still provide some of the more
"academic paper" sounding details of the task, but remember that the paper here is
intended for a general audience, and not a niche psychology community.

Review: openjournals/joss-reviews#1179

Make sure to add tags and a description to your repository so it's better indexed on Github

Review: openjournals/joss-reviews#1179