clariah / software-quality-guidelines Goto Github PK

I think it would be hugely useful to have a Google Form-version of the criteria. They are currently formulated as a survey or questionnaire.

Having to print the document, and tick the boxes by hand is not a very sustainable model. Especially if we want to somehow analyze the results (semi) automatically.

A commenter observed the following is missing:

The analysis of computer software known as 'static program analysis' is missing. For each language and paradigm there are numbers of tools that analyse code and produce metrics that can be readily used in a quality assessment.

http://softwarequality.clariah.nl/ is down, probably caused by our to server migration.

How do we prioritize the various indicators? This is missing in the current draft and we rightly got some comments on it:

• Prioritizing: must have/should have/nice to have.

@reinierdevalk

Some of the feedback we got relates to the high amount of criteria. Here are some comments we received regarding this:

• the guidelines are very detailed, and rather seem to lean towards an aftermath assessment than guidelines to be used during the work process. For the latter, we might want to point to the document of the eScience center. (https://nlesc.gitbooks.io/guide/content/software/software_overview.html)
• To applied for assessment it would also be better to have less criteria
• is nu wel een erg lange lijst om door te lezen.

I think Jauco's "actionable steps" initiative already mitigates most of these concerns by offering a different perspective on the criteria.

All indicators are currently closed questions that are all to be answered affirmatively. Some comments we got:

• checklist is now yes/no/no available; [...] For the time being the questions are made more closed. DSA operates with more open questions, legitimate your efforts.

From my point of view, the argument for closed questions is one of easier measurability, if all questions have to be answered affirmatively it's relatively easy to compute a score for each dimension and for the whole. It would make it easy to transform into a survey form as well.

We have to strike a balance between a consumer and developer perspective. This was worded nicely in the following comment:

The guidelines are envisioned from a consumer point of view. There is nothing wrong with that, but the document lacks an explicit statement that it is envisioned from this viewpoint and there should be a justification for doing so. After all, CLARIAH is
a consumer, but also a producer of software.
Why is the document consumer centric? Indications are that the words architecture and/or design are very sparse in the document. Architecture pops up 2 times, design 3 times, one time in conjuncture with architecture. A consumer centric approach looks
at software, ‘the thing that is’; a producer centric approach looks at software as the thing that is the result of a process in which architecture and design play a prominent role.

Various indicators need to be worked out more thoroughly. Some comments we got:

• a lot of questions could be merged and also made more qualitative - more descriptions
• De volgende items moeten denk ik nog wat uitgewerkt worden en concreter worden. Nu zou ik niet weten hoe ik software kan schrijven die eraan voldoet: D4 PF1 PF2 PF3 CH1

de template bevat bijvoorbeeld alleen de kopjes.

Deze is voor @reinierdevalk (maar die kan ik nog niet assignen want hij is pas net geinvite)

We received the following comment:

This chapter lacks more recent developments in installing and deploying software on server-like machines with automated build- and install scripts. Vagrant, Ansible, Docker cannot be missed when it comes to deploying software.

Nice presentation during the CLARIAH tech day!

We had a very quick chat after the presentation. There are definitely a lot of similarities with the escience center software development guidelines:

https://nlesc.gitbooks.io/guide/content/software/software_overview.html
https://nlesc.gitbooks.io/guide/content/software/checklist.html

Might be interesting for you to have a look. Nice to see we are all working along similar lines :-).