cylc / cylc-ui Goto Github PK

The current regular expressions in the Tornado handlers miss routes to assets like /app.js. This file is created as /js/app.randomvalue.js when we build the application for production.

But after #36 , we have a development mode. In this case, the produced file does not have a /js prefix. So it ends up simply returning a 404.

See:

• cylc/cylc-flow#1320 (comment)
• cylc/cylc-flow#1320 (comment)

Suite run dbs contain all the info needed for nice summary reports and stats about suite runs, but we don't have a built-in utility to do that yet.

For accessibility, trying to cover everything in one interface might be too hard. There are certain visual elements such as buttons and icons provided by Vuetify components that do not pass the accessibility tests for WAVE tool (which appears to focus more on screen readers).

For colour blindness, there are over three types of colour blindness, and they depend on certain rules for contrast and colors.

Using themes would give us options to have things like a high contrast theme, a theme compatible for protanomaly, another one for protanopia, perhaps even one with higher font size (or that allows the user to alter the font size and store in a cookie), etc.

The Vuetify template used in the current PoC is the Vuetify Material Dashboard, by Tim Creative.

It provides several options and libraries out of the box. But I don't believe it includes themes. It is possible to have themes within a template like this, or the theme coming from a lower level library, that controls which template to use.

Not sure which approach is the best/easier to implement. So will have to start researching in the Vue communities. Will open an issue in the vuetify-material-dashboard GitHub repository too, to ask them about it.

I suspect this is yet another one of the issues that if you leave to implement later, it might be much too late and hard to implement. Causing issues with regressions, code refactoring, etc. So better to start soon-ish if possible.

Upon logging in to JupyterHub, it creates a cookie that can be used to retrieve user information.

The cookie can be used to access information about the user.

This ticket is to investigate if we are able to access the Cookie from the Vue.js application too, then query this endpoint, and use the returned information afterwards. Here's what the information returned should look like.

• Find out which calls and what data we need
• Find a way to pass the API token from the Spawner to the UI Server
• Decide how/where to store the API token in the UI Server (securely)
• Reading the Hub user profile in Vue.js Read the value of the jupyterhub-hub-login Cookie in Vue.js
• Retrieve user information using Axios HTTP client in Vue.js
• Enforce that certain routes can be accessed only when logged in

Context

It has become clear that many users run complex suites with an inordinate number of tasks for dependency visualisation, which is currently only manifest with our 'graph' (node-link) view & in such cases exhibits as an illegible muddle of intersecting edges.

Accordingly, I have been browsing the literature on visualisations for directed acyclic graphs to see if there are alternatives which are suited better to workflows containing (very) large numbers of tasks. [1] is a very good summary of different approaches.

Concept

In light of the above, I would like to tentatively propose a new 'view' i.e. visualisation mode for suites, to complement the 'graph', 'dot' & 'text' task views we will want to re-establish in improved form for our new GUI (https://github.com/cylc/cylc/issues/1873), either as part of its initial development, or as a later enhancement.

The underlying concept would be (dynamic) adjacency matrix depictions of (D)AGs, which are essentially boolean arrays for edge existence between nodes. Research shows that for layered graphs with more than ~20 nodes, matrix representations are consistently clearer, except in some cases for path-finding [2].

However, this can yet be improved on [3] with an adapted, condensed form denoted a 'quilt' [4]. "Quilts
scale much more successfully to large graphs than our other two [sorted & 'centered & sorted'] matrix depictions." [5].

Status

Obviously the details of this idea need fleshing out, however the matrix-based underlying nature of this view lends itself intuitively to NumPy functionality. I will conduct some investigation into spatial requirements, scalability with 'skip links' & complications/subtleties I may have neglected as yet.

References

1. 'The State of the Art in Visualizing Dynamic Graphs'; Beck et al. (2014)
2. 'On the readability of graphs using node-link and matrix-based representations'; Ghoniem, Fekete & Castagliola (2005)
3. 'Understanding Which Graph Depictions Are Best for Viewers'; Christensen et al. (2014)
4. 'Visualizing very large layered graphs with quilts'; Watson et al. (2008)
5. 'Matrix depictions for large layered graphs'; Watson et al. (2008)

Only have experience measuring coverage for JavaScript with lcov, Karma, qunit. No idea what's used by Mocha + chai.

Best example I found, was this comment to an open issue in one of the vue.js repositories.

node-sass and sass-loader are listed under dependencies. But I suspect they could be marked as devDep instead.

Now that the list of suites is coming from a GraphQL endpoint, it was possible to notice the delay to display the results.

Instead, initially the user sees the message that there are no suites are available. Only later when the response is parsed and the results objects created in Vue then the table is updated.

We can work on the user experience a bit more here.

I was thinking it might be useful to be able to, in graph view, do a search for a specific task. In particular, I've done a cylc graph suite.rc.processed on a particular system, curious to see the triggering around a specific task (I find this suites graph section a bit large to follow from casual inspection). But then, I had to find the task in the cylc graph display. It would be handy to have a navigate to task matching some glob or regex pattern, and you can cycle through the matches.

Is this at all feasible?

I'm not talking about a running suite, as typically they wouldn't display every single task all at once, so are a bit more manageable.

As part of #33, we won't need to have a user profile form that allows the user to submit values. The user information is now read-only, coming from the Hub.

So we can replace the form and use normal HTML elements such as <p>.

As in the i18n component.

While reading about Vue.js and auth, found an example with routes to handle these errors. Might be a good idea to ship with some basic pages too. These can be changed later.

https://github.com/websanova/vue-auth/blob/706b151a337a27b99c86516cbf42dfcd865450d0/demos/2.x/src/app.js#L75

Now that we have a working Apollo Client for GraphQL, and also have already tested with a backend implemented by @dwsutherland, we have a few more things to test, like performance, pagination, and also GraphQL Subscriptions, which can be implemented via WebSockets.

This could be used for giving a better user experience, and also alleviating the pressure on the server by replacing multiple Ajax calls.

Now cylc-jupyterhub supports pointing to any directory with an index.html to serve as the Web GUI for the Cylc UI Server.

It would be good to have a better way to code cylc-web and see the results with and without the Hub. Preferably something quick, as generating production files takes some 20 seconds that could be better spent doing something else.

Not likely we will need it. Sites may add it if necessary (i.e. add a note somewhere to the Wiki saying how to add GA... who knows)

• Description: cylc/cylc-flow#2900 (comment)
• Code: https://github.com/dwsutherland/cylc/tree/flask-gevent-graphql

We should be able to use the current Apollo Client to connect to the API. Things to consider are CORS, headers, authentication (how/where/when it would be done), interceptors (see #40 ), error handling, and also test the one mutation provided to stop suites.

The port of the Cylc process also needs to be unique, so we will need to limit the range to a single port number.

We have two material icons assets now. The second is much heavier, and was introduced while creating the login page (mainly because I didn't really know much about that part of Vuetify).

Now I found a site with the list of icons that match the first icon set. We just need to find the three replacements for the login page, remove the other library, and profit!

http://code.meta-platform.com/assets/mdi/preview.html

Pre work of #43 , or done as part of #43. We started the project with only 4% of test coverage, and now are above 30. With this service covered, we should get close to the 50's.

The @ symbol is used as alias for the root directory, and appears to be working fine. However, the IDE is not recognizing that option, marking it as an error.

Unit tests with @ are also not working.

It is much easier to have a single place with the setLoading(true) and once complted setLoading(false), rather than having to implement that in each call.

e.g. greyby/vue-spinner#19

The vue.config.js created by vue-cli some weeks ago appears to be using now a deprecated function escape. It suggests to use encodeURI or encodeURIComponent instead.

For some reason the next / previous start in the opposite state (next missing, previous displaying), though the links work as expected (i.e. click on nothing to go next, and previous is not working).

After that, both links disappear :o|

It would be helpful to add some indicator of task groups via their visualisation colour in the LED view.

Vue.js takes care of view/visualization. It means that you are free to architect the rest of your application. That is quite different from other frameworks such as React or Angular.

However, certain patterns from these other frameworks can be useful. In AngularJS, for instance, most developers are familiar with services. And when well designed, it is fairly simple to write tests mocking the services.

Without doing this way, we should still be able to create the application, run, get the right results, etc. However, writing tests without being able to mock a service, forces you to hack your API. By inserting methods that replace parts of an object.

And then eventually you end up having to spend a lot of time maintaining your main code, and also your test code. Which may become a blocker when you have to upgrade or change the framework.

This issue is to design how the components & services should play together, and then create the UserService for #33 . That service should be able to use Vuex actions to store the state. The only method necessary now is to retrieve the user profile (i.e. one user, for a given URL like /user/kinow/userprofile).

Not necessary, but it would be better if we could have unit tests designed too. IOW, have a test for the UserService, in a simple way, that can be replicated later for future services. Later we may have to update the Suite component to remove graphql dependency from Vuex.

• running unit tests #15
• running functional tests #16
• reporting coverage
• check security & static analysis issues

(Maybe even report issues to Riot some day?)

The current mock server, GraphCMS, doesn't play nicely with CORS. So when running with Node (i.e. npm run serve, where there is an express server (I think it's using express js) that handles/proxies the requests).

We have to use hash mode for routers, because if we use URL's like /suites/ the Hub redirects to /hub/suites (unless we add every route in the UI to the hub, which may not be practical).

This means that we have nobody proxying the requests to GraphCMS. In other words, they leave the browser, from the client. And the new browsers being well secure, they - thankfully - block such requests, unless servers expose the headers for CORS (which is not the case here).

This is a good opportunity to practice more GraphQL and HTTP.

• How well does GraphQL work with CORS? Should be fully transparent to GraphQL Apollo as it happens in the HTTP layer right? We just add headers, and if the server is OK with that, all good.
• Would it be good to have an extra Tornado handler to proxy requests? While this is useful, it could be misused and introduce security vulnerabilities.
• Can we guess where the GraphQL server will be? If UI server, it may be easier... if in another server (possibly a Service connected in the Hub) then we may have this issue, and have to add CORS headers to cylc workflow services too.

Probably initially with a dummy authentication. Could even copy the JupyterHub dummy authenticator.

Then we need to confirm:

• It will work with different authentication methods (e.g. OAuth, JWT, LDAP, local user PAM, dummy)
• Can show/hide content based on the user authenticated
• Allows for auditing

Creating the login form (#12) the need to validate the fields appeared. Vuetify comes with two libraries ready to be used for that:

• vee-validate
• Vuelidate

Both look good, but we need to pick one...

The vue-cli utility, when used to create a new project, offers to include e2e, or end to end testing. These tests go beyond unit tests, playing the role of functional tests. They are able to test the application similarly to Selenium.

The two options offered by the plugin are Cypress (chrome only) or Nightwatch (Selenium based). The first option is Cypress, and given the bad reputation Selenium got in the past (it was hard to maintain, hard to write tests for complex JS applications, and needed somewhat complicated setup), I think Cypress to start with is fine.

Related to #2 , after creating a basic JWT backend example, realized we still do not have a UI with the form/fields for authentication.

• An example with Material (Not clear what's the license)
• Another example with Material (Not clear what's the license)
• The page about Forms from Vuetify has some examples
• A Vuetify example (Not clear what's the license)
• A gist with a Vuetify example, alas without screen shot (Not clear what's the license)
• Another example
• And http://designify.me/tutorials/how-to-create-a-beautiful-material-design-login-form-with-html-css/ (Not clear what's the license)
• One from Colorlib (CC By 3.0, we would have to add a footer/header note about the author)

In the Vue Mastery videos, they showed how to display the progress for loading a view/component - from what I understood at least.

This could be useful, besides the other spinner/loaders in other components, to display that the system is busy loading a route.

e.g. https://scotch.io/tutorials/add-loading-indicators-to-your-vuejs-application

We can use the image node attribute to draw images within nodes in
cylc graph and the cylc gui graph view. In fact, you can already embed
an image within nodes for cylc graph in the visualization section (add:
image="/some/path/to/image.png", imagescale="both" to a task in [[node attributes]].

It would be best if we could get the image to avoid the text in the node (top right?).

Adjusting some view options in gcontrol's graph view affects other gcontrol instances - for example 'Group All Families'. This would be a problem for shared suites, and would probably be best kept on the client side rather than stored remotely.

Depends on #1872 DONE.

cylc/cylc-flow#1881 added family expand/collapse to the dot view, but only for top level families. To be consistent with the other views we should be able to click down through intermediate family levels too, if there are any.

@oliver-sanders - assigning you as this might possibly be an easy extension to your original change (if we're lucky 😬).

npm install returns the following errors ATM:

npm WARN [email protected] requires a peer of ajv@^5.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 2 packages from 3 contributors and audited 31353 packages in 8.897s
found 8 vulnerabilities (1 low, 1 moderate, 5 high, 1 critical)
  run `npm audit fix` to fix them, or `npm audit` for details

Right now we have EcmaScript 2015 or EcmaScript 6 - or ES6 - through the Vue/Babel/Webpack wizardry . Where Vue client creates the project scaffold, Babel transpiles from EcmaScript 6 to JavaScript, and Webpack puts everything together, combining files and calling plug-ins to interpret the right files.

The examples below are unrelated, used only to illustrate the syntax differences. With JavaScript we would have some code similar to this snippet from the Annif application.

Vue.component('annif-results', {
  delimiters: ["<%","%>"],
  props: ['results'],
  template: '<div>\
  <h2 class="mt-4">Results</h2>\
  <ul class="list-group list-group-flush" id="results">\
    <li class="list-group-item p-0" v-for="result in results">\
      <meter class="mr-2" v-bind:value="result.score"></meter>\
      <a v-bind:href="result.uri"><% result.label %></a>\
    </li>\
  </ul>\
</div>'
});

Instead, we have with ES6.

https://github.com/kinow/cylc-web/blob/06153cac6805782fab1404f7a052689cd63b8068/src/views/Login.vue#L64-L103

Here's what it looks like with TypeScript.

import { Component, Vue } from 'vue-property-decorator';
import { mapState } from 'vuex';
@Component({
  computed: {
    ...mapState(['url', 'html', 'git']),
  },
})
export default class Home extends Vue {
}

In Melbourne we had a discussion about which language to adopt for the project, and the idea was to use JavaScript/EcmaScript, instead of TypeScript. The rationale was that it was simpler for others not familiar with Vue.js, and single-page-applications, JS frameworks, etc, to get used and contribute to the code base.

Need something for main page, as well as for the user management, so that work on #2 can be done more easily.

e.g. https://akryum.github.io/vue-apollo/

I think it might be better to add the menu to the left sidebar. I suspect the UI will be longer, rather than wider. So users will keep scrolling for finding tasks, suites, etc.

So the navigation might be simpler with a menu. Perhaps once that can also be hidden.

Other useful features of the menu:

• Able to show/hide items depending on the user authenticated
• Use icons in the menu for distinguishing items
• Proper aria/HTML tags

The GUI needs somewhere for users to enter Jinja2 var=value pairs for use with any spawned command that takes the --set option. As of cylc/cylc-flow#745 this can be achieved for cylc run and cylc restart with a new "other options" entry in the suite start popup, but not for validation etc.

Links

• Vue.js documentation about Unit Testing
• Vue.js documentation about Unit Testing Vue Components

Choosing a unit test framework

Vue.js (and vue-cli) provide ways to integrate Vue.js with various testing frameworks like Jasmine, Karma, Mocha, Chai, Sinon, etc. NIWA has AngularJS apps tested with Karma, and Vue.js (also some other components like i18n) use Karma+Jasmine for testing with BDD. It sounds easy to set up, and not so hard to change later.

Will implement a minimal example now with Karma and Jasmine, and then check with others in the next meeting (or will start a thread somewhere before).

Just released https://medium.com/the-vue-point/vue-2-6-released-66aa6c8e785e

Time to upgrade 🎉

Notifications such as alert, error, etc, are being displayed in the browser console. But we should find an alternative to display it to the user (when appropriate).

Ref email from @trwhitcomb -
https://groups.google.com/forum/#!topic/cylc/fbPYBaquZ6o

I'm not sure how best to display this information. Another text view column might be appropriate, or the "view prerequisites and outputs" pop-up, or additional task tool-tip information...

JupyterHub has authentication, and also a thin layer of authorization in its reverse proxy. But the Cylc UI server will need to redefine parts of its authorization.

This issue is to record discussions on authorization for the web layer, which will interface with JupyterHub and with Cylc UI Server.

See related discussion in cylc/cylc: cylc/cylc-flow#1901

See related discussion in cylc/cylc-jupyterhub: cylc/cylc-uiserver#10

The gscan GUI currently displays running suites, or recently(?) stopped ones.

We could (perhaps optionally) have it display all of a users registered suites, including ones that have never been started or were stopped non-recently, or a given subset of them. This would make it a good way to see which members of a group of suites that should be running are still running, and to restart them if they're not running (perhaps by a shortcut "restart" right-click menu option in gscan).

Only text is being displayed with

<v-text-field
                ref="username"
                v-model="username"
                :rules="[() => !!username || 'This field is required']"
                :error-messages="errorMessages"
                prepend-icon="person"
                label="User name"
                placeholder="john.doe"
                required
              />

Which produces

<i aria-hidden="true" class="v-icon material-icons theme--light">person</i>

And renders

When running npm run test:unit, the following message is displayed:

 10% building 1/1 modules 0 active(node:18358) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead

Could be reminiscent from #11