Giter Club home page Giter Club logo

flow's Introduction

Flow Build Status

A flexible data management, workflow execution, and visualization application.

See the project homepage and documentation for more details.

flow's People

Contributors

betsymcphail avatar curtislisle avatar danlamanna avatar dorukozturk avatar jeffbaumes avatar lukejharmon avatar manthey avatar zachmullen avatar zackgalbreath avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flow's Issues

editable tables

Allow users to make changes to their tabular data in the web browser after it has been uploaded.

table.rows to CSV converter fails on LifeMapper data

A multicolumn table with some complex cell contents (URLS, parenthetical lists), fails for me. The failure is during the encoding process. I hate these.. To recreate this, run "LifeMapper occurrences from Experiment Number" in the Arbor collection and enter experiment number 1530489. Browser console shows:

POST https://arbor.kitware.com/girder/api/v1/item/romanesco/table/rows.json/csv 500 (Internal Server Error) d3.js:1919
{"message": "Exception: 'ascii' codec can't encode character u'\xed' in position 22: ordinal not in range(128)\n

Better organization for analyses

A drop-down menu of arbitrary (and expanding) length does not seem like the ideal UI element to use for analysis selection. Instead, consider a pane where the user can browse / search / filter through the analyses. This new UI selection element should at a minimum display the name of the analysis and its description.

better handling of missing workflow inputs

Currently if your workflow is missing an input connection it just says "undefined" when you attempt to run it. We should probably check that the entire workflow pipeline is properly connected when the user attempts to save it.

programmatic access for power users

A common question during the hackathon was "is there an API to interact with arbor?" Apparently power users prefer to use the keyboard as opposed to the mouse. This issue mostly came up when discussing the possibility of batch processing (performing the same workflow on many different inputs).

Another related request / potential solution was the ability to download a whole workflow as a standalone script for offline use. I could see this working as long as the workflow doesn't swap back and forth between languages.

refresh button for workflow analyses

My understanding is that when a workflow is created, it makes a deep copy of its component analysis steps. So later on if one of the analyses is edited, the workflow will still be using the old version. This is a good idea for stability (someone else can't break your finely-tuned workflow) but it makes it awkward to fix problems in the underlying analytic logic. For this reason, we should implement a refresh button within the workflow editor that will copy in the latest version of a particular analysis step.

Fix client tests

Client tests are failing with the following. We may need help from the Girder team to see how they are doing consistent client testing nowadays, e.g. @jbeezley, @cpatrick.

3: /usr/local/bin/tangelo start --host 127.0.0.1 --port 50001 --root /home/travis/build/Kitware/tangelohub/app -nd --logdir . --girder-path /girder --no-config
3: ()
3: ====stdout:
3: waiting for blanket...
3: -----------------------
3: Jasmine runner starting
3: -----------------------
3: . App is running -> app had rendered
3: 
3: App is running: 1 of 1 passed.
3: F Able to create account -> account created
3: 
3:  Error: timeout: timed out after 5000 msec waiting for user to be logged in
3: 
3: Able to create account: 1 of 2 passed.
3: . Able to upload data -> data uploaded
3: {"message": "Administrator access required.", "type": "access"}
3: Unauthorized
3: F Able to upload data -> data downloaded
3: 
3:  Error: timeout: timed out after 5000 msec waiting for converted data to be stored
3: 
3: Able to upload data: 3 of 4 passed.
3: 401 Unauthorized {"message": "Administrator access required.", "type": "access"}
3: Unauthorized
3: F Able to create collection -> collection created
3: 
3:  Error: timeout: timed out after 5000 msec waiting for collection to be available
3: 
3: Able to create collection: 0 of 1 passed.
3: F Able to create analysis -> analysis created
3: 
3:  Error: timeout: timed out after 5000 msec waiting for save location to be updated
3: 
3: Able to create analysis: 0 of 1 passed.
3: F Able to delete collection -> collection deleted
3: 
3:  Error: timeout: timed out after 5000 msec waiting for confirm dialog to appear
3: 
3: Able to delete collection: 0 of 1 passed.
3: ----------------
3: Testing Finished
3: ----------------
3: 7 specs, 5 failures in 25.775s.

single column CSV files don't work?

I tried the following file saved as columnsToKeepBad.csv but it doesn't seem to parse successfully uploaded as a CSV file into tangelo-hub:

columnHeader
awesomeness
name
island

When the file is changed to this (columnsToKeep.csv), it parses correctly:

columnHeader,junk
awesomeness,0
name,1
island,2

Basic workflow support

  • Ability to assign fixed inputs
  • Show inputs/outputs in different shape or color
  • More general way to see input/output type

display errors from uploads & conversions

Similar to #42, we need to provide better feedback to users when something goes wrong during the upload step. This involves capturing output from romanesco's converters and presenting it to the user on the website.

Better handling of analyses in non-admin collections

Only administrators can successfully run analyses in a collection created by a normal (non-admin) user. We need to do the following to improve this situation:

  1. create the ability for an admin to "whitelist" this collection from the web interface.
  2. notify the user that they will not be able to successfully run analyses, and explain why.
  3. provide a "request access" button (or similar) so that the user can notify the admins that they'd like to have analysis execution enabled on their collection.

Incorrect Heatmap layout in Chrome

In heatmap, row labels flow off the screen. Because of this I also was not able to click on row labels to sort by that row. This happens in Chrome not FF for some reason.

Allow upload/download of nested-json graphs

The Data Management view in tangelohub doesn't support some I/O options for nested format trees. Tangelohub should support upload/download/renaming of nested trees. This lets us move datasets between TangeloHub instances late in the ETL process on a dataset.

Manually specify whether there are column headers

User should be able to specify this on per-file basis, or let it do auto-detection. Will require changes to Romanesco and TangeloHub UI. One option is to have formats "csv" (with header) and "csv.nohead" (without header). This may also mean TangeloHub needs to be able to specify whether Romanesco should do header detection if nothing specified by user, or to just assign the type specified by the user. Splitting out header detection (i.e. format sniffing) as a separate Romanesco operation seems like the right approach.

Fix empty list elements

When logging in, new items in collection list are empty. Similarly, inputs and outputs in newly uploaded analysis are empty.

Error when deleting last dataset in a collection

Uncaught TypeError: Cannot read property 'get' of undefined DatasetManagementView.js:192
flow.DatasetManagementView.Backbone.View.extend.updateDataset DatasetManagementView.js:192
jQuery.event.dispatch jquery.js:5095
elemData.handle jquery.js:4766
jQuery.event.trigger jquery.js:5007
(anonymous function) jquery.js:5691
jQuery.extend.each jquery.js:657
jQuery.fn.jQuery.each jquery.js:266
jQuery.fn.extend.trigger jquery.js:5690
jQuery.fn.(anonymous function) jquery.js:7562
(anonymous function) DatasetManagementView.js:96
(anonymous function) d3.js:1916
event d3.js:434
respond d3.js:1869

Enable Heatmap transpose option and change default

In heatmap, default is a transposed version of what you see in the table view. Should initialize to the table layout (rows along y, columns along x) with perhaps an option for transposing the view.

non-admin user cannot run analyses in self-made collections

I tried creating a public collection "OTLIntegration" and then uploaded a table and several analyses. I receive an immediate "Error: Unauthorized" when trying to run an analysis from the collection. Related to permissions on the Analysis girder subfolder?

Support permalinks into arbor

Right now users have no method of directly linking to an analysis or data set within arbor. We need to provide a method to create hyperlinks that drill directly down to some specific view within our web app.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.