Giter Club home page Giter Club logo

data-visualization-tools's Introduction

Example invocation (URL):

http://localhost/viirs/?source=bin/viirs-flaring-2013-12-10.bin

where bin/viirs-flaring-2013-12-10.bin is an (in this case relative) URL to the binary file to visualize.

Being a description of the contents of the directory viirs/ contaimned within some older eldritch directory pertaining to the documentation and expose of the ritual and repeated penetration of the earth's crust with the intent of extracting voltaile and noxious gasses and fluids that were heretofor contained within by entities whose motives are best understood to be govenered and driven by greed and worship of the cult of mithras

Directory Organization
bin/
  Binary formated files. Probably output from scripts in ruby/.
csv/
  CSV formatted files containing VIIRS data from the Suomi NPP satellite.
json/
  JSON formatted files. Probably output from script in ruby/.
ruby/
  Scripts for doing stuff with stuff.

Sample pipeline Take a CSV file, produce a JSON file.

$ cd drilling-data-tools/visualize/viirs
$ ./ruby/csv-to-json.rb -f csv/viirs-csv-2013-08-05-to-2013-09-03.csv 
$ mv viirs-csv-2013-08-05-to-2013-09-03.json json/

Take a JSON FILE, create a binary file.

  • latitidue float
  • longitude float
  • date int
  • radiantoutput float
  • temperature int
  • radiativeheat float
  • footprint float

All data must be little endian

["31.459555".to_f].pack("e")
[DateTime.strptime("2013-08-28 00:05:01+00:00", '%Y-%m-%d %H:%M:%S%z').to_time.to_i].pack("l<")

Tools and data

Tools for converting between csv, json and geojson are available in the swissarmykitchensink pip package.

Example data is available in the data branch of this repo. NOTE: Do not merge between the data branch and the master branch - they are entierly separate.

data-visualization-tools's People

Contributors

amgottsegen avatar chrisbartley avatar createlab-harry avatar gabrielo avatar hjhawkinsiv avatar lzmunch avatar pdille avatar rsargent avatar saylorsd avatar sufyanabbasi avatar yenchiah avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

data-visualization-tools's Issues

scrollwheel-to-zoom is too slow on Macs with mechanical scroll wheel

The hack in timelapse.js to slow down Mac scrolling is useful for laptop scrollpad and "magic mouse" with scrollpad on the mouse, but is way too slow for Macs with mechanical wheels. This isn't a problem with zooming UIs on similar platforms like Google Maps.

Should we upgrade to a more modern interpretation of WheelEvent honoring DOM_DELTA_PIXEL vs DOM_DELTA_LINE?

Fake data back-end

Fake data format that serves random data.
Also: Generalize TiledBinFormat to handle any format class for tiles.

Time/spatial tiling

connect time zoom to inverse spatial zoom - when zoomed out, have small time tiles, when zoomed in, send all/nearly all time in one chunk.

CLEANUP: Explain all html files

I'd like us to start the big cleanup by making a list of visualizations, and explaining what they are (for), and then deleting any that are old and not really relevant any more. here are the files that I'd like comments on (you can just make the list in this ticket):

  • ./viirs/00.html
  • ./viirs/02.html
  • ./viirs/06.html
  • ./viirs/01.html
  • ./viirs/03.html
  • ./viirs/05.html
  • ./viz.html
  • ./frac-focus.html
  • ./google-maps.html
  • ./index.html - this is the current version used by skytruth, generalized to handle any size of dataset, no hard-coded filenames or the like

Many vector layers flash brighter when transitioning levels during zoom

Parent and child tiles are sometimes drawn at the same time during zoom, when a parent is loaded and fewer than all 4 children are loaded. The parent and partial children are all drawn. For raster without transparency, the child covers the parent, for a good result. But with vectors or alpha, sometimes lines don't line up (e.g. WDPA) or things become brighter (e.g. dotmaps). Ideally we should either not draw a partial set of children when the parent is present, or draw partial parent where children don't overlap . The latter might be unworkable.

Improved Play/Pause/Time Slider

Add an enhanced time-domain control that works and looks more like the time slider in the timelapse tool with a graduated time slider and a play/pause control.

Improve performance of spinner management

Spinner management code, such as WebGLVectorTile2._removeLoadingSpinner and WebglMapTile._removeLoadingSpinner, slows image tile loading by factor of three, due to the complex dom search. It would be better to manage the spinner without use of the dom.

Data Streaming

For large data sets, it is desirable to enable display of some data while the rest of the data is still loading. Add support for processing incoming data in chunks instead of waiting for the entire data set to be downloaded.

Ability to include spreadsheets from other spreadsheets for layer definition

Proposal: implement a simple form of inclusion -- optional rows in a spreadsheet that could look something like "include https://docs.google.com/spreadsheets/d/1rCiksJv4aXi1usI0_9zdl4v5vuOfiHgMRidiDPt1WfE/edit#gid=870361385".

The idea is that if you are developing new non-public CSV or dotmap additions (e.g. davos2019), you could put only the additions and changes in your spreadsheet tab, then include the corresponding main public tab.

...

I like the simple form of inclusion idea. It seems to minimize the number of steps and reduces the chance at human error that was possible when manually copying and pasting from different spreadsheets. I’m all for it!

Using apostrophe in theme and story titles makes the mobile story viewer not work

"Randy's story" in this spreadsheet
https://docs.google.com/spreadsheets/d/1AaJtLP9Xzr4EHMNfFmBaYeqUQ3IVHjfnC_3ruaTjfLs/edit#gid=0

works for desktop:
https://earthtime.org/stories/randys_story#waypoints=1AaJtLP9Xzr4EHMNfFmBaYeqUQ3IVHjfnC_3ruaTjfLs.0

but not mobile:
https://earthtime.org/m/stories/randys_story#waypoints=1AaJtLP9Xzr4EHMNfFmBaYeqUQ3IVHjfnC_3ruaTjfLs.0

In this spreadsheet both the theme and the story have apostrophes -- not sure if that's significant.

By removing both apostrophes, the mobile started working.

Minimum marker display size at high zoom levels

Consider the case where the markers on the map represent physical features that are 1000m across. In that case, when you zoom in very far, the marker should become larger in pixel size once you reach the point where the default display size becomes smaller than the physical size in the map projection.

This could be accomplished by supplying a Minimum Size attribute that is used to compute the display size of markers. This attribute would be expressed in map projection units.

Merge json-to-bin

json-to-bin.py started out as a straight port, but has now diverged from json-to-bin.rb as it implements the new format with a json header, and json-to-bin.rb has been updated with new changes from another repo to support more options.

Could we somehow merge this? Esp, could we settle on one binary format?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.