Giter Club home page Giter Club logo

jsondiscoverer's Introduction

jsonDiscoverer

Visit the webpage of the project to play with the service.

What is this project about?

The JSON discoverer is a tool that allows you to get the implicit schema of your JSON documents. The tool also supports the discovery of composition links among the elements of the schemas of several APIs.

These are currently the main functionalities provided:

  • Simple discovery, which discovers the schema of a given set of JSON documents returned by a single service.
  • Advanced discovery, which discovers the schema from a set of JSON-based services. First, the schema of each JSON-based service is discovered (by using the simple discoverer), then the resulting schemas are composed to obtain a general one.
  • API Composer, which takes a set of API schemas, looks for composition links (i.e., common concepts or attributes) and generates a composition graph. The result is used to help developers to compose APIs. The tool currently incoporates a sequence diagram generator to visualize API compositions.

Our tool draws schema information as UML class diagrams, including concepts (i.e., classes) and their properties (i.e., attributes and associations linking the different concepts). Potential API compositions are represented by means of UML sequence diagrams showing the possible sequence of API calls.

This video summarizes the main features of the tool:

JSON Discoverer v2.1.1

The tool is inspired in the research papers:

What can you find in this repository?

So far, these are the projects (all of them developed as Eclipse plugins):

  • jsonDiscoverer. Includes the core implementation of the discoverers to get schemas (and data models) out of JSON documents. Schemas are represented as Ecore files while data models are XMI files conforming to the Ecore ones. This therefore means that the tool depends on Eclipse Modeling Framework.
  • jsonDiscoverer.coverage. EMF-generated Java Code from an Ecore model to represent coverage models. Coverage models are used to specify which parts in a global Ecore model are covered by smaller Ecore models.
  • jsonDiscoverer.docs. The javadoc documentation generated out of the classes of the other projects. Also available here.
  • jsonDiscoverer.web. Web project including (1) an angularJS-based client-side application and (2) a servlet-based server-side application to provide access to the discoverers. Servlets are basically the facade to the discoverers implemented in the jsonDiscoverer project.
  • jsonDiscoverer.tests. Includes some JUnit-based classes to test the discoverers.
  • jsonDiscoverer.examples. Includes some code examples to launch the discoverers from Java. The project also includes some generation examples for the discoverers (see folder exampleData in the project).
  • jsonDiscoverer.zoo. Some JSON documents collected from several Web APIs to test the discoverers. The project depends on EMF2GV.

Additionally, the projects use the GSON library to parse JSON documents.

Documentation

You can find further details about how the tool works in the Inner Workings section of the webpage.

The generated javadoc is also available here

Using JSONDiscoverer

You can use the tool in three different ways:

Accesing the tool website. The easiest way to use our tool is to access to the webpage of the project. Once in the landing page you can use the menus on the top bar to access to the different functionalities.

In Java. We recommend you to download all the Eclipse projects included in this GitHub project and import them in your Eclipse instalation. The project dependencies require your Eclipse to include: EMF and EMF2GV

You can directly use the Java implementation of the discoverers provided in the jsonDiscoverer and jsonDiscoverer.coverage projects. You will also find the corresponding jar files in the root of these projects (you need both jars if you want to integrate our tool in your projects). You can build the jars by using the Ant build file located at the root of the projects. The build.xml file defines a default target that will build the JAR file automatically.

To understand better how each class works you can have a look at the inner workings section of the webpage. and the javadoc documentation.

We also recomment you to have a look at the project jsonDiscoverer.examples, where you will find some Java examples to use the discoverers and the toolset.

As Web application. The web application can be deployed as a WAR container file in your Tomcat (currently tested in Tomcat 7). We provide two ways in this option: (1) using a pre-compiled WAR or (2) create your own WAR file.

The pre-compiled version of the WAR has to be configured as follows.

  1. Download the jsonDiscoverer.web project
  2. Locate the file jsonDiscoverer.war on the root of the project
  3. Open the file with any container manager (e.g., WinRAR, WinZIP, etc...)
  4. Go to WebContent/WEB-INFfolder and edit the file config.properties. Each line in this file has been commented to help you configure the tool. Please, pay special attention to those variables regarding configuration folders (e.g., workingDir) and the path to the DOT executable (e.g., dotExePath), which is required to generate the pictures.
  5. Close the WAR file in your container manager
  6. Deploy the file in your Tomcat.

If this option does not work, follow the indications provided below to build your own WAR.

  1. Download the jsonDiscoverer.web project
  2. Go to WebContent/WEB-INFfolder
  3. Open and configure the config.properties.server file. Each line in this file has been commented to help you configure the tool. Please, pay special attention to those variables regarding configuration folders (e.g., workingDir) and the path to the DOT executable (e.g., dotExePath), which is required to generate the pictures.
  4. Once you have edited the configuration, you can build the WAR by using the Ant build file located at the root of the project. The build.xml file defines a default target that will build the WAR file automatically
  5. Take the generated WAR and copy to the webapps folder of your Tomcat instalation.

Can I collaborate?

Absolutely!.You can follow the typical GitHub contribution flow to ask/contribute:

  • For bugs or feature requests you can directly create an issue.
  • For implementing new features or directly address any issue, please fork the project, perform the patch and send us a pull request.

Please refer to our contribution guidelines for more details.

You can find our governance rules in this file.

Who is behind this project?

License

This project is licensed under the EPL license. Documentation is licensed under the under CC BY 3.0.

jsondiscoverer's People

Contributors

abelgomez avatar jcabot avatar jlcanovas avatar wandertaker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jsondiscoverer's Issues

Polution in the generated schema

The current version of the JSON discoverer generates some extra annotations in the schema that can later be a problem when generating the picture.

Modal windows are double

When showing modal windows (advanced and composition discoverers) they are shown in a double window, see screenshot.

modal

Like to have a bit more specific error message

Hi

I try to visualize the following JSON, but discoverer claims that the Json has an error. The visualized well in http://jsonviewer.stack.hu/. Cannot find out, what is wrong.

[{"print": [{"t":"all", "v":[1,2,3,4], "s": [[1,2],[3,4]], "f":[1,3], "j":[1]},
{"t": "only 1", "v":[1], "s": [[1,2],[3,4]], "f":[1,3], "j":[1]},
{"t": "sopran, alt", "v":[1,2], "s":[[1,2]], "f":[1], "j":[1,2]},
{"t": "tenor, bass", "v":[3, 4], "s":[[1, 2], [3,4]], "f":[3 ], "j":[1, 3]}]},
{"legend": [10,23]},
{"note": [[[10, 50], "Folge: A A B B A", "regular"]]}
]

"Calculate" button in composer discoverer

The "Calculate" button has to be enabled/disabled according to the state of the discovery process. RIght now there is no control and an exception is shown at server-side.

Combining two classes that are not supposed to be combined

Hi, and thanks for making this utility.

I am getting an unexpected combination of two classes because one contains "type" and "coordinates" and the other contains "type", "id", and "url". You can see in the image that they are combined into a single "geometry" class (second from the left on the bottom row).

image

It is combinind these:

"geometry": {
        "type": "Point",
        "coordinates": ["lat", "long", "elev"]
    }
"identifier" : [
        {"type": "doi",
        "id": "10.1038/nature07717",
        "url": "http://dx.doi.org/10.1038/nature07717"}
        ]

Attachments don't work with big schemas/data

The current support for downloading ecore/xmi files relies on base64 encoding and therefore has limitations (depending on the browser, e.g., it seems that Mozilla admits arount 65000 characteres while IE8 admits 32KB).

This can also happens if the pictures are too big since the same encoding is used.

"Your Data" not showing first member in array elements

The "Your Data" output is treating the first member value in an element of an array as the value of that array element.

It's entirely possible that this is intentional and I just don't fully understand what it's supposed to be doing, but it seems odd.

Example:

[
    {
        "Value": [
            {
                "SchoolDay": "false",
                "ScheduleDate": "20150921",
                "CloseDay": "0"
            }
        ]
    }
]

Results in the attached. I expected something more like

+-------------------------+
| Value                   |
+-------------------------+
| SchoolDay : false       |
| ScheduleDate : 20150921 |
| CloseDay : 0            |
+-------------------------+

untitled

utf8 support

Hello. Can you include utf8 support?
example:
[{"Тест":{}}]
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.