kazarena / json-gold Goto Github PK

View Code? Open in Web Editor NEW

115.0 115.0 17.0 319 KB

A JSON-LD processor for Go

License: Apache License 2.0

Go 99.71% Makefile 0.29%

json-gold's People

Contributors

Stargazers

Watchers

Forkers

dkolbly dennwc kc1116 burntsushi verisart ciceroverneck samuell hilalisa uppercaveman vikramk9852 vlmold backwardn jonnycrunch glitchfix cleverpebble

json-gold's Issues

How to use multiple context in compact and expand functions

Releases + Tags

Hey there.

Is it possible to do releases or tags here?

Love, someone having to maintain a Gopkg.toml

Expand a compact document

Maybe I'm doing something wrong, but given the following trivial example describing a compact document, attempting to expand it:

package main

import "github.com/kazarena/json-gold/ld"

func main() {
        proc := ld.NewJsonLdProcessor()
        options := ld.NewJsonLdOptions(``)

        /*
                {
                        "@context": "http://schema.org/",
                        "@type": "Person",
                        "name": "Jane Doe",
                        "jobTitle": "Professor",
                        "telephone": "(425) 123-4567",
                        "url": "http://www.janedoe.com"
                }
        */
        doc := make(map[string]interface{})
        doc[`@context`] = `http://schema.org/`
        doc[`@type`] = `Person`
        doc[`name`] = `Jane Doe`
        doc[`jobTitle`] = `Professor`
        doc[`telephone`] = `(425) 123-4567`
        doc[`url`] = `http://www.janedoe.com`

        expanded, err := proc.Expand(doc, options)
        if err != nil {
                panic(err)
        }

        ld.PrintDocument("JSON-LD expansion succeeded", expanded)
}

I get the following result, where I expected it to succeed:

panic: interface conversion: interface is nil, not string

goroutine 1 [running]:
github.com/kazarena/json-gold/ld.(*Context).parse(0xc8200d9d80, 0x6d06c0, 0xc8200dd500, 0xc8200d4f70, 0x0, 0x0, 0x5, 0x0, 0x0)
        /storage/workspace/golang/src/github.com/kazarena/json-gold/ld/context.go:91 +0x15c4
github.com/kazarena/json-gold/ld.(*Context).Parse(0xc8200d9d80, 0x6d06c0, 0xc8200dd500, 0x8, 0x0, 0x0)
        /storage/workspace/golang/src/github.com/kazarena/json-gold/ld/context.go:61 +0x7a
github.com/kazarena/json-gold/ld.(*JsonLdApi).Expand(0xc8200d5c37, 0xc8200d9d80, 0x0, 0x0, 0x6ceec0, 0xc82000fbf0, 0x0, 0x0, 0x0, 0x0)
        /storage/workspace/golang/src/github.com/kazarena/json-gold/ld/api_expand.go:61 +0x1d7
github.com/kazarena/json-gold/ld.(*JsonLdProcessor).expand(0xc8200d5e50, 0x6ceec0, 0xc82000fbf0, 0xc820086b00, 0x0, 0x0, 0x0, 0x0, 0x0)
        /storage/workspace/golang/src/github.com/kazarena/json-gold/ld/processor.go:141 +0x6d7
github.com/kazarena/json-gold/ld.(*JsonLdProcessor).Expand(0xc8200d5e50, 0x6ceec0, 0xc82000fbf0, 0xc820086b00, 0x0, 0x0, 0x0, 0x0, 0x0)
        /storage/workspace/golang/src/github.com/kazarena/json-gold/ld/processor.go:79 +0x69
main.main()
        /storage/workspace/golang/src/ldtest/main.go:27 +0x612

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1
exit status 2

Blank nodes across graphs / RDF lists

The problem was described in json-ld/json-ld.org#357 and jsonld-java/jsonld-java#118. As JSON-goLD inherits its implementation of FromRDF algorithm from JSONLD-JAVA, this issue is still out there and as a result 3 tests fail:

fromRdf-t0020: list with node shared across graphs
fromRdf-t0021: list with node shared across graphs (same triple in different graphs)
fromRdf-t0022: list from duplicate triples

In order to claim compliance with the spec I need to reinstate these tests and fix the code. I believe following digitalbazaar/pyld@a489b50 from PyLD is a good idea.

Examples showing how to build a graph

Hi, I'm really interested in this library. Do you have any examples that demonstrate how to build up a json-ld graph? I can see the structures in there just looking for some examples. Thanks!

Force Nesting

Hi Kazarena, this may be a limitation of json-ld, but I have a json-ld object below. I generate this object by going from RDFDataset to using api.FromRDF(). In order to make it more succinct, it would be nice if instead of having an array of objects here, I could express it as a single object, with _:b1 nested inside _:b0. Do you know of any json-ld calls that will achieve that? I tried compact, but didn't work.

[
  {
    "@id": "_:b0",
    "@type": [
      "http://www.cidoc-crm.org/cidoc-crm/E15_Identifier_Assignment"
    ],
    "http://www.cidoc-crm.org/cidoc-crm/P140_assigned_attribute_to": [
      {
        "@id": "http://www.verisart.com/objects/e7822ade-0a3a-493c-8e83-16580ce8e866"
      }
    ],
    "http://www.cidoc-crm.org/cidoc-crm/P141_assigned": [
      {
        "@id": "_:b1"
      }
    ]
  },
  {
    "@id": "_:b1",
    "@type": [
      "http://www.cidoc-crm.org/cidoc-crm/E35_Title"
    ],
    "http://www.w3.org/2000/01/rdf-schema#label": [
      {
        "@language": "en",
        "@value": "Mona Lisa"
      }
    ]
  }
]

Schema Validation

It doesn't appear like there is a way to validate that a piece of conforms to a schema.

I would expect it to behave similarly to this tool and check that the provided schema(s) match the fields used: https://search.google.com/structured-data/testing-tool

Not sure how much work this would be to implement but I'm happy to lend a hand.

Frame processing of lists appears to be broken

Here's a program that reproduces the problem. It also lists a few more general issues with ld that I've come across.

/*
This program provides examples of two cases where frame fails to properly process a graph.

The example contains a list that is broken by frame. The content of the list is an array of node references to nodes that don't exist.

See below for a description of other ld package issues ...
*/
package main

import

/*
ld package issues:

    * NewJsonLdApi does not accept a JsonLdOptions parameter as it documents. Instead it appears that JsonLdOptions is given to only
    subset of JsonLdApi functions. This implies that only these functions make use of it.
    For instance, only these use a Document Loader to resolve remote context/document references.
    "NewJsonLdApi creates a new instance of JsonLdApi and initialises it with the given JsonLdOptions structure."

    * Frame does not process lists correctly. It appears it loses their content after they have been flattened and then does
    not later embed the content. Instead it results in a hanging node reference to their content.

    * The output of the Node jsonld module wraps a graph object around Frame output - this package does not.

    * It also does not do the 'empty context' compact as specified by the framing spec.

    * The spec is very unclear about how to construct the input frame and exactly what features it provides.
    The ld package doesn't provide any additional description.

*/
(
"encoding/json"
"fmt"

"github.com/kazarena/json-gold/ld"

)

/*
Canonicalize filters and transforms an unmarshalled JSON LD graph int o a consistent form for processing. This is done in three steps:

Expand removes all context if any has been included.
Frame extracts only the node(s) or value object(s) of the types specified. The outgoing edges of these nodes are
fully 'unrolled'. If the unrolled edges include a node with multiple incoming edges, a copy of the node is attached to each edge.
Compact is used to remove array 'wrappers' from singleton arrays.

The input must be unmarshalled JSON.
If only one node matches the typeFilter, it is returned; if no nodes are matched, the result is nil; otherwise an array of the matched nodes are returned.
*/
func Canonicalize(input interface{}, types []interface{}) (interface{}, error) {
var (
emptyCtx = ld.NewContext(nil, ld.NewJsonLdOptions(""))
ldapi = ld.NewJsonLdApi()
err error
frame []interface{}
obj map[string]interface{}
expanded interface{}
framed interface{}
framedArray []interface{}
compactInput interface{}
compacted interface{}
)

obj = map[string]interface{}{"@type": types}
frame = []interface{}{obj}

expanded, err = ldapi.Expand(emptyCtx, "", input)
if err != nil {
    return nil, err
}

framed, err = ldapi.Frame(expanded, frame, nil)
if err != nil {
    return nil, err
}
switch framed.(type) {
case map[string]interface{}:
    compactInput = framed
case []interface{}:
    framedArray = framed.([]interface{})
    switch len(framedArray) {
    case 0:
        return nil, nil
    case 1:
        compactInput = framedArray[0]
    default:
        compactInput = framed
    }
default:
    return nil, nil
}

compacted, err = ldapi.Compact(emptyCtx, "", compactInput, true)
if err != nil {
    return nil, err
}
return compacted, nil

}

func main() {

var (
    examples      = make([]string, 1)
    unmarshalled  interface{}
    canonicalized interface{}
    err           error
)

examples[0] = `{
    "@context": {
        "p_type": "http://tn.resilient-networks.com/policy/type/#",
        "p_prop": "http://tn.resilient-networks.com/policy/property/#"
    },
    "@id": "_:per",
    "@type": ["p_type:policy_evaluation_request"],
    "p_prop:policy": {
        "@id": "_:1",
        "@type": "p_type:policy_def",
        "p_prop:px": {
            "@id": "_:1.1",
            "p_prop:px": {
                "@id": "_:1.1",
                "@type": "p_type:phase",
                "p_prop:step_list": {
                    "@list": [{
                        "@id": "_:1.1.1",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.1.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.1.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.1.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.2",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.2.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.2.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.2.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.3",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.3.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.3.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.3.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.4",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.4.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.4.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.4.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.5",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.5.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.5.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.5.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.6",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.6.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.6.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.6.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }]
                }
            }
        }
    }
}`

for _, example := range examples {
    err = json.Unmarshal([]byte(example), &unmarshalled)
    if err != nil {
        fmt.Printf("Unmarshal Error: %v", err)
    }

    canonicalized, err = Canonicalize(unmarshalled, []interface{}{"http://tn.resilient-networks.com/policy/type/#policy_evaluation_request"})
    if err != nil {
        fmt.Printf("Canonicalize Error: %v", err)
    }
    ld.PrintDocument("Bad Graph", canonicalized)
}

}

Using JSON-LD string in toRDF

So I see the examples where you can do

    triples, err := proc.ToRDF("URL here", options)

triples, err := proc.ToRDF(map[string]interface{}, options)

What is I have a JSON-LD string? What is the process of getting the string into an interface{} format that can be used here?

triples, err := proc.ToRDF("Valid JSON-LD string", options)

It seems when I pass it a URL it's simply reading in the JSON-LD at some point, so I do not see how to get to the point where I am just passing it a JSON-LD string to work with.

Thanks!

URDNA2015 Normalization

Any chance of updating this to support URDNA2015 (http://json-ld.github.io/normalization/spec/) normalization?

How to get framing results into struct

I'm looking to use the framing aspect of this library to help me parse down schema.org JSON-LD into known chunks.

My example is at https://github.com/ESIPFed/snapHacks/blob/master/sh01-jsonldCrawl/simpleCrawler/main.go

I am using the JSON-LD at line 25
The frame defined at line 67

and it works! Two points...

The @context is huge... is this returned from the net somehow based on all of schame.org types or something?
The @graph is parsed out at line 83 and I am trying to figure out how to cast it to a []map[string]string which I think is the collection type. However, my assert fails.

Is there is a best practice for dealing with the results from the proc.Frame call at line 78?

The code is self contained, so it should download and run if you want to try it. Many of the functions are just placeholders for now.

Any guidance appreciated!

Thanks
Doug

Change of maintainer (Dec 2017)

Dear users of JSON-goLD,

We are planning to add new features to this library, including support for Linked Data Signatures, content-addressable JSON-LD, etc. In order to facilitate this development, the current maintainer of JSON-goLD (@kazarena) is going to copy this repository to the GitHub organisation of his company (@piprate) under the following terms:

@piprate team will be the maintainer of the new library (github.com/piprate/json-gold)
This repo (github.com/kazarena/json-gold) will remain available
Interfaces of the new library will be preserved, but may deviate in future versions
Licensing will not change
Past contributors will be recognised
Commit history will not be preserved in the new library
Versions of the new library will be reset

I've decided the standard GitHub repo transfer process will not work well because of Go imports. Fork isn't ideal either because the original library will receive no further updates.

Please leave your feedback here.

Frame adds an extra empty string element to some singleton sets

Here's a program that reproduces this issue.
tst.txt