soleinas / morph-kgc Goto Github PK

Morph-KGC is an engine that constructs RDF and RDF-star knowledge graphs from heterogeneous data sources with the R2RML, RML and RML-star mapping languages. Morph-KGC is built on top of pandas and it leverages mapping partitions to significantly reduce execution times and memory consumption for large data sources.

Citing Morph-KGC: If you used Morph-KGC in your work, please cite the SWJ paper:

@article{arenas2022morph,
  title   = {{Morph-KGC: Scalable Knowledge Graph Materialization with Mapping Partitions}},
  author  = {Arenas-Guerrero, Julián and Chaves-Fraga, David and Toledo, Jhon and Pérez, María S. and Corcho, Oscar},
  journal = {Semantic Web},
  year    = {2022},
  url     = {http://www.semantic-web-journal.net/system/files/swj3135.pdf}
}

• Supports R2RML, RML and RML-star mapping languages.
• Input data formats:
  • Relational databases: MySQL, PostgreSQL, Oracle, Microsoft SQL Server, MariaDB, SQLite.
  • Tabular files: CSV, TSV, Excel, Parquet, Feather, ORC, Stata, SAS, SPSS, ODS.
  • Hierarchical files: JSON, XML.
• Output RDF and RDF-star serializations: N-Triples, N-Triples-star, N-Quads, N-Quads-star.
• Integration with RDFLib and Oxigraph.
• Remote data files and mapping files.
• Runs on Linux, Windows and macOS systems.
• Compatible with Python 3.7 or higher.
• Optimized to materialize large knowledge graphs.

Read the documentation.

Learn quickly with the tutorial in Google Colaboratory!

PyPi is the fastest way to install Morph-KGC:

pip install morph-kgc

We recommend to use virtual environments to install Morph-KGC.

To run the engine via command line you just need to execute the following:

python3 -m morph_kgc config.ini

Check the documentation to can see how to generate the configuration INI file. Here you can also see an example INI file.

It is also possible to run Morph-KGC as a library with RDFLib and Oxigraph:

import morph_kgc

# generate the triples and load them to an RDFLib graph
g_rdflib = morph_kgc.materialize('/path/to/config.ini')
# work with the RDFLib graph
q_res = g_rdflib.query(' SELECT DISTINCT ?classes WHERE { ?s a ?classes } ')

# generate the triples and load them to Oxigraph
g_oxigraph = morph_kgc.materialize_oxigraph('/path/to/config.ini')
# work with Oxigraph
q_res = graph.query(' SELECT DISTINCT ?classes WHERE { ?s a ?classes } ')

# the methods above also accept the config as a string
config = """
            [DataSource1]
            mappings: /path/to/mapping/mapping_file.rml.ttl
            db_url: mysql+pymysql://user:password@localhost:3306/db_name
         """
g_rdflib = morph_kgc.materialize(config)

Morph-KGC is available under the permissive Apache License 2.0.

• Julián Arenas-Guerrero - [email protected]

Ontology Engineering Group, Universidad Politécnica de Madrid.

See the full list of contributors here.