Comments (7)
Hi @wetneb I now have an understanding of what is expected and how to achieve that. In case of any clarifications I will reach out
from sparqlextension.
I am not sure why @j-sal used one or the other, you could ask her directly. I think it would work with both.
from sparqlextension.
Yes. I would expect this extension not to rely in the json importer at all actually, even if it retrieves the results in json internally
from sparqlextension.
Sure. I think i get the logic
from sparqlextension.
Hi @wetneb I am working on a custom json parser but am having challenges in understanding how the parsePreview and TabularImportingParserBase.readTable methods in the existing extensions work.
- Should the raw data be passed to the initialize-parser-ui sub command?
- How is the raw data handled by the parsePreview and TabularImportingParserBase.readTable methods?
- How is the raw data then returned to the frontend for preview?
I have gone through several extensions but didnt understand how the raw data gets parsed before being displayed at the preview page
from sparqlextension.
Hi @WaltonG,
The raw data is generally read from the backend directly. That means, in your case I would expect that the SPARQL request is not done by the frontend (as is currently the case) but rather by the backend.
To get a simple example, have you looked at the CommonsExtension? It is very similar, because it also lets people create projects from an API. In particular, the ImportingController should be interesting for you. I think it should answer your second question.
I would expect the general process to look like this.
- the frontend lets the user type in a SPARQL query (in the CommonsExtension: from a list of Wikimedia Commons categories)
- the user validates the form, which calls the preview sub-command of the importing controller defined by the extension;
- inside the server (in Java), this preview command calls the SPARQL endpoint (in CommonsExtension: the MediaWiki API) to execute the query, and transforms the query results into an OpenRefine project.
- the frontend displays this previewed project to the user (by retrieving the rows from the temporary project created by the preview command)
- when the user creates the project, that triggers again a request to the server, which then performs the SPARQL query again and creates the project out of it.
The initialize-parser-ui
sub-command does not do any data processing itself, it just returns the default values for the options displayed in the preview stage.
As you can see from the CommonsExtension, the preview and project creation stage share a lot of the same code:Β the only difference between the two is the limit on the number of rows. In your case it could make sense to include this limit in the SPARQL query, if you have a safe way to do so (it might require parsing the SPARQL query).
One thing I would encourage you to get an overview of the requests that are made, is to try out the CommonsExtension with your web developer panel open, so that you can see the network requests made by the frontend to the backend (and inspect their parameters and responses).
Let me know if any of this is unclear.
from sparqlextension.
@wetneb What could be the reason behind switching from Apache HttpClient in main OpenRefine to OkHttpClient in the CommonsExtension?
from sparqlextension.
Related Issues (13)
- Linked Data Fragments importer HOT 1
- Create project from WCQS SPARQL query (special case: requires Wikimedia authentication) HOT 11
- Entity column should be reconciled HOT 10
- Color syntaxing and auto-completion
- SparqlExtension should work with OpenRefine v3.5.x
- Translations of the strings of text inside the extension HOT 2
- Bug: The Extensions menu is broken when this extension is installed. HOT 1
- The HTTP request to the Query Service should be sent from the back-end HOT 2
- Write Tests for The Available Functionalities
- Trying to query ArtsData SPARQL end-point fails HOT 5
- Documentation of features of the SPARQL Importer
- Provide a list of SPARQL Importer end-points
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sparqlextension.