rhizome-ai / apache-age-python Goto Github PK
View Code? Open in Web Editor NEWPython driver for Apache AGE
License: Apache License 2.0
apache-age-python's People
Contributors
Stargazers
Watchers
apache-age-python's Issues
how do i perform a json dump?
I tried querying with to_json, json_build_object or row_jo_json but had an error of function not available
edge queries are not executed in a loop.
When i tried running the following query in the loop, I was able to insert the graphs, but I am not able to create the edges.
here is the code below.
The purpose is to traverse the dom tree and save their properties which can be used for fast modifications when using a templating engine.
Can't the edges be connected based on the ID of the vertices?
import age
from age.gen.ageParser import *
GRAPH_NAME = "text_test"
DSN = "host=localhost port=5432 dbname=texttest user=afidegnum password=mypass"
ag = age.connect(graph=GRAPH_NAME, dsn=DSN)
print(ag)
bare = "/home/afidegnum/projects/prototypes/apps/mydemo/resources/bare.html"
from itertools import tee
from lxml import html
tree = html.parse(bare)
lc = [x for x in tree.getiterator()]
def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = tee(iterable)
next(b, None)
return zip(a, b)
for z, w in pairwise(lc):
cur = ag.execCypher("CREATE (n:Node {name: %s}) RETURN n", params=(z.tag,))
sor = ag.execCypher("CREATE (n:Node {name: %s}) RETURN n", params=(w.tag,))
b = [x[0].id for x in cur]
c = [x[0].id for x in sor]
ag.execCypher("MATCH (c:Person), (p:Person) WHERE c.id = %s AND p.id = %s CREATE (a)-[r:connects]->(b)", params=(b[0], c[0]))
cursor = ag.execCypher("MATCH p=()-[:connects]->() RETURN p")
for row in cursor:
print(row)
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8"/>
<title>Document</title>
</head>
<body>
<ul class="menu">
<div class="itm">home</div>
<div class="itm">About us</div>
<div class="itm">Contact us</div>
</ul>
<div id="idone" class="classone">
<li class="item1">First</li>
<li class="item2">Second</li>
<li class="item3">Third</li>
<div id="innerone"><h1>This Title</h1></div>
<div id="innertwo"><h2>Subheads</h2></div>
</div>
<div id="second" class="below">
<div class="inner">
<h1>welcome</h1>
<h1>another</h1>
<h2>third</h2>
</div>
</div>
</body>
</html>
Traversal does not provide a nested loop
The code I developed so far, closes the html and other tags prematurely. I need to be able to parse the trees from the graph database and write a nested html code with it. Can you please give a hint on how do i proceed?
def graph_dom(t_id):
parent = ag.execCypher("MATCH (n:node) WHERE id(n) = %s RETURN n", params=(t_id,))
p = [x[0] for x in parent]
pp = p[0]["tag"]
parent_tag = soup.new_tag(name=p[0]["tag"])
soup.append(parent_tag)
# d = None
children = ag.execCypher("MATCH (v:node)-[R:connect]->(V2) WHERE id(v) = %s RETURN V2", params=(p[0].id,))
print(f"Parent ----------{pp}")
# print(f"Children: {children}")
for d in children:
check_parent = ag.execCypher("MATCH (v:node)<-[R:connect]-(V2) WHERE id(v) = %s RETURN V2", params=(d[0].id,))
cp = [c[0] for c in check_parent]
# print(f"{pp}---------------------")
print(p[0].id, cp[0].id)
if (p[0].id == cp[0].id):
children_tag = soup.new_tag(name=d[0]["tag"])
parent_tag.append(children_tag)
print(d[0]["tag"])
graph_dom(d[0].id)
graph_dom(t[0])
file_soup = soup.prettify()
with open("helloworld.html", "w") as file:
file.write(str(file_soup))
############################################### HTML
<html>
<head>
</head>
<body>
</body>
</html>
<head>
<meta/>
<title>
</title>
</head>
<meta/>
<title>
</title>
<body>
<ul>
</ul>
<div>
</div>
<div>
</div>
</body>
<ul>
<div>
</div>
<div>
</div>
<div>
</div>
</ul>
<div>
</div>
<div>
</div>
<div>
</div>
<div>
<ul>
</ul>
<div>
</div>
<div>
</div>
</div>
<ul>
<li>
</li>
<li>
</li>
<li>
</li>
</ul>
<li>
</li>
<li>
</li>
<li>
</li>
<div>
<h1>
</h1>
</div>
<h1>
</h1>
<div>
<h2>
</h2>
</div>
<h2>
</h2>
<div>
<div>
</div>
</div>
<div>
<h1>
</h1>
<h1>
</h1>
<h2>
</h2>
</div>
<h1>
</h1>
<h1>
</h1>
<h2>
</h2>
Composite data types escaped with few errors
I had an error of not able to insert a dictonary data, I had to pass the value with json_dumps which got escaped.
{'id': 'second', 'class': 'below'} # this gave an error
{"id": "second", "class": "below"} # this got escaped. to {\"id\": \"second\", \"class\": \"below\"}
how do i handle composite data with age-python?
Any way to perform bulk inserts and upserts?
Coming from Neo4J cypher's implementation, I could do the following:
q = '''
UNWIND $rows as row
MATCH (g_child:Group {uid: row.child_uid})
MATCH (g_parent:Group {uid: row.parent_uid})
MERGE (g_child)-[:CHILD_GROUP_OF]->(g_parent)
RETURN count(*) as total
'''
and then
res = neo4j.query(q, parameters = {'rows': r})
This would take a list of dictionaries as rows and unwind them on the server, lighting fast.
Is there any similar way of doing this with AGE? I've looked through documentation and the source code and it doesn't seem to be an option. The closest one is use
with ag.connection.cursor() as cursor:
and perform multiple single calls within that cursor, committing at the end.
This is quite slow on large volumes of data (testing on 500K vertices).
Separately, what is the suggested way to UPSERT the data (MERGE in Neo4j implementation)? Or do we need to manually verify for existence/duplicate before creating?
not all arguments converted during string formatting
I'm greeted with the following error where I am finding it hard to trace. I'm suspecting it's from the recursive function. the same error appears in the previous example you gave #7
and the recursion kept printing the same result over and over
def node_list(node):
print(node.tag)
for child in node.iterchildren():
print(child)
node_list(node)
for n in dom_tree.iter():
node_list(n)
``
html
<Element head at 0x7f87a057bf40>
<Element body at 0x7f87a05656d0>
html
<Element head at 0x7f87a057bf40>
<Element body at 0x7f87a05656d0>
html
<Element head at 0x7f87a057bf40>
<Element body at 0x7f87a05656d0>
html
<Element head at 0x7f87a057bf40>
<Element body at 0x7f87a05656d0>
html
here is the complete log report
TypeError Traceback (most recent call last)
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(conn, graphName, cypherStmt, cols, params)
106 try:
--> 107 cursor.execute(stmt, params)
108 return cursor
TypeError: not all arguments converted during string formatting
During handling of the above exception, another exception occurred:
SqlExcutionError Traceback (most recent call last)
/tmp/ipykernel_63525/1068908748.py in
1 for n in dom_tree.iter():
----> 2 node_pick(n)
/tmp/ipykernel_63525/4168182535.py in node_pick(node)
1 def node_pick(node):
2 # parent = node
----> 3 cursor = ag.execCypher("CREATE (t:node {tag: %s} ) RETURN id(t)", params=(node.tag))
4 eid = cursor.fetchone()[0] # get last inserted ID
5 ag.commit()
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(self, cypherStmt, cols, params)
156
157 def execCypher(self, cypherStmt:str, cols:list=None, params:tuple=None) -> ext.cursor :
--> 158 return execCypher(self.connection, self.graphName, cypherStmt, cols=cols, params=params)
159
160 def cypher(self, cursor:ext.cursor, cypherStmt:str, cols:list=None, params:tuple=None) -> ext.cursor :
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(conn, graphName, cypherStmt, cols, params)
112 except Exception as cause:
113 conn.rollback()
--> 114 raise SqlExcutionError("Excution ERR[" + str(cause) +"](" + stmt +")", cause)
115
116
SqlExcutionError: ("Excution ERR[not all arguments converted during string formatting](SELECT * from cypher('text_test', $$ CREATE (t:node {tag: %s} ) RETURN id(t) $$) as (v agtype);)", TypeError('not all arguments converted during string formatting'))
import json
from lxml import etree
from itertools import tee
from lxml import html
bare = "/home/afidegnum/projects/prototypes/apps/mydemo/resources/bare.html"
dom_tree = html.parse(bare)
import age
from age.gen.ageParser import *
GRAPH_NAME = "text_test"
DSN = "host=localhost port=5432 dbname=texttest user=mydemo password=demo123"
ag = age.connect(graph=GRAPH_NAME, dsn=DSN)
def node_pick(node):
# parent = node
cursor = ag.execCypher("CREATE (t:node {tag: %s} ) RETURN id(t)", params=(node.tag))
eid = cursor.fetchone()[0] # get last inserted ID
ag.commit()
for child in node.iterchildren():
cursor = ag.execCypher("CREATE (t:node {tag: %s} ) RETURN id(t)", params=(node.tag))
xid = cursor.fetchone()[0] # get last inserted ID
ag.commit()
ag.execCypher("MATCH (c:node {id: %s}), (p:node {id: %s}) CREATE (c)-[r:childOf}]->(p)", params=(eid, xid))
ag.commit()
node_pick(node)
for n in dom_tree.iter():
node_pick(n)
the demo html
######### bare.html
<!doctype html>
home
About us
Contact us
This Title
Subheads
welcome
another
third
Issue passing the IDs to the dom trees evenly,
Hi, I have reopened the question again, on SO, what happened after the example you showed me, the traversal wasn't properly represented. I have only added the cypher so users from various graph databases can also participate. Here is the new question:
sample code setGraph
ag = age.setGraph(GRAPH_NAME)
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'age' has no attribute 'setGraph'
Did you mean:
ag = age.Graph(GRAPH_NAME)
Update the documentation
There are lots of broken modules while trying the notebook examples.
Excution ERR['int' object does not support indexing]
I'm trying to run a recursive query that will enable me to process the nodes which can be used for additional operations, and came across the error above:
when I tested individual queries, they works perfectly
def graph_nodes(tag_id):
top = ag.execCypher("MATCH (n:node) WHERE id(n) = %s RETURN n", params=(tag_id,))
x = [r[0] for r in top]
print(x[0]["tag"])
children = ag.execCypher("MATCH (v:node)-[R:connect]-(V2) WHERE id(v) = %s RETURN V2", params=(x[0]).id,)
for c in children:
print(c)
graph_nodes(c[0].id)
cursor = ag.execCypher("MATCH (n:node {tag: 'html'}) RETURN n")
t = [x[0].id for x in cursor]
print(t[0])
graph_nodes(t[0])
TypeError Traceback (most recent call last)
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(conn, graphName, cypherStmt, cols, params)
106 try:
--> 107 cursor.execute(stmt, params)
108 return cursor
TypeError: 'int' object does not support indexing
During handling of the above exception, another exception occurred:
SqlExcutionError Traceback (most recent call last)
/tmp/ipykernel_117955/2082054997.py in <module>
11 t = [x[0].id for x in cursor]
12 print(t[0])
---> 13 graph_nodes(t[0])
/tmp/ipykernel_117955/2082054997.py in graph_nodes(tags)
3 x = [r[0] for r in top]
4 print(x[0]["tag"])
----> 5 children = ag.execCypher("MATCH (v:node)-[R:connect]-(V2) WHERE id(v) = %s RETURN V2", params=(x[0].id,)
6 for c in children:
7 print(c)
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(self, cypherStmt, cols, params)
156
157 def execCypher(self, cypherStmt:str, cols:list=None, params:tuple=None) -> ext.cursor :
--> 158 return execCypher(self.connection, self.graphName, cypherStmt, cols=cols, params=params)
159
160 def cypher(self, cursor:ext.cursor, cypherStmt:str, cols:list=None, params:tuple=None) -> ext.cursor :
~/.virtualenvs/orgs/lib/python3.9/site-packages/age/age.py in execCypher(conn, graphName, cypherStmt, cols, params)
112 except Exception as cause:
113 conn.rollback()
--> 114 raise SqlExcutionError("Excution ERR[" + str(cause) +"](" + stmt +")", cause)
115
116
SqlExcutionError: ("Excution ERR['int' object does not support indexing](SELECT * from cypher('text_test', $$ MATCH (v:node)-[R:connect]-(V2) WHERE id(v) = %s RETURN V2 $$) as (v agtype);)", TypeError("'int' object does not support indexing"))
import age report an error
when I execute command：
sudo apt-get update
sudo apt-get install python3-dev libpq-dev
pip install --no-binary :all: psycopg2
pip install antlr4-python3-runtime
pip install apache-age-python，
when import age, report an error:
Guidelines for extending the driver to other languages
I would like to write this driver to Rust language.
what are the API guidelines?
Nodes not printing, error: line 1:100 missing '::path' at '<EOF>'
apache-age-python returns an error over result formatting.
from the AGEviewerer, this query successfully prints the results as right-left nodes containing the list of the right nodes as shown below:
SELECT * from cypher('text_test', $$
MATCH (V:node)-[connect]->(V2:node)
WITH V as parent, COLLECT(V2) as children
RETURN parent, children
$$) as (parent agtype, children agtype);
and the result below:
{"id":844424930131987,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"second\", \"class\": \"below\"}"}} [{"id":844424930131988,"label":"node","properties":{"tag":"div","attrib":"{\"class\": \"inner\"}"}}]
{"id":844424930131973,"label":"node","properties":{"tag":"body","attrib":"null"}}
[{"id":844424930131987,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"second\", \"class\": \"below\"}"}},{"id":844424930131978,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"idone\", \"class\": \"classone\"}"}},{"id":844424930131974,"label":"node","properties":{"tag":"ul","attrib":"{\"class\": \"menu\"}"}}]
{"id":844424930131970,"label":"node","properties":{"tag":"head","attrib":"null"}}
[{"id":844424930131972,"label":"node","properties":{"tag":"title","attrib":"null"}},
{"id":844424930131971,"label":"node","properties":{"tag":"meta","attrib":"{\"charset\": \"UTF-8\"}"}}]
{"id":844424930131985,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"innertwo\"}"}} [{"id":844424930131986,"label":"node","properties":{"tag":"h2","attrib":"null"}}]
{"id":844424930131978,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"idone\", \"class\": \"classone\"}"}}
[{"id":844424930131985,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"innertwo\"}"}},{"id":844424930131983,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"innerone\"}"}},{"id":844424930131979,"label":"node","properties":{"tag":"ul","attrib":"{\"class\": \"listing\"}"}}]
{"id":844424930131974,"label":"node","properties":{"tag":"ul","attrib":"{\"class\": \"menu\"}"}}
[{"id":844424930131977,"label":"node","properties":{"tag":"div","attrib":"{\"class\": \"itm\"}"}},{"id":844424930131976,"label":"node","properties":{"tag":"div","attrib":"{\"class\": \"itm\"}"}},{"id":844424930131975,"label":"node","properties":{"tag":"div","attrib":"{\"class\": \"itm\"}"}}]
{"id":844424930131988,"label":"node","properties":{"tag":"div","attrib":"{\"class\": \"inner\"}"}}
[{"id":844424930131991,"label":"node","properties":{"tag":"h2","attrib":"null"}},{"id":844424930131990,"label":"node","properties":{"tag":"h1","attrib":"null"}},{"id":844424930131989,"label":"node","properties":{"tag":"h1","attrib":"null"}}]
{"id":844424930131983,"label":"node","properties":{"tag":"div","attrib":"{\"id\": \"innerone\"}"}} [{"id":844424930131984,"label":"node","properties":{"tag":"h1","attrib":"null"}}]
{"id":844424930131979,"label":"node","properties":{"tag":"ul","attrib":"{\"class\": \"listing\"}"}} [{"id":844424930131982,"label":"node","properties":{"tag":"li","attrib":"{\"class\": \"item3\"}"}},
{"id":844424930131981,"label":"node","properties":{"tag":"li","attrib":"{\"class\": \"item2\"}"}},{"id":844424930131980,"label":"node","properties":{"tag":"li","attrib":"{\"class\": \"item1\"}"}}]
{"id":844424930131969,"label":"node","properties":{"tag":"html","attrib":"{\"lang\": \"en\"}"}} [{"id":844424930131973,"label":"node","properties":{"tag":"body","attrib":"null"}},{"id":844424930131970,"label":"node","properties":{"tag":"head","attrib":"null"}}]
while using the same procedure on apache-python-age, here is the resulting output.
import psycopg2
import age
GRAPH_NAME = "text_test"
DSN = "host=localhost port=5432 dbname=texttest user=afidegnum password=chou1979"
conn = psycopg2.connect(DSN)
age.setUpAge(conn, GRAPH_NAME)
def get_tree(n_id):
""" Get the trees"""
with conn.cursor() as cursor:
try :
cursor.execute("""
SELECT * from cypher(%s, $$
MATCH (V:node)-[r:connect]->(V2:node)
WITH r as nodes, COLLECT(V2) as children
RETURN nodes, children
$$) as (nodes agtype, children agtype);
""", (GRAPH_NAME,))
for row in cursor:
print(row)
except Exception as ex:
print(ex)
conn.rollback()
node_id = 844424930131969
get_tree(node_id)
here is the result obtained:
__init__() missing 1 required positional argument: 'cause'
line 1:189 mismatched input '::vertex' expecting '::edge'
line 1:408 mismatched input '::vertex' expecting '::edge'
line 1:682 mismatched input '::vertex' expecting '::edge'
line 1:910 mismatched input '::vertex' expecting '::edge'
line 1:1138 mismatched input '::vertex' expecting '::edge'
line 1:1370 mismatched input '::vertex' expecting '::edge'
line 1:1602 mismatched input '::vertex' expecting '::edge'
line 1:1832 mismatched input '::vertex' expecting '::edge'
line 1:2026 mismatched input '::vertex' expecting '::edge'
line 1:2239 mismatched input '::vertex' expecting '::edge'
line 1:2433 mismatched input '::vertex' expecting '::edge'
Care to look into it ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.