Comments (7)
No, what's the relation between asynch and pandas? What's the meaning to select directly to dataframe?
from asynch.
Well, we are talking here about clickhouse database, and how to access it asynchronously.
Most of the time, we will selecting data from database.
When you are selecting data from database you need some complex structure to hold the result.
If we are talking about python, pandas dataframe has no alternative for such purpose, if you need to further do something with the data (data wrangling, machine learning...).
At the moment I am using clickhouse_driver and DB API connection and pandas.read_sql function (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_sql.html).
As aioch doesn't have DB API connection I am kind of stuck with that approach.
I will try to do the same with your asynch driver. I hope it will work.
from asynch.
Well, so did you try it? I never use pandas, and asynch also support DB API.
from asynch.
I tried this:
async def main():
conn = await connect(
host="127.0.0.1",
port=9001,
database="default",
)
async with conn.cursor() as cursor:
await cursor.execute("SELECT 1")
ret = cursor.fetchone()
print(ret)
I got (1,)
as a result, but where are column types? As I can see you are using with_column_types=True
in response = await execute(query, args=args, with_column_types=True, **execute_kwargs)
, but you are not returning them.
Anyway I hoped that the following will work, but it doesn't:
import asyncio
from asynch import connect
import pandas as pd
from jinjasql import JinjaSql
async def main():
conn = await connect(
host="127.0.0.1",
port=9001,
database="default",
)
jsql = JinjaSql(param_style='pyformat')
sql_templ = 'select 1'
params = {}
query, bind_params = jsql.prepare_query(sql_templ, params)
df = pd.read_sql_query(query, conn, params=bind_params) # I tried df = await pd.read_sql(...) but it hasn't worked
print(df)
asyncio.run(main())
RuntimeWarning: coroutine 'Cursor.execute' was never awaited
cur.execute(*args, **kwargs)
Looks like you should create/delete the cursor on the fly in the background for such usage.
from asynch.
@long2ice can you please comment my findings?
from asynch.
Does pandas support asyncio?
from asynch.
No
from asynch.
Related Issues (20)
- Not able to connect using secure=True
- Error when toggle stream_results=True HOT 3
- IndexError: bytearray index out of range HOT 13
- Connection hangs up
- Why aren't the connections to Clickhouse closed? HOT 3
- Compressed = True does not work HOT 6
- facing problem with installing asynch
- The cursor closes the connection incorrectly
- Cannot connect with secure=True, verify=False HOT 1
- Can not connect to Clickhouse version 22.3.8.39 HOT 1
- RuntimeError: read() called while another coroutine is already waiting for incoming data HOT 4
- pypi wheels
- Problem with Decimal
- Compression still not working for 2.1.0 - 2.2.0 HOT 3
- DictCursor fetch* methods raise exceptions when no rows are returned HOT 2
- Streaming queries fail when user is readonly type 1
- Connection pool initialization HOT 2
- Bug IndexError HOT 4
- Connection live time HOT 1
- ProgrammingError on async race condition when using pool execute many HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asynch.