Working with this library and the SQL Alchemy version I have found that it is not poss

Add Endpoint_URL param to SQLAlchemy about pyathena HOT 2 CLOSED

Xl1001 commented on June 12, 2024

Add Endpoint_URL param to SQLAlchemy

from pyathena.

Comments (2)

laughingman7743 commented on June 12, 2024

https://github.com/laughingman7743/PyAthena/blob/master/pyathena/sqlalchemy/base.py#L996-L1006
It would need to be fixed to be able to handle the case of VPCE.

from pyathena.

laughingman7743 commented on June 12, 2024

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.client
https://github.com/laughingman7743/PyAthena/blob/master/pyathena/connection.py#L44
https://github.com/laughingman7743/PyAthena/blob/master/pyathena/connection.py#L155-L157
The SQLAlchemy URL is not associated with a VPC endpoint URL.
You should be able to use VPCE by specifying the VPCE URL in the parameter endpoint_url in the SQLAlchemy URL as follows:

conn_str = "awsathena+rest://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com:443/"\
           "{schema_name}?s3_staging_dir={s3_staging_dir}&endpoint_url={endpoint_url}"
engine = create_engine(conn_str.format(
    aws_access_key_id="YOUR_ACCESS_KEY_ID",
    aws_secret_access_key="YOUR_SECRET_ACCESS_KEY",
    region_name="us-west-2",
    schema_name="default",
    s3_staging_dir="s3://YOUR_S3_BUCKET/path/to/",
    endpoint_url="VPC_ENDPOINT_ID.athena.us-west-2.vpce.amazonaws.com"))

from pyathena.

Recommend Projects