Giter Club home page Giter Club logo

sql-odbc's Introduction

OpenSearch ODBC Driver

OpenSearchODBC is a read-only ODBC driver for Windows and Mac for connecting to OpenSearch SQL support.

ODBC Driver

Build ODBC CI

Specifications

The driver is compatible with ODBC 3.51.

Supported Versions

Operating System Version Supported Bitness
Windows Windows 10, Windows 11 32-bit, 64-bit
MacOS Catalina 10.15.4, Mojave 10.14.6 64-bit

BI Tool Connectors

Download and Installation the Driver

You can use the installers generated as part of the most recent release. They are available for download from Artifacts page on OpenSearch.org at the very bottom and from automated CI workflow.

Windows

  1. Run the .msi installer to install the OpenSearch SQL ODBC Driver.
  2. Test connection using ODBC Data Source Administrator.

To use the driver with Tableau:

  1. Copy the .tdc file from <driver-install-dir>/resources to <windows-user-dir>/Documents/My Tableau Repository/Datasources.

This will customize the connection from Tableau to OpenSearch, ensuring that the correct forms of queries are used.

Mac

iODBC Driver Manager should be installed before installing the OpenSearch SQL ODBC Driver on Mac.

  1. Run the .pkg installer to install the OpenSearch SQL ODBC Driver.
  2. Configure a Driver and DSN entry for the OpenSearch SQL ODBC Driver, following the instructions here.

To use the driver with Tableau:

  1. Copy the .tdc file from <driver-install-dir>/resources to <mac-user-dir>/Documents/My Tableau Repository/Datasources.

This will customize the connection from Tableau to OpenSearch, ensuring that the correct forms of queries are used.

Using the Driver

The driver comes in the form of a library file:

  • Windows: sqlodbc.dll
  • Mac: libsqlodbc.dylib

If using with ODBC compatible BI tools, refer to the tool documentation on configuring a new ODBC driver. In most cases, you will need to make the tool aware of the location of the driver library file and then use it to setup OpenSearch database connections.

Connection Strings and Configuring the Driver

A list of options available for configuring driver behaviour is available here.

To setup a connection, the driver uses an ODBC connection string. Connection strings are semicolon-delimited strings specifying the set of options to use for a connection. Typically, a connection string will either:

  1. specify a Data Source Name containing a pre-configured set of options (DSN=xxx;User=xxx;Password=xxx;)
  2. or configure options explicitly using the string (Host=xxx;Port=xxx;LogLevel=OPENSEARCH_DEBUG;...)

Building from source

Building

Please refer to the build instructions for detailed build instructions on your platform. If your PC is already setup to build the library, you can simply invoke cmake using

cmake ./src

From the projects root directory, then build the project using Visual Studio (Windows) or make (Mac).

  • Visual Studio: Open ./global_make_list.sln
  • Make: Run make from the build root.

Testing

NOTE: Some tests in ITODBCConnection will fail if a test DSN (Data Source Name) is not configured on your system. Refer to "Running Tests" in the build instructions for more information on configuring this.

Documentation

Please refer to the documentation for detailed information on installing and configuring OpenSearch.

Code of Conduct

This project has adopted an Open Source Code of Conduct.

Security issue notifications

If you discover a potential security issue in this project, notify OpenSearch Security directly by emailing [email protected]. To prevent any additional risk caused by the potential issue, do not create a public GitHub issue.

Licensing

See the LICENSE file for our project's licensing. We will ask you to confirm the licensing of your contribution.

Copyright

Copyright OpenSearch Contributors. See NOTICE for details.

sql-odbc's People

Contributors

alinalibq avatar amazon-auto avatar anirudha avatar chloe-zh avatar dai-chen avatar davidcui1225 avatar dblock avatar derek-ho avatar eugenesk24 avatar forestmvey avatar gaiksaya avatar jordanw-bq avatar joshuali925 avatar lyndonbauto avatar mend-for-github-com[bot] avatar penghuo avatar peterzhuamazon avatar rupal-bq avatar tengda-he avatar vamsi-amazon avatar varun-lodaya avatar yang-db avatar yury-fridlyand avatar zhongnansu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sql-odbc's Issues

AWS OpenSearch compatibility issues?

SQL-ODBC seems to be not compatible with AWS OpenSearch
I have made a build from the latest source code and now I am trying to reach my AWS OpenSearch domain and I am getting this error
Screenshot 2021-09-30 at 17 24 39

The good news is that the driver itself seems to work correctly as it can tell exactly what the problem is.

The bad news is that /_cat/plugins on my AWS OpenSearch returns something strange and it makes me think that the driver expects the remote OpenSearch instance to have a certain plugin installed and it expects this plugin to be called sql (and I guess it used to be called that way as it is still mentioned in some AWS docs ). Anyway the latest OpenSearch 1.0 AWS OpenSearch domains list the following plugins (I mean that opensearch- prefix), so I would say this is a kind of naming issue
Screenshot 2021-09-30 at 18 10 50

This can not be called a bug as the product seems to be functional so far, but it would be great to have an option to specify what is the name of plugin (?). I will also appreciate pointing me to a file I could tweak to make this done.

[BUG] OpenSearch ODBC driver installer conflicts with already installed OpenDistro ODBC driver

Describe the bug
OpenSearch and OpenDistro ODBC drivers conflicts with each other. It is impossible to install both of the on the computer.
Furthermore, it is impossible to install 32 bit driver if 64 bit one is already installed and otherwise.

To Reproduce
Steps to reproduce the behavior:

  1. Install OpenDistro ODBC driver taken from https://opendistro.github.io/for-elasticsearch/downloads.html
  2. Download OpenSearch ODBC driver with the same bitness. I used artifacts from this workflow run: https://github.com/opensearch-project/sql/actions/runs/1343276721
  3. Try to install it
  4. Error says that a newer version is already installed

Test 2

  1. Install OpenSearch ODBC 32-bit
  2. Try installing OpenSearch ODBC 64-bit
  3. Install succeeds, but the first driver is deleted

Expected behavior
OpenSearch ODBC driver is installed and co-exist with OpenDistro ODBC driver. No conflicts between them.
OpenSearch ODBC 32-bit driver is installed and co-exist with OpenSearch ODBC 64-bit driver. No conflicts between them.

Host/Environment (please complete the following information):

md5sum *.msi
82604ef8a21b3e048ed3e45545f38839  opendistro-sql-odbc-1.13.0.1-windows-x64.msi
69217d4692d91221700388f48649f16a  opendistro-sql-odbc-1.13.0.1-windows-x86.msi
52c3a8aa2214880cf480ed5208c13048  OpenSearch SQL ODBC Driver 32-bit-1.1.0.0-Windows.msi
94f078df2bb41b95dd7b13f5469b71cb  OpenSearch SQL ODBC Driver 64-bit-1.1.0.0-Windows.msi

Additional context
I guess this happened because both OpenSearch and OpenDistro ODBC drivers are registered as the same product in the system.

Update ODBC driver docs to address authentication issues and limitations

Is your feature request related to a problem? Please describe.
Bugs #22, #20 and #19 address different authentication issues. All information, including limitations and possible workarounds, should be documented in one place.

Describe the solution you'd like
This task is to update md documentation. It should help ODBC driver users to understand possible issues and avoid them.

[BUG] ODBC driver ignores `Use SSL` flag and requires https:// prefix for SSL connections

Describe the bug
The ODBC driver requires https:// prefix for Host field for SSL connections. Without this prefix SSL is not used actually. Flag 'Use SSL' is ignored.

To reproduce
Flag is unset, https:// prefix is added, connection succeeds:
image
image

Otherwise connection fails:
image
image

Host/Environment (please complete the following information):

  • OS: Win 10
  • Version: 1.2.0

[BUG] ODBC driver requires additional configuration to use AWS_SIGv4 authentication

Describe the bug

  1. ODBC driver accepts credentials from opensearchodbc profile only from ~/.aws/credentials file. The default profile is ignored.
  2. ODBC driver ignores credentials set in environment variables. See doc.

Expected behavior
The JDBC driver accepts credentials from the default profile` and from the environment variables as well. I expected ODBC to do the same.

Version: 1.2.0

image

[BUG] ODBC driver doesn't work properly when SSL is used

Describe the bug
I detected that ODBC driver refers to ODFE when SSL is activated. Maybe it is not a reason, but just a coincidence.

To Reproduce
Steps to reproduce the behavior:

  1. I started a local cluster with security plugin on port 9300 instead of 9200 (opensearch-docker.zip).
  2. Configured a DSN as shown on the picture
  3. Tried to test the connection
  4. Connection fails, SSL wasn't actually used (HTTP traffic is not encrypted).

image
image
image

See also ODBC log: mylog_odbcad32_22088.log

I also tried the same test with an AWS instance and it also failed. Despite of local one, it has valid SSL certificates.
Perhaps, it also affects AWS_SIGv4 authentication.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS] Win 10 + Docker on WSL2
  • Version [e.g. 22] ODBC v.1.2.0, OS v.1.2.0

[BUG]Aggregate functions in PowerBI fail

Describe the bug
Visualizations with Minimum, Maximum, Standard deviation, Variance, or Median options selected fail to load.

To Reproduce
Steps to reproduce the behavior:

  1. Start PowerBI
  2. Connect to OpenSearch
  3. Drag a numerical column into the workspace
  4. Select one of the aggregate functions listed above
  5. See visualization error in the workspace

Expected behavior
The result should be have the expected value of the corresponding aggregate function.

Screenshots
pbi_issue_aggregation

Additional context
Using OpenSearch version 1.2.0

[FEATURE][ODBC] Support SIGV4 along with Basic Auth

Is your feature request related to a problem?
Currently there are three auth types in ODBC driver: basic, sigv4, none.
Users can provide their FGAC credentials (username and password) using basic auth, and sigv4 will read opensearchodbc aws profile which contains IAM credentials.

There is a domain that uses both types of authentication. opensearch.log shows 401 authn error for basic and none auth types, and 403 authz error for sigv4 auth type. Looks like sigv4 got passed the aws validation, but since there were no username and password send, it did not go through the FGAC validation.

Feel free to edit/comment if the above assumption is wrong.

What solution would you like?
Provide an option in ODBC driver to allow user use their aws credentials with basic auth (username and password).

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

ODBC driver does not recognize its support for conversions[BUG]

Describe the bug
ODBC driver does not recognize its support for conversions of any types. The SQL plugin supports casting of different types and should be reflected by the ODBC driver.

To Reproduce
Steps to reproduce the behavior:

  1. Install the ODBC driver from https://github.com/opensearch-project/sql/actions/runs/1555197058
  2. Open ODBCTest
  3. Click on the Conn button
  4. Select SQLGetInfo
  5. Select SQL_CONVERT_INTEGER=61 (1.0) for info type
  6. See the result InfoValuePtr = 0x00000000

Expected behavior
Should not be 0 as that represents unsupported.

[FEATURE] Create integration test platform for ODBC driver

Is your feature request related to a problem?

There are automated tests for ODBC driver with a real OpenSearch cluster

What solution would you like?

  1. New or modified CI workflow which run an OpenSearch cluster and run tests
  2. New test sub-project which loads and tests ODBC driver

What alternatives have you considered?

N/A

Do you have any additional context?

See also opensearch-project/sql-jdbc#18

[BUG] PBI connector: Connectivity issue

I received an email saying that an OpenSearch user experiencing a PBI connectivity issue. Quoting:

User can refresh the report in Power BI Desktop but failing in Power BI service with the below error. Tried to refresh the Dataset in Power BI web which is failing.

We cannot discover all data sources in this dataset right now because the dataset users a custom connector. Make sure the data gateway associated with the dataset is online and has the custom connector loaded, then click again on Discover Data Sources.

I asked for details and will update this post when/if I have more info.

[BUG] Build for macOS is currently failing.

macOS build is failing. See here for an example.

Here's a snippet of the log:

CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
    Command failed: /usr/local/Cellar/cmake/3.25.2/bin/cmake --build . --config Debug --target install -- -v -j4
    Working Directory: /usr/local/share/vcpkg/buildtrees/aws-sdk-cpp/x64-osx-dbg
    See logs for more information:
      /usr/local/share/vcpkg/buildtrees/aws-sdk-cpp/install-x64-osx-dbg-out.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_build_cmake.cmake:78 (vcpkg_execute_build_process)
  scripts/cmake/vcpkg_install_cmake.cmake:18 (vcpkg_build_cmake)
  buildtrees/versioning_/versions/aws-sdk-cpp/948e791fc4effed6d27b6ad4da335a51ba6111bd/portfile.cmake:37 (vcpkg_install_cmake)
  scripts/ports.cmake:147 (include)


error: building aws-sdk-cpp:x64-osx failed with: BUILD_FAILED
error: Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
    https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+aws-sdk-cpp
You can submit a new issue at:
    https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[aws-sdk-cpp]+Build+error
Include '[aws-sdk-cpp] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
    vcpkg-tool version: 2023-01-24-8a88d63f241d391772fbde69af9cab96c3c64c75
    vcpkg-scripts version: a32522820 2023-02-18 (11 days ago)

[2.12.0] Ensure compatibility for 2.12.0 default admin credentials changes

Is your feature request related to a problem?
With the 2.12.0 release of OpenSearch, the security plugin's install demo configuration script requires an initial admin password to be passed in to run. We want to ensure all repos are aware/complaint with this change.

What solution would you like?
A clear and concise description of what you want to happen.

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

Update screenshots in ODBC user manual

There are many screenshots in ODBC documentation which still shows ODFE namespace and driver name. Need to update with new OpenSearch screenshots to avoid confusion.

Power BI is getting an unexpected data type for date columns from the ODBC driver[BUG]

Describe the bug
Power BI is getting an unexpected data type for date columns from the ODBC driver. The driver should return type 93 which is SQL_TYPE_TIMESTAMP. These columns do not show up on Power BI as there is a data type mismatch.

To Reproduce

  1. Install the ODBC driver from https://github.com/opensearch-project/sql/actions/runs/1555197058
  2. Open ODBCTest
  3. Click on the Catalog button
  4. Select SQLColumns
  5. See data type for date columns is not 93

Expected behavior
Data type in ODBC test should have data type 93 for date columns

[BUG] Windows installer is missing files.

What is the bug?
A driver can't run if installed by package created by CI. Dependency files are missing.
image
It should contain zlib, libcurl, vld, aws-cpp-sdk, aws-c-sdk and all dependencies.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Build installer from main or download from GH CI
  2. Install
  3. Try to use driver

What is the expected behavior?
Installer should contain all required files.

What is your host/environment?
main @ 9b8d366

Do you have any screenshots?
TODO

Do you have any additional context?
Maybe caused by #27

Manual approval required for workflow run 5458110121: Release ODBC driver version 1.5.0.0

Please approve or deny the release of ODBC Driver TAG: 1.5.0.0 COMMIT: becefce VERSION : 1.5.0.0

Workflow is pending manual review.
URL: https://github.com/opensearch-project/sql-odbc/actions/runs/5458110121

Required approvers: [pjfitzgibbons ps48 kavithacm derek-ho joshuali925 dai-chen YANG-DB rupal-bq mengweieric vamsi-amazon swiddis penghuo seankao-az MaxKsyunz anirudha forestmvey acarbonetto GumpacG]

Respond "approved", "approve", "lgtm", "yes" to continue workflow or "denied", "deny", "no" to cancel.

[BUG] Tableau is incorrectly receiving non-null values as null from the ODBC driver

Describe the bug
The TDVT smoke tests are failing due to all values of a query being null when the expected are not null. The issue has been isolated to the ODBC driver as a query to the OpenSearch sql plugin returns accurate non-null values.

To Reproduce
Steps to reproduce the behavior using ODBCTest:

  1. Execute SELECT staples.Discount FROM staples GROUP BY 1
  2. SQLSetStmtAttr
  • Attribute SQL_ATTR_ROW_ARRAY_SIZE
  • ValuePtr: 100
  1. SQLBindCol
  • Column Number: 21
  • Target Type: SQL_C_Double
  1. SQL fetch
  2. Actual Result: SQLDescribeCol returned: SQL_ERROR=-1

Screenshots
Actual results:
image
Expected results:
image

Host/Environment (please complete the following information):

  • OS: Windows 10
  • Version 21H1

[FEATURE] OAuth2 implementation

Is your feature request related to a problem?
Is more often required this type of authentication/authorization.

What solution would you like?
A working solution is provided in the pull request #67

What alternatives have you considered?
A working solution is provided in the pull request #67

Do you have any additional context?

Add option to specify keys in the connection string for AWS_SIGv4 authentication

Is your feature request related to a problem? Please describe.
According to the doc, a JDBC driver user should

  • Record access/secret keys in file ~/.aws/credentials before trying to connect
    or
  • Set env variables called AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with corresponding values before opening an application to browse the DB (e.g. Tableau Desktop or DBeaver)

I find both options not-user-friendly for a regular user.

Describe the solution you'd like
Add option to specify keys in the connection string as it is already done for username and password
For example:

jdbc:opensearch://https://blah-blah-blah.us-east-1.es.amazonaws.com?auth=AWS_SIGV4&access_key=$AWS_ACCESS_KEY_ID&secret_key=$AWS_SECRET_ACCESS_KEY

UPD: Once both keys are specified in the connection string, the driver shouldn't create an instance of AWSCredentialsProvider. Specified = both keys are not empty and valid (checksum validation if available).

Additional context
It is relevant for JDBC driver and for ODBC if it supports AWS_SIGv4 authentication method

[BUG] Broken link in the README

What is the bug?
Tableau connector link should be removed from the BI tools section of the readme. The link to the PowerBI connector in the same section is broken and should be fixed.

[BUG] Connection fails if server URL has slash in the end

What is the bug?
ODBC can't connect if server URL has slash in the end.

How can one reproduce the bug?
image
image

What is the expected behavior?
Driver should automatically delete the slash if it is causes troubles or ignore it.

What is your host/environment?
OpenSearch 2.3, ODBC 1.4

Release `ODBC` driver for Linux

Is your feature request related to a problem? Please describe.
A user can't create Linux applications which use the ODBC driver.

Describe the solution you'd like
ODBC driver to be compiled to most popular Linux kernels.

Describe alternatives you've considered
A user might use the JDBC driver, but it adds difficulties for embedding it to non-java applications.

[FEATURE] Support SSH tunnel with AWS_SIGv4 auth

Is your feature request related to a problem?
An OpenSearch cluster located into AWS VPC can be reached with SSH tunnel or VPN only.
AWS_SIGv4 authentication requires extra header to set if SSH tunnel is used.

What solution would you like?
Add optional configuration parameter when AWS_SIGv4 is selected.

What alternatives have you considered?
Use BASIC auth instead.

Do you have any additional context?
https://stackoverflow.com/a/67683064

[FEATURE] Request an option to inform the indices alias in the driver configuration

Hello, I would like to report an issue we are having, the users who not being allowed to view all OpenSearch indices ("indices:admin/get") are getting an error to connect. Our OpenSearch users can only view their indexes, as the same server is shared between several projects so we cannot give permissions to view all indexes for security reasons.

A solution to this problem would be to add a option to inform the alias of the indices we want to connect, especially when the driver executing SHOW TABLES LIKE '%' in the connection test to the db, since we cannot directly execute this SQL because the lack of the "indices:admin/get".

Only to reinforce the idea, we can't add the permission to our users.

[BUG] ODBC driver not properly paging

Describe the bug
When using the ODBC driver from some SQL clients, only the first 200 rows are retrieved by default. Normally we would use a "LIMIT" clause to get more than 200 rows, but LIMIT is not supported by all SQL clients. We can manually increase the cluster opendistro.query.size_limit and the index.max_result_window settings, but this has significant memory overhead.

Can the ODBC driver support an efficient paging mechanism for these?

To Reproduce
Steps to reproduce the behavior:

  1. In SQL Server (or a different SQL client tool), add a linked server to the OpenSearch ODBC DSN:
    EXEC sp_addlinkedserver
    @server = N'OpenSearch',
    @srvproduct = N'',
    @Provider = N'MSDASQL',
    @datasrc = N'OpenSearch SQL ODBC DSN';

  2. try to query the Linked Server using SQL (ensure the OpenSearch index has more than 200 rows)
    select top 1000 * from [OpenSearch]...[myopensearchindex]

What happens
only the first 200 rows are retrieved. We can force more rows by changing the cluster's opendistro.query.size_limit and each index's index.max_result_window --- but that is not optimal for memory.

Expected behavior
The ODBC driver should implement proper paging to get the data down to the client. Maybe it could honor the T-SQL "top" syntax instead of a "LIMIT" clause?

[FEATURE] Support EC2 Instance Authentication

I would like to use the ODBC driver on Windows EC2 instances that have an instance profile and IAM role with permissions for OpenSearch. This would eliminate the need to configure or store credentials within the EC2 instance. Unfortunately, this ODBC driver has hard coded the use of the profile credential provider when selecting IAM authentication.

The ODBC driver should switch from using Aws::Auth::ProfileConfigFileAWSCredentialsProvider to instead using Aws::Auth::DefaultAWSCredentialsProviderChain which would use EnvironmentAWSCredentialsProvider, ProfileConfigFileAWSCredentialsProvider, InstanceProfileCredentialsProvider in that order. This would provide functionality with parity to the JDBC driver and most tooling that uses the AWS SDK.

Build zip artifact for ODBC driver on Mac

Is your feature request related to a problem? Please describe.
Currently ODBC driver is built into .pkg on Mac which introduces more complexity for artifact signing.

Describe the solution you'd like
Instead, build zip artifact for ODBC driver on MacOS. This is similar as MySQL which only provides dmg or zip for Mac users.

Describe alternatives you've considered
Configure the makefile for cpack to generate zip artifact which includes same files as before.

Baseline MAINTAINERS, CODEOWNERS, and external collaborator permissions

Follow opensearch-project/.github#125 to baseline MAINTAINERS, CODEOWNERS, and external collaborator permissions.

Close this issue when:

  1. MAINTAINERS.md has the correct list of project maintainers.
  2. CODEOWNERS exists and has the correct list of aliases.
  3. Repo permissions only contain individual aliases as collaborators with maintain rights, admin, and triage teams.
  4. All other teams are removed from repo permissions.

If this repo's permissions was already baselined, please confirm the above when closing this issue.

[BUG] ODBC driver doesn't use SSL

Describe the bug
ODBC driver doesn't use SSL.

To Reproduce
Steps to reproduce the behavior:

  1. Launch an OS cluster, enable SSL
  2. Install the driver
  3. Test connection

image
image

Expected behavior
The driver should be able to use SSL according to the user settings. Option to trust self-signed certificates should be also available.

Version tested: 1.2.0

[BUG] OpenSearch ODBC client will not connect to OpenSearch cluster

Describe the bug
OpenSearch ODBC client will not connect to OpenSearch cluster.

Using the 'Test' button in the ODBC Data Sources Configuration dialog, I get the "Connection error: SQL plugin is not available, please install the SQL plugin to use this driver" error message. Using curl with the same parameters in a WSL window on the same system works as expected.

Tracing through the code, this message is generated in the CheckSQLPluginAvailability() method in opensearch_communication.cpp. Based on what I can see in the log on the Windows side (see below), it is being triggered by an exception thrown in the ParseErrorResponse() method -- which probably means it is getting a non-JSON response to parse. The problem is, I can't see what that response is -- I can only sometimes get the OpenSearch logs to show a connection (though communication is confirmed using tcpdump on the server side), much less tell me what it is sending back. The root log level of the node I'm trying to connect to is set to DEBUG ... I've tried to set to TRACE, but couldn't sift any additional useful information out -- but if y'all can tell me what to search for, I'll set it to TRACE again and send the results.

When using curl to test, I can see messages from o.o.s.l.p.RestSqlAction showing the query received, but not when I test the ODBC driver -- which makes me wonder if the driver is sending the appropriate API call at all? Again, I can't see enough in the logs to answer this question.

I see similar issues raised in several different threads in the forums, with the frequent cause being SSL configuration, possible certificate issues, but (as you see below) I've worked through those issues to make sure I've got those bases covered. I installed the cluster's Root CA in the windows trusted authorities keystore, and verified using a browser that the cluster's cert is accepted.

To Reproduce
Steps to reproduce the behavior:

  1. Download ODBC driver (see issue opensearch-project/sql#244 )
  2. Install driver and open ODBC Data Sources (64 bit) tool
  3. Enter parameters (see screenshot below) on main screen, selecting 'SSL' from advanced options sub-dialog (hostname validation is not set, but appears to make no difference)
  4. Click the 'test' button
  5. Error dialog pops up after timeout period (see below)
  6. Logs from the Windows side are also below

Expected behavior
The ODBC driver should connect with the OpenSearch cluster

Plugins
These are the plugins currently installed:

opensearch-alerting
opensearch-anomaly-detection
opensearch-asynchronous-search
opensearch-cross-cluster-replication
opensearch-index-management
opensearch-job-scheduler
opensearch-knn
opensearch-notebooks
opensearch-performance-analyzer
opensearch-reports-scheduler
opensearch-security
opensearch-sql

Screenshots
Configuration Dialog:
image

Error Dialog:
image

Windows Log:

opensearch[OpenSearchCommunication::LogMsg]832: Verifying connection options.
opensearch[OpenSearchCommunication::LogMsg]832: Required connection option are valid.
opensearch[OpenSearchCommunication::LogMsg]832: Starting DB connection.
opensearch[OpenSearchCommunication::LogMsg]832: Attempting to establish DB connection.
opensearch[OpenSearchCommunication::LogMsg]832: 
opensearch[OpenSearchCommunication::LogMsg]832: Checking for SQL plugin status.
opensearch[OpenSearchCommunication::LogMsg]832: Parsing error response (with schema validation)
opensearch[OpenSearchCommunication::LogMsg]832: Unexpected exception thrown from the server, the SQL plugin is not installed or in unhealthy status.
opensearch[OpenSearchCommunication::LogMsg]832: SQL plugin is not available, please install the SQL plugin to use this driver.
opensearch[OpenSearchCommunication::LogMsg]832: Dropping DB connection.
connection[CC_set_error_statements]564: entering self=0000025EB7BD36E0
connection[CC_log_error]652: CONN ERROR: func=LIBOPENSEARCH_connect, desc='', errnum=202, errmsg='Connection error: [OpenSearch][SQL ODBC Driver][SQL Plugin] Connection error: SQL plugin is not available, please install the SQL plugin to use this driver.'
connection[CC_log_error]655:             ------------------------------------------------------------
connection[CC_log_error]658:             henv=0000025EB7BAD060, conn=0000025EB7BD36E0, status=0, num_stmts=16
connection[CC_log_error]660:             opensearchconn=0000000000000000, stmts=0000025EB7BA2140, lobj_type=-999
connection[CC_log_error]652: CONN ERROR: func=OPENSEARCHAPI_DriverConnect, desc='Error from CC_Connect', errnum=202, errmsg='Connection error: [OpenSearch][SQL ODBC Driver][SQL Plugin] Connection error: SQL plugin is not available, please install the SQL plugin to use this driver.'
connection[CC_log_error]655:             ------------------------------------------------------------
connection[CC_log_error]658:             henv=0000025EB7BAD060, conn=0000025EB7BD36E0, status=0, num_stmts=16
connection[CC_log_error]660:             opensearchconn=0000000000000000, stmts=0000025EB7BA2140, lobj_type=-999
odbcapi30w[SQLGetDiagRecW]208: entering
opensearch[OPENSEARCHAPI_GetDiagRec]50: entering type=2 rec=1
 environ.c[OPENSEARCHAPI_ConnectError]239: entering hdbc=0000025EB7BD36E0 <512>
connection[CC_get_error]629: entering
connection[CC_get_error]641: leaving
 environ.c[OPENSEARCHAPI_ConnectError]256: CC_get_error: status = 202, msg = #Connection error: [OpenSearch][SQL ODBC Driver][SQL Plugin] Connection error: SQL plugin is not available, please install the SQL plugin to use this driver.#
 environ.c[OPENSEARCHAPI_ConnectError]343: 	     szSqlState = '08001',len=156, szError='Connection error: [OpenSearch][SQL ODBC Driver][SQL Plugin] Connection error: SQL plugin is not available, please install the SQL plugin to use this driver.'
opensearch[OPENSEARCHAPI_GetDiagRec]71: leaving 0
win_unicod[utf8_to_ucs2_lf]254: ilen=156 bufcount=512 ocount=156
odbcapi30.[SQLFreeHandle]230: entering
connection[OPENSEARCHAPI_FreeConnect]218: entering...hdbc=0000025EB7BD36E0
connection[CC_Destructor]325: entering self=0000025EB7BD36E0
connection[CC_cleanup]424: entering self=0000025EB7BD36E0
connection[CC_set_error_statements]564: entering self=0000025EB7BD36E0
connection[CC_log_error]652: CONN ERROR: func=CC_cleanup, desc='', errnum=1, errmsg='Connection not open'
connection[CC_log_error]655:             ------------------------------------------------------------
connection[CC_log_error]658:             henv=0000025EB7BAD060, conn=0000025EB7BD36E0, status=0, num_stmts=16
connection[CC_log_error]660:             opensearchconn=0000000000000000, stmts=0000025EB7BA2140, lobj_type=-999
connection[CC_cleanup]439: after LIBOPENSEARCH_disconnect
dlg_specif[CC_conninfo_init]463: entering opt=1
connection[CC_cleanup]505: leaving
connection[CC_Destructor]332: after CC_Cleanup
connection[CC_Destructor]343: after free statement holders
connection[CC_Destructor]354: leaving
connection[OPENSEARCHAPI_FreeConnect]236: leaving...
odbcapi30.[SQLFreeHandle]230: entering
 environ.c[OPENSEARCHAPI_FreeEnv]80: entering env=0000025EB7BAD060
 environ.c[EN_Destructor]453: entering self=0000025EB7BAD060
 environ.c[EN_Destructor]476: clearing conns count=128
 environ.c[EN_Destructor]488: leaving rv=1
 environ.c[OPENSEARCHAPI_FreeEnv]83:    ok

Host/Environment (please complete the following information):
Client - Windows 10 (latest updates)
ODBC Driver - tested with versions 1.1.0.1 and 1.2 - same behavior
Cluster - OpenSearch 1.1.0 (tarball) running on Debian 11 & Fedora 34 hosts (the host parameter was pointed to a Debian host)

Additional context
I'm curious if the 299 Deprecation warning is interfering with the exchange between the ODBC driver and the cluster, but I cannot find a way to shut it off to validate that hypothesis.

[BUG] `fetch_size` is not supported in PBI connector

What is the bug?

Currently, fetch_size can be configured in ODBC by adding ;fetchSize=x block in the connection string.
PowerBI connector has no option to set this.

What is the expected behavior?

One of:

  • Update PBI connector: allow user to optionally set fetch_size and use it in connection string if set
  • Update ODBC driver to properly comply with ODBC API. For example, if SQLSetStmtAttr called with SQL_ATTR_MAX_ROWS, driver doesn't set fetch_size parameter. Maybe PBI connector update is also required.

What is your host/environment?

main @ 1f0620f

Do you have any additional context?

Inspired by #16

[BUG] Using Power BI with aliased indices

What is the bug?
We have been trying to connect Power BI to our OpenSearch domains but are having troubles with using aliased indexes.

This looks to be down to SHOW tables LIKE %; reporting an instance of the alias for every index that is using it. This in itself is handled by Power BI, but then when you try to connect to the aliased index/table Power BI reports an error 'Expression.Error: The key matched more than one row in the table.'

Can you offer any guidance?

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Make a new OpenSearch data source:
    image

Select one of the instances of the aliased index
image
3. See error
image

What is the expected behavior?
Ideally the aliased index should only show up once in the list of indices, but even if it does selecting it should continue and not show an error.

What is your host/environment?

  • OS: Windows 10
  • Version: OpenSearch 1.3, but same behavior on OpenSearch 2.7
    Driver version: signed_opensearch-sql-odbc-win64-1.1.0.1.msi
  • Plugins: Nothing added just the standard AWS managed OpenSearch

Update Power BI connector to use a workaround for bug #330

Is your feature request related to a problem? Please describe.
As of #22, the PBI can't connect to a server using SSL.

Describe the solution you'd like
Fortunately, it is possible to use a workaround to avoid this bug and keep using SSL. The connector should add https:// prefix for the server address to enforce driver to use SSL.

Internal tracking id: AOS-248

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.