Giter Club home page Giter Club logo

Comments (16)

karinazhou avatar karinazhou commented on May 22, 2024 2

@kitsuyui Thank you for pointing this out:

Yeah, Python3 seems to be changed about locale. But it is not a python's bug.
Because slash separated locales are totally legal in BSD family.
macOS is BSD descendant.

The reason for this issue is that we call something like std::locale(setlocale(LC_ALL, NULL)) to check the locale of current environment in the driver. When using Python3 on Linux and Mac, the returned locale string from setlocale(LC_ALL, NULL) are with different format if locale is set to UTF-8, for example:

  • Slash string on Mac
  • Semicolon string on Linux

If you add this to your test code without calling the ODBC driver on Mac:
printf("std::locale name : %s\n", std::locale(setlocale(LC_ALL, NULL)).name().c_str());

You will see the same abort with slash string. Obviously, the std::locale doesn't like slash string on Mac.

Instead of checking the LC_ALL, the LC_CTYPE will be sufficient for the driver. I have filed a bug in our tasks and the fix will come with the next release of the ODBC driver.

Thank you again for the report. We really appreciate it.

from homebrew-mssql-release.

anmuzychuk avatar anmuzychuk commented on May 22, 2024 2

interestingly, for those who still have issues check if

import locale
locale.setlocale(locale.LC_ALL, locale.getlocale())

gives an error, and if yes quick fix that works for me was:
$export LC_ALL=en_US.UTF-8
$export LANG=en_US.UTF-8

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024 1

@karinazhou Thank you so much.

I finally found these reproducible code.

It seems that python3 somehow breaks the behavior of the old setlocale(LC_ALL, NULL) function on Mac.

Yeah, Python3 seems to be changed about locale. But it is not a python's bug.
Because slash separated locales are totally legal in BSD family.
macOS is BSD descendant.

Look this Apple Libc source code. (lines from 156 to 191.)
https://opensource.apple.com/source/Libc/Libc-1244.30.3/locale/FreeBSD/setlocale.c.auto.html

Note: OpenBSD also accepts slash separated locales. OpenBSD setlocale(3) Examples

reproducible code

#include <stdio.h>
#include <wchar.h>
#include <sql.h>
#include <sqlext.h>
#include <string.h>
#include <locale>

void showData(SQLHDBC dbc)
{
    SQLHSTMT stmt = NULL;
    SQLCHAR szData[10];
    SQLLEN cbData = 0;
    SQLRETURN ret;

    SQLAllocHandle(SQL_HANDLE_STMT, dbc, &stmt);

    ret= SQLExecDirect(stmt, (SQLCHAR*)"SELECT 1", SQL_NTS);
    if (ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO) {
        while (true) {
            ret = SQLFetch(stmt);
            if (ret != SQL_SUCCESS)
                break;

            // Get data
            SQLGetData(stmt, 1, SQL_C_CHAR, szData, 10, &cbData);
            printf("data : %s\n", szData);
        }
    }
    else {
        printf("Failed to select the data\n");
    }

    SQLFreeHandle(SQL_HANDLE_STMT, stmt);
}

void HandleDiagnosticRecord (SQLHANDLE      hHandle,
                             SQLSMALLINT    hType,
                             RETCODE        RetCode)
{
    SQLSMALLINT iRec = 0;
    SQLINTEGER  iError;
    SQLCHAR       wszMessage[1000];
    SQLCHAR       wszState[10];

    if (RetCode == SQL_INVALID_HANDLE)
    {
        printf("Invalid handle!\n");
        return;
    }

    while (SQLGetDiagRec(hType,
                         hHandle,
                         ++iRec,
                         wszState,
                         &iError,
                         wszMessage,
                         (SQLSMALLINT)(sizeof(wszMessage) / sizeof(WCHAR)),
                         (SQLSMALLINT *)NULL) == SQL_SUCCESS)
    {
        printf("[%5.5s] %s (%d)\n", wszState, wszMessage, iError);
    }

}

int main() {
    setlocale(LC_ALL, "C/UTF-8/C/C/C/C");
    HDBC dbc = SQL_NULL_HANDLE;
    HENV env = SQL_NULL_HANDLE;
    SQLRETURN ret;
    SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
    SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, sizeof(int));
    SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
    wchar_t connectionStr[] = L"driver={ODBC Driver 17 for SQL Server}";
    ret = SQLDriverConnectW(dbc, NULL, (SQLWCHAR*)&connectionStr, SQL_NTS, NULL, 0, NULL, SQL_DRIVER_NOPROMPT);
    HandleDiagnosticRecord(dbc, SQL_HANDLE_DBC, ret);
    if (SQL_SUCCEEDED(ret)) {
        printf("Connected\n");
    }
    else {
        printf("Failed to connect\n");
        HandleDiagnosticRecord(dbc, SQL_HANDLE_DBC, ret);
    }
    showData(dbc);
    SQLDisconnect(dbc);
    SQLFreeHandle(SQL_HANDLE_DBC, dbc);
    SQLFreeHandle(SQL_HANDLE_ENV, env);
    return 0;
}
$ g++ -fshort-wchar -std=c++11 -stdlib=libc++ -I /usr/local/opt/msodbcsql17/include/msodbcsql17/ -o locale_test.dylib locale_test-g -lodbc

$ ./locale_test
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: collate_byname<char>::collate_byname failed to construct for C/UTF-8/C/C/C/C
Abort trap: 6

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

Hi kitsuyui,

The issue you reported may be caused by the incorrect locale setting. The ODBC Driver 17 introduces the encoding support and you can find more details about this from section Character Set Support:

https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/programming-guidelines?view=sql-server-2017#character-set-support

It mentions

if an application needs to use one of the encodings above, it should use the setlocale function to set the locale appropriately before connecting;

The application has its own locale which may be different from the operating system locale. For example, the application can be encoded in Shift-JIS while your system environment is UTF-8.

If both of your application and system locales are UTF-8 e.g. en_US.UTF-8, it should be fine without setlocale() function call in the application. If it doesn't work, you may explicitly call

setlocale(LC_ALL "")

If your application is encoded in other code page rather than UTF-8, to make the driver works properly, it's necessary to call setlocale() with proper character set, e.g.

setlocale(LC_ALL, "en_US.cp932")

Hope it help a bit.

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou Thank you for your information.
Okay, I understood why that is happened. But I have still some questions.

The URL says

Thus, in a typical Linux or Mac environment where the encoding is UTF-8, users of ODBC Driver 17 upgrading from 13 or 13.1 will not observe any differences. However, applications that use a non-UTF-8 encoding in the above list via setlocale() need to use that encoding for data to/from the driver instead of UTF-8.

But the sudden aborting happens on normally macOS environment with UTF-8. I don't set charset such as ja_JP.CP932 or zh_cn.CP936.
And it seems to be only accepted with LC_CTYPE=C, otherwise the aborting is happen.

This aborting is not happened when in Linux with both of default POSIX system locale and explicitly setting UTF-8 by Debian (locale-gen), Ubuntu(locale-update).

These occurring sudden abortion is the intended too? Why it happens on macOS only?

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

@kitsuyui Thanks for the reply.

I tested with a clean Mac High Sierra with OS:
Darwin 17.0.0 Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64 x86_64

I have the following system locale:

bamboos-Mac:~ bamboo$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

I installed the msodbcsql17 from
https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-2017

After installing the driver, I ran a simple cpp application to connect to the server and retrieve data. The server is SQL Server 2016. I didn't call setlocale() in my application. I could not get the aborting message you met with.

You may try the following code on your side as well to see whether you still get the aborting message or not.

Create test table charTest on your server and insert some data into it:

CREATE TABLE [dbo].[charTest](
	[col] [int] NULL,
	[data] [char](10) NULL
)

Create a locale_test.cpp with the following code:
NOTE: the cpp file must be with UTF-8 encoding. You can check it from Notepad++ or other editor.

#include <stdio.h>
#include <sql.h>
#include <sqlext.h>

void showData(SQLHDBC dbc)
{
    SQLHSTMT stmt = NULL;
    SQLCHAR szData[10];
    SQLLEN cbData = 0;
    SQLRETURN ret;   

    SQLAllocHandle(SQL_HANDLE_STMT, dbc, &stmt);
    
    ret= SQLExecDirect(stmt, (SQLCHAR*)"SELECT data FROM charTest", SQL_NTS);
    if (ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO) {
        while (true) { 
            ret = SQLFetch(stmt);     
            if (ret != SQL_SUCCESS)
                break;
            
            // Get data
            SQLGetData(stmt, 1, SQL_C_CHAR, szData, 10, &cbData);  
            printf("data : %s\n", szData);
        }
    }
    else {
        printf("Failed to select the data\n");
    }   
    
    SQLFreeHandle(SQL_HANDLE_STMT, stmt);
} 

int main() {
    SQLHENV env;
    SQLHDBC dbc;    
    SQLRETURN ret; // ODBC API return status 
    SQLCHAR outstr[1024] = {};
    SQLSMALLINT outstrlen = 0;

    SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
    SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (void *)SQL_OV_ODBC3, 0);
    SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);

    ret = SQLDriverConnect(dbc, NULL, (SQLCHAR *)"DRIVER={ODBC Driver 17 for SQL Server};SERVER=<replace with your server>;DATABASE=<replace with your db>;UID=<your id>;PWD=<your pwd>", SQL_NTS, outstr, 0, &outstrlen, SQL_DRIVER_COMPLETE);

    if (SQL_SUCCEEDED(ret)) {
        printf("Connected\n");
    }
    else {
        printf("Failed to connect\n");
    }
    
    showData(dbc);
    
    SQLDisconnect(dbc);     // disconnect from driver
    // free up allocated handles 
    SQLFreeHandle(SQL_HANDLE_DBC, dbc);
    SQLFreeHandle(SQL_HANDLE_ENV, env);
}

Build the cpp file:
g++ -std=c++11 -I /usr/local/opt/msodbcsql17/include/msodbcsql17/ -o locale_test locale_test.cpp -g -lodbc

And run it on your machine.

Moreover, could you please also run locale on your side to check whether they are all in en_US.UTF-8 ?
And also please check the encoding of the files of your application?

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou Thank you so much for your help!
I compiled and ran the code with various locale (nothing, C, POSIX, en_US.UTF-8), and it works fine.
That's great.

And I tried with debugging pyodbc, then I found the error happens on SQLDriverConnectW, not SQLDriverConnect.

here

libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: collate_byname<char>::collate_byname failed to construct for C/en_US.UTF-8/C/C/C/C
Abort trap: 6

Would you be able to check a version of SQLDriverConnectW (wide char version of SQLDriverConnect) possible?
I tried with just replacing SQLDriverConnect to SQLDriverConnectW and SQLCHAR to SQLWCHAR, then it can't connect.

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

Hi @kitsuyui ,

The wide version of SQLDriverConnect will be automatically called if the application is unicode. SQLDriverConnect is just the entry point and the actual functionality in the back stage is the same.
Yes, you can replace the code like this:

SQLDriverConnectW(dbc, NULL, (SQLWCHAR *)"DRIVER={ODBC Driver 17 for SQL Server};SERVER=<replace with your server>;DATABASE=<replace with your db>;UID=<your id>;PWD=<your pwd>", SQL_NTS, NULL, 0, NULL, SQL_DRIVER_COMPLETE);

NOTE: you can ignore outstr and outstrlen here since they are not necessary.

To make the change work, you need to add -fshort-wchar to the compile command:

g++ -fshort-wchar -std=c++11 -I /usr/local/opt/msodbcsql17/include/msodbcsql17/ -o locale_test locale_test.cpp -g -lodbc

I can connect to the server with this change without error. Could you please try it again?

In order to further investigate this issue, we need more information of your system environment. Could you please run the following commands on your side and let me know the result?

locale -a
locale
odbcinst -j

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou
Thank you for your kind instruction.
I tried but I couldn't connect when with SQLDriverConnectW version.

locale -a

$ locale -a
en_NZ
nl_NL.UTF-8
pt_BR.UTF-8
fr_CH.ISO8859-15
eu_ES.ISO8859-15
en_US.US-ASCII
af_ZA
bg_BG
cs_CZ.UTF-8
fi_FI
zh_CN.UTF-8
eu_ES
sk_SK.ISO8859-2
nl_BE
fr_BE
sk_SK
en_US.UTF-8
en_NZ.ISO8859-1
de_CH
sk_SK.UTF-8
de_DE.UTF-8
am_ET.UTF-8
zh_HK
be_BY.UTF-8
uk_UA
pt_PT.ISO8859-1
en_AU.US-ASCII
kk_KZ.PT154
en_US
nl_BE.ISO8859-15
de_AT.ISO8859-1
hr_HR.ISO8859-2
fr_FR.ISO8859-1
af_ZA.UTF-8
am_ET
fi_FI.ISO8859-1
ro_RO.UTF-8
af_ZA.ISO8859-15
en_NZ.UTF-8
fi_FI.UTF-8
hr_HR.UTF-8
da_DK.UTF-8
ca_ES.ISO8859-1
en_AU.ISO8859-15
ro_RO.ISO8859-2
de_AT.UTF-8
pt_PT.ISO8859-15
sv_SE
fr_CA.ISO8859-1
fr_BE.ISO8859-1
en_US.ISO8859-15
it_CH.ISO8859-1
en_NZ.ISO8859-15
en_AU.UTF-8
de_AT.ISO8859-15
af_ZA.ISO8859-1
hu_HU.UTF-8
et_EE.UTF-8
he_IL.UTF-8
uk_UA.KOI8-U
be_BY
kk_KZ
hu_HU.ISO8859-2
it_CH
pt_BR
ko_KR
it_IT
fr_BE.UTF-8
ru_RU.ISO8859-5
zh_TW
zh_CN.GB2312
no_NO.ISO8859-15
de_DE.ISO8859-15
en_CA
fr_CH.UTF-8
sl_SI.UTF-8
uk_UA.ISO8859-5
pt_PT
hr_HR
cs_CZ
fr_CH
he_IL
zh_CN.GBK
zh_CN.GB18030
fr_CA
pl_PL.UTF-8
ja_JP.SJIS
sr_YU.ISO8859-5
be_BY.CP1251
sr_YU.ISO8859-2
sv_SE.UTF-8
sr_YU.UTF-8
de_CH.UTF-8
sl_SI
pt_PT.UTF-8
ro_RO
en_NZ.US-ASCII
ja_JP
zh_CN
fr_CH.ISO8859-1
ko_KR.eucKR
be_BY.ISO8859-5
nl_NL.ISO8859-15
en_GB.ISO8859-1
en_CA.US-ASCII
is_IS.ISO8859-1
ru_RU.CP866
nl_NL
fr_CA.ISO8859-15
sv_SE.ISO8859-15
hy_AM
en_CA.ISO8859-15
en_US.ISO8859-1
zh_TW.Big5
ca_ES.UTF-8
ru_RU.CP1251
en_GB.UTF-8
en_GB.US-ASCII
ru_RU.UTF-8
eu_ES.UTF-8
es_ES.ISO8859-1
hu_HU
el_GR.ISO8859-7
en_AU
it_CH.UTF-8
en_GB
sl_SI.ISO8859-2
ru_RU.KOI8-R
nl_BE.UTF-8
et_EE
fr_FR.ISO8859-15
cs_CZ.ISO8859-2
lt_LT.UTF-8
pl_PL.ISO8859-2
fr_BE.ISO8859-15
is_IS.UTF-8
tr_TR.ISO8859-9
da_DK.ISO8859-1
lt_LT.ISO8859-4
lt_LT.ISO8859-13
zh_TW.UTF-8
bg_BG.CP1251
el_GR.UTF-8
be_BY.CP1131
da_DK.ISO8859-15
is_IS.ISO8859-15
no_NO.ISO8859-1
nl_NL.ISO8859-1
nl_BE.ISO8859-1
sv_SE.ISO8859-1
pt_BR.ISO8859-1
zh_CN.eucCN
it_IT.UTF-8
en_CA.UTF-8
uk_UA.UTF-8
de_CH.ISO8859-15
de_DE.ISO8859-1
ca_ES
sr_YU
hy_AM.ARMSCII-8
ru_RU
zh_HK.UTF-8
eu_ES.ISO8859-1
is_IS
bg_BG.UTF-8
ja_JP.UTF-8
it_CH.ISO8859-15
fr_FR.UTF-8
ko_KR.UTF-8
et_EE.ISO8859-15
kk_KZ.UTF-8
ca_ES.ISO8859-15
en_IE.UTF-8
es_ES
de_CH.ISO8859-1
en_CA.ISO8859-1
es_ES.ISO8859-15
en_AU.ISO8859-1
el_GR
da_DK
no_NO
it_IT.ISO8859-1
en_IE
zh_HK.Big5HKSCS
hi_IN.ISCII-DEV
ja_JP.eucJP
it_IT.ISO8859-15
pl_PL
ko_KR.CP949
fr_CA.UTF-8
fi_FI.ISO8859-15
en_GB.ISO8859-15
fr_FR
hy_AM.UTF-8
no_NO.UTF-8
es_ES.UTF-8
de_AT
tr_TR.UTF-8
de_DE
lt_LT
tr_TR
C
POSIX

locale

LANG=
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

odbcinst -j

$ odbcinst -j
unixODBC 2.3.6
DRIVERS............: /usr/local/etc/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources
USER DATA SOURCES..: /Users/kitsuyui/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

./locale_test (SQLDriverConnect version.)

$ ./locale_test
Connected

./locale_test (SQLDriverConnectW version)

$ ./locale_test
Failed to connect
Failed to select the data

./locale_test (SQLDriverConnectW version with setting LANG)

$ LANG=en_US.UTF-8 ./locale_test
Failed to connect
Failed to select the data

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

@kitsuyui Thank you for the detailed information!

The environment you provided looks good to me. Could you try to turn on the ODBC tracing on your mac and run the SQLDriverConnectW version test again?

Here's the instruction about how to do the ODBC tracing on Mac and Linux:
ODBC Tracing on Linux and Mac

Please attach the trace log so we can investigate more.

One more thing you can try is to add SQLGetDiagRec after you get the return code. Here's the sample code to use SQLGetDiagRec :

void HandleDiagnosticRecord (SQLHANDLE      hHandle,    
                             SQLSMALLINT    hType,  
                             RETCODE        RetCode)
{
    SQLSMALLINT iRec = 0;
    SQLINTEGER  iError;
    SQLCHAR       szMessage[1000];
    SQLCHAR       szState[10];

    if (RetCode == SQL_INVALID_HANDLE)
    {
        printf("Invalid handle!\n");
        return;
    }

    while (SQLGetDiagRec(hType,
                         hHandle,
                         ++iRec,
                         szState,
                         &iError,
                         szMessage,
                         1000,
                         (SQLSMALLINT *)NULL) == SQL_SUCCESS)
    {
        printf("[%5.5s] %s (%d)\n", wszState, wszMessage, iError);        
    }
}

if (SQL_SUCCEEDED(ret)) {
        printf("Connected\n");
    }
    else {
        printf("Failed to connect\n");
        HandleDiagnosticRecord(dbc, SQL_HANDLE_DBC, ret);        
    }

Also use SQL_DRIVER_NOPROMPT in SQLDriverConnectW if you meet with Dialog Failed error:
SQLDriverConnectW(dbc, NULL, (SQLWCHAR *)"DRIVER={ODBC Driver 17 for SQL Server};SERVER=<replace with your server>;DATABASE=<replace with your db>;UID=<your id>;PWD=<your pwd>", SQL_NTS, NULL, 0, NULL, SQL_DRIVER_NOPROMPT);

It should be able to print some error messages for the connection failure.

Thanks,

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

@kitsuyui I have updated the previous post with some modification.

The wprintf may not work with driver on Mac so please just use printf instead.
Additionally, please use SQL_DRIVER_NOPROMPT other than SQL_DRIVER_COMPLETE to make SQLGetDiagRec works properly.

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou Thank you so much for your patience and kindness!

Your sample code works fine. But I found more specific conditions that it happen.
So I wrote more minimal reproducible code without using pyodbc.

  1. It doesn't happen on Python2 but it happens only on Python3.
  2. It doesn't happen on Python2 installed with Homebrew too.
  3. It doesn't happen by executing directly C++ executable but it only happens by calling via shared library.

HandleDiagnosticRecord doesn't show error when suddenly abort happens.
Abort happens before HandleDiagnosticRecord called.
Also ODBC Trace log is not helpful for the same reason.

However I got crash report. This dumps more detailed stack trace.

These become hint?

Reproducible codes

#include <stdio.h>
#include <wchar.h>
#include <sql.h>
#include <sqlext.h>
#include <string.h>
#include <thread>

void showData(SQLHDBC dbc)
{
    SQLHSTMT stmt = NULL;
    SQLCHAR szData[10];
    SQLLEN cbData = 0;
    SQLRETURN ret;

    SQLAllocHandle(SQL_HANDLE_STMT, dbc, &stmt);

    ret= SQLExecDirect(stmt, (SQLCHAR*)"SELECT 1", SQL_NTS);
    if (ret == SQL_SUCCESS || ret == SQL_SUCCESS_WITH_INFO) {
        while (true) {
            ret = SQLFetch(stmt);
            if (ret != SQL_SUCCESS)
                break;

            // Get data
            SQLGetData(stmt, 1, SQL_C_CHAR, szData, 10, &cbData);
            printf("data : %s\n", szData);
        }
    }
    else {
        printf("Failed to select the data\n");
    }

    SQLFreeHandle(SQL_HANDLE_STMT, stmt);
}

void HandleDiagnosticRecord (SQLHANDLE      hHandle,
                             SQLSMALLINT    hType,
                             RETCODE        RetCode)
{
    SQLSMALLINT iRec = 0;
    SQLINTEGER  iError;
    SQLCHAR       wszMessage[1000];
    SQLCHAR       wszState[10];

    if (RetCode == SQL_INVALID_HANDLE)
    {
        printf("Invalid handle!\n");
        return;
    }

    while (SQLGetDiagRec(hType,
                         hHandle,
                         ++iRec,
                         wszState,
                         &iError,
                         wszMessage,
                         (SQLSMALLINT)(sizeof(wszMessage) / sizeof(WCHAR)),
                         (SQLSMALLINT *)NULL) == SQL_SUCCESS)
    {
        printf("[%5.5s] %s (%d)\n", wszState, wszMessage, iError);
    }

}

extern "C" void test() {
    HDBC dbc = SQL_NULL_HANDLE;
    HENV env = SQL_NULL_HANDLE;
    SQLRETURN ret;
    SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
    SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, sizeof(int));
    SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
    wchar_t connectionStr[] = L"driver={ODBC Driver 17 for SQL Server}";
    ret = SQLDriverConnectW(dbc, NULL, (SQLWCHAR*)&connectionStr, SQL_NTS, NULL, 0, NULL, SQL_DRIVER_NOPROMPT);
    HandleDiagnosticRecord(dbc, SQL_HANDLE_DBC, ret);
    if (SQL_SUCCEEDED(ret)) {
        printf("Connected\n");
    }
    else {
        printf("Failed to connect\n");
        HandleDiagnosticRecord(dbc, SQL_HANDLE_DBC, ret);
    }
    showData(dbc);
    SQLDisconnect(dbc);
    SQLFreeHandle(SQL_HANDLE_DBC, dbc);
    SQLFreeHandle(SQL_HANDLE_ENV, env);
}
$ g++ -fshort-wchar -shared -std=c++11 -stdlib=libc++ -I /usr/local/opt/msodbcsql17/include/msodbcsql17/ -o locale_test.dylib locale_test.cpp -g -lodbc
$ brew install python3

Python2

It works. It shows an usual error message and it shows either Hello by print("Hello").

$ python -c '
import ctypes
try:
    ctypes.cdll.LoadLibrary("locale_test.dylib").test()
except Exception:
    pass
print("Hello")
'
[     ] [Microsoft][ODBC Driver 17 for SQL Server]Neither DSN nor SERVER keyword supplied (0)
Failed to connect
[     ] [Microsoft][ODBC Driver 17 for SQL Server]Neither DSN nor SERVER keyword supplied (0)
Failed to select the data
Hello

Python3

It doesn't work. Suddenly abort happen.
(It omits print("Hello") in spite of the error happen in try-except clause.)

$ python3 -c '
import ctypes
try:
    ctypes.cdll.LoadLibrary("locale_test.dylib").test()
except Exception:
    pass
print("Hello")
'
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: collate_byname<char>::collate_byname failed to construct for C/en_US.UTF-8/C/C/C/C
Abort trap: 6

ODBC Trace log

[ODBC][7198][1528908082.055555][__handles.c][460]
		Exit:[SQL_SUCCESS]
			Environment = 0x7f89e2857000
[ODBC][7198][1528908082.061593][SQLSetEnvAttr.c][189]
		Entry:
			Environment = 0x7f89e2857000
			Attribute = SQL_ATTR_ODBC_VERSION
			Value = 0x3
			StrLen = 4
[ODBC][7198][1528908082.061824][SQLSetEnvAttr.c][381]
		Exit:[SQL_SUCCESS]
[ODBC][7198][1528908082.062025][SQLAllocHandle.c][377]
		Entry:
			Handle Type = 2
			Input Handle = 0x7f89e2857000
[ODBC][7198][1528908082.062167][SQLAllocHandle.c][493]
		Exit:[SQL_SUCCESS]
			Output Handle = 0x7f89e2857600
[ODBC][7198][1528908082.068684][SQLDriverConnectW.c][290]
		Entry:
			Connection = 0x7f89e2857600
			Window Hdl = 0x0
			Str In = [driver={ODBC Driver 17 for SQL Server}][length = 38 (SQL_NTS)]
			Str Out = 0x0
			Str Out Max = 0
			Str Out Ptr = 0x0
			Completion = 0
		UNICODE Using encoding ASCII 'UTF-8' and UNICODE 'UCS-2-INTERNAL'

Crash Report

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called
terminating with uncaught exception of type std::runtime_error: collate_byname<char>::collate_byname failed to construct for C/en_US.UTF-8/C/C/C/C

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff7aebcb6e __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff7b087080 pthread_kill + 333
2   libsystem_c.dylib             	0x00007fff7ae181ae abort + 127
3   libc++abi.dylib               	0x00007fff78d13f8f abort_message + 245
4   libc++abi.dylib               	0x00007fff78d14113 default_terminate_handler() + 241
5   libobjc.A.dylib               	0x00007fff7a154eab _objc_terminate() + 105
6   libc++abi.dylib               	0x00007fff78d2f7c9 std::__terminate(void (*)()) + 8
7   libc++abi.dylib               	0x00007fff78d2f47a __cxa_rethrow + 99
8   libc++.1.dylib                	0x00007fff78ce87e1 std::__1::locale::__imp::__imp(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long) + 2117
9   libc++.1.dylib                	0x00007fff78ce9e06 std::__1::locale::locale(char const*) + 186
10  libmsodbcsql.17.dylib         	0x0000000108ee8f59 SystemLocale::SystemLocale(char const*) + 41
11  libmsodbcsql.17.dylib         	0x0000000108ee76d8 SystemLocale::Singleton() + 56
12  libmsodbcsql.17.dylib         	0x0000000108e0bca9 unsigned long SystemLocale::FromUtf16<ArrayTAllocator<char> >(unsigned int, wchar_t const*, long, char**, bool*, unsigned int*) + 41
13  libmsodbcsql.17.dylib         	0x0000000108e0bba1 LoadResourceDLL(wchar_t const*, void*, void**) + 97
14  libmsodbcsql.17.dylib         	0x0000000108e0ba55 InitializeSharedModules() + 101
15  libmsodbcsql.17.dylib         	0x0000000108e7c14b DvrInit() + 267
16  libmsodbcsql.17.dylib         	0x0000000108e7b229 ExportImp::SQLAllocEnv(tagENV**) + 25
17  libmsodbcsql.17.dylib         	0x0000000108e7b07d SQLAllocHandle + 109
18  libodbc.2.dylib               	0x0000000108daad08 __connect_part_one + 2120
19  libodbc.2.dylib               	0x0000000108dca28a SQLDriverConnectW + 1307
20  locale_test.dylib             	0x0000000108da0d46 test + 246 (locale_test.cpp:75)
21  _ctypes.cpython-36m-darwin.so 	0x0000000108d47457 ffi_call_unix64 + 79
22  _ctypes.cpython-36m-darwin.so 	0x0000000108d47bfb ffi_call + 703
23  _ctypes.cpython-36m-darwin.so 	0x0000000108d435a3 _ctypes_callproc + 662
24  _ctypes.cpython-36m-darwin.so 	0x0000000108d3e225 PyCFuncPtr_call + 977
25  org.python.python             	0x00000001084046e3 _PyObject_FastCallDict + 143
26  org.python.python             	0x00000001084a30c6 call_function + 443
27  org.python.python             	0x000000010849b631 _PyEval_EvalFrameDefault + 1659
28  org.python.python             	0x00000001084a3876 _PyEval_EvalCodeWithName + 1747
29  org.python.python             	0x000000010849af3c PyEval_EvalCode + 42
30  org.python.python             	0x00000001084c3acf run_mod + 54
31  org.python.python             	0x00000001084c2ade PyRun_FileExFlags + 164
32  org.python.python             	0x00000001084c21c9 PyRun_SimpleFileExFlags + 283
33  org.python.python             	0x00000001084d6faa Py_Main + 3466
34  org.python.python             	0x00000001083f6e1d 0x1083f5000 + 7709
35  libdyld.dylib                 	0x00007fff7ad6c015 start + 1

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

@kitsuyui Thank you so much for attaching more clues and reports. That's really helpful :)

By following your repro steps, I can get the aborting error on my Mac when using python3 with shared library. That's right. There's no errors when using python2 and executable.

From the crash report, line 10:
10 libmsodbcsql.17.dylib 0x0000000108ee8f59 SystemLocale::SystemLocale(char const*) + 41

When the SystemLocale singleton is initialized, there's a setlocale(LC_ALL, NULL) call. This is to query the name of the current locale. The return value of setlocale() will be passed to the constructor of std::locale . I tried to put the following line at the beginning of the test() in the cpp file to see the result:

printf("setlocale() : %s\n", setlocale(LC_ALL, NULL));

When I set the locale to en_US.UTF-8, I get the following output.

This is the result with python2 :
setlocale() : C

This is the result with python3 :
setlocale() : C/en_US.UTF-8/C/C/C/C

This is the result with executable only :
setlocale() : C

When I set the locale to C, the result with python3 changed:

setlocale() : C

I tried it on Linux with python3 and get this:
setlocale() : LC_CTYPE=en_US.UTF-8; LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C

It seems that python3 somehow breaks the behavior of the old setlocale(LC_ALL, NULL) function on Mac.

In order to make your code work properly in python3 with non-C environment, it's better to explicitly add something silimar to
setlocale(LC_ALL, "")
at the beginning in your code and it will get the correct current locale from the environment variable.

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou

printf("std::locale name : %s\n", std::locale(setlocale(LC_ALL, NULL)).name().c_str());

Obviously, the std::locale doesn't like slash string on Mac.

Wow, that's scary!

That's great the new version will be released. I'm looking forward to it.
Thank you.

from homebrew-mssql-release.

karinazhou avatar karinazhou commented on May 22, 2024

Hi @kitsuyui ,

We are pleased to let you know that a new version of ODBC Driver 17 for SQL Server has been released. The new version 17.2 contains the fix for the locale issue. Please feel free to try the new driver :)

Thanks,

from homebrew-mssql-release.

kitsuyui avatar kitsuyui commented on May 22, 2024

@karinazhou

It works fine. I will close this issue.

Thank you very much!!!

from homebrew-mssql-release.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.