tableau / connector-plugin-sdk Goto Github PK

I noticed that this line in the documentation

tableau.exe -DConnectPluginsPath =C:\tableau_connectors

should not have a space after DConnectPluginsPath.

I was able to get things working with this:

tableau.exe -DConnectPluginsPath=C:\tableau_connectors

Hi,

I am a developer at MaxCompute.

There is a case in TDVT, the test category is calcs_data and the test name is calcs_data.

The SQL statement is:

SELECT `calcs`.`time1` AS `temp_test__665897456__0_`
FROM `odpsdemo_dev`.`calcs` `calcs`
GROUP BY `calcs`.`time1`

The type of calcs.time1 is TIME and in dataset TestV1 this field only contains data like "hh:MM:ss".

But the expected value is:

%null%
#1901-01-31 00:05:57#
#1901-01-31 02:05:25#
#1901-01-31 04:40:49#
#1901-01-31 04:48:07#
#1901-01-31 09:33:31#
#1901-01-31 12:33:57#
#1901-01-31 18:58:41#
#1901-01-31 19:36:22#
#1901-01-31 19:48:23#
#1901-01-31 19:57:33#
#1901-01-31 22:20:14#
#1901-01-31 22:50:16#

The "1901-01-31" part is quite confusing. I guess it may be an internal default date of Tableau. And it can be really hard for MaxCompute to pass this case.

Here is the snapshot of the test result:

Thank you very much for your time!

My airtable connector does not need a server field, because it is a well-known API endpoint. Is there a way to remove this from the dialog?

Name: Bob Looney
Company: Senturus

We're looking to support stored procedures via an odbc based connector plugin. Is this possible?

We see that we can set the capability CAP_CONNECT_STORED_PROCEDURE. Do we then just need to support ODBC's SQLProcedures and SQLProcedureColumns methods in order for this feature to work? Anything else?

I know the plugin drives partly off the underlying connection type, and I didn't think Tableau's ODBC connector supports stored procedures based on this support file (only SAP, SQL Server, Teradata, and Oracle connectors):
https://onlinehelp.tableau.com/current/pro/desktop/en-gb/connect_basic_stored_procedures.htm

Just making sure this is possible before we invest the time and if anything unique to Tableau is needed to make this work. Thanks!

It's unclear what the authentication-options could be. Right now my connector does not actually require a Username, but the only example is UsernamePassword.

Hello,

I'm checking out the connector SDK and trying to build a small custom example. I noticed that the current examples contain very basic connection dialogs. Would it be possible to expose the full set of available options in Tableau? Also, can you add to the repository more complex examples that show how to make dropdowns, how to disable certain options when a particular field value is chosen, how to redirect connection to a web page etc?

Bogdan

Hi,

Like what we can see in Hortonworks Hadoop connector I want to design dialog with a ComboBox choose the authentication type from and each type should have specific fields that will be shown in the dialog if the realted type is chosen. For example if "username and password" type was set on the ComboBox just username and password field should be shown to the user.
Currently the following is my .tcd file contents"
`

<authentication-mode value='ComboBox' />
<authentication-options>
  <option name="UsernameAndPassword" />
  <option name="Username" />
  <option name="No Authentication" />
</authentication-options>
<db-name-prompt value="Database: " />
<has-pre-connect-database value="true" />
<port-prompt value="Port: " default="1234" />

How can I design such a form?

One of our fields is boolean. We'd like to somehow define it in the .tcd file and have it shown as a checkbox instead of a textbox where the user needs to type "true" or "false".

Is there an easy way to understand how a query running through TDVT was formed so I can handle it with the right dialect section?
I have few queries that fail, but I do not know how to map them to the right element in the .tdd file. I am using the full tdd example.

Is there any way to take the username, password, and server from the dialog the TCD creates, then call an API / ODBC function, then update the TCD dialog with choices in a dropdown?

Or is there a way to override the browse action of the database browse button to do something like that?

I want to configure additional parameters in the dialog box. For this, I'm trying to use the extended-dialog option, but it doesn't seem to work:

ConnectionDialogParser::ParseDialog: Error: Error(15,23): element 'extended-properties' is not allowed for content model '(authentication-mode?,authentication-options?,connection-options?,db-name-prompt?,default-ssl-mode?,extended-properties?,has-databases?,has-pre-connect-database?,has-schemas?,port-prompt?,server-prompt?,service-prompt?,show-connection-prompt?,show-db-browse-button?,show-encryption-checkbox?,show-import-ssl-cert-label?,show-service-prompt?,show-ssl-checkbox?,show-uncommitted-data-checkbox?,ssl-requires-certificate?,warehouse-prompt?)' (id: )"}
{"ts":"2019-02-27T12:31:56.603","pid":7548,"tid":"af0","sev":"error","req":"-","sess":"-","site":"-","user":"-","k":"connector-plugin-error","e":{"category":"needs-classification","log-code":"0e9a4f13"},"v":"Error loading connection dialog file (.tcd) for class 'example'. Skipping plugin."}

I am testing a JDBC connector I'm working on and I get mixed results based on the base dialects I'm using in my tests.

According to the documentation here:

My dialect.tdd file header is the following (and the rest of the file is empty):

<?xml version="1.0" encoding="utf-8"?>
<dialect name='Custom'
         class='custom_jdbc'
         version='18.1'>
		 <!--base='SQL92Dialect'-->
   <function-map>

Having the dialect.tdd file without a base (as shown above) my tests output:

Total time: 194.51126885414124
Total failed tests 633
Total tests ran 715

According to the documentation, if there is a base, that dialect will be used, if there is no base then SQL-92 default behavior is used. My second test was adding as base SQL92Dialect:

<?xml version="1.0" encoding="utf-8"?>
<dialect name='Custom'
         class='custom_jdbc'
         base='SQL92Dialect'
         version='18.1'>
		 <!---->
   <function-map>

This dialect is not listed here, but the tests ran and they generated a different output, even though I would have assumed (based on the docs) that the same dialect should have been used and, implicitly, the same output should have been generated:

Total time: 227.23913717269897
Total failed tests 690
Total tests ran 811

As you can see, the number of tests that ran is different, the same can be said about the number of failed tests.

1. is the documentation correct regarding the base dialect fallback?
2. is SQL92Dialect a valid base dialect? If it is, the statement saying SQL-92 default behavior is used means a different base dialect is used by default?
3. in the samples one can find full_dialect.tdd file, but I would be interested in seeing other DBs dialects, for example PostgreSQL90Dialect. Are these available somewhere?

Many thanks.

About You:
Name: Jon
Company: Max compute

Your question:
Hi.

Thanks for providing Hive12Dialect! But with this dialect, it seems some generated SQL statement is not correct.

Here is the dialect we are using:

<?xml version="1.0" encoding="utf-8"?>
<dialect name='MaxCompute'
         class='maxcompute_jdbc'
         base='Hive12Dialect'
         version='18.1'>
  <function-map>
      <date-function name='DATENAME' return-type='str'>
      <formula part='year'>CAST(YEAR(%2) AS STRING)</formula>
      <formula part='quarter'>CAST(CAST((MONTH(%2) - 1) / 3 + 1 AS BIGINT) AS STRING)</formula>
      <formula part='month'>CAST(MONTH(%2) AS STRING)</formula>
      <formula part='dayofyear'>CAST(DATEDIFF(TO_DATE(%2), TO_DATE(CAST(TRUNC(%2,&apos;YY&apos;) AS DATE))) + 1 AS STRING)</formula>
      <formula part='day'>CAST(DAY(%2) AS STRING)</formula>
      <formula part='weekday'>CAST((8 + DATEDIFF(CAST(%2 AS DATE),NEXT_DAY(%2,&apos;SU&apos;))) AS STRING)</formula>
      <formula part='week'>CAST(FLOOR((14 + DATEDIFF(%2, TRUNC(%2,&apos;YY&apos;)) + DATEDIFF(TRUNC(%2,&apos;YY&apos;),NEXT_DAY(TRUNC(%2,&apos;YY&apos;),&apos;SU&apos;)))/7) AS STRING)</formula>
      <formula part='hour'>CAST(HOUR(%2) AS STRING)</formula>
      <formula part='minute'>COALESCE(CAST(MINUTE(%2) AS STRING), &apos;&apos;)</formula>
      <formula part='second'>COALESCE(CAST(SECOND(%2) AS STRING), &apos;&apos;)</formula>
      <argument type='localstr' />
      <argument type='datetime' />
    </date-function>
    <date-function name='DATENAME' return-type='str'>
      <formula part='year'>CAST(YEAR(%2) AS STRING)</formula>
      <formula part='quarter'>CAST(CAST((MONTH(%2) - 1) / 3 + 1 AS BIGINT) AS STRING)</formula>
      <formula part='month'>CAST(MONTH(%2) AS STRING)</formula>
      <formula part='dayofyear'>CAST(DATEDIFF(TO_DATE(%2), TO_DATE(CAST(TRUNC(%2,&apos;YY&apos;) AS DATE))) + 1 AS STRING)</formula>
      <formula part='day'>CAST(DAY(%2) AS STRING)</formula>
      <formula part='weekday'>CAST((8 + DATEDIFF(CAST(%2 AS DATE),NEXT_DAY(%2,&apos;SU&apos;))) AS STRING)</formula>
      <formula part='week'>CAST(FLOOR((14 + DATEDIFF(%2, TRUNC(%2,&apos;YY&apos;)) + DATEDIFF(TRUNC(%2,&apos;YY&apos;),NEXT_DAY(TRUNC(%2,&apos;YY&apos;),&apos;SU&apos;)))/7) AS STRING)</formula>
      <formula part='hour'>CAST(HOUR(%2) AS STRING)</formula>
      <formula part='minute'>COALESCE(CAST(MINUTE(%2) AS STRING), &apos;&apos;)</formula>
      <formula part='second'>COALESCE(CAST(SECOND(%2) AS STRING), &apos;&apos;)</formula>
      <argument type='localstr' />
      <argument type='datetime' />
      <argument type='localstr' />
    </date-function>
    </function-map>
</dialect>

And for test case date.datename.sow.week, DATENAME('week', [date2], 'monday'), the generated SQL statement was:

SELECT  AS `temp_test__499182808__0_`
FROM `odpsdemo_dev`.`calcs` `calcs`

Also, I noticed that in the hive dialect above, the formula part of the two 'DATENAME' functions are identical even through they have different argument list. It seems that the third argument of the second 'DATENAME' function does not present in the formula part at all.

Thank you very much for your time!

Jon
Max Compute, Alibaba

We're trying to make the database name optional on our Connector.

We created the connectionRequired.js file...

We set it up in the TDR file...

But when we view the connector, it is always required / won't let you hit "Sign In" until you type in something.

(after you type a letter in database, the button enables)

Is this a bug or are we missing a trick?

I'm struggling with below error with custom named connector TDVT running some some Test Cases.

"LoadDatasource DataSourceException: Execution of a connector plugin script component exceeded the 3000ms timeout."

I'm using Tableau Main version and TDVT 1.4.4

Can you please help what changes i have to made.

I'm writing a plugin for a JDBC driver that has, well, no chance of being SQL92 compliant. I would like to see an example of an extract-only plugin.

{"ts":"2019-04-28T22:05:20.586","pid":17338,"tid":"41ee7b","sev":"error","req":"-","sess":"-","site":"-","user":"-","k":"connector-plugin-error","e":{"log-code":"0e9acd07","log-source":"needs-classification"},"v":"Class already registered: memsql"}

Describe the bug
We created a custom connector using the Actian JDBC Driver and found that JDBC Connectivity is inconsistent with many "Unable to complete action. An error occurred while communicating with the data source." errors. The connection is already established as it is able to obtain metadata.

Note that the same errors happen with the Generic JDBC option.

A couple of examples where the error occurs consistently:

• data source data preview: When Clicking on the button labeled "Update Now":

Unable to complete action
An error occurred while communicating with the data source.

• when trying to add a calculated field in a worksheet (dropping it in the sheet). The query that is given with the error message runs fine outside of Tableau:

An error occurred while communicating with data source 'my_table (actian)'.

Bad Connection: Tableau could not connect to the data source.
An error occurred while communicating with the data source.

SELECT 'test' AS "Calculation_3586272695231557632"
FROM "actian"."my_table" "my_table"
HAVING (COUNT(1) > 0)

Desktop (please complete the following information):

• OS: Windows (10)
• Tableau Version 2019.2.1

About you:
Name: Cris Ianculovici
Company: Actian

Some of the dialect base classes listed in the documentation make sense, and some do not. For instance; I'm not aware of any database that is a snowflake derivative. It seems like the base classes should be limited to things which would be reasonably extended.

https://tableau.github.io/connector-plugin-sdk/docs/design

It would also be good to have more information on the dialect itself. I'm not sure what each dialect actually provides for each base class, and therefore what I need to override. Is there a way to create a "dialect base class" reference?

I do like the xsd available at:

https://github.com/tableau/connector-plugin-sdk/blob/master/validation/tdd_latest.xsd

... but I would like to see the existing implementations for some common classes

When all was configured according to the document, I run
python3 -m tdvt.tdvt --run test4dmp
to test mysql connector(via jdbc), however, the following error occur

hyperd server version 9.1.0 build version master.0.0.0.5690.r3a6612de
LoadDatasource TableauException:
Unable to establish connection: Data source "MySQL" is not licensed) 无法建立连接: 数据源“MySQL”未得到许可。

any wrong with my config?

What is the supported set of javascript globals that we can use in the connection builder? Is it possible to create other dialogs and make REST API calls from this script?

I am following the same steps as given in the document to achieve connector in tableau server, but i'm getting below error. Please suggest

C:\Users\Administrator>tsm configuration set -k native_api.connect_plugins_path -v C:/tableau_connectors
Logging in as user 'Workstation\Administrator' since no --username flag was given.
Password:
Configuration error: At least one configuration value you specified does not match a known configuration key. This applies to the following keys: '[native_api.connect_plugins_path]'

Use this parameter to override unknown key error: --force-keys

Currently we have to copy the JDBC jars to %ProgramFiles%\Tableau\Drivers. This raises certain issues:

• It requires elevated privileges, which we'd like to avoid.
• It may cause conflicts with other vendors drivers (all drivers in the same classpath)
• We'd like to support side-by-side versions of our driver (e.g. bug fixes or performance enhancements).
  Proposed solution: If a user activates a custom connector, extend the JDBC classpath with that connector's classpath, either via a fixed folder (e.g. a 'jars' folder within the connector), or from some config file within the connector (which should be relative to the connector's location).

In my database the column names of the Staples table use underscore instead of space. I'm using the following LogicalConfig in the .ini file, but the generated Staple tests ignore the underscore requirement:

[LogicalConfig]
Name = tableau_tdvt
tablename = $dsName
tablePrefix = [tableau_tdvt].
tablenameUpper = True
tablenameLower = True
boolUnderscore = True
fieldnameDate_underscore = True
fieldnameLower = True
fieldnameUpper = True
fieldnameNoSpace = True
fieldnameLower_underscore = True
fieldnameUnderscoreNotSpace = True

Since many of us are likely coming from the ODBC connector, one thing that wasn't clear right away is that all the "String Extras" should go into params in the connectionBuilder.js file.

Can you update this page, section 4 to speak more clearly to that for anyone else that runs into this issue?
https://tableau.github.io/connector-plugin-sdk/docs/example

Between the connection dialog options, manifest options, and connectionBuilder options all sounding very similar, its a bit hard to tell where each setting lands.

Hi Logan,
Thanks for publishing detailed article and example provided. I managed to use the Postgres example and build a mysql connector to connect to MEMSQL. I am currently trying to add Kerberos authentication and wanted to check if you could either share example or SDK documentation?

About You:
Name: Jon
Company: MaxCompute

Your question:
Hi,

The test case 'logical.case.null' generates a SQL statements as follows:

SELECT (CASE WHEN ((8 + DATEDIFF(`calcs`.`date1`,NEXT_DAY(CAST(`calcs`.`date1` AS DATE),'SU'))) IN (7, 1)) THEN CAST(NULL AS TIMESTAMP) ELSE `calcs`.`date1` END) AS `temp_test__4257957843__0_`
FROM `odpsdemo_dev`.`calcs` `calcs`
GROUP BY (CASE WHEN ((8 + DATEDIFF(`calcs`.`date1`,NEXT_DAY(CAST(`calcs`.`date1` AS DATE),'SU'))) IN (7, 1)) THEN CAST(NULL AS TIMESTAMP) ELSE `calcs`.`date1` END)

The data type of the field after 'CASE WHEN xxx THEN' is TIMESTAMP while the data type of the field calcs.date1 after 'ELSE' is DATE.

We have tried to run the same query on hive and hive could not accept this query as well.

So, is it possible for us to safely skip this test case? Or if this test case is really important, what should we do to get it passed?

Thank you very much for your time!
Jon

In the API reference under CAP_FAST_METADATA, it says "This capability is available in 9.0 and later." I suggest striking that sentence, since connector plug-ins don't work with versions that far back.

I'm using the following manifest.xml file:

<?xml version='1.0' encoding='utf-8' ?>

<connector-plugin class='myconnector' superclass='odbc' plugin-version='0.0.0' name='MyConnector' version='18.1'>
  <connection-customization class="myconnector" enabled="true" version="10.0">
    <vendor name="vendor"/>
    <driver name="driver"/>
    <customizations>
    ....
    </customizations>
  </connection-customization>
  <connection-dialog file='connection-dialog.tcd'/>
  <connection-resolver file="connectionResolver.tdr"/>
  <dialect file='dialect.tdd'/>
</connector-plugin>

I successfully connect to my data source, that is the calcs tdvt table and I save the .tds file with the 'Add to the saved datasources' option. The saved file does not contain the list of customizations I added in the manifest.xml file. Is this normal? If not, how can it be fixed? @

@lriggs

Hi

Can you please provide sample plugin with Hive authentication mode using SSL. Thanks

I have a fairly new attempt at a plugin and while 2019.1 and 2019.2 Beta load the shipped plugins fine I'm unable to get much debugging information out of the one I built. If the git repro repository is simply cloned from the following repository

https://github.com/jkew/airtable.tableau

I'll get an error about the .git directory:

{"ts":"2019-04-28T22:11:34.574","pid":17463,"tid":"4201b9","sev":"error","req":"-","sess":"-","site":"-","user":"-","k":"connector-plugin-error","e":{"log-code":"3834cf96","log-source":"needs-classification"},"v":"Failed to validate /Users/johnkew/Develop/airtable.tableau/.git"}

Removing that doesn't provide any additional insight into what went wrong though.

I'm using the following command line options:

/Applications/Tableau\ Desktop\ 2019.2.app/Contents/MacOS/Tableau -DConnectPluginsPath=/Users/johnkew/Develop/airtable.tableau/ -DLogLevel=Debug

Started implementing Tableau plugin feature for our named connector. I have the following questions,

1. When I open the saved workbook by double-click on the workbook, Tableau Desktop shows "Tableau doesn't recognize the data source type....". Every time I need to run Tableau using the "-DConnectPluginsPath" command-line argument and open the workbook from there. Is there any solution for that?

2. We have existing workbooks that are all created using "Other Database(ODBC)" with our ODBC connector in the Tableau Desktop. How can we open and work with that existing workbook with the newly developed tableau plugin named connector?

Name: Cris Ianculovici
Company: Actian

The Actian/Ingres query syntax supports the behavior required for CAP_QUERY_TOP_N=yes in the form of
SELECT FIRST n ... FROM ...
How can I get that to work?

Hi,
Started implementing Tableau plugin feature for our named connector. The connection is successful to my data source from my named Plugin and I can able to list the schema and tables. But we have a few problems,

1. I'm unable to see any Search option under the Schema (Select Schema), with that search option we are doing some customized search. So how to get that?

2. On the initial connection dialog for the plugin, how to show the list of DSN under the system?

Thanks.

Hi,
great to see the SDK is now available for public consumption / production. Building a simple connector wrapping an existing ODBC or JDBC driver seems straightforward (we'll be starting that very soon), but we're curious about possibilities to append additional information to the ODBC (or JDBC) catalog content that Tableau automatically retrieves once the connection has been established. More to the point, our database supports some back-end BI modeling capabilities of its own (dimension info, pretty names, ...) and we'd like to be able to leverage that metadata when Tableau builds a .tds file or at least the data source browser in which you build that.
Any suggestions on how we could inject that information are much appreciated.

Thanks,
benjamin

The problem with a command line arg is that the user has to launch tableau with it each time, instead of using the shortcut created by the installation.
We provide a batch script to simply this, which the user can shortcut, but it's a compromise.
Suggestion: if "-DConnectPluginsPath" is not specified, scan for plugins at "%USERPROFILE%\Documents\My Tableau Repository\Plugins" - standard tableau folder which does not require elevated permissions.

Generic TCD types with specified parameters would be useful for building and customizing UI:

String: Custom name, optional default value
Boolean: Custom name, optional default value
Int: Custom Name, optional default value
Enum: Custom Name, custom enum types, optional default value

I have many TDVT queries that their Error Message and Error Type (in the combined output file) say "Error.". They are all of Test Type = "expression".
How do I find what is the issue?

I tun TDVT and have a long section of queries that end with an error -
Tableau was unable to generate a query to perform this operation.

How can I find the details why did that happen?
I am using a .tdd file

Hi Team,
Are we able to customize the initial plugin connection dialog like adding a new textbox and using that value when building a connection string?

The example dialog for "new_text_field" does not appear to work, at least in 2019.2 Beta. The dialog does not pass validation on startup

https://github.com/tableau/connector-plugin-sdk/blob/master/samples/components/dialogs/new_text_field/connection-dialog.tcd

<service-prompt value="Table" />

We have a couple of unique fields in our authentication string that have to be available in the dialog.
IS there a reference or an example of how such thing should be done?

We organized the fields in the .tcd files in a certain way (popular/required fields first, then the rest), but they're not displayed in the same order in the actual dialog.
We'd like them to be displayed in the same order, or alternatively a syntax to enforce order.

Well, someone's got to file the first issue, so why not let it be the obvious question when we'll be able to see the actual SDK on this repo. The timing works really well for us so we're eager to get started.
Or should we really give you guys a chance to get home from New Orleans first? :-)

I can't find a reference to the minimum Tableau version needed anywhere in the docs. One good place to put it could be in the bulleted list under "Before you begin" at the bottom of the Get Started page. If it requires a pre-release version at this point, we might also want to link to the directions for getting a beta.

About You:
Name: Jon
Company: MaxCompute

Your question:
Hi,

I am trying to figure out the root cause of two failed test cases, lod.17_Nesting and lod.16_As Group.

Since you said you didn't get much information from the Tableau log, I checked the log of our db and it seemed everything worked fine. Tabquery did nothing except read the metadata of table 'Staples'. I am afraid that Tabquery failed before executing any test case.

Then I noticed that both of the failed case printed an error message like "Abstract query [field] is not defined".

Here is an example:

Abstract query [ship_mode] is not defined

Then I found a related issue: https://kb.tableau.com/articles/issue/error-abstract-query-is-not-defined-when-viewing-data-in-a-google-bigquery-connection?lang=fr-fr

Is it possible that the failures are caused by this issue?

I have attached the test case, hope that could help.

setup.lod.17_Nesting.odpsdemo_dev_logical.xml.zip

Thank you very much!
Jon

Can we update the "share your connector" page to include a note about the Mac command line path and arguments needed to test the connector?
https://tableau.github.io/connector-plugin-sdk/docs/share

i.e.,
/Applications/Tableau\ Desktop\ 2019.1.app/Contents/MacOS/Tableau -DConnectPluginsPath=/Users/tableau_connectors

My Airtable connector cannot support initial sql right now, or at least it doesn't make sense within the context of this connection. Is there a way to disable it?

The document of TDVT mentioned following properties in "INI file structure" part:

#You can add a new logical config here and use it above. These are example attributes; you wouldn't set them all since some are mutually exclusive.

tablenameUpper = True
tablenameLower = True
boolUnderscore = True
fieldnameDate_underscore = True
fieldnameLower = True
fieldnameUpper = True
fieldnameNoSpace = True
fieldnameLower_underscore = True
fieldnameUnderscoreNotSpace = True

As mentioned in the comment some of the attributes are "mutually exclusive" so we tried to turn some of them to "False".

The confusing part is that it makes no difference to change the value to "False". TDVT code (eg. line 40 of tdvt/tdvt/config_gen/gentests.py) only checks if the corresponding key is present and ignore whatever the value is. The only way to turn the switch off is to remove or comment the line from INI file.

It might be better to clarify this behavior in the comment since the form "key = True" is really misleading.