Giter Club home page Giter Club logo

excelerator's Introduction

Excelerator: Use Snowflake in Excel

Excelerator is an Excel Add-In to help you pull data from Snowflake into Excel and push new or updated data from Excel into Snowflake. Excelerator is only compatible with the Windows operating system, not the MacOS. View a demo of Excelerator and see it upload over 1 million rows (77MB).

Example use-cases include:

  • financial budgeting write-back,
  • look-up table maintenance,
  • pricing analysis,
  • and more!
Excelerator is not a supported product by Snowflake or any company. Excelerator will write data to the Snowflake database and should be used with great care. Use at your own risk.

Get Started with Excelerator

Important: When following the instructions below, make sure to 'Unblock' the Addin after downloading. The details are in the link located in Step 2 below.

To get started, you'll need to:

Install the ODBC Driver

The Excel Add-In requires the ODBC driver. To download the driver, go to: https://sfc-repo.snowflakecomputing.com/odbc/index.html Select the version of the ODBC driver, 64-bit or 32-bit, based on the local Excel installation. Make sure it matches or the Add-In will not work. Once downloaded, install the driver.

Install Excelerator

Step 1: Set Required Privileges

Below are the privileges required for each capability.

To query data:

Object Privilege
Database USAGE
Schema USAGE
Table SELECT

*Stage can be provided in the login instead

To upload data requires everything in query, plus:

Object Privilege
Schema CREATE TABLE,CREATE STAGE
Table INSERT, UPDATE, TRUNCATE

To rollback data requires the schema privileges from upload, plus:

Object Privilege
Table Ownership

Optional The following script will create a new role with the proper privileges, except the table level privileges: SnowflakeExcelAdd-In_Create_Role.sql You’ll have to update the script before executing it with the information specific to your environment. In the script, you will be providing the role of the user that will be using this Add-In. The script will assign the new to the existing role, which will inherit all the privileges defined in the script.

Step 2 – Install Excel Add-in

There are 2 versions of the Excel Add-in, one for only reading data from Snowflake and one for reading and writing data to snowflake. The Excel Add-ins are Excel files with an extension of ".xlam". The .xlam file for the read-only version is called "SnowflakeExcelAdd-InReadOnly.xlam". The full read-write version is called "SnowflakeExcelAdd-In.xlam". These files are both stored in the repository. In order to install these add-ins, follow the instructions here: https://exceloffthegrid.com/install-uninstall-excel-add/.

The Excelerator is now available on the Home tab of the Ribbon.

Use Excelerator

With Excelerator installed, now you need to connect it to Snowflake.

First, confirm you can find the appropriate buttons within the "Home" tab.

Connection Parameters

Click the connect button and enter your Snowflake connection information. There are two security types standard: login/password and SSO. We recommend using SSO for more secure authentication.

The user's default role and warehouse will be used. If the user does not have a default warehouse, they will be prompted to enter one. Both the role and warehouse can be changed by clicking on the 'Config' menu item in the Excel ribbon.

Overview

Now we'll walk you through each section of the Add-In. You can work with data sourced from other data sources within Excel (such as files) or you can pull data from Snowflake into Excel. You can manipulate the data using VBA scripts and vlookups, but be sure to write that final data back to Snowflake! The Add-In handles both reading and writing scenarios.

Execute a Query

Pull data into Excel by selecting the “Query" button. A search dialog will open that allows you to select a database, schema and table in order to download data. Once you select a table you can then choose which columns you would like to reutrn from the selected table. You can select all the columns by clicking the "All" button, or select a subset of columns by clicking the "Choose" button. Once selected, a SQL statement is created and entered in the bottom text area. This SQL statement can be manually updated. Click the "Execute" button to execute the query in Snowflake and pull down the result set.

You'll get results that look similar to this:

Repeat as-needed to gather data from Snowflake into your Excel sheet.

Write Data to Snowflake

To write data to Snowflake, click the "Upload" button in the ribbon.

Select Upload Table

In the top section, choose the database, schema and table to upload the data to.

Basic Upload section

There are 3 basic ways to upload data into a table:

  • Update existing rows & Append new rows
  • Append data - This will insert all the data into the table.
  • Replace all data in the table with the data from the spreadsheet - All the data will be deleted from the table and then the new data will be inserted.

When selecting the first option, you will have to define the columns that represent the table key. The key defines how each row is unique, and is used to update existing rows. This can be done by entering the column position, for example: A,B,C. In this case the first 3 columns in the excel sheet and target table will be the key.

With any of the above options, if a column exists in the spreadsheet, but does not exist in the table, it will be added automatically to the Snowflake table. The data type can be specified by clicking the "Define Data Types" button in the ribbon. If you prefer to have Snowflake determine the data types, select the "Auto-generate data types" in the "Advance Options" section below.

"Advanced Options" section

This section allows more options:

  • Create a new table - This will create a new table with the columns specified in the first row.
  • Recreate and existing table - This will drop the table and recreate the table with the columns specified in the first row.

Specifying Data Types

For any new columns, the data type needs to be manually specified by clicking the "Define Data Types" button in the ribbon. A new row will be created in the first line of the spreadsheet containing a drop down of the Snowflake supported data types. Choose the appropriate data types for each column.

🔧 Limitations
Timestamps don’t handle anything less than seconds. To get around this, manually cast the time to a varchar in the SQL.

excelerator's People

Contributors

adamd avatar jdanielmyers avatar lukegalbraithrussell avatar sfc-gh-awong avatar ssegal100 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

excelerator's Issues

Support encryption key size 256

After we have updated CLIENT_ENCRYPTION_KEY_SIZE to 256 (from 128 default) users were not able to decrypt data from stage when using Excelerator. All other tools work well. According to documentation change could have affected only JDBC drivers, not ODBC.
the actual error message from SF logs is: "Failed to decrypt. Check file key and master key."
Excelerator can still connect and read data, but not write/update.
Could you please look at that? I tried searching through the VBA but could not find anything which would help.

Thank you

Miro

Issue Logging in

when logging into snowflake; Excelerator first runs these commands bellow.

show databases;
WITH databases (a,name,b,c,d,e,f,g,h) as (select * from table(result_scan(last_query_id()))) select name from databases;

it seems there are 10 columns now when declaring "show databases". I am unable to use excelerator and I am getting the error "Mismatch between number of columns produced by 'DATABASES' and the number of aliases specified."

Connection via external browser

Hello everyone,

Thank you for the great project. I was wondering if there is a way to connect to Snowflake via an external browser.
I appreciate your support.

Greetings,
Aymen

Cannot Clear Excelerator AddIn from Excel

The AddIn was installed without checking the unblock property. When this was attempted to be corrected the excel addin cannot be removed from excel. Excel "keeps" finding it even after deleting all the excelerator files and unchecking the addin from the excel configuration. Excel gives the message to remove the addin. We click ok to remove it and uncheck the addin from the list. The addin is not listed. Excel is shutdown. the laptop rebooted. When Excel is started again, it finds the configuration for the previous addin. No matter what we do we cannot get rid of that reference.

We tired restoring the addin to the location excel is looking for and now excel is really acting weird. It is displaying double addin icons and throwing errors. See the attached screenshots.
Snowflake errors.xlsx

Can't find project or library error

Hi - while trying to install the add-in, I receive the "Can't find project or library error" in relation to this library: Dim mdConnections As New Scripting.Dictionary.

Please see attached image.
Excel Add-in 1
Snowflake Add-in 2

Excelerator as COM Add In

Hi Team

Can you please let us know if the Excelerator has COM Add in version instead of Excel Add in

Video demo

Just checking if you have posted a video of the Excelerator demo. I'd like to share it with some of my customers.

Craig Warman

  • Snowflake SE

Readme Link for ODBC driver doesn't contain 64bit windows download

Hey there,

Was trying to walk through the readme, and clicked on the link for the ODBC driver. For some reason, there's no 64 bit driver for Windows. I tried using the 32 bit one, which didn't appear to work. After a google search, I landed on https://developers.snowflake.com/odbc/ which did get me the 64 bit driver, and I appear to be properly connected now. I would have opened a PR, but not sure exactly how the readme should be updated, as the current link seems to provide more potential choices for non-windows than the link I used.

Invalid UTF8 Detected in string when using data sourced from SQL Server 2008R2

Data enters Snowflake via Fivetran using the SQL Server connector. Source SQL Server Collation is SQL_Latin1_General_CP1_CI_AS. Querying using the connector is fine, but when I upload, I receive an Invalid UTF8 error; I believe due to the source systems collation.

I have a named stage with a defined file format for Excelerator. I've set the following parameters in my file format with no success:

replace_invalid_characters = true
validate_utf8 = false

I created and populated a table with all the values enclosed in double quotes and set that parameter on my file format but receive the same error.

All works fine when I use data funneled from another cloud source such as Salesforce.

How to build + contribute?

Hi, I'd like to be able to build from source and possibly create a PR for improving

I cloned the repo but since there isn't a VS solution, I'm not sure how to compile?

Would appreciate build instructions so I can contribute

Thanks

No access to CREATE STAGES

Can Excelerator be modified to optionally use the user stage (@~) instead of creating a stage to upload data?

"SSL peer certificate or SSH remote key was not OK" error occurred

Hi,

I've tried to use the Excelerator Add-in today, but when I connect to snowflake server, following error occurred. Does anybody have an idea how to resolve following error?
image

OS: Windows10 Pro
Excel: Office 2016 (64bit)
ODBC Driver: snowflake64_odbc-2.22.2.msi

Best Regards,

Character encoding mismatch when uploading data

When uploading data that contains non-ASCII characters, such as ä (C3 A4 in UTF-8, E4 in Windows-1252), Snowflake returns an error complaining of invalid UTF-8.

I have no experience with Excel VBA. My guess would be that either Workbook.SaveAs needs the FileFormat:=xlCSVUTF8 option or Snowflake's FILE_FORMAT needs the ENCODING = 'WINDOWS1252' option. I recommend something like the former if possible, in order to support characters not found in Windows-1252. I also recommend testing emoji and supplementary ideographs, as some Unicode implementations stumble over characters outside the basic multilingual plane.

wb.SaveAs fileName:=fileName, FileFormat:=xlCSV, CreateBackup:=False

Does not allow to change the snowflake role

The role in snowflake is changed and it does not reflect in the excel. When I try to edit in config it does not give an option to edit the new role or it does not fetch the new role in snowflake.

Snowflake Add-In Querying Problems

Hello,

I am having issues with the Snowflake Excel Add-In and was wondering if anyone has any idea what the problem is?

When querying Snowflake using the Excel Add-In I am having issues retrieving data. When I execute the SQL for a table/view it returns no data, no matter how long I wait. However if I then click the Connection icon twice, the data is then retrieved into Excel. If I then create a new sheet in the workbook and click Query, then the same table of data is retrieved automatically again. If I then change the SQL to execute against a different table/view, again no data is rertieved. If I click Connect the original set of data is retrieved and not the new table.

When I try to change my role or warehouse I get the error "Problem Initializing"

I get the error when I click either link in User Data. When I close the error an Upload Status box appears with the text "Getting Roles..." and it never goes away. I can close it and it give me a dialog box with two dropdowns to select a Role and Warehouse.

If I click OK on the config the debugger shows up along with a compile error "Method or data member not found". The break is on OKButton_Click.

Connection error

Could be my thing only but I keep on having a connection issue due to SSL/SSH (see attached error). ODBC and Snowflake account seem to be fine, any hints?

screenshot

Thanks in advance!

Upload Not Working

Hello,

I downloaded the Excelerator for writebacks on May 16 (for Windows)
The Worksheet Version Number is 7.
The version # in the Configuration popup window is 1.1.19
Excel Version 2208

After modifying a cell and specifying a primary key column, then clicking UPLOAD, the following happens:

image

image

In Snowflake's Activity page I can see the stage is created successfully and the PUT is successful.
I am not able to access the Excelerator Log tab - clicking UNHIDE doesn't reveal it, and when I create the tab myself and run the select & upload my LOG tab doesn't get populated.

Thanks for your help!

Add In gets Removed when Excel is closed and reopened

The latest version of the AddIn (SnowflakeExcelAddin.xlam) does not persist between closing and reopening Excel (Version 2008 Build 13127.21216). The menu items are removed and the addin must be readded via the Addin management window.

JSON(variant) & BINARY data type support

I notice that when ingesting data, VARIANT & BINARY data types are not ingested... is there any workaround or anything you could suggest for being able to do this?

snowflake excelerator error

I installed snowflake excelerator add-in on Excel 365 (32bit). I have the 32bit Snowflake ODBC (v2.22) installed as well.
When try to connect to Snowflake, I receive the curl error code 3, "url using bad/illegal format or missing URL" oscode 9, "Bad file descriptor".
I am using Windows 10 Ent edition.
Any help is appreciated

Error on SHOW GRANTS When Uploading Data to Snowflake

Symptom:
I am able to connect and get a list of databases, but when I attempt to upload a small table to Snowflake, I get the following error:

"SQL Compilation Error:
Mismatch between the number of columns produced by "SHOWGRANTS" and the number of aliases specified."
Excelerator Error

Last statement executed on Snowflake:
with ShowGrants (created_on,privilege,granted_on,name,granted_to,grantee_name,grant_option,granted_by) as (select * from table(result_scan(last_query_id()))) select 'grant '|| privilege|| ' on table ' || '"DB_NAME"."DBO"."TEST2234100_BackupForExcel"' || ' to '|| granted_to ||' ' || grantee_name || IFF( grant_option='true' , ' with grant option' ,'' ) from ShowGrants where privilege <>'OWNERSHIP' order by created_on desc

System Info:
Excelerator: The latest version on the Main branch as of Today
Windows 10 Pro Version 10.0.19045
Microsoft® Excel® for Microsoft 365 MSO (Version 2306 Build 16.0.16529.20100) 64-bit

TimeStamp

Hi,
When I load data I am facing issue if the data type is Timestamp in the table ...
I teseted with mutilple tables I am unable to load data is data type is Timestamp
Any idea why this is happening
Warm Regards,

Manually set http proxy

Hi team !
It's working to specify a proxy in ODBC connection but do you have guideline to manually set proxy in Excelerator ?

Connect Cancel Button Cause Connect Error Going Forward

When using the SnowflakeExcelAddin.xlam Add In, if I click Connect and then Cancel. Every time afterwards I receive an error "ERROR: Problem connecting: SQL compilation error: Object does not exist, or operation cannot be performed."

This did go away once after my computer went to sleep over night but I was able to recreate by clicking Cancel from the Connect Window.

Excelarator Add-in crashing when trying to use upload feature

When I try to use Excelerator upload feature, it crashes my excel and reloads it. I have traced it back and it goes as far as creating a temporary stage and the put command in snowflake. but doesn't really put the file in the temporary stage or load the data to the snowflake table.

Upload error to new table in schema with quoted, lowercase name containing "/"

Hi,

I'm having issues uploading table data to a new table in a database and schema with a case-sensitive, lowercase quoted name. The names of the database and schema also contain the "/" character, I'm not sure if this is relevant.

The error happens after the "create or replace stage ... " statement is issued to Snowflake.

Upload of the same data succeeds to a database/schema with a case-insensitive, unquoted name.

I'm using version 1.1.10 of the add-in

Could you identify and eliminate the cause? We have a lot of case-sensitive, quoted database object names which we can't change.

Thanks,
Daniel

Question on "Update existing rows & Append new rows"

Hi,

This is not an issue but I need some clarification and guidance. Let me know if you have a forum where I can ask this if this is not the place.

Suppose I download 10 rows of data from table and then I add another row. I then upload it using the option "Update existing rows & Append new rows".
My understanding is that it will append the new row and also overwrite the existing 10 rows of data in the table in snowflake.
Is there a way to just append the new row of data and keep the remaining rows of data in snowflake table as they are?

Also if the table has a column set to autoincrement then is there a way to push data from Excelerator and let snowflake deal with incrementing the value in the column? I do not want to force my end users to increment and add id value while uploading the data.

Regards
PK

Excelerator stopped returning data

Hi all.
Three of us were testing the tool and data retrieval was working as expected the first day (we are testing the read-only add-on). Next day, two of us have been experiencing these issues which have not been resolved:

  • The connection to Snowflake works initially (using SSO) and they're able to select the schema, table, and columns. When they hit execute, the "Execute SQL" dialog closes and they're back on the current sheet, but data is not retrieved. After waiting some time and no data, one of two things happens: if they click back on either the Query or Connect options, the data will show up in the background on the sheet, as if it was paused and only continues once you select one of the Excelerator options again. Or, the other person still saw no data at all returned no matter what options are selected.
  • Also, the Config options look corrupted (it had all fields filled with "E" which was one of the data result column values)
  • Could something be happening wrong with the hidden worksheets for the config items?
    We compared our Trust Center options, but no differences were found. The 3rd person was able to continue getting data with various queries without issue and could not duplicate the same issue.
    Any ideas?

Cannot change the role

the default role does not have any warehouse attached to it. want to change the default role, but it popped up the warehouse list which is empty.

Inserting data does overrides columns with defaults or autoincrement set.

If I insert a row and leave the Primary key field empty since it is supposed to autogenerate the id on insert and\or I leave another column with the default set to CURRENT_TIMESTAMP(). Neither the ID or the date gets set. They end up being set to empty strings. Without the key this also makes deletes a bit of an issue and the user will need rules to add additional key fields.
Great Addin though. I'll just have to add some instructions for the users to look up the last ID and increment it by 1 and type in the date and time for the CREATE DATE field.

Cal SP

Hi,
Can you please guide me by letting me know if there is a way by which I can excute Call stored procedure from the Plug in
Regards,
G

Toolbar options revert to read-only ("upload" button disappears)

The add-in keeps reverting to the "read only" version. That is, even though I downloaded the .xlam file for the full version of the add-in, excel only displays the "Query" button in the Home toolbar, not the "Upload" button.

I haven't found any pattern to when this happens or steps to reproduce - except that it inevitably does after a couple uses. This has occured on two different machines.

MicrosoftTeams-image (1)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.