Summary of the flow - The user initiates a request by posing a natural language question related to the data stored in Snowflake to the LLM. This natural language query is transmitted to Amazon Bedrock (Anthropic Claude in Bedrock), where it is utilized to formulate a SQL query by using the sample prompts as a context. Subsequently, the generated SQL query is executed against the tables in Snowflake to initiate data retrieval . The retrieved data is then sent back to Amazon Bedrock, where it is used to generate a natural language response based on the acquired information. Finally, the Language Model (LLM) delivers a natural language response to the user, leveraging the retrieved data. Get Started - As the initial step, we establish a Snowflake database, configuring an appropriate table schema tailored for the Netflix data we're working with, and proceed to load the data into the table. If assistance is needed, please refer to the provided steps in this link. Snowflake table sample outputGenerate sample input prompts, outputs, and table information that will be referenced in the subsequent Python code. Store this information separately in a YAML file for future reference here. Create AWS CLI profile with access to Bedrock LLMs and setup your python environment in Jupyter notebook. Create the environment reference file to store your snowflake account information variables. Best practice is to store these information in AWS Secrets manager. Load the import the required python, langchain, packages. We will load environment variables using load_dotenv(). Initialise an instance of the Bedrock class, specifying parameters such as credentials profile name, model ID, endpoint URL, AWS region, and enabling verbosity. Additionally, a function named get_snowflake_uri() is defined to construct a Snowflake URI based on specified parameters such as user, password, Snowflake account, database, schema, warehouse name, and role, returning the generated URI. Overall, the code is involved in setting up the necessary environment for language model operations and establishing connections with Snowflake, encapsulating these functionalities in a Python script. #load env variables, setup llm instance profile and snowflake URI

load_dotenv()

llm = Bedrock( credentials_profile_name=os.getenv("snowadmin"), model_id="anthropic.claude-v1", endpoint_url="https://bedrock-runtime.us-east-1.amazonaws.com", region_name="us-east-1", verbose=True )

def get_snowflake_uri(): snowflake_url = f"snowflake://{user}:{password}@{snowflake_account}/{database}/{schema}?warehouse={warehouse_name}&role={role}" return snowflake_url Define a function load_samples() that is responsible for loading SQL examples for few-shot prompting from "netflix_examples.yaml." The function initializes a variable called sql_samples. Using the safe_loadmethod, we load the YAML content into thesql_samplesvariable. Finally, return the loaded SQL examples, providing a means to access and utilize them within the script. #Load the sql examples for few-shot prompting examples def load_samples(): sql_samples = None with open("netflix_examples.yaml", "r") as stream: sql_samples = yaml.safe_load(stream) return sql_samples We have set up a few-shot prompting chain for interacting with a language model and a database. It takes parameters from the yaml and function defines an sample prompt using a template that incorporates input variables such as table information, input, SQL command, SQL result, and answer. It utilise embeddings with the model for local embeddings and establishes anexample_selector` using semantic similarity based on vector db. Then create a few-shot prompt template, incorporating the provided examples and relevant variables. Finally return a SQLDatabaseChain, effectively setting up the few-shot prompting chain with specified configurations, including the use of a query checker and verbose output. def load_few_shot_chain(llm, db, examples): example_prompt = PromptTemplate( input_variables=["table_info", "input", "sql_cmd", "sql_result", "answer"], template=( "{table_info}\n\nQuestion: {input}\nSQLQuery: {sql_cmd}\nSQLResult:" " {sql_result}\nAnswer: {answer}" ), ) local_embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") example_selector = SemanticSimilarityExampleSelector.from_examples( examples, local_embeddings, Chroma, k=min(3, len(examples)), ) few_shot_prompt = FewShotPromptTemplate( example_selector=example_selector, example_prompt=example_prompt, prefix=_postgres_prompt + "Provide no preamble" + " Here are some examples:", suffix=PROMPT_SUFFIX, input_variables=["table_info", "input", "top_k"], ) return SQLDatabaseChain.from_llm( llm, db, prompt=few_shot_prompt, use_query_checker=True, verbose=True, return_intermediate_steps=True, ) Finally an operation to execute an SQL query on Snowflake is defined . It first retrieves the Snowflake URI using the previously defined and initializes a SQL database sample rows and tables. Subsequently, SQL examples are loaded. Then create a few-shot prompting chain with the provided language model and examples. Finally, the function applies the created SQL database chain to answer the input question. The example usage section demonstrates how to utilize this functionality. If the script is executed as the main program, This serves as an illustration of the overall workflow, where a question is processed through the few-shot prompting chain to obtain an answer based on the configured SQL database and language model.

if name == "main": # Sample question sample_question = "List all titles with TYPE show and have SEASON lesser than 4 and contains GENRE comedy?"

snowflake_answer(sample_question)

Output for above query - Entering new SQLDatabaseChain chain... List all titles with TYPE show and have SEASON lesser than 4 and contains GENRE comedy? SQLQuery:SELECT title FROM data WHERE type = 'SHOW' AND seasons < 4 AND genres LIKE '%comedy%' SQLResult: [('Monty Pythons Fliegender Zirkus',), ('High Risk',), ('Cowboy Bebop',), ('Sonic X',), ('Chappelles Show',), ('Gurren Lagann',), ('Black Butler',), ('Boys Over Flowers',), ('Dead Set',), ('Toradora!',), ('Angel Beats!',), ('Monty Python: Almost the Truth (The Lawyers Cut)',), ('Ouran High School Host Club',), ('Iris',), ('DreamWorks Shreks Swamp Stories',), ('Ben & Hollys Little Kingdom',), ('Den-noh Coil',), ('Zig and Sharko',), ('The Cartel',), ('Kung Fu Panda Awesome Secrets',), ('Dreamworks Happy Holidays from Madagascar',), ('Timmy Time',), ('Oscars Oasis',), ('Fated to Love You',), ('Nuevo Rico Nuevo Pobre',), ('Pecados capitales',), ('Pedro El Escamoso',), ('Monty Pythons Personal Best',), ('Together',), ('Black & White',), ('The Prince Who Turns into a Frog',), ('At Dolphin Bay',), ('Pink Zone',), ('Happy Endings',), ('Hunter x Hunter',), ('Lilyhammer',), ('TIGER & BUNNY',), ('Magi',), ('My Babysitters a Vampire',), ('Kurokos Basketball',), ('Smile PreCure!',), ('Justin Time',), ('Meet the Adebanjos',), ('Inborn Pair',), ('Office Girls',), ('Love, Now',), ('Masameer',), ('Drunken to Love You',), ('Yours Fatefully',), ('KILL la KILL',), ('Derek',), ('Lovesick',), ('Zach Stone Is Gonna Be Famous',), ('The Cat in the Hat Knows a Lot About That!',), ('Sam & Cat',), ('The Heirs',), ('The Haunted Hathaways',), ('Pac-Man and the Ghostly Adventures',), ('Dokidoki! PreCure',), ('Incomplete Life',), ('Turbo FAST',), ('Boss & Me',), ('Two Fathers',), ('Mystic Whispers',), ('Ojos in d House',), ('Ash vs Evil Dead',), ('One-Punch Man',), ('Servant of the People',), ('Crashing',), ('Norsemen',), ('Scream: The TV Series',), ('Versailles',), ('Flowers',), ('Love',), ('Wet Hot American Summer: First Day of Camp',), ('Master of None',), ('Easy',), ('The Disastrous Life of Saiki K.',), ('Trollhunters: Tales of Arcadia',), ('March Comes in Like a Lion',), ('Gilmore Girls: A Year in the Life',), ('The Windsors',), ('Flaked',), ('A Very Secret Service',), ('Crazyhead',), ('Lady Dynamite',), ('No Tomorrow',), ('Grizzy & the Lemmings',), ('The Letdown',), ('Molang',), ('Reply 1988',), ('Cinderella and Four Knights',), ('Simon',), ('The Sound of Your Heart',), ('Spotless',), ('Kuromukuro',), ('Puffin Rock',), ('Skylanders Academy',), ('Cant Cope, Wont Cope',), ('Danger Mouse',), ('Haters Back Off',), ('Hello, My Twenties!',), ('Kong: King of the Apes',), ('Bella and the Bulldogs',), ('Unnatural Selection',), ('Netflix Presents: The Characters',), ('Good Morning Call',), ('Beat Bugs',), ('Another Miss Oh',), ('Ask the Storybots',), ('World of Winx',), ('Richie Rich',), ('The Good Cop',), ('My Runway',), ('Dear My Friends',), ('Toon',), ('H20: Mermaid Adventures',), ('Paquita Salas',), ('Три кота',), ('Old Money',), ('Atelier',), ('Chelsea',), ('Real Rob',), ('Miss in Kiss',), ('Trailer Park Boys: Out of the Park: Europe',), ('Kazoops!',), ('Fangbone!',), ('The Miracle',), ('Magic Cellphone',), ('Diamond Lover',), ('My Little Baby',), ('Bibik Bibikku',), ('Derry Girls',), ('Maniac',), ('HAPPY!',), ('Imposters',), ('Santa Clarita Diet',), ('The End of the F***ing World',), ('The Kominsky Method',), ('A Series of Unfortunate Events',), ('American Vandal',), ('Mystery Science Theater 3000',), ('Wet Hot American Summer: Ten Years Later',), ('Everything Sucks!',), ('James Acaster: Repertoire',), ('VeggieTales in the City',), ('Perfume',), ('Cells at Work!',), ('Stranger',), ('Rascal Does Not Dream of Bunny Girl Senpai',), ('Hilda',), ('Friends from College',), ('My Little Pony: Equestria Girls',), ('Insatiable',), ('Sick Note',), ('Teasing Master Takagi-san',), ('Scissor Seven',), ('Dave Chappelle',), ('Great News',), ('Zoids Wild',), ('Hotel Transylvania: The Series',), ('Magic for Humans',), ('Bill Nye Saves the World',), ('Life Sentence',), ('Shes Gotta Have It',), ('The Toys That Made Us',), ('The Standups',), ('Disjointed',), ('White Gold',), ('The Good Cop',), ('Because This Is My First Life',), ('Abby Hatcher',), ('Sisters',), ('Norm Macdonald Has a Show',), ('The House of Flowers',), ('Hot Date',), ('Welcome to the Family',), ('A Korean Odyssey',), ('Cloudy with a Chance of Meatballs',), ('Man Like Mobeen',), ('Meteor Garden',), ('Fallet',), ('The Hollow',), ('Transformers: Rescue Bots Academy',), ('The Fix',), ('The Guest',), ('Thieves of the Wood',), ('Dope',), ('Paradise PD',), ('Welcome to Waikiki',), ('The Hook Up Plan',), ('The Bride of Habaek',), ('The New Legends of Monkey',), ('The Velvet Collection',), ('The Indian Detective',), ('My Shy Boss',), ('Selection Day',), ('Cupcake & Dino - General Services',), ('Daniel Sloss: Live Shows',), ('Free Rein',), ('Bumping Mics with Jeff Ross & Dave Attell',), ('Hi Score Girl',), ('Samurai Gourmet',), ('Lego Jurassic World: The Secret Exhibit',), ('Girlboss',), ('Spy Kids: Mission Critical',), ('Polly Pocket',), ('Pop Team Epic',), ('Standup and Away! with Brian Regan',), ('A Love So Beautiful',), ('The Piano Forest',), ('Super Drags',), ('Bad Guys: Vile City',), ('Busted!',), ('3Below: Tales of Arcadia',), ('Live',), ('My Secret Romance',), ('The Comedy Lineup',), ('Prison Playbook',), ('Marlon',), ('My Sassy Girl',), ('Million Yen Women',), ('My ID is Gangnam Beauty',), ('Champions',), ('100 Days My Prince',), ('The Who Was? Show',), ('Back Street Girls: Goku Dolls',), ('Edgar Rice Burroughs Tarzan and Jane',), ('Robot Trains',), ('My Dead Ex',), ('Kantaro: The Sweet Tooth Salaryman',), ('ReBoot: The Guardian Code',), ('Neo Yokio',), ('Super Monsters',), ('The Degenerates',), ('Motown Magic',), ('Larva Island',), ('Clean with Passion for Now',), ('Ari Shaffir: Double Negative',), ('Relatable',), ('Prince of Peoria',), ('Yummy Mummies',), ('Ashes of Love',), ('Accidentally In Love',), ('The Joel McHale Show with Joel McHale',), ('Best.Worst.Weekend.Ever.',), ('Samantha!',), ('Legend Quest',), ('Trailer Park Boys: Out of the Park: USA',), ('The Curious Creations of Christine McConnell',), ('Heidi, bienvenida a casa',), ('My Only Love Song',), ('Borges',), ('Stretch Armstrong & the Flex Fighters',), ('Dinotrux: Supercharged',), ('Revolutionary Love',), ('Dennis & Gnasher Unleashed!',), ('Take My Brother Away',), ('Jimmy: The True Story of a True Idiot',), ('Smoking',), ('Oh My Ghost',), ('Wrong Kind of Black',), ('All About the Washingtons',), ('Part-Time Idol',), ('The Honeymoon Stand Up Special',), ('A Boy Named Flora A',), ('The Couldve-Gone-All-the-Way Committee',), ('The Many Faces of Ito',), ('Dave Chappelle: Equanimity & The Bird Revelation',), ('Running Man',), ('Julies Greenroom',), ('All Hail King Julien: Exiled',), ('Buddy Thunderstruck',), ('Blazing Transfer Students',), ('Club de Cuervos Presents: The Ballad of Hugo Sánchez',), ('Check The Store Next Door',), ('Pucca: Love Recipe',), ('The Perfect Match',), ('The Break with Michelle Wolf',), ('The Sound of Your Heart: Reboot',), ('YG Future Strategy Office',), ('Russian Doll',), ('After Life',), ('Love, Death & Robots',), ('The Umbrella Academy',), ('Space Force',), ('Dead to Me',), ('I Think You Should Leave with Tim Robinson',), ('Emily in Paris',), ('Tuca & Bertie',), ('Feel Good',), ('The Daily Life of the Immortal King',), ('Living with Yourself',), ('Pokémon Journeys: The Series',), ('I Am Not Okay with This',), ('The Politician',), ('The Untamed',), ('Daybreak',), ('Merli: Dare to Know',), ('DC Super Hero Girls',), ('Its Okay to Not Be Okay',), ('Trinkets',), ('Dorohedoro',), ('Bonding',), ('Lego Jurassic World: Legend of Isla Nublar',), ('Green Eggs and Ham',), ('L.A.s Finest',), ('Almost Happy',), ('Dash & Lily',), ('Blown Away',), ('Medical Police',), ('Start-Up',), ('Middleditch & Schwartz',), ('Teenage Bounty Hunters',), ('GREAT PRETENDER',), ('Kipo and the Age of Wonderbeasts',), ('Valeria',), ('Aunty Donnas Big Ol House of Fun',), ('Historical Roasts',), ('How to Sell Drugs Online (Fast)',), ('Gentefied',), ('Hotel Del Luna',), ('The Baker and the Beauty',), ('Her Private Life',), ('Power Rangers Beast Morphers',), ('Home for Christmas',), ('Rainbow High',), ('Special',), ('Turn Up Charlie',), ('Abyss',), ('Gameboys',), ('Dolly Partons Heartstrings',), ('Julie and the Phantoms',), ('The Naked Director',), ('The Baby-Sitters Club',), ('Romance Is a Bonus Book',), ('No Good Nick',), ('Connected',), ('Love Alarm',), ('Love & Anarchy',), ('#blackAF',), ('Touch Your Heart',), ('The Duchess',), ('Love 101',), ('Hospital Playlist',), ('Power Players',), ('Ashley Garcia: Genius in Love',), ('Family Business',), ('The InBESTigators',), ('AJ and the Queen',), ('Mythomaniac',), ('Twice Upon a Time',), ('Lunatics',), ('Schulz Saves America',), ('Do Do Sol Sol La La Sol',), ('Taj Mahal 1989',), ('Dragons: Rescue Riders',), ('Mr. Iglesias',), ('Mystic Pop-up Bar',), ('Nobodys Looking',), ('Hi Bye, Mama!',), ('Heroes of Goo Jit Zu',), ('The Last Word',), ('Losers',), ('Kota Factory',), ('The School Nurse Files',), ('Hoops',), ('Twelve Forever',), ('The Cabin with Bert Kreischer',), ('Reality Z',), ('Rookie Historian Goo Hae-Ryung',), ('Can You Hear Me?',), ('Trailer Park Boys: The Animated Series',), ('Malibu Rescue: The Series',), ('The Club',), ('Sneakerheads',), ('Search: WWW',), ('I Hear You',), ('Merry Happy Whatever',), ('The Neighbor',), ('The Healing Powers of Dude',), ('Back with the Ex',), ('The Big Show Show',), ('Go! Live Your Way',), ('When the Camellia Blooms',), ('The Search',), ('Shaun the Sheep: Adventures from Mossy Bottom',), ('Mystery Lab',), ('Its Bruno!',), ('Maradona in Mexico',), ('Brews Brothers',), ('Alien TV',), ('Was It Love?',), ('Followers',), ('Social Distance',), ('Huge in France',), ('Felipe Esparza: Bad Decisions',), ('Over Christmas',), ('Mismatched',), ('Rilakkuma and Kaoru',), ('Stay Tuned!',), ('Booba: Food Puzzle',), ('COMEDIANS of the world',), ('How To Ruin Christmas',), ('Trigger Warning with Killer Mike',), ('Masaba Masaba',), ('Until Dawn',), ('The Good Bandit',), ('Astronomy Club: The Sketch Show',), ('Handsome Siblings',), ('Archibalds Next Big Thing',), ('Prank Encounters',), ('The Iliza Shlesinger Sketch Show',), ('Team Kaylie',), ('Put Your Head on My Shoulder',), ('Green Door',), ('GAME ON: A Comedy Crossover Event',), ('The Unremarkable Juanquini',), ('Mighty Little Bheem',), ('Rhyme Time Town',), ('Sakho & Mangane',), ('Bhaag Beanie Bhaag',), ('Hasmukh',), ('A Little Thing Called First Love',), ('Tiffany Haddish Presents: They Ready',), ('Oldsters',), ('Larry Charles Dangerous World of Comedy',), ('Hong Kong West Side Stories',), ('Dino Girl Gauko',), ('Sexy Central',), ('Encounters',), ('Lugar de Mulher',), ('Mighty Little Bheem: Festival of Colors',), ('Six Windows in the Desert',), ('Bangkok Buddies',), ('Pinky Malinky',), ('The Charming Stepmom',), ('Once Upon a Time in Lingjian Mountain',), ('Answer for Heaven',), ('Valentino',), ('Buddi',), ('Ladies Up',), ('The House Arrest of Us',), ('Fary: Hexagone',), ('Mighty Little Bheem: Diwali',), ('The Pentaverate',), ('Murderville',), ('Clark',), ('Hard Cell',), ('The Woman in the House Across the Street from the Girl in the Window',), ('Maid',), ('Lupin',), ('Human Resources',), ('Heirs to the Land',), ('The Cuphead Show!',), ('Cowboy Bebop',), ('Komi Cant Communicate',), ('Tomorrow',), ('Ginny & Georgia',), ('Inside Job',), ('The Chair',), ('Sex/Life',), ('True Story',), ('Anxious People',), ('The Last Bus',), ('Masters of the Universe: Revelation',), ('Maya and the Three',), ('Samurai Rabbit: The Usagi Chronicles',), ('Vincenzo',), ('Smother-In-Law',), ('Pretend Its a City',), ('Hometown Cha-Cha-Cha',), ('An astrological guide for broken hearts',), ('Standing Up',), ('History of Swear Words',), ('Hes Expecting',), ('He-Man and the Masters of the Universe',), ('The Way of the Househusband',), ('Shaman King',), ('Saturday Morning All Star Hits!',), ('Post Mortem: No One Dies in Skarnes',), ('Pretty Smart',), ('On the Verge',), ('The Club',), ('Falling Into Your Smile',), ('Wild Abandon',), ('The Upshaws',), ('Q-Force',), ('Centaurworld',), ('Power Rangers Dino Fury',), ('Our Beloved Summer',), ('Tear Along the Dotted Line',), ('Feels Like Ishq',), ('The Creature Cases',), ('The Crew',), ('Sex, Love & Goop',), ('Thomas & Friends: All Engines Go!',), ('Elite Short Stories: Patrick',), ('Getting Curious with Jonathan Van Ness',), ('Inspector Koo',), ('Back to 15',), ('Battle Kitty',), ('Misfit: The Series',), ('Once Upon a Time... Happily Never After',), ('Penguin Town',), ('Elite Short Stories: Phillipe Caye Felipe',), ('Zero Chill',), ('Action Pack',), ('Finding Ola',), ('Pui Pui Molcar',), ('Generation 56K',), ('Elite Short Stories: Samuel Omar',), ('Call My Agent Bollywood',), ('Kid Cosmic',), ('Framed! A Sicilian Murder Mystery',), ('50M2',), ('Angry Birds: Summer Madness',), ('Russell Howard: Lubricant',), ('Chicago Party Aunt',), ('Sexify',), ('The Time It Takes',), ('That Girl Lay Lay',), ('Alessandro Cattelan: One Simple Question',), ('Everything Will Be Fine',), ('Soil',), ('Why Are You Like This?',), ('Eternally Confused and Eager for Love',), ('Love, Life & Everything in Between',), ('Daughter from Another Mother',), ('Dad Stop Embarrassing Me!',), ('Mad for Each Other',), ('Sharkdog',), ('Country Comfort',), ('Adventure Beast',), ('Transformers: BotBots',), ('Zero',), ('A Tale Dark & Grimm',), ('Your Life is a Joke',), ('Supercães',), ('Hello, Me!',), ('Ridley Jones',), ('Dogs in Space',), ('Taarak Mehta Kka Chhota Chashmah',), ('Ada Twist, Scientist',), ('Racket Boys',), ('Decoupled',), ('Rebelde',), ('Karmas World',), ('Scaredy Cats',), ('We the People',), ('The War Next-door',), ('The Wedding Coach',), ('The New Adventures of Oggy',), ('I Heart Arlo',), ('Magic for Humans Spain',), ('Kayko and Kokosh',), ('The Ingenuity of the Househusband',), ('Juanpis González - The Series',), ('Lets Eat',), ('Creators File: GOLD',), ('Abla Fahita: Drama Queen',), ('Christmas Flow',), ('HQ Barbers',), ('Mighty Little Bheem: Kite Festival',)] Answer:Here are the results:

Question: List all titles with TYPE show and have SEASON lesser than 4 and contains GENRE comedy? SQLQuery: SELECT title FROM data WHERE type = 'SHOW' AND seasons < 4 AND genres LIKE '%comedy%' LIMIT 5;

Finished chain. In conclusion, this blog post we delved into Snowflake data exploration through natural language queries using the Large Language Models (LLMs) hosted on Amazon Bedrock. The seamless integration of RAG proves to be a potent solution and user-friendly approach to harness the power of generative AI for enterprise-level data inquiries. The entire process is exemplified through straightforward Python code, making the implementation accessible to a wide audience. I have also shared the complete code in a GitHub repository and providing a sample output for a specific SQL query related to Netflix data. The contents of this blog reflect my personal experience and approach to solve problem and are not representative of my employers, including Amazon Web Services (AWS). All third-party libraries, modules, plugins, and SDKs belong to their respective owners. Additionally, it is important to note that utilising AWS and Snowflake services may incur costs, and users are advised to be aware of and use these services mindfully and bring down the services spun up during the setup process.