Data Copilot: My goal is to create an assistant that helps me quickly access to product data. Why? There are two reasons:
-
- I need rapid, anytime access to 360-degree information of feature performance & operation (I am the Product Manger)
-
- I have worked directly with ChatGPT and encountered several limitations:
- ChatGPT does not allow me to handle Google Sheets directly
- Uploading and processing data with ChatGPT is a costly (tokens), time-consuming process, and sometimes the risk of having to share data is inevitable --> I need a cheaper, more secure solution
During my research, I discovered PandasAI (a fantastic library, which I recommend to anyone with similar goals) and to help you quickly get started, I created a demo project on Streamlit. You can freely download and explore it.
- How to Combine PandasAI + ChatGPT + Streamlit
- Streamlit: allows you to build chat interfaces very quickly
- Basic techniques with PandasAI
- PandasAI only sends metadata instead of the full data set -> no worries about data leak risks
- How to handle the response and display it on Streamlit
- How to use ChatGPT and PlanUML to draw ER Diagrams (I use it to check if ChatGPT really understands the relationships between data tables, as a misunderstanding could lead to incorrect processing)
- How to work with Google Sheets
You have to create two environment variables to store the OPENAI_API_KEY and the Credentials information when connecting to Google Sheets before running
export OPENAI_API_KEY="sk-iOZXf..................gqObIX5cd8Z"
export GS_ACCOUNT_JSON='{"type":"service_account","project_id":"...........,"universe_domain":"googleapis.com"}'
- Update after I have more time and further develop my idea
![image](https://private-user-images.githubusercontent.com/31585927/323473657-fc70141c-21e0-46fb-b114-a6e090d67f93.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4MTgyNTAsIm5iZiI6MTcyMzgxNzk1MCwicGF0aCI6Ii8zMTU4NTkyNy8zMjM0NzM2NTctZmM3MDE0MWMtMjFlMC00NmZiLWIxMTQtYTZlMDkwZDY3ZjkzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDE0MTkxMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFiNGU3NDU2NTJjMTdkYTQzMjQyZDM0OGIxYzY0MjM5OTQxZTlhZjU5M2RhYWU4MjVkMjY3OGQxM2Y1YTA4MWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.W5uFvNcoIFP84EqxDaZzkfJYkD53IobADniLr8eGmOQ)