technoculture / echoes Goto Github PK

• Add OG image
• Add metadata for title

The OG should use the title of the chat, echoes logo as the the thumbnail, OG image generated by vercel/og. The description could be generic with the format:

“Confidential conversation. Intended for sharing within the respective organisation only.”

• Override style for rendering of html tags within the rendered markdown text (react-markdown, remark-gfm)

Note, we need to have styling that matches or directly uses the styling in shadcn components for visual consistency

Clerkjs leads to a sluggish UX due to the way it handles fetching user info.

Alternatives

• NextAuth: https://youtu.be/famNru_i7ao
• https://kinde.com/docs/developer-tools/nextjs-sdk/

• Continuation Token
• Setting up Qdrant DB again from S3

• Create a branch with tursodb based app
• Migrate the data from planetscale to turso at night for complete db switch

• Replace organization switcher having manage account feature same
  
• Replace user profile components having manage account feature same
  
• allow user to change user preferences (language etc.)
• allow org admin to add/remove from knowledge base

Add pagination for cards list

Echoes uses tldraw to provide diagramming. Add visual chat to it using GPT-4V

• improve performance as much as possible

• Navigation Menu
• Add pagination
• Changing order of the audio's (dragging to change order)
• Removing from queue
• Play button at the card level summarizes the chat and starts playing the summary

• Integrate app's s3 bucket for file uploading inside a chat
• Show a list of files using shadcn data table (Modal)
• Allow users to use one or more of company’s files in the chat

NOTE:

• If there are files inside folders, just show each file in a flat table like (foldername/subfoldername/filename.pdf) instead of creating a hierarchical file browser

• Originally it was planned to use uploadthing to simplify development, however given the data is confidential it cannot be stored in an external service.

Some useful blogs

• How To Upload a File to S3 With Next.js
  
• Multi-file Uploads Using Next.js 13 Serverless Functionality, Express 4, and Amazon S3 Pre-signed URLs in TypeScript and Node.js 18
  

• https://docs.elevenlabs.io/api-reference/text-to-speech-websockets
• https://github.com/kevinamiri/elevenlabs-react-example

• By default Echo(@ Echo) is tagged without writing an @
• once someone start's writing @ to tag people's, a suggestion list should appear the top of the input bar to choose from
• suggestion list should have all the users of that particular organization

—-

https://upstash.com/blog/realtime-notifications

Currently the playlist is based on drawer, it should be refactored for it to be based on a drawer instead when the user is on a smaller screen such as mobile (sm).

https://www.youtube.com/watch?v=t3Qzo-hgEFc&t=296s

• #40
• Add component for RNA viz

• Implement parallel routes
• use multiple slots in layout components

App Like Usability when Offline

• App Icons, status bar (https://github.com/elegantapp/pwa-asset-generator)
• Pre-cache last 25 chats
• Cache all routes for 3 months
• Cache Audio Files and Clerk User Images (https://ducanh-next-pwa.vercel.app/docs/next-pwa/offline-fallbacks)

Mobile Friendly UI

• Toast for offline (https://ui.shadcn.com/docs/components/toast), (https://dev.to/alexgurr/turning-a-react-app-into-a-pwa-with-offline-detection-service-workers-and-theming-142i#chapter-3)
• Floating Tab bar with Search, Playlist and New Chat Button
• Disable send and mic on offline, hide new chat when offline
• Pull to refresh (https://github.com/cuongdevjs/pull-to-refresh-react) with offline state
• Hide scrollbars in various places (https://github.com/reslear/tailwind-scrollbar-hide)

Better App Start

• Add a static splash screen
• Refactor routes so that /user and /org are not dynamic routes @PrinceBaghel258025
• The PWA should start at /org @PrinceBaghel258025

Optimisations

• App Shortcut (https://learn.microsoft.com/en-us/microsoft-edge/progressive-web-apps-chromium/how-to/shortcuts)
• https://stackoverflow.com/questions/13334397/white-flicker-between-launch-image-splash-screen-and-app-homepage
• PWA Compat

—-

Additional References

• https://medium.com/@addyosmani/progressive-web-apps-with-react-js-part-3-offline-support-and-network-resilience-c84db889162c
• https://ducanh-next-pwa.vercel.app/docs/next-pwa/getting-started

Components

• Jupyter Kernel Gateway for serving python sandboxes
• a function calling interface to the sandbox

References

• https://github.com/iamgreggarcia/codesherpa/tree/main/frontend/src/components
• Code Interpreter for Bioinformatics: Are We There Yet?
• https://towardsdatascience.com/expose-endpoints-using-jupyter-kernel-gateway-e55951b0f5ad
• https://github.com/jupyter/kernel_gateway_demos

• Assistant id on the org level (default: echoes-orgslug, ex: echo-exrna)
• some kind of evaluation test for knowledge base (allows users to upload csv containing MCQ's to test the reliability of knowledge base

• Adding knowledge base
• use user access permission for allowing to add data to knowledge base

Allow users to speak their queries instead of typing.

• Integrate whisper
• Add UI for audio transcription

• Allow to choose from all the available GPT's when the user initiates chat
• Documentation GPT (supporting special kind of syntax @ for tagging people and echo itself(by default) in the org, #{chatId/title} to reference another chat

Currently the title generation uses too many tokens because it sends the entire chat. Chat titles should ideally be generated after the response to the first chat query response, using just the first chat interaction with a cheap model (GPT3.5 or cheaper (replicate))

• Add sentry to create issues against runtime exceptions in prod
• Add langsmith for LLM/Langchain

• Scroll up and down through chat (anchored at each user question)
• Add floating buttons at bottom right corner of the screen with up and down arrows
• When the up or down button is held pressed for longer than a second, go all the way to top or bottom respectively

• split the chat message in two equal halfs for a certain screen resolution and above
• question (user message) and answer (ai response) should be visible at the same level if the screen is wider than a certain size (ex- 1200px)
• question and it's answer should behave in sticky manner (question should not be moved from visible viewport until it's answer is fully scrolled to the bottom)
• Inputs(questions) having mentions or # tags should be in the center (acc. to screen width)

• Use liveblocks API to add presence indication (A border around avatars)
• Synchronise chat state using liveblocks API

• Replace copy button with a https://ui.shadcn.com/docs/components/dropdown-menu

Include in the dropdown menu:

• Copy (Works same as currently)
• Edit (For both queries and output)
  • On edit, regenerate the corresponding output (not all outputs)
• Regenerate (For outputs only)

• Add and remove from embeddings from ChromaDB on file upload and delete
• Allow for a simple "chat with PDF" flow
• Add a pdf viewer in a split screen layout for “Chat with PDF” flow

References

• https://github.com/react-pdf-viewer/react-pdf-viewer

Move to gpt-4-turbo vision model

Allow prompting on a text selection:

• Are You Sure? ["Is the following statement by you (an AI) true?: "{selection}""] (S)
• Explain This ["Explain the meaning of "{selection}""]
• Elaborate ["Go in great detail, and demonstrate expertise in the internal workings of the concepts involved: "{selection}""] (E)
• Analyse This Critically ["Criticise this viewpoint by presenting facts. Criticism presented should be scientific in nature and spirit. Statement: "{statement}""] (C)
• I Need Examples ["Present some examples of: "{selection}""] (E)
• I Need References ["Provide some references for this: "{selection}""] (R)

https://ui.shadcn.com/docs/components/context-menu

Would speed up loading of dashboard and also allow us to fetch 25 or more cards.

• Automatically switch from gpt-4 to gpt-3.5-16k and from gpt-3.5-16k to gpt-3.5-32k
• If the context size is still exceeded, Show an error UI stating the chat has reached its maximum length

• Server side search for chats
• Improved nextjs caching
• https://ui.shadcn.com/docs/components/command