approximatelabs / datadm Goto Github PK

View Code? Open in Web Editor NEW

378.0 378.0 28.0 11.71 MB

DataDM is your private data assistant. Slide into your data's DMs

License: MIT License

Python 97.79% Dockerfile 0.55% Jupyter Notebook 1.67%

datadm's People

Contributors

Stargazers

Watchers

datadm's Issues

Cost estimation for openai?

Have you made a cost estimation for this running over openai?
Thanks

OpenAI Models don't stream anymore

After an update to guidance, streaming is broken.

See: guidance-ai/guidance#244

For now, we are going to pin guidance to 0.0.61 and release

--> Keeping this issue open to remember to check if we can remove the version pin.

Some Bug

Hi Team !
I'm just start on this and getting this issue when initial your repo

pip install "datadm[cuda]"
datadm

Then getting this issue

/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/gradio/components/base.py:181: UserWarning: show_label has no effect when container is False.
  warnings.warn("show_label has no effect when container is False.")
Traceback (most recent call last):
  File "/home/lthun/anaconda3/envs/wisey/bin/datadm", line 5, in <module>
    from datadm.app import main
  File "/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/datadm/app.py", line 310, in <module>
    results.extend(container.value.gradio_gen(upload_magic_thens))
  File "/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/datadm/app.py", line 157, in gradio_gen
    for obj in obj.gradio_gen(upload_magic_thens):
  File "/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/datadm/app.py", line 115, in gradio_gen
    objs.append(self.component('download', container=False))
  File "/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/datadm/app.py", line 107, in component
    return self.gradios.get(name, {}).get('class', gr.Text)(**kwargs, **extra_kwargs)
  File "/home/lthun/anaconda3/envs/wisey/lib/python3.9/site-packages/gradio/component_meta.py", line 159, in wrapper
    return fn(self, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'container'

Support GGML based Models

In order to do this, I'd like to use ctransformers in guidance

See this issue: marella/ctransformers#13

Build Image Fails

For the non-cuda image. Something about safetensors is failing.

Folder/Directory import support

As A Strava.com Athlete with thousands of recorded activities, I Want To have datadm ingest a folder/directory with all all of those activities in CSV format (one file per activity), In Order To ask it questions about many years worth of activities.

I'm using Strava and they do bulk exports of activities as individual files. Presumably I'm asking for recursive Folder/Directory file loading of data.

OpenAI integration is broken

When trying to use an OpenAI model, no response is returned and DataDM shows "Error" in the middle of the screen.

Logs:

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1352, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1093, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 341, in async_iteration
    return await iterator.__anext__()
  File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 334, in __anext__
    return await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 317, in run_sync_iterator_async
    return next(iterator)
  File "/datadm/datadm/app.py", line 49, in bot
    yield from agent.bot(repl, conversation, model_selection)
  File "/datadm/datadm/agent.py", line 22, in bot
    for conversation in self._bot(repl, conversation, llm):
  File "/datadm/datadm/agents/baseline.py", line 57, in _bot
    for result in precode(conversation=clean_conversation_list(starting_convo), silent=True, stream=True):
  File "/usr/local/lib/python3.10/site-packages/guidance/_program.py", line 251, in _stream_run
    loop.run_until_complete(execute_task) # this will stop each time the program wants to emit a new state
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/site-packages/guidance/_program.py", line 385, in execute
    await self._executor.run(llm_session)
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 106, in run
    raise e
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 102, in run
    await self.visit(self.parse_tree)
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 437, in visit
    visited_children.append(await self.visit(child, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 437, in visit
    visited_children.append(await self.visit(child, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 404, in visit
    command_output = await command_function(*positional_args, **named_args)
  File "/usr/local/lib/python3.10/site-packages/guidance/library/_assistant.py", line 13, in assistant
    return await role(name="assistant", hidden=hidden, _parser_context=_parser_context)
  File "/usr/local/lib/python3.10/site-packages/guidance/library/_role.py", line 14, in role
    out = await parser.visit(
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 437, in visit
    visited_children.append(await self.visit(child, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 437, in visit
    visited_children.append(await self.visit(child, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 226, in visit
    visited_children = [await self.visit(child, next_node, next_next_node, prev_node, node, parent_node) for child in node.children]
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 226, in <listcomp>
    visited_children = [await self.visit(child, next_node, next_next_node, prev_node, node, parent_node) for child in node.children]
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 437, in visit
    visited_children.append(await self.visit(child, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
  File "/usr/local/lib/python3.10/site-packages/guidance/_program_executor.py", line 300, in visit
    command_output = await command_function(*positional_args, **named_args)
  File "/usr/local/lib/python3.10/site-packages/guidance/library/_gen.py", line 155, in gen
    for resp in gen_obj:
  File "/usr/local/lib/python3.10/site-packages/guidance/llms/_openai.py", line 192, in stream_then_save
    for curr_out in gen:
  File "/usr/local/lib/python3.10/site-packages/guidance/llms/_openai.py", line 44, in add_text_to_chat_mode_generator
    c['text'] = c['delta']['content']
  File "/usr/local/lib/python3.10/site-packages/openai/openai_object.py", line 71, in __setitem__
    raise ValueError(
ValueError: You cannot set text to an empty string. We interpret empty strings as None in requests.You may set {
  "index": 0,
  "delta": {
    "role": "assistant",
    "content": ""
  },
  "finish_reason": null
}.text = None to delete the property

Looks like this is due to: guidance-ai/guidance#245

Automatically install libraries

Multliple times I've run into scenarios where DataDM would like to use a library that is not installed, such as when I ask it to plot data on a map. If I ask it to install the library, it complains that it can't, but it does it anyway. It'd be nice if it just installed required dependencies automatically, esp for hosted DataDM which is run in a Docker container such that installing deps is a bit safer.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.