spcl / graph-of-thoughts Goto Github PK
View Code? Open in Web Editor NEWOfficial Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
Home Page: https://arxiv.org/pdf/2308.09687.pdf
License: Other
Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
Home Page: https://arxiv.org/pdf/2308.09687.pdf
License: Other
Dear authors,
When I run the example in Quick Start, I have the following error:
Traceback (most recent call last):
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/quick_start.py", line 32, in <module>
ctrl.run()
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/controller/controller.py", line 77, in run
current_operation.execute(
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/operations/operations.py", line 120, in execute
self._execute(lm, prompter, parser, **kwargs)
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/operations/operations.py", line 453, in _execute
lm.query(prompt, num_responses=self.num_branches_response)
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/language_models/chatgpt.py", line 79, in query
response = self.chat([{"role": "user", "content": query}], num_responses)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/backoff/_sync.py", line 105, in retry
ret = target(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/language_models/chatgpt.py", line 121, in chat
response = openai.ChatCompletion.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 25, in create
return super().create(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
response, _, api_key = requestor.request(
^^^^^^^^^^^^^^^^^^
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/openai/api_requestor.py", line 298, in request
resp, got_stream = self._interpret_response(result, stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/openai/api_requestor.py", line 700, in _interpret_response
self._interpret_response_line(
File "/home/weijie/miniconda3/envs/aca/lib/python3.11/site-packages/openai/api_requestor.py", line 765, in _interpret_response_line
raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, you requested 5715 tokens (1619 in the messages, 4096 in the completion). Please reduce the length of the messages or completion.
Note that I saved the example as quick_start.py. How should this be fixed?
Thank you for your time and consideration.
Best regards,
Weijie Liu
sorry it wasn't an issue
thank you!
Hi,
I'm experimenting with the GoT framework, but I keep encountering an AttributeError.
There's an error when running the example files i.e. doc_merg.py, keyword_counting.py, set_intersection.py files, and sorting.py files. Error is traced back to chatgpt.py. It's strange that this error is present ONLY when running the example files, but it doesn't show up when running the quick start prompt.
Code:
from openai import OpenAI, OpenAIError
@backoff.on_exception(backoff.expo, OpenAIError, max_time=10, max_tries=6)
def chat(self, messages: List[Dict], num_responses: int = 1) -> ChatCompletion:
"""
Send chat messages to the OpenAI model and retrieves the model's response.
Implements backoff on OpenAI error.
:param messages: A list of message dictionaries for the chat.
:type messages: List[Dict]
:param num_responses: Number of desired responses, default is 1.
:type num_responses: int
:return: The OpenAI model's response.
:rtype: ChatCompletion
"""
response = self.client.chat.completions.create(
model=self.model_id,
messages=messages,
temperature=self.temperature,
max_tokens=self.max_tokens,
n=num_responses,
stop=self.stop,
)
self.prompt_tokens += response.usage.prompt_tokens
self.completion_tokens += response.usage.completion_tokens
prompt_tokens_k = float(self.prompt_tokens) / 1000.0
completion_tokens_k = float(self.completion_tokens) / 1000.0
self.cost = (
self.prompt_token_cost * prompt_tokens_k
+ self.response_token_cost * completion_tokens_k
)
self.logger.info(
f"This is the response from chatgpt: {response}"
f"\nThis is the cost of the response: {self.cost}"
)
return response
The error message:
line 104, in ChatGPT
backoff.expo, openai.error.OpenAIError, max_time=10, max_tries=6
^^^^^^^^^^^^
AttributeError: module 'openai' has no attribute 'error'
This is preventing me from running any example file including my own. Any suggestions?
It's always great to see a lack of ego and respect for others in academia.
at least give a example config file ?
hi! this is great work and super interesting!! - looking to experiment and implement this approach using the llama-2 huggingface api. I was wondering if there is a sample implementation as i'm struggling to import the library, therefore need to rely on locally defining it on my cloud notebook.
This is my training loop if that helps (it's not generating a response, i believe it's not connecting to the api)
import json
prompter = SortingPrompter()
# Access the got_merge_prompt attribute
merge_prompt = prompter.got_merge_prompt
# Create an instance of Controller with the SortingPrompter's merge_prompt
controller = Controller(lm, graph, merge_prompt, parse)
# Initialize an empty list to store the generated responses
generated_responses = []
# Set the save frequency and the number of epochs
save_frequency = 5
num_epochs = 1
for epoch in range(num_epochs):
for data_point in known_data["examples"]:
prompts = data_point['input'].split('\n')
for prompt in prompts:
response = controller.run(prompt)
print(f"Instruction: {prompt}")
print(f"Response: {response}")
# Append the response to the list
generated_responses.append(response)
# Check if it's time to save the responses
if len(generated_responses) % save_frequency == 0:
# Save the responses to a file (e.g., JSON)
with open('known-got.json', 'w') as json_file:
json.dump(generated_responses, json_file)
# Save the responses to a final JSON file after all epochs are done
with open('known-got.json', 'w') as json_file:
json.dump(generated_responses, json_file)
Thanks again!
Why I quote your demo and got error that "ImportError: cannot import name 'GraphOfOperations' from partially initialized module 'graph_of_thoughts.operations' (most likely due to a circular import)
"?
I read the partly source code and found after from graph_of_thoughts import operations
and controller.py quote this package again.
But I dont know how to fix it
Hi! Much appreciated for the excellent work!
I would like to use the GoT frame to do vision-QA task. I have already prepared the FLAN-T5 LLM model. I wonder if it's possible to employ the GoT in my model to do inference?
And does GoT frame support multimodel LM, such as BLIP2?
Hi there,
First of all, great work on implementing GoT and the publication of your paper---really appreciate you guys making your source code publicly available!
Operation
s effectively halt their execution for paths that fail to meet some criteria?Operation
? Or does the paper only mean that one can refine over Thought
s and not Operation
s?I tried to use llama7b-hf to sort numbers according to the code in readme by changing chatGPT to llama7b-hf
lm = controller.Llama2HF(
"graph_of_thoughts/controller/config.json",
model_name="llama7b-hf"
)
But I got the following results
WARNING:root:Could not parse step answer: Sure, I'd be happy to help you sort the list of numbers! Can you please provide the list of numbers for me to work with?. Returning empty list.
May I know whether this is a correct outcome?
In the line above, if current
is either None
or an empty string, should the string formatting be num=len(documents)
instead?
Is there a way to use Graph-of-Thoughts with local model, such as donwloaded Vicuna or LLama?
i'd like to use qwen model to perform GoT reasoning,thank you
Hi, thanks for the great work!
In GoT, thought is a subarray. But what is thought in ToT?
In other words, what is your idea of using ToT to solve the array sorting problem? Is there prompt examples?
First of all, thank you for making this code public and for coming up with such a clever extension of tree-of-thoughts. I'm building a Q&A system over a set of documents, and I'm wondering if there are any examples out there of using graph-of-thoughts in a RAG (retrieval augmented generation) pipeline?
It's unclear looking at the code if there is a way to dynamically generate new "successors" and add it into the execution queue. It would of course be pushed to the front of the queue and not the back. Say you have a node A that has the thought that it should explore certain direction of thoughts, say thought B and C.
The initial graph structure may look simple: generate node A -> score node.
Is there a way that after running node A, it will be then explore an arbitrary number of nodes due to its thoughts so it looks like:
generate node A -> generate node B -> generate node C -> score node
Currently it seems like the initial graph structure is fixed?
Dear authors,
I don't quite understand the Controller Instantiation part in graph_of_thoughts/controller/README.md.
How to instantiate Prompter, Parser, GraphOfOperations, and AbstractLanguageModel respectively? Do I need to paste the code block into a Python file and run the Python file? Could you please further explain it?
I didn't complete this part and ran from examples.sorting.sorting_032 import SortingPrompter, SortingParser, utils
. As a result, I got an error
File "/home/weijie/graph-of-thoughts/graph_of_thoughts/language_models/chatgpt.py", line 104, in ChatGPT
backoff.expo, openai.error.OpenAIError, max_time=10, max_tries=6
^^^^^^^^^^^^
AttributeError: module 'openai' has no attribute 'error'
Best regards,
Weijie Liu
Firstly, thanks a lot for your work and sharing this code!
I'd like to try applying this approach to my own cognitive tasks, but I'm having trouble grasping how one should design a Graph of Operations for a given task. Are there different categories of cognitive tasks that would each tend to call for a certain architecture of GoO?
Or am I maybe misunderstanding, and perhaps the implemented example (document merging) showcases what a typical GoO would be for a language-based cognitive task, and not much variation is necessarily needed or expected?
In the paper you say the following which are nice tips I think, but I don't understand how to apply them to building a GoO:
The overall goal when conducting graph decomposition is
to break down a task to the point, where the LLM can solve
it correctly for the majority of time using a single prompt
(or with a few additional improvement steps). This signifi-
cantly lowers the number of improvement/refinement steps
needed during the later stages of the graph exploration. Fur-
thermore, as indicated by our results, combining or concate-
nating sub-results is usually an easier task than solving large
task instances from scratch. Hence, the LLM is often suc-
cessful when aggregating the final solution.
Any further tips or insight would be greatly appreciated!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.