fafrd / aquarium Goto Github PK

With the advancements in at home AI it would be amazing to see support for the following backend:

https://abetlen.github.io/llama-cpp-python/

Web Server
llama-cpp-python offers a web server which aims to act as a drop-in replacement for the OpenAI API. This allows you to use llama.cpp compatible models with any OpenAI compatible client (language libraries, services, etc).

To install the server package and get started:

pip install llama-cpp-python[server]
export MODEL=./models/7B
python3 -m llama_cpp.server
Navigate to http://localhost:8000/docs to see the OpenAPI documentation.

Redirecting the API URL within the source of the golang library used for openAI gets about this far:

Then I am met with this error:

19c31e44 iteration 1: executing                                                                                                                                                                                                                                                                
panic: runtime error: slice bounds out of range [4:3]                                                                                                       

goroutine 70 [running]:
aquarium/actor.(*Actor).iteration(0xc00018b100)
	/home/raven/aquarium/actor/actor.go:327 +0x1079
aquarium/actor.(*Actor).Loop.func2()
	/home/raven/aquarium/actor/actor.go:148 +0x68
created by aquarium/actor.(*Actor).Loop
	/home/raven/aquarium/actor/actor.go:141 +0xabb

Sick project.

One thing I've noticed is that it will hang while trying to CD to a directory.

/bin/bash: line 1: exec: cd: not found

And it doesn't seem to know what to do when cd isn't found so it just keeps trying, at least in my last attempt.

panic: runtime error: slice bounds out of range [721:720]

goroutine 82 [running]:
aquarium/actor.(*Actor).iteration(0xc000096000)
/Users/ksimpson/git/aquarium/actor/actor.go:299 +0xf6a
aquarium/actor.(*Actor).Loop.func2()
/Users/ksimpson/git/aquarium/actor/actor.go:141 +0x68
created by aquarium/actor.(*Actor).Loop
/Users/ksimpson/git/aquarium/actor/actor.go:134 +0x97b

Bordom is part of agency .. right?

for example if agent throws something into the background with some simple job control foo &
it could, via bordom timer be given +1 to max procs...

Or perhaps an interactive element, where the output screen shows the current # of procs and the user can increase this value:

also the split output screen is neat, I could envision sending this data to gpt4 with a prefixed with a general explanation prompt and see if it could come up with good ideas to escape the loop.

cool project.

edit: 7 dollars?!
This thing is pretty dangerous... I think that feeding the last 10 or 20 lines of stdout would be almost as effective,
and cost a lot less.

I've asked GPT-4 to modify the genDialogue() function to use gpt-3.5-turbo instead. Seems to work fine for me. I'm not familiar with Go can someone take a look and see if it looks good?

func genDialogue(aiPrompt string) (string, error) {
	apiKey := os.Getenv("OPENAI_API_KEY")
	if apiKey == "" {
		return "", errors.New("undefined env var OPENAI_API_KEY")
	}

	ctx := context.Background()
	client := gpt3.NewClient(apiKey)

	messages := []gpt3.ChatCompletionRequestMessage{
		{
			Role:    "user",
			Content: aiPrompt,
		},
	}

	logger.Debugf("### Sending request to OpenAI:\n%s\n\n", aiPrompt)

	request := gpt3.ChatCompletionRequest{
		Model:        "gpt-3.5-turbo",
		Messages:     messages,
		MaxTokens:    tokens,
		Temperature:  gpt3.Float32Ptr(0.0),
	}

	resp, err := client.ChatCompletion(ctx, request)
	if err != nil {
		logger.Debugf("### ERROR from OpenAI:\n%s\n\n", err)
		return "", err
	}

	trimmedResponse := strings.TrimSpace(resp.Choices[0].Message.Content)
	logger.Debugf("### Received response from OpenAI:\n%s\n\n\n", trimmedResponse)
	return trimmedResponse, nil
}

When the AI uses add-apt-repository it hangs forever. Could the same workaround as for apt be used?

Running this program can be expensive due to the OpenAI api cost. Part of the reason is that we are sending the entire output of previous commands, and for very long messages this can add up.

We can probably fix this by only sending the last ~10 or 20 output lines to OpenAI- we should implement that as a flag or something. If it works very well i'll make this the default behavior.

Running:
OPENAI_API_KEY="..." ./aquarium --goal "Your goal is to check for any security vulnerabilities."

Running the following command and gets stuck:
sudo apt-get install -q -y rkhunter

It would be nice to have a timeout that can fire if the terminal's content doesn't change in x seconds. In case of fire triggering the loop of getting the terminal's output back to the AI.