kolbytn / mindcraft Goto Github PK

NOTE: It Mixtral can at times be... Fragile. Let's call it that. Keep the temperature LOW. You can indeed drive it nuts, at least with the system prompt I was using.

I intend to make a fork that just barely supports this service.
GroqCloud indeed supports JavaScript-based inferences/completions.

The thing here is that Groq is, in fact, not it's own AI model. It's a service.

You can choose multiple Groq-provided models, including a new one, Mixtral-8x7b with a 32,768 max token limit.

The JavaScript code snippet is provided by GroqCloud itself, just enter the parameters (including the system prompt) and ask it to give you the code. It's also a small snippet, thank god.

Mixtral-8x7b on GroqCloud can be found here

I have personal experience with GroqCloud and I fully intend to test out function calls myself with mixtral.

(Note: If it was not clear, I have little experience in JavaScript.)

file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/coder.js:141
}, settings.code_timeout_mins);
^

ReferenceError: settings is not defined
at Coder.generateCodeLoop (file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/coder.js:141:16)
at async Coder.generateCode (file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/coder.js:86:19)
at async Object.perform (file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/commands/actions.js:29:20)
at async executeCommand (file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/commands/index.js:89:20)
at async Agent.handleMessage (file:///C:/Users/JohnDoe/Desktop/mindcraft-main/src/agent/agent.js:125:35)

you can try using https://console.groq.com/ for this (and obv the actual openai gpt api)
the functions api is the same as it is on gpt4+

https://console.groq.com/docs/tool-use#models
https://platform.openai.com/docs/guides/function-calling

this a local LLM model.

I think if done right you could train model to work with the game system you made

it has API docs that mostly a copy open AI api

So i wanted to try this ai bot and after reading the readme, i decided to go for the ollama models because they are free.
I did everything but it says i need to set the api key as environment variable...
Is the Ollama thing still in development or i missed something ?
Thanks in advance !
-Philipou

Are there any plans to make the bot check for a new version and be able to install it via cli or interface?

Bot does not follow instructions given in minecraft, sometimes even using unrelated commands

Log:

Log.txt
andy.json
settings.json

Bot will repeatedly place a torch in the water as the torch automatically breaks each time.

Reproduce: Give the bot some torches and place it in shallow water with torch placement mode on.

The Bot only writes Hello, im a friendly minecraft bot and so on, i ask him to get some wood, go to me, build a house, but he only says i see a skeleton, i am a friendly bot. If I should send any logs, please ask me. I hope you can help me.

Now he stands next to the coal and says he can't find the coal, but with baritone he's sure to find it.

Ollama models can be ran locally and it shouldn't be too hard to make a bridge between these two
Some good models include:
https://ollama.com/library/mistral
https://ollama.com/library/gemma
https://ollama.com/library/llama2
https://ollama.com/library/mixtral

I am currently experimenting both the gpt 4o and the 4-turbo and after it starts building, it basically stops and has the running effect (check the picture)

these are also the logs:
{
role: 'user',
content: 'Topo1717: with what you have in your inventory, please build me a short cute tiny house, with also internal decorationsw'
},
{
role: 'assistant',
content: "Absolutely! I'll get started on your tiny house right away. 🏠 !newAction"
},
{
role: 'system',
content: 'Code generation started. Write code in codeblock in your response:'
}
]
Received.
Generated code: """
const buildTinyHouse = async (bot) => {
const position = world.getPosition(bot);
const startX = position.x;
const startY = position.y;
const startZ = position.z;
const width = 5;
const depth = 5;
const height = 3;

// Build the floor
for (let x = startX; x < startX + width; x++) {
    for (let z = startZ; z < startZ + depth; z++) {
        await skills.placeBlock(bot, 'spruce_planks', x, startY, z);
    }
}

// Build the walls
for (let x = startX; x < startX + width; x++) {
    for (let y = startY + 1; y < startY + height; y++) {
        for (let z = startZ; z < startZ + depth; z++) {
            // Leave a space for the door
            if ((x === startX || x === startX + width - 1 || z === startZ || z === startZ + depth - 1) && !(x === startX + 2 && z === startZ && y === startY + 1)) {
                await skills.placeBlock(bot, 'birch_planks', x, y, z);
            }
        }
    }
}

// Build the roof
for (let x = startX; x < startX + width; x++) {
    for (let z = startZ; z < startZ + depth; z++) {
        await skills.placeBlock(bot, 'oak_stairs', x, startY + height, z);
    }
}

// Place the door
await skills.placeBlock(bot, 'jungle_door', startX + 2, startY + 1, startZ);

// Internal decorations
// Place a bed
await skills.placeBlock(bot, 'red_bed', startX + 1, startY + 1, startZ + 1);

// Place a chest
await skills.placeBlock(bot, 'chest', startX + 3, startY + 1, startZ + 1);

// Place an enchanting table
await skills.placeBlock(bot, 'enchanting_table', startX + 1, startY + 1, startZ + 3);

// Place a furnace
await skills.placeBlock(bot, 'furnace', startX + 3, startY + 1, startZ + 3);

// Place an armor stand
await skills.placeBlock(bot, 'armor_stand', startX + 2, startY + 1, startZ + 2);

};

await buildTinyHouse(bot);"""
executing code...

placing block...
placing block...
placing block...
placing block...
placing block...

Loading examples... (node:61) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (Use `node --trace-deprecation ...` to show where the warning was created) Examples loaded. Logging in... Mineflayer detected that you are using a deprecated event (physicTick)! Please use this event (physicsTick) instead. Agent process exited with code 0 and signal null

C:\Users\won'tshowmypc'suser\mindcraft\node_modules\mineflayer\lib\loader.js:108
    const version = require('minecraft-data')(bot._client.version).version
                                                                  ^

TypeError: Cannot read properties of null (reading 'version')
    at next (C:\Users\won'tshowmypc'suser\mindcraft\node_modules\mineflayer\lib\loader.js:108:67)
    at createBot (C:\Users\won'tshowmypc'suser\mindcraft\node_modules\mineflayer\lib\loader.js:105:34)
    at initBot (file:///C:/Users/won'tshowmypc'suser/mindcraft/src/utils/mcdata.js:51:15)
    at Agent.start (file:///C:/Users/won'tshowmypc'suser/mindcraft/src/agent/agent.js:24:20)

Node.js v18.16.0
Agent process exited with code 1 and signal null
Agent process exited too quickly. Killing entire process. Goodbye.

i've done all the readme says, this happens
i'm using only a claude api key

hey i saw this from a vid on youtube and thought it was pretty sweet and wanted to check it out but it just spams for me lol, i think it might be responding to itself.
im using gemini 1.0 if that matters

Hello. Im installed the bot and this happend: when im say something to bot for example: Hello andy, the bot says weird stuff. im checked and that what bot is saying is the instruction, that are given to bot to program it as minecraft bot. please help.

i want to run mindcraft but i have problem with rate limit and i dont want to buy a tier account

Hello,

Whenever I am trying to run "node main.js" I get the following error:

Loading examples...
file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/node_modules/openai/error.mjs:59
return new RateLimitError(status, error, message, headers);
^

RateLimitError: 429 You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.
at APIError.generate (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/node_modules/openai/error.mjs:59:20)
at OpenAI.makeStatusError (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/node_modules/openai/core.mjs:256:25)
at OpenAI.makeRequest (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/node_modules/openai/core.mjs:299:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async GPT.embed (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/src/models/gpt.js:56:27)
at async file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/src/utils/examples.js:20:31
at async Promise.all (index 5)
at async Examples.load (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/src/utils/examples.js:23:25)
at async Prompter.initExamples (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/src/agent/prompter.js:44:9)
at async Agent.start (file:///Users/iluca/Documents/Leiden%20Univ/Master/Games%20AI/Assignment%203/mindcraft/src/agent/agent.js:15:9) {
status: 429,
headers: {
'alt-svc': 'h3=":443"; ma=86400',
'cf-cache-status': 'DYNAMIC',
'cf-ray': '875c95ac888a970d-AMS',
connection: 'keep-alive',
'content-length': '337',
'content-type': 'application/json; charset=utf-8',
date: 'Wed, 17 Apr 2024 12:58:06 GMT',
server: 'cloudflare',
'set-cookie': '__cf_bm=5smTpOKE2NHa2D_1BAOenwmYG2oJ2Xt3KR1Q7Eixqeo-1713358686-1.0.1.1-NeSwO9K.H2l.uO3xrSS53kmWK9Hhp81qc_eyroOcJ04Bl.U_HgELA9fQgrFEGq7JMH6wxE.o7VVJ4Nw.tE8j_g; path=/; expires=Wed, 17-Apr-24 13:28:06 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=XxvlQAw4wGsfsEiI7hpBtXi9qpDNxJCBSWKk443MZW4-1713358686295-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None',
'strict-transport-security': 'max-age=15724800; includeSubDomains',
vary: 'Origin',
'x-request-id': 'req_f74558cd10c5b7f498219040d82a022f'
},
request_id: 'req_f74558cd10c5b7f498219040d82a022f',
error: {
message: 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.',
type: 'insufficient_quota',
param: null,
code: 'insufficient_quota'
},
code: 'insufficient_quota',
param: null,
type: 'insufficient_quota'
}

Node.js v18.17.1
Agent process exited with code 1 and signal null
Agent process exited too quickly. Killing entire process. Goodbye.

Any ideas how to perhaps fix this?

I followed the tutorial and everything, signed up for chatgpt twice because i bought the wrong product the first time lol and finally made progress to this screen. Now im stuck and cannot get past this error, am i just dumb? how do i fix this.

P.S. amazing work, have not used it yet but i am looking forward to doing so.

Hello everyone!!! I would like to see a free model that I don't have to pay for.... Can anyone make it? I'm not really good at coding, and I can't even find a free alternative to ebedding.

I got this error after cloning the repo, adding my key, doing pnpm install, then running node main.js

[vince@vinux mindcraft]$ node main.js
node:internal/modules/esm/resolve:845
  throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base), null);
        ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'vec3' imported from /home/vince/Source/mindcraft/src/agent/library/skills.js
    at packageResolve (node:internal/modules/esm/resolve:845:9)
    at moduleResolve (node:internal/modules/esm/resolve:918:18)
    at defaultResolve (node:internal/modules/esm/resolve:1148:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:390:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:359:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:234:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:87:39)
    at link (node:internal/modules/esm/module_job:86:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Node.js v21.7.2
Agent process exited with code 1 and signal null
Agent process exited too quickly. Killing entire process. Goodbye.

I resolved this issue by doing pnpm install vec3

Easy fix, but I figured I'd make an issue in case vec3 was unintentionally omitted from the package.json

yeah, it would be nice if they had more documentation on using llama3 I have no idea how to use it with these bots (unless its meant to be extremely slow)

the only thing it does is freaks out when it sees a zombie
I think I found the issue which is that the ollama server seems to be constantly restarting (I have no idea why) every time a new message is sent, the server thingy decides to start loading something (you can see if you have the console window open) and it takes around 2m to do one request and then just IMMEDIATELY RELOADS

Ok I think I got it figured out: every time the user sends a message, it creates a new agent (instead of using the same one for some reason) and sends the starting stuff again:

received message from BorbTheBird : hello there
selected examples:
zZZn98: come here
brug: Remember that your base is here.
Awaiting local response... (model: llama3)
Messages: (intents here)

(yes, I just copied my comment, but I believe this issue deserves its own thread)

any idea what should i do to fix this?

G:\Artificial_Intelilgence\mindcraft\mindcraft>node main.js
Loading examples...
Examples loaded.
Logging in...
(node:12324) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Mineflayer detected that you are using a deprecated event (physicTick)! Please use this event (physicsTick) instead.
Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
Agent process exited with code 0 and signal null

I followed the installation and local run guideline but I can't get it works.
I did the installation and run the command below, then nothing happens and no agent shows up when I go back to game.
Could anyone suggest a solution? thanks！

~/mindcraft$ node main.js
Loading examples...
Examples loaded.
Logging in...
Mineflayer detected that you are using a deprecated event (physicTick)! Please use this event (physicsTick) instead.
Agent process exited with code 0 and signal null

Hi all, just watched your videos and I have to say, you got me. I'm a developer, minecraft player, and use AI tools regularly. Oh and I love beneficial bots.

I haven't yet played with Mineflayer, it's on my list but I never really got around to it. The closest was JsMacros running in my client. But I really should. The tool that I do often use is Baritone. There may be a moderate overlap in functionality, but I can't help but think that giving AI another toolset that it can use to interact with the world could be really useful.

Some features that come to mind where I think Baritone may offer some advantages:

• It seems to be notable for pathfinding, whether for building or long range navigation, supports waypoints etc.
• Numerous settings that allow configuring of building/clearing related options, things like buildInLayers/layerHeight (could have helped the bots immensely with the building video tasks
• Can select areas by relative position and/or coordinates, manipulate the selection like worldedit, and perform clear, fill and replace operations. Also can copy and paste areas.

Overall it's an amazing tool. The main two challenges (aside from a lot of overlapping functionality) is A, Baritone requires text input which may be challenging [edit, it seems to have an API], B I have no clue if it would integrate with Mineflayer, which I think is the client. If it won't then this entire idea is dead in the water.

Anyways, very cool content, I cloned the repo and will have to start playing with it.

Endlessly restarts with this error and each restart resends requests to the API again

I am trying to log into my Alt account using the Microsoft authenticator. The json file has the name as the correct Minecraft account but the console is asking if it owns Minecraft. I have used this account with mindcraft a while back but now it's not working. How can I re log into the Microsoft account?

Error: Cannot find module 'C:\Users\win\main.js'
at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
at Module._load (node:internal/modules/cjs/loader:986:27)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
at node:internal/main/run_main_module:28:49 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}

Node.js v20.14.0

Hello, i can load the bot into the world perfectly, but if i speak to him to mine some sand, he only says stuff like hi, im andy, a friendly Mc bot, here is the command line output: https://pastebin.com/kYsKQRS1
to install, i only git cloned the package, ran npm install and then node main.js (i had my openai key as environment variable saved) and it's also requesting API Keys.

The bot has connected to the server and has just been standing there for a long time.

But he has running effects near his feet.

Broke the block next to him, now he's running back and forth.

Like i dont know how to name the variable, nor anything??? And when i name it as "GEMINI_API_KEY=(my private key)", when i add it to all of them, i notice that it just overwrite and only set to the last one! what the hell is this, i need quick help

and the code says to put openai api key but i want ollama 3. how to do?

I am currently using the Ollama bot, however it doesn't do anything. I dropped him so blocks, he picked them up, but when I tell him in chat to build something with them or to come where I am, he doesn't move and doesn't do anything.

Am I doing something wrong or am I perhaps using it in the wrong way?

Is there any way to disable embeddings when using gpt?

GRAHH

Minecraft version: Minecraft 1.20.4
AI model: llama3

It's in the title - not many other details I can provide. Console registers that I've said something in chat, however, the bot itself will not do anything. "Follow me", or any simple commands do not process at all. It joins the game and does its own thing.

Hi! This is a insanely sick project and I want to try it on a 1.20.4 server. Is there going to be a update to it? I understand this is a really complex project and it would take a lot of effort.

Having some issues wtih the bot overall.
Example:

Received.
Full response: ""Looks like we're maxed out here. Moving up! !moveAway(2)""
parsed command: { commandName: '!moveAway', args: [ 2 ] }
executing code...

Code execution triggered catch: ReferenceError: move is not defined
Agent executed: !moveAway and got: Code output:
!!Code threw exception!! Error: ReferenceError: move is not defined

Since it says it's not defined, I suspect I have some outdated version of it? Or perhaps it could be some other issue?

GPT4o seems highly incompetent, as if there's some issue with the code, as if it's incomplete? Are we supposed to define these methods ourselves?

Whats required to update the release support? Happy to help if you need

Been playing around with this a lot, and seems like the prompting is great and all but what I want is repetitive actions to get done based on how I show/catalog actions to an NPC. Think of a prompt session like this:

Follow me and record the actions i take starting now. I will tell you when to stop recording the actions i take. Watch what I do and lets call it sorterArray. 

(i place a bunch of blocks)

Stop recording. List the items needed to complete 1 sorterArray

Thoughts on this? Feels like a natural progression of newAction where it could rely on memory feedback and store executable code.

How can i add +3 bots in a minecraft server? like have andy, pollux and radley in a single instance?

for some reason this is sending too many requests before even joining the server, it does this until it gets a ratelimit error and then crashes and tries restarting which again, same thing happens :'((

RateLimitError: 429 Rate limit reached for text-embedding-ada-002 in organization org-n0dBBLrWyfW8gBN32Z8hu5N9 on requests per min (RPM): Limit 3, Used 3, Requested 1. Please try again in 20s. Visit https://platform.openai.com/account/rate-limits to learn more. You can increase your rate limit by adding a payment method to your account at https://platform.openai.com/account/billing.
    at APIError.generate (file:///G:/Artificial_Intelilgence/mindcraft/node_modules/openai/error.mjs:58:20)
    at OpenAI.makeStatusError (file:///G:/Artificial_Intelilgence/mindcraft/node_modules/openai/core.mjs:256:25)
    at OpenAI.makeRequest (file:///G:/Artificial_Intelilgence/mindcraft/node_modules/openai/core.mjs:299:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async GPT.embed (file:///G:/Artificial_Intelilgence/mindcraft/mindcraft/src/models/gpt.js:59:27)
    at async file:///G:/Artificial_Intelilgence/mindcraft/mindcraft/src/utils/examples.js:20:31
    at async Promise.all (index 13)
    at async Examples.load (file:///G:/Artificial_Intelilgence/mindcraft/mindcraft/src/utils/examples.js:23:25)
    at async Prompter.initExamples (file:///G:/Artificial_Intelilgence/mindcraft/mindcraft/src/agent/prompter.js:44:9)
    at async Agent.start (file:///G:/Artificial_Intelilgence/mindcraft/mindcraft/src/agent/agent.js:15:9) {
  status: 429

What do i do???

so i was testing this project and noticed theres alot of frequent ratelimits that leads to the AI being a incredibly slow, it is not necessary to implement as the code basicly works fine and people who have paid API access will experience no problems, but i imagine there's gonna be quite alot of people who will not be able to use this project because of the god damn rate limits :/
So i came up with 2 ideas/features that can potentially be very useful, and you guys could consider adding it when you have time for it.

firstly, the ability to use custom endpoints, theres quite alot of free chatgpt api endpoints that people could use to get higher ratelimits while having no cost, i don't think something like this is hard to add but since i have near to no experience in nodejs i didnt know how to do it myself ( or i wouldve done it myself )

alternatively, being able to add multiple openai api key to be used at the same time could double the ratelimit, (by switching API keys between each request ) but this doesnt seem quite fitting as you would need multiple accounts and im pretty sure that would be against the TOS, but yeah it'd still work

So yeah please consider implementing one of these features as im pretty certain itll help alot of people who are trying to test this project in the future ;))

Agent cannot connect to a modded minecraft LAN server because of an issue that's hard to define. Keeps saying ECONNREFUSED errorno. -4078

how do i selfhost the ai instead of api key

The self preservation and hunting modes conflict with underwater creatures. We should remove fish and squids from the hunting list.

It's actually pretty common that the bot gets stuck between two different modes. For example running towards a mob to pick up a nearby item and them away from it in cowardice. I don't have any reproducible examples of this yet, but we should consider either leaving other modes disabled for a time after executing one mode. Or ensuring that the radius for all the different modes are the same.