Handling model code with prefix ``` about lagent HOT 7 CLOSED

The PythonInterpreter supports three formats

• solution()
• print(answer)
• get the final line answer

You can modify PythonInterpreter'description and PythonInterpreter'parameter to get the best practice

from lagent.

@Harold-lkk question is, how to decide which format use before LLM prints it's code?

from lagent.

That maybe depend on LLM.

1. More powerful LLM can follow more strict user instructions.
2. You may need different prompts for different LLM, which is prompt-engineering

from lagent.

@Harold-lkk Can u provide some workable example using interLM7B? it should not differs too much for most opensource LLMs.

from lagent.

Hi @lucasjinreal ,
Thanks for your report. There are still some issues with the code capability and the code interpreter ability. The current workaround might be manually deal with '```python' and remove it. We will further enhance the model in the future to fix that.

from lagent.

I also tried to make a code-interpreter-like project. The biggest problem is how to extract code block in response string. I'm struggling on such solutions like providing sample responses(prompt skill) to LLM but not everytime it works.

from lagent.

Yes, it is a bit difficult for some models. We will update the functionality of code interpreter in the future.

from lagent.