question about LM capcapacity, thanks about rot HOT 1 CLOSED

Thank you for your interest in our work, and sorry for the late reply.

First of all, "To gain mathematical computing power, the model needs to be fine-tuned" is far from the point of our paper.
We are proposing RoT as an inference framework, which demonstrates the importance of utilizing multiple contexts to solve complex reasoning problems.
Fine-tuning is not the only way to make an LM perform RoT.
For example, one may prepend a CoT prompting, which includes several demonstrations of RoT, to every context to induce an LM to perform RoT without any fine-tuning.
However, as we deal with extremely complex problems that require hundreds of thousands of tokens of reasoning steps, each token generation should be extremely accurate to achieve a reasonable score.
We found that CoT prompting is not sufficient for such accuracy, so we chose to rely on fine-tuning for this work.

Of course, fine-tuning for solving just a few types of problems will damage the general capability of LMs.
We did not quantitatively measure the degradation because (i) it seemed obvious, and (ii) evaluating the general capability of LMs is costly.

Hope this answers your question :)

from rot.