This project introduces REFINER, a system that helps language models improve their reasoning abilities through feedback. It has one model that generates initial reasoning steps, and another model that critiques those steps. By getting feedback on the intermediate reasoning, the first model can refine its final answer. This allows language models to solve complex reasoning tasks more accurately.

REFINER is an interaction-based framework for natural language reasoning tasks. It has a CRITIC model that provides structured feedback on intermediate reasoning steps, and a GENERATOR model that solves the reasoning task by first generating intermediate steps. The core idea is the interaction between the generator and critic, where the generator's steps are improved via feedback from the critic.