Repair Is Nearly Generation: Multilingual Program Repair with LLMs
- Harshit Joshi ,
- José Cambronero ,
- Sumit Gulwani ,
- Vu Le ,
- Ivan Radicek ,
- Gust Verbruggen
Published by AAAI | February 2023
Most programmers make mistakes when writing code. Some of these mistakes are
small and require few edits to the original program – a class of errors recently
termed last mile mistakes. These errors break the flow for experienced
developers and can stump novice programmers. Existing automated repair
techniques targeting this class of errors are language-specific and do not
easily carry over to new languages. Transferring symbolic approaches requires
substantial engineering and neural approaches require data and retraining. We
introduce RING, a multilingual repair engine powered by a large language model
trained on code (LLMC) such as Codex. Such a multilingual engine enables a
flipped model for programming assistance, one where the programmer writes code
and the AI assistance suggests fixes, compared to traditional code suggestion
technology. Taking inspiration from the way programmers manually fix bugs, we
show that a prompt-based strategy that conceptualizes repair as localization,
transformation, and candidate ranking, can successfully repair programs in
multiple languages with minimal effort. We present the first results for such a
multilingual repair engine by evaluating on 6 different languages and comparing
performance to language-specific repair engines. We show that RING can
outperform language specific repair engines for three of these languages.