GPT-4 Equipped with Numeric Calculation

MSR-TR-2023-48 |

Published by GitHub

https://github.com/githubnext/gpt4-with-calc/blob/main/docs/report.md

Download BibTex

GPT-4 with Calc

An exploration of using calculation generation to improve GPT-4’s capabilities for numeric reasoning.

GPT-4 is weak at calculating with numbers, compared to other resasoning capabilities. It makes mistakes that lead to problematic user experiences on texts involving basic numeric problems. We describe a simple, general technique to address this, by adding a step generating calculation code, apply it to some widely reported real-world failures of GPT-4-based AI, do some evaluation and discuss related issues.