Leveraging QLoRA on Code Large Language Models for Multilingual Code Translation

초록

This paper presents an efficient fine-tuning and comparison of various code Large Language Models (code LLMs) for multilingual code translation using Quantized Low-Rank Adaptation (QLoRA). The study evaluates the code translation performance of five representative code LLMs across 42 programming language pairs utilizing the XLCoST dataset. The results indicate that the CodeLlama-7B-Instruct and CodeT5+ 770M models achieved the highest CodeBLEU scores of 0.740 and 0.735, respectively, for Java-Python translation. However, the average performance across all models was similar. Translation quality varied significantly between language pairs, with PHP-C++ underperforming, while translations targeting Python excelled, likely due to the composition of the pre-training data. Additionally, we confirmed that large models like CodeLlama can be effectively fine-tuned on a single 48GB GPU using QLoRA.

키워드