Writing tests is a time-consuming yet essential task during software development. We propose to leverage recent advances in deep learning for text and code generation to assist developers in writing tests. We formalize the novel task of test completion to automatically complete the next statement in a test method based on the context of prior statements and the code under test. We develop TeCo---a deep learning model using code semantics for test completion. The key insight underlying TeCo is that predicting the next statement in a test method requires reasoning about code execution, which is hard to do with only syntax-level data that existing code completion models use. TeCo extracts and uses six kinds of code semantics data, including the execution result of prior statements and the execution context of the test method. To provide a testbed for this new task, as well as to evaluate TeCo, we collect a corpus of 130,934 test methods from 1,270 open-source Java projects. Our results show that TeCo achieves an exact-match accuracy of 18, which is 29 percent higher than the best baseline using syntax-level data only. When measuring functional correctness of generated next statement, TeCo can generate runnable code in 29 percent of the cases compared to 18 percent obtained by the best baseline. Moreover, TeCo is significantly better than prior work on test oracle generation.

PDF

International Conference on Software Engineering (2023).

@article{nie:icse23, title={Learning Deep Semantics for Test Completion}, author={Pengyu Nie and Rahul Banerjee and Junyi Jessy Li and Raymond Mooney and Milos Gligoric}, journal={International Conference on Software Engineering}, month={May}, url="http://www.cs.utexas.edu/users/ai-labpub-view.php?PubID=128006", year={2023} }

Slides (PDF)