Mid Sweden University

 Understanding large-scale software systems remains asignificant challenge in software engineering, particularly fortasks such as on-boarding, maintenance, and documentation.While existing code summarization tools can assist inunderstanding isolated code snippets or individualfunctions, they often fall short in delivering comprehensive,system-level descriptions of complex code-bases.In this thesis, we propose the System Description Generator(SDG), a proof-of-concept tool that automatically generateshigh-level textual descriptions of Java software systems bycombining static code analysis, knowledge graph (KG)modeling, and large language models (LLMs). SDG extractsarchitectural data using abstract syntax trees (ASTs), stores itin a Neo4j graph, ranks key components based on outgoingrelationships, and prompts an LLM with a structured JSONrepresentation to generate natural language descriptions.To evaluate SDG, we applied it to 34 open-source Javaprojects and assessed the generated descriptions across threedimensions: correctness, relevance, and usefulness. We usedthe automated G-Eval framework for scalable assessmentand selected a subset of four representative projects formanual evaluation to ensure alignment between humanjudgments and automated scores. The results showed highmedian scores for correctness (0.81) and relevance (0.77),confirming the structural and contextual fidelity. Theusefulness score (0.52), though lower than correctness andrelevance, remains reasonably strong given the subjectivenature of user-perceived value, rather than indicating adeficiency of the tool.Overall, we demonstrate the feasibility of combining ASTs,KGs, and LLMs to support system-level softwarecomprehension. The SDG tool and evaluation framework areavailable on the project’s GitHub repository to supportreproducibility and inspire future tools that aid large-scalesoftware comprehension.