中文 | English
In the domain of large-scale software development, the demands for dynamic and multifaceted static code analysis exceed the capabilities of traditional tools. To bridge this gap, we present CodeFuse-Query, a system that redefines static code analysis through the fusion of Domain Optimized System Design and Logic Oriented Computation Design. CodeFuse-Query reimagines code analysis as a data computation task, support scanning over 10 billion lines of code daily and more than 300 different tasks. It optimizes resource utilization, prioritizes data reusability, applies incremental code extraction, and introduces tasks types specially for Code Change, underscoring its domain-optimized design. The system's logic-oriented facet employs Datalog, utilizing a unique two-tiered schema, COREF, to convert source code into data facts. Through Godel, a distinctive language, CodeFuse-Query enables formulation of complex tasks as logical expressions, harnessing Datalog's declarative prowess.
Overall, the CodeFuse-Query platform is divided into three main parts: code data model, code query DSL, and platform productization services.
News!! A paper describing the data-centric and language-agnostic static analysis has been accepted by ICSE 2025:
Title: Datalog-Based Language-Agnostic Change Impact Analysis for Microservices
Authors: Qingkai Shi (Nanjing University), Xiaoheng Xie (Ant Group), Xianjin Fu (Ant Group), Peng Di (Ant Group), Huawei Li (Alibaba Inc.), Ang Zhou (Ant Group), and Gang Fan (Ant Group)
We have defined a code data and standardization model: COREF, which requires all code to be converted to this model through various language extractors. COREF mainly contains the following information: COREF = AST (Abstract Syntax Tree) + ASG (Abstract Semantic Graph) + CFG (Control Flow Graph) + PDG (Program Dependency Graph) + Call Graph + Class Hierarchy + Documentation (Documentation/Comments) Note: Since the computation difficulty of each type of information varies, not all languages' COREF information includes all the above. The basic information mainly consists of AST, ASG, Call Graph, Class Hierarchy, and Documentation, while other information (CFG and PDG) is still under construction and will be gradually supported.
Based on the generated COREF code data, CodeFuse-Query uses a custom DSL language called Gödel for queries to meet code analysis needs. Gödel is a logical reasoning language based on the logical reasoning language Datalog, which derives new facts through "facts" and "rules". Gödel is also a declarative language, which, compared to imperative programming, focuses more on describing "what is needed" and leaves the implementation to the computation engine. Since the code has been transformed into relational data (COREF data is stored in the form of relational data tables), one might wonder why not use SQL directly or use an SDK, but instead learn a new DSL language. The reason is that Datalog has monotonicity and termination properties, meaning that Datalog sacrifices some expressive power, and Gödel inherits this characteristic.
- Compared to SDKs, Gödel's main advantage is ease of learning and use; its declarative nature means users do not need to focus on intermediate computations but can describe their needs simply, like with SQL.
- Compared to SQL, Gödel's advantages are stronger descriptive ability and faster computation speed, for example, in describing recursive algorithms and multi-table joint queries, which are difficult for SQL.
CodeFuse-Query includes the Sparrow CLI and the online service Query Center. Sparrow CLI contains all components and dependencies, such as extractors, data model, compiler, etc., allowing users to generate code data and conduct queries locally (for Sparrow CLI usage, please see Section 3: Installation, Configuration, and Running). If users require online queries, they can experiment using the Query Center.
As of now, CodeFuse-Query supports data analysis for 11 programming languages. Among them, support for 5 languages (Java, JavaScript, TypeScript, XML, Go) is very mature, while the remaining 6 languages (Object-C, C++, Python3, Swift, SQL, Properties) are in beta stage and have room for further improvement and perfection. The specific support status is shown in the table below:
| Language | Status | COREF Model Node Count |
|---|---|---|
| Java | Mature | 162 |
| XML | Mature | 12 |
| TS/JS | Mature | 392 |
| Go | Mature | 40 |
| OC/C++ | Beta | 53/397 |
| Python3 | Beta | 93 |
| Swift | Beta | 248 |
| SQL | Beta | 750 |
| Properties | Beta | 9 |
Note: The maturity level of the language status is determined based on the types of information contained in COREF and the actual implementation. Except for OC/C++, all languages support complete AST information and Documentation, and in the case of Java, COREF for Java also supports ASG, Call Graph, Class Hierarchy, and some CFG information.
Installation, Configuration, and Running
- Abstract
- Introduction
- User Case
- Installation, Configuration, and Running
- GödelScript Query Language
- Developing Plugins (VSCode)
- COREF API
Our work is described in the following paper:
CodeFuse-Query: A Data-Centric Static Code Analysis System for Large-Scale Organizations
If you find CodeFuse-Query useful in your research, please cite our paper:
@misc{xie2024codefusequerydatacentricstaticcode,
title={CodeFuse-Query: A Data-Centric Static Code Analysis System for Large-Scale Organizations}, author={Xiaoheng Xie and Gang Fan and Xiaojun Lin and Ang Zhou and Shijie Li and Xunjin Zheng and Yinan Liang and Yu Zhang and Na Yu and Haokun Li and Xinyu Chen and Yingzhuang Chen and Yi Zhen and Dejun Dong and Xianjin Fu and Jinzhou Su and Fuxiong Pan and Pengshuai Luo and Youzheng Feng and Ruoxiang Hu and Jing Fan and Jinguo Zhou and Xiao Xiao and Peng Di},
year={2024},
eprint={2401.01571},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2401.01571}, }