GoCoMA: Hyperbolic Multimodal Representation Fusion for Large Language Model-Generated Code Attribution

1. Problem Statement

Large Language Models (LLMs) trained on massive code corpora can generate code that is nearly indistinguishable from human-written code. This capability raises critical concerns regarding:

• Security: LLMs may generate vulnerable code or hallucinate non-existent packages.
• Licensing & Plagiarism: Ambiguity regarding the origin of code (human vs. specific LLM) creates legal and ethical risks.
• Forensics: There is an urgent need for robust Code Authorship Attribution (CAA) to trace code back to its generating model.

Limitations of Existing Approaches:

• Source Code Only: Relying solely on source code features (stylometry, ASTs) misses deeper behavioral signals.
• Watermarking: Often unreliable as it requires the model provider to actively implement it and fails for pre-existing code.
• Deep Representation Gaps: Current deep learning approaches struggle with multilingual setups and fail to utilize low-level signals derived from Binary Pre-Executable Artifacts (BPEA) (e.g., compiled object files, bytecode).
• Euclidean Fusion: Standard multimodal fusion in Euclidean space often fails to preserve the intrinsic hierarchical relationships between high-level code semantics and low-level binary structural patterns.

2. Methodology: GoCoMA Framework

The authors propose GoCoMA (Geodesic Cosine similarity-based cross-Modal Attention), a geometry-aware multimodal framework that fuses source code and binary artifacts within a hyperbolic space.

A. Data Representation

1. Code Modality (High-Level):
   • Source code is processed using Code Pre-trained Language Models (Code PLMs) such as CodeT5+, Qwen2.5-Coder, UniXcoder, and CodeBERT.
   • These models extract high-level lexical, syntactic, and semantic embeddings.
2. Image Modality (Low-Level):
   • Source code is compiled into Binary Pre-Executable Artifacts (BPEA) (e.g., .o, .class, .pyc).
   • These binary files are transformed into RGB images by grouping consecutive bytes into pixel triplets.
   • Vision Pre-trained Models (Vision PTMs) like ViT-Base, ConvNeXT, EfficientNetV2-M, and MaxViT-T analyze these images to capture low-level compilation footprints, byte-layouts, and optimization patterns.

B. Hyperbolic Fusion Strategy

The core innovation is projecting embeddings into a Poincaré ball (hyperbolic space with constant negative curvature) to model the extrinsic hierarchy between code and binary artifacts.

1. Projection (Euclidean $\to$ Hyperbolic):
   • Euclidean embeddings ($h^{(E)}$) from PLMs and PTMs are mapped to the Poincaré ball ($D^n_c$) using the Riemannian exponential map:
     $$h^{(H)} = \tanh(\sqrt{c} \|h^{(E)}\|) \frac{h^{(E)}}{\sqrt{c} \|h^{(E)}\|}$$
2. Geodesic-Cosine Similarity Attention (GCSA):
   • Instead of standard dot-product attention, GoCoMA uses Geodesic Cosine Similarity to compute attention weights between code and image embeddings within the hyperbolic manifold.
   • This utilizes Möbius linear maps for Query, Key, and Value transformations to ensure curvature consistency.
   • The similarity metric is defined as: $GCS(x, y) = \cos(\frac{\sqrt{c}}{2} d_c(x, y))$, where $d_c$ is the geodesic distance.
3. Aggregation & Back-Projection:
   • Features are aggregated using Möbius addition ($\oplus_c$) to preserve hierarchical dependencies.
   • The fused hyperbolic representation is mapped back to Euclidean space via the logarithmic map for final classification.

3. Key Contributions

1. Multimodal Perspective: First framework to jointly model source code and BPEA images as complementary signals, establishing an "extrinsic hierarchy" where code captures high-level semantics and BPEA captures low-level structural/optimization patterns.
2. Geometry-Aware Fusion: Introduction of GoCoMA, which leverages hyperbolic geometry (Poincaré ball) and GCSA to fuse modalities. This approach theoretically preserves tree-like growth and hierarchical relations better than Euclidean concatenation or attention.
3. Empirical Validation: Demonstrated consistent outperformance over strong unimodal baselines, Euclidean multimodal baselines, and state-of-the-art LLM prompting methods on two major benchmarks.

4. Experimental Results

The framework was evaluated on two open-source benchmarks:

• CoDET-M4: Large-scale, multi-language (C++, Java, Python), multi-domain dataset with human vs. LLM code.
• LLMAuthorBench: Fine-grained, 8-way attribution across closely related modern LLMs (e.g., GPT-4o, Claude-3.5, Llama 3.3).

Key Findings:

• Superior Performance: GoCoMA achieved the highest accuracy and macro-F1 scores.
  • On LLMAuthorBench, the best configuration (CodeT5+ + ViT with GCSA) reached 98.92% Accuracy and 93.82% Macro-F1.
  • On CoDET-M4, the same configuration achieved 88.67% Accuracy.
• Comparison with Baselines:
  • vs. Unimodal: Multimodal fusion significantly outperformed single-modality models.
  • vs. Euclidean Fusion: Hyperbolic fusion (Möbius addition) outperformed Euclidean concatenation and cross-attention. GoCoMA (GCSA) further improved upon standard hyperbolic fusion.
  • vs. LLM Baselines: GoCoMA significantly surpassed general-purpose LLMs (e.g., GPT-5.1, DeepSeek) used in zero-shot and few-shot settings. For instance, on CoDET-M4, GoCoMA (89.00%) vastly outperformed GPT-5.1 few-shot (75.20%).
• Ablation Insights:
  • Code PLMs alone are strong, but BPEA images provide complementary low-level signals.
  • Moving fusion to hyperbolic space yields the most significant gains, confirming that the hierarchical relationship between code and binary artifacts is best captured in negatively curved space.

5. Significance and Limitations

Significance:

• Forensic Capability: Provides a robust, model-agnostic method to trace code back to specific LLMs, addressing security and licensing concerns.
• Theoretical Advancement: Validates that hyperbolic geometry is superior for fusing multimodal data with hierarchical structures (high-level semantics vs. low-level bytes).
• Practical Application: Offers a "recipe" for high-accuracy attribution: pairing a high-capacity Code PLM with a ViT-based image encoder fused via GCSA.

Limitations:

• Generalization: Results are based on two specific benchmarks; generalization to other languages or build systems is unproven.
• Toolchain Sensitivity: The binary-to-image transformation is sensitive to compilers, optimization flags, and linkers, which can alter byte layouts.
• Static Analysis Only: The method analyzes static artifacts and omits runtime traces or build metadata.
• Robustness: Performance against style obfuscation, formatting normalization, or binary padding has not been tested.

In conclusion, GoCoMA represents a significant step forward in software forensics by effectively bridging the gap between high-level code semantics and low-level binary artifacts through the lens of hyperbolic geometry.