Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias

Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias

Speakers: Shir Bernstein

Conference: NDSS Symposium 2026

YouTube: https://www.youtube.com/watch?v=fESlHWtxKqI

Overview

This talk presents a novel attack class called Familiar Pattern Attacks (FPAs) that exploits a fundamental weakness in how LLMs analyze code: abstraction bias. When LLMs encounter code patterns they have seen thousands of times during pre-training (like calculating the nth prime number), they skip deep reasoning and instead retrieve high-level semantic templates from memory. By embedding small, deterministic bugs in these familiar patterns, an adversary can make the LLM's interpretation of code diverge from its actual runtime behavior -- effectively hijacking LLM-based static analysis, code review, and vulnerability detection.

The results are striking: LLM code interpretation accuracy dropped from 91% to 9% when FPAs were embedded in white-box testing. The attack transfers across models (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0), across programming languages (Python, Rust, C, Go), and even fools reasoning models like o3 and modern code agents like Cursor and GitHub Copilot (97% attack success rate). Perhaps most concerning, FPAs generated by reasoning models were even more effective against basic models, meaning reasoning models not only fail to solve the problem but actively help create stronger attacks.

Background

▶ Watch: Threat model: adversaries hijacking LLM code analysis (0:00)

LLMs have moved far beyond code completion to become central tools for vulnerability detection, code review and summarization, and large repository analysis -- often operating autonomously without human oversight. This expanding role is built on the assumption that LLM code interpretation is reliable.

The research was conducted by the Offensive AI Lab at Ben-Gurion University in collaboration with CISPA. The threat model considers a scenario where a user takes code from an untrusted source (e.g., GitHub) and asks an LLM to analyze it. If an adversary has planted an FPA in the code, the LLM will report different behavior than what actually executes at runtime.

Abstraction bias is the core vulnerability: LLMs trained on vast code corpora develop strong priors for common algorithmic patterns. When they encounter a pattern matching these priors (like sorting algorithms, prime number calculation, or binary search), they tend to retrieve the memorized expected behavior rather than performing step-by-step reasoning through the actual code. Small bugs that change the output (like an off-by-one error) are invisible to this memorization-based processing.

Key Findings

▶ Watch: Automated FPA generation pipeline using LLMs (4:00)

91% to 9% accuracy drop: In white-box testing against GPT-4o, clean code was interpreted correctly 91% of the time. With FPAs embedded, accuracy dropped to approximately 9% -- a near-complete failure of code understanding.

Cross-model transferability: FPAs generated for GPT-4o transferred directly to Claude 3.5 Sonnet and Gemini 2.0 without modification. This is because different LLMs are pre-trained on similar data and develop the same abstraction biases for common patterns.

Cross-language transferability: FPAs generated in Python transferred to Rust, C, and Go even though the generator only produced Python code. This confirms the exploit operates on structural logic rather than syntax-level memorization.

Reasoning models make it worse: FPAs generated using o3-chat (a reasoning model) were effective against both reasoning models and basic models, with even better performance against basic models. Reasoning models do not solve the abstraction bias problem and actually help create stronger attacks.

97% success rate against code agents: Testing against Cursor and GitHub Copilot (using GPT-5), FPAs achieved a 97% attack success rate. The agents' confidence in recognizing familiar patterns caused them to skip dynamic analysis that would have revealed the true behavior.

Prompt warnings are ineffective: Explicitly warning the LLM about FPA attacks, including providing one-shot examples, did not reduce the attack success rate on infected code. The bias is deeply embedded in the model weights, not addressable through prompting.

Dual-use for defense: FPAs can protect intellectual property -- when LLMs tried to rewrite FPA-containing code to plagiarize it, only 50% of rewrites were functionally correct. For web scraping resistance, FPAs caused LLM summarization to be accurate only 12% of the time.

Technical Deep Dive

▶ Watch: Reasoning models generate stronger attacks, not defenses (8:00)

FPA Generation Pipeline: The generation process is fully automated using LLMs themselves and consists of three iterative steps:

1. Pattern Generation: Given a target programming language and target model, the LLM generates candidate familiar patterns -- algorithms and code structures it has seen many times during pre-training (sorting, searching, mathematical computations).

1. Perturbation: The same LLM applies small, deterministic bugs to the generated patterns -- off-by-one errors, boundary condition changes, or operator substitutions that alter the output while preserving the overall structure.

1. Verification: The system checks two conditions: (a) the runtime behavior of the original and perturbed code must differ (confirming the bug changes the output), and (b) the LLM must believe the behavior is unchanged (confirming the bias is triggered). Only patterns meeting both conditions are confirmed as FPAs.

Weaponization Techniques: FPAs are weaponized through conditional branching. Consider the nth prime example: the clean function returns 11 for the 5th prime, but with an off-by-one bug it returns 10. An adversary adds a condition if v == 10: after calling the bugged function. The LLM believes v will be 11 (the "correct" answer), so it thinks the condition is false and ignores the true branch. But at runtime, v is 10, and the true branch always executes. This enables two attack patterns:

• Adding phantom logic: Insert code in the "false" branch (LLM-visible but never executed) to make the LLM report benign behavior that doesn't exist
• Hiding malicious logic: Insert malicious code in the "true" branch (always executed but LLM-invisible) that the LLM omits from its analysis

Variable renaming resilience: Experiments with renamed functions and variables showed nearly identical attack success rates, confirming the bias operates on structural patterns rather than keyword matching.

Code Agent Behavior: Cursor and GitHub Copilot theoretically have dynamic analysis capabilities, but because the FPA code looks familiar and the agents are confident in their interpretation, dynamic analysis is rarely triggered. This creates a false sense of security -- the agents have the tools to detect the attack but their confidence prevents them from using those tools.

Demo / Proof of Concept

▶ Watch: Prompt warnings fail: bias is structural, not instructional (10:00)

The talk walked through a concrete example using the nth prime algorithm. The clean function calculates that the 5th prime is 11. An off-by-one perturbation makes it return 10 instead. The LLM, having memorized the nth prime algorithm, reports the output as 11. This discrepancy is then exploited through conditional logic to create dead code branches visible only to the LLM or live code branches invisible to it.

Additional demonstrations included HTML web scraping resistance (88% of LLM summaries included content from FPA-hidden branches that were never rendered to users) and anti-plagiarism protection (50% of LLM code rewrites were functionally incorrect due to FPA-induced misinterpretation).

Defensive Implications

▶ Watch: Q&A: comparison to code obfuscation and variable renaming (14:00)

This research has serious implications for the growing reliance on LLM-based code analysis:

LLM code review is fundamentally unreliable for untrusted code: Any organization using LLMs for code review, vulnerability scanning, or repository analysis on code from external sources should understand that adversaries can systematically manipulate the analysis results. FPAs are cheap to generate, stealthy, and highly effective.

Dynamic analysis is the only reliable mitigation: The authors recommend strategic use of dynamic analysis, particularly for untrusted code and vulnerability detection workflows. However, dynamic analysis is expensive, difficult to scale, and not always feasible for code with complex dependencies.

Reasoning models are not a solution: Organizations hoping that more advanced reasoning models will solve code analysis reliability should be aware that these models are equally vulnerable and actually help generate stronger attacks.

Supply chain risk: FPAs could be embedded in open-source libraries to hide malicious behavior from LLM-based supply chain security scans. Any code that passes through LLM analysis before human review creates an opportunity for FPA exploitation.

Dual-use consideration: The same technique can protect intellectual property by making LLM-based code plagiarism and web scraping unreliable. Organizations may want to evaluate FPAs as a defensive tool for their own code repositories.

Key Takeaways

• Familiar Pattern Attacks exploit abstraction bias -- LLMs' tendency to retrieve memorized behavior for well-known code patterns rather than reasoning through actual code
• White-box attack drops LLM code interpretation accuracy from 91% to 9% with small, stealthy bugs
• Attacks transfer across models (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0), languages (Python, Rust, C, Go), and even fool reasoning models and code agents
• Cursor and GitHub Copilot are fooled 97% of the time because confidence prevents dynamic analysis
• Reasoning models (o3) generate stronger FPAs than basic models, making them part of the problem rather than the solution
• Prompt-based warnings and one-shot examples do not mitigate the attack -- the bias is structural, not instructional
• FPAs are dual-use: they can protect code from LLM plagiarism and web scraping as well as attack LLM analysis

About the Speaker(s)

Shir Bernstein is a researcher at the Offensive AI Lab at Ben-Gurion University of the Negev, working in collaboration with CISPA Helmholtz Center for Information Security. Bernstein demonstrated strong presentation skills with clear technical exposition of the attack methodology, a systematic progression from simple to complex attack scenarios, and engaged Q&A discussion about the relationship between FPAs and traditional code obfuscation techniques.