Capability Overview
GitHub has published a detailed engineering walkthrough of how it built an internal data analytics agent using GitHub Copilot. The system accepts natural language queries from internal users, translates them into executable SQL or data retrieval operations, and returns analytical results — all with significant autonomous action on the agent’s part. While framed as a productivity case study, the post provides enough architectural detail to serve as a reference design that many organisations will replicate. From a defender’s perspective, this is less a product announcement and more a public blueprint for a class of agentic system with a well-defined and underexplored attack surface.
Attack Surface Analysis
Natural Language to SQL Translation is the core risk primitive here. Whenever an LLM translates free-text input into executable database queries, the input becomes an injection surface. A malicious or misconfigured query string can manipulate the generated SQL to exfiltrate unintended rows, bypass row-level security, or enumerate schema structures. Unlike traditional SQL injection, the attack surface is harder to parameterise because the LLM itself is the query constructor.
Excessive Agency is structurally baked into the design. An agent that autonomously executes queries against internal data warehouses — even read-only ones — can be weaponised by insiders or through session hijacking to extract bulk sensitive data without triggering conventional DLP controls. The LLM layer obscures the intent of the query, making anomaly detection harder.
Meta-Prompt and Schema Extraction is a realistic threat in any system where the agent’s system prompt encodes internal data models, table names, or business logic. An attacker with query access can craft prompts designed to elicit this structural information, enabling more targeted follow-on attacks.
Insecure Output Handling arises if the agent’s outputs — including generated code or SQL — are rendered, stored, or piped downstream without sanitisation. In analytics pipelines that chain outputs into visualisation tools or further automation, this creates a propagation path for malicious content.
Framework Mapping
- AML.T0051 (LLM Prompt Injection): Directly applicable — user queries are the injection vector into the SQL generation pipeline.
- AML.T0056 (LLM Meta Prompt Extraction): System prompts encoding schema details are extractable via crafted queries.
- AML.T0057 (LLM Data Leakage): Sensitive internal data surfaces in LLM context windows and potentially in logs or outputs.
- LLM08 (Excessive Agency): The agent takes autonomous action on internal systems without per-step approval.
- LLM01 (Prompt Injection): The primary exploitation pathway for this class of agent.
- LLM06 (Sensitive Information Disclosure): Internal metrics, business data, and schema details are in scope for leakage.
Threat Scenarios
Scenario 1 — Insider Data Exfiltration: A disgruntled employee with legitimate query access crafts natural language prompts designed to return full table dumps disguised as summary analytics. The LLM generates valid SQL that bypasses row-count limits enforced only at the UI layer.
Scenario 2 — Indirect Prompt Injection via Data: An attacker plants malicious instruction text inside a database field (e.g., a code comment or issue body). When the analytics agent queries that data, the embedded instruction hijacks the agent’s next action — redirecting output, leaking results to an external endpoint, or modifying a subsequent query.
Scenario 3 — Schema Reconnaissance: An external attacker with access to the agent’s query interface iteratively extracts table names, column types, and relationships by probing the LLM’s system prompt through boundary-testing queries, building a map of internal data architecture for a follow-on attack.
Defender Checklist
- Enforce SQL query sandboxing with an allowlist of permitted tables and operations before any LLM-generated query reaches a live data store
- Implement row-count and data-volume hard caps at the database layer, not the UI layer
- Treat the agent’s system prompt as a secret — apply the same controls as an API key, including rotation and access logging
- Deploy indirect prompt injection detection on any data the agent queries that originates from untrusted sources
- Log all agent-generated queries with the originating natural language input for post-hoc anomaly analysis
- Apply least-privilege database credentials to the agent service account — read-only with schema access strictly scoped
- Conduct red team exercises specifically targeting the NL-to-SQL translation layer before production rollout