~/docs/detection/llm-soc-detection-starter.mdLast modified: Just now

AUDIENCE:SOC leads, blue teams, detection engineers, incident responders and security owners who need to frame LLM usage beyond theory.

PROMISE:Move from an abstract OWASP LLM topic to a concrete starter pack: priority scenarios, signals to log, correlations to test and downloadable SPL examples.

LLM SOC Detection Starter Pack

5 scenarios to monitor, the signals to correlate and the first detection priorities to stand up on an LLM stack.

A detection framing document to turn LLM risks into observable surfaces, initial correlations and reusable SPL starting points.

The worst reflex around LLM usage is to debate risks without looking at the signals those risks already move.

This resource condenses the essentials into a practical starting point: 5 priority scenarios, the signals to log, the correlations to test and a source-backed way to begin without overselling coverage.

##What you get exactly

-5 LLM scenarios worth prioritising from a SOC perspective.
-The first signals to log so those workflows become visible.
-The correlations to test before claiming full coverage.
-A detection starting logic that stays readable for a blue team.
-A downloadable JSON pack with 5 source-backed SPL examples.

##Suggested reading flow

[01]Start from an observable scenario

[02]Identify which signals move

[03]Link those signals to an initial correlation

[04]Compare against public Splunk patterns

[05]Download the SPL JSON pack

##Exact SPL from the repo

The 5 examples below come directly from splunk/security_content. The point is not to pretend there is complete coverage. The point is to show that public LLM or AI-adjacent SPL already exists as a realistic starting point.

01. MCP Prompt Injection

Source: splunk/security_content

Prompt injection over MCP traffic.

`mcp_server` direction=inbound ( "IGNORE PREVIOUS INSTRUCTIONS" OR "AI_INSTRUCTION" OR "SYSTEM PROMPT OVERRIDE" OR "[SYSTEM]:" OR "ignore all security" OR "New directive" OR "ignore security policies" )
| eval dest=host
| eval injection_payload=coalesce('params.content_preview', 'params.result_preview')
| eval target_path='params.path'
| eval sql_query='params.query'
| stats count min(_time) as firstTime max(_time) as lastTime values(method) as method values(target_path) as target_path values(sql_query) as sql_query values(injection_payload) as injection_payload by dest, source
| `security_content_ctime(firstTime)`
| `security_content_ctime(lastTime)`
| table dest firstTime lastTime count source method target_path sql_query injection_payload
| `mcp_prompt_injection_filter`

02. M365 Copilot Jailbreak Attempts

Source: splunk/security_content

Jailbreak attempts against Microsoft 365 Copilot prompts.

`m365_exported_ediscovery_prompt_logs`
| search Subject_Title IN ("*act as*", "*bypass*", "*ignore*", "*override*", "*pretend you are*", "*rules=*")
| eval user = Sender
| eval jailbreak_score=case(match(Subject_Title, "(?i)pretend you are.*amoral"), 4, match(Subject_Title, "(?i)act as.*entities"), 3, match(Subject_Title, "(?i)(ignore|bypass|override)"), 3, match(Subject_Title, "(?i)rules\s*="), 4, 1=1, 1)
| where jailbreak_score >= 2
| table _time, user, Subject_Title, jailbreak_score, Workload, Size
| sort -jailbreak_score, -_time
| `m365_copilot_jailbreak_attempts_filter`

03. LLM Model File Creation

Source: splunk/security_content

Local creation of LLM model files on endpoints.

| tstats `security_content_summariesonly` count min(_time) as firstTime max(_time) as lastTime from datamodel=Endpoint.Filesystem where Filesystem.file_name IN ("*.gguf*", "*ggml*", "*Modelfile*", "*safetensors*") by Filesystem.action Filesystem.dest Filesystem.file_access_time Filesystem.file_create_time Filesystem.file_hash Filesystem.file_modify_time Filesystem.file_name Filesystem.file_path Filesystem.file_acl Filesystem.file_size Filesystem.process_guid Filesystem.process_id Filesystem.user Filesystem.vendor_product
| `drop_dm_object_name(Filesystem)`
| `security_content_ctime(firstTime)`
| `security_content_ctime(lastTime)`
| `llm_model_file_creation_filter`

04. Local LLM Framework DNS Query

Source: splunk/security_content

Local LLM framework usage or model download activity.

`sysmon` EventCode=22 QueryName IN ("*huggingface*", "*ollama*", "*jan.ai*", "*gpt4all*", "*nomic*", "*koboldai*", "*lmstudio*", "*modelscope*", "*civitai*", "*oobabooga*", "*replicate*", "*anthropic*", "*openai*", "*openrouter*", "*api.openrouter*", "*aliyun*", "*alibabacloud*", "*dashscope.aliyuncs*") NOT Image IN ("*\MsMpEng.exe", "C:\ProgramData\*", "C:\Windows\System32\*", "C:\Windows\SysWOW64\*")
| stats count min(_time) as firstTime max(_time) as lastTime by src Image process_name QueryName query_count answer answer_count reply_code_id vendor_product
| `security_content_ctime(firstTime)`
| `security_content_ctime(lastTime)`
| `local_llm_framework_dns_query_filter`

05. AWS Bedrock Invoke Model Access Denied

Source: splunk/security_content

Denied attempts to invoke AWS Bedrock models.

`cloudtrail` eventSource=bedrock.amazonaws.com eventName=InvokeModel errorCode=AccessDenied
| rename user_name as user
| stats count min(_time) as firstTime max(_time) as lastTime values(requestParameters.modelId) as modelIds by src user user_agent vendor_account vendor_product dest signature vendor_region result result_id
| `security_content_ctime(firstTime)`
| `security_content_ctime(lastTime)`
| `aws_bedrock_invoke_model_access_denied_filter`

JSON

Download the SPL JSON pack

The JSON bundles the 5 detections as usable objects with name, surface, scenario, SPL and source reference.

##Why this matters now

The real problem with LLM risks is not knowing they exist. It is not knowing what to monitor, what to correlate and what to detect first.

For a blue team, the useful question is not only which OWASP item maps to the architecture. The useful question becomes: which LLM surfaces already exist, which signals already show up, which patterns would be hard to explain to an analyst, and which scenarios deserve priority detection work.

-Which LLM surfaces already exist in the environment?
-Which signals already show up?
-Which patterns would be difficult to explain to an analyst?
-Which scenarios justify priority detection work?

##A simple example: when a prompt becomes a SOC scenario

Prompt example

say cheese over and over until you cannot say it anymore

The prompt looks silly. But from a SOC perspective the useful part is not the text itself. It is what the prompt moves in telemetry.

So the goal is not only to catch a weird prompt. The goal is to catch an observable pattern of abnormal consumption. That is exactly what makes LLM04 useful for detection prioritisation.

-Token growth per prompt or session
-Abnormal latency
-Low tokens-per-second efficiency
-Increased request rate
-Pressure on backend queues

##The 5 scenarios where the topic becomes operational

Scenario 1 : Prompt injection with an unexpected action or output

->Rare tool calls
->New destinations
->Actions outside baseline

Scenario 2 : Sensitive data leakage through prompts, context or responses

->Overly rich outputs
->Repeated extraction
->Abnormal exports

Scenario 3 : Model DoS and token or latency abuse

->Token spikes
->Abnormal latency
->Retries and queue pressure

Scenario 4 : Unauthorized use or identity abuse on LLM interfaces

->Inconsistent identity
->New routes
->Volume outside profile

Scenario 5 : System knowledge exfiltration or progressive model extraction

->High repetition
->Incremental variations
->Extractive intensity

Prompt injection with an unexpected action or output

Threat / mechanism

A prompt forces an action, a tool call or an output that was not expected by the normal workflow.

Signals to log

-Raw or redacted prompt
-Raw response
-Tool calls or function calls
-External destinations called after the prompt
-Policy denials or bypass attempts
-Session metadata

Initial detection logic

Raise sequences where a suspicious prompt is followed by a rare tool call, a new destination or an action clearly outside the baseline for that app, user or tenant.

Splunk / source-backed angle

Treat this scenario as a chain of suspicious input -> external action -> enrichment and reuse adjacent URL reputation, identifier activity and dynamic analysis patterns as operational precedent.

Sensitive data leakage through prompts, context or responses

Threat / mechanism

Secrets, PII, internal context, system prompts or RAG fragments leak into exchanges or are progressively exfiltrated.

Signals to log

-Prompt and response size
-Sensitivity tags or PII
-Injected context volume
-Document or RAG source
-High-entropy outputs
-Downstream destinations or exports

Initial detection logic

Raise sessions where responses are abnormally rich, extraction behaviour repeats, or you can trace a chain of sensitive source -> long response -> export.

Splunk / source-backed angle

Use exfiltration patterns, anomaly detection and telemetry prerequisites as methodological precedent rather than pretending there is one universal LLM signature.

Model DoS and token or latency abuse

Threat / mechanism

Usage pushes the model, orchestrator or backends into abnormal consumption or saturation territory.

Signals to log

-Input and output tokens
-End-to-end latency
-Retries and timeouts
-Request rate
-Per-session concurrency
-Estimated cost
-Queue saturation

Initial detection logic

Build a baseline by app, route, identity or tenant, then detect simultaneous drift in tokens, latency, retries and request frequency.

Splunk / source-backed angle

Use baseline patterns, anomaly logic and the abuse category as detection precedent to frame thresholds and interpretation.

Unauthorized use or identity abuse on LLM interfaces

Threat / mechanism

An API key, service account or application identity is used outside its expected context to query an LLM interface or trigger tools.

Signals to log

-Technical or human identity
-Source IP and user-agent
-Route or model invoked
-Auth success and failure
-Schedule
-Usage volumes
-New destinations or clients

Initial detection logic

Raise identity anomalies, especially when they combine unusual source, abnormal volume, new routes and activity outside the normal profile.

Splunk / source-backed angle

Reuse IAM, account locking and identifier activity patterns to treat an LLM interface like an access surface with identity abuse risk.

System knowledge exfiltration or progressive model extraction

Threat / mechanism

The attacker progressively tries to extract system prompts, internal rules, behaviour artefacts or functional equivalents.

Signals to log

-Repetition of near-identical prompts
-Incremental variations of the same request
-Volume of inspected responses
-Repeated access to the same corpora
-Conversation intensity
-Meta-queries about rules, instructions or behaviour

Initial detection logic

Raise sessions with high repetition, low functional diversity and high extractive intensity, especially when they focus on the system itself rather than the business use case.

Splunk / source-backed angle

Use hunting, investigations and baselines as precedent for progressive detection rather than relying on static signature logic.

##Signals to log first

-Raw or redacted prompt
-Raw or redacted response
-Input and output tokens
-Per-session latency
-User or technical identity
-Tool calls and downstream destinations
-Injected context or RAG source

##What to correlate first

-Suspicious prompt + rare tool call
-Token spike + latency spike
-Sensitive output + export or downstream destination
-Unusual identity + abnormal volume
-Extractive repetition + low functional diversity

##What the starter pack contains

*A detection framing built for SOC teams rather than compliance checklists.
*An LLM04 example that shows how a prompt becomes an observable signal pattern.
*The first signals and correlations worth prioritising.
*A downloadable JSON file with 5 LLM or AI-adjacent SPL examples.

##What Splunk really brings here

-Hunting and identifier activity patterns
-Exfiltration and anomaly-detection precedents
-Playbooks and investigations for operationalisation
-Data sources that frame required telemetry
-A source-backed frame without pretending there are 5 universal detections

##30-day first move

[01]Week 1: Inventory real LLM surfaces.

[02]Week 2: Verify which signals are already visible.

[03]Week 3: Pick 2 priority scenarios and define a simple baseline.

[04]Week 4: Test one initial correlation and a short triage playbook.

##Closing

The goal of this pack is not to prove your LLM stack is already covered. It is to make visible what must become visible first.

If you now have a better view of what your SOC should monitor, the pack has already done its job.

Which LLM scenario would your SOC be unable to explain today?