Data Normalization, Correlation & Threat Data Enrichment

These concepts often work together in security pipelines to transform raw event logs into actionable security intelligence.

 Data Normalization

What It Is

Data normalization is the process of standardizing logs and events from different systems into a common format.

Different devices produce logs in inconsistent formats:

• Firewalls
• Endpoint detection agents
• IDS/IPS
• Cloud services
• Authentication systems

Normalization makes them easy to process, query, and correlate.

🔍 Example

Raw logs:

• Cisco ASA: src=10.0.1.2 dst=8.8.8.8
• Windows Event: SourceNetworkAddress=10.0.1.2
• AWS CloudTrail: "sourceIPAddress": "10.0.1.2"

After normalization:

source.ip = 10.0.1.2
destination.ip = 8.8.8.8
event.type = connection_attempt

Why It Matters

• Enables analytics across multiple data sources
• Reduces complexity of SIEM rules
• Improves detection accuracy

 Data Correlation

What It Is

Correlation links together multiple normalized events to identify patterns or attacks that cannot be detected in individual logs.

It can be rule-based or machine-learning-driven.

🔍 Example Correlations

Brute force + successful login → Account Compromise

1. Multiple failed logins
2. Followed by a successful login
3. From the same IP

Lateral movement correlation:

• Authentication attempts
• File access logs
• Remote command executions

MITRE ATT&CK correlation:
Mapping event sequences to adversary techniques.

🎯 Why It Matters

Correlation turns noise into insight by linking events into a storyline, enabling detection of:

• Multi-step attacks
• Insider threats
• APT activity
• Coordinated external attacks

Threat Data Enrichment

✅ What It Is

Enrichment adds context to events by referencing external intelligence sources or internal metadata.

It transforms raw data into meaningful security intelligence.

Types of Enrichment

🔍 Example

A raw event:

source.ip = 45.155.205.10

After enrichment:

source.ip = 45.155.205.10
geo.country = Russia
threat.reputation = malicious
threat.confidence = high
asn = AS35415
ioc_type = known C2 server

🎯 Why It Matters

• Prioritizes alerts
• Provides context for faster investigations
• Reduces false positives
• Enables automated blocking/remediation

How These Three Work Together

1. Normalization → Standardizes logs
2. Enrichment → Provides additional context
3. Correlation → Connects enriched events to detect threats

Example Scenario
A failed login event from one source and a network connection from another source are normalized.
Enrichment reveals:

• The source IP is a known malicious actor.

Correlation shows:

• Multiple failed logins → followed by a successful login.
• Same malicious IP address.

➡️ High-confidence detection of account compromise.