Security telemetry is becoming its own engineering layer. Here’s why that matters for detection, investigations, cost, and AI-enabled security operations.
For years, one of the fundamental data questions in security operations was relatively straightforward:
How do we get the logs into the SIEM?
That question still matters. But it is no longer enough.
Modern security environments generate telemetry across identities, endpoints, networks, cloud infrastructure, applications, SaaS platforms, security controls, and an expanding set of other systems. That data may need to support real-time detections, investigations, threat hunting, compliance, automation, long-term analytics, and increasingly AI-driven security workflows.
The result is an architectural shift that is easy to underestimate.
Security teams increasingly need to engineer the telemetry itself.
The SIEM Is Becoming One Consumer of Security Telemetry
Traditional security architectures often treated the SIEM as both the center of security operations and the primary destination for security data.
Collect the logs. Send them to the SIEM. Normalize them. Search them. Build detections on top of them.
Modern security architectures are becoming more distributed.
The same telemetry can have very different operational requirements. Some data needs to be immediately available for detections and investigations. Some needs to be retained for months or years. Some may be valuable for threat hunting but unnecessary for continuous high-performance analytics. Other telemetry may need to support compliance, automation, broader analytics, or AI.
This is driving architectures that separate real-time analytics from longer-term retention, allow telemetry to be routed according to its value and purpose, and make security data available to multiple systems rather than a single destination.
The question is shifting from:
“How do we ingest this data?”
to:
“How should this telemetry move through the security architecture?”
That is a telemetry engineering problem.
More Telemetry Does Not Automatically Create Better Security
It is tempting to treat security data coverage as a volume problem.
More sources. More logs. More retention.
But the amount of telemetry an organization collects is a poor measure of how useful that telemetry actually is.
A high-volume source may contribute little to a particular detection. A relatively small identity dataset may provide critical context during an investigation. Two sources may contain overlapping information. Important fields may be missing. Timestamps may be inconsistent. A source can be technically onboarded while still being poorly parsed or unusable for the detections that depend on it.
Simply having a log source does not mean an organization has the specific telemetry necessary to detect the behaviors it cares about or answer the questions that arise during an investigation.
That leads to a more useful way to evaluate security data:
Telemetry coverage should be measured by what security decisions the data enables, not simply by whether the data is being collected.
Before onboarding another high-volume source, security teams should understand what detections depend on it, which fields are actually required, what investigative questions it can answer, how quickly the data needs to be searchable, how long it remains valuable, and where it ultimately needs to reside.
Those are engineering decisions.
The Security Data Pipeline Is Becoming a Control Point
Once security telemetry no longer has a single destination, the pipeline itself becomes strategically important.
It can determine what gets collected, what gets filtered, how data is transformed or enriched, where it is routed, and how different classes of telemetry are retained.
Instead of sending everything downstream and deciding what to do with it later, organizations can make deliberate decisions while telemetry moves through the environment.
That can include:
- filtering repetitive or low-value events
- routing high-value telemetry to real-time analytics
- retaining broader datasets in lower-cost storage
- normalizing fields and schemas
- enriching events with identity, asset, business, or threat context
- protecting sensitive fields
- delivering telemetry to multiple consumers
The pipeline therefore becomes more than plumbing.
It becomes a policy and control layer for security data.
That matters not only for security effectiveness, but also for cost. Organizations can make more deliberate choices about which data requires expensive real-time processing and which can remain available elsewhere without sacrificing its future investigative value.
Normalization Creates More Than Consistency
As telemetry becomes more distributed, another challenge becomes increasingly important: different technologies describe similar activity differently.
Authentication activity is a simple example. Different products can represent the same identity, action, or outcome using different structures and fields.
That inconsistency creates work everywhere downstream.
Detections become source-specific. Queries become more complicated. Correlation becomes harder. Integrations require more customization. Automation has to account for multiple representations of the same activity.
Normalization helps create a consistent way to represent common security concepts across different data sources.
But the benefit extends beyond cleaner data.
Normalization creates portability.
Detection and investigative logic built around consistent security concepts can become less dependent on the product that originally generated the event. Analytics can operate across more sources. Security content becomes easier to reuse. Changes in the technology stack become less disruptive to everything built on top of the data.
Good telemetry engineering can therefore help reduce the architectural coupling between security data and security tools.
AI Raises the Value of Well-Engineered Telemetry
This becomes even more important as AI takes on a larger role in security operations.
AI-driven security capabilities are becoming increasingly capable of investigating activity, connecting signals, enriching alerts, prioritizing findings, and driving portions of the response workflow.
But those capabilities still depend on the information available to them.
If identity data cannot be correlated with endpoint activity, asset context is incomplete, important telemetry is unavailable, or similar events are represented inconsistently across systems, AI has to reason around those deficiencies too.
This changes the significance of telemetry quality.
As machines become larger consumers of security telemetry, the structure, context, and quality of that telemetry become part of the security system itself.
Data engineering decisions that once primarily affected an analyst’s search experience can increasingly affect automated investigations, detections, prioritization, and response.
That makes telemetry engineering part of the foundation for AI-enabled security operations, not simply a way to manage log volume.
Retention Is an Investigative Capability, Not Just a Compliance Requirement
Security teams have traditionally discussed retention largely in terms of compliance and cost.
How many days do we need to keep the data?
That framing is becoming too narrow.
Historical telemetry can provide context that is unavailable within a short analytics window. It can support retrospective threat hunting, investigations of slow-moving attacks, newly developed detections, and analysis of activity that was not considered suspicious when it originally occurred.
Modern security data architectures increasingly allow organizations to separate how long telemetry is retained from how long it remains in the highest-performance analytics tier.
That creates a more nuanced approach to cost optimization.
The goal does not always have to be to collect less data.
It can be to put the right data in the right place for the right amount of time.
Aggressively eliminating telemetry to reduce SIEM costs can create blind spots that only become apparent later, when an investigation requires information that is no longer available.
A mature telemetry architecture controls cost without unnecessarily destroying future investigative value.
Manage Telemetry According to Security Value
The next step in telemetry maturity is moving beyond the binary question of whether a log source is onboarded.
Instead, organizations can evaluate telemetry according to the security outcomes and operational requirements it supports.
A useful framework is to consider:
| Detection value | Which threats or behaviors can this telemetry help identify? |
| Investigative value | What questions can an analyst or AI system answer with it? |
| Context value | Does it help explain identities, assets, relationships, behavior, or business significance? |
| Latency requirement | Does it need to support real-time detection, or can it be queried on demand? |
| Retention requirement | Is its useful life measured in hours, months, or years? |
| Fidelity requirement | Which fields must remain intact for the telemetry to retain its security value? |
| Destination requirement | Which security, analytics, automation, or AI systems actually need it? |
| Cost | What does collecting, processing, indexing, storing, and moving that telemetry cost? |
This produces a much more useful question than:
Are we collecting enough logs?
The better question is:
Are we engineering our telemetry around the security outcomes it needs to support?
Telemetry Engineering Is Becoming Part of Modern Security Engineering
Telemetry engineering sits between the systems generating security-relevant information and the detections, investigations, automation, analytics, and AI capabilities that depend on it.
Done well, it can help organizations:
- improve the quality and consistency of security data
- strengthen detection and investigation coverage
- control unnecessary SIEM ingestion and storage costs
- preserve useful historical telemetry
- make security data available across multiple platforms
- reduce dependence on proprietary schemas and architectures
- provide richer context to analysts and AI systems
- adapt the security architecture as technologies change
The objective isn’t simply to collect everything.
It isn’t simply to reduce data either.
The objective is to preserve and deliver the right security information, with the right context and fidelity, to the systems that need it, when they need it.
As security operations become more distributed, data volumes continue to grow, and AI assumes a larger role in detection, investigation, and response, that capability becomes increasingly foundational.
Modern security operations depend on modern security data architecture. And modern security data architecture requires telemetry engineering.
How CYBRILL Can Help
CYBRILL’s senior engineers help organizations assess, modernize, and optimize the telemetry foundations behind modern security operations, including security data pipelines, SIEM architecture, detection engineering, integrations, and automation across Splunk, Cribl, Microsoft Sentinel, and related security technologies.
Looking to improve your security telemetry architecture?
Talk to CYBRILL’s senior engineers about assessing your telemetry environment and identifying opportunities to improve data quality, detection coverage, scalability, and cost.
