Publikations-Information

Download

Derek Wegerbauer, Revital Marbel, Harel Berger, Florian Alt
Fluent but Hollow: The Operational Limits of LLMs in DNS Security
In LCN'26: Proceedings of the 51st IEEE Conference on Local Computer Networks. September 29, 2026. IEEE. (bib)

Security practitioners increasingly use Large Language Models (LLMs) to extract operational parameters from attack literature for IDS configuration and incident response. However, the reliability of these parameters is poorly understood. We evaluate five state-of-the-art LLMs on sixteen canonical DNS cache-poisoning papers, comparing citation-only and document-grounded prompting via independent human annotation. We identify a systematic semantic-procedural gap: while models excel at narrative attack descriptions, they frequently hallucinate or fail to recover the precise quantitative parameters required for defensive deployment. Our analysis reveals a `Temporal Erosion' of technical accuracy, characterized by a performance drop of up to 20\% when processing historical literature. This creates a `Silent Failure' mode–-producing technically plausible but operationally incorrect parameters for legacy primitives. We demonstrate that while document grounding significantly improves performance, ungrounded LLMs introduce systemic risk by generating `operationally hollow' defenses. Our findings establish that document grounding is a mandatory but insufficient prerequisite for safety. Even with grounding, persistent stochastic errors in quantitative extraction mean that LLMs cannot yet be deployed in high-assurance security workflows without rigorous human-in-the-loop verification.