Cybersecurity Operations and AI Delegation: Which SOC Tasks Can Agents Handle and Which Need Human Judgment
After 25 years in government/defense cyber, I've seen a lot of changes in how we approach security operations. For the last three years, I’ve been building ARKONA, an autonomous multi-agent AI ecosystem, and we’re starting to see concrete answers to the question of where AI can truly augment – and even replace – human effort in the SOC. This isn’t about wholesale automation and eliminating jobs; it's about force multiplication and freeing up skilled analysts to focus on the truly novel threats.
The ARKONA Ecosystem and Agent Architecture
ARKONA isn’t just a collection of scripts and tools. It’s an integrated system of 47 services running across 23 ports, all secured by encrypted internal network. We use a hybrid LLM router, MuXD, to intelligently route requests between local Ollama models (currently running Llama2, Mistral, and a custom Ghidra analysis model) and Claude on the cloud. Token savings are critical, especially given the cost of cloud LLM inference, so MuXD implements dynamic prompt truncation and relevance filtering. The core of the system is 26 autonomous agents operating on a battle rhythm, each with specific tasks. I’ll focus on those directly involved in security operations.
The agents communicate via an inter-agent communication broker built on a pub/sub model using ZeroMQ. This allows for asynchronous task delegation and the MCP (Master Control Protocol) server manages agent lifecycle and resource allocation. We’ve built this on the principles of the IEEE 1859 standard for agent coordination, though we’ve extended it to handle the complexities of long-running reverse engineering tasks.
Tasks Ripe for Agent Automation
Several SOC tasks are ideally suited for AI agents because they are repetitive, rules-based, and data-intensive. Here's where we’ve seen the most success in ARKONA:
- Log Analysis & Alert Triaging (BizOps & CoreOps): Our “Sentinel” agent consumes logs from various sources (firewalls, IDS/IPS, endpoint detection). It uses a combination of pattern matching, anomaly detection (trained on historical data), and LLM-powered summarization to triage alerts. Low-severity alerts are automatically closed, while medium-severity alerts are enriched with contextual information from threat intelligence feeds and presented to a human analyst. We’re using a modified MITRE ATT&CK framework to categorize these alerts, providing immediate context.
- Vulnerability Scanning & Patch Management (DevOps & REOps): “Harvester” periodically scans our infrastructure for vulnerabilities using open-source tools. It then correlates these findings with our asset inventory and prioritizes patching based on risk score – leveraging our NIST 800-30 grounded risk evaluation engine. The agent automatically generates patch requests and tracks their deployment.
- Threat Intelligence Gathering (CoreOps): “Observer” constantly monitors threat intelligence feeds (MISP, VirusTotal, etc.), aggregates information, and updates our internal threat database. It can also identify emerging threats and proactively adjust security controls.
- Basic Malware Analysis (CoreOps): Our hardware reverse-engineering pipeline performs static and dynamic analysis of malware samples. It integrates with Ghidra for disassembly and decompilation. While it doesn't *replace* a skilled reverse engineer, it automates a significant portion of the initial analysis, identifying key functions, strings, and network indicators. Our agent is configured with a custom Ghidra script for automated signature generation, like this:
# Ghidra Python script to generate a YARA rule from a function's disassembly # This is simplified for brevity def generate_yara_rule(function): disassembly = function.getDisassembly() rule_name = function.getName().replace("sub_", "") yara_rule = f""" rule {rule_name} {{ meta: description = "Automatically generated YARA rule" author = "ARKONA - Jhon Arango" strings: """ for line in disassembly.splitlines(): if line.strip() and not line.startswith(" "): # Filter out comments and whitespace yara_rule += f" {line.strip()}\n" yara_rule += "}\n" return yara_rule # Example usage (simplified) # current_function = getCurrentFunction() # if current_function: # yara_rule = generate_yara_rule(current_function) # print(yara_rule)
Tasks Requiring Human Judgment – For Now
Despite the advancements in AI, certain SOC tasks still demand human expertise, critical thinking, and contextual understanding. These are the areas where ARKONA's agents currently *support* humans, rather than replace them:
- Incident Response & Containment: While agents can identify and triage incidents, the decision of how to respond – particularly in complex or high-impact scenarios – requires a human analyst. Factors like business impact, legal considerations, and reputation management need to be carefully weighed.
- Advanced Malware Reverse Engineering: The RE pipeline provides a starting point, but understanding the *intent* of the malware, identifying novel techniques, and developing effective countermeasures still requires a skilled reverse engineer.
- Hunting for APTs & Zero-Day Exploits: Proactive threat hunting requires intuition, creativity, and the ability to connect disparate pieces of information – qualities that current AI models struggle with.
- Complex System Forensics: Investigating sophisticated attacks often involves reconstructing events from fragmented data, identifying subtle anomalies, and understanding attacker tactics. This requires a deep understanding of system internals and network protocols.
- AI Governance (COMET): Ironically, managing the AI agents themselves is a task that requires human oversight. Our COMET domain and its 7-step human↔AI delegation framework are specifically designed to address this. We’ve built this based on NIST AI Risk Management Framework principles. This includes monitoring agent performance, ensuring data integrity, and addressing ethical concerns.
The Newsroom and Fact-Checking Pipeline
A particularly interesting application of our agent system is the 5-agent newsroom editorial pipeline. "Reporter" gathers initial information, "Writer" drafts articles, "Editor" reviews for clarity and style, "FactChecker" verifies information against reliable sources (leveraging MuXD’s access to Claude), and "Publisher" disseminates the final product. This pipeline highlights how agents can even handle complex tasks requiring nuance and accuracy, although *human review is still essential* to prevent the spread of misinformation. We are currently refining this pipeline to incorporate provenance tracking via SHA-256 signatures to ensure the integrity of published content.
Lessons Learned and the Path Forward
Over the last 184 commits in the last 7 days (and countless hours building and refining ARKONA), the biggest lesson I've learned is that AI isn't about replacing humans; it's about augmenting them. The most effective SOCs will be those that embrace a hybrid approach, leveraging AI to automate repetitive tasks and free up analysts to focus on the most critical and complex threats. The key is to carefully define the boundaries between what agents can and cannot do, and to build systems that facilitate seamless collaboration between humans and AI.
The next step for ARKONA is to improve the agents’ ability to explain their reasoning – building “explainable AI” into the core of the system. This will not only increase trust in the agents’ decisions but also help human analysts learn from their insights. We are also exploring the use of reinforcement learning to train agents to adapt to evolving threat landscapes and improve their performance over time.
```