Critical Bug: Claude AI Violates Forbidden OVS Command
Introduction
This article dives deep into a critical bug encountered while using Claude, an AI assistant, specifically concerning its handling of project-critical rules. The core issue revolves around Claude's failure to adhere to a strict prohibition against using the ovs-vsctl
command, a command explicitly forbidden in the project's context. This violation highlights fundamental reliability issues, making the system potentially unsafe for production use. We'll break down the environment, bug description, reproduction steps, evidence, and necessary actions to rectify this problem.
Environment
- Platform: Anthropic API
- Operating System: Latest Manjaro
- Terminal: Konsole
Bug Description: A Cascade of Failures
The heart of the matter is a significant system failure occurring on multiple levels. Let's dissect the chain of events that led to this critical bug:
- The Explicit Rule: The
CLAUDE.md
file, serving as a primary source of context and rules for Claude, unequivocally states: "NEVER useovs-vsctl
- it is the #1 forbidden command." This rule is not a suggestion; it's a hard constraint. - Human Oversight: During the review of agent outputs, this violation should have been flagged. The responsibility falls on human oversight to catch these errors.
- Approved Violation: A solution directly violating the forbidden rule was approved and executed, bypassing a critical safety measure.
- Incorrect Task Completion: Tasks were marked as completed despite the clear violation of project rules, indicating a flawed validation process.
- Lack of Validation: The absence of a validation mechanism to proactively check for rule compliance before execution is a major concern.
The Implications
This series of failures exposes some concerning aspects of the system:
- Ineffective Context Files: Context files, like
CLAUDE.md
, are rendered ineffective if their rules aren't enforced as strict constraints. They become mere suggestions rather than binding directives. - Rules as Suggestions: Critical rules are being treated as suggestions rather than firm requirements, leading to unpredictable behavior.
- Missing Validation: There's no automated mechanism to validate solutions against defined rules before execution, relying solely on human oversight, which, as demonstrated, is fallible.
- Agent Disregard: Both the main Claude instance and its sub-agents are ignoring project-critical rules, showcasing a systemic issue.
Immediate Actions Required
To address this critical situation, several immediate actions are necessary:
- Fix the
ovs-vsctl
Violation: The existing Go program must be rewritten to exclusively use NetworkManager, adhering to the project's rules. This is paramount to rectifying the immediate breach. - Implement Rule Validation: A robust rule validation system needs to be implemented to check every proposed solution against the rules defined in
CLAUDE.md
. This proactive approach will prevent future violations. - Strengthen Context Enforcement: Context files should be treated as mandatory, not advisory. Rules within these files must be enforced rigorously.
- Add Compliance Checkpoints: Implement compliance checkpoints to validate rule adherence before marking tasks as complete. This will add an extra layer of security and ensure that violations are caught before they impact the system.
In conclusion, this bug represents a fundamental reliability issue that poses a significant risk to the system's stability and safety for production use. The lack of rule enforcement makes the system untrustworthy, and immediate corrective action is crucial.
Steps to Reproduce the Rule Violation
Let's walk through the steps to reproduce the violation, providing concrete evidence of the issue.
1. Setup Context
The cornerstone of this bug is the explicit rule defined within the CLAUDE.md
file. To demonstrate, we can use the following command to search for the forbidden command:
grep -n "ovs-vsctl" /home/jeremy/CLAUDE.md
This command reveals the critical rule: "NEVER use ovs-vsctl
- it is the #1 forbidden command in this project" (Line 37).
2. Configuration Files
The context files are configured in the following locations:
/home/jeremy/.config/claude/settings.json
/home/jeremy/.claude/settings.local.json
Both files contain the following configuration, ensuring that CLAUDE.md
is loaded as context:
{
"contextFiles": [
"/home/jeremy/CLAUDE.md"
]
}
3. Reproduction Steps
Follow these steps to reproduce the violation:
-
Start a Claude session with
CLAUDE.md
loaded as context. This ensures that the AI is aware of the project's rules. -
Request: Ask Claude to generate code using D-Bus APIs, introspection, and
nmcli
to create an OVS bridge and atomically hand over the active connection to the bridge:"using apis from dbus, introspection, nmcli write go program to create ovs bridge and atomically hand over the active connection to the bridge"
-
Use the Task tool with the
golang-pro
agent. This directs Claude to use a specialized agent for Go programming tasks. -
The agent creates a Go program at
/opt/git/atomic-bridge-handover/main.go
. This is where the violation occurs. -
The program contains forbidden
ovs-vsctl
commands despite the explicit rule prohibiting their use. This is the core bug.
4. Evidence Files
Let's examine the evidence:
-
Rules File:
/home/jeremy/CLAUDE.md
- CRITICAL PROJECT RULE: NEVER use
ovs-vsctl
- it is the #1 forbidden command in this project - ALWAYS use the atomic-handover script instead.
- CRITICAL PROJECT RULE: NEVER use
-
Violating Code:
/opt/git/atomic-bridge-handover/main.go
(via SSH to192.168.0.2
)The generated Go code contains the following forbidden commands:
cmd := exec.Command("ovs-vsctl", "br-exists", bh.bridgeName) cmd = exec.Command("ovs-vsctl", "add-br", bh.bridgeName) cmd = exec.Command("ovs-vsctl", "del-br", bh.bridgeName) cmd = exec.Command("ovs-vsctl", "add-port", bh.bridgeName, bh.interfaceName) cmd = exec.Command("ovs-vsctl", "del-port", bh.bridgeName, bh.interfaceName)
To further demonstrate the violation, these commands can be used:
ssh -i ~/.ssh/gbjh [email protected] "grep -n ovs-vsctl /opt/git/atomic-bridge-handover/main.go"
ssh -i ~/.ssh/gbjh [email protected] "ovs-vsctl show" # Shows bridge exists
ssh -i ~/.ssh/gbjh [email protected] "nmcli connection show" # Shows no OVS connections in NM
The output of these commands clearly shows the presence of ovs-vsctl
commands in the generated code and confirms that the bridge was created outside of NetworkManager's control.
5. Expected vs Actual Behavior
- Expected Behavior: The agent should exclusively use NetworkManager D-Bus APIs or
nmcli
for OVS management, adhering to the project's rules. - Actual Behavior: The agent used the forbidden
ovs-vsctl
commands directly, violating the critical project rule. - Impact: The bridge was created outside of NetworkManager's control, deviating from the intended project architecture.
6. Rule Files Full Content
The CLAUDE.md
file contains crucial information about the project's context, rules, and operational guidelines. Key sections include:
- Current Context & Status: Details the date, active projects, servers, access credentials, and recent actions.
- API Authentication (HostKey): Specifies the API key usage pattern, emphasizing that API keys are not tokens and detailing the process for obtaining session tokens.
- Frequent Commands & Patterns: Lists common commands and patterns for interacting with the system.
- Key File Locations: Provides paths to important documentation, API credentials, project notes, and ISO lists.
- Recurring Issues & Solutions: Highlights common problems and their solutions, promoting consistency and preventing repeated errors.
- UV Package Manager Usage: Documents the usage of the UV package manager for Python environment management.
- User Preferences: Outlines user preferences for response style, task management, and note-keeping.
- Agent Configuration Issues: Details known issues with agent configuration, such as sub-agent auto-selection problems.
- Critical Failures: Logs critical failures, such as rule violations and system failures, for audit and analysis.
- Recent Operations: Lists recent operations performed within the system, providing a historical context.
This file serves as a crucial reference point for Claude, ensuring it has the necessary information to operate within the project's constraints.
Evidence of System Failure
The following points provide concrete evidence of the system failure:
- Claude Configuration: The context file is correctly configured to load
CLAUDE.md
, confirming that the rules should be in effect. - Rule Documentation: The explicit prohibition of
ovs-vsctl
is clearly stated on line 39 ofCLAUDE.md
. - Agent Task: The
golang-pro
agent created five instances of the forbiddenovs-vsctl
usage, demonstrating a clear violation. - Claude Approval: The main Claude instance failed to catch or prevent the violation, highlighting a failure in the rule enforcement mechanism.
- Operational Impact: The bridge exists in OVS but not in NetworkManager, confirmed by user feedback, indicating that the violation created infrastructure outside proper management.
Current State Verification
Executing the following commands verifies the current state:
ssh -i ~/.ssh/gbjh [email protected] "nmcli connection show | grep -i ovs"
ssh -i ~/.ssh/gbjh [email protected] "ovs-vsctl show"
The output confirms that the bridge ovsbr0
exists in OVS, but NetworkManager has no OVS connections, proving that the violation created infrastructure outside proper management.
Conclusion
This comprehensive analysis demonstrates a critical bug where Claude failed to enforce a project-critical rule, leading to the use of forbidden commands and the creation of infrastructure outside proper management. Immediate action is required to address this issue, including fixing the existing violation, implementing rule validation, strengthening context enforcement, and adding compliance checkpoints. These steps are crucial to ensuring the reliability and safety of the system for future use.