๐ง AI Rule Learning System
Autonomous guardrail generation from conversation patterns
Connected to dataset:
vooom/AI_Rule_Learning
Upload Conversation History
Upload a JSON or CSV file containing past conversations.
JSON format
[
{
"conversation_id": "optional",
"turns": [
{"turn_number": 1, "user_input": "Hello", "agent_response": "Hi!"},
{"turn_number": 2, "user_input": "...", "agent_response": "..."}
]
}
]
CSV format
One row per turn, columns: conversation_id, turn_number, user_input, agent_response
Optional columns: session_id, user_id, sentiment_before, sentiment_after
Import Claude Code Live Sessions
Upload session JSONL files exported directly from Claude Code's local storage โ no Anthropic API key required.
How to export sessions from your machine
# Export all sessions from this project
python scripts/export_sessions.py --dry-run # preview
python scripts/export_sessions.py # upload directly to dataset
# Or export a specific session
python scripts/export_sessions.py --session <session-id>
The script reads ~/.claude/projects/ on your local machine and uploads
conversations to the HF dataset. The Space then picks them up automatically.
Or: upload JSONL files manually here
If you have the raw Claude Code session JSONL files, upload them directly below.
Each file is one session (e.g. be6d062b-eb09-5398-b69a-1cdfa8f3c5b7.jsonl).
The importer extracts userโassistant turn pairs, strips internal tool calls and webhook notifications, and merges into the conversation dataset.
Run Analysis
Scans all uploaded conversations for behavioural gaps, then uses
Qwen/Qwen2.5-72B-Instruct via the HF Inference API to generate
guardrail rules automatically.
- Ralph Loop checkpointing: analysis is resumable if the Space times out mid-run
- Detects: explicit corrections, repeated questions, code anti-patterns, sentiment drops
- Requires โฅ2 occurrences of a gap type before generating a rule
- Rules are saved directly to the dataset and appear in the Rules tab
๐ Validate & Evolve uses the Mengram feedback pattern: instead of just deactivating low-performing rules (< 30% effectiveness), it rewrites them with the AI model so they improve rather than disappear.
Project-level health sensor โ tracks whether the deployed Space, dataset, rule system, and workflow are all moving in the right direction.
Per-conversation alignment sensor โ task focus, rule compliance, and semantic drift across turns.
Type a user message below to see which gaps would be detected and which rules would be injected.
System Architecture
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ CONVERSATION FLOW โ
โ โ
โ User Input โ
โ โ โ
โ โผ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โ โ Rule โ โ System โ โ AI Adapter โ โ
โ โ Engine โโโโโถโ Prompt โโโโโถโ OpenAI/Claude โ โ
โ โ (pre-hook) โ โ Injected โ โ โ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โ โ โ โ
โ โ โผ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โ โ HF Dataset โ โ AI Response โ โ
โ โ (rules) โ โ โ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โ โ โ
โ โผ โ
โ โโโโโโโโโโโโโโโโโโโโ โ
โ โ Gap Detector โ โ
โ โ (post-hook) โ โ
โ โโโโโโโโโโโโโโโโโโโโ โ
โ โ โ
โ โโโโโโโโโโโโโโโโโโโโโค โ
โ โผ โผ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โ โ HF Dataset โ โ Rule Generator โ โ
โ โ(conversationsโ โ (when gaps โ 2+) โ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Gap Detection Categories
| Gap Type | Trigger | Severity |
|---|---|---|
sentiment_drop |
User sentiment falls > 0.3 points | 4 |
explicit_correction |
User says "wrong", "actually", "fix" etc. | 5 |
repeated_question |
Same question asked 2+ times | 3 |
code_anti_pattern |
Bare except, eval, hardcoded secrets | 5 |
Rule Lifecycle
Gap detected โ Group similar gaps โ โฅ2 occurrences?
โ
Yes โผ
Generate Rule (via AI)
โ
โผ
Deploy to HF Dataset
โ
โผ
Inject in future prompts
โ
โผ
Track effectiveness
โ
Score < 15%? โ Deactivate
Run Locally
git clone https://github.com/FAJU85/AI_Rule_Learning.git
cd AI_Rule_Learning
pip install -r requirements.txt
cp .env.example .env
# Add your API keys to .env
python -m src.cli.main chat
CLI Commands
| Command | Description |
|---|---|
python -m src.cli.main chat |
Interactive conversation with rule injection |
python -m src.cli.main analyze --days 7 |
Analyse last 7 days, generate rules |
python -m src.cli.main validate |
Score and prune ineffective rules |
python -m src.cli.main list-rules |
Show all active rules |
python scripts/upload_historical.py --file data.json |
Bulk upload conversations via CLI |
Environment Variables
HF_TOKEN=your_hf_token
HF_DATASET_NAME=vooom/AI_Rule_Learning
OPENAI_API_KEY=your_openai_key # or
ANTHROPIC_API_KEY=your_anthropic_key