feat: Major directory reorganization and cleanup
Reorganized project structure for better maintainability and reduced disk usage by 95.9% (11 GB -> 451 MB). Directory Reorganization (85% reduction in root files): - Created docs/ with subdirectories (deployment, testing, database, etc.) - Created infrastructure/vpn-configs/ for VPN scripts - Moved 90+ files from root to organized locations - Archived obsolete documentation (context system, offline mode, zombie debugging) - Moved all test files to tests/ directory - Root directory: 119 files -> 18 files Disk Cleanup (10.55 GB recovered): - Deleted Rust build artifacts: 9.6 GB (target/ directories) - Deleted Python virtual environments: 161 MB (venv/ directories) - Deleted Python cache: 50 KB (__pycache__/) New Structure: - docs/ - All documentation organized by category - docs/archives/ - Obsolete but preserved documentation - infrastructure/ - VPN configs and SSH setup - tests/ - All test files consolidated - logs/ - Ready for future logs Benefits: - Cleaner root directory (18 vs 119 files) - Logical organization of documentation - 95.9% disk space reduction - Faster navigation and discovery - Better portability (build artifacts excluded) Build artifacts can be regenerated: - Rust: cargo build --release (5-15 min per project) - Python: pip install -r requirements.txt (2-3 min) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
151
docs/database/SQL_INJECTION_FIXES_VERIFICATION.txt
Normal file
151
docs/database/SQL_INJECTION_FIXES_VERIFICATION.txt
Normal file
@@ -0,0 +1,151 @@
|
||||
SQL INJECTION VULNERABILITY FIXES - VERIFICATION GUIDE
|
||||
=====================================================
|
||||
|
||||
FILES MODIFIED:
|
||||
--------------
|
||||
1. api/services/conversation_context_service.py
|
||||
2. api/routers/conversation_contexts.py
|
||||
|
||||
CHANGES SUMMARY:
|
||||
---------------
|
||||
|
||||
FILE 1: api/services/conversation_context_service.py
|
||||
----------------------------------------------------
|
||||
|
||||
Line 13: ADDED import
|
||||
OLD: from sqlalchemy import or_, text
|
||||
NEW: from sqlalchemy import or_, text, func
|
||||
|
||||
Lines 178-201: FIXED search_term SQL injection
|
||||
OLD:
|
||||
if search_term:
|
||||
fulltext_match = text(
|
||||
"MATCH(title, dense_summary) AGAINST(:search_term IN NATURAL LANGUAGE MODE)"
|
||||
).bindparams(search_term=search_term)
|
||||
|
||||
query = query.filter(
|
||||
or_(
|
||||
fulltext_match,
|
||||
ConversationContext.title.like(f"%{search_term}%"), # VULNERABLE
|
||||
ConversationContext.dense_summary.like(f"%{search_term}%") # VULNERABLE
|
||||
)
|
||||
)
|
||||
|
||||
NEW:
|
||||
if search_term:
|
||||
try:
|
||||
fulltext_condition = text(
|
||||
"MATCH(title, dense_summary) AGAINST(:search_term IN NATURAL LANGUAGE MODE)"
|
||||
).bindparams(search_term=search_term)
|
||||
|
||||
like_condition = or_(
|
||||
ConversationContext.title.like(func.concat('%', search_term, '%')), # SECURE
|
||||
ConversationContext.dense_summary.like(func.concat('%', search_term, '%')) # SECURE
|
||||
)
|
||||
|
||||
query = query.filter(or_(fulltext_condition, like_condition))
|
||||
except Exception:
|
||||
like_condition = or_(
|
||||
ConversationContext.title.like(func.concat('%', search_term, '%')),
|
||||
ConversationContext.dense_summary.like(func.concat('%', search_term, '%'))
|
||||
)
|
||||
query = query.filter(like_condition)
|
||||
|
||||
Lines 210-220: FIXED tags SQL injection
|
||||
OLD:
|
||||
if tags:
|
||||
tag_filters = []
|
||||
for tag in tags:
|
||||
tag_filters.append(ConversationContext.tags.like(f'%"{tag}"%')) # VULNERABLE
|
||||
if tag_filters:
|
||||
query = query.filter(or_(*tag_filters))
|
||||
|
||||
NEW:
|
||||
if tags:
|
||||
# Use secure func.concat to prevent SQL injection
|
||||
tag_filters = []
|
||||
for tag in tags:
|
||||
tag_filters.append(
|
||||
ConversationContext.tags.like(func.concat('%"', tag, '"%')) # SECURE
|
||||
)
|
||||
if tag_filters:
|
||||
query = query.filter(or_(*tag_filters))
|
||||
|
||||
|
||||
FILE 2: api/routers/conversation_contexts.py
|
||||
--------------------------------------------
|
||||
|
||||
Lines 79-90: ADDED input validation for search_term
|
||||
NEW:
|
||||
search_term: Optional[str] = Query(
|
||||
None,
|
||||
max_length=200,
|
||||
pattern=r'^[a-zA-Z0-9\s\-_.,!?()]+$', # Whitelist validation
|
||||
description="Full-text search term (alphanumeric, spaces, and basic punctuation only)"
|
||||
),
|
||||
|
||||
Lines 86-90: ADDED validation for tags
|
||||
NEW:
|
||||
tags: Optional[List[str]] = Query(
|
||||
None,
|
||||
description="Filter by tags (OR logic)",
|
||||
max_items=20 # Prevent DoS
|
||||
),
|
||||
|
||||
Lines 121-130: ADDED runtime tag validation
|
||||
NEW:
|
||||
# Validate tags to prevent SQL injection
|
||||
if tags:
|
||||
import re
|
||||
tag_pattern = re.compile(r'^[a-zA-Z0-9\-_]+$')
|
||||
for tag in tags:
|
||||
if not tag_pattern.match(tag):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"Invalid tag format: '{tag}'. Tags must be alphanumeric with hyphens or underscores only."
|
||||
)
|
||||
|
||||
|
||||
TESTING THE FIXES:
|
||||
-----------------
|
||||
|
||||
Test 1: Valid Input (should work - HTTP 200)
|
||||
curl "http://172.16.3.30:8001/api/conversation-contexts/recall?search_term=test" \
|
||||
-H "Authorization: Bearer $JWT_TOKEN"
|
||||
|
||||
Test 2: SQL Injection Attack (should be rejected - HTTP 422)
|
||||
curl "http://172.16.3.30:8001/api/conversation-contexts/recall?search_term=%27%20OR%20%271%27%3D%271" \
|
||||
-H "Authorization: Bearer $JWT_TOKEN"
|
||||
|
||||
Test 3: Tag Injection (should be rejected - HTTP 400)
|
||||
curl "http://172.16.3.30:8001/api/conversation-contexts/recall?tags[]=%27%20OR%20%271%27%3D%271" \
|
||||
-H "Authorization: Bearer $JWT_TOKEN"
|
||||
|
||||
|
||||
KEY SECURITY IMPROVEMENTS:
|
||||
-------------------------
|
||||
|
||||
1. NO F-STRING INTERPOLATION IN SQL
|
||||
- All LIKE patterns use func.concat()
|
||||
- All parameterized queries use .bindparams()
|
||||
|
||||
2. INPUT VALIDATION AT ROUTER LEVEL
|
||||
- Regex pattern enforcement
|
||||
- Length limits
|
||||
- Character whitelisting
|
||||
|
||||
3. RUNTIME TAG VALIDATION
|
||||
- Additional validation in endpoint
|
||||
- Prevents bypass of Query validation
|
||||
|
||||
4. DEFENSE IN DEPTH
|
||||
- Multiple layers of protection
|
||||
- Validation + Parameterization + Database escaping
|
||||
|
||||
|
||||
DEPLOYMENT NEEDED:
|
||||
-----------------
|
||||
These changes are in D:\ClaudeTools but need to be deployed to the running API server at 172.16.3.30:8001
|
||||
|
||||
After deployment, run: bash test_sql_injection_simple.sh
|
||||
|
||||
Reference in New Issue
Block a user