Django Middleware Integration

Django Middleware Integration

DBCrust provides powerful Django middleware for automatic ORM performance analysis. The middleware captures Django queries in real-time, detects N+1 query problems, and provides actionable optimization recommendations without requiring code changes.

🚀 Quick Start

Basic Middleware Setup

Add DBCrust middleware to your Django project in 3 simple steps:

settings.py

INSTALLED_APPS = [
    # ... your existing apps
    'dbcrust.django',
]

# Add middleware (for development only)
if DEBUG:
    MIDDLEWARE = [
        'dbcrust.django.PerformanceAnalysisMiddleware',
        # ... your existing middleware
    ]

That’s it! DBCrust now automatically analyzes all ORM queries and reports performance issues.

Environment-Specific Configuration

settings/base.py

INSTALLED_APPS = [
    # ... your apps
    'dbcrust.django',
]

# settings/development.py
from .base import *

MIDDLEWARE = [
    'dbcrust.django.PerformanceAnalysisMiddleware',
    # ... other middleware
]

DBCRUST_ANALYSIS = {
    'ENABLED': True,
    'AUTO_REPORT': True,
    'REPORT_THRESHOLD': 5,  # Report if more than 5 queries
}

# settings/production.py
from .base import *

# Don't include middleware in production
# But keep the app for management commands

🛠️ Middleware Configuration

Performance Analysis Settings

settings.py

DBCRUST_ANALYSIS = {
    # Enable/disable analysis
    'ENABLED': True,

    # Automatic reporting
    'AUTO_REPORT': True,           # Print reports automatically
    'REPORT_THRESHOLD': 3,         # Report if query count > threshold
    'SLOW_QUERY_THRESHOLD': 100,   # Report queries slower than 100ms

    # N+1 Detection
    'DETECT_N_PLUS_ONE': True,
    'N_PLUS_ONE_THRESHOLD': 3,     # Flag if >3 similar queries

    # Missing optimization detection
    'DETECT_MISSING_SELECT_RELATED': True,
    'DETECT_MISSING_PREFETCH_RELATED': True,
    'DETECT_LARGE_RESULT_SETS': True,
    'LARGE_RESULT_SET_THRESHOLD': 100,

    # Query pattern analysis
    'ANALYZE_QUERY_PATTERNS': True,
    'SUGGEST_INDEXES': True,
    'DETECT_INEFFICIENT_QUERIES': True,

    # Reporting options
    'REPORT_FORMAT': 'console',    # 'console', 'json', 'html'
    'INCLUDE_STACK_TRACE': True,   # Show where queries originated
    'INCLUDE_QUERY_DETAILS': True, # Show actual SQL
    'MAX_QUERIES_IN_REPORT': 10,  # Limit report size

    # Storage options
    'STORE_RESULTS': False,        # Store results in database
    'STORE_DURATION_DAYS': 7,     # How long to keep stored results
}

Advanced Middleware Options

settings.py

DBCRUST_MIDDLEWARE = {
    # Middleware behavior
    'ANALYZE_ONLY_VIEWS': True,    # Only analyze view requests (not API calls)
    'SKIP_ADMIN': True,            # Skip Django admin requests
    'SKIP_STATIC': True,           # Skip static file requests

    # Request filtering
    'ANALYZE_PATHS': [             # Only analyze these URL patterns
        r'^/api/',
        r'^/dashboard/',
    ],
    'SKIP_PATHS': [                # Skip these URL patterns
        r'^/health/',
        r'^/metrics/',
    ],

    # User filtering
    'ANALYZE_USERS': ['admin', 'developer'],  # Only analyze these users
    'SKIP_ANONYMOUS': False,       # Analyze anonymous user requests

    # Performance limits
    'MAX_ANALYSIS_TIME': 1000,     # Max 1 second for analysis
    'SKIP_LONG_REQUESTS': True,    # Skip requests longer than 5 seconds
    'LONG_REQUEST_THRESHOLD': 5000,
}

📊 Real-Time Analysis Output

Console Output Format

When AUTO_REPORT = True, you’ll see real-time analysis:

🚨 DBCrust Django ORM Analysis - /books/
============================================
Request: GET /books/ (user: admin)
Duration: 2.34 seconds | Queries: 26

🔴 CRITICAL ISSUES (1):
   N+1 Query Detected:
   - Query: SELECT * FROM books_book ORDER BY created_at DESC
   - Followed by: 25x SELECT * FROM authors_author WHERE id = ?

   💡 Fix: Use select_related()
   books = Book.objects.select_related('author').all()
   Estimated improvement: 2.1s → 0.12s (94% faster)

🟡 OPTIMIZATIONS (2):
   Missing prefetch_related:
   - Model: Book → reviews (accessed 25 times)
   💡 Fix: Book.objects.prefetch_related('reviews')

   Large result set without pagination:
   - Query returned 500 rows, consider pagination
   💡 Fix: Use Django's Paginator class

📈 PERFORMANCE SUMMARY:
   Total queries: 26 (24 duplicates)
   Total time: 2.34s (2.1s in duplicates)
   Potential improvement: 94% faster with optimizations

View file: books/views.py:42
Query origins:
  → books/views.py:45 (Book.objects.all())
  → books/templates/books/book_item.html:12 ({{ book.author.name }})

JSON Output Format

settings.py

DBCRUST_ANALYSIS = {
    'REPORT_FORMAT': 'json',
    'JSON_OUTPUT_FILE': '/tmp/dbcrust_analysis.json',
}

JSON output structure:

{
  "request": {
    "path": "/books/",
    "method": "GET",
    "user": "admin",
    "timestamp": "2024-01-15T14:30:00Z",
    "duration_ms": 2340
  },
  "query_analysis": {
    "total_queries": 26,
    "duplicate_queries": 24,
    "total_duration_ms": 2340,
    "potential_improvement_percent": 94
  },
  "issues": [
    {
      "severity": "critical",
      "type": "n_plus_one",
      "description": "N+1 query detected accessing author.name",
      "query": "SELECT * FROM authors_author WHERE id = ?",
      "occurrence_count": 25,
      "fix_suggestion": "Use select_related('author')",
      "estimated_improvement": {
        "current_ms": 2100,
        "optimized_ms": 120,
        "improvement_percent": 94
      },
      "location": {
        "file": "books/views.py",
        "line": 45,
        "function": "book_list"
      }
    }
  ],
  "optimizations": [
    {
      "type": "missing_prefetch",
      "model": "Book",
      "relation": "reviews",
      "access_count": 25,
      "suggestion": "Use prefetch_related('reviews')"
    }
  ]
}

🎯 Advanced Usage Patterns

Custom Analysis Rules

Create custom analysis rules for your specific needs:

myapp/dbcrust_rules.py

from dbcrust.django.analyzers import BaseAnalyzer

class CustomModelAnalyzer(BaseAnalyzer):
    """Custom analyzer for specific model patterns"""

    def analyze_queryset(self, queryset, context):
        """Analyze custom business logic patterns"""
        issues = []

        # Custom rule: Check for missing status filters
        if hasattr(queryset.model, 'status'):
            query_sql = str(queryset.query)
            if 'WHERE' not in query_sql or 'status' not in query_sql:
                issues.append({
                    'type': 'missing_status_filter',
                    'severity': 'warning',
                    'message': 'Query missing status filter - may return inactive records',
                    'suggestion': 'Add .filter(status="active") to queryset'
                })

        # Custom rule: Check for expensive aggregations
        if any(op in str(queryset.query) for op in ['COUNT(*)', 'SUM(', 'AVG(']):
            if 'GROUP BY' not in str(queryset.query):
                issues.append({
                    'type': 'expensive_aggregation',
                    'severity': 'warning',
                    'message': 'Aggregation without GROUP BY may be slow',
                    'suggestion': 'Consider adding appropriate grouping or using database views'
                })

        return issues

# Register custom analyzer
from dbcrust.django import register_analyzer
register_analyzer(CustomModelAnalyzer)

settings.py

DBCRUST_ANALYSIS = {
    'CUSTOM_ANALYZERS': [
        'myapp.dbcrust_rules.CustomModelAnalyzer',
    ]
}

View-Specific Analysis

Analyze specific views in detail:

views.py

from dbcrust.django.decorators import analyze_performance

@analyze_performance(
    max_queries=5,
    max_duration=1000,  # 1 second
    detect_n_plus_one=True
)
def book_list(request):
    """Book list view with performance monitoring"""
    # This view will be analyzed even if middleware is disabled
    books = Book.objects.all()
    return render(request, 'books/list.html', {'books': books})

@analyze_performance(
    custom_rules=['check_pagination', 'check_caching']
)
def expensive_report(request):
    """Complex report with custom analysis rules"""
    # Complex query logic here
    return render(request, 'reports/expensive.html', context)

Class-Based View Integration

views.py

from django.views.generic import ListView
from dbcrust.django.mixins import PerformanceAnalysisMixin

class BookListView(PerformanceAnalysisMixin, ListView):
    model = Book
    template_name = 'books/list.html'

    # Performance analysis settings
    performance_max_queries = 10
    performance_detect_n_plus_one = True
    performance_suggest_optimizations = True

    def get_queryset(self):
        # This queryset will be analyzed automatically
        return Book.objects.select_related('author').prefetch_related('reviews')

class AuthorDetailView(PerformanceAnalysisMixin, DetailView):
    model = Author

    # Custom performance rules for this view
    performance_custom_rules = [
        'check_related_objects',
        'check_expensive_annotations'
    ]

🔧 Integration with Development Workflow

Pre-Commit Hooks

Catch performance issues before they reach production:

.pre-commit-config.yaml

repos:
  - repo: local
    hooks:
      - id: django-orm-analysis
        name: Django ORM Performance Analysis
        entry: python manage.py dbcrust_analyze_code
        language: system
        files: \.py$
        pass_filenames: true

management/commands/dbcrust_analyze_code.py

from django.core.management.base import BaseCommand
from dbcrust.django.static_analysis import analyze_python_files

class Command(BaseCommand):
    help = 'Analyze Python files for potential ORM performance issues'

    def add_arguments(self, parser):
        parser.add_argument('files', nargs='*', help='Python files to analyze')

    def handle(self, *args, **options):
        issues_found = False

        for file_path in options['files']:
            issues = analyze_python_files([file_path])
            if issues:
                issues_found = True
                self.stdout.write(f"\n🚨 Issues found in {file_path}:")
                for issue in issues:
                    self.stdout.write(f"  - {issue['message']} (line {issue['line']})")
                    self.stdout.write(f"    Fix: {issue['suggestion']}")

        if issues_found:
            self.stdout.write("\n💡 Run 'python manage.py dbcrust_fix_auto' to auto-fix simple issues")
            exit(1)
        else:
            self.stdout.write("✅ No ORM performance issues detected")

IDE Integration

VS Code Extension Integration:

.vscode/settings.json

{
    "python.linting.enabled": true,
    "python.linting.pylintEnabled": false,
    "dbcrust.analysis.enabled": true,
    "dbcrust.analysis.realtime": true,
    "dbcrust.analysis.showInlineWarnings": true
}

PyCharm Plugin Integration:

# PyCharm external tool configuration
# Program: python
# Arguments: manage.py dbcrust_analyze_file $FilePath$
# Working directory: $ProjectFileDir$

Testing Integration

test_performance.py

from django.test import TestCase
from dbcrust.django.testing import PerformanceTestCase

class BookViewPerformanceTest(PerformanceTestCase):
    """Test view performance with DBCrust"""

    # Performance constraints
    max_queries = 3
    max_duration = 500  # 500ms
    detect_n_plus_one = True

    def setUp(self):
        # Create test data
        self.author = Author.objects.create(name="Test Author")
        self.books = [
            Book.objects.create(title=f"Book {i}", author=self.author)
            for i in range(10)
        ]

    def test_book_list_performance(self):
        """Test that book list view meets performance requirements"""
        with self.assert_performance():
            response = self.client.get('/books/')
            self.assertEqual(response.status_code, 200)

        # Performance analysis runs automatically
        # Test fails if constraints are violated

    def test_book_detail_performance(self):
        """Test book detail view performance"""
        book = self.books[0]

        with self.assert_performance(max_queries=2):
            response = self.client.get(f'/books/{book.id}/')
            self.assertEqual(response.status_code, 200)

# Run performance tests
# python manage.py test test_performance --keepdb

📈 Performance Monitoring

Continuous Performance Monitoring

settings/production.py

# Enable lightweight monitoring in production
DBCRUST_MONITORING = {
    'ENABLED': True,
    'SAMPLE_RATE': 0.1,  # Monitor 10% of requests
    'STORE_RESULTS': True,
    'ALERT_THRESHOLDS': {
        'query_count': 20,
        'duration_ms': 5000,
        'n_plus_one_count': 3,
    },
    'ALERTING': {
        'SLACK_WEBHOOK': os.getenv('SLACK_WEBHOOK_URL'),
        'EMAIL_RECIPIENTS': ['dev-team@company.com'],
    }
}

Performance Dashboard

urls.py

urlpatterns = [
    # ... your URLs
    path('dbcrust/', include('dbcrust.django.dashboard.urls')),
]

Access performance dashboard at /dbcrust/dashboard/:

• Real-time performance metrics
• N+1 query detection trends
• Slow query analysis
• Optimization recommendations
• Historical performance data

Metrics Integration

settings.py

DBCRUST_METRICS = {
    'PROMETHEUS_ENABLED': True,
    'PROMETHEUS_PREFIX': 'dbcrust_django',
    'STATSD_ENABLED': True,
    'STATSD_HOST': 'localhost',
    'STATSD_PORT': 8125,
    'CUSTOM_METRICS': {
        'query_count': 'counter',
        'query_duration': 'histogram',
        'n_plus_one_detected': 'counter',
    }
}

Prometheus metrics exposed:

# HELP dbcrust_django_queries_total Total number of queries
# TYPE dbcrust_django_queries_total counter
dbcrust_django_queries_total{view="book_list",method="GET"} 156

# HELP dbcrust_django_query_duration_seconds Query duration
# TYPE dbcrust_django_query_duration_seconds histogram
dbcrust_django_query_duration_seconds_bucket{view="book_list",le="0.1"} 45
dbcrust_django_query_duration_seconds_bucket{view="book_list",le="0.5"} 120

# HELP dbcrust_django_n_plus_one_total N+1 queries detected
# TYPE dbcrust_django_n_plus_one_total counter
dbcrust_django_n_plus_one_total{view="book_list"} 3

🚨 Troubleshooting

Common Issues

Middleware not running:

# Check middleware is properly installed
python manage.py shell
>>> from django.conf import settings
>>> 'dbcrust.django.PerformanceAnalysisMiddleware' in settings.MIDDLEWARE
True

# Check DEBUG mode is enabled
>>> settings.DEBUG
True

No analysis output:

# Check configuration
DBCRUST_ANALYSIS = {
    'ENABLED': True,
    'AUTO_REPORT': True,  # Must be True for console output
}

# Check logging configuration
LOGGING = {
    'loggers': {
        'dbcrust.django': {
            'level': 'DEBUG',
            'handlers': ['console'],
        }
    }
}

Performance impact:

# Reduce middleware overhead
DBCRUST_MIDDLEWARE = {
    'SKIP_ADMIN': True,          # Skip admin pages
    'SKIP_STATIC': True,         # Skip static files
    'ANALYZE_ONLY_VIEWS': True,  # Only analyze views
    'MAX_ANALYSIS_TIME': 100,    # Limit analysis time
}

Debug Mode

Enable detailed debugging:

settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'dbcrust.django': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': False,
        },
    },
}

# Enable debug mode
DBCRUST_DEBUG = True

📚 See Also

• Django Management Commands - CLI tools for Django
• CI/CD Integration - Automated performance testing
• Team Workflows - Collaborative performance optimization
• Django ORM Analyzer - Complete analyzer documentation