Detection Systems - Technical Documentation#

Overview#

The Detection Systems form a critical intelligence layer in Marcus that automatically analyzes kanban board state and user context to recommend the optimal operating mode. This system bridges the gap between raw board data and intelligent mode selection, enabling Marcus to adapt dynamically to different project scenarios without explicit configuration.

System Architecture#

The detection systems consist of two primary components working in tandem:

1. Board Analyzer (src/detection/board_analyzer.py)#

  • Primary Function: Analyzes kanban board characteristics to determine structure quality and recommend initial operating modes

  • Input: List of tasks from a kanban board

  • Output: BoardState object with comprehensive analysis metrics

2. Context Detector (src/detection/context_detector.py)#

  • Primary Function: Combines board analysis with user interaction patterns to make final mode recommendations

  • Input: Board state, user messages, interaction history

  • Output: ModeRecommendation with confidence scores and reasoning

Integration in Marcus Ecosystem#

Position in Architecture#

The Detection Systems sit at the intelligence layer of Marcus, specifically between:

  • Data Layer: Raw kanban board data (tasks, labels, descriptions)

  • Decision Layer: Mode selection and agent coordination

  • Execution Layer: Actual task assignment and work coordination

Dependencies#

  • Core Models: Uses Task, TaskStatus, Priority from src.core.models

  • Kanban Integration: Receives board data via kanban providers

  • MCP Server: Detection results influence MCP tool behavior

Integration Points#

  1. Project Creation: Analyzes empty boards to recommend Creator mode

  2. Agent Registration: Considers board state when agents request tasks

  3. Mode Switching: Continuously monitors board evolution for mode recommendations

  4. User Interaction: Parses user messages for explicit intent signals

Workflow Integration#

In the typical Marcus workflow scenario:

create_project → register_agent → request_next_task → report_progress → report_blocker → finish_task
                      ↑                    ↑                     ↑
                   Detection           Detection              Detection
                   (initial)         (continuous)          (evolution)

Specific Invocation Points#

  1. Project Creation:

    • Triggered when create_project receives a new board ID

    • Analyzes board to determine if Creator mode should generate initial structure

    • Confidence: 95% for empty boards

  2. Agent Registration:

    • Called during register_agent to understand board maturity

    • Influences which types of agents should be deployed

    • Helps determine agent skill requirements

  3. Task Request Cycles:

    • Invoked before each request_next_task to ensure optimal mode

    • Adapts to board evolution as tasks are completed

    • Monitors for chaos/organization threshold crossings

  4. Progress Monitoring:

    • Tracks how report_progress calls affect board structure

    • Detects when boards transition between organization levels

    • Adjusts recommendations based on completion patterns

What Makes This System Special#

1. Multi-Dimensional Analysis#

Unlike simple rule-based systems, Marcus detection performs comprehensive analysis across:

  • Structural Metrics: Task descriptions, labels, estimates, dependencies

  • Workflow Patterns: Sequential vs parallel vs phased execution

  • Component Detection: Frontend, backend, database, infrastructure components

  • Phase Recognition: Setup, design, development, testing, deployment phases

2. Confidence-Based Recommendations#

Every recommendation includes:

  • Confidence Score: 0.0-1.0 based on signal strength

  • Reasoning: Human-readable explanation of decision factors

  • Alternatives: Ranked alternative modes with reasoning

3. Context-Aware Intelligence#

The system maintains user context including:

  • Message History: Recent interaction patterns

  • Mode Preferences: Historical usage patterns by user

  • Intent Detection: Real-time parsing of user goals

4. Dynamic Adaptation#

Board recommendations evolve as projects mature:

  • Empty Board → Creator mode (95% confidence)

  • Chaotic Board → Enricher mode (85% confidence)

  • Well-Structured → Adaptive mode (90% confidence)

Technical Implementation Details#

Board Analysis Algorithm#

async def calculate_structure_score(self, tasks: List[Task]) -> float:
    """
    Weighted scoring across multiple dimensions:
    - Descriptions (25%): Quality and length of task descriptions
    - Labels (20%): Presence of categorization labels
    - Estimates (25%): Time/complexity estimates
    - Priority Distribution (15%): Balanced priority assignment
    - Dependencies (15%): Task relationship definition
    """

Scoring Thresholds:

  • 0.0-0.3: Chaotic (task titles only)

  • 0.3-0.6: Basic organization

  • 0.6-0.8: Good structure

  • 0.8-1.0: Excellent metadata

Workflow Pattern Detection#

The system recognizes four primary workflow patterns:

  1. Sequential: One task in progress, orderly completion

  2. Parallel: Multiple concurrent tasks (3+ in progress)

  3. Phased: Clear development phases detected in task names

  4. Ad Hoc: No discernible pattern

Detection Logic:

if in_progress == 1 and total > 5:
    return WorkflowPattern.SEQUENTIAL
elif in_progress > 3:
    return WorkflowPattern.PARALLEL
elif len(phases) >= 3:
    return WorkflowPattern.PHASED

Intent Recognition System#

Uses regex pattern matching across user messages:

INTENT_PATTERNS = {
    UserIntent.CREATE: [
        r"create.*project", r"new.*project", r"start.*from.*scratch",
        r"generate.*tasks", r"from.*prd", r"build.*mvp"
    ],
    UserIntent.ORGANIZE: [
        r"organize", r"structure", r"clean.*up", r"add.*metadata"
    ],
    UserIntent.COORDINATE: [
        r"assign", r"next.*task", r"who.*should", r"coordinate"
    ]
}

Component and Phase Detection#

Component Patterns:

  • Frontend: (frontend|ui|client|react|vue|angular)

  • Backend: (backend|api|server|endpoint|service)

  • Database: (database|db|sql|mongo|redis|cache)

  • Infrastructure: (infra|devops|ci|cd|docker|k8s)

Phase Patterns:

  • Setup: (setup|init|initialize|config|configure|scaffold)

  • Design: (design|architect|plan|model|schema)

  • Development: (implement|build|create|develop|code)

  • Testing: (test|qa|quality|verify|validate)

  • Deployment: (deploy|release|launch|ship|production)

Simple vs Complex Task Handling#

Simple Tasks (< 5 tasks, low structure)#

  • Recommendation: Creator mode

  • Confidence: High (0.90+)

  • Reasoning: Limited scope suggests need for structure generation

  • Fallback: Enricher mode if creation fails

Complex Tasks (10+ tasks, varied structure)#

  • Analysis: Multi-dimensional scoring across all metrics

  • Recommendation: Based on structure score and workflow patterns

  • Confidence: Moderate (0.70-0.85)

  • Adaptability: Continuous monitoring for threshold crossings

Chaos Detection#

Special handling for “chaotic” boards:

  • Threshold: Structure score < 0.3 with 10+ tasks

  • Mode: Enricher (immediate organization needed)

  • Priority: High confidence (0.85) due to clear need

Board-Specific Considerations#

Empty Boards#

  • Immediate Response: Creator mode recommendation

  • User Guidance: “Describe your project and I’ll help create tasks”

  • Alternative: Manual task creation with Enricher support

Legacy Boards#

  • Assessment: Structure score calculation across existing tasks

  • Migration Path: Enricher mode to add missing metadata

  • Preservation: Maintain existing organization patterns

Multi-Component Projects#

  • Detection: Component pattern recognition across tasks

  • Coordination: Adaptive mode for cross-component dependencies

  • Specialization: Component-specific agent deployment

Agile Boards#

  • Pattern Recognition: Sprint/iteration detection

  • Mode Selection: Adaptive mode for established processes

  • Enhancement: Enricher mode for metadata gaps

Cato Integration#

The Detection Systems provide critical input to Cato (Marcus AI Intelligence Engine):

Decision Context#

  • Board Analysis Results: Structure scores, workflow patterns, component detection

  • User Intent Signals: Parsed from interaction history

  • Confidence Metrics: For decision quality assessment

Learning Feedback Loop#

  • Mode Effectiveness: Track success rates of mode recommendations

  • Pattern Refinement: Improve detection algorithms based on outcomes

  • User Adaptation: Learn user-specific preferences and patterns

AI-Enhanced Detection#

Cato can override detection recommendations when:

  • Historical Data: Previous similar projects suggest different approach

  • User Patterns: Individual user shows strong mode preferences

  • Context Clues: Advanced NLP detects nuanced requirements

Pros and Cons#

Advantages#

  1. Automatic Adaptation

    • No manual configuration required

    • Responds to project evolution in real-time

    • Scales across different project types and sizes

  2. Intelligent Analysis

    • Multi-dimensional board assessment

    • Pattern recognition across workflow styles

    • Context-aware recommendations

  3. User-Centric Design

    • Parses user intent from natural language

    • Maintains interaction history and preferences

    • Provides transparent reasoning for decisions

  4. Confidence-Based Decisions

    • Quantified uncertainty in recommendations

    • Alternative modes for edge cases

    • Graceful degradation when signals are weak

Disadvantages#

  1. Complexity Overhead

    • Multiple analysis dimensions increase computational cost

    • Pattern matching requires regular expression maintenance

    • Context storage grows with user interaction history

  2. False Positive Risk

    • Regex patterns may misinterpret user messages

    • Board structure analysis might misclassify organization level

    • Intent detection can be fooled by similar language patterns

  3. Cold Start Problem

    • New users have no interaction history

    • Empty boards provide minimal signals for analysis

    • Initial recommendations rely heavily on heuristics

  4. Maintenance Burden

    • Pattern libraries require updates as language evolves

    • Threshold tuning needed as board complexity increases

    • Component/phase patterns need domain-specific customization

Why This Approach Was Chosen#

Design Philosophy#

The detection system embodies Marcus’s core principle of intelligent automation. Rather than requiring users to explicitly configure modes, the system observes and adapts.

Alternative Approaches Considered#

  1. Manual Mode Selection

    • Pros: Simple, explicit, no false positives

    • Cons: Requires user expertise, static, poor UX

    • Rejection Reason: Contradicts Marcus automation goals

  2. Pure Rule-Based System

    • Pros: Predictable, debuggable, fast

    • Cons: Brittle, requires manual tuning, limited adaptability

    • Rejection Reason: Insufficient intelligence for complex scenarios

  3. ML-Based Classification

    • Pros: Learning capability, pattern discovery, accuracy improvement

    • Cons: Training data requirements, black box decisions, deployment complexity

    • Rejection Reason: Overkill for current scale, transparency requirements

Chosen Hybrid Approach#

Combines rule-based reliability with pattern recognition flexibility:

  • Deterministic: Core logic is predictable and debuggable

  • Adaptive: Pattern recognition allows for nuanced decisions

  • Transparent: All recommendations include human-readable reasoning

  • Extensible: New patterns and components can be added easily

Future Evolution#

Short-Term Enhancements (Next 6 Months)#

  1. Pattern Learning

    • Machine learning layer to improve pattern recognition

    • User feedback integration for recommendation quality

    • A/B testing framework for algorithm improvements

  2. Advanced Context Detection

    • Sentiment analysis for user frustration/satisfaction

    • Temporal patterns in user interaction cycles

    • Cross-project learning for similar domain detection

  3. Integration Expansion

    • GitHub integration for code-based component detection

    • Time tracking integration for workflow pattern validation

    • Communication platform integration for team coordination signals

Medium-Term Evolution (6-18 Months)#

  1. Predictive Capabilities

    • Board evolution prediction based on current trajectory

    • Early warning system for potential project issues

    • Proactive mode recommendations before problems emerge

  2. Domain Specialization

    • Industry-specific pattern libraries (fintech, healthcare, e-commerce)

    • Project type classification (MVP, enterprise, migration)

    • Role-based analysis (technical, business, design teams)

  3. Advanced AI Integration

    • Large language model integration for sophisticated intent detection

    • Computer vision for board visualization analysis

    • Multi-modal learning combining text, structure, and interaction patterns

Long-Term Vision (18+ Months)#

  1. Autonomous Board Optimization

    • Self-improving board structures based on team productivity

    • Automatic task generation and refinement

    • Dynamic workflow optimization based on team performance

  2. Ecosystem Intelligence

    • Cross-organization learning and pattern sharing

    • Industry benchmarking and best practice recommendations

    • Automated compliance and governance checking

  3. Generative Capabilities

    • AI-generated project structures from minimal requirements

    • Automatic task decomposition and dependency inference

    • Intelligent agent skill matching and deployment

Performance Characteristics#

Computational Complexity#

  • Board Analysis: O(n) where n = number of tasks

  • Pattern Matching: O(m*p) where m = message length, p = pattern count

  • Context Retrieval: O(1) with user ID indexing

  • Total Analysis Time: < 100ms for typical boards (< 50 tasks)

Memory Requirements#

  • Board State Cache: ~1KB per analyzed board

  • User Context Storage: ~500 bytes per user

  • Pattern Libraries: ~10KB static data

  • Total System Footprint: < 1MB for 1000 users

Scalability Limits#

  • Concurrent Analysis: Limited by regex processing (CPU-bound)

  • User Context Storage: Linear growth with user base

  • Pattern Complexity: Exponential growth with regex complexity

  • Recommended Limits: 10,000 users, 1,000 concurrent analyses

Error Handling and Resilience#

Fallback Strategies#

  1. Pattern Match Failures: Default to board structure analysis

  2. Board Analysis Errors: Fall back to task count heuristics

  3. Context Retrieval Issues: Use anonymous user context

  4. Complete System Failure: Default to Adaptive mode

Graceful Degradation#

  • Partial Data: Reduced confidence scores with available signals

  • Timeout Protection: Hard limits on analysis time (5 seconds max)

  • Memory Constraints: LRU cache eviction for user contexts

  • Service Dependencies: Independent operation from external services

The Detection Systems represent Marcus’s commitment to intelligent automation, providing the foundation for adaptive project management that responds to real-world complexity while maintaining transparency and user control.