Skip to main content
Manual event tracking gives you complete control over what interactions are monitored and how they’re recorded. Use this approach when you need custom tracking logic, are integrating with non-supported LLM providers, or want granular control over the monitoring process.

When to Use Manual Tracking

Choose manual tracking when you need to:
  • Custom Integration: Work with LLM providers not yet supported by automatic wrappers
  • Granular Control: Track specific events or add custom metadata
  • Complex Workflows: Monitor multi-step processes with custom logic
  • Legacy Systems: Integrate with existing agent implementations
  • Custom Events: Track domain-specific events beyond standard LLM interactions

Core Tracking Methods

Conversation Lifecycle

Track the complete lifecycle of agent conversations:
import { AgentMonitor } from '@agent-governance/node';

const monitor = new AgentMonitor({
  apiKey: 'your-api-key',
  organizationId: 'your-org-id',
  enableComplianceChecks: true
});

const agentId = 'customer-service-agent';
const sessionId = 'session-' + Date.now();

// 1. Start conversation
monitor.trackConversationStart(agentId, sessionId, 'customer-123');

// 2. Track user input
monitor.trackUserMessage(agentId, sessionId, 'I need help with my account', 'customer-123');

// 3. Track agent response
monitor.trackAgentResponse(agentId, sessionId, 'I can help you with your account. What specific assistance do you need?');

// 4. End conversation
monitor.trackConversationEnd(agentId, sessionId, {
  duration: 45000, // 45 seconds
  messageCount: 4,
  userSatisfaction: 8, // 1-10 scale
  resolutionStatus: 'resolved'
});

Tool and Function Calls

Track when your agent uses tools or external functions:
// Track tool call and result together
monitor.trackToolCall(
  agentId,
  sessionId,
  'get_account_balance',
  { accountId: 'acc_123', accountType: 'checking' }, // Parameters
  { balance: 2547.83, currency: 'USD', lastUpdated: '2024-01-15' }, // Result
  320 // Execution time in milliseconds
);

// Or track them separately for more control
monitor.track(agentId, {
  sessionId: sessionId,
  interactionType: 'tool_call',
  toolName: 'transfer_funds',
  toolParameters: {
    fromAccount: 'acc_123',
    toAccount: 'acc_456',
    amount: 500.00
  }
});

// Track the result separately
monitor.track(agentId, {
  sessionId: sessionId,
  interactionType: 'tool_result',
  toolName: 'transfer_funds',
  toolResult: {
    transactionId: 'txn_789',
    status: 'completed',
    confirmationNumber: 'CF123456'
  },
  metadata: {
    toolExecutionTime: 1250
  }
});

Error Tracking

Comprehensive error tracking with context:
// Simple error tracking
monitor.trackError(agentId, sessionId, 'Database connection failed');

// Detailed error with metadata
monitor.trackError(agentId, sessionId, new Error('API timeout'), {
  errorType: 'APITimeoutError',
  errorCode: 'TIMEOUT_001',
  severity: 'high',
  recoverable: true,
  userImpact: 'moderate',
  systemImpact: 'minimal',
  errorSource: 'external'
});

// Using the generic track method for full control
monitor.track(agentId, {
  sessionId: sessionId,
  interactionType: 'error',
  content: 'Failed to process payment request',
  errorType: 'PaymentProcessingError',
  errorCode: 'PAY_ERR_001',
  severity: 'critical',
  metadata: {
    recoverable: false,
    userImpact: 'high',
    systemImpact: 'moderate',
    errorSource: 'server',
    originalError: {
      message: 'Payment gateway unreachable',
      code: 502,
      timestamp: Date.now()
    }
  }
});

Advanced Tracking Patterns

Banking Workflow Example

Here’s a comprehensive example tracking a complete banking interaction:
async function trackBankingWorkflow(monitor) {
  const agentId = 'personal-banking-agent';
  const sessionId = `banking-${Date.now()}`;
  const userId = 'customer-456';

  try {
    // Start the conversation
    monitor.trackConversationStart(agentId, sessionId, userId, {
      userAgent: 'Mozilla/5.0...',
      ipAddress: '192.168.1.100',
      referrer: 'https://mybank.com/dashboard'
    });

    // User asks about balance
    monitor.trackUserMessage(
      agentId,
      sessionId,
      'Can you show me my checking account balance?',
      userId
    );

    // Agent decides to check account
    monitor.trackToolCall(
      agentId,
      sessionId,
      'get_account_summary',
      { accountType: 'checking', includeRecentTransactions: true },
      {
        balance: 3247.89,
        currency: 'USD',
        recentTransactions: [
          { date: '2024-01-14', amount: -45.67, description: 'Grocery Store' },
          { date: '2024-01-13', amount: 2500.00, description: 'Payroll Deposit' }
        ]
      },
      450 // Execution time
    );

    // Agent responds with balance information
    monitor.trackAgentResponse(
      agentId,
      sessionId,
      'Your checking account balance is $3,247.89. I can see your most recent transactions include a payroll deposit and a grocery purchase.',
      {
        llmLatency: 850,
        tokensUsed: { input: 125, output: 67, total: 192 },
        responseQuality: 92,
        cost: 0.0034
      }
    );

    // User asks about suspicious transaction
    monitor.trackUserMessage(
      agentId,
      sessionId,
      'I see a charge I don\'t recognize from last week. Can you help me check if it\'s fraudulent?',
      userId
    );

    // Agent runs fraud check
    monitor.trackToolCall(
      agentId,
      sessionId,
      'check_transaction_fraud',
      {
        accountId: 'acc_123',
        lookbackDays: 7,
        flagSuspicious: true
      },
      {
        suspiciousTransactions: [
          {
            id: 'txn_suspicious_001',
            amount: 127.45,
            merchant: 'Unknown Merchant LLC',
            location: 'Los Angeles, CA',
            riskScore: 75
          }
        ],
        recommendations: ['contact_customer', 'freeze_card']
      },
      1200
    );

    // Agent provides fraud analysis
    monitor.trackAgentResponse(
      agentId,
      sessionId,
      'I found one potentially suspicious transaction for $127.45 from "Unknown Merchant LLC" in Los Angeles. This has a risk score of 75/100. I recommend we freeze your card immediately and investigate this charge.',
      {
        llmLatency: 920,
        tokensUsed: { input: 245, output: 89, total: 334 },
        responseQuality: 95,
        flagged: true, // High-risk response
        requiresReview: true
      }
    );

    // End conversation successfully
    monitor.trackConversationEnd(agentId, sessionId, {
      duration: 180000, // 3 minutes
      messageCount: 6,
      userSatisfaction: 9,
      resolutionStatus: 'escalated',
      endReason: 'user_initiated',
      followUpRequired: true,
      escalationReason: 'potential_fraud_detected'
    });

  } catch (error) {
    // Track any errors that occur
    monitor.trackError(agentId, sessionId, error, {
      errorType: 'WorkflowError',
      severity: 'high',
      recoverable: false,
      userImpact: 'high'
    });
  }
}

Custom Event Types

Track domain-specific events using the generic track method:
// Track compliance review trigger
monitor.track(agentId, {
  sessionId: sessionId,
  interactionType: 'compliance_violation',
  metadata: {
    violationType: 'fair_lending',
    severity: 'warning',
    description: 'Potential discriminatory language detected',
    ruleId: 'fair_lending_001',
    triggeredContent: 'people like you typically...',
    recommendedActions: ['human_review', 'response_revision'],
    requiresImmediateAction: false
  }
});

// Track risk threshold exceeded
monitor.track(agentId, {
  sessionId: sessionId,
  interactionType: 'risk_threshold_exceeded',
  metadata: {
    riskScore: 85,
    threshold: 75,
    riskFactors: [
      { factor: 'suspicious_transaction_inquiry', weight: 40 },
      { factor: 'high_value_account_access', weight: 30 },
      { factor: 'unusual_time_of_day', weight: 15 }
    ],
    recommendedActions: ['escalate_to_human', 'additional_verification']
  }
});

Metadata and Context

Enhance your tracking with rich metadata:

Performance Metrics

monitor.trackAgentResponse(agentId, sessionId, responseText, {
  // LLM Performance
  llmLatency: 1250,
  tokensUsed: {
    input: 156,
    output: 89,
    total: 245
  },
  cost: 0.0067,
  temperature: 0.7,
  maxTokens: 500,

  // Quality Metrics
  responseQuality: 88,
  relevanceScore: 92,
  coherenceScore: 85,
  helpfulnessScore: 90,

  // Compliance and Risk
  complianceFlags: [
    {
      rule: 'pii_detection',
      severity: 'info',
      description: 'Account number mentioned in context',
      context: { detected: 'account ending in 1234' }
    }
  ],
  riskScore: 25,
  requiresReview: false
});

Business Context

monitor.trackUserMessage(agentId, sessionId, userMessage, userId, {
  // Customer Context
  customerTier: 'premium',
  accountAge: 1825, // days
  totalAccountValue: 125000,

  // Interaction Context
  messageId: 'msg_unique_123',
  threadId: 'thread_abc',
  channel: 'web_chat',

  // Technical Context
  userAgent: 'Mozilla/5.0...',
  ipAddress: '192.168.1.100',
  sessionDuration: 45000,

  // Business Context
  previousIssues: 2,
  satisfactionHistory: [8, 9, 7, 9],
  preferredLanguage: 'en-US'
});

Batch Operations and Performance

Efficient Batch Tracking

For high-volume applications, use efficient batching:
// Configure for high-volume scenarios
const monitor = new AgentMonitor({
  apiKey: 'your-api-key',
  organizationId: 'your-org-id',
  batchSize: 500, // Larger batches for efficiency
  flushInterval: 10000, // 10 second intervals
  enableLogging: false // Disable verbose logging
});

// Track multiple events efficiently
const events = [
  {
    sessionId: 'session-1',
    interactionType: 'user_message',
    content: 'Hello'
  },
  {
    sessionId: 'session-1',
    interactionType: 'agent_response',
    content: 'Hi there!'
  },
  {
    sessionId: 'session-2',
    interactionType: 'user_message',
    content: 'Help me'
  }
];

// Track all events
events.forEach(event => monitor.track(agentId, event));

// Force flush when needed
await monitor.flush();

Conditional Tracking

Track selectively based on business logic:
function trackInteractionConditionally(monitor, agentId, interaction) {
  const { sessionId, userMessage, agentResponse, riskScore } = interaction;

  // Always track high-risk interactions
  if (riskScore > 70) {
    monitor.trackUserMessage(agentId, sessionId, userMessage);
    monitor.trackAgentResponse(agentId, sessionId, agentResponse, {
      riskScore,
      flagged: true,
      requiresReview: true
    });
    return;
  }

  // Sample normal interactions (10% sampling)
  if (Math.random() < 0.1) {
    monitor.trackUserMessage(agentId, sessionId, userMessage);
    monitor.trackAgentResponse(agentId, sessionId, agentResponse, { sampled: true });
  }

  // Always track errors and tool calls
  if (interaction.hasError) {
    monitor.trackError(agentId, sessionId, interaction.error);
  }

  if (interaction.toolCalls?.length > 0) {
    interaction.toolCalls.forEach(tool => {
      monitor.trackToolCall(agentId, sessionId, tool.name, tool.params, tool.result);
    });
  }
}

Integration Patterns

Express.js Middleware

Create middleware for automatic tracking in web applications:
function createAgentTrackingMiddleware(monitor) {
  return (req, res, next) => {
    const sessionId = req.sessionID || `session-${Date.now()}`;
    const userId = req.user?.id;

    // Add tracking utilities to request
    req.agentTracking = {
      trackUserMessage: (agentId, message) => {
        monitor.trackUserMessage(agentId, sessionId, message, userId, {
          ipAddress: req.ip,
          userAgent: req.get('User-Agent'),
          referer: req.get('Referer')
        });
      },

      trackAgentResponse: (agentId, response, metadata = {}) => {
        monitor.trackAgentResponse(agentId, sessionId, response, {
          ...metadata,
          endpoint: req.path,
          method: req.method
        });
      },

      trackError: (agentId, error) => {
        monitor.trackError(agentId, sessionId, error, {
          endpoint: req.path,
          method: req.method,
          statusCode: res.statusCode
        });
      }
    };

    next();
  };
}

// Use in Express app
app.use(createAgentTrackingMiddleware(monitor));

app.post('/api/chat', (req, res) => {
  const { message } = req.body;

  // Track user message
  req.agentTracking.trackUserMessage('web-agent', message);

  // Process message and generate response
  const response = processMessage(message);

  // Track agent response
  req.agentTracking.trackAgentResponse('web-agent', response);

  res.json({ response });
});

Custom Agent Wrapper

Create a custom wrapper for your agent class:
class MonitoredAgent {
  constructor(agentImplementation, monitor, agentId) {
    this.agent = agentImplementation;
    this.monitor = monitor;
    this.agentId = agentId;
  }

  async handleMessage(sessionId, userMessage, userId = null) {
    const startTime = Date.now();

    try {
      // Track conversation start if needed
      if (!this.activeSessions?.has(sessionId)) {
        this.monitor.trackConversationStart(this.agentId, sessionId, userId);
        this.activeSessions = this.activeSessions || new Set();
        this.activeSessions.add(sessionId);
      }

      // Track user message
      this.monitor.trackUserMessage(this.agentId, sessionId, userMessage, userId);

      // Process with underlying agent
      const result = await this.agent.processMessage(userMessage, { sessionId, userId });

      // Track any tool calls
      if (result.toolCalls) {
        for (const tool of result.toolCalls) {
          this.monitor.trackToolCall(
            this.agentId,
            sessionId,
            tool.name,
            tool.parameters,
            tool.result,
            tool.executionTime
          );
        }
      }

      // Track agent response
      this.monitor.trackAgentResponse(this.agentId, sessionId, result.response, {
        llmLatency: Date.now() - startTime,
        tokensUsed: result.tokenUsage,
        responseQuality: result.qualityScore,
        cost: result.cost
      });

      return result;

    } catch (error) {
      // Track error
      this.monitor.trackError(this.agentId, sessionId, error, {
        errorType: error.constructor.name,
        severity: 'high',
        recoverable: error.recoverable || false
      });
      throw error;
    }
  }

  async endSession(sessionId, metadata = {}) {
    this.monitor.trackConversationEnd(this.agentId, sessionId, metadata);
    this.activeSessions?.delete(sessionId);
  }
}

// Usage
const monitoredAgent = new MonitoredAgent(myAgentImplementation, monitor, 'my-agent');
const response = await monitoredAgent.handleMessage('session-123', 'Hello!', 'user-456');

Compliance Integration

Manual tracking allows for sophisticated compliance monitoring:

Real-time Compliance Checks

async function trackWithCompliance(monitor, agentId, sessionId, userMessage, agentResponse) {
  // Track user message first
  monitor.trackUserMessage(agentId, sessionId, userMessage);

  // Pre-response compliance check
  const complianceResult = await monitor.complianceEngine?.evaluateInteraction({
    agentId,
    agentCategory: 'tool_calling',
    agentSpecialty: 'personal_banking',
    userMessage,
    agentResponse,
    sessionId,
    timestamp: Date.now()
  });

  // Track response with compliance data
  monitor.trackAgentResponse(agentId, sessionId, agentResponse, {
    complianceFlags: complianceResult?.violations || [],
    riskScore: complianceResult?.riskScore || 0,
    requiresReview: complianceResult?.requiresReview || false
  });

  // If compliance violations detected, track separately
  if (complianceResult?.violations?.length > 0) {
    for (const violation of complianceResult.violations) {
      monitor.track(agentId, {
        sessionId,
        interactionType: 'compliance_violation',
        metadata: {
          violationType: violation.rule,
          severity: violation.severity,
          description: violation.description,
          context: violation.context,
          recommendation: violation.recommendation
        }
      });
    }
  }
}

Custom Compliance Rules

// Add custom compliance rule
if (monitor.complianceEngine) {
  monitor.complianceEngine.addRule({
    id: 'banking-hours-check',
    name: 'Banking Hours Compliance',
    description: 'Ensures agents mention banking hours appropriately',
    category: 'consumer_protection',
    severity: 'info',
    isActive: true,
    ruleFunction: (context) => {
      const violations = [];
      const response = context.agentResponse || '';

      // Check if banking hours are mentioned without proper context
      if (response.includes('banking hours') && !response.includes('Monday through Friday')) {
        violations.push({
          rule: 'banking-hours-check',
          severity: 'info',
          description: 'Banking hours mentioned without full schedule',
          recommendation: 'Include complete banking hours schedule'
        });
      }

      return {
        isCompliant: violations.length === 0,
        violations,
        riskScore: violations.length * 5,
        requiresReview: false
      };
    }
  });
}

Analytics and Reporting

Track custom metrics for business intelligence:

Custom Analytics Events

// Track business-specific metrics
function trackBusinessMetrics(monitor, agentId, sessionId, interaction) {
  const { customerSegment, productInterest, conversionEvent } = interaction;

  // Track customer segment interaction
  monitor.track(agentId, {
    sessionId,
    interactionType: 'custom',
    metadata: {
      eventType: 'customer_segment_interaction',
      segment: customerSegment,
      products: productInterest,
      timestamp: Date.now()
    }
  });

  // Track conversion events
  if (conversionEvent) {
    monitor.track(agentId, {
      sessionId,
      interactionType: 'custom',
      metadata: {
        eventType: 'conversion',
        conversionType: conversionEvent.type,
        value: conversionEvent.value,
        currency: conversionEvent.currency
      }
    });
  }
}

// Usage in banking context
trackBusinessMetrics(monitor, 'loan-agent', 'session-123', {
  customerSegment: 'high_value',
  productInterest: ['mortgage', 'investment'],
  conversionEvent: {
    type: 'mortgage_application_started',
    value: 450000,
    currency: 'USD'
  }
});

Session Analytics

class SessionAnalytics {
  constructor(monitor, agentId) {
    this.monitor = monitor;
    this.agentId = agentId;
    this.sessions = new Map();
  }

  startSession(sessionId, userId = null) {
    this.sessions.set(sessionId, {
      startTime: Date.now(),
      messageCount: 0,
      toolCallCount: 0,
      errorCount: 0,
      userId,
      events: []
    });

    this.monitor.trackConversationStart(this.agentId, sessionId, userId);
  }

  trackMessage(sessionId, type, content, metadata = {}) {
    const session = this.sessions.get(sessionId);
    if (!session) return;

    session.messageCount++;
    session.events.push({
      type,
      timestamp: Date.now(),
      content: content.substring(0, 100) // Store preview
    });

    if (type === 'user_message') {
      this.monitor.trackUserMessage(this.agentId, sessionId, content, session.userId, metadata);
    } else if (type === 'agent_response') {
      this.monitor.trackAgentResponse(this.agentId, sessionId, content, metadata);
    }
  }

  trackToolCall(sessionId, toolName, params, result, executionTime) {
    const session = this.sessions.get(sessionId);
    if (session) session.toolCallCount++;

    this.monitor.trackToolCall(this.agentId, sessionId, toolName, params, result, executionTime);
  }

  trackError(sessionId, error) {
    const session = this.sessions.get(sessionId);
    if (session) session.errorCount++;

    this.monitor.trackError(this.agentId, sessionId, error);
  }

  endSession(sessionId, userSatisfaction = null) {
    const session = this.sessions.get(sessionId);
    if (!session) return;

    const duration = Date.now() - session.startTime;
    const resolutionStatus = session.errorCount > 0 ? 'unresolved' : 'resolved';

    this.monitor.trackConversationEnd(this.agentId, sessionId, {
      duration,
      messageCount: session.messageCount,
      toolCallCount: session.toolCallCount,
      errorCount: session.errorCount,
      userSatisfaction,
      resolutionStatus,
      avgResponseTime: duration / Math.max(session.messageCount, 1)
    });

    this.sessions.delete(sessionId);
  }
}

// Usage
const analytics = new SessionAnalytics(monitor, 'customer-service-agent');
analytics.startSession('session-456', 'customer-123');
analytics.trackMessage('session-456', 'user_message', 'I need help');
analytics.trackMessage('session-456', 'agent_response', 'How can I assist you?');
analytics.endSession('session-456', 8);

Error Handling and Resilience

Implement robust error handling for tracking operations:

Resilient Tracking Wrapper

class ResilientTracker {
  constructor(monitor, options = {}) {
    this.monitor = monitor;
    this.options = {
      maxRetries: 3,
      retryDelay: 1000,
      fallbackToLocalStorage: true,
      ...options
    };
    this.failedEvents = [];
  }

  async safeTrack(agentId, event, retries = 0) {
    try {
      this.monitor.track(agentId, event);

      // Process any previously failed events
      if (this.failedEvents.length > 0) {
        await this.retryFailedEvents();
      }

    } catch (error) {
      console.warn('Tracking failed:', error.message);

      if (retries < this.options.maxRetries) {
        setTimeout(() => {
          this.safeTrack(agentId, event, retries + 1);
        }, this.options.retryDelay * Math.pow(2, retries));
      } else {
        // Store for later retry
        this.failedEvents.push({ agentId, event, timestamp: Date.now() });

        if (this.options.fallbackToLocalStorage && typeof localStorage !== 'undefined') {
          this.saveToLocalStorage();
        }
      }
    }
  }

  async retryFailedEvents() {
    const eventsToRetry = [...this.failedEvents];
    this.failedEvents = [];

    for (const { agentId, event } of eventsToRetry) {
      try {
        this.monitor.track(agentId, event);
      } catch (error) {
        // Put back in failed events
        this.failedEvents.push({ agentId, event, timestamp: Date.now() });
      }
    }
  }

  saveToLocalStorage() {
    try {
      localStorage.setItem('agent-tracking-failed-events', JSON.stringify(this.failedEvents));
    } catch (error) {
      console.warn('Failed to save to localStorage:', error.message);
    }
  }

  loadFromLocalStorage() {
    try {
      const stored = localStorage.getItem('agent-tracking-failed-events');
      if (stored) {
        this.failedEvents = JSON.parse(stored);
        localStorage.removeItem('agent-tracking-failed-events');
      }
    } catch (error) {
      console.warn('Failed to load from localStorage:', error.message);
    }
  }
}

// Usage
const resilientTracker = new ResilientTracker(monitor);
resilientTracker.safeTrack('agent-id', {
  sessionId: 'session-123',
  interactionType: 'user_message',
  content: 'Hello'
});

Performance Optimization

Optimize tracking for high-throughput scenarios:

Debounced Tracking

class DebouncedTracker {
  constructor(monitor, debounceMs = 100) {
    this.monitor = monitor;
    this.debounceMs = debounceMs;
    this.pendingEvents = new Map();
    this.timers = new Map();
  }

  track(agentId, event) {
    const key = `${agentId}-${event.sessionId}`;

    // Clear existing timer
    if (this.timers.has(key)) {
      clearTimeout(this.timers.get(key));
    }

    // Add to pending events
    if (!this.pendingEvents.has(key)) {
      this.pendingEvents.set(key, []);
    }
    this.pendingEvents.get(key).push(event);

    // Set new timer
    const timer = setTimeout(() => {
      this.flushEvents(agentId, key);
    }, this.debounceMs);

    this.timers.set(key, timer);
  }

  flushEvents(agentId, key) {
    const events = this.pendingEvents.get(key) || [];

    // Process all pending events
    events.forEach(event => {
      this.monitor.track(agentId, event);
    });

    // Cleanup
    this.pendingEvents.delete(key);
    this.timers.delete(key);
  }

  async flush() {
    // Flush all pending events immediately
    for (const [key, timer] of this.timers) {
      clearTimeout(timer);
      const agentId = key.split('-')[0];
      this.flushEvents(agentId, key);
    }

    await this.monitor.flush();
  }
}

Best Practices

Troubleshooting

Check that you’re calling monitor.flush() or waiting for the automatic flush interval. Verify your API key and organization ID are correct.
Reduce batch size and flush interval. Implement event sampling for high-volume scenarios. Check for event accumulation without proper flushing.
Ensure you’re passing metadata objects correctly. Check that metadata keys match expected field names. Verify data types are serializable.
Confirm that enableComplianceChecks: true is set. Verify that compliance checks are running on the correct event types (typically agent responses).
Use consistent session IDs across related events. Ensure session IDs are unique per conversation. Check that conversation start events are tracked.

Next Steps

I