API Reference
Complete API documentation for @orka-js/collector
collector()
Start the collector dashboard server. Returns a promise that resolves when the server is ready.
import { collector } from '@orka-js/collector';
const { tracer, server, stop } = await collector({
port: 3001,
host: 'localhost',
open: true,
cors: true,
maxTraces: 1000,
retentionMs: 24 * 60 * 60 * 1000, // 24 hours
});
// Later, stop the server
await stop();
Returns
| Property | Type | Description |
|---|---|---|
| tracer | TraceCollector | The tracer instance (TraceCollector) |
| server | DevToolsServer | The Express server instance |
| stop | () => Promise<void> | Function to stop the server |
Configuration
Configure the collector server with custom port, host, and other options when calling collector().
import { collector } from '@orka-js/collector';
// Start collector with custom configuration
const { tracer, server, stop } = await collector({
port: 3001, // Custom port (default: 3001)
host: 'localhost', // Custom host (default: 'localhost')
open: true, // Auto-open browser (default: false)
cors: true, // Enable CORS (default: true)
maxTraces: 1000, // Max traces to keep in memory (default: 1000)
retentionMs: 86400000, // Trace retention time in ms (default: 24h)
});
console.log(`DevTools running at http://${host}:${port}`);
// Stop the server when done
await stop();
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
| port | number | 3001 | Port number for the DevTools server |
| host | string | 'localhost' | Host address to bind the server (use "0.0.0.0" for external access) |
| open | boolean | false | Automatically open the dashboard in your default browser |
| cors | boolean | true | Enable Cross-Origin Resource Sharing for API access |
| maxTraces | number | 1000 | Maximum number of traces to keep in memory before eviction |
| retentionMs | number | 86400000 | Trace retention time in milliseconds (24 hours by default) |
Common Use Cases
port: 3001 and {host: 'localhost'}{host: '0.0.0.0'} to allow external connectionsopen: false to prevent browser auto-launch in headless environmentsmaxTraces and adjust retentionMs based on your needsRemote TracingNEW
Debug production AI apps by sending traces to a central collector, then view them in real-time from anywhere.
Agent Mode (Production)
Send traces from your production app to a remote collector service.
import { collector } from '@orka-js/collector';
// Production: Send traces to remote collector
const { tracer, stop } = await collector({
source: 'remote',
mode: 'agent',
remote: {
endpoint: 'https://traces.mycompany.com',
apiKey: process.env.COLLECTOR_API_KEY,
projectId: 'my-ai-app',
environment: 'production',
sampling: 0.1 // Sample 10% of traces
}
});
// Your app runs normally - traces are sent automatically
Viewer Mode (Monitoring)
View production traces in real-time from your local machine.
import { collector } from '@orka-js/collector';
// Viewer: Watch production traces locally
const { tracer, server, stop } = await collector({
source: 'remote',
mode: 'viewer',
remote: {
endpoint: 'https://traces.mycompany.com',
apiKey: process.env.COLLECTOR_API_KEY,
projectId: 'my-ai-app',
filters: {
environment: 'production',
timeRange: 'last-1h'
}
},
port: 3001,
open: true
});
// Dashboard at localhost:3001 shows live production traces!
Remote Configuration Options
| Option | Type | Description |
|---|---|---|
| source | 'local' | 'remote' | Mode: local dashboard or remote collector |
| mode | 'agent' | 'viewer' | Agent sends traces, viewer receives them |
| remote.endpoint | string | Remote collector URL |
| remote.apiKey | string | Authentication token |
| remote.projectId | string | Project identifier |
| remote.environment | string | Environment tag (production, staging, etc.) |
| remote.sampling | number | Sampling rate 0-1 (agent mode only) |
| remote.filters | object | Filter traces (viewer mode only) |
trace.start()
Start a new trace run. Returns a run ID that you can use with trace.end() or trace.error().
import { trace } from '@orka-js/collector';
const runId = trace.start('llm', 'generate-response', {
prompt: 'Hello, world!'
}, {
model: 'gpt-4',
temperature: 0.7
});
// ... do work ...
trace.end(runId, { content: 'Response text' });
Parameters
| type | TraceRunType | Type of run (agent, llm, tool, retrieval, etc.) |
| name | string | Name of the run |
| input? | unknown | Input data (optional) |
| metadata? | TraceMetadata | Additional metadata (optional) |
trace.end()
End a trace run successfully with output data.
import { trace } from '@orka-js/collector';
const runId = trace.start('llm', 'generate', { prompt: 'Hello' });
// ... perform operation ...
const result = await llm.generate('Hello');
// End the trace with output
trace.end(runId, { content: result.content }, {
promptTokens: result.usage.promptTokens,
completionTokens: result.usage.completionTokens,
model: 'gpt-4'
});
Parameters
| runId | string | The run ID returned by trace.start() |
| output? | unknown | Output data from the operation |
| metadata? | TraceMetadata | Additional metadata (tokens, model, etc.) |
trace.error()
End a trace run with an error. Records the error message and stack trace.
import { trace } from '@orka-js/collector';
const runId = trace.start('tool', 'web-search', { query: 'AI news' });
try {
const results = await searchTool.execute({ query: 'AI news' });
trace.end(runId, results);
} catch (error) {
// Record the error
trace.error(runId, error, {
toolName: 'web-search',
retryCount: 3
});
}
Parameters
| runId | string | The run ID returned by trace.start() |
| error | Error | string | The error object or message |
| metadata? | TraceMetadata | Additional error context |
trace.wrap()
Wrap an async function to automatically trace its execution. Handles start, end, and error automatically.
import { trace } from '@orka-js/collector';
const result = await trace.wrap('agent', 'research', async () => {
const agent = new ReActAgent({ llm, tools });
return agent.run("Analyze market trends");
}, {
model: 'gpt-4',
userId: 'user-123'
});
trace.session()
Create a session to group related traces together. Useful for tracking user requests or workflows.
import { trace } from '@orka-js/collector';
// Start a session for a user request
const sessionId = trace.session('User Request #123', {
userId: 'user-456',
requestType: 'research'
});
// All traces within this session are grouped
await trace.wrap('agent', 'step1', async () => {
return agent.analyze(input);
});
await trace.wrap('agent', 'step2', async () => {
return agent.synthesize(results);
});
// End the session
trace.endSession(sessionId);
Parameters
| name? | string | Optional session name for identification |
| metadata? | Record<string, unknown> | Additional session metadata |
@Trace
TypeScript decorator for automatically tracing class methods.
import { Trace } from '@orka-js/collector';
class MyAgent {
@Trace({ type: 'agent', name: 'process' })
async process(input: string) {
// Automatically traced
return this.llm.generate(input);
}
@Trace({ type: 'llm' }) // name defaults to method name
async generateResponse(prompt: string) {
return this.llm.generate(prompt);
}
}
withTrace()
Higher-order function to wrap any function with tracing. Alternative to the @Trace decorator for non-class functions.
import { withTrace } from '@orka-js/collector';
// Wrap a function with tracing
const tracedGenerate = withTrace(
async (prompt: string) => {
return llm.generate(prompt);
},
{ type: 'llm', name: 'generate' }
);
// Use the traced function
const result = await tracedGenerate('Hello, world!');
// Works with any async function
const tracedSearch = withTrace(searchTool.execute, {
type: 'tool',
name: 'web-search'
});
createDevToolsHook()
Create a hook to integrate DevTools with the @orka-js/observability Tracer.
import { Tracer } from '@orka-js/observability';
import { createDevToolsHook, collector } from '@orka-js/collector';
// Start collector first
await collector();
// Create tracer with DevTools hook
const tracer = new Tracer({
hooks: [createDevToolsHook()]
});
// All tracer events now appear in DevTools
const trace = tracer.startTrace('my-operation');
tracer.addEvent(trace.id, { type: 'llm', name: 'generate' });
tracer.endTrace(trace.id);
TraceCollector
The central collector that manages all trace sessions, runs, and metrics. Access it via getCollector().
import { getCollector } from '@orka-js/collector';
const tracer = getCollector();
// Get all sessions
const sessions = tracer.getSessions();
console.log(`Total sessions: ${sessions.length}`);
// Get metrics summary
const metrics = tracer.getMetrics();
console.log(`Total runs: ${metrics.totalRuns}`);
console.log(`Avg latency: ${metrics.avgLatencyMs}ms`);
console.log(`Error rate: ${metrics.errorRate * 100}%`);
// Find a specific session
const session = sessions.find(s => s.name === 'User Request #123');
// Export traces as JSON
const exportData = tracer.export();
fs.writeFileSync('traces.json', JSON.stringify(exportData, null, 2));
// Import traces
tracer.import(JSON.parse(fs.readFileSync('traces.json', 'utf-8')));
// Clear all traces
tracer.clear();
Methods
| getSessions() | TraceSession[] | Get all trace sessions |
| getMetrics() | TraceMetrics | Get aggregated metrics |
| getTimeSeriesData(options?) | TimeSeriesData | Get time series data for charts (latency, tokens, cost) |
| export() | ExportData | Export all traces as JSON |
| import(data) | void | Import traces from JSON |
| clear() | void | Clear all traces and sessions |
| on(event, handler) | void | Subscribe to trace events |
getTimeSeriesData()NEW
Extract time series data for latency, tokens, and cost metrics. Perfect for building custom dashboards and visualizations.
import { getCollector } from '@orka-js/collector';
const tracer = getCollector();
// Get all time series data
const allData = tracer.getTimeSeriesData();
console.log(allData.latency); // [{ timestamp: 1234567890, value: 342 }, ...]
console.log(allData.tokens); // [{ timestamp: 1234567890, value: 1500 }, ...]
console.log(allData.cost); // [{ timestamp: 1234567890, value: 0.03 }, ...]
// Filter by session
const sessionData = tracer.getTimeSeriesData({
sessionId: 'chat-session-123'
});
// Get only latency data
const latencyOnly = tracer.getTimeSeriesData({
metric: 'latency'
});
// Get only tokens data
const tokensOnly = tracer.getTimeSeriesData({
metric: 'tokens'
});
// Get only cost data
const costOnly = tracer.getTimeSeriesData({
metric: 'cost'
});
Parameters
| options? | object | Optional filter options |
| sessionId? | string | Filter by specific session ID |
| metric? | 'latency' | 'tokens' | 'cost' | 'all' | Which metric(s) to extract (default: 'all') |
Return Type
interface TimeSeriesData {
latency: Array<{ timestamp: number; value: number }>;
tokens: Array<{ timestamp: number; value: number }>;
cost: Array<{ timestamp: number; value: number }>;
}
// All arrays are sorted by timestamp (ascending)
Use Case: Building Charts
This method is designed to work seamlessly with charting libraries like Chart.js, Recharts, or D3.js:
import { Line } from 'react-chartjs-2';
const data = tracer.getTimeSeriesData({ metric: 'latency' });
const chartData = {
labels: data.latency.map(d => new Date(d.timestamp).toLocaleTimeString()),
datasets: [{
label: 'Latency (ms)',
data: data.latency.map(d => d.value),
borderColor: 'rgb(139, 92, 246)',
backgroundColor: 'rgba(139, 92, 246, 0.1)',
}]
};
return <Line data={chartData} />;
TypeScript Types
type TraceRunType =
| 'agent'
| 'llm'
| 'tool'
| 'retrieval'
| 'chain'
| 'workflow'
| 'graph'
| 'node'
| 'embedding'
| 'custom';
interface TraceMetadata {
model?: string;
provider?: string;
promptTokens?: number;
completionTokens?: number;
totalTokens?: number;
cost?: number;
temperature?: number;
maxTokens?: number;
toolName?: string;
toolArgs?: Record<string, unknown>;
nodeId?: string;
threadId?: string;
[key: string]: unknown;
}
interface TraceRun {
id: string;
parentId?: string;
type: TraceRunType;
name: string;
startTime: number;
endTime?: number;
latencyMs?: number;
status: 'running' | 'success' | 'error';
input?: unknown;
output?: unknown;
error?: string;
metadata?: TraceMetadata;
children: TraceRun[];
}
interface TraceSession {
id: string;
name?: string;
startTime: number;
endTime?: number;
runs: TraceRun[];
metadata?: Record<string, unknown>;
}
Examples of Use
Discover how to integrate the OrkaJS collector into your applications with practical and detailed examples.
Basic Integration
Objective: Configure OrkaJS with DevTools local to automatically trace all operations.
Ideal for local development where you want to visualize traces in real-time in your browser.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { collector } from '@orka-js/collector';
// 1 - Initialize DevTools in local mode
const { tracer, stop } = await collector({
source: 'local',
port: 3001,
open: true // Automatically open the dashboard
});
// 2 - Configure Orka with the integrated tracer
const orka = createOrka({
llm: new OpenAIAdapter({
apiKey: process.env.OPENAI_API_KEY!,
timeoutMs: 30000
}),
vectorDB: new MemoryVectorAdapter(),
// Tracer injection for automatic tracing
tracer,
defaults: {
chunkSize: 500,
topK: 3,
},
});
// 3 - Usage - automatic and transparent tracing
console.log('🚀 Starting the demo...');
// Create a knowledge base
await orka.knowledge.create({
name: 'documentation',
source: [
'OrkaJS is a TypeScript framework for LLM systems.',
'It supports OpenAI, Anthropic, Mistral and Ollama.',
'Vector databases: Pinecone, Qdrant, Chroma, in-memory.',
],
});
// Question with automatic RAG tracing
const result = await orka.ask({
knowledge: 'documentation',
question: 'What LLM providers are supported?',
});
console.log('Result:', result);
// 4 - Cleanup
await orka.knowledge.delete('documentation');
await stop(); // Stop the DevTools server
//🎯 RESULT IN THE DASHBOARD:
//• Main trace: "ask-operation"
//• Sub-traces: knowledge retrieval, LLM call, response generation
//• Metrics: latency, tokens, cost
//• Hierarchical tree visualization
Avantages
- • Automatic tracing without additional code
- • Real-time web dashboard
- • Minimal configuration
- • Ideal for development
Key points
- • The tracer is injected directly into Orka
- • All operations are traced automatically
- • Dashboard accessible on http://localhost:3001
- • Don't forget to call stop() to clean up
Manual tracing with sessions
Objective: Fine-grained tracing with manual sessions and runs for precise observability.
Perfect for complex workflows where you want to trace specific steps with custom metadata.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { trace } from '@orka-js/collector';
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! })
});
// 1 - Start a tracing session (logical group of operations)
const sessionId = trace.session('Support Client - Ticket #TCK-1234');
console.log('Session started: ' + sessionId);
// 2 - Trace a complete RAG operation with detailed metadata
const ragRunId = trace.start('workflow', 'RAG Query Processing', {
query: 'How to configure Redis cache in OrkaJS?',
knowledge: 'documentation',
userId: 'user-456',
ticketId: 'TCK-1234',
priority: 'high'
}, {
category: 'technical-support',
complexity: 'medium',
estimatedTokens: 1500
});
try {
// Step 1: Retrieval from knowledge base
const retrievalRunId = trace.start('retrieval', 'Knowledge Search', {
query: 'Redis cache configuration OrkaJS',
topK: 5
});
// Simulation of search (replace with actual call)
const retrievedDocs = [
{ content: 'Redis cache configuration...', score: 0.95 },
{ content: 'Cache best practices...', score: 0.87 }
];
trace.end(retrievalRunId, {
documents: retrievedDocs.length,
avgScore: retrievedDocs.reduce((sum, doc) => sum + doc.score, 0) / retrievedDocs.length,
searchTime: 45
});
// Step 2: Generate response with LLM
const llmRunId = trace.start('llm', 'Generate Technical Response', {
prompt: 'Based on the retrieved docs...',
model: 'gpt-4',
maxTokens: 500
});
const result = await orka.ask({
knowledge: 'documentation',
question: 'How to configure Redis cache in OrkaJS?'
});
trace.end(llmRunId, result, {
responseLength: result.length,
actualTokens: 342,
model: 'gpt-4',
temperature: 0.1
});
// 3 - Finalize the RAG tracing
trace.end(ragRunId, {
finalResponse: result,
totalProcessingTime: 1250,
customerSatisfaction: 'resolved'
});
console.log('Ticket processed successfully');
} catch (error) {
// Error handling with tracing
trace.error(ragRunId, error instanceof Error ? error: new Error(String(error)));
console.error('Error during processing:', error);
}
// 4 - Finalize the session
trace.endSession(sessionId);
// 5 - Retrieve session metrics
const collector = getCollector();
const sessionMetrics = collector.getMetrics(sessionId);
console.log('Metrics of the session:', {
totalRuns: sessionMetrics.totalRuns,
avgLatency: `${sessionMetrics.avgLatencyMs.toFixed(2)}ms`,
totalTokens: sessionMetrics.totalTokens,
errorRate: `${(sessionMetrics.errorRate * 100).toFixed(1)}%`
});
//🎯 RESULT:
//• Session structured with business context
//• Hierarchical runs with rich metadata
//• Error handling with preserved context
//• Metrics per session for analysis
Best practices
- • Use sessions to group related operations (user ticket, transaction, etc.)
- • Add rich metadata for business context
- • Structure runs in hierarchy to reflect your workflow
- • Manage errors systematically with trace.error()
- • Calculate metrics per session for analysis
Automatic wrapper with withTrace()
Objective : Automate tracing of existing functions without modifying their internal code.
• Elegant solution to add observability to legacy functions or external libraries.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { withTrace } from '@orka-js/collector';
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! })
});
// 1 - Existing function (without modification)
async function processUserQuery(query: string, options?: {
maxResults?: number;
includeSources?: boolean;
}) {
console.log('Processing query: ' + query);
// Complex business logic
const result = await orka.ask({
question: query,
knowledge: 'documentation',
topK: options?.maxResults ?? 3
});
if (options?.includeSources) {
return {
answer: result,
sources: ['doc-1', 'doc-2', 'doc-3'], // Simulation
metadata: {
processedAt: new Date().toISOString(),
queryLength: query.length
}
};
}
return result;
}
// 2 - Automatic wrapping with tracing
const tracedProcessQuery = withTrace(
processUserQuery,
{
name: 'Query Processing Pipeline',
type: 'workflow' as const,
// Automatic metadata for each call
metadata: {
version: '2.1.0',
service: 'user-support',
environment: 'production'
}
}
);
// 3 - Usage - automatic tracing
async function handleCustomerRequests() {
const queries = [
'Comment installer OrkaJS ?',
'Quels sont les providers LLM supportés ?',
'Comment configurer le cache ?'
];
for (const query of queries) {
try {
console.log('Processing:' + query);
// Normal call - automatic tracing in background
const result = await tracedProcessQuery(query, {
maxResults: 5,
includeSources: true
});
console.log('Result:', result);
} catch (error) {
console.error('Error for "' + query + '":', error);
}
}
}
// 4 - Wrapper for utility functions
const tracedLogger = withTrace(
async (level: 'info' | 'warn' | 'error', message: string, data?: any) => {
// Existing logging logic
const timestamp = new Date().toISOString();
console.log('[' + timestamp + '] [' + level.toUpperCase() + '] ' + message, data || '');
// Simulation of sending to external service
await new Promise(resolve => setTimeout(resolve, 10));
},
{
name: 'External Logging',
type: 'tool' as const,
metadata: {
provider: 'datadog',
retention: '30d'
}
}
);
// 5 - Execution of the demo
await handleCustomerRequests();
// Logging with tracing
await tracedLogger('info', 'Demo completed successfully', {
totalQueries: 3,
avgLatency: 850
});
//🎯 ADVANTAGES:
//• No modification of the original code
//• Automatic and consistent tracing
//• Automatically enriched metadata
//• Error handling preserved
//• Reusable for any function
🔧 Non-intrusive
No need to modify the existing code
⚡ Performance
Minimal overhead (~1-2ms)
🔄 Reusable
Applicable to any function
Decorator for classes and methods
Objective: Use TypeScript decorators for elegant tracing at the class level.
• Modern and declarative approach to add observability to your object-oriented architecture.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { MemoryVectorAdapter } from '@orka-js/memory';
import { Trace } from '@orka-js/collector';
// 1- Initial configuration
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! }),
vectorDB: new MemoryVectorAdapter()
});
// 2- Service class with decorators
class CustomerSupportAgent {
private readonly agentId: string;
private readonly version: string = '1.2.0';
constructor(agentId: string) {
this.agentId = agentId;
}
@Trace({ name: 'Generate Technical Response', type: 'agent' as const,
metadata: {
category: 'customer-support',
criticality: 'high'
}
})
async generateResponse(query: string, context?: {
userId?: string;
sessionId?: string;
priority?: 'low' | 'medium' | 'high';
}): Promise<{
response: string;
confidence: number;
sources: string[];
processingTime: number;
}> {
const startTime = Date.now();
try {
// Business logic
const result = await orka.ask({
question: query,
knowledge: 'support-docs',
system: `You are a technical support agent (ID: ${this.agentId}). Respond with precision and empathy.`
});
const processingTime = Date.now() - startTime;
return {
response: result,
confidence: 0.92,
sources: ['kb-123', 'kb-456'],
processingTime
};
} catch (error) {
// Errors are automatically traced by the decorator
throw new Error(`Failed to generate response: ${error instanceof Error ? error.message : String(error)}`);
}
}
@Trace({ name: 'Create Knowledge Base', type: 'retrieval' as const,
metadata: {
operation: 'knowledge-ingestion',
batch: true
}
})
async createKnowledgeBase(docs: {
title: string;
content: string;
category: string;
tags: string[];
}[]): Promise<void> {
console.log(`📚 Ingesting ${docs.length} documents...`);
// Document preparation
const processedDocs = docs.map(doc => ({
...doc,
processedAt: new Date().toISOString(),
agentVersion: this.version
}));
// Creation in Orka
await orka.knowledge.create({
name: 'support-docs',
source: processedDocs.map(doc => doc.content)
});
console.log('Knowledge base created successfully');
}
@Trace({ name: 'Update Agent Configuration', type: 'custom' as const })
updateConfiguration(config: {
temperature?: number;
maxTokens?: number;
model?: string;
}): void {
// Configuration logic
console.log(' Configuration updated:', config);
// Simulation of validation
if (config.temperature && (config.temperature < 0 || config.temperature > 2)) {
throw new Error('Temperature must be between 0 and 2');
}
}
// Method without decorator (not traced)
private internalHelper(data: any): void {
// Internal logic - no automatic tracing
console.log('🔧 Internal processing:', data);
}
}
// 3 - Inherited class with tracing
class AdvancedSupportAgent extends CustomerSupportAgent {
@Trace({ name: 'Multi-step Problem Resolution', type: 'workflow' as const })
async resolveComplexIssue(issue: {
description: string;
severity: 'low' | 'medium' | 'high' | 'critical';
category: string;
}): Promise<{
resolution: string;
steps: string[];
escalation: boolean;
}> {
const steps: string[] = [];
try {
// Step 1: Problem analysis
steps.push('Problem analysis');
const analysis = await this.generateResponse(
`Analyze this problem: ${issue.description}`,
{ priority: issue.severity }
);
// Step 2: Solution search
steps.push('Solution search');
const solutions = await this.generateResponse(
`Find solutions for: ${issue.description}`,
{ priority: issue.severity }
);
// Step 3: Resolution generation
steps.push('Resolution generation');
const resolution = await this.generateResponse(
`Generate a complete resolution based on the analysis and solutions`,
{ priority: issue.severity }
);
return {
resolution: resolution.response,
steps,
escalation: issue.severity === 'critical'
};
} catch (error) {
steps.push('Resolution failed');
throw error;
}
}
}
// 4 - Usage
async function demoSupportAgent() {
console.log('🚀 Starting support agent...');
// Create agent
const agent = new CustomerSupportAgent('agent-007');
// Configuration (automatically traced)
agent.updateConfiguration({
temperature: 0.1,
maxTokens: 1000,
model: 'gpt-4'
});
// Create knowledge base (automatically traced)
await agent.createKnowledgeBase([
{
title: 'Installation Guide',
content: 'OrkaJS is installed with npm install orajs...',
category: 'installation',
tags: ['setup', 'npm', 'installation']
},
{
title: 'Redis configuration',
content: 'To configure Redis cache...',
category: 'configuration',
tags: ['redis', 'cache', 'configuration']
}
]);
// Generate response (automatically traced)
const response = await agent.generateResponse(
'How to configure Redis cache?',
{
userId: 'user-123',
sessionId: 'session-456',
priority: 'high'
}
);
console.log('Response generated:', response);
// Advanced agent
const advancedAgent = new AdvancedSupportAgent('agent-elite');
const resolution = await advancedAgent.resolveComplexIssue({
description: 'The application does not start after update',
severity: 'critical',
category: 'technical'
});
console.log('Complex resolution:', resolution);
}
// 5 - Execution
await demoSupportAgent();
//🎯 ADVANTAGES OF DECORATORS:
//• Declarative and elegant syntax
//• No pollution of business code
//• Inheritance preserved
//• Centralized configuration
//• Errors traced automatically
//• Ideal for object-oriented architecture
Usage tips
- • Use descriptive names for traced methods
- • Add contextual metadata
- • Combine with class inheritance
- • Use for critical methods only
- • Available only in TypeScript
- • Requires experimental decorator activation
- • Does not work with arrow functions
- • Slight overhead at runtime
Remote production with centralized collector
Objective : Configure tracing for a production environment with a remote collector.
Ideal solution for production applications where you want to centralize traces and monitor performance.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { devtools } from '@orka-js/collector';
// 1 - Configuration Production - Mode Agent (send traces)
const { tracer, stop } = await devtools({
source: 'remote',
mode: 'agent',
remote: {
endpoint: 'https://traces.company.com',
apiKey: process.env.COLLECTOR_API_KEY!,
projectId: 'customer-support-ai',
environment: 'production',
sampling: 0.1, // 10% of traces to control costs
batchSize: 50, // Send in batches of 50 traces
flushInterval: 5000 // Force send every 5s
},
verbose: true,
// Local fallback configuration
fallback: {
enableLocalCache: true,
maxCacheSize: 1000,
retryAttempts: 3,
retryDelay: 1000
}
});
// 2 - Configuration Orka for production
const orka = createOrka({
llm: new OpenAIAdapter({
apiKey: process.env.OPENAI_API_KEY!,
timeoutMs: 30000,
maxRetries: 3
}),
tracer,
// Resilience configuration
resilience: {
retry: { attempts: 3, backoff: 'exponential' },
fallback: {
primary: 'gpt-4',
fallback: 'gpt-3.5-turbo'
}
}
});
// 3 - Production service with tracing
class ProductionSupportService {
private readonly serviceVersion = '2.3.1';
private readonly region = process.env.AWS_REGION || 'us-east-1';
async handleCustomerRequest(request: {
userId: string;
query: string;
sessionId: string;
priority: 'low' | 'medium' | 'high' | 'urgent';
}): Promise<{
response: string;
confidence: number;
processingTime: number;
traceId?: string;
}> {
const startTime = Date.now();
try {
// Automatic tracing via injected tracer
const result = await orka.ask({
question: request.query,
knowledge: 'production-docs',
system: `Service de support (v${this.serviceVersion}) - Priorité: ${request.priority}`,
metadata: {
userId: request.userId,
sessionId: request.sessionId,
region: this.region,
priority: request.priority,
serviceVersion: this.serviceVersion
}
});
const processingTime = Date.now() - startTime;
return {
response: result,
confidence: 0.89,
processingTime
};
} catch (error) {
// Errors are automatically tracked.
console.error(`Request processing error ${request.userId}:`, error);
throw error;
}
}
// Health metrics of the service
async getHealthMetrics(): Promise<{
status: 'healthy' | 'degraded' | 'unhealthy';
avgLatency: number;
errorRate: number;
totalRequests: number;
uptime: number;
}> {
const collector = getCollector();
const metrics = collector.getMetrics();
const avgLatency = metrics.avgLatencyMs;
const errorRate = metrics.errorRate;
const totalRequests = metrics.totalRuns;
// Determine health status
let status: 'healthy' | 'degraded' | 'unhealthy';
if (errorRate > 0.1 || avgLatency > 5000) {
status = 'unhealthy';
} else if (errorRate > 0.05 || avgLatency > 3000) {
status = 'degraded';
} else {
status = 'healthy';
}
return {
status,
avgLatency,
errorRate,
totalRequests,
uptime: process.uptime()
};
}
}
// 4 - Monitoring and alerts
class ProductionMonitor {
private alertThresholds = {
maxLatency: 3000, // 3s
maxErrorRate: 0.05, // 5%
minSuccessRate: 0.95 // 95%
};
async startMonitoring(): Promise<void> {
const collector = getCollector();
// Subscription to tracing events
collector.subscribe((event) => {
if (event.type === 'run:end') {
const { run } = event;
// Alertes de latence
if (run.latencyMs && run.latencyMs > this.alertThresholds.maxLatency) {
this.sendAlert('HIGH_LATENCY', {
operation: run.name,
latency: run.latencyMs,
threshold: this.alertThresholds.maxLatency
});
}
}
if (event.type === 'run:error') {
this.sendAlert('OPERATION_ERROR', {
operation: event.run.name,
error: event.error,
severity: this.getSeverity(event.run.type)
});
}
});
// Periodic monitoring
setInterval(async () => {
const metrics = collector.getMetrics();
await this.checkHealth(metrics);
}, 30000); // Every 30s
}
private async sendAlert(type: string, data: any): Promise<void> {
// Sending alerts to your monitoring system
console.log(`🚨 ALERT [${type}]:`, data);
// Integration with PagerDuty, Slack, etc.
await this.pagerDuty.trigger(type, data);
await this.slack.sendAlert(type, data);
}
private getSeverity(operationType: string): 'low' | 'medium' | 'high' | 'critical' {
const severityMap = {
'llm': 'critical',
'agent': 'high',
'retrieval': 'medium',
'tool': 'low'
};
return severityMap[operationType as keyof typeof severityMap] || 'medium';
}
private async checkHealth(metrics: any): Promise<void> {
if (metrics.errorRate > this.alertThresholds.maxErrorRate) {
await this.sendAlert('HIGH_ERROR_RATE', {
currentRate: metrics.errorRate,
threshold: this.alertThresholds.maxErrorRate
});
}
}
}
// 5 - Deployment and usage
async function deployProductionService() {
console.log('Deployment of the production service...');
const supportService = new ProductionSupportService();
const monitor = new ProductionMonitor();
// Starting monitoring
await monitor.startMonitoring();
// Simulation of production requests
const requests = [
{ userId: 'user-001', query: 'How to reset my password?', sessionId: 'sess-001', priority: 'urgent' },
{ userId: 'user-002', query: 'API key configuration', sessionId: 'sess-002', priority: 'high' },
{ userId: 'user-003', query: 'Installation guide', sessionId: 'sess-003', priority: 'low' }
];
for (const req of requests) {
try {
const result = await supportService.handleCustomerRequest(req);
console.log(`${req.userId} request traited in ${result.processingTime}ms`);
} catch (error) {
console.error(`Failure request ${req.userId}:`, error);
}
}
// Checking health
const health = await supportService.getHealthMetrics();
console.log('Service health:', health);
// Proper shutdown
await stop();
}
// 6 - Execution
await deployProductionService();
//🎯 AVANTAGES PRODUCTION:
//• Centralized traces for analysis
//• Sampling for cost control
//• Real-time monitoring with alerts
//• Built-in fallback and resilience
//• Automatic health metrics
//• Compatible with cloud infrastructure
Production Considerations
- • Configure sampling to control costs
- • Use secure API keys
- • Set up latency alerts
- • Monitor error rates
Best Practices
- • Implement retries and fallbacks
- • Use structured metadata
- • Configure appropriate timeouts
- • Test under load before deployment
Complex multi-step workflow
Objective: Implement a complex workflow with detailed tracing of each step.
Perfect for enterprise applications with multi-step processes and conditional decisions.
import { createOrka } from '@orka-js/core';
import { OpenAIAdapter } from '@orka-js/openai';
import { trace, withTrace } from '@orka-js/collector';
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! })
});
// 1 - Complex workflow for ticket processing
const processTicket = withTrace(
async (ticket: {
id: string;
message: string;
userId: string;
category: 'technical' | 'billing' | 'general';
priority: 'low' | 'medium' | 'high' | 'urgent';
}) => {
const sessionId = trace.session(`Ticket ${ticket.id}`);
console.log(`🎫 Début traitement ticket ${ticket.id}`);
try {
// Step 1: Analysis and classification
const analysisResult = await analyzeTicket(ticket);
// Step 2: Search for solutions based on category
const solutions = await findSolutions(ticket, analysisResult);
// Step 3: Generate personalized response
const response = await generateResponse(ticket, analysisResult, solutions);
// Step 4: Additional actions based on category
const actions = await performAdditionalActions(ticket, response);
// Step 5: Finalization and tracking
const result = await finalizeTicket(ticket, response, actions);
trace.endSession(sessionId);
return result;
} catch (error) {
trace.error(sessionId, error instanceof Error ? error: new Error(String(error)));
throw error;
}
},
{
name: 'Ticket Processing Workflow',
type: 'workflow' as const,
metadata: {
version: '3.0.0',
complexity: 'high',
estimatedDuration: '30-60s'
}
}
);
// 2 - Functions of the workflow with detailed tracing
async function analyzeTicket(ticket: any) {
const analysisId = trace.start('agent', 'Ticket Analysis', {
ticketId: ticket.id,
category: ticket.category,
priority: ticket.priority
});
try {
const analysis = await orka.ask({
question: `
Analyze this customer ticket:
Message: "${ticket.message}"
Category: ${ticket.category}
Priority: ${ticket.priority}
Provide:
1. Sentiment (positive/negative/neutral)
2. Real urgency(1-10)
3. Complexity (simple/medium/complex)
4. Main keywords
5. Category suggestion if necessary
`,
system: 'You are an expert customer support analyst. Be precise and structured.',
metadata: {
ticketId: ticket.id,
analysisType: 'comprehensive'
}
});
const parsedAnalysis = parseAnalysis(analysis);
trace.end(analysisId, parsedAnalysis, {
sentiment: parsedAnalysis.sentiment,
urgency: parsedAnalysis.urgency,
complexity: parsedAnalysis.complexity,
keywords: parsedAnalysis.keywords.length
});
return parsedAnalysis;
} catch (error) {
trace.error(analysisId, error instanceof Error ? error: new Error('Analysis failed'));
throw error;
}
}
async function findSolutions(ticket: any, analysis: any) {
const searchId = trace.start('retrieval', 'Solution Search', {
ticketId: ticket.id,
category: ticket.category,
complexity: analysis.complexity
});
try {
// Build search query
const searchQuery = buildSearchQuery(ticket, analysis);
// Knowledge base search
const searchResults = await orka.ask({
knowledge: 'solutions-db',
question: searchQuery,
topK: analysis.complexity === 'complexe' ? 8 : 5,
metadata: {
searchType: 'solutions',
ticketId: ticket.id,
category: ticket.category
}
});
// Evaluation and filtering of solutions
const evaluatedSolutions = await evaluateSolutions(searchResults, analysis);
trace.end(searchId, evaluatedSolutions, {
totalFound: searchResults.length,
relevantSolutions: evaluatedSolutions.length,
avgRelevance: evaluatedSolutions.reduce((sum: number, sol: any) => sum + sol.relevance, 0) / evaluatedSolutions.length
});
return evaluatedSolutions;
} catch (error) {
trace.error(searchId, error instanceof Error ? error: new Error('Solution search failed'));
throw error;
}
}
async function generateResponse(ticket: any, analysis: any, solutions: any[]) {
const responseId = trace.start('llm', 'Response Generation', {
ticketId: ticket.id,
sentiment: analysis.sentiment,
solutionCount: solutions.length
});
try {
const responsePrompt = buildResponsePrompt(ticket, analysis, solutions);
const response = await orka.ask({
question: responsePrompt,
system: `Generate an empathetic and professional response. Customer sentiment: ${analysis.sentiment}`,
model: analysis.complexity === 'complexe' ? 'gpt-4' : 'gpt-3.5-turbo',
maxTokens: analysis.complexity === 'complexe' ? 800 : 400,
metadata: {
responseType: 'customer_reply',
ticketId: ticket.id,
complexity: analysis.complexity
}
});
const parsedResponse = parseResponse(response);
trace.end(responseId, parsedResponse, {
responseLength: parsedResponse.text.length,
tone: parsedResponse.tone,
hasSolution: parsedResponse.solutionProvided,
nextSteps: parsedResponse.nextSteps?.length || 0
});
return parsedResponse;
} catch (error) {
trace.error(responseId, error instanceof Error ? error: new Error('Response generation failed'));
throw error;
}
}
async function performAdditionalActions(ticket: any, response: any) {
const actionsId = trace.start('workflow', 'Additional Actions', {
ticketId: ticket.id,
category: ticket.category,
hasEscalation: response.requiresEscalation
});
const actions = [];
try {
// Action 1: Climb if necessary
if (response.requiresEscalation) {
const escalationId = trace.start('tool', 'Escalation Process');
const escalationResult = await escalateTicket(ticket, response);
actions.push({ type: 'escalation', result: escalationResult });
trace.end(escalationId, escalationResult);
}
// Action 2: Creation of follow-up ticket if technical problem
if (ticket.category === 'technical' && response.requiresFollowUp) {
const followUpId = trace.start('tool', 'Follow-up Ticket Creation');
const followUpResult = await createFollowUpTicket(ticket, response);
actions.push({ type: 'followup', result: followUpResult });
trace.end(followUpId, followUpResult);
}
// Action 3: Update of the knowledge base
if (response.suggestsKnowledgeUpdate) {
const updateId = trace.start('tool', 'Knowledge Base Update');
const updateResult = await suggestKnowledgeUpdate(ticket, response);
actions.push({ type: 'knowledge_update', result: updateResult });
trace.end(updateId, updateResult);
}
// Action 4: Notification client according to priority
if (ticket.priority === 'urgent' || ticket.priority === 'high') {
const notificationId = trace.start('tool', 'Customer Notification');
const notificationResult = await sendCustomerNotification(ticket, response);
actions.push({ type: 'notification', result: notificationResult });
trace.end(notificationId, notificationResult);
}
trace.end(actionsId, actions, {
totalActions: actions.length,
escalationTriggered: response.requiresEscalation,
followUpCreated: response.requiresFollowUp
});
return actions;
} catch (error) {
trace.error(actionsId, error instanceof Error ? error: new Error('Additional actions failed'));
throw error;
}
}
async function finalizeTicket(ticket: any, response: any, actions: any[]) {
const finalizationId = trace.start('workflow', 'Ticket Finalization', {
ticketId: ticket.id,
responseProvided: !!response,
actionsTaken: actions.length
});
try {
// Calculation of metrics
const metrics = {
totalProcessingTime: Date.now() - ticket.startTime,
customerSatisfaction: predictSatisfaction(response),
resolutionCompleteness: calculateCompleteness(response, actions),
escalationRate: actions.some(a => a.type === 'escalation') ? 1 : 0
};
// Final recording
const finalResult = {
ticketId: ticket.id,
status: 'resolved',
response: response.text,
actions: actions.map(a => ({ type: a.type, success: a.result.success })),
metrics,
timestamp: new Date().toISOString()
};
trace.end(finalizationId, finalResult, metrics);
return finalResult;
} catch (error) {
trace.error(finalizationId, error instanceof Error ? error: new Error('Finalization failed'));
throw error;
}
}
// 3 - Utility functions (simplified)
function parseAnalysis(analysis: string) {
// Parsing of the analysis response
return {
sentiment: 'neutre',
urgency: 5,
complexity: 'moyenne',
keywords: ['installation', 'configuration'],
suggestedCategory: 'technical'
};
}
function buildSearchQuery(ticket: any, analysis: any) {
return `${ticket.message} ${analysis.keywords.join(' ')}`;
}
async function evaluateSolutions(results: string[], analysis: any) {
// Evaluation of found solutions
return results.slice(0, 3).map((result, index) => ({
content: result,
relevance: 0.9 - (index * 0.1),
confidence: 0.85
}));
}
// 4 - Demo of the workflow
async function demoComplexWorkflow() {
console.log('🚀 Demo of complex workflow...');
const tickets = [
{
id: 'TCK-001',
message: 'My application won't start after the last update',
userId: 'user-123',
category: 'technical' as const,
priority: 'high' as const
},
{
id: 'TCK-002',
message: 'I would like to understand my bill from last month',
userId: 'user-456',
category: 'billing' as const,
priority: 'medium' as const
}
];
for (const ticket of tickets) {
try {
console.log(`\n📋 Ticket processing ${ticket.id}...`);
const result = await processTicket({
...ticket,
startTime: Date.now()
});
console.log('✅ Ticket resolved:', {
id: result.ticketId,
status: result.status,
processingTime: result.metrics.totalProcessingTime,
satisfaction: result.metrics.customerSatisfaction
});
} catch (error) {
console.error(`❌ Failure ticket ${ticket.id}:`, error);
}
}
}
// 5 - Execution
await demoComplexWorkflow();
//🎯 WORKFLOW RESULTS:
//• Full hierarchical tracing
//• Detailed metrics per step
//• Contextual error handling
//• Traced conditional actions
//• Performance analysis
//• Optimization potential
Workflow Architecture
- • Analysis and classification of the ticket
- • Search for relevant solutions
- • Generation of personalized response
- • Automatic conditional actions
- • Finalization with metrics
- • Total processing time
- • Predicted customer satisfaction
- • Escalation rate
- • Resolution completeness
- • Performance by step
Monitoring and metrics
Objective: Set up comprehensive monitoring with alerts, dashboards, and performance analysis.
Enterprise solution for monitoring your AI applications with real-time metrics and intelligent alerts.
import { createOrka, OpenAIAdapter } from 'orkajs';
import { getCollector, trace } from '@orka-js/collector';
const orka = createOrka({
llm: new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY! })
});
// 1 - Advanced monitoring system
class AdvancedMonitoringSystem {
private collector = getCollector();
private metrics: Map<string, number[]> = new Map();
private alerts: Alert[] = [];
private dashboards: Map<string, Dashboard> = new Map();
constructor() {
this.setupEventListeners();
this.initializeMetrics();
this.startPeriodicMonitoring();
}
// Configuration of listeners for all events
private setupEventListeners(): void {
this.collector.subscribe((event) => {
switch (event.type) {
case 'run:start':
this.handleRunStart(event);
break;
case 'run:end':
this.handleRunEnd(event);
break;
case 'run:error':
this.handleRunError(event);
break;
case 'session:start':
this.handleSessionStart(event);
break;
case 'session:end':
this.handleSessionEnd(event);
break;
}
});
}
// Handling of run start
private handleRunStart(event: any): void {
const { run } = event;
// Tracking of active runs
this.updateMetric('active_runs', 1);
// Alerts if too many simultaneous executions
if (this.getMetric('active_runs') > 100) {
this.triggerAlert('HIGH_CONCURRENCY', {
current: this.getMetric('active_runs'),
threshold: 100,
severity: 'warning'
});
}
// Structured logging
console.log(`🟢 Start: ${run.type}/${run.name} [${run.id}]`);
}
// Handling of run end
private handleRunEnd(event: any): void {
const { run } = event;
// Update metrics
this.updateMetric('active_runs', -1);
this.updateMetric('completed_runs', 1);
this.addMetric('latency_ms', run.latencyMs || 0);
// Type-specific metrics
this.updateTypeMetrics(run.type, run);
// Performance alerts
if (run.latencyMs && run.latencyMs > 5000) {
this.triggerAlert('SLOW_OPERATION', {
operation: run.name,
latency: run.latencyMs,
threshold: 5000,
severity: 'warning'
});
}
// Token metrics if available
if (run.metadata?.totalTokens) {
this.addMetric('tokens_used', run.metadata.totalTokens);
this.addMetric('cost_usd', run.metadata.cost || 0);
}
console.log(`✅ End: ${run.type}/${run.name} - ${run.latencyMs}ms`);
}
// Handling of errors
private handleRunError(event: any): void {
const { run, error } = event;
this.updateMetric('error_count', 1);
this.updateMetric('active_runs', -1);
// Critical alerts based on operation type
const severity = this.getErrorSeverity(run.type, error);
this.triggerAlert('OPERATION_ERROR', {
operation: run.name,
type: run.type,
error: error,
severity,
timestamp: new Date().toISOString()
});
// Tracking of errors by type
const errorKey = `${run.type}_errors`;
this.updateMetric(errorKey, 1);
console.error(`❌ Error: ${run.type}/${run.name} - ${error}`);
}
// Monitoring of sessions
private handleSessionStart(event: any): void {
this.updateMetric('active_sessions', 1);
console.log(`🎯 Session start: ${event.sessionId}`);
}
private handleSessionEnd(event: any): void {
this.updateMetric('active_sessions', -1);
this.updateMetric('completed_sessions', 1);
// Session performance analysis
const session = this.collector.getSession(event.sessionId);
if (session) {
const sessionDuration = (session.endTime || Date.now()) - session.startTime;
this.addMetric('session_duration_ms', sessionDuration);
// Alerts if sessions too long
if (sessionDuration > 300000) { // 5 minutes
this.triggerAlert('LONG_SESSION', {
sessionId: event.sessionId,
duration: sessionDuration,
threshold: 300000,
severity: 'warning'
});
}
}
console.log(`🏁 Session end: ${event.sessionId}`);
}
// 2 - Intelligent alert system
private triggerAlert(type: string, data: any): void {
const alert: Alert = {
id: this.generateId(),
type,
severity: data.severity || 'info',
message: this.buildAlertMessage(type, data),
data,
timestamp: new Date().toISOString(),
acknowledged: false
};
this.alerts.push(alert);
// Filtering of alerts (duplicates, frequency, etc.)
if (this.shouldSendAlert(alert)) {
this.sendAlert(alert);
}
// Cleaning up old alerts
this.cleanupOldAlerts();
}
private buildAlertMessage(type: string, data: any): string {
const messages = {
'HIGH_CONCURRENCY': `High concurrency: ${data.current} active runs (threshold: ${data.threshold})`,
'SLOW_OPERATION': `Slow operation: ${data.operation} took ${data.latency}ms (threshold: ${data.threshold}ms)`,
'OPERATION_ERROR': `Operation error in ${data.operation} (${data.type}): ${data.error}`,
'LONG_SESSION': `Long session detected: ${data.duration}ms (threshold: ${data.threshold}ms)`,
'HIGH_ERROR_RATE': `High error rate: ${(data.rate * 100).toFixed(1)}% (threshold: ${(data.threshold * 100).toFixed(1)}%)`,
'COST_SPIKE': `Cost spike detected: $${data.cost.toFixed(4)} in ${data.period}`
};
return messages[type as keyof typeof messages] || `Alert: ${type}`;
}
private shouldSendAlert(alert: Alert): boolean {
// Éviter les doublons récents
const recentSimilar = this.alerts.filter(a =>
a.type === alert.type &&
!a.acknowledged &&
Date.now() - new Date(a.timestamp).getTime() < 60000 // 1 minute
);
return recentSimilar.length === 0;
}
private async sendAlert(alert: Alert): void {
console.log(`🚨 ALERT [${alert.severity.toUpperCase()}]: ${alert.message}`);
// Intégrations externes
if (alert.severity === 'critical') {
await this.pagerDuty.trigger(alert);
await this.slack.sendCriticalAlert(alert);
} else if (alert.severity === 'warning') {
await this.slack.sendWarning(alert);
}
// Email for serious alerts
if (alert.severity === 'critical') {
await this.emailService.sendAlert(alert);
}
}
// 3 - Real-time dashboards
private initializeDashboards(): void {
// Main dashboard
this.dashboards.set('main', {
id: 'main',
name: 'Main Dashboard',
widgets: [
{ type: 'metric', name: 'Active Runs', key: 'active_runs', format: 'number' },
{ type: 'metric', name: 'Error Rate', key: 'error_rate', format: 'percentage' },
{ type: 'metric', name: 'Avg Latency', key: 'avg_latency', format: 'duration' },
{ type: 'metric', name: 'Total Cost', key: 'total_cost', format: 'currency' }
]
});
// Performance dashboard
this.dashboards.set('performance', {
id: 'performance',
name: 'Performance Dashboard',
widgets: [
{ type: 'chart', name: 'Latency Trend', key: 'latency_ms', chartType: 'line' },
{ type: 'chart', name: 'Throughput', key: 'completed_runs', chartType: 'bar' },
{ type: 'chart', name: 'Error Rate', key: 'error_rate', chartType: 'area' }
]
});
}
// 4 - Metrics and statistics
private initializeMetrics(): void {
// Basic metrics
this.metrics.set('active_runs', []);
this.metrics.set('completed_runs', []);
this.metrics.set('error_count', []);
this.metrics.set('latency_ms', []);
this.metrics.set('tokens_used', []);
this.metrics.set('cost_usd', []);
// Metrics by operation type
['llm', 'agent', 'retrieval', 'tool', 'workflow'].forEach(type => {
this.metrics.set(`${type}_latency`, []);
this.metrics.set(`${type}_errors`, []);
this.metrics.set(`${type}_count`, []);
});
}
private updateMetric(key: string, delta: number): void {
const current = this.getMetric(key);
this.metrics.set(key, [current + delta]);
}
private addMetric(key: string, value: number): void {
const values = this.metrics.get(key) || [];
values.push(value);
// Keep only the last 1000 values
if (values.length > 1000) {
values.splice(0, values.length - 1000);
}
this.metrics.set(key, values);
}
private getMetric(key: string): number {
const values = this.metrics.get(key) || [];
return values.length > 0 ? values[values.length - 1] : 0;
}
private updateTypeMetrics(type: string, run: any): void {
this.updateMetric(`${type}_count`, 1);
if (run.latencyMs) {
this.addMetric(`${type}_latency`, run.latencyMs);
}
}
// 5 - Periodic monitoring
private startPeriodicMonitoring(): void {
setInterval(() => {
this.performHealthCheck();
this.updateDashboardMetrics();
this.checkAnomalies();
}, 30000); //Every 30 seconds
}
private performHealthCheck(): void {
const health = this.getSystemHealth();
if (health.status !== 'healthy') {
this.triggerAlert('SYSTEM_HEALTH', {
status: health.status,
issues: health.issues,
severity: health.status === 'critical' ? 'critical' : 'warning'
});
}
}
private getSystemHealth(): { status: 'healthy' | 'warning' | 'critical', issues: string[] } {
const issues: string[] = [];
// Check key metrics
const errorRate = this.calculateErrorRate();
if (errorRate > 0.1) issues.push('High error rate');
const avgLatency = this.calculateAverageLatency();
if (avgLatency > 5000) issues.push('High average latency');
const activeRuns = this.getMetric('active_runs');
if (activeRuns > 200) issues.push('High concurrency');
let status: 'healthy' | 'warning' | 'critical' = 'healthy';
if (issues.length >= 2) status = 'critical';
else if (issues.length >= 1) status = 'warning';
return { status, issues };
}
private updateDashboardMetrics(): void {
this.dashboards.forEach(dashboard => {
dashboard.widgets.forEach(widget => {
widget.value = this.calculateWidgetValue(widget.key);
widget.lastUpdated = new Date().toISOString();
});
});
}
private calculateWidgetValue(key: string): any {
switch (key) {
case 'active_runs':
return this.getMetric('active_runs');
case 'error_rate':
return this.calculateErrorRate();
case 'avg_latency':
return this.calculateAverageLatency();
case 'total_cost':
return this.calculateTotalCost();
default:
return this.getMetric(key);
}
}
private checkAnomalies(): void {
// Anomaly detection based on statistics
const recentLatency = this.getRecentValues('latency_ms', 10);
const avgRecentLatency = recentLatency.reduce((a, b) => a + b, 0) / recentLatency.length;
const overallAvgLatency = this.calculateAverageLatency();
// Alert if recent latency > 2x overall average
if (avgRecentLatency > overallAvgLatency * 2) {
this.triggerAlert('LATENCY_ANOMALY', {
recent: avgRecentLatency,
average: overallAvgLatency,
ratio: avgRecentLatency / overallAvgLatency,
severity: 'warning'
});
}
}
// 6 - Utility methods
private calculateErrorRate(): number {
const totalRuns = this.getMetric('completed_runs');
const errorCount = this.getMetric('error_count');
return totalRuns > 0 ? errorCount / totalRuns: 0;
}
private calculateAverageLatency(): number {
const latencies = this.metrics.get('latency_ms') || [];
return latencies.length > 0
? latencies.reduce((a, b) => a + b, 0) / latencies.length
: 0;
}
private calculateTotalCost(): number {
const costs = this.metrics.get('cost_usd') || [];
return costs.reduce((a, b) => a + b, 0);
}
private getRecentValues(key: string, count: number): number[] {
const values = this.metrics.get(key) || [];
return values.slice(-count);
}
private getErrorSeverity(operationType: string, error: string): 'info' | 'warning' | 'critical' {
// Severity based on operation type and error
if (operationType === 'llm') return 'critical';
if (error.includes('timeout')) return 'warning';
if (error.includes('rate limit')) return 'critical';
return 'warning';
}
private generateId(): string {
return Math.random().toString(36).substr(2, 9);
}
private cleanupOldAlerts(): void {
const oneHour = 60 * 60 * 1000;
this.alerts = this.alerts.filter(alert =>
Date.now() - new Date(alert.timestamp).getTime() < oneHour
);
}
// 7 - Public API
public getDashboard(id: string): Dashboard | undefined {
return this.dashboards.get(id);
}
public getAlerts(severity?: string): Alert[] {
return severity
? this.alerts.filter(a => a.severity === severity)
: this.alerts;
}
public acknowledgeAlert(alertId: string): void {
const alert = this.alerts.find(a => a.id === alertId);
if (alert) {
alert.acknowledged = true;
}
}
public getMetrics(): any {
return {
activeRuns: this.getMetric('active_runs'),
completedRuns: this.getMetric('completed_runs'),
errorRate: this.calculateErrorRate(),
avgLatency: this.calculateAverageLatency(),
totalCost: this.calculateTotalCost(),
tokensUsed: this.metrics.get('cost_usd')?.length || 0
};
}
}
// Types pour le monitoring
interface Alert {
id: string;
type: string;
severity: 'info' | 'warning' | 'critical';
message: string;
data: any;
timestamp: string;
acknowledged: boolean;
}
interface Dashboard {
id: string;
name: string;
widgets: Widget[];
}
interface Widget {
type: 'metric' | 'chart';
name: string;
key: string;
format?: 'number' | 'percentage' | 'duration' | 'currency';
chartType?: 'line' | 'bar' | 'area';
value?: any;
lastUpdated?: string;
}
// 8 - Demonstration
async function demoAdvancedMonitoring() {
console.log('🚀 Advanced monitoring demonstration...');
// Initialize monitoring system
const monitoring = new AdvancedMonitoringSystem();
// Simulation of operations with different characteristics
const operations = [
{ type: 'llm', name: 'Generate Response', latency: 1200, tokens: 150, cost: 0.0045 },
{ type: 'retrieval', name: 'Knowledge Search', latency: 200, tokens: 0, cost: 0 },
{ type: 'agent', name: 'Process Request', latency: 3500, tokens: 300, cost: 0.009 },
{ type: 'tool', name: 'API Call', latency: 150, tokens: 0, cost: 0 },
{ type: 'workflow', name: 'Complex Pipeline', latency: 8000, tokens: 500, cost: 0.015 }
];
// Simulation of executions
for (let i = 0; i < 20; i++) {
const op = operations[i % operations.length];
const runId = trace.start(op.type, op.name, {
iteration: i,
testData: true
});
// Simulation of variable latency
await new Promise(resolve => setTimeout(resolve, op.latency / 10));
// Simulation of occasional errors
if (Math.random() < 0.1) {
trace.error(runId, `Simulated error in ${op.name}`);
} else {
trace.end(runId, `Result for ${op.name}`, {
totalTokens: op.tokens,
cost: op.cost,
latency: op.latency
});
}
}
// Wait a bit for monitoring
await new Promise(resolve => setTimeout(resolve, 2000));
// Display metrics
console.log('\n📊 Final metrics:', monitoring.getMetrics());
// Display alerts
const alerts = monitoring.getAlerts();
if (alerts.length > 0) {
console.log('\n🚨 Alerts generated:');
alerts.forEach(alert => {
console.log(` [${alert.severity}] ${alert.message}`);
});
}
// Main dashboard
const dashboard = monitoring.getDashboard('main');
if (dashboard) {
console.log('\n📈 Main dashboard:');
dashboard.widgets.forEach(widget => {
console.log(` ${widget.name}: ${widget.value}`);
});
}
}
// 9 - Execution
await demoAdvancedMonitoring();
//🎯 MONITORING RESULT:
//• Intelligent alert system
//• Real-time dashboards
//• Anomaly detection
//• Detailed metrics
//• Automatic health checks
//• Interface de gestion des alertes
Monitoring Capabilities
- • Latencies by operation type
- • Error and success rates
- • Token consumption
- • Costs and budget
- • Configurable thresholds
- • Anomaly detection
- • Anti-duplication filtering
- • External integrations
- • Customizable widgets
- • Real-time charts
- • Health checks
- • Data export
Master Observability with OrkaJS
These examples cover all use cases, from local development to enterprise production. Choose the approach that matches your needs and adapt it to your architecture.