Skip to content
🔗 分享本题
查看我的学习进度 →

23 模块 Q12 教学图:OpenTelemetry 在 Agent 系统中的完整接入实战

🧠 图解记忆:用统一 Span 语义跨模型、工具和服务传播上下文,才能获得端到端可追溯性;点击图片可查看原图。

💡 答案要点

为什么 Agent 需要 OpenTelemetry?

传统监控:单次 API 调用(输入 → 输出)
Agent 监控:多步骤轨迹链(Thought → Action → Observation → ... → Final)

OpenTelemetry = 统一采集 + 跨服务追踪 + 上下文传播

架构图:

┌─────────────────────────────────────────────────────────────┐
│                    OpenTelemetry Agent 接入架构              │
├─────────────────────────────────────────────────────────────┤
│  1. Trace 采集(跨 Agent 链路追踪)                          │
│     ├── traceparent header 传递(TraceID/SpanID)           │
│     ├── 每个 Tool 调用 = 一个 Span                           │
│     └── 父Span → 子Span 自动关联                            │
│                                                              │
│  2. Metrics 采集(指标监控)                                 │
│     ├── token_consumption_total(累计 Token 消耗)           │
│     ├── tool_call_duration_seconds(工具调用延迟)           │
│     ├── step_count_per_task(每任务步数分布)                │
│     └── agent_retry_count(重试次数分布)                    │
│                                                              │
│  3. Logs 采集(结构化日志)                                  │
│     ├── trace_id 关联(同一请求的所有日志)                   │
│     ├── span_id(定位到具体步骤)                            │
│     └── attributes(tool_name、model、temperature 等)        │
│                                                              │
│  4. Baggage 传播(跨服务上下文)                              │
│     ├── user_id、session_id、feature_flags                  │
│     └── 在所有 Span 间自动传递                               │
└─────────────────────────────────────────────────────────────┘

最小接入实现:

展开 Python 代码示例(65 行)
python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.trace import Status, StatusCode

# 1. 初始化 Provider
provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://collector:4317"))
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

tracer = trace.get_tracer(__name__)

# 2. 用装饰器自动追踪 Agent 步骤
class OpenTelemetryAgent:
    def __init__(self, name: str):
        self.name = name
        self.tracer = tracer
    
    async def run(self, task: str, tools: list):
        with self.tracer.start_as_current_span(
            f"agent.{self.name}.run",
            attributes={"task": task, "tool_count": len(tools)}
        ) as span:
            try:
                context = span.get_span_context()
                
                for step_idx, (thought, action, obs) in enumerate(self.reasoning_loop(task)):
                    with self.tracer.start_as_current_span(
                        f"agent.step.{step_idx}",
                        kind=trace.SpanKind.CLIENT,
                        attributes={
                            "step.thought": thought,
                            "step.action": action,
                            "step.observation": str(obs)[:200]
                        }
                    ) as step_span:
                        step_span.set_attribute("step.index", step_idx)
                        step_span.set_attribute(
                            "llm.token_usage", 
                            obs.get("token_count", 0) if isinstance(obs, dict) else 0
                        )
                        if "error" in obs:
                            step_span.set_status(Status(StatusCode.ERROR, obs["error"]))
                
                span.set_status(Status(StatusCode.OK))
                return final_result
                
            except Exception as e:
                span.set_status(Status(StatusCode.ERROR, str(e)))
                span.record_exception(e)
                raise

# 3. 自动注入 trace_id 到工具调用
def call_tool_with_trace(tool_name: str, tool_args: dict, parent_span):
    with tracer.start_as_current_span(
        f"tool.{tool_name}",
        context=parent_span.get_span_context()
    ) as tool_span:
        tool_span.set_attribute("tool.name", tool_name)
        tool_span.set_attribute("tool.args", str(tool_args))
        result = tool_execute(tool_name, tool_args)
        tool_span.set_attribute("tool.result_type", type(result).__name__)
        return result

关键配置(docker-compose):

yaml
# otel-collector.yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s

exporters:
  prometheus:
    endpoint: 0.0.0.0:8889
  jaeger:
    endpoint: http://jaeger:14250

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]

面试话术:

"我在生产环境中用 OpenTelemetry 做 Agent 可观测性,核心是三点:① 每个 Tool 调用是一个 Span,父Span自动关联子Span,能看清整个轨迹;② token消耗、步数、重试次数都入库,可以做成本分析和异常检测;③ trace_id 在所有日志里,打通了日志和链路。最实用的经验是用装饰器包装Agent的run方法,零侵入接入,不用改业务代码。"

📚 参考:OpenTelemetry 官方文档(Python 接入)