DATA
6de0bf9f5b
Vendored deer-flow upstream (bytedance/deer-flow) plus prompt-injection hardening: - New deerflow.security package: content_delimiter, html_cleaner, sanitizer (8 layers — invisible chars, control chars, symbols, NFC, PUA, tag chars, horizontal whitespace collapse with newline/tab preservation, length cap) - New deerflow.community.searx package: web_search, web_fetch, image_search backed by a private SearX instance, every external string sanitized and wrapped in <<<EXTERNAL_UNTRUSTED_CONTENT>>> delimiters - All native community web providers (ddg_search, tavily, exa, firecrawl, jina_ai, infoquest, image_search) replaced with hard-fail stubs that raise NativeWebToolDisabledError at import time, so a misconfigured tool.use path fails loud rather than silently falling back to unsanitized output - Native client back-doors (jina_client.py, infoquest_client.py) stubbed too - Native-tool tests quarantined under tests/_disabled_native/ (collect_ignore_glob via local conftest.py) - Sanitizer Layer 7 fix: only collapse horizontal whitespace, preserve newlines and tabs so list/table structure survives - Hardened runtime config.yaml references only the searx-backed tools - Factory overlay (backend/) kept in sync with deer-flow tree as a reference / source See HARDENING.md for the full audit trail and verification steps.
106 lines
3.5 KiB
Markdown
106 lines
3.5 KiB
Markdown
# Langfuse Tracing Implementation Plan
|
|
|
|
**Goal:** Add optional Langfuse observability support to DeerFlow while preserving existing LangSmith tracing and allowing both providers to be enabled at the same time.
|
|
|
|
**Architecture:** Extend tracing configuration from a single LangSmith-only shape to a multi-provider config, add a tracing callback factory that builds zero, one, or two callbacks based on environment variables, and update model creation to attach those callbacks. If a provider is explicitly enabled but misconfigured or fails to initialize, tracing initialization during model creation should fail with a clear error naming that provider.
|
|
|
|
**Tech Stack:** Python 3.12, Pydantic, LangChain callbacks, LangSmith, Langfuse, pytest
|
|
|
|
---
|
|
|
|
### Task 1: Add failing tracing config tests
|
|
|
|
**Files:**
|
|
- Modify: `backend/tests/test_tracing_config.py`
|
|
|
|
**Step 1: Write the failing tests**
|
|
|
|
Add tests covering:
|
|
- Langfuse-only config parsing
|
|
- dual-provider parsing
|
|
- explicit enable with missing required Langfuse fields
|
|
- provider enable detection without relying on LangSmith-only helpers
|
|
|
|
**Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_tracing_config.py -q`
|
|
Expected: FAIL because tracing config only supports LangSmith today.
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Update tracing config code to represent multiple providers and expose helper functions needed by the tests.
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_tracing_config.py -q`
|
|
Expected: PASS
|
|
|
|
### Task 2: Add failing callback factory and model attachment tests
|
|
|
|
**Files:**
|
|
- Modify: `backend/tests/test_model_factory.py`
|
|
- Create: `backend/tests/test_tracing_factory.py`
|
|
|
|
**Step 1: Write the failing tests**
|
|
|
|
Add tests covering:
|
|
- LangSmith callback creation
|
|
- Langfuse callback creation
|
|
- dual callback creation
|
|
- startup failure when an explicitly enabled provider cannot initialize
|
|
- model factory appends all tracing callbacks to model callbacks
|
|
|
|
**Step 2: Run tests to verify they fail**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_model_factory.py tests/test_tracing_factory.py -q`
|
|
Expected: FAIL because there is no provider factory and model creation only attaches LangSmith.
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Create tracing callback factory module and update model factory to use it.
|
|
|
|
**Step 4: Run tests to verify they pass**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_model_factory.py tests/test_tracing_factory.py -q`
|
|
Expected: PASS
|
|
|
|
### Task 3: Wire dependency and docs
|
|
|
|
**Files:**
|
|
- Modify: `backend/packages/harness/pyproject.toml`
|
|
- Modify: `README.md`
|
|
- Modify: `backend/README.md`
|
|
|
|
**Step 1: Update dependency**
|
|
|
|
Add `langfuse` to the harness dependencies.
|
|
|
|
**Step 2: Update docs**
|
|
|
|
Document:
|
|
- Langfuse environment variables
|
|
- dual-provider behavior
|
|
- failure behavior for explicitly enabled providers
|
|
|
|
**Step 3: Run targeted verification**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_tracing_config.py tests/test_model_factory.py tests/test_tracing_factory.py -q`
|
|
Expected: PASS
|
|
|
|
### Task 4: Run broader regression checks
|
|
|
|
**Files:**
|
|
- No code changes required
|
|
|
|
**Step 1: Run relevant suite**
|
|
|
|
Run: `cd backend && uv run pytest tests/test_tracing_config.py tests/test_model_factory.py tests/test_tracing_factory.py -q`
|
|
|
|
**Step 2: Run lint if needed**
|
|
|
|
Run: `cd backend && uv run ruff check packages/harness/deerflow/config/tracing_config.py packages/harness/deerflow/models/factory.py packages/harness/deerflow/tracing`
|
|
|
|
**Step 3: Review diff**
|
|
|
|
Run: `git diff -- backend/packages/harness backend/tests README.md backend/README.md`
|