readme update

README.md

@@ -1,6 +1,24 @@
-# 本地 Agent 框架（实训 B 方向）
+# 本地多工具智能 Agent 与语音交互系统（实训 B 方向）
 
-本项目使用 Python 3.10 实现一个本地文件驱动的 Agent 框架。B1–B5 均保留独立命令行入口，使用服务器本地 Qwen3.5-4B。
+本项目使用 Python 3.10 实现一个完整的本地智能 Agent 系统。系统以本地 Qwen3.5-4B 为决策核心，将 Agent 总控、Tools Schema、八类本地 Skill、长期记忆、KV Cache、音频预处理、Faster-Whisper 语音识别和 CosyVoice2 语音合成串联为统一工作流。用户可以输入文本或麦克风语音，Agent 会自主规划并调用工具，最终返回文字回答，也可以通过独立 CosyVoice 子进程生成语音。
+
+> **⚠️ 使用前必须按实际电脑与服务器修改配置**
+>
+> 仓库中的绝对路径、SSH 地址和端口均为团队开发环境示例，不能直接照搬。运行前必须检查 `agent/configs/model.yaml`、`agent/configs/tts.yaml` 和 `agent/configs/voice_chat.yaml`：将模型、tokenizer、CosyVoice Python/仓库/模型/参考音频改为当前电脑或服务器的实际路径；将 `connection.host` 改为本人可访问的 `用户名@服务器地址`，将 `connection.port` 改为目标服务器实际开放的 SSH 端口，并同步修改 `connection.remote_root`、`connection.python` 和 `connection.template`。详细字段见“1.3 首次运行前必须检查的私有路径”。
+
+完整任务的数据流如下：
+
+```text
+文本 / 麦克风音频
+  → 音频预处理与 ASR（仅音频输入）
+  → 加载全局记忆和会话记忆
+  → Qwen3.5-4B 规划任务
+  → B3 校验并调用 B2 Skills
+  → 工具结果返回模型继续推理
+  → 生成最终回答
+  → 保存消息、轨迹、Memory 和报告
+  → CosyVoice2 子进程合成语音（按需启用）
+```
 
 模块边界如下：
 
@@ -12,23 +30,77 @@
 | B4 | `code/b4_local_agent_llm.py` | 使用 mock 或本地 LLM 生成标准 AIMessage，不执行工具。 |
 | B5 | `code/b5_memory.py` | 查找、截断、保存 memory 文档并维护索引。 |
 | 完整演示 | `code/run_full_demo.py` | 调用 B1 跑通完整 Agent，并生成汇总报告。 |
+| 语音服务端 | `code/voice_chat_server.py` | 接收音频、调用 ASR 和 Agent，并维护多轮会话。 |
+| Windows 语音客户端 | `code/voice_chat_windows_client.py` | 麦克风录音、SSH 通信、本地/远程 TTS 和扬声器播放。 |
+| TTS Worker | `skills/text_to_speech.py`、`skills/tts_worker.py` | 从 Agent 环境启动独立 CosyVoice 子进程并生成 WAV。 |
 
 B4的mock模式不真实加载、运行模型，作为无 GPU、无模型或模块联调时的调试模式。`prompt_json`模式则加载本地模型真实运行。
 
 ## 1. 环境准备
 
-所有模块统一使用项目根目录下的 `requirements.txt`。推荐每位同学新建自己的conda环境，安装步骤如下：
+Agent 与 CosyVoice 建议使用两个独立的 Conda 环境。Agent 环境负责本地模型、工具调用、Memory 和语音识别；CosyVoice 环境只负责 TTS。两套环境分离可以避免 PyTorch、Transformers、音频库等依赖版本互相冲突。
+
+### 1.1 Agent 环境
+
+在项目根目录执行：
 
 ```bash
-conda create -n your_env python=3.10 -y
-conda activate your_env
-export PYTHONNOUSERSITE=1
-pip install -r requirements.txt
+conda create -n agent python=3.10 -y
+conda activate agent
+pip install -r agent/requirements.txt
 ```
-其中"export PYTHONNOUSERSITE=1"的作用是：让Python启动时禁止加载用户级site-packages目录，保证只用当前环境自己的包
 
-模型使用Qwen3.5-4B
-统一配置文件为 `configs/model.yaml`，实际使用时需要把model_name_or_path和tokenizer_name_or_path改为模型所在的文件路径。
+本项目默认使用 Qwen3.5-4B。`agent/configs/model.yaml` 中提供的是团队服务器路径，其他使用者必须将 `model.model_name_or_path` 和 `model.tokenizer_name_or_path` 改为自己机器上的模型目录。
+
+### 1.2 独立 CosyVoice TTS 环境
+
+本项目采用子进程方式运行本地 TTS：Agent 环境中的 `agent/skills/text_to_speech.py` 不直接导入 CosyVoice，而是读取 `agent/configs/tts.yaml`，使用 `runtime.python_executable` 指定的 CosyVoice Python 启动 `agent/skills/tts_worker.py`。因此，仅安装 `agent/requirements.txt` 不能运行 TTS，还需要单独准备 CosyVoice 环境。
+
+```bash
+conda create -n cosyvoice python=3.10 -y
+conda activate cosyvoice
+pip install -r CosyVoice/requirements.txt
+```
+
+Linux 如果遇到 SoX 依赖问题，还需要安装系统库：
+
+```bash
+# Ubuntu / Debian
+sudo apt-get install sox libsox-dev
+
+# CentOS
+sudo yum install sox sox-devel
+```
+
+安装后，将 `agent/configs/tts.yaml` 中的 Python 路径改成该环境的真实解释器：
+
+```yaml
+runtime:
+  # Windows 示例
+  python_executable: "C:/Users/<你的用户名>/anaconda3/envs/cosyvoice/python.exe"
+
+  # Linux 示例：使用时保留其中一项即可
+  # python_executable: "/home/<你的用户名>/miniconda3/envs/cosyvoice/bin/python"
+```
+
+还需要确认 `voice.cosyvoice_repo`、`voice.model_dir` 和各音色的 `prompt_wav` 指向真实存在的 CosyVoice 代码、模型及参考音频。当前仓库已经包含 `CosyVoice/` 和 `CosyVoice2-0.5B/`，如果目录结构不变，可以继续使用配置中的相对路径。
+
+### 1.3 首次运行前必须检查的私有路径
+
+仓库配置保留了团队开发机器和服务器的示例路径，不能直接照搬到其他电脑。至少检查以下字段：
+
+| 配置文件 | 字段 | 当前内容的性质 | 应修改为 |
+|---|---|---|---|
+| `agent/configs/model.yaml` | `model_name_or_path`、`tokenizer_name_or_path` | 团队服务器绝对路径 | 本机 Qwen3.5-4B 模型目录 |
+| `agent/configs/tts.yaml` | `runtime.python_executable` | 团队成员 Windows Conda 路径 | 本机 CosyVoice 环境的 Python |
+| `agent/configs/tts.yaml` | `cosyvoice_repo`、`model_dir`、`prompt_wav` | 项目相对路径 | 若目录被移动，改为实际路径 |
+| `agent/configs/voice_chat.yaml` | `connection.host`、`port` | 团队 SSH 服务器地址和端口 | 自己可访问的服务器账号、地址和端口 |
+| `agent/configs/voice_chat.yaml` | `connection.remote_root` | 团队服务器项目路径 | 服务器上的项目根目录 |
+| `agent/configs/voice_chat.yaml` | `connection.python` | 团队服务器 Agent 解释器 | 服务器 Agent 环境的 Python |
+| `agent/configs/voice_chat.yaml` | `connection.template` | 团队项目中的运行模板路径 | 远程项目内实际存在的 runtime JSON 路径 |
+| `agent/configs/voice_chat.yaml` | `audio.sample_rate`、TTS 与 conversation 配置 | 团队演示参数 | 根据本机音频设备、TTS 部署位置和会话需求调整 |
+
+`agent/configs/voice_chat.yaml` 中的 IP、用户名、SSH 端口和远程目录均属于部署示例。运行语音客户端之前必须替换，否则客户端会尝试连接团队原服务器，或因端口、目录和解释器不匹配而启动失败。
 
 演示命令均从 `agent/code` 目录执行：
 
@@ -36,6 +108,90 @@ pip install -r requirements.txt
 cd agent/code
 ```
 
+### 1.4 整体系统运行方法
+
+以下是团队项目最常用的完整运行入口。B2–B5 的独立命令主要用于模块调试，首次使用建议先按本节跑通整体流程。
+
+#### 方式一：完整文本 Agent Demo
+
+确认 Qwen 模型路径和配置后，从 `agent/code` 目录执行：
+
+```bash
+python run_full_demo.py \
+  --input ../data/runtime_input_4.json \
+  --tools_config ../configs/tools.yaml \
+  --memory_config ../configs/memory.yaml \
+  --model_config ../configs/model.yaml \
+  --llm_mode prompt_json \
+  --outdir ../outputs/full_demo
+```
+
+`runtime_input_4.json` 会让真实模型选择 `table_analyzer` 分析示例 CSV。一次完整任务依次经过 B1 Runtime、B5 Memory、B4 本地模型、B3 Tool Layer 和 B2 Skill，最终在 `agent/outputs/full_demo/` 生成回答、消息流、工具结果、执行轨迹和汇总报告。
+
+可以替换 `--input` 运行其他整体任务：
+
+| Runtime 文件 | 整体任务 |
+|---|---|
+| `runtime_input.json` | 读取本地文档并总结 |
+| `runtime_input_2.json` | 调用安全计算工具 |
+| `runtime_input_3.json` | 检索本地资料 |
+| `runtime_input_4.json` | 分析 CSV 表格 |
+| `runtime_input_5.json` | 格式转换并生成文件 |
+| `runtime_input_audio.json` | 音频预处理 → ASR → Agent → 最终回答 |
+| `runtime_input_tts.json` | Agent 完成任务后调用 CosyVoice2 生成 WAV |
+
+若只想检查编排流程而不加载 GPU 模型，可临时使用 `--llm_mode mock`；正式演示应使用 `prompt_json`。
+
+#### 方式二：完整语音对话
+
+语音模式通常采用“Windows 客户端 + GPU 服务器 Agent”的部署方式。先完成以下检查：
+
+1. 修改 `voice_chat.yaml` 中的 SSH 地址、端口、远程项目目录和远程 Agent Python；
+2. 确认远程服务器可以运行 Qwen 和 Faster-Whisper；
+3. 本地 TTS 模式下，确认 `tts.yaml` 指向本机 CosyVoice Python、代码和模型；
+4. 在 `voice_chat.yaml` 的 `tts.voice` 中选择音色：`default` 或 `funingna`；
+5. 根据需要选择 `tts.mode`、`tts.stream_mode` 和 `conversation.mode`。
+
+随后在项目根目录的 Windows PowerShell 中执行：
+
+```powershell
+python .\agent\code\voice_chat_windows_client.py --config .\agent\configs\voice_chat.yaml
+```
+
+客户端会自动完成：麦克风录音 → SSH 发送音频 → 服务器预处理与识别 → Agent 调用工具并回答 → 本地或远程 CosyVoice 合成 → 扬声器播放。
+
+常用的临时模式切换示例：
+
+```powershell
+# 本地流式合成，使用 funingna 音色，连续多轮对话
+python .\agent\code\voice_chat_windows_client.py `
+  --config .\agent\configs\voice_chat.yaml `
+  --tts-mode local `
+  --tts-stream-mode stream `
+  --tts-voice funingna `
+  --conversation-mode append
+
+# 切换为 default 音色和整段合成
+python .\agent\code\voice_chat_windows_client.py `
+  --config .\agent\configs\voice_chat.yaml `
+  --tts-mode local `
+  --tts-stream-mode full `
+  --tts-voice default
+```
+
+#### 运行成功判断
+
+文本完整 Demo 成功时，应满足：
+
+- 进程退出码为 0；
+- `outputs/full_demo/demo_report.md` 中状态为 `success`；
+- `messages.json` 包含 `system → user → assistant → tool → assistant` 消息链；
+- `tool_messages.json` 中目标工具状态为 `success`；
+- `final_answer.md` 包含最终回答；
+- 启用 TTS 时输出目录中存在可播放的 WAV 文件。
+
+语音模式成功时，终端会先返回识别文本和 Agent 回答；启用 TTS 后还会显示音频块进度及实际使用的 `voice` ID，并通过扬声器播放。
+
 ## 2. 公共数据格式
 
 ### 2.1 SkillResult
@@ -275,7 +431,7 @@ python b4_local_agent_llm.py --model_config ../configs/model.yaml --messages ../
 python b4_local_agent_llm.py --model_config ../configs/model.yaml --messages ../data/messages/messages_with_tool.json --tools_schema ../data/messages/tools_schema_basic.json --mode mock --outdir ../outputs/B4_llm
 ```
 
-mock 不加载模型、不占用显存，适合无 GPU、无本地模型或其他模块同学联调，不作为正式基础演示截图。
+mock 不加载模型、不占用显存，适合无 GPU、无本地模型或其他模块同学联调。
 
 ### 6.3 B4 输出
 
@@ -414,6 +570,107 @@ python run_full_demo.py --input ../data/runtime_input.json --tools_config ../con
 system → user → assistant(tool_calls) → tool → assistant(final)
 ```
 
+### 8.4 本地 TTS 与语音交互配置
+
+#### TTS 子进程运行方式
+
+调用 `text_to_speech` 时，Agent 会将请求写入临时 JSON，再用 CosyVoice 环境的 Python 启动 TTS Worker。`agent/configs/tts.yaml` 中：
+
+- `persistent_worker: false`：每次合成启动一次独立子进程，结束后释放模型；占用更容易回收，但每次都要重新加载模型。
+- `persistent_worker: true`：复用常驻子进程和已加载模型，适合连续对话，后续语音生成更快。
+- `load_jit`、`fp16`：控制 CosyVoice 加速和精度选项，需结合运行设备支持情况设置。
+- `min_chunk_chars`、`max_chunk_chars`、`hard_max_chunk_chars`：控制流式 TTS 的文本切块长度。
+- `inter_chunk_silence_sec`：控制合并语音块之间的静音时长。
+
+也可以临时使用环境变量覆盖主要路径，而不修改 YAML：
+
+```powershell
+$env:COSYVOICE_PYTHON="C:\path\to\envs\cosyvoice\python.exe"
+$env:COSYVOICE_REPO="C:\path\to\assignment_B\CosyVoice"
+$env:COSYVOICE_MODEL="C:\path\to\assignment_B\CosyVoice2-0.5B"
+$env:COSYVOICE_PROMPT_WAV="C:\path\to\prompt.wav"
+```
+
+#### 音色切换
+
+项目内置 `default` 和 `funingna` 两个音色配置，定义在 `agent/configs/tts.yaml` 的 `voice.voices` 下。每个音色由参考音频 `prompt_wav` 和对应文本 `prompt_text` 组成，二者应与实际音频内容一致。
+
+修改语音客户端默认音色：
+
+```yaml
+# agent/configs/voice_chat.yaml
+tts:
+  voice: "default"   # 可改为 funingna
+```
+
+也可以在启动时临时覆盖：
+
+```powershell
+python .\agent\code\voice_chat_windows_client.py --config .\agent\configs\voice_chat.yaml --tts-voice default
+```
+
+在 Runtime JSON 中启用最终回答 TTS 时，可通过 `tts_voice` 选择音色：
+
+```json
+{
+  "enable_tts": true,
+  "tts_voice": "funingna"
+}
+```
+
+新增音色时，在 `agent/configs/tts.yaml` 的 `voice.voices` 下增加新的 ID，并配置匹配的 `prompt_wav` 与 `prompt_text`，随后把 `tts_voice` 或 `tts.voice` 改为该 ID。
+
+#### TTS 和会话模式切换
+
+`agent/configs/voice_chat.yaml` 支持以下模式：
+
+| 字段 | 可选值 | 说明 |
+|---|---|---|
+| `tts.mode` | `local` | 在 Windows 本地通过独立 CosyVoice 子进程合成并播放；当前项目默认模式 |
+| `tts.mode` | `remote` | 在远程服务器合成，再将音频结果传回客户端 |
+| `tts.mode` | `none` | 关闭语音合成，只显示识别文本和回答 |
+| `tts.stream_mode` | `stream` | 回答按文本块合成，生成一块就播放一块 |
+| `tts.stream_mode` | `full` | 完整回答合成为一个 WAV 后再播放 |
+| `tts.language` | `auto`、`zh`、`en` | 自动判断、中文零样本或英文跨语言合成 |
+| `conversation.mode` | `append` | 同一服务进程持续追加消息，适合多轮对话与 KV Cache 复用 |
+| `conversation.mode` | `restart` | 每轮重新执行一次独立 Agent Runtime，通过已保存 Memory 衔接上下文 |
+
+这些设置也可以通过命令行覆盖，无需修改配置文件：
+
+```powershell
+# 本地流式 TTS，使用 funingna 音色，并保留连续对话上下文
+python .\agent\code\voice_chat_windows_client.py `
+  --config .\agent\configs\voice_chat.yaml `
+  --tts-mode local `
+  --tts-stream-mode stream `
+  --tts-language auto `
+  --tts-voice funingna `
+  --conversation-mode append
+
+# 关闭 TTS
+python .\agent\code\voice_chat_windows_client.py `
+  --config .\agent\configs\voice_chat.yaml `
+  --tts-mode none
+```
+
+如不想修改 `tts.yaml`，语音客户端还支持 `--local-tts-python`、`--local-cosyvoice-repo`、`--local-tts-model`、`--local-prompt-wav` 和 `--local-prompt-text`，可在启动时覆盖本地 TTS 的环境、模型和参考音频路径。
+
+#### 单独验证本地流式 TTS
+
+建议在运行完整语音对话前先验证 CosyVoice 子进程：
+
+```powershell
+python .\agent\code\stream_tts_demo.py `
+  --text "你好，这是本地语音合成测试。" `
+  --language zh `
+  --tts-voice default `
+  --cosyvoice-python "C:\path\to\envs\cosyvoice\python.exe" `
+  --cosyvoice-repo ".\CosyVoice" `
+  --model-dir ".\CosyVoice2-0.5B"
+```
+
+终端应逐块显示 TTS 事件并播放语音；加入 `--no-play` 可以只生成和检查音频，不调用扬声器。
+
 ## 9. CLI 退出码
 
 | 退出码 | 含义 |
@@ -512,3 +769,4 @@ system → user → assistant(tool_calls) → tool → assistant(final)
 | `llm_calls/llm_call_002_ai_message.json` | `b4_local_agent_llm.py` | 第二次 AIMessage，包含最终回答。 | JSON 对象 |
 | `llm_calls/llm_run_log.jsonl` | `b4_local_agent_llm.py` | 完整演示 LLM 调用日志。 | JSONL |
 | `demo_report.md` | `run_full_demo.py` | 对完整演示状态、数据流、最终回答和文件清单的汇总。 | Markdown |
+

code/b1_agent_runtime.py

@@ -50,6 +50,12 @@ def _validate_runtime_input(payload: dict) -> dict:
         raise ValueError("tts_language must be auto, zh, or en")
     payload["tts_language"] = tts_language.strip().lower()
 
+    tts_voice = payload.get("tts_voice")
+    if tts_voice is not None:
+        if not isinstance(tts_voice, str) or not tts_voice.strip():
+            raise ValueError("tts_voice must be a non-empty string when provided")
+        payload["tts_voice"] = tts_voice.strip()
+
     if input_type == "text":
         if not isinstance(payload.get("user_input"), str) or not payload["user_input"].strip():
             raise ValueError("text input requires a non-empty user_input")
@@ -138,136 +144,135 @@ def _append_system_instruction(messages: list[dict], instruction: str) -> list[d
     raise ValueError("messages must contain a system message")
 
 
-def _extract_asr_transcript(tool_messages: list[dict]) -> str:
-    """Extract the recognized text from B3's speech_to_text ToolMessage."""
+def _append_plan_step_instruction(messages: list[dict], instruction: str) -> list[dict]:
+    """Return a copied message sequence with one transient plan-step suffix."""
+    call_messages = deepcopy(messages)
+    call_messages.append(
+        {
+            "role": "user",
+            "content": "[PLAN_STEP_INSTRUCTION]\n" + instruction.strip(),
+        }
+    )
+    return call_messages
+
+
+def _skill_output(
+    tool_messages: list[dict],
+    expected_name: str,
+    required_fields: tuple[str, ...] = (),
+) -> dict:
+    """Validate one successful ToolMessage and return its SkillResult output."""
     if len(tool_messages) != 1:
-        raise ValueError("audio bootstrap expects exactly one ToolMessage")
+        raise ValueError(f"{expected_name} expects exactly one ToolMessage")
     message = tool_messages[0]
-    if message.get("name") != "speech_to_text":
-        raise ValueError("audio bootstrap received a ToolMessage from an unexpected tool")
+    if message.get("name") != expected_name:
+        raise ValueError(f"expected {expected_name} ToolMessage, got {message.get('name')}")
     if message.get("status") != "success":
-        raise ValueError("speech_to_text did not complete successfully")
+        raise ValueError(f"{expected_name} did not complete successfully")
 
     try:
         result = json.loads(message.get("content", ""))
-    except json.JSONDecodeError as exc:
-        raise ValueError("speech_to_text ToolMessage content is not valid JSON") from exc
+    except (TypeError, json.JSONDecodeError) as exc:
+        raise ValueError(f"{expected_name} ToolMessage content is not valid JSON") from exc
 
-    if result.get("status") != "success":
-        raise ValueError("speech_to_text returned an error SkillResult")
+    if not isinstance(result, dict) or result.get("status") != "success":
+        error = result.get("error") if isinstance(result, dict) else None
+        raise ValueError(f"{expected_name} returned an error SkillResult: {error}")
     output = result.get("output")
     if not isinstance(output, dict):
-        raise ValueError("speech_to_text SkillResult has no output object")
-    transcript = output.get("text")
-    if not isinstance(transcript, str) or not transcript.strip():
-        raise ValueError("speech_to_text SkillResult contains an empty transcript")
-    return transcript.strip()
-
-
-def _extract_processed_audio_path(tool_messages: list[dict]) -> str:
-    """Extract processed_audio_path from audio_preprocess ToolMessage."""
-    if len(tool_messages) != 1:
-        raise ValueError("audio_preprocess expects exactly one ToolMessage")
-
-    message = tool_messages[0]
-    if message.get("name") != "audio_preprocess":
-        raise ValueError("unexpected ToolMessage during audio preprocessing")
-    if message.get("status") != "success":
-        raise ValueError("audio_preprocess did not complete successfully")
-
-    try:
-        result = json.loads(message.get("content", ""))
-    except json.JSONDecodeError as exc:
-        raise ValueError(
-            "audio_preprocess ToolMessage content is not valid JSON"
-        ) from exc
-
-    if result.get("status") != "success":
-        raise ValueError("audio_preprocess returned an error SkillResult")
-
-    output = result.get("output")
-    if not isinstance(output, dict):
-        raise ValueError("audio_preprocess SkillResult has no output object")
-
-    processed_audio_path = output.get("processed_audio_path")
-    if (
-        not isinstance(processed_audio_path, str)
-        or not processed_audio_path.strip()
-    ):
-        raise ValueError(
-            "audio_preprocess SkillResult has no processed_audio_path"
-        )
-
-    return processed_audio_path.strip()
-
-
-def _extract_tts_output(tool_messages: list[dict]) -> dict:
-    """Extract text_to_speech output from one B3 ToolMessage."""
-    if len(tool_messages) != 1:
-        raise ValueError("text_to_speech expects exactly one ToolMessage")
-
-    message = tool_messages[0]
-    if message.get("name") != "text_to_speech":
-        raise ValueError("unexpected ToolMessage during text-to-speech output")
-    if message.get("status") != "success":
-        raise ValueError("text_to_speech did not complete successfully")
-
-    try:
-        result = json.loads(message.get("content", ""))
-    except json.JSONDecodeError as exc:
-        raise ValueError("text_to_speech ToolMessage content is not valid JSON") from exc
-
-    if result.get("status") != "success":
-        error = result.get("error")
-        raise ValueError(f"text_to_speech returned an error SkillResult: {error}")
-
-    output = result.get("output")
-    if not isinstance(output, dict):
-        raise ValueError("text_to_speech SkillResult has no output object")
-
-    audio_path = output.get("audio_path")
-    if not isinstance(audio_path, str) or not audio_path.strip():
-        raise ValueError("text_to_speech SkillResult has no audio_path")
-
+        raise ValueError(f"{expected_name} SkillResult has no output object")
+    missing = [field for field in required_fields if not output.get(field)]
+    if missing:
+        raise ValueError(f"{expected_name} SkillResult missing output: {', '.join(missing)}")
     return output
 
 
-def _is_empty_local_search_result(tool_messages: list[dict]) -> bool:
-    """
-    判断 local_file_search 是否执行成功但没有检索到任何候选文件。
-    """
-    if len(tool_messages) != 1:
-        return False
+def _tool_failures(tool_messages: list[dict]) -> list[dict]:
+    """Return protocol, execution, and unusable-result failures for any skill."""
+    failures = []
+    for message in tool_messages:
+        if not isinstance(message, dict):
+            failures.append(
+                {
+                    "tool_call_id": None,
+                    "tool": "unknown",
+                    "reason": "invalid_tool_message",
+                    "error": "ToolMessage must be an object",
+                }
+            )
+            continue
+        failure = {
+            "tool_call_id": message.get("tool_call_id"),
+            "tool": message.get("name", "unknown"),
+        }
+        try:
+            result = json.loads(message.get("content", ""))
+            if not isinstance(result, dict):
+                raise ValueError("SkillResult must be an object")
+        except (TypeError, json.JSONDecodeError, ValueError) as exc:
+            failures.append({**failure, "reason": "invalid_skill_result", "error": str(exc)})
+            continue
 
-    tool_message = tool_messages[0]
+        if message.get("status") != "success" or result.get("status") != "success":
+            failures.append(
+                {
+                    **failure,
+                    "reason": "skill_error",
+                    "input": result.get("input"),
+                    "error": result.get("error"),
+                }
+            )
+            continue
 
-    if tool_message.get("name") != "local_file_search":
-        return False
-
-    if tool_message.get("status") != "success":
-        return False
-
-    try:
-        skill_result = json.loads(tool_message.get("content", ""))
-    except json.JSONDecodeError:
-        return False
-
-    if skill_result.get("status") != "success":
-        return False
-
-    output = skill_result.get("output") or {}
-    results = output.get("results")
-
-    return isinstance(results, list) and len(results) == 0
+        output = result.get("output")
+        if (
+            message.get("name") == "local_file_search"
+            and isinstance(output, dict)
+            and output.get("results") == []
+        ):
+            failures.append(
+                {
+                    **failure,
+                    "reason": "empty_result",
+                    "input": result.get("input"),
+                    "error": None,
+                }
+            )
+    return failures
 
 
-def _replan_after_empty_search_instruction() -> str:
+def _failure_signature(failure: dict) -> str:
+    """Identify the same failed operation independently of generated call IDs."""
+    failure_input = failure.get("input")
+    if failure.get("reason") == "empty_result" and isinstance(failure_input, dict):
+        # Increasing only the result-window size cannot make an empty search useful.
+        # Treat that as the same operation so the planner cannot loop by escalating top_k.
+        failure_input = {
+            key: value
+            for key, value in failure_input.items()
+            if key not in {"top_k", "limit", "offset"}
+        }
+    return json.dumps(
+        {
+            "tool": failure.get("tool"),
+            "reason": failure.get("reason"),
+            "input": failure_input,
+            "error": failure.get("error"),
+        },
+        ensure_ascii=False,
+        sort_keys=True,
+        separators=(",", ":"),
+        default=str,
+    )
+
+
+def _replan_instruction(failures: list[dict]) -> str:
+    details = json.dumps(failures, ensure_ascii=False, separators=(",", ":"))
     return (
-        "REPLAN: local_file_search returned results=[]. "
-        "The old plan is invalid. "
-        "Do not call file_reader without a returned path. "
-        "Do not repeat the same search. "
-        "Return DIRECT, another valid tool plan, or state that no local file was found."
+        f"REPLAN: one or more tool calls failed or returned an unusable result: {details}. "
+        "The previous decision is invalid. Use the ToolMessages to diagnose the failure. "
+        "Do not repeat an identical failed call unless you correct its arguments or have a clear reason. "
+        "Return DIRECT, a new valid tool plan, or explain that the task cannot be completed."
     )
 
 
@@ -502,6 +507,7 @@ def run_agent(
     planning_llm_calls = 0
     planning_latency_ms = 0.0
     planning_events = []
+    failure_attempts: dict[str, int] = {}
     audio_transcript = None
     audio_bootstrap = []
     output_tool_rounds = 0
@@ -570,9 +576,11 @@ def run_agent(
                     "args": {"audio_path": runtime["audio_path"]},
                 },
             )
-            processed_audio_path = _extract_processed_audio_path(
-                preprocess_messages
-            )
+            processed_audio_path = _skill_output(
+                preprocess_messages,
+                "audio_preprocess",
+                ("processed_audio_path",),
+            )["processed_audio_path"].strip()
 
             asr_args = {"audio_path": processed_audio_path}
             if runtime.get("audio_language"):
@@ -586,7 +594,11 @@ def run_agent(
                     "args": asr_args,
                 },
             )
-            audio_transcript = _extract_asr_transcript(asr_messages)
+            audio_transcript = _skill_output(
+                asr_messages,
+                "speech_to_text",
+                ("text",),
+            )["text"].strip()
 
             messages.append(
                 {
@@ -635,6 +647,7 @@ def run_agent(
             mode,
             str(output_dir / "agent_decision"),
             decision_stem,
+            conversation_id=runtime["conversation_id"],
         )
         decision_latency_ms = round((perf_counter() - planning_started) * 1000, 3)
         planning_latency_ms = round(planning_latency_ms + decision_latency_ms, 3)
@@ -701,7 +714,7 @@ def run_agent(
                             "Do not provide the final user answer yet."
                         )
 
-                    call_messages = _append_system_instruction(
+                    call_messages = _append_plan_step_instruction(
                         messages,
                         step_instruction,
                     )
@@ -713,6 +726,7 @@ def run_agent(
                     mode,
                     str(output_dir / "llm_calls"),
                     f"llm_call_{llm_calls:03d}",
+                    conversation_id=runtime["conversation_id"],
                 )
             if not isinstance(llm_result, dict) or not isinstance(llm_result.get("ai_message"), dict):
                 raise ValueError("B4 result must contain an ai_message object")
@@ -817,27 +831,100 @@ def run_agent(
             )
         tool_rounds += 1
 
-        # 先把真实工具结果写进上下文。
-        # 重新规划时，B4 必须能够看到 local_file_search 返回的 results=[]。
+        # Always expose real tool results to B4 before asking it to replan.
         messages.extend(tool_messages)
         all_tool_messages.extend(tool_messages)
         turn["tool_messages"] = tool_messages
 
-        # 无论本轮来自 Direct 还是 Plan，只要实际执行的是 local_file_search
-        # 且结果为空，旧决策都不能继续使用：后续 file_reader 没有合法 path，
-        # 或普通循环会重复调用 B4 而忽略空检索这一关键信号。
-        if (
-            len(tool_calls) == 1
-            and tool_calls[0].get("name") == "local_file_search"
-            and _is_empty_local_search_result(tool_messages)
-        ):
+        # This is protocol-driven: every current or future skill participates
+        # without a per-skill branch. Empty search remains an unusable outcome.
+        tool_failures = _tool_failures(tool_messages)
+        if tool_failures:
+            repeated_failures = []
+            for failure in tool_failures:
+                signature = _failure_signature(failure)
+                failure_attempts[signature] = failure_attempts.get(signature, 0) + 1
+                if failure_attempts[signature] >= 2:
+                    repeated_failures.append(failure)
+
+            if repeated_failures:
+                failed_tools = ", ".join(
+                    dict.fromkeys(failure["tool"] for failure in repeated_failures)
+                )
+                warnings.append(
+                    f"stopped repeated identical tool failure and forced a direct answer: {failed_tools}"
+                )
+                turn["latency_ms"] = round(
+                    (perf_counter() - turn_start) * 1000,
+                    3,
+                )
+                turns.append(turn)
+
+                direct_messages = _append_system_instruction(
+                    messages,
+                    "FINAL DIRECT ANSWER REQUIRED. Tool use is now disabled because "
+                    "the same operation returned an unusable result twice. Answer the "
+                    "user directly using the conversation and ToolMessages already "
+                    "available. Clearly state any missing information, but do not call "
+                    "a tool and do not mention internal retry, cache, or orchestration details.",
+                )
+                direct_started = perf_counter()
+                llm_calls += 1
+                direct_result = generate_ai_message(
+                    str(model_file),
+                    direct_messages,
+                    [],
+                    mode,
+                    str(output_dir / "llm_calls"),
+                    f"llm_call_{llm_calls:03d}_forced_direct",
+                    conversation_id=runtime["conversation_id"],
+                )
+                direct_ai_message = direct_result.get("ai_message", {})
+                direct_tool_calls = direct_ai_message.get("tool_calls", [])
+                direct_content = direct_ai_message.get("content", "")
+                direct_turn = {
+                    "turn_index": llm_calls,
+                    "ai_message": direct_ai_message,
+                    "llm_status": direct_result.get("status"),
+                    "llm_error": direct_result.get("error"),
+                    "tool_messages": [],
+                    "latency_ms": round((perf_counter() - direct_started) * 1000, 3),
+                    "forced_direct_after_repeated_failure": True,
+                }
+                turns.append(direct_turn)
+
+                if (
+                    direct_result.get("status") == "success"
+                    and isinstance(direct_content, str)
+                    and direct_content.strip()
+                    and not direct_tool_calls
+                ):
+                    messages.append(direct_ai_message)
+                    final_answer = direct_content.strip()
+                    print(f"content: {final_answer}")
+                else:
+                    final_answer = (
+                        "I could not find enough information to complete the request. "
+                        "Please provide more specific details or a different source."
+                    )
+                    status = "repeated_tool_failure"
+                    terminal_error = {
+                        "type": "RepeatedToolFailure",
+                        "message": final_answer,
+                        "failures": repeated_failures,
+                        "forced_direct_error": direct_result.get("error"),
+                        "forced_direct_tool_calls": direct_tool_calls,
+                    }
+                break
+
+            failed_tools = ", ".join(failure["tool"] for failure in tool_failures)
             warnings.append(
-                "local_file_search returned no results; previous Agent decision was invalidated and replanning started"
+                f"tool result invalidated the previous Agent decision; replanning started: {failed_tools}"
             )
 
             messages = _append_system_instruction(
                 messages,
-                _replan_after_empty_search_instruction(),
+                _replan_instruction(tool_failures),
             )
 
             planning_started = perf_counter()
@@ -849,7 +936,8 @@ def run_agent(
                 agent_tools_schema,
                 mode,
                 str(output_dir / "agent_decision"),
-                f"replan_after_empty_search_turn_{llm_calls:03d}",
+                f"replan_after_tool_failure_turn_{llm_calls:03d}",
+                conversation_id=runtime["conversation_id"],
             )
 
             decision_latency_ms = round(
@@ -864,15 +952,16 @@ def run_agent(
 
             planning_events.append(
                 {
-                    "stage": "replan_after_empty_local_search",
+                    "stage": "replan_after_tool_failure",
                     "latency_ms": decision_latency_ms,
-                    "trigger_tool": "local_file_search",
+                    "trigger_tools": [failure["tool"] for failure in tool_failures],
+                    "failures": tool_failures,
                 }
             )
 
             write_json(
                 decision,
-                output_dir / f"replan_after_empty_search_turn_{llm_calls:03d}.json",
+                output_dir / f"replan_after_tool_failure_turn_{llm_calls:03d}.json",
             )
 
             interpreted = _interpret_agent_decision(decision)
@@ -934,6 +1023,8 @@ def run_agent(
                     "output_filename": "final_answer.wav",
                 },
             }
+            if runtime.get("tts_voice"):
+                tts_call["args"]["tts_voice"] = runtime["tts_voice"]
             tts_started = perf_counter()
             try:
                 tts_tool_messages = execute_tool_calls(
@@ -944,7 +1035,11 @@ def run_agent(
                 )
                 output_tool_rounds += 1
                 all_tool_messages.extend(tts_tool_messages)
-                tts_result = _extract_tts_output(tts_tool_messages)
+                tts_result = _skill_output(
+                    tts_tool_messages,
+                    "text_to_speech",
+                    ("audio_path",),
+                )
                 tts_output = {
                     "enabled": True,
                     "status": "success",
@@ -987,6 +1082,7 @@ def run_agent(
         ),
         "audio_bootstrap": audio_bootstrap if input_type == "audio" else [],
         "tts_output": tts_output,
+        "tts_voice": runtime.get("tts_voice"),
         "status": status,
         "toolset": runtime["toolset"],
         "max_turns": runtime["max_turns"],
@@ -1283,4 +1379,4 @@ def main(argv: list[str] | None = None) -> int:
 
 
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(main())

code/b3_tool_layer.py

@@ -264,10 +264,10 @@ def execute_tool_calls(
                     output = function(**kwargs)
                     latency_ms = round((perf_counter() - start) * 1000, 3)
                     result = make_skill_result(name, "success", args, output, None, latency_ms)
-                except (ImportError, AttributeError) as exc:
-                    raise RuntimeError(f"cannot load configured tool {name}: {exc}") from exc
                 except Exception as exc:
                     latency_ms = round((perf_counter() - start) * 1000, 3)
+                    if isinstance(exc, (ImportError, AttributeError)):
+                        exc = RuntimeError(f"cannot load configured tool {name}: {exc}")
                     result = _error_result(name, args, exc, latency_ms)
         content = json.dumps(result, ensure_ascii=False, separators=(",", ":"))
         message = make_tool_message(call["id"], call["name"], content, result["status"])

code/b4_schema_injection_benchmark.py

@@ -0,0 +1,308 @@
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import statistics
+import sys
+from copy import deepcopy
+from pathlib import Path
+from time import perf_counter
+
+from b4_local_agent_llm import generate_ai_message
+from common.io_utils import read_json, read_text, read_yaml, write_json, write_text
+from common.path_utils import resolve_cli_path, resolve_from_file
+from common.schemas import make_skill_result, make_tool_message
+
+
+INJECTION_MODES = ("prompt", "native")
+
+
+def _write_model_variant(source: Path, injection: str, target: Path) -> None:
+    try:
+        import yaml
+    except ImportError as exc:
+        raise RuntimeError("PyYAML is required; install requirements.txt") from exc
+
+    config = deepcopy(read_yaml(source))
+    model = config.setdefault("model", {})
+    for key in ("model_name_or_path", "tokenizer_name_or_path"):
+        value = model.get(key)
+        if isinstance(value, str) and not Path(value).expanduser().is_absolute():
+            model[key] = str(resolve_from_file(value, source))
+    config.setdefault("tool_calling", {})["schema_injection"] = injection
+    write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False), target)
+
+
+def _required_by_tool(tools_schema: list[dict]) -> dict[str, list[str]]:
+    required = {}
+    for entry in tools_schema:
+        function = entry.get("function") if isinstance(entry, dict) else None
+        if not isinstance(function, dict) or not isinstance(function.get("name"), str):
+            continue
+        parameters = function.get("parameters") or {}
+        required[function["name"]] = list(parameters.get("required") or [])
+    return required
+
+
+def _tool_message(call: dict, status: str, output=None, error=None) -> dict:
+    name = call.get("name", "unknown")
+    args = call.get("args") if isinstance(call.get("args"), dict) else {}
+    result = make_skill_result(name, status, args, output, error, 0.0)
+    return make_tool_message(
+        call.get("id") or "benchmark_call",
+        name,
+        json.dumps(result, ensure_ascii=False, separators=(",", ":")),
+        status,
+    )
+
+
+def _run_case(
+    task: dict,
+    injection: str,
+    repeat: int,
+    model_config: Path,
+    tools_schema: list[dict],
+    system_prompt: str,
+    required_by_tool: dict[str, list[str]],
+    max_llm_calls: int,
+    llm_mode: str,
+    output_dir: Path,
+) -> dict:
+    messages = [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": task["user_input"]},
+    ]
+    expected = task["expected_tools"]
+    outputs = task["tool_outputs"]
+    actual_calls = []
+    required_present = required_total = 0
+    expected_index = 0
+    reasoning_latency_ms = 0.0
+    status = "max_llm_calls"
+    final_answer = ""
+
+    for llm_call in range(1, max_llm_calls + 1):
+        started = perf_counter()
+        result = generate_ai_message(
+            str(model_config),
+            messages,
+            tools_schema,
+            llm_mode,
+            str(output_dir / "llm_calls"),
+            f"call_{llm_call:03d}",
+        )
+        reasoning_latency_ms += (perf_counter() - started) * 1000
+        ai_message = result["ai_message"]
+        messages.append(ai_message)
+        if result.get("status") != "success":
+            status = "llm_error"
+            break
+
+        calls = ai_message.get("tool_calls", [])
+        if not calls:
+            final_answer = ai_message.get("content", "")
+            status = "success"
+            break
+
+        tool_messages = []
+        for call in calls:
+            actual_calls.append(call)
+            name = call.get("name", "unknown")
+            args = call.get("args") if isinstance(call.get("args"), dict) else {}
+            required = required_by_tool.get(name, [])
+            present = sum(key in args and args[key] is not None for key in required)
+            required_present += present
+            required_total += len(required)
+
+            expected_name = expected[expected_index] if expected_index < len(expected) else None
+            missing = [key for key in required if key not in args or args[key] is None]
+            if name != expected_name:
+                tool_messages.append(
+                    _tool_message(
+                        call,
+                        "error",
+                        error={"type": "UnexpectedTool", "message": f"expected {expected_name}, got {name}"},
+                    )
+                )
+            elif missing:
+                tool_messages.append(
+                    _tool_message(
+                        call,
+                        "error",
+                        error={"type": "MissingArguments", "message": ", ".join(missing)},
+                    )
+                )
+            else:
+                tool_messages.append(_tool_message(call, "success", outputs[expected_index], None))
+                expected_index += 1
+        messages.extend(tool_messages)
+
+    actual_names = [call.get("name", "unknown") for call in actual_calls]
+    write_json(messages, output_dir / "messages.json")
+    return {
+        "schema_injection": injection,
+        "task_id": task["id"],
+        "repeat": repeat,
+        "status": status,
+        "expected_tools": json.dumps(expected, ensure_ascii=False),
+        "actual_tools": json.dumps(actual_names, ensure_ascii=False),
+        "tool_call_correct": int(actual_names == expected),
+        "required_args_present": required_present,
+        "required_args_total": required_total,
+        "tool_call_rounds": sum(
+            message.get("role") == "assistant" and bool(message.get("tool_calls"))
+            for message in messages
+        ),
+        "llm_calls": sum(message.get("role") == "assistant" for message in messages),
+        "reasoning_latency_ms": round(reasoning_latency_ms, 3),
+        "final_answer": final_answer,
+    }
+
+
+def _mean(rows: list[dict], key: str) -> float:
+    return round(statistics.fmean(float(row[key]) for row in rows), 3) if rows else 0.0
+
+
+def _summarize(rows: list[dict]) -> list[dict]:
+    summary = []
+    for injection in INJECTION_MODES:
+        selected = [row for row in rows if row["schema_injection"] == injection]
+        required_total = sum(row["required_args_total"] for row in selected)
+        latencies = [row["reasoning_latency_ms"] for row in selected]
+        summary.append(
+            {
+                "schema_injection": injection,
+                "runs": len(selected),
+                "success_rate": round(sum(row["status"] == "success" for row in selected) / len(selected), 4),
+                "tool_call_accuracy": round(sum(row["tool_call_correct"] for row in selected) / len(selected), 4),
+                "parameter_completeness": round(
+                    sum(row["required_args_present"] for row in selected) / required_total, 4
+                ) if required_total else 1.0,
+                "avg_tool_call_rounds": _mean(selected, "tool_call_rounds"),
+                "avg_llm_calls": _mean(selected, "llm_calls"),
+                "avg_reasoning_latency_ms": _mean(selected, "reasoning_latency_ms"),
+                "median_reasoning_latency_ms": round(statistics.median(latencies), 3),
+            }
+        )
+    return summary
+
+
+def _pct(value: float) -> str:
+    return f"{value * 100:.2f}%"
+
+
+def _report(summary: list[dict], run_count: int, repeats: int) -> str:
+    indexed = {row["schema_injection"]: row for row in summary}
+    prompt, native = indexed["prompt"], indexed["native"]
+    lines = [
+        "# B4 Tools Schema Injection Benchmark",
+        "",
+        f"- Repeats per task and injection mode: `{repeats}`",
+        f"- Measured B4 runs: `{run_count}` (warm-up excluded)",
+        "- Tool-call accuracy: exact match of the complete emitted tool-name sequence.",
+        "- Parameter completeness: non-null required arguments / required arguments in emitted calls.",
+        "- Tool-call rounds: B4 responses containing at least one tool call.",
+        "- Reasoning latency: cumulative wall time of B4 `generate_ai_message`; canned tools take no measured time.",
+        "",
+        "| Injection | Success | Tool accuracy | Parameter completeness | Avg tool rounds | Avg LLM calls | Avg latency ms | Median latency ms |",
+        "|---|---:|---:|---:|---:|---:|---:|---:|",
+    ]
+    for row in summary:
+        lines.append(
+            f"| {row['schema_injection']} | {_pct(row['success_rate'])} | "
+            f"{_pct(row['tool_call_accuracy'])} | {_pct(row['parameter_completeness'])} | "
+            f"{row['avg_tool_call_rounds']:.3f} | {row['avg_llm_calls']:.3f} | "
+            f"{row['avg_reasoning_latency_ms']:.3f} | {row['median_reasoning_latency_ms']:.3f} |"
+        )
+    lines.extend(
+        [
+            "",
+            "## Native minus prompt",
+            "",
+            f"- Tool-call accuracy: `{(native['tool_call_accuracy'] - prompt['tool_call_accuracy']) * 100:+.2f}` percentage points",
+            f"- Parameter completeness: `{(native['parameter_completeness'] - prompt['parameter_completeness']) * 100:+.2f}` percentage points",
+            f"- Average tool-call rounds: `{native['avg_tool_call_rounds'] - prompt['avg_tool_call_rounds']:+.3f}`",
+            f"- Average reasoning latency: `{native['avg_reasoning_latency_ms'] - prompt['avg_reasoning_latency_ms']:+.3f} ms`",
+            "",
+            "Positive accuracy/completeness deltas favor native injection; negative rounds/latency deltas favor native injection.",
+            "",
+        ]
+    )
+    return "\n".join(lines)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="B4-only prompt vs native tools-schema benchmark.")
+    parser.add_argument("--benchmark", required=True)
+    parser.add_argument("--model_config", required=True)
+    parser.add_argument("--outdir", required=True)
+    parser.add_argument("--repeats", type=int, default=None)
+    parser.add_argument("--llm_mode", choices=["mock", "prompt_json"], default="prompt_json")
+    parser.add_argument("--skip_warmup", action="store_true")
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    benchmark_path = resolve_cli_path(args.benchmark)
+    model_path = resolve_cli_path(args.model_config)
+    outdir = resolve_cli_path(args.outdir)
+    benchmark = read_json(benchmark_path)
+    tasks = benchmark.get("tasks")
+    repeats = args.repeats if args.repeats is not None else benchmark.get("repeats", 3)
+    max_llm_calls = benchmark.get("max_llm_calls", 4)
+    if not isinstance(tasks, list) or not tasks:
+        raise ValueError("benchmark tasks must be a non-empty array")
+    if not isinstance(repeats, int) or repeats < 1:
+        raise ValueError("repeats must be a positive integer")
+
+    system_prompt = read_text(resolve_from_file(benchmark["system_prompt_path"], benchmark_path)).strip()
+    tools_schema = read_json(resolve_from_file(benchmark["tools_schema_path"], benchmark_path))
+    required = _required_by_tool(tools_schema)
+    variants = {}
+    for injection in INJECTION_MODES:
+        variant = outdir / "configs" / f"model_{injection}.yaml"
+        _write_model_variant(model_path, injection, variant)
+        variants[injection] = variant
+
+    if not args.skip_warmup:
+        for injection in INJECTION_MODES:
+            print(f"warmup injection={injection}", flush=True)
+            _run_case(
+                tasks[0], injection, 0, variants[injection], tools_schema,
+                system_prompt, required, max_llm_calls, args.llm_mode,
+                outdir / "warmup" / injection,
+            )
+
+    rows = []
+    for repeat in range(1, repeats + 1):
+        for task in tasks:
+            for injection in INJECTION_MODES:
+                print(f"run injection={injection} task={task['id']} repeat={repeat}", flush=True)
+                row = _run_case(
+                    task, injection, repeat, variants[injection], tools_schema,
+                    system_prompt, required, max_llm_calls, args.llm_mode,
+                    outdir / "runs" / injection / task["id"] / f"repeat_{repeat:03d}",
+                )
+                rows.append(row)
+                write_json(rows, outdir / "run_results.json")
+
+    summary = _summarize(rows)
+    write_json(summary, outdir / "summary.json")
+    columns = list(rows[0])
+    with (outdir / "run_results.csv").open("w", encoding="utf-8-sig", newline="") as handle:
+        writer = csv.DictWriter(handle, fieldnames=columns)
+        writer.writeheader()
+        writer.writerows(rows)
+    write_text(_report(summary, len(rows), repeats), outdir / "comparison_report.md")
+    print(outdir / "comparison_report.md")
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        raise SystemExit(main())
+    except Exception as exc:
+        print(f"fatal: {type(exc).__name__}: {exc}", file=sys.stderr)
+        raise SystemExit(1)

code/benchmark_conversation_kv_cache.py

@@ -0,0 +1,262 @@
+from __future__ import annotations
+
+import argparse
+import json
+import tempfile
+from copy import deepcopy
+from pathlib import Path
+from time import perf_counter
+from typing import Any
+
+import yaml
+
+from b4_local_agent_llm import _clear_kv_cache, generate_ai_message
+from common.io_utils import ensure_dir, read_json, read_yaml, write_json
+from common.path_utils import resolve_cli_path
+
+
+def _write_temp_model_config(
+    base_config: dict,
+    enabled: bool,
+    max_new_tokens: int | None,
+    directory: Path,
+) -> Path:
+    config = deepcopy(base_config)
+    kv_cache = config.setdefault("kv_cache", {})
+    if not isinstance(kv_cache, dict):
+        kv_cache = {}
+        config["kv_cache"] = kv_cache
+    kv_cache["enabled"] = enabled
+    if max_new_tokens is not None:
+        generation = config.setdefault("generation", {})
+        if not isinstance(generation, dict):
+            generation = {}
+            config["generation"] = generation
+        generation["max_new_tokens"] = max_new_tokens
+
+    handle = tempfile.NamedTemporaryFile(
+        "w",
+        encoding="utf-8",
+        suffix=".yaml",
+        dir=directory,
+        delete=False,
+    )
+    with handle:
+        yaml.safe_dump(config, handle, allow_unicode=True, sort_keys=False)
+    return Path(handle.name)
+
+
+def _long_context(repeat: int) -> str:
+    sentence = (
+        "Benchmark context paragraph: KV cache reuse matters when the same conversation "
+        "history is repeatedly prefetched across turns; append-only messages allow safe "
+        "token-prefix validation and reuse of past_key_values. "
+    )
+    return sentence * max(0, repeat)
+
+
+def _conversation_turns(count: int, payload_repeat: int) -> list[str]:
+    topics = [
+        "KV cache reuse in multi-turn conversations",
+        "why append-only message history matters",
+        "how CPU offload protects GPU memory",
+        "how token-level prefix checks keep cache reuse safe",
+        "how benchmark logs prove cache effectiveness",
+    ]
+    payload = (
+        " Extra turn payload for stable prefill measurement: "
+        "conversation history grows while the required output stays constant. "
+    ) * max(0, payload_repeat)
+    turns = []
+    for index in range(count):
+        topic = topics[index % len(topics)]
+        turns.append(
+            f"This is conversation turn {index + 1}. "
+            f"Topic marker: {topic}. "
+            f"{payload}"
+            'Return exactly this JSON object and nothing else: {"content":"OK","tool_calls":[]}'
+        )
+    return turns
+
+
+def _run_variant(
+    *,
+    label: str,
+    model_config: Path,
+    tools_schema: list[dict],
+    outdir: Path,
+    conversation_id: str,
+    turns: list[str],
+    context_repeat: int,
+) -> dict[str, Any]:
+    _clear_kv_cache()
+    messages = [
+        {
+            "role": "system",
+            "content": (
+                "You are a deterministic benchmark assistant. Never call tools. "
+                'For every user turn, return exactly {"content":"OK","tool_calls":[]} '
+                "with no extra text.\n\n"
+                + _long_context(context_repeat)
+            ),
+        }
+    ]
+    rows = []
+    started = perf_counter()
+
+    for index, user_content in enumerate(turns, start=1):
+        messages.append({"role": "user", "content": user_content})
+        turn_started = perf_counter()
+        result = generate_ai_message(
+            str(model_config),
+            messages,
+            tools_schema,
+            mode="prompt_json",
+            artifact_dir=str(outdir / label),
+            artifact_stem=f"turn_{index:03d}",
+            conversation_id=conversation_id,
+        )
+        latency_ms = round((perf_counter() - turn_started) * 1000, 3)
+        ai_message = result["ai_message"]
+        messages.append(ai_message)
+
+        raw_path = outdir / label / f"turn_{index:03d}_raw_model_output.json"
+        raw_record = read_json(raw_path)
+        kv_cache = raw_record.get("kv_cache") or {}
+        raw_text = raw_record.get("raw_text", "")
+        rows.append(
+            {
+                "turn": index,
+                "latency_ms": latency_ms,
+                "status": result["status"],
+                "kv_event": kv_cache.get("event"),
+                "kv_reason": kv_cache.get("reason"),
+                "cache_tier": kv_cache.get("cache_tier"),
+                "reused_token_count": kv_cache.get("reused_token_count", 0),
+                "new_prefill_token_count": kv_cache.get("new_prefill_token_count", 0),
+                "current_token_count": kv_cache.get("current_token_count", 0),
+                "content_chars": len(ai_message.get("content", "")),
+                "raw_output_chars": len(raw_text),
+            }
+        )
+
+    total_ms = round((perf_counter() - started) * 1000, 3)
+    raw_lengths = [int(row["raw_output_chars"] or 0) for row in rows]
+    return {
+        "label": label,
+        "conversation_id": conversation_id,
+        "turn_count": len(turns),
+        "total_ms": total_ms,
+        "avg_ms": round(total_ms / max(1, len(turns)), 3),
+        "hit_count": sum(1 for row in rows if row["kv_event"] == "hit"),
+        "reused_token_count": sum(int(row["reused_token_count"] or 0) for row in rows),
+        "new_prefill_token_count": sum(int(row["new_prefill_token_count"] or 0) for row in rows),
+        "raw_output_chars_min": min(raw_lengths) if raw_lengths else 0,
+        "raw_output_chars_max": max(raw_lengths) if raw_lengths else 0,
+        "rows": rows,
+    }
+
+
+def _print_summary(summary: dict[str, Any]) -> None:
+    print(json.dumps(summary, ensure_ascii=False, indent=2))
+    baseline = summary["disabled"]["total_ms"]
+    optimized = summary["enabled"]["total_ms"]
+    if optimized > 0:
+        speedup = round(baseline / optimized, 3)
+        saved_ms = round(baseline - optimized, 3)
+        print(f"kv_cache_speedup={speedup}x saved_ms={saved_ms}")
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=(
+            "Benchmark conversation-level KV cache reuse with long shared context "
+            "and short fixed outputs."
+        )
+    )
+    parser.add_argument("--model_config", required=True)
+    parser.add_argument("--tools_schema")
+    parser.add_argument("--outdir", required=True)
+    parser.add_argument("--turns", type=int, default=5)
+    parser.add_argument("--max_new_tokens", type=int, default=12)
+    parser.add_argument("--conversation_id", default="bench_conversation_kv")
+    parser.add_argument("--context_repeat", type=int, default=80, help="Repeat count for long system context.")
+    parser.add_argument("--turn_payload_repeat", type=int, default=8, help="Repeat count for per-turn payload.")
+    args = parser.parse_args()
+
+    model_config_path = resolve_cli_path(args.model_config)
+    outdir = ensure_dir(resolve_cli_path(args.outdir))
+
+    base_config = read_yaml(model_config_path)
+    if not isinstance(base_config, dict):
+        raise ValueError("model config must be an object")
+    tools_schema_path = resolve_cli_path(args.tools_schema) if args.tools_schema else None
+    tools_schema = read_json(tools_schema_path) if tools_schema_path else []
+    if not isinstance(tools_schema, list):
+        raise ValueError("tools schema must be a list")
+
+    enabled_config = _write_temp_model_config(
+        base_config,
+        True,
+        args.max_new_tokens,
+        model_config_path.parent,
+    )
+    disabled_config = _write_temp_model_config(
+        base_config,
+        False,
+        args.max_new_tokens,
+        model_config_path.parent,
+    )
+    turns = _conversation_turns(args.turns, args.turn_payload_repeat)
+
+    # Warm up model loading outside the measured variants.
+    warmup_messages = [
+        {"role": "system", "content": "Warm up."},
+        {"role": "user", "content": "Answer with one short sentence. Do not call tools."},
+    ]
+    generate_ai_message(
+        str(disabled_config),
+        warmup_messages,
+        [],
+        mode="prompt_json",
+        artifact_dir=str(outdir / "warmup"),
+        artifact_stem="warmup",
+        conversation_id=f"{args.conversation_id}_warmup",
+    )
+
+    disabled = _run_variant(
+        label="disabled",
+        model_config=disabled_config,
+        tools_schema=tools_schema,
+        outdir=outdir,
+        conversation_id=f"{args.conversation_id}_disabled",
+        turns=turns,
+        context_repeat=args.context_repeat,
+    )
+    enabled = _run_variant(
+        label="enabled",
+        model_config=enabled_config,
+        tools_schema=tools_schema,
+        outdir=outdir,
+        conversation_id=f"{args.conversation_id}_enabled",
+        turns=turns,
+        context_repeat=args.context_repeat,
+    )
+
+    summary = {
+        "model_config": str(model_config_path),
+        "tools_schema": str(tools_schema_path) if tools_schema_path else None,
+        "turns": args.turns,
+        "max_new_tokens": args.max_new_tokens,
+        "context_repeat": args.context_repeat,
+        "turn_payload_repeat": args.turn_payload_repeat,
+        "disabled": disabled,
+        "enabled": enabled,
+    }
+    write_json(summary, outdir / "benchmark_summary.json")
+    _print_summary(summary)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

code/run_full_demo.py

@@ -24,6 +24,7 @@ def _build_report(result: dict, outdir: Path) -> str:
             f"- Status: `{tts_output.get('status')}`\n"
             f"- Audio path: `{tts_output.get('audio_path')}`\n"
             f"- Language: `{tts_output.get('language')}`\n"
+            f"- Voice: `{tts_output.get('voice_id') or (tts_output.get('voice') or {}).get('id')}`\n"
             f"- Duration: `{tts_output.get('duration_sec')}` seconds\n\n"
         )
     return (

code/stream_tts_demo.py

@@ -0,0 +1,95 @@
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+AGENT_ROOT = PROJECT_ROOT / "agent"
+if str(AGENT_ROOT) not in sys.path:
+    sys.path.insert(0, str(AGENT_ROOT))
+if str(AGENT_ROOT / "code") not in sys.path:
+    sys.path.insert(0, str(AGENT_ROOT / "code"))
+
+from skills.text_to_speech import shutdown_tts_worker, stream_text_to_speech
+
+
+DEFAULT_TEXT = (
+    "你好，这是第一阶段流式语音测试。"
+    "现在系统会按短句逐段合成，并在每一段完成后立刻返回音频。"
+    "后续再接入模型文本流。"
+)
+
+
+def play_wav(path: Path) -> None:
+    try:
+        import sounddevice as sd
+        import soundfile as sf
+    except ImportError as exc:
+        raise RuntimeError("Please install sounddevice and soundfile in the agent environment.") from exc
+    audio, rate = sf.read(str(path), dtype="float32")
+    sd.play(audio, rate)
+    sd.wait()
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="Stream CosyVoice chunks and play each chunk immediately.")
+    parser.add_argument("--text", default=DEFAULT_TEXT)
+    parser.add_argument("--language", choices=["auto", "zh", "en"], default="zh")
+    parser.add_argument("--output-dir", default=str(PROJECT_ROOT / "agent" / "outputs" / "stream_tts_demo"))
+    parser.add_argument("--output-filename", default="stream_demo.wav")
+    parser.add_argument("--no-play", action="store_true", help="Only print chunk events; do not play audio.")
+    parser.add_argument("--cosyvoice-python", default=None, help="Optional CosyVoice Python executable override.")
+    parser.add_argument("--cosyvoice-repo", default=None, help="Optional CosyVoice repo path override.")
+    parser.add_argument("--model-dir", default=None, help="Optional CosyVoice model directory override.")
+    parser.add_argument("--tts-voice", default=None, help="Optional voice profile id from agent/configs/tts.yaml.")
+    parser.add_argument("--prompt-wav", default=None, help="Optional prompt wav override.")
+    parser.add_argument("--prompt-text", default=None, help="Optional prompt text override.")
+    return parser
+
+
+def main() -> int:
+    args = build_parser().parse_args()
+    if args.cosyvoice_python:
+        os.environ["COSYVOICE_PYTHON"] = str(Path(args.cosyvoice_python).resolve())
+    if args.cosyvoice_repo:
+        os.environ["COSYVOICE_REPO"] = str(Path(args.cosyvoice_repo).resolve())
+    if args.model_dir:
+        os.environ["COSYVOICE_MODEL"] = str(Path(args.model_dir).resolve())
+    if args.prompt_wav:
+        os.environ["COSYVOICE_PROMPT_WAV"] = str(Path(args.prompt_wav).resolve())
+    if args.prompt_text:
+        os.environ["COSYVOICE_PROMPT_TEXT"] = args.prompt_text
+    if args.tts_voice:
+        os.environ["COSYVOICE_VOICE"] = args.tts_voice
+
+    events = []
+    try:
+        for event in stream_text_to_speech(
+            args.text,
+            language=args.language,
+            output_filename=args.output_filename,
+            tts_voice=args.tts_voice,
+            output_dir=args.output_dir,
+        ):
+            events.append(event)
+            chunk_path = Path(event["audio_path"])
+            print(
+                f"chunk {event.get('index')}/{event.get('chunk_count')} "
+                f"synth={event.get('latency_ms')}ms audio={event.get('duration_sec')}s "
+                f"path={chunk_path}",
+                flush=True,
+            )
+            if not args.no_play:
+                play_wav(chunk_path)
+    finally:
+        shutdown_tts_worker()
+
+    print(f"EVENT_COUNT {len(events)}", flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

code/voice_chat_server.py

@@ -0,0 +1,666 @@
+from __future__ import annotations
+
+import argparse
+import base64
+import json
+import os
+import sys
+import uuid
+from copy import deepcopy
+from pathlib import Path
+from time import perf_counter
+
+from b1_agent_runtime import (
+    _append_plan_step_instruction,
+    _append_system_instruction,
+    _default_llm_mode,
+    _interpret_agent_decision,
+    _replan_instruction,
+    _skill_output,
+    _tool_failures,
+    generate_agent_decision,
+    generate_ai_message,
+    run_agent,
+)
+from common.io_utils import read_json, read_text, write_json, write_text
+from common.path_utils import PROJECT_ROOT, bootstrap_project_root, resolve_from_file
+
+bootstrap_project_root()
+
+from skills.text_to_speech import shutdown_tts_worker, stream_text_to_speech, text_to_speech
+
+
+RESPONSE_PREFIX = "__VOICE_CHAT_RESPONSE__"
+VOICE_RESPONSE_INSTRUCTION = """
+当前是中文语音对话模式，最终回答将由语音合成系统朗读。
+
+请使用自然、直接、适合听觉理解的中文口语回答。先说结论，再用少量完整句子解释。不要使用 Markdown 标题、项目符号、编号列表、表格、引用、代码块、链接或加粗标记，也不要把回答写成报告或文档。需要表达多个要点时，使用“首先、另外、最后”等自然衔接词。除非用户明确要求，否则不要朗读网址、文件路径、JSON、工具参数或内部实现细节，并尽量保持简洁。
+""".strip()
+
+
+def _voice_system_prompt(base_prompt: str) -> str:
+    return base_prompt.rstrip() + "\n\n" + VOICE_RESPONSE_INSTRUCTION + "\n"
+
+
+def _response(payload: dict) -> None:
+    encoded = base64.b64encode(json.dumps(payload, ensure_ascii=False).encode("utf-8")).decode("ascii")
+    print(RESPONSE_PREFIX + encoded, flush=True)
+
+
+def _save_audio(message: dict, session_dir: Path, turn: int) -> Path:
+    if isinstance(message.get("audio_path"), str):
+        path = Path(message["audio_path"]).expanduser().resolve()
+        if not path.is_file():
+            raise ValueError(f"audio file not found: {path}")
+        return path
+    encoded = message.get("audio_b64")
+    if not isinstance(encoded, str) or not encoded:
+        raise ValueError("request requires audio_b64 or audio_path")
+    suffix = str(message.get("suffix", ".wav")).lower()
+    if suffix not in {".wav", ".flac", ".mp3", ".m4a", ".ogg", ".webm"}:
+        raise ValueError(f"unsupported audio suffix: {suffix}")
+    path = session_dir / f"input_{turn:03d}{suffix}"
+    path.write_bytes(base64.b64decode(encoded, validate=True))
+    return path
+
+
+def _trace_transcript(trace: dict) -> str | None:
+    input_audio = trace.get("input_audio")
+    if isinstance(input_audio, dict) and isinstance(input_audio.get("transcript"), str):
+        return input_audio["transcript"]
+    input_payload = trace.get("input")
+    if isinstance(input_payload, dict) and isinstance(input_payload.get("transcript"), str):
+        return input_payload["transcript"]
+    return None
+
+
+def _agent_tools_schema(tools_schema: list[dict]) -> list[dict]:
+    return [
+        tool
+        for tool in tools_schema
+        if not (
+            isinstance(tool, dict)
+            and isinstance(tool.get("function"), dict)
+            and tool["function"].get("name") in {"audio_preprocess", "speech_to_text"}
+        )
+    ]
+
+
+def _run_forced_tool(
+    *,
+    tool_call: dict,
+    tools_config: str,
+    toolset: str,
+    output_dir: Path,
+) -> tuple[dict, list[dict]]:
+    from b3_tool_layer import execute_tool_calls
+
+    started = perf_counter()
+    ai_message = {
+        "role": "assistant",
+        "content": "",
+        "tool_calls": [tool_call],
+    }
+    tool_messages = execute_tool_calls(
+        [tool_call],
+        tools_config,
+        toolset,
+        str(output_dir),
+    )
+    return (
+        {
+            "ai_message": ai_message,
+            "tool_messages": tool_messages,
+            "latency_ms": round((perf_counter() - started) * 1000, 3),
+        },
+        tool_messages,
+    )
+
+
+def _transcribe_audio_for_append(
+    *,
+    audio_path: Path,
+    language: str,
+    runtime: dict,
+    tools_config: str,
+    output_dir: Path,
+) -> tuple[str, list[dict], list[dict]]:
+    audio_bootstrap = []
+    all_tool_messages = []
+
+    preprocess_record, preprocess_messages = _run_forced_tool(
+        tool_call={
+            "id": "runtime_audio_preprocess_001",
+            "name": "audio_preprocess",
+            "args": {"audio_path": str(audio_path)},
+        },
+        tools_config=tools_config,
+        toolset=runtime["toolset"],
+        output_dir=output_dir,
+    )
+    preprocess_record["stage"] = "audio_preprocess"
+    audio_bootstrap.append(preprocess_record)
+    all_tool_messages.extend(preprocess_messages)
+    processed_audio_path = _skill_output(
+        preprocess_messages,
+        "audio_preprocess",
+        ("processed_audio_path",),
+    )["processed_audio_path"].strip()
+
+    asr_args = {"audio_path": processed_audio_path}
+    if language:
+        asr_args["language"] = language
+    asr_record, asr_messages = _run_forced_tool(
+        tool_call={
+            "id": "runtime_speech_to_text_001",
+            "name": "speech_to_text",
+            "args": asr_args,
+        },
+        tools_config=tools_config,
+        toolset=runtime["toolset"],
+        output_dir=output_dir,
+    )
+    asr_record["stage"] = "speech_to_text"
+    audio_bootstrap.append(asr_record)
+    all_tool_messages.extend(asr_messages)
+    transcript = _skill_output(asr_messages, "speech_to_text", ("text",))["text"].strip()
+    return transcript, audio_bootstrap, all_tool_messages
+
+
+def _run_append_reasoning_turn(
+    *,
+    session_messages: list[dict],
+    transcript: str,
+    runtime: dict,
+    tools_config: str,
+    model_config: str,
+    tools_schema: list[dict],
+    mode: str,
+    output_dir: Path,
+    conversation_id: str,
+    audio_bootstrap: list[dict],
+    initial_tool_messages: list[dict],
+) -> dict:
+    session_messages.append(
+        {
+            "role": "user",
+            "content": "[TRANSCRIBED_USER_REQUEST]\n" + transcript,
+        }
+    )
+
+    agent_tools_schema = _agent_tools_schema(tools_schema)
+    turns = []
+    all_tool_messages = list(initial_tool_messages)
+    warnings = []
+    status = "success"
+    terminal_error = None
+    final_answer = ""
+    tool_rounds = 0
+    llm_calls = 0
+    planning_llm_calls = 0
+    planning_latency_ms = 0.0
+    planning_events = []
+    plan_mode = "not_started"
+    execution_plan = None
+    plan_steps = []
+    plan_step_index = 0
+    pending_llm_result = None
+
+    planning_started = perf_counter()
+    planning_llm_calls += 1
+    decision = generate_agent_decision(
+        model_config,
+        session_messages,
+        agent_tools_schema,
+        mode,
+        str(output_dir / "agent_decision"),
+        "initial_decision",
+        conversation_id=conversation_id,
+    )
+    decision_latency_ms = round((perf_counter() - planning_started) * 1000, 3)
+    planning_latency_ms = round(planning_latency_ms + decision_latency_ms, 3)
+    planning_events.append(
+        {"stage": "initial_audio_append_decision", "latency_ms": decision_latency_ms}
+    )
+    write_json(decision, output_dir / "agent_decision.json")
+
+    interpreted = _interpret_agent_decision(decision)
+    if not interpreted["ok"]:
+        status = interpreted["status"]
+        terminal_error = interpreted["error"]
+    else:
+        plan_mode = interpreted["plan_mode"]
+        execution_plan = interpreted["execution_plan"]
+        plan_steps = interpreted["plan_steps"]
+        pending_llm_result = interpreted["pending_llm_result"]
+
+    while status == "success":
+        llm_calls += 1
+        turn_start = perf_counter()
+        current_plan_step = None
+
+        if pending_llm_result is not None:
+            llm_result = pending_llm_result
+            pending_llm_result = None
+        else:
+            call_messages = session_messages
+            if plan_mode == "plan":
+                if plan_step_index >= len(plan_steps):
+                    raise RuntimeError("execution plan ended before the Agent produced a final answer")
+                current_plan_step = plan_steps[plan_step_index]
+                suggested_tool = current_plan_step["suggested_tool"]
+                if suggested_tool is None:
+                    step_instruction = (
+                        "PLAN EXECUTION MODE\n"
+                        f"Current step: {current_plan_step['step_id']}\n"
+                        f"Goal: {current_plan_step['goal']}\n"
+                        "This is the final answer step. Do not call a tool. "
+                        "Use previous ToolMessages and return the final answer for the user."
+                    )
+                else:
+                    step_instruction = (
+                        "PLAN EXECUTION MODE\n"
+                        f"Current step: {current_plan_step['step_id']}\n"
+                        f"Goal: {current_plan_step['goal']}\n"
+                        f"Required tool: {suggested_tool}\n"
+                        "Return exactly one tool call using the required tool. "
+                        "Do not provide the final user answer yet."
+                    )
+                call_messages = _append_plan_step_instruction(
+                    session_messages,
+                    step_instruction,
+                )
+
+            llm_result = generate_ai_message(
+                model_config,
+                call_messages,
+                agent_tools_schema,
+                mode,
+                str(output_dir / "llm_calls"),
+                f"llm_call_{llm_calls:03d}",
+                conversation_id=conversation_id,
+            )
+
+        ai_message = llm_result["ai_message"]
+        llm_status = llm_result.get("status")
+        llm_error = llm_result.get("error")
+        session_messages.append(ai_message)
+        turn = {
+            "turn_index": llm_calls,
+            "ai_message": ai_message,
+            "llm_status": llm_status,
+            "llm_error": llm_error,
+            "tool_messages": [],
+            "latency_ms": None,
+        }
+
+        if llm_status != "success":
+            status = "llm_parse_error"
+            terminal_error = {
+                "type": "LLMParseError",
+                "message": "B4 failed to parse the model output as a valid AIMessage JSON object.",
+                "llm_call_index": llm_calls,
+                "cause": llm_error,
+            }
+            turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+            turns.append(turn)
+            break
+
+        tool_calls = ai_message.get("tool_calls", [])
+        if current_plan_step is not None:
+            expected_tool = current_plan_step["suggested_tool"]
+            if expected_tool is None and tool_calls:
+                status = "plan_step_violation"
+                final_answer = "任务计划执行失败：最终回答步骤不允许继续调用工具。"
+                terminal_error = {
+                    "type": "PlanStepViolation",
+                    "message": final_answer,
+                    "step": current_plan_step,
+                    "tool_calls": tool_calls,
+                }
+                turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+                turns.append(turn)
+                break
+            if expected_tool is not None and (
+                not tool_calls or tool_calls[0].get("name") != expected_tool
+            ):
+                status = "plan_step_violation"
+                final_answer = "任务计划执行失败：模型没有按当前计划步骤调用指定工具。"
+                terminal_error = {
+                    "type": "PlanStepViolation",
+                    "message": final_answer,
+                    "step": current_plan_step,
+                    "expected_tool": expected_tool,
+                    "tool_calls": tool_calls,
+                }
+                turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+                turns.append(turn)
+                break
+
+        if not tool_calls:
+            if current_plan_step is not None:
+                plan_step_index += 1
+            final_answer = ai_message["content"]
+            turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+            turns.append(turn)
+            break
+
+        if tool_rounds >= runtime["max_turns"]:
+            requested = ", ".join(call.get("name", "unknown") for call in tool_calls)
+            status = "max_turns_exceeded"
+            final_answer = f"任务因超过最大工具调用轮次而终止，最后一次模型仍请求调用工具：{requested}。"
+            terminal_error = {
+                "type": "MaxTurnsExceeded",
+                "message": final_answer,
+                "unexecuted_tool_calls": tool_calls,
+            }
+            turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+            turns.append(turn)
+            break
+
+        from b3_tool_layer import execute_tool_calls
+
+        tool_messages = execute_tool_calls(
+            tool_calls,
+            tools_config,
+            runtime["toolset"],
+            str(output_dir),
+        )
+        tool_rounds += 1
+        session_messages.extend(tool_messages)
+        all_tool_messages.extend(tool_messages)
+        turn["tool_messages"] = tool_messages
+
+        tool_failures = _tool_failures(tool_messages)
+        if tool_failures:
+            failed_tools = ", ".join(failure["tool"] for failure in tool_failures)
+            warnings.append(
+                f"tool result invalidated the previous Agent decision; replanning started: {failed_tools}"
+            )
+            replan_messages = _append_system_instruction(
+                session_messages,
+                _replan_instruction(tool_failures),
+            )
+            planning_started = perf_counter()
+            planning_llm_calls += 1
+            decision = generate_agent_decision(
+                model_config,
+                replan_messages,
+                agent_tools_schema,
+                mode,
+                str(output_dir / "agent_decision"),
+                f"replan_after_tool_failure_turn_{llm_calls:03d}",
+                conversation_id=conversation_id,
+            )
+            decision_latency_ms = round((perf_counter() - planning_started) * 1000, 3)
+            planning_latency_ms = round(planning_latency_ms + decision_latency_ms, 3)
+            planning_events.append(
+                {
+                    "stage": "replan_after_tool_failure",
+                    "latency_ms": decision_latency_ms,
+                    "trigger_tools": [failure["tool"] for failure in tool_failures],
+                    "failures": tool_failures,
+                }
+            )
+            write_json(
+                decision,
+                output_dir / f"replan_after_tool_failure_turn_{llm_calls:03d}.json",
+            )
+            interpreted = _interpret_agent_decision(decision)
+            if not interpreted["ok"]:
+                status = interpreted["status"]
+                terminal_error = interpreted["error"]
+                turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+                turns.append(turn)
+                break
+            plan_mode = interpreted["plan_mode"]
+            execution_plan = interpreted["execution_plan"]
+            plan_steps = interpreted["plan_steps"]
+            plan_step_index = 0
+            pending_llm_result = interpreted["pending_llm_result"]
+        elif current_plan_step is not None:
+            plan_step_index += len(tool_calls)
+
+        turn["latency_ms"] = round((perf_counter() - turn_start) * 1000, 3)
+        turns.append(turn)
+
+    write_json(session_messages, output_dir / "messages.json")
+    write_json(all_tool_messages, output_dir / "tool_messages.json")
+    write_text(final_answer, output_dir / "final_answer.md")
+    trace = {
+        "conversation_id": conversation_id,
+        "voice_conversation_mode": "append",
+        "input_type": "audio",
+        "input_audio": {"transcript": transcript},
+        "audio_bootstrap": audio_bootstrap,
+        "tts_output": {
+            "enabled": runtime["enable_tts"],
+            "status": "not_requested" if runtime["enable_tts"] else "disabled",
+        },
+        "tts_voice": runtime.get("tts_voice"),
+        "status": status,
+        "toolset": runtime["toolset"],
+        "max_turns": runtime["max_turns"],
+        "tool_rounds_used": tool_rounds,
+        "llm_call_count": llm_calls,
+        "turns": turns,
+        "final_answer_path": "final_answer.md",
+        "memory_save": {"requested": runtime["save_memory"], "status": "skipped", "reason": "append_session_history"},
+        "agent_execution_mode": plan_mode,
+        "execution_plan": execution_plan,
+        "planning_latency_ms": planning_latency_ms,
+        "planning_llm_calls": planning_llm_calls,
+        "planning_events": planning_events,
+        "warnings": warnings,
+        "error": terminal_error,
+    }
+    write_json(trace, output_dir / "trace.json")
+    return {
+        "conversation_id": conversation_id,
+        "status": status,
+        "final_answer": final_answer,
+        "trace_path": str(output_dir / "trace.json"),
+        "messages_path": str(output_dir / "messages.json"),
+    }
+
+
+def serve(args: argparse.Namespace) -> int:
+    # A voice session owns one long-running TTS worker. Standalone B2 skill
+    # runs remain one-shot so their CUDA process can exit normally.
+    os.environ["COSYVOICE_PERSISTENT_WORKER"] = "1" if args.tts_mode == "remote" else "0"
+    remote_streaming = args.tts_mode == "remote" and args.tts_stream_mode == "stream"
+    remote_full = args.tts_mode == "remote" and args.tts_stream_mode == "full"
+    template_path = Path(args.template).resolve()
+    template = read_json(template_path)
+    session_id = args.session_id or uuid.uuid4().hex[:12]
+    session_dir = (PROJECT_ROOT / "data" / "voice_sessions" / session_id).resolve()
+    output_dir = (PROJECT_ROOT / "outputs" / "voice_sessions" / session_id).resolve()
+    session_dir.mkdir(parents=True, exist_ok=True)
+    output_dir.mkdir(parents=True, exist_ok=True)
+    base_system_prompt_path = resolve_from_file(template["system_prompt_path"], template_path)
+    voice_system_prompt_path = session_dir / "voice_system_prompt.txt"
+    write_text(_voice_system_prompt(read_text(base_system_prompt_path)), voice_system_prompt_path)
+    memory_id = None
+    turn = 1
+    tools_config = args.tools_config or str(PROJECT_ROOT / "configs" / "tools.yaml")
+    memory_config = args.memory_config or str(PROJECT_ROOT / "configs" / "memory.yaml")
+    model_config = args.model_config or str(PROJECT_ROOT / "configs" / "model.yaml")
+    conversation_mode = args.conversation_mode
+    llm_mode = args.llm_mode or _default_llm_mode(Path(model_config))
+    tools_schema = None
+    session_messages = None
+    if conversation_mode == "append":
+        from b3_tool_layer import get_tools_schema
+
+        session_messages = [{"role": "system", "content": read_text(voice_system_prompt_path)}]
+        tools_schema = get_tools_schema(tools_config, "multimodal_tools", str(output_dir))
+    print(
+        f"[VoiceChat] session={session_id}; mode={conversation_mode}; waiting for audio",
+        file=sys.stderr,
+        flush=True,
+    )
+    _response({"status": "ready", "session_id": session_id, "conversation_mode": conversation_mode})
+
+    try:
+        for line in sys.stdin:
+            if not line.strip():
+                continue
+            try:
+                message = json.loads(line)
+                if message.get("command") in {"quit", "exit", "shutdown"}:
+                    _response({"status": "bye", "session_id": session_id})
+                    return 0
+                audio_path = _save_audio(message, session_dir, turn)
+                runtime = deepcopy(template)
+                turn_conversation_id = (
+                    session_id
+                    if conversation_mode == "append"
+                    else f"{session_id}_turn_{turn:03d}"
+                )
+                runtime.update(
+                    {
+                        "execution_mode": "integrated",
+                        "conversation_id": turn_conversation_id,
+                        "input_type": "audio",
+                        "audio_path": str(audio_path),
+                        "audio_language": message.get("language") or args.language,
+                        "toolset": "multimodal_tools",
+                        "max_turns": max(3, int(runtime.get("max_turns", 6))),
+                        "save_memory": "conversation",
+                        "enable_tts": remote_full,
+                        "tts_language": args.tts_language,
+                        "selected_memory_ids": [memory_id] if memory_id else [],
+                        "use_global_memory": False,
+                        "system_prompt_path": str(voice_system_prompt_path),
+                    }
+                )
+                if args.tts_voice:
+                    runtime["tts_voice"] = args.tts_voice
+                runtime_path = session_dir / f"runtime_{turn:03d}.json"
+                turn_dir = output_dir / f"turn_{turn:03d}"
+                write_json(runtime, runtime_path)
+                if conversation_mode == "append":
+                    if session_messages is None or tools_schema is None:
+                        raise RuntimeError("append conversation state was not initialized")
+                    turn_dir.mkdir(parents=True, exist_ok=True)
+                    transcript, audio_bootstrap, bootstrap_tool_messages = _transcribe_audio_for_append(
+                        audio_path=audio_path,
+                        language=runtime.get("audio_language") or args.language,
+                        runtime=runtime,
+                        tools_config=tools_config,
+                        output_dir=turn_dir,
+                    )
+                    result = _run_append_reasoning_turn(
+                        session_messages=session_messages,
+                        transcript=transcript,
+                        runtime=runtime,
+                        tools_config=tools_config,
+                        model_config=model_config,
+                        tools_schema=tools_schema,
+                        mode=llm_mode,
+                        output_dir=turn_dir,
+                        conversation_id=session_id,
+                        audio_bootstrap=audio_bootstrap,
+                        initial_tool_messages=bootstrap_tool_messages,
+                    )
+                else:
+                    result = run_agent(
+                        str(runtime_path), tools_config, memory_config,
+                        model_config, str(turn_dir), args.llm_mode,
+                    )
+                    saved_path = turn_dir / "saved_memory.json"
+                    if saved_path.is_file():
+                        memory_id = read_json(saved_path).get("memory_id") or memory_id
+                trace = read_json(result["trace_path"])
+                answer = result.get("final_answer", "")
+                tts_output = trace.get("tts_output") or {}
+                audio_b64 = None
+                audio_suffix = None
+                tts_status = "disabled"
+                if remote_streaming and answer.strip():
+                    tts_status = "streaming"
+                elif remote_full:
+                    if conversation_mode == "append" and answer.strip():
+                        tts_output = text_to_speech(
+                            answer,
+                            language=args.tts_language,
+                            output_filename=f"turn_{turn:03d}.wav",
+                            tts_voice=runtime.get("tts_voice"),
+                            output_dir=str(turn_dir),
+                        )
+                        trace["tts_output"] = tts_output
+                        write_json(trace, result["trace_path"])
+                    tts_status = str(tts_output.get("status", "skipped"))
+                    generated_audio_path = tts_output.get("audio_path")
+                    if tts_status == "success" and isinstance(generated_audio_path, str):
+                        generated_path = Path(generated_audio_path)
+                        if generated_path.is_file():
+                            audio_b64 = base64.b64encode(generated_path.read_bytes()).decode("ascii")
+                            audio_suffix = generated_path.suffix or ".wav"
+                base_payload = {
+                    "status": result.get("status", "success"), "session_id": session_id,
+                    "turn": turn, "transcript": _trace_transcript(trace),
+                    "answer": answer, "audio_b64": audio_b64, "audio_suffix": audio_suffix,
+                    "tts": {"enabled": args.tts_mode == "remote", "status": tts_status, "mode": args.tts_stream_mode},
+                    "output_dir": str(turn_dir),
+                    "conversation_mode": conversation_mode,
+                }
+                _response(base_payload)
+
+                if remote_streaming and answer.strip() and result.get("status", "success") == "success":
+                    chunk_count = 0
+                    for event in stream_text_to_speech(
+                        answer,
+                        language=args.tts_language,
+                        output_filename=f"turn_{turn:03d}.wav",
+                        tts_voice=runtime.get("tts_voice"),
+                        output_dir=str(turn_dir),
+                    ):
+                        audio_path = event.get("audio_path")
+                        audio_b64 = (
+                            base64.b64encode(Path(audio_path).read_bytes()).decode("ascii")
+                            if isinstance(audio_path, str) and Path(audio_path).is_file()
+                            else None
+                        )
+                        chunk_count = max(chunk_count, int(event.get("index", 0) or 0))
+                        _response({
+                            "status": "audio_chunk",
+                            "session_id": session_id,
+                            "turn": turn,
+                            "audio_b64": audio_b64,
+                            "audio_suffix": ".wav",
+                            "tts_event": event,
+                        })
+                    _response({
+                        "status": "done",
+                        "session_id": session_id,
+                        "turn": turn,
+                        "chunk_count": chunk_count,
+                    })
+            except Exception as exc:
+                _response({"status": "error", "turn": turn, "error": f"{type(exc).__name__}: {exc}"})
+            turn += 1
+    finally:
+        shutdown_tts_worker()
+    return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="Persistent stdin/stdout voice-chat server for an SSH session.")
+    parser.add_argument("--template", required=True, help="Base runtime JSON")
+    parser.add_argument("--tools-config", default=None)
+    parser.add_argument("--memory-config", default=None)
+    parser.add_argument("--model-config", default=None)
+    parser.add_argument("--llm-mode", choices=["mock", "prompt_json"], default=None)
+    parser.add_argument("--language", default="zh")
+    parser.add_argument("--tts-language", choices=["auto", "zh", "en"], default="auto")
+    parser.add_argument("--tts-voice", default=None)
+    parser.add_argument("--tts-mode", choices=["remote", "none"], default="none")
+    parser.add_argument("--tts-stream-mode", choices=["stream", "full"], default="stream")
+    parser.add_argument("--session-id", default=None)
+    parser.add_argument("--conversation-mode", choices=["restart", "append"], default="restart")
+    return parser
+
+
+if __name__ == "__main__":
+    raise SystemExit(serve(build_parser().parse_args()))

code/voice_chat_windows_client.py

@@ -0,0 +1,284 @@
+from __future__ import annotations
+
+import argparse
+import base64
+import io
+import json
+import os
+import shlex
+import subprocess
+import sys
+from pathlib import Path
+
+from common.io_utils import read_yaml
+
+
+PREFIX = "__VOICE_CHAT_RESPONSE__"
+AGENT_ROOT = Path(__file__).resolve().parents[1]
+DEFAULT_CONFIG = AGENT_ROOT / "configs" / "voice_chat.yaml"
+
+
+def _nonempty(value):
+    return value.strip() if isinstance(value, str) and value.strip() else value
+
+
+def _config_value(config: dict, section: str, name: str, default=None):
+    value = config.get(section, {}).get(name) if isinstance(config.get(section), dict) else None
+    value = _nonempty(value)
+    return default if value is None else value
+
+
+def _load_client_config(path: str | None) -> dict:
+    if not path:
+        return {}
+    config_path = Path(path).expanduser().resolve()
+    if not config_path.is_file():
+        if config_path == DEFAULT_CONFIG.resolve():
+            return {}
+        raise RuntimeError(f"Voice chat config does not exist: {config_path}")
+    config = read_yaml(config_path)
+    if not isinstance(config, dict):
+        raise RuntimeError(f"Voice chat config must be an object: {config_path}")
+    return config
+
+
+def _apply_config_defaults(args) -> None:
+    config = _load_client_config(args.config)
+    args.host = args.host or _config_value(config, "connection", "host")
+    args.port = args.port if args.port is not None else int(_config_value(config, "connection", "port", 22))
+    args.remote_root = args.remote_root or _config_value(config, "connection", "remote_root")
+    args.python = args.python or _config_value(config, "connection", "python", "python")
+    args.template = args.template or _config_value(config, "connection", "template", "agent/data/runtime_input_audio.json")
+    args.language = args.language or _config_value(config, "audio", "language", "zh")
+    args.sample_rate = args.sample_rate if args.sample_rate is not None else int(_config_value(config, "audio", "sample_rate", 16000))
+    args.tts_mode = args.tts_mode or _config_value(config, "tts", "mode", "local")
+    args.tts_stream_mode = args.tts_stream_mode or _config_value(config, "tts", "stream_mode", "stream")
+    args.tts_language = args.tts_language or _config_value(config, "tts", "language", "auto")
+    args.tts_voice = args.tts_voice or _config_value(config, "tts", "voice")
+    args.conversation_mode = args.conversation_mode or _config_value(config, "conversation", "mode", "restart")
+    args.local_tts_python = args.local_tts_python or _config_value(config, "local_tts", "python")
+    args.local_cosyvoice_repo = args.local_cosyvoice_repo or _config_value(config, "local_tts", "cosyvoice_repo")
+    args.local_tts_model = args.local_tts_model or _config_value(config, "local_tts", "model_dir")
+    args.local_prompt_wav = args.local_prompt_wav or _config_value(config, "local_tts", "prompt_wav")
+    args.local_prompt_text = args.local_prompt_text or _config_value(config, "local_tts", "prompt_text")
+    if not args.host:
+        raise RuntimeError("Missing SSH host. Set connection.host in agent/configs/voice_chat.yaml or pass --host.")
+    if not args.remote_root:
+        raise RuntimeError("Missing remote root. Set connection.remote_root in agent/configs/voice_chat.yaml or pass --remote-root.")
+
+
+def record_wav(sample_rate: int) -> bytes:
+    try:
+        import numpy as np
+        import sounddevice as sd
+        import soundfile as sf
+    except ImportError as exc:
+        raise RuntimeError("Please install local audio dependencies: python -m pip install numpy sounddevice soundfile") from exc
+    chunks = []
+
+    def callback(indata, frames, time_info, status):
+        if status:
+            print(status, file=sys.stderr)
+        chunks.append(indata.copy())
+
+    input("Press Enter to start recording...")
+    with sd.InputStream(samplerate=sample_rate, channels=1, dtype="float32", callback=callback):
+        input("Recording; press Enter again to stop...")
+    if not chunks:
+        raise RuntimeError("No audio was recorded")
+    buffer = io.BytesIO()
+    sf.write(buffer, np.concatenate(chunks, axis=0), sample_rate, format="WAV", subtype="PCM_16")
+    return buffer.getvalue()
+
+
+def play_wav(data: bytes) -> None:
+    import sounddevice as sd
+    import soundfile as sf
+    audio, rate = sf.read(io.BytesIO(data), dtype="float32")
+    sd.play(audio, rate)
+    sd.wait()
+
+
+def configure_local_tts(args):
+    assignment_root = Path(__file__).resolve().parents[2]
+    inferred_python = Path(sys.executable).resolve().parent.parent / "cosyvoice" / "python.exe"
+    paths = {
+        "CosyVoice Python": Path(args.local_tts_python).resolve() if args.local_tts_python else inferred_python,
+        "CosyVoice repo": Path(args.local_cosyvoice_repo).resolve() if args.local_cosyvoice_repo else assignment_root / "CosyVoice",
+        "TTS model": Path(args.local_tts_model).resolve() if args.local_tts_model else assignment_root / "CosyVoice2-0.5B",
+    }
+    prompt = Path(args.local_prompt_wav).resolve() if args.local_prompt_wav else None
+    if prompt is not None:
+        paths["prompt WAV"] = prompt
+    for label, path in paths.items():
+        if not path.exists():
+            raise RuntimeError(f"Local {label} does not exist: {path}")
+    os.environ.update({
+        "COSYVOICE_PYTHON": str(paths["CosyVoice Python"]),
+        "COSYVOICE_REPO": str(paths["CosyVoice repo"]),
+        "COSYVOICE_MODEL": str(paths["TTS model"]),
+        "COSYVOICE_PERSISTENT_WORKER": "1",
+    })
+    if prompt is not None:
+        os.environ["COSYVOICE_PROMPT_WAV"] = str(prompt)
+    if args.local_prompt_text:
+        os.environ["COSYVOICE_PROMPT_TEXT"] = args.local_prompt_text
+    if args.tts_voice:
+        os.environ["COSYVOICE_VOICE"] = args.tts_voice
+    from common.path_utils import bootstrap_project_root
+    bootstrap_project_root()
+    from skills.text_to_speech import shutdown_tts_worker, stream_text_to_speech, text_to_speech
+    return text_to_speech, stream_text_to_speech, shutdown_tts_worker, assignment_root / "agent" / "outputs" / "local_voice_sessions"
+
+
+def _read_response(line: str) -> dict:
+    return json.loads(base64.b64decode(line[len(PREFIX):]).decode("utf-8"))
+
+
+def _play_local_stream(local_tts_stream, answer: str, args, output_root: Path, response: dict) -> None:
+    print("[local TTS] streaming synthesis...")
+    for event in local_tts_stream(
+        answer,
+        language=args.tts_language,
+        output_filename=f"turn_{response.get('turn', 0):03d}.wav",
+        tts_voice=args.tts_voice,
+        output_dir=str(output_root / response.get("session_id", "session")),
+    ):
+        voice = event.get("voice_id")
+        voice_text = f" voice={voice}" if voice else ""
+        print(f"[TTS] chunk {event.get('index')}/{event.get('chunk_count')}{voice_text}")
+        play_wav(Path(event["audio_path"]).read_bytes())
+
+
+def _play_local_full(local_tts, answer: str, args, output_root: Path, response: dict) -> None:
+    print("[local TTS] full synthesis...")
+    result = local_tts(
+        answer,
+        language=args.tts_language,
+        output_filename=f"turn_{response.get('turn', 0):03d}.wav",
+        tts_voice=args.tts_voice,
+        output_dir=str(output_root / response.get("session_id", "session")),
+    )
+    play_wav(Path(result["audio_path"]).read_bytes())
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Windows microphone/speaker bridge to the SSH voice-chat server.")
+    parser.add_argument("--config", default=str(DEFAULT_CONFIG))
+    parser.add_argument("--host")
+    parser.add_argument("--port", type=int, default=None)
+    parser.add_argument("--remote-root")
+    parser.add_argument("--python", default=None)
+    parser.add_argument("--template", default=None)
+    parser.add_argument("--language", default=None)
+    parser.add_argument("--sample-rate", type=int, default=None)
+    parser.add_argument("--tts-mode", choices=["local", "remote", "none"], default=None)
+    parser.add_argument("--tts-stream-mode", choices=["stream", "full"], default=None)
+    parser.add_argument("--tts-language", choices=["auto", "zh", "en"], default=None)
+    parser.add_argument("--tts-voice", default=None)
+    parser.add_argument("--conversation-mode", choices=["restart", "append"], default=None)
+    parser.add_argument("--local-tts-python")
+    parser.add_argument("--local-cosyvoice-repo")
+    parser.add_argument("--local-tts-model")
+    parser.add_argument("--local-prompt-wav")
+    parser.add_argument("--local-prompt-text")
+    args = parser.parse_args()
+    _apply_config_defaults(args)
+
+    local_tts = local_tts_stream = local_tts_shutdown = local_output_root = None
+    if args.tts_mode == "local":
+        local_tts, local_tts_stream, local_tts_shutdown, local_output_root = configure_local_tts(args)
+    remote_tts_mode = "remote" if args.tts_mode == "remote" else "none"
+    remote = (
+        f"cd {shlex.quote(args.remote_root)} && {shlex.quote(args.python)} "
+        f"agent/code/voice_chat_server.py --template {shlex.quote(args.template)} "
+        f"--language {shlex.quote(args.language)} --tts-mode {remote_tts_mode} "
+        f"--tts-stream-mode {shlex.quote(args.tts_stream_mode)} "
+        f"--conversation-mode {shlex.quote(args.conversation_mode)}"
+    )
+    if args.tts_mode == "remote" and args.tts_voice:
+        remote += f" --tts-voice {shlex.quote(args.tts_voice)}"
+    process = subprocess.Popen(
+        ["ssh", "-T", "-p", str(args.port), args.host, remote],
+        stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True,
+        encoding="utf-8", errors="replace", bufsize=1,
+    )
+    try:
+        ready = False
+        for line in process.stdout:
+            if line.startswith(PREFIX):
+                response = _read_response(line)
+                if response.get("status") == "ready":
+                    print(f"Remote voice service ready, session={response.get('session_id')}")
+                    ready = True
+                    break
+            else:
+                print("[server]", line.rstrip())
+        if not ready:
+            raise RuntimeError(f"Remote voice service failed to start; SSH exit code={process.poll()}")
+
+        while process.poll() is None:
+            wav = record_wav(args.sample_rate)
+            process.stdin.write(json.dumps({"audio_b64": base64.b64encode(wav).decode("ascii"), "suffix": ".wav", "language": args.language}) + "\n")
+            process.stdin.flush()
+            waiting_for_remote_audio = False
+
+            for line in process.stdout:
+                if not line.startswith(PREFIX):
+                    print("[server]", line.rstrip())
+                    continue
+
+                response = _read_response(line)
+                status = response.get("status")
+                if status == "error":
+                    print("Error:", response.get("error"))
+                    break
+                if status == "audio_chunk":
+                    event = response.get("tts_event") or {}
+                    voice = event.get("voice_id")
+                    voice_text = f" voice={voice}" if voice else ""
+                    print(f"[TTS] chunk {event.get('index')}/{event.get('chunk_count')}{voice_text}")
+                    if response.get("audio_b64"):
+                        play_wav(base64.b64decode(response["audio_b64"]))
+                    waiting_for_remote_audio = True
+                    continue
+                if status == "done":
+                    waiting_for_remote_audio = False
+                    break
+
+                print("You:", response.get("transcript") or "(no transcript)")
+                answer = response.get("answer", "")
+                print("Agent:", answer)
+                if response.get("audio_b64"):
+                    play_wav(base64.b64decode(response["audio_b64"]))
+                    break
+                if local_tts_stream is not None and args.tts_stream_mode == "stream" and answer.strip():
+                    _play_local_stream(local_tts_stream, answer, args, local_output_root, response)
+                    break
+                if local_tts is not None and args.tts_stream_mode == "full" and answer.strip():
+                    _play_local_full(local_tts, answer, args, local_output_root, response)
+                    break
+                if response.get("tts", {}).get("status") == "streaming":
+                    waiting_for_remote_audio = True
+                    continue
+                break
+
+            if waiting_for_remote_audio:
+                print("[TTS] audio stream ended unexpectedly")
+            if input("Continue? [Enter/q] ").strip().lower() in {"q", "quit", "exit"}:
+                break
+    finally:
+        if local_tts_shutdown is not None:
+            local_tts_shutdown()
+        if process.poll() is None:
+            try:
+                process.stdin.write('{"command":"quit"}\n')
+                process.stdin.flush()
+                process.wait(timeout=10)
+            except Exception:
+                process.terminate()
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

configs/model.yaml

@@ -16,12 +16,22 @@ generation:
   temperature: 0
   top_p: 1
 
+kv_cache:
+  enabled: true
+  ttl_seconds: 1200
+  max_entries: 8
+  max_tokens_per_entry: 8192
+  max_total_gpu_mb: 2048
+  cpu_enabled: true
+  cpu_max_entries: 32
+  cpu_max_total_mb: 8192
+
 context:
   max_input_tokens: 4096
 
 tool_calling:
   mode: prompt_json
-  schema_injection: native 
+  schema_injection: native
   
 outputs:
   save_raw_output: true

configs/tools.yaml

@@ -47,7 +47,7 @@ tools:
         description: Path relative to the configured data root.
       max_chars:
         type: integer
-        description: Maximum number of characters to return.
+        description: Maximum number of characters to return. For document summaries, omit this field or use at least 1200 so the result includes content beyond front matter.
     required: [path]
     returns:
       content: {type: string, description: File content.}
@@ -61,7 +61,7 @@ tools:
     description: Search local txt and md files for query terms.
     parameters:
       query: {type: string, description: Search query.}
-      root_dir: {type: string, description: Directory relative to data root.}
+      root_dir: {type: string, description: "Subdirectory inside data_root; use docs for project documents."}
       file_types:
         type: array
         items: {type: string}
@@ -175,6 +175,9 @@ tools:
       output_filename:
         type: string
         description: Optional output WAV file name. Defaults to final_answer.wav.
+      tts_voice:
+        type: string
+        description: Optional voice profile id from agent/configs/tts.yaml.
     required: [text]
     returns:
       audio_path:
@@ -195,4 +198,3 @@ tools:
       model:
         type: object
         description: CosyVoice runtime metadata.
-

configs/tts.yaml

@@ -1,11 +1,28 @@
 runtime:
-  python_executable: "/opt/conda/envs/cosyvoice/bin/python"
+  python_executable: "C:/Users/30703/anaconda3/envs/cosyvoice/python.exe"
   timeout_sec: 600
   max_total_chars: 1200
-  max_chunk_chars: 120
-  inter_chunk_silence_sec: 0.18
+  min_chunk_chars: 20
+  max_chunk_chars: 60
+  hard_max_chunk_chars: 80
+  inter_chunk_silence_sec: 0.08
+  persistent_worker: false
+  load_jit: false
+  fp16: false
+  # Keep the text frontend enabled when wetext is installed; disabling it makes
+  # Chinese normalization fall back to raw text and can produce unstable speech.
+  disable_text_frontend: false
 
 voice:
+  default_voice_id: default
   cosyvoice_repo: "../../CosyVoice"
   model_dir: "../../CosyVoice2-0.5B"
   prompt_wav: "../../CosyVoice/asset/zero_shot_prompt.wav"
+  prompt_text: "希望你以后能够做得比我还好。"
+  voices:
+    default:
+      prompt_wav: "../../CosyVoice/asset/zero_shot_prompt.wav"
+      prompt_text: "希望你以后能够做得比我还好。"
+    funingna:
+      prompt_wav: "../data/voices/funingna.wav"
+      prompt_text: "我对自己的歌唱水平还是很有自信的，不过值得我开口唱诵的段落可不多。希望那些剧团里的创作者们加把劲，不要让我等太久。"

configs/voice_chat.yaml

@@ -0,0 +1,28 @@
+connection:
+  host: "root@202.199.13.141"
+  port: 20092
+  remote_root: "/root/siton-tmp/assignment_B"
+  python: " /opt/conda/envs/agent/bin/python"
+  template: "agent/data/runtime_input_audio.json"
+
+audio:
+  language: "zh"
+  sample_rate: 16000
+
+tts:
+  mode: "local"
+  stream_mode: "stream"
+  language: "auto"
+  voice: "funingna"
+
+conversation:
+  # restart: keep the old per-turn run_agent behavior.
+  # append: keep one voice session message history so KV cache can reuse prior turns.
+  mode: "append"
+
+local_tts:
+  python: null
+  cosyvoice_repo: null
+  model_dir: null
+  prompt_wav: null
+  prompt_text: null

data/b4_schema_benchmark.json

@@ -0,0 +1,53 @@
+{
+  "repeats": 3,
+  "max_llm_calls": 4,
+  "system_prompt_path": "../prompts/local_tool_agent.txt",
+  "tools_schema_path": "messages/tools_schema_basic.json",
+  "tasks": [
+    {
+      "id": "no_tool",
+      "user_input": "什么是原神？请简要回答。",
+      "expected_tools": [],
+      "tool_outputs": []
+    },
+    {
+      "id": "calculator",
+      "user_input": "请调用 calculator 计算 56*29+81。",
+      "expected_tools": ["calculator"],
+      "tool_outputs": [{"result": 1705}]
+    },
+    {
+      "id": "file_reader",
+      "user_input": "请调用 file_reader 读取 docs/agent_intro.txt，并总结内容。",
+      "expected_tools": ["file_reader"],
+      "tool_outputs": [{"content": "Agent 系统由模型、工具、记忆和执行循环组成。", "source": "docs/agent_intro.txt"}]
+    },
+    {
+      "id": "local_file_search",
+      "user_input": "请调用 local_file_search 在 docs 目录搜索‘工具调用’，返回最相关文件。",
+      "expected_tools": ["local_file_search"],
+      "tool_outputs": [{"results": [{"path": "docs/01_agent_tool_call_exact.md", "score": 1.0}]}]
+    },
+    {
+      "id": "table_analyzer",
+      "user_input": "请调用 table_analyzer 分析 tables/results.csv，告诉我行数和列名。",
+      "expected_tools": ["table_analyzer"],
+      "tool_outputs": [{"rows": 3, "columns": ["task", "score", "latency"]}]
+    },
+    {
+      "id": "format_converter",
+      "user_input": "请调用 format_converter，把 name: Agent Demo 转换成 JSON。",
+      "expected_tools": ["format_converter"],
+      "tool_outputs": [{"formatted_text": "{\"name\":\"Agent Demo\"}"}]
+    },
+    {
+      "id": "search_then_read",
+      "user_input": "请先在 docs 目录搜索 Agent 工具调用，再读取最相关文档并总结。",
+      "expected_tools": ["local_file_search", "file_reader"],
+      "tool_outputs": [
+        {"results": [{"path": "docs/01_agent_tool_call_exact.md", "score": 1.0}]},
+        {"content": "Agent 先规划工具调用，B3 校验并执行 Skill，结果再交回模型。", "source": "docs/01_agent_tool_call_exact.md"}
+      ]
+    }
+  ]
+}

data/runtime_input_genshin.json

@@ -0,0 +1,10 @@
+{
+  "conversation_id": "conv_genshin_001",
+  "user_input": "什么是原神",
+  "system_prompt_path": "../prompts/local_tool_agent.txt",
+  "selected_memory_ids": [],
+  "use_global_memory": false,
+  "toolset": "basic_tools",
+  "max_turns": 3,
+  "save_memory": "none"
+}

data/runtime_input_tts.json

@@ -8,5 +8,6 @@
   "max_turns": 3,
   "save_memory": "conversation",
   "enable_tts": true,
-  "tts_language": "auto"
+  "tts_language": "auto",
+  "tts_voice": "default"
 }

data/runtime_kv_cache_hit.json

@@ -0,0 +1,12 @@
+{
+  "execution_mode": "integrated",
+  "input_type": "text",
+  "conversation_id": "conv_kv_cache_hit_001",
+  "system_prompt_path": "../prompts/local_tool_agent.txt",
+  "selected_memory_ids": [],
+  "use_global_memory": false,
+  "toolset": "basic_tools",
+  "max_turns": 4,
+  "save_memory": "none",
+  "user_input": "Please answer in Chinese. First use exactly one file_reader tool call to read docs/03_agent_development_exact.md with max_chars 2000. After the tool result is available, summarize the document into three concise Chinese bullet points. Do not call local_file_search because the exact file path is already provided."
+}

data/runtime_plan_kv_cache_hit.json

@@ -0,0 +1,12 @@
+{
+  "execution_mode": "integrated",
+  "input_type": "text",
+  "conversation_id": "conv_plan_kv_cache_hit_001",
+  "system_prompt_path": "../prompts/local_tool_agent.txt",
+  "selected_memory_ids": [],
+  "use_global_memory": false,
+  "toolset": "basic_tools",
+  "max_turns": 5,
+  "save_memory": "none",
+  "user_input": "Please answer in Chinese. Search the local docs directory for documents about Agent tool calling, then read the most relevant document returned by the search, and finally summarize the evidence into three concise Chinese bullet points. This requires the dependent sequence local_file_search -> file_reader -> final answer; do not guess the file path before searching."
+}

prompts/local_tool_agent.txt

@@ -1,3 +1,3 @@
-You are a local tool-using agent. Use available tools when needed. Do not invent file contents. If the user asks about a local file, use the file reading tool. If a tool is needed, choose exactly one tool and wait for its ToolMessage before deciding whether another tool is needed. Never request multiple tools in the same response. If tool results are provided, answer based on the tool results. Follow the user's output requirements.
+You are a local tool-using agent. Use available tools when needed. Do not invent file contents. If the user asks about a local file, use the file reading tool. If the user says according to the document or 根据文档 without giving a file path, search local documents first and read the relevant result before answering. If a tool is needed, choose exactly one tool and wait for its ToolMessage before deciding whether another tool is needed. Never request multiple tools in the same response. If tool results are provided, answer based on the tool results. Do not repeatedly read the same file only to change max_chars; after a successful file_reader result, answer if the content is enough, or make a targeted search/read for missing information. Follow the user's output requirements.
 
 Audio-input rule: when the current user message is marked [INPUT_TYPE: AUDIO], the audio file itself is the user's input rather than the filename being the request. During the audio bootstrap phase, call audio_preprocess first, then call speech_to_text with the processed_audio_path returned by audio_preprocess. Do not answer or use downstream tools until speech_to_text succeeds. When a later message is marked [TRANSCRIBED_USER_REQUEST], treat its transcript as the user's real current request and complete it using the normal tool loop.

skills/file_reader.py

@@ -8,7 +8,10 @@ from skills.tool_decorator import tool
     description="Read a local UTF-8 txt or md file from the data directory.",
     parameters={
         "path": "Path relative to the configured data root.",
-        "max_chars": "Maximum number of characters to return.",
+        "max_chars": (
+            "Maximum number of characters to return. For document summaries, "
+            "omit this field or use at least 1200 so the result includes content beyond front matter."
+        ),
     },
     returns={
         "content": {"type": "string", "description": "File content."},

skills/local_file_search.py

@@ -1,37 +1,57 @@
 from __future__ import annotations
 
+from dataclasses import dataclass
+from math import sqrt
 import re
+from pathlib import Path
+from typing import Any
 
 from skills import resolve_data_path
 from skills.tool_decorator import tool
 
-def _snippet(text: str, terms: list[str], radius: int = 60) -> str:
+
+MIN_SEMANTIC_MATCH_THRESHOLD = 0.015
+LEXICAL_WEIGHT = 0.58
+SEMANTIC_WEIGHT = 0.42
+
+
+@dataclass
+class _Document:
+    path: Path
+    text: str
+    filename: str
+
+
+def _snippet(
+    text: str,
+    terms: list[str],
+    radius: int = 60,
+    center: int | None = None,
+) -> str:
     lowered = text.casefold()
-    positions = [lowered.find(term.casefold()) for term in terms]
+    positions = [lowered.find(term.casefold()) for term in terms] if terms else []
     positions = [position for position in positions if position >= 0]
-    start = max(0, (min(positions) if positions else 0) - radius)
+    anchor = center if center is not None else (min(positions) if positions else 0)
+    start = max(0, anchor - radius)
     end = min(len(text), start + radius * 2)
     prefix = "..." if start else ""
     suffix = "..." if end < len(text) else ""
     return prefix + text[start:end].replace("\n", " ").strip() + suffix
 
+
 def _score_document(
     text: str,
     filename: str,
     phrase: str,
     terms: list[str],
-) -> tuple[float, list[str], bool]:
-    """
-    phrase：完整查询短语，用于精确匹配。
-    terms：拆分后的辅助词，用于扩大召回范围。
-    """
+) -> tuple[float, list[str], bool, float]:
     lowered_text = text.casefold()
     lowered_filename = filename.casefold()
 
     score = 0.0
+    filename_score = 0.0
     matched_terms: list[str] = []
 
-    # 1. 辅助词只提供较低分数，且限制重复词的贡献。
     for term in terms:
         text_hits = lowered_text.count(term)
         filename_hits = lowered_filename.count(term)
@@ -40,51 +60,41 @@ def _score_document(
             matched_terms.append(term)
 
         score += min(text_hits, 5)
-        # 文件名命中说明主题更相关，额外加分。
         if filename_hits > 0:
             score += 3
+            filename_score += 3
 
-    # 2. 完整短语是另一个排序依据。
     phrase_matched = False
-
-    # 一个普通单词如 agent 不需要再额外做短语奖励。
-    # 只有查询被拆成多个辅助词时，才判断完整短语。
     if len(terms) > 1:
         compact_phrase = re.sub(r"\s+", "", phrase.casefold())
         compact_text = re.sub(r"\s+", "", lowered_text)
         compact_filename = re.sub(r"\s+", "", lowered_filename)
 
         if compact_phrase and compact_phrase in compact_text:
-            score += 2*len(terms) * compact_text.count(compact_phrase)
+            score += 2 * len(terms) * compact_text.count(compact_phrase)
             phrase_matched = True
 
         if compact_phrase and compact_phrase in compact_filename:
             score += 20
+            filename_score += 20
             phrase_matched = True
 
-    return round(score, 3), matched_terms, phrase_matched
+    return round(score, 3), matched_terms, phrase_matched, round(filename_score, 3)
+
+
+def _normalize_score(score: float) -> float:
+    if score <= 0:
+        return 0.0
+    return score / (score + 8.0)
+
 
 def _build_query(query: str) -> dict:
-    """
-    同时保存完整查询短语和拆分后的辅助词。
-
-    phrase 用于精确匹配；
-    terms 用于召回可能相关的文档。
-    """
     normalized = query.casefold().strip()
-
-    # 去掉查询中的空白，统一用于短语精确匹配。
     phrase = re.sub(r"\s+", "", normalized)
-
-    # 拆出英文片段、数字/下划线标识符、连续中文片段。
-    raw_terms = re.findall(
-        r"[a-z0-9_]+|[\u4e00-\u9fff]+",
-        normalized,
-    )
+    raw_terms = re.findall(r"[a-z0-9_]+|[\u4e00-\u9fff]+", normalized)
 
     terms = []
     seen = set()
-
     for term in raw_terms:
         if term not in seen:
             terms.append(term)
@@ -96,11 +106,120 @@ def _build_query(query: str) -> dict:
     }
 
 
+def _tokenize_for_vector(text: str) -> list[str]:
+    normalized = text.casefold()
+    tokens: list[str] = []
+
+    try:
+        import jieba
+
+        tokens.extend(token.strip() for token in jieba.cut(normalized) if token.strip())
+    except Exception:
+        tokens.extend(re.findall(r"[a-z0-9_]+|[\u4e00-\u9fff]+", normalized))
+
+    tokens.extend(re.findall(r"[a-z0-9_]+", normalized))
+    chinese = re.sub(r"[^\u4e00-\u9fff]", "", normalized)
+    tokens.extend(chinese[index : index + 2] for index in range(max(0, len(chinese) - 1)))
+
+    deduped = []
+    seen = set()
+    for token in tokens:
+        if token and token not in seen:
+            deduped.append(token)
+            seen.add(token)
+    return deduped
+
+
+def _fallback_vector_scores(query: str, documents: list[_Document]) -> list[float]:
+    query_tokens = _tokenize_for_vector(query)
+    doc_tokens = [
+        _tokenize_for_vector(f"{document.filename} {document.text}")
+        for document in documents
+    ]
+    all_tokens = doc_tokens + [query_tokens]
+
+    document_frequency: dict[str, int] = {}
+    for tokens in all_tokens:
+        for token in set(tokens):
+            document_frequency[token] = document_frequency.get(token, 0) + 1
+
+    doc_count = len(all_tokens)
+
+    def vectorize(tokens: list[str]) -> dict[str, float]:
+        counts: dict[str, int] = {}
+        for token in tokens:
+            counts[token] = counts.get(token, 0) + 1
+
+        vector = {}
+        for token, count in counts.items():
+            idf = 1.0 + doc_count / (1 + document_frequency.get(token, 0))
+            vector[token] = count * idf
+        return vector
+
+    def cosine(left: dict[str, float], right: dict[str, float]) -> float:
+        if not left or not right:
+            return 0.0
+        dot = sum(value * right.get(token, 0.0) for token, value in left.items())
+        left_norm = sqrt(sum(value * value for value in left.values()))
+        right_norm = sqrt(sum(value * value for value in right.values()))
+        if left_norm == 0 or right_norm == 0:
+            return 0.0
+        return dot / (left_norm * right_norm)
+
+    query_vector = vectorize(query_tokens)
+    return [cosine(vectorize(tokens), query_vector) for tokens in doc_tokens]
+
+
+def _semantic_scores(query: str, documents: list[_Document]) -> list[float]:
+    if not documents:
+        return []
+
+    corpus = [f"{document.filename}\n{document.text}" for document in documents]
+    try:
+        from sklearn.feature_extraction.text import TfidfVectorizer
+        from sklearn.metrics.pairwise import cosine_similarity
+
+        vectorizer = TfidfVectorizer(
+            analyzer="word",
+            tokenizer=_tokenize_for_vector,
+            token_pattern=None,
+            ngram_range=(1, 2),
+            lowercase=False,
+        )
+        matrix = vectorizer.fit_transform(corpus + [query])
+        similarities = cosine_similarity(matrix[:-1], matrix[-1]).reshape(-1)
+        return [max(0.0, float(score)) for score in similarities]
+    except Exception:
+        return _fallback_vector_scores(query, documents)
+
+
+def _resolve_search_root(root_dir: str, data_root: str | None) -> tuple[Path, Path]:
+    """Resolve a search scope and remove one redundant data-root name if needed."""
+    if not isinstance(root_dir, str) or not root_dir.strip():
+        raise ValueError("root_dir must be a non-empty string")
+
+    search_root, data_root_path = resolve_data_path(root_dir, data_root)
+    if search_root.is_dir():
+        return search_root, data_root_path
+
+    parts = Path(root_dir.replace("\\", "/")).parts
+    if parts and parts[0].casefold() == data_root_path.name.casefold():
+        relative = Path(*parts[1:]) if len(parts) > 1 else Path(".")
+        normalized_root, _ = resolve_data_path(
+            relative.as_posix(),
+            str(data_root_path),
+        )
+        if normalized_root.is_dir():
+            return normalized_root, data_root_path
+
+    raise FileNotFoundError(f"search directory not found: {root_dir}")
+
+
 @tool(
     description="Search local txt and md files for query terms.",
     parameters={
         "query": "Search query.",
-        "root_dir": "Directory relative to data root.",
+        "root_dir": "Subdirectory inside data_root; use docs for project documents.",
         "file_types": "File extensions to search.",
         "top_k": "Maximum number of matches.",
     },
@@ -120,37 +239,68 @@ def local_file_search(
         raise ValueError("query must be a non-empty string")
     if not isinstance(top_k, int) or isinstance(top_k, bool) or top_k <= 0:
         raise ValueError("top_k must be a positive integer")
-    search_root, data_root_path = resolve_data_path(root_dir, data_root)
-    if not search_root.is_dir():
-        raise FileNotFoundError(f"search directory not found: {root_dir}")
+
+    search_root, data_root_path = _resolve_search_root(root_dir, data_root)
+
     extensions = file_types or ["txt", "md"]
     normalized_extensions = {f".{item.lower().lstrip('.')}" for item in extensions}
     if not normalized_extensions.issubset({".txt", ".md"}):
         raise ValueError("local_file_search only supports txt and md")
-    query_info = _build_query(query)
-    phrase = query_info["phrase"]
-    terms = query_info["terms"]
-    results = []
+
+    documents = []
     for path in sorted(search_root.rglob("*")):
         if not path.is_file() or path.suffix.lower() not in normalized_extensions:
             continue
-        text = path.read_text(encoding="utf-8")
-        score, matched_terms, phrase_matched = _score_document(
-            text=text,
-            filename=path.stem,
+        documents.append(_Document(path=path, text=path.read_text(encoding="utf-8"), filename=path.stem))
+
+    query_info = _build_query(query)
+    phrase = query_info["phrase"]
+    terms = query_info["terms"]
+    semantic_scores = _semantic_scores(query, documents)
+    max_semantic_score = max(semantic_scores, default=0.0)
+    semantic_threshold = max(
+        MIN_SEMANTIC_MATCH_THRESHOLD,
+        max_semantic_score * 0.6,
+    )
+
+    results = []
+    for document, semantic_score in zip(documents, semantic_scores):
+        raw_lexical_score, matched_terms, phrase_matched, filename_score = _score_document(
+            text=document.text,
+            filename=document.filename,
             phrase=phrase,
             terms=terms,
         )
-        if not matched_terms:
+        semantic_matched = semantic_score > 0 and semantic_score >= semantic_threshold
+        if not matched_terms and not semantic_matched:
             continue
+
+        lexical_score = _normalize_score(raw_lexical_score)
+        final_score = (LEXICAL_WEIGHT * lexical_score) + (SEMANTIC_WEIGHT * semantic_score)
+        if phrase_matched:
+            final_score += 0.08
+        if filename_score:
+            final_score += min(0.07, filename_score / 100)
+
         results.append(
             {
-                "path": path.relative_to(data_root_path).as_posix(),
-                "score": score,
+                "path": document.path.relative_to(data_root_path).as_posix(),
+                "score": round(final_score, 4),
+                "lexical_score": round(lexical_score, 4),
+                "semantic_score": round(semantic_score, 4),
                 "matched_terms": matched_terms,
                 "phrase_matched": phrase_matched,
-                "snippet": _snippet(text, matched_terms),
+                "semantic_matched": semantic_matched,
+                "snippet": _snippet(document.text, matched_terms),
             }
         )
-    results.sort(key=lambda item: (not item["phrase_matched"], -item["score"], item["path"]))
+
+    results.sort(
+        key=lambda item: (
+            not item["phrase_matched"],
+            -item["score"],
+            not item["semantic_matched"],
+            item["path"],
+        )
+    )
     return {"results": results[:top_k]}

skills/text_to_speech.py

@@ -6,6 +6,8 @@ import re
 import subprocess
 import sys
 import uuid
+import atexit
+import threading
 from pathlib import Path
 from time import perf_counter
 from typing import Any
@@ -18,6 +20,10 @@ from common.path_utils import PROJECT_ROOT
 TTS_CONFIG_PATH = PROJECT_ROOT / "configs" / "tts.yaml"
 TTS_WORKER_PATH = Path(__file__).resolve().with_name("tts_worker.py")
 SUPPORTED_LANGUAGES = {"auto", "zh", "en"}
+TTS_EVENT_PREFIX = "__TTS_WORKER_EVENT__:"
+_WORKER_LOCK = threading.Lock()
+_WORKER_PROCESS: subprocess.Popen[str] | None = None
+_WORKER_KEY: tuple[str, str] | None = None
 
 
 def _require(condition: bool, code: str, message: str) -> None:
@@ -35,8 +41,8 @@ def _resolve_setting(
     default: str | None = None,
 ) -> str | None:
     return (
-        _nonempty_string(configured)
-        or _nonempty_string(os.environ.get(environment_name))
+        _nonempty_string(os.environ.get(environment_name))
+        or _nonempty_string(configured)
         or default
     )
 
@@ -55,7 +61,33 @@ def _resolve_config_path(value: str, *, executable: bool = False) -> str:
     return str(candidate.resolve())
 
 
-def _load_tts_settings() -> dict[str, Any]:
+def _select_voice_profile(voice_config: dict[str, Any], requested_voice: str | None) -> tuple[str, dict[str, Any]]:
+    default_voice_id = _nonempty_string(voice_config.get("default_voice_id")) or "default"
+    selected_voice = (
+        _nonempty_string(requested_voice)
+        or _nonempty_string(os.environ.get("COSYVOICE_VOICE"))
+        or default_voice_id
+    )
+
+    legacy_profile = {
+        key: voice_config.get(key)
+        for key in ("cosyvoice_repo", "model_dir", "prompt_wav", "prompt_text")
+        if key in voice_config
+    }
+    profiles = voice_config.get("voices")
+    if profiles is None:
+        return selected_voice, legacy_profile
+
+    _require(isinstance(profiles, dict), "TTS-1003", "tts.yaml.voice.voices must be an object")
+    _require(
+        selected_voice in profiles and isinstance(profiles[selected_voice], dict),
+        "TTS-1003",
+        f"tts voice does not exist: {selected_voice}",
+    )
+    return selected_voice, {**legacy_profile, **profiles[selected_voice]}
+
+
+def _load_tts_settings(tts_voice: str | None = None) -> dict[str, Any]:
     _require(
         TTS_CONFIG_PATH.is_file(),
         "TTS-1003",
@@ -69,16 +101,18 @@ def _load_tts_settings() -> dict[str, Any]:
     _require(isinstance(runtime, dict), "TTS-1003", "tts.yaml.runtime must be an object")
     _require(isinstance(voice, dict), "TTS-1003", "tts.yaml.voice must be an object")
 
+    voice_id, selected_voice = _select_voice_profile(voice, tts_voice)
+
     python_executable = _resolve_setting(
         runtime.get("python_executable"),
         "COSYVOICE_PYTHON",
         sys.executable,
     )
-    repo = _resolve_setting(voice.get("cosyvoice_repo"), "COSYVOICE_REPO")
-    model_dir = _resolve_setting(voice.get("model_dir"), "COSYVOICE_MODEL")
-    prompt_wav = _resolve_setting(voice.get("prompt_wav"), "COSYVOICE_PROMPT_WAV")
+    repo = _resolve_setting(selected_voice.get("cosyvoice_repo"), "COSYVOICE_REPO")
+    model_dir = _resolve_setting(selected_voice.get("model_dir"), "COSYVOICE_MODEL")
+    prompt_wav = _resolve_setting(selected_voice.get("prompt_wav"), "COSYVOICE_PROMPT_WAV")
     prompt_text = _resolve_setting(
-        voice.get("prompt_text"),
+        selected_voice.get("prompt_text"),
         "COSYVOICE_PROMPT_TEXT",
         "希望你以后能够做的比我还好呦。",
     )
@@ -112,8 +146,23 @@ def _load_tts_settings() -> dict[str, Any]:
 
     timeout_sec = runtime.get("timeout_sec", 600)
     max_total_chars = runtime.get("max_total_chars", 1200)
-    max_chunk_chars = runtime.get("max_chunk_chars", 120)
+    min_chunk_chars = runtime.get("min_chunk_chars", 20)
+    max_chunk_chars = runtime.get("max_chunk_chars", 60)
+    hard_max_chunk_chars = runtime.get("hard_max_chunk_chars", 80)
     silence_sec = runtime.get("inter_chunk_silence_sec", 0.18)
+    persistent_worker = runtime.get("persistent_worker", False)
+    persistent_override = _nonempty_string(os.environ.get("COSYVOICE_PERSISTENT_WORKER"))
+    if persistent_override is not None:
+        normalized_override = persistent_override.lower()
+        _require(
+            normalized_override in {"1", "true", "yes", "on", "0", "false", "no", "off"},
+            "TTS-1003",
+            "COSYVOICE_PERSISTENT_WORKER must be a boolean value",
+        )
+        persistent_worker = normalized_override in {"1", "true", "yes", "on"}
+    load_jit = runtime.get("load_jit", False)
+    fp16 = runtime.get("fp16", False)
+    disable_text_frontend = runtime.get("disable_text_frontend", False)
 
     _require(
         isinstance(timeout_sec, int) and not isinstance(timeout_sec, bool) and 30 <= timeout_sec <= 3600,
@@ -130,40 +179,257 @@ def _load_tts_settings() -> dict[str, Any]:
         "TTS-1003",
         "tts.yaml.runtime.max_chunk_chars must be an integer from 20 to 1000",
     )
+    _require(
+        isinstance(min_chunk_chars, int) and not isinstance(min_chunk_chars, bool) and 1 <= min_chunk_chars <= max_chunk_chars,
+        "TTS-1003",
+        "tts.yaml.runtime.min_chunk_chars must be an integer from 1 to max_chunk_chars",
+    )
+    _require(
+        isinstance(hard_max_chunk_chars, int)
+        and not isinstance(hard_max_chunk_chars, bool)
+        and max_chunk_chars <= hard_max_chunk_chars <= 1000,
+        "TTS-1003",
+        "tts.yaml.runtime.hard_max_chunk_chars must be an integer from max_chunk_chars to 1000",
+    )
     _require(
         isinstance(silence_sec, (int, float)) and not isinstance(silence_sec, bool) and 0 <= float(silence_sec) <= 3,
         "TTS-1003",
         "tts.yaml.runtime.inter_chunk_silence_sec must be between 0 and 3",
     )
+    _require(isinstance(persistent_worker, bool), "TTS-1003", "tts.yaml.runtime.persistent_worker must be boolean")
+    _require(isinstance(load_jit, bool), "TTS-1003", "tts.yaml.runtime.load_jit must be boolean")
+    _require(isinstance(fp16, bool), "TTS-1003", "tts.yaml.runtime.fp16 must be boolean")
+    _require(
+        isinstance(disable_text_frontend, bool),
+        "TTS-1003",
+        "tts.yaml.runtime.disable_text_frontend must be boolean",
+    )
 
     return {
         "python_executable": _resolve_config_path(python_executable, executable=True),
+        "voice_id": voice_id,
         "cosyvoice_repo": str(repo_path),
         "model_dir": str(model_path),
         "prompt_wav": str(prompt_path),
         "prompt_text": prompt_text,
         "timeout_sec": timeout_sec,
         "max_total_chars": max_total_chars,
+        "min_chunk_chars": min_chunk_chars,
         "max_chunk_chars": max_chunk_chars,
+        "hard_max_chunk_chars": hard_max_chunk_chars,
         "inter_chunk_silence_sec": float(silence_sec),
+        "persistent_worker": persistent_worker,
+        "load_jit": load_jit,
+        "fp16": fp16,
+        "disable_text_frontend": disable_text_frontend,
     }
 
 
+def shutdown_tts_worker() -> None:
+    global _WORKER_PROCESS, _WORKER_KEY
+    with _WORKER_LOCK:
+        process, _WORKER_PROCESS, _WORKER_KEY = _WORKER_PROCESS, None, None
+        if process is not None and process.poll() is None:
+            try:
+                process.stdin.write('{"command":"shutdown","id":"shutdown"}\n')
+                process.stdin.flush()
+                process.wait(timeout=5)
+            except Exception:
+                process.terminate()
+
+
+atexit.register(shutdown_tts_worker)
+
+
+def _get_persistent_worker(settings: dict[str, Any], env: dict[str, str]) -> subprocess.Popen[str]:
+    global _WORKER_PROCESS, _WORKER_KEY
+    key = (settings["python_executable"], settings["model_dir"])
+    if _WORKER_PROCESS is not None and _WORKER_PROCESS.poll() is None and _WORKER_KEY == key:
+        return _WORKER_PROCESS
+    if _WORKER_PROCESS is not None and _WORKER_PROCESS.poll() is None:
+        _WORKER_PROCESS.terminate()
+    process = subprocess.Popen(
+        [settings["python_executable"], str(TTS_WORKER_PATH), "--serve"],
+        stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=None,
+        text=True, bufsize=1, env=env,
+    )
+    while True:
+        line = process.stdout.readline()
+        if not line:
+            raise SkillError("TTS-1007", "persistent CosyVoice worker exited during startup")
+        if line.strip() == "__TTS_WORKER_READY__":
+            break
+    _WORKER_PROCESS, _WORKER_KEY = process, key
+    return process
+
+
+def _run_persistent(settings: dict[str, Any], env: dict[str, str], request_path: Path, response_path: Path) -> int:
+    with _WORKER_LOCK:
+        process = _get_persistent_worker(settings, env)
+        request_id = uuid.uuid4().hex
+        process.stdin.write(json.dumps({"id": request_id, "request": str(request_path), "response": str(response_path)}) + "\n")
+        process.stdin.flush()
+        marker = f"__TTS_WORKER_DONE__:{request_id}:"
+        while True:
+            line = process.stdout.readline()
+            if not line:
+                raise SkillError("TTS-1007", "persistent CosyVoice worker exited during synthesis")
+            if line.strip().startswith(marker):
+                return int(line.strip().rsplit(":", 1)[1])
+
+
+def _run_persistent_stream(
+    settings: dict[str, Any],
+    env: dict[str, str],
+    request_path: Path,
+    response_path: Path,
+):
+    with _WORKER_LOCK:
+        process = _get_persistent_worker(settings, env)
+        request_id = uuid.uuid4().hex
+        process.stdin.write(json.dumps({"id": request_id, "request": str(request_path), "response": str(response_path)}) + "\n")
+        process.stdin.flush()
+        event_marker = f"{TTS_EVENT_PREFIX}{request_id}:"
+        done_marker = f"__TTS_WORKER_DONE__:{request_id}:"
+        while True:
+            line = process.stdout.readline()
+            if not line:
+                raise SkillError("TTS-1007", "persistent CosyVoice worker exited during stream synthesis")
+            stripped = line.strip()
+            if stripped.startswith(event_marker):
+                yield json.loads(stripped[len(event_marker):])
+            elif stripped.startswith(done_marker):
+                returncode = int(stripped.rsplit(":", 1)[1])
+                if returncode != 0:
+                    response = read_json(response_path) if response_path.is_file() else {}
+                    raise SkillError("TTS-1006", _worker_error_message(response, None))
+                return
+
+
 def _detect_language(text: str) -> str:
     # 对最终回答使用保守检测：只要出现中文字符，就选中文 zero-shot 音色合成。
     return "zh" if re.search(r"[\u4e00-\u9fff]", text) else "en"
 
 
-def _split_text(text: str, max_chunk_chars: int) -> list[str]:
-    normalized = re.sub(r"\s+", " ", text).strip()
+def _markdown_to_speech_text(text: str) -> str:
+    """Convert common Markdown into natural speech and retain block pauses."""
+    text = text.replace("\r\n", "\n").replace("\r", "\n")
+    text = re.sub(r"```[^\n]*\n.*?```", "\n代码内容已省略。\n", text, flags=re.DOTALL)
+    text = re.sub(r"!\[([^]]*)\]\([^)]*\)", r"\1", text)
+    text = re.sub(r"\[([^]]+)\]\([^)]*\)", r"\1", text)
+    text = re.sub(r"<https?://[^>]+>", "", text)
+    text = re.sub(r"`([^`]+)`", r"\1", text)
+
+    spoken_lines: list[str] = []
+    for raw_line in text.split("\n"):
+        line = raw_line.strip()
+        if not line:
+            if spoken_lines and spoken_lines[-1] != "":
+                spoken_lines.append("")
+            continue
+        if re.fullmatch(r"(?:[-*_]\s*){3,}", line):
+            continue
+        if re.fullmatch(r"\|?[\s:|-]+\|?", line) and "-" in line:
+            continue
+
+        line = re.sub(r"^#{1,6}\s+", "", line)
+        line = re.sub(r"^>+\s*", "", line)
+        line = re.sub(r"^(?:[-+*]|\d+[.)])\s+", "", line)
+        if "|" in line:
+            cells = [cell.strip() for cell in line.strip("|").split("|") if cell.strip()]
+            line = "，".join(cells)
+        line = re.sub(r"[*_~]", "", line)
+        line = re.sub(r"\s+", " ", line).strip()
+        if not line:
+            continue
+        if not re.search(r"[。！？!?；;：:]$", line):
+            line += "。"
+        spoken_lines.append(line)
+
+    return "\n".join(spoken_lines).strip()
+
+
+def _validate_tts_input(
+    text: str,
+    language: str,
+    tts_voice: str | None = None,
+) -> tuple[dict[str, Any], str, str, list[str]]:
+    _require(isinstance(text, str) and text.strip(), "TTS-1001", "text must be a non-empty string")
+    _require(isinstance(language, str), "TTS-1002", "language must be auto, zh, or en")
+
+    normalized_language = language.strip().lower()
+    _require(
+        normalized_language in SUPPORTED_LANGUAGES,
+        "TTS-1002",
+        "language must be auto, zh, or en",
+    )
+
+    if tts_voice is not None:
+        _require(isinstance(tts_voice, str) and bool(tts_voice.strip()), "TTS-1003", "tts_voice must be a non-empty string")
+        tts_voice = tts_voice.strip()
+
+    settings = _load_tts_settings(tts_voice)
+    normalized_text = _markdown_to_speech_text(text)
+    _require(
+        len(normalized_text) <= settings["max_total_chars"],
+        "TTS-1001",
+        f"text exceeds configured max_total_chars={settings['max_total_chars']}",
+    )
+
+    resolved_language = _detect_language(normalized_text) if normalized_language == "auto" else normalized_language
+    chunks = _split_text(
+        normalized_text,
+        settings["max_chunk_chars"],
+        settings["min_chunk_chars"],
+        settings["hard_max_chunk_chars"],
+    )
+    _require(bool(chunks), "TTS-1001", "text contains no speakable content")
+    return settings, normalized_text, resolved_language, chunks
+
+
+def _worker_settings(settings: dict[str, Any]) -> dict[str, Any]:
+    return {
+        "voice_id": settings["voice_id"],
+        "cosyvoice_repo": settings["cosyvoice_repo"],
+        "model_dir": settings["model_dir"],
+        "prompt_wav": settings["prompt_wav"],
+        "prompt_text": settings["prompt_text"],
+        "inter_chunk_silence_sec": settings["inter_chunk_silence_sec"],
+        "load_jit": settings["load_jit"],
+        "fp16": settings["fp16"],
+        "disable_text_frontend": settings["disable_text_frontend"],
+    }
+
+
+def _worker_env(settings: dict[str, Any]) -> dict[str, str]:
+    env = os.environ.copy()
+    env.update(
+        {
+            "COSYVOICE_REPO": settings["cosyvoice_repo"],
+            "COSYVOICE_MODEL": settings["model_dir"],
+            "COSYVOICE_PROMPT_WAV": settings["prompt_wav"],
+        }
+    )
+    return env
+
+
+def _split_text_legacy_unused(
+    text: str,
+    max_chunk_chars: int,
+    min_chunk_chars: int = 20,
+    hard_max_chunk_chars: int | None = None,
+) -> list[str]:
+    normalized = _markdown_to_speech_text(text)
     if len(normalized) <= max_chunk_chars:
         return [normalized]
+    hard_limit = hard_max_chunk_chars or max_chunk_chars
 
     sentences = [
         piece.strip()
-        for piece in re.split(r"(?<=[。！？!?；;：:])\s*|\n+", normalized)
+        for piece in re.split(r"(?<=[。！？!?；;])\s*|\n+", normalized)
         if piece.strip()
     ]
+    soft_breaks = re.compile(r"(?<=[，,、：:])\s*")
     chunks: list[str] = []
     current = ""
 
@@ -174,18 +440,77 @@ def _split_text(text: str, max_chunk_chars: int) -> list[str]:
             current = ""
 
     for sentence in sentences:
+        pieces = [sentence]
         if len(sentence) > max_chunk_chars:
-            flush()
-            for start in range(0, len(sentence), max_chunk_chars):
-                chunks.append(sentence[start : start + max_chunk_chars])
-            continue
+            pieces = [piece for piece in soft_breaks.split(sentence) if piece]
 
-        candidate = sentence if not current else f"{current} {sentence}"
-        if len(candidate) <= max_chunk_chars:
-            current = candidate
-        else:
-            flush()
-            current = sentence
+        expanded: list[str] = []
+        for piece in pieces:
+            if len(piece) <= hard_limit:
+                expanded.append(piece)
+            else:
+                expanded.extend(piece[start : start + hard_limit] for start in range(0, len(piece), hard_limit))
+
+        for piece in expanded:
+            candidate = piece if not current else f"{current} {piece}"
+            if len(candidate) <= max_chunk_chars or len(current) < min_chunk_chars:
+                current = candidate
+            else:
+                flush()
+                current = piece
+
+    flush()
+    return chunks
+
+
+def _split_text(
+    text: str,
+    max_chunk_chars: int,
+    min_chunk_chars: int = 20,
+    hard_max_chunk_chars: int | None = None,
+) -> list[str]:
+    normalized = _markdown_to_speech_text(text)
+    if not normalized:
+        return []
+
+    hard_limit = hard_max_chunk_chars or max_chunk_chars
+    sentence_breaks = r"(?<=[\u3002\uff01\uff1f!?；;])\s*|\n+"
+    soft_breaks = re.compile(r"(?<=[\uff0c,\u3001\uff1a:])\s*")
+    sentences = [piece.strip() for piece in re.split(sentence_breaks, normalized) if piece.strip()]
+    chunks: list[str] = []
+    current = ""
+
+    def flush() -> None:
+        nonlocal current
+        if current:
+            chunks.append(current)
+            current = ""
+
+    for sentence in sentences:
+        pieces = [sentence]
+        if len(sentence) > max_chunk_chars:
+            pieces = [piece.strip() for piece in soft_breaks.split(sentence) if piece.strip()]
+
+        expanded: list[str] = []
+        for piece in pieces:
+            if len(piece) <= hard_limit:
+                expanded.append(piece)
+            else:
+                expanded.extend(piece[start : start + hard_limit] for start in range(0, len(piece), hard_limit))
+
+        for piece in expanded:
+            candidate = piece if not current else f"{current} {piece}"
+            if len(candidate) <= max_chunk_chars and len(current) < min_chunk_chars:
+                current = candidate
+                continue
+            if current and len(current) >= min_chunk_chars:
+                flush()
+                current = piece
+            elif len(candidate) <= hard_limit:
+                current = candidate
+            else:
+                flush()
+                current = piece
 
     flush()
     return chunks
@@ -206,13 +531,13 @@ def _safe_output_path(output_dir: str | None, output_filename: str | None) -> Pa
     return candidate
 
 
-def _worker_error_message(result: dict[str, Any], completed: subprocess.CompletedProcess[str]) -> str:
+def _worker_error_message(result: dict[str, Any], completed: subprocess.CompletedProcess[str] | None) -> str:
     worker_error = result.get("error") if isinstance(result, dict) else None
     if isinstance(worker_error, dict):
         detail = worker_error.get("message")
         if isinstance(detail, str) and detail.strip():
             return detail.strip()
-    stderr = (completed.stderr or "").strip()
+    stderr = (completed.stderr or "").strip() if completed is not None else ""
     return stderr[-2000:] if stderr else "CosyVoice worker failed without an error message"
 
 
@@ -220,32 +545,12 @@ def text_to_speech(
     text: str,
     language: str = "auto",
     output_filename: str | None = None,
+    tts_voice: str | None = None,
     *,
     output_dir: str | None = None,
 ) -> dict:
     """Run CosyVoice in its own configured Python environment and return one WAV artifact."""
-    _require(isinstance(text, str) and text.strip(), "TTS-1001", "text must be a non-empty string")
-    _require(isinstance(language, str), "TTS-1002", "language must be auto, zh, or en")
-
-    normalized_language = language.strip().lower()
-    _require(
-        normalized_language in SUPPORTED_LANGUAGES,
-        "TTS-1002",
-        "language must be auto, zh, or en",
-    )
-
-    settings = _load_tts_settings()
-    normalized_text = re.sub(r"\s+", " ", text).strip()
-    _require(
-        len(normalized_text) <= settings["max_total_chars"],
-        "TTS-1001",
-        f"text exceeds configured max_total_chars={settings['max_total_chars']}",
-    )
-
-    resolved_language = _detect_language(normalized_text) if normalized_language == "auto" else normalized_language
-    chunks = _split_text(normalized_text, settings["max_chunk_chars"])
-    _require(bool(chunks), "TTS-1001", "text contains no speakable content")
-
+    settings, _normalized_text, resolved_language, chunks = _validate_tts_input(text, language, tts_voice)
     output_path = _safe_output_path(output_dir, output_filename)
     request_path = output_path.with_name(f".{output_path.stem}_{uuid.uuid4().hex}_request.json")
     response_path = output_path.with_name(f".{output_path.stem}_{uuid.uuid4().hex}_response.json")
@@ -254,43 +559,25 @@ def text_to_speech(
         "chunks": chunks,
         "language": resolved_language,
         "output_path": str(output_path),
-        "settings": {
-            "cosyvoice_repo": settings["cosyvoice_repo"],
-            "model_dir": settings["model_dir"],
-            "prompt_wav": settings["prompt_wav"],
-            "prompt_text": settings["prompt_text"],
-            "inter_chunk_silence_sec": settings["inter_chunk_silence_sec"],
-        },
+        "settings": _worker_settings(settings),
     }
 
     write_json(request, request_path)
-    env = os.environ.copy()
-    env.update(
-        {
-            "COSYVOICE_REPO": settings["cosyvoice_repo"],
-            "COSYVOICE_MODEL": settings["model_dir"],
-            "COSYVOICE_PROMPT_WAV": settings["prompt_wav"],
-        }
-    )
+    env = _worker_env(settings)
 
     started = perf_counter()
     try:
         try:
-            completed = subprocess.run(
-                [
-                    settings["python_executable"],
-                    str(TTS_WORKER_PATH),
-                    "--request",
-                    str(request_path),
-                    "--response",
-                    str(response_path),
-                ],
-                capture_output=True,
-                text=True,
-                env=env,
-                timeout=settings["timeout_sec"],
-                check=False,
-            )
+            if settings["persistent_worker"]:
+                completed = None
+                returncode = _run_persistent(settings, env, request_path, response_path)
+            else:
+                completed = subprocess.run(
+                    [settings["python_executable"], str(TTS_WORKER_PATH), "--request", str(request_path), "--response", str(response_path)],
+                    capture_output=True, text=True, env=env,
+                    timeout=settings["timeout_sec"], check=False,
+                )
+                returncode = completed.returncode
         except FileNotFoundError as exc:
             raise SkillError(
                 "TTS-1004",
@@ -310,7 +597,7 @@ def text_to_speech(
         response = read_json(response_path)
         _require(isinstance(response, dict), "TTS-1007", "CosyVoice worker response must be an object")
 
-        if completed.returncode != 0 or response.get("status") != "success":
+        if returncode != 0 or response.get("status") != "success":
             raise SkillError("TTS-1006", _worker_error_message(response, completed))
 
         output = response.get("output")
@@ -325,6 +612,7 @@ def text_to_speech(
         return {
             **output,
             "language": resolved_language,
+            "voice_id": settings["voice_id"],
             "chunk_count": len(chunks),
             "controller_latency_ms": round((perf_counter() - started) * 1000, 3),
             "worker_python": settings["python_executable"],
@@ -335,3 +623,68 @@ def text_to_speech(
                 temporary.unlink(missing_ok=True)
             except OSError:
                 pass
+
+
+def stream_text_to_speech(
+    text: str,
+    language: str = "auto",
+    output_filename: str | None = None,
+    tts_voice: str | None = None,
+    *,
+    output_dir: str | None = None,
+):
+    """Yield one event per synthesized chunk while keeping the CosyVoice worker loaded."""
+    settings, _normalized_text, resolved_language, chunks = _validate_tts_input(text, language, tts_voice)
+    output_path = _safe_output_path(output_dir, output_filename)
+    request_path = output_path.with_name(f".{output_path.stem}_{uuid.uuid4().hex}_stream_request.json")
+    response_path = output_path.with_name(f".{output_path.stem}_{uuid.uuid4().hex}_stream_response.json")
+    chunk_dir = output_path.with_name(f"{output_path.stem}_chunks_{uuid.uuid4().hex[:8]}")
+
+    request = {
+        "stream": True,
+        "chunks": chunks,
+        "language": resolved_language,
+        "output_path": str(output_path),
+        "chunk_dir": str(chunk_dir),
+        "settings": _worker_settings(settings),
+    }
+    write_json(request, request_path)
+    env = _worker_env(settings)
+
+    started = perf_counter()
+    try:
+        if settings["persistent_worker"]:
+            event_source = _run_persistent_stream(settings, env, request_path, response_path)
+        else:
+            process = subprocess.Popen(
+                [settings["python_executable"], str(TTS_WORKER_PATH), "--request", str(request_path), "--response", str(response_path)],
+                stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=env, bufsize=1,
+            )
+
+            def one_shot_events():
+                assert process.stdout is not None
+                request_marker = f"{TTS_EVENT_PREFIX}oneshot:"
+                for line in process.stdout:
+                    stripped = line.strip()
+                    if stripped.startswith(request_marker):
+                        payload_text = stripped[len(request_marker):]
+                        yield json.loads(payload_text)
+                _, stderr = process.communicate(timeout=5)
+                if process.returncode != 0:
+                    response = read_json(response_path) if response_path.is_file() else {}
+                    message = _worker_error_message(response, None) or stderr
+                    raise SkillError("TTS-1006", message)
+            event_source = one_shot_events()
+
+        for event in event_source:
+            event["language"] = resolved_language
+            event["voice_id"] = settings["voice_id"]
+            event["controller_latency_ms"] = round((perf_counter() - started) * 1000, 3)
+            event["worker_python"] = settings["python_executable"]
+            yield event
+    finally:
+        for temporary in (request_path, response_path):
+            try:
+                temporary.unlink(missing_ok=True)
+            except OSError:
+                pass

skills/tts_worker.py

@@ -4,10 +4,14 @@ import argparse
 import json
 import sys
 import traceback
+import warnings
+import uuid
 from pathlib import Path
 from time import perf_counter
 from typing import Any
 
+TTS_EVENT_PREFIX = "__TTS_WORKER_EVENT__:"
+
 
 def _write_response(path: Path, payload: dict[str, Any]) -> None:
     path.parent.mkdir(parents=True, exist_ok=True)
@@ -19,7 +23,89 @@ def _require(value: bool, message: str) -> None:
         raise ValueError(message)
 
 
-def _synthesize(request: dict[str, Any]) -> dict[str, Any]:
+_MODEL_CACHE: dict[tuple[str, bool, bool, bool], Any] = {}
+
+
+def _quiet_known_dependency_warnings() -> None:
+    warnings.filterwarnings("ignore", message=r".*pkg_resources is deprecated.*")
+    warnings.filterwarnings("ignore", message=r".*LoRACompatibleLinear.*deprecated.*")
+    warnings.filterwarnings("ignore", message=r".*torch\.nn\.utils\.weight_norm is deprecated.*")
+    warnings.filterwarnings("ignore", message=r".*Sliding Window Attention is enabled.*")
+    warnings.filterwarnings("ignore", message=r".*Torch was not compiled with flash attention.*")
+
+
+def _patch_onnxruntime_providers() -> None:
+    import onnxruntime
+
+    if getattr(onnxruntime, "_agent_provider_patch", False):
+        return
+    available = set(onnxruntime.get_available_providers())
+    original_session = onnxruntime.InferenceSession
+
+    def patched_session(*args, **kwargs):
+        providers = kwargs.get("providers")
+        if providers is not None:
+            filtered = []
+            for provider in providers:
+                provider_name = provider[0] if isinstance(provider, tuple) else provider
+                if provider_name in available:
+                    filtered.append(provider)
+            if not filtered and "CPUExecutionProvider" in available:
+                filtered = ["CPUExecutionProvider"]
+            kwargs["providers"] = filtered
+        return original_session(*args, **kwargs)
+
+    onnxruntime.InferenceSession = patched_session
+    onnxruntime._agent_provider_patch = True
+
+
+def _get_model(repo: Path, model_dir: Path, settings: dict[str, Any]) -> tuple[Any, float, bool]:
+    key = (
+        str(model_dir),
+        bool(settings.get("load_jit", False)),
+        bool(settings.get("fp16", False)),
+        bool(settings.get("disable_text_frontend", False)),
+    )
+    if key in _MODEL_CACHE:
+        return _MODEL_CACHE[key], 0.0, True
+    repo_text = str(repo)
+    matcha_text = str(repo / "third_party" / "Matcha-TTS")
+    if repo_text not in sys.path:
+        sys.path.insert(0, repo_text)
+    if matcha_text not in sys.path:
+        sys.path.insert(0, matcha_text)
+    # WeText lazily downloads its normalizer model from ModelScope.  On an
+    # offline inference server, make the optional import fail so CosyVoice
+    # falls back to its built-in lightweight normalization instead of waiting
+    # through repeated 60-second network timeouts.
+    if settings.get("disable_text_frontend", False):
+        sys.modules["wetext"] = None
+    _quiet_known_dependency_warnings()
+    _patch_onnxruntime_providers()
+    from cosyvoice.cli.cosyvoice import AutoModel
+    started = perf_counter()
+    model = AutoModel(model_dir=str(model_dir), load_jit=key[1], fp16=key[2])
+    elapsed = round((perf_counter() - started) * 1000, 3)
+    _MODEL_CACHE[key] = model
+    return model, elapsed, False
+
+
+def _synthesize_one(cosyvoice: Any, language: str, text: str, prompt_text: str, prompt_wav: Path):
+    if language == "zh":
+        return cosyvoice.inference_zero_shot(
+            text,
+            prompt_text,
+            str(prompt_wav),
+            stream=False,
+        )
+    return cosyvoice.inference_cross_lingual(
+        f"<|en|>{text}",
+        str(prompt_wav),
+        stream=False,
+    )
+
+
+def _synthesize(request: dict[str, Any], event_callback=None) -> dict[str, Any]:
     settings = request.get("settings")
     _require(isinstance(settings, dict), "request.settings must be an object")
     chunks = request.get("chunks")
@@ -42,43 +128,46 @@ def _synthesize(request: dict[str, Any]) -> dict[str, Any]:
     _require(isinstance(prompt_text, str) and prompt_text.strip(), "prompt_text must be a non-empty string")
     _require(isinstance(silence_sec, (int, float)) and 0 <= float(silence_sec) <= 3, "inter_chunk_silence_sec must be between 0 and 3")
 
-    repo_text = str(repo)
-    matcha_text = str(repo / "third_party" / "Matcha-TTS")
-    if repo_text not in sys.path:
-        sys.path.insert(0, repo_text)
-    if matcha_text not in sys.path:
-        sys.path.insert(0, matcha_text)
-
     import torch
     import torchaudio
-    from cosyvoice.cli.cosyvoice import AutoModel
-
-    model_started = perf_counter()
-    cosyvoice = AutoModel(model_dir=str(model_dir))
-    model_load_latency_ms = round((perf_counter() - model_started) * 1000, 3)
+    cosyvoice, model_load_latency_ms, model_cache_hit = _get_model(repo, model_dir, settings)
     sample_rate = int(cosyvoice.sample_rate)
 
     synth_started = perf_counter()
     pieces = []
-    for index, text in enumerate(chunks):
-        if language == "zh":
-            generator = cosyvoice.inference_zero_shot(
-                text,
-                prompt_text,
-                str(prompt_wav),
-                stream=False,
-            )
-        else:
-            generator = cosyvoice.inference_cross_lingual(
-                f"<|en|>{text}",
-                str(prompt_wav),
-                stream=False,
-            )
+    chunk_outputs = []
+    stream = bool(request.get("stream"))
+    chunk_dir = Path(request.get("chunk_dir") or output_path.with_name(f"{output_path.stem}_chunks")).expanduser().resolve()
+    if stream:
+        chunk_dir.mkdir(parents=True, exist_ok=True)
 
+    for index, text in enumerate(chunks):
+        chunk_started = perf_counter()
+        generator = _synthesize_one(cosyvoice, language, text, prompt_text, prompt_wav)
         chunk_audio = [item["tts_speech"] for item in generator]
         _require(bool(chunk_audio), f"CosyVoice returned no audio for chunk {index + 1}")
         audio = torch.cat(chunk_audio, dim=1)
         pieces.append(audio)
+        chunk_duration_sec = round(float(audio.shape[1]) / sample_rate, 3)
+        chunk_latency_ms = round((perf_counter() - chunk_started) * 1000, 3)
+
+        if stream:
+            chunk_path = chunk_dir / f"{output_path.stem}_chunk_{index + 1:03d}.wav"
+            torchaudio.save(str(chunk_path), audio.detach().cpu(), sample_rate)
+            event = {
+                "type": "audio_chunk",
+                "index": index + 1,
+                "chunk_count": len(chunks),
+                "text": text,
+                "audio_path": str(chunk_path),
+                "sample_rate": sample_rate,
+                "duration_sec": chunk_duration_sec,
+                "latency_ms": chunk_latency_ms,
+                "model_cache_hit": model_cache_hit,
+            }
+            chunk_outputs.append(event)
+            if event_callback is not None:
+                event_callback(event)
 
         if index < len(chunks) - 1 and float(silence_sec) > 0:
             silence_samples = int(round(sample_rate * float(silence_sec)))
@@ -86,54 +175,106 @@ def _synthesize(request: dict[str, Any]) -> dict[str, Any]:
 
     merged = torch.cat(pieces, dim=1).detach().cpu()
     output_path.parent.mkdir(parents=True, exist_ok=True)
-    torchaudio.save(str(output_path), merged, sample_rate)
+    if not stream:
+        torchaudio.save(str(output_path), merged, sample_rate)
     synthesis_latency_ms = round((perf_counter() - synth_started) * 1000, 3)
     duration_sec = round(float(merged.shape[1]) / sample_rate, 3)
 
     return {
-        "audio_path": str(output_path),
+        "audio_path": None if stream else str(output_path),
+        "chunk_dir": str(chunk_dir) if stream else None,
         "sample_rate": sample_rate,
         "duration_sec": duration_sec,
         "chunk_count": len(chunks),
+        "chunks": chunk_outputs if stream else None,
+        "voice": {
+            "id": settings.get("voice_id"),
+            "prompt_wav": str(prompt_wav),
+        },
         "model": {
             "name": "CosyVoice2-0.5B",
             "model_dir": str(model_dir),
             "model_load_latency_ms": model_load_latency_ms,
+            "model_cache_hit": model_cache_hit,
             "synthesis_latency_ms": synthesis_latency_ms,
         },
     }
 
 
-def main(argv: list[str] | None = None) -> int:
-    parser = argparse.ArgumentParser(description="CosyVoice subprocess worker for the B2 TTS Skill.")
-    parser.add_argument("--request", required=True)
-    parser.add_argument("--response", required=True)
-    args = parser.parse_args(argv)
-
-    request_path = Path(args.request).resolve()
-    response_path = Path(args.response).resolve()
+def _run_request(request_path: Path, response_path: Path) -> int:
     try:
         request = json.loads(request_path.read_text(encoding="utf-8"))
         if not isinstance(request, dict):
             raise ValueError("request JSON must be an object")
-        output = _synthesize(request)
+        event_callback = None
+        if request.get("stream"):
+            def emit(event: dict[str, Any]) -> None:
+                print(
+                    TTS_EVENT_PREFIX + "oneshot:" + json.dumps(event, ensure_ascii=False),
+                    flush=True,
+                )
+            event_callback = emit
+        output = _synthesize(request, event_callback=event_callback)
         _write_response(response_path, {"status": "success", "output": output, "error": None})
         return 0
     except Exception as exc:
-        _write_response(
-            response_path,
-            {
-                "status": "error",
-                "output": None,
-                "error": {
-                    "type": type(exc).__name__,
-                    "message": str(exc),
-                    "traceback": traceback.format_exc(limit=5),
-                },
-            },
-        )
+        _write_response(response_path, {"status": "error", "output": None, "error": {
+            "type": type(exc).__name__, "message": str(exc), "traceback": traceback.format_exc(limit=5)}})
         return 1
 
 
+def _serve() -> int:
+    print("__TTS_WORKER_READY__", flush=True)
+    for line in sys.stdin:
+        request_id = "unknown"
+        status = 1
+        try:
+            command = json.loads(line)
+            request_id = str(command.get("id") or uuid.uuid4().hex)
+            if command.get("command") == "shutdown":
+                status = 0
+                return 0
+            request_path = Path(command["request"]).resolve()
+            response_path = Path(command["response"]).resolve()
+            request = json.loads(request_path.read_text(encoding="utf-8"))
+            if isinstance(request, dict) and request.get("stream"):
+                def emit(event: dict[str, Any]) -> None:
+                    print(
+                        TTS_EVENT_PREFIX + request_id + ":" + json.dumps(event, ensure_ascii=False),
+                        flush=True,
+                    )
+
+                try:
+                    output = _synthesize(request, event_callback=emit)
+                    _write_response(response_path, {"status": "success", "output": output, "error": None})
+                    status = 0
+                except Exception as exc:
+                    _write_response(response_path, {"status": "error", "output": None, "error": {
+                        "type": type(exc).__name__, "message": str(exc), "traceback": traceback.format_exc(limit=5)}})
+                    status = 1
+            else:
+                status = _run_request(request_path, response_path)
+        finally:
+            print(f"__TTS_WORKER_DONE__:{request_id}:{status}", flush=True)
+    return 0
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="CosyVoice subprocess worker for the B2 TTS Skill.")
+    parser.add_argument("--request")
+    parser.add_argument("--response")
+    parser.add_argument("--serve", action="store_true")
+    args = parser.parse_args(argv)
+
+    if args.serve:
+        return _serve()
+    if not args.request or not args.response:
+        parser.error("--request and --response are required unless --serve is used")
+
+    request_path = Path(args.request).resolve()
+    response_path = Path(args.response).resolve()
+    return _run_request(request_path, response_path)
+
+
 if __name__ == "__main__":
     raise SystemExit(main())

tests/test_b1_tool_failures.py

@@ -0,0 +1,276 @@
+from __future__ import annotations
+
+import json
+import sys
+import tempfile
+import types
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "code"))
+
+import b1_agent_runtime
+from b1_agent_runtime import (
+    _append_plan_step_instruction,
+    _failure_signature,
+    _replan_instruction,
+    _skill_output,
+    _tool_failures,
+)
+
+
+def tool_message(name: str, status: str, *, output=None, error=None) -> dict:
+    result = {
+        "skill_name": name,
+        "status": status,
+        "input": {},
+        "output": output,
+        "error": error,
+        "latency_ms": 1.0,
+    }
+    return {
+        "role": "tool",
+        "tool_call_id": f"call_{name}",
+        "name": name,
+        "content": json.dumps(result),
+        "status": status,
+    }
+
+
+class ToolFailureTests(unittest.TestCase):
+    def test_future_skill_error_is_detected_without_name_registration(self) -> None:
+        messages = [
+            tool_message(
+                "future_skill",
+                "error",
+                error={"type": "RuntimeError", "message": "boom"},
+            )
+        ]
+
+        failures = _tool_failures(messages)
+
+        self.assertEqual(failures[0]["tool"], "future_skill")
+        self.assertEqual(failures[0]["reason"], "skill_error")
+        self.assertIn("future_skill", _replan_instruction(failures))
+
+    def test_all_failures_in_a_multi_tool_round_are_reported(self) -> None:
+        messages = [
+            tool_message("ok_skill", "success", output={"value": 1}),
+            tool_message("bad_skill", "error", error={"message": "bad input"}),
+            {"name": "broken_protocol", "content": "not-json", "status": "success"},
+        ]
+
+        failures = _tool_failures(messages)
+
+        self.assertEqual([item["tool"] for item in failures], ["bad_skill", "broken_protocol"])
+
+    def test_empty_local_search_remains_an_unusable_result(self) -> None:
+        failures = _tool_failures(
+            [tool_message("local_file_search", "success", output={"results": []})]
+        )
+
+        self.assertEqual(failures[0]["reason"], "empty_result")
+
+    def test_failure_signature_ignores_call_id_but_tracks_input(self) -> None:
+        first = _tool_failures(
+            [tool_message("local_file_search", "success", output={"results": []})]
+        )[0]
+        second = dict(first, tool_call_id="another_call")
+        changed = dict(first, input={"query": "different"})
+
+        self.assertEqual(_failure_signature(first), _failure_signature(second))
+        self.assertNotEqual(_failure_signature(first), _failure_signature(changed))
+
+    def test_empty_search_signature_ignores_result_window_escalation(self) -> None:
+        first = {
+            "tool": "local_file_search",
+            "reason": "empty_result",
+            "input": {"query": "中北大学", "top_k": 10},
+            "error": None,
+        }
+        second = dict(first, input={"query": "中北大学", "top_k": 100})
+
+        self.assertEqual(_failure_signature(first), _failure_signature(second))
+
+    def test_shared_output_parser_replaces_skill_specific_helpers(self) -> None:
+        output = _skill_output(
+            [tool_message("speech_to_text", "success", output={"text": "hello"})],
+            "speech_to_text",
+            ("text",),
+        )
+
+        self.assertEqual(output, {"text": "hello"})
+
+    def test_plan_step_instruction_is_transient_user_suffix(self) -> None:
+        messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "task"},
+        ]
+
+        call_messages = _append_plan_step_instruction(messages, "PLAN EXECUTION MODE")
+
+        self.assertEqual(messages[-1]["content"], "task")
+        self.assertEqual(call_messages[-1]["role"], "user")
+        self.assertTrue(call_messages[-1]["content"].startswith("[PLAN_STEP_INSTRUCTION]"))
+        self.assertNotIn("PLAN EXECUTION MODE", call_messages[0]["content"])
+
+    def test_run_agent_replans_after_an_unknown_future_skill_fails(self) -> None:
+        first_decision = {
+            "status": "success",
+            "execution_mode": "direct",
+            "execution_plan": {"use_plan": False, "steps": []},
+            "ai_message": {
+                "role": "assistant",
+                "content": "",
+                "tool_calls": [
+                    {"id": "call_future", "name": "future_skill", "args": {}}
+                ],
+            },
+        }
+        recovery_decision = {
+            "status": "success",
+            "execution_mode": "direct",
+            "execution_plan": {"use_plan": False, "steps": []},
+            "ai_message": {
+                "role": "assistant",
+                "content": "recovered",
+                "tool_calls": [],
+            },
+        }
+
+        def execute_tool_calls(calls, *_args):
+            return [
+                tool_message(
+                    calls[0]["name"],
+                    "error",
+                    error={"type": "RuntimeError", "message": "future failure"},
+                )
+            ]
+
+        b3 = types.ModuleType("b3_tool_layer")
+        b3.execute_tool_calls = execute_tool_calls
+        b3.get_tools_schema = lambda *_args: []
+        b5 = types.ModuleType("b5_memory")
+        b5.load_memory = lambda *_args: {"status": "success", "selected_memory_docs": []}
+
+        with tempfile.TemporaryDirectory() as temp_dir:
+            root = Path(temp_dir)
+            prompt = root / "prompt.txt"
+            prompt.write_text("system", encoding="utf-8")
+            runtime_input = root / "input.json"
+            runtime_input.write_text(
+                json.dumps(
+                    {
+                        "execution_mode": "integrated",
+                        "input_type": "text",
+                        "conversation_id": "replan-test",
+                        "system_prompt_path": str(prompt),
+                        "toolset": "test_tools",
+                        "max_turns": 2,
+                        "save_memory": "none",
+                        "user_input": "run a future skill",
+                    }
+                ),
+                encoding="utf-8",
+            )
+
+            with (
+                patch.dict(sys.modules, {"b3_tool_layer": b3, "b5_memory": b5}),
+                patch.object(
+                    b1_agent_runtime,
+                    "generate_agent_decision",
+                    side_effect=[first_decision, recovery_decision],
+                ),
+            ):
+                result = b1_agent_runtime.run_agent(
+                    str(runtime_input),
+                    str(root / "tools.yaml"),
+                    str(root / "memory.yaml"),
+                    str(root / "model.yaml"),
+                    str(root / "output"),
+                    "mock",
+                )
+
+            trace = json.loads((root / "output" / "trace.json").read_text(encoding="utf-8"))
+            self.assertEqual(result["final_answer"], "recovered")
+            self.assertEqual(trace["planning_events"][1]["stage"], "replan_after_tool_failure")
+            self.assertEqual(trace["planning_events"][1]["trigger_tools"], ["future_skill"])
+
+    def test_run_agent_passes_runtime_tts_voice_to_output_tool(self) -> None:
+        direct_decision = {
+            "status": "success",
+            "execution_mode": "direct",
+            "execution_plan": {"use_plan": False, "steps": []},
+            "ai_message": {
+                "role": "assistant",
+                "content": "final answer",
+                "tool_calls": [],
+            },
+        }
+        captured_calls = []
+
+        def execute_tool_calls(calls, *_args):
+            captured_calls.extend(calls)
+            return [
+                tool_message(
+                    "text_to_speech",
+                    "success",
+                    output={"audio_path": "answer.wav", "voice_id": calls[0]["args"].get("tts_voice")},
+                )
+            ]
+
+        b3 = types.ModuleType("b3_tool_layer")
+        b3.execute_tool_calls = execute_tool_calls
+        b3.get_tools_schema = lambda *_args: []
+        b5 = types.ModuleType("b5_memory")
+        b5.load_memory = lambda *_args: {"status": "success", "selected_memory_docs": []}
+
+        with tempfile.TemporaryDirectory() as temp_dir:
+            root = Path(temp_dir)
+            prompt = root / "prompt.txt"
+            prompt.write_text("system", encoding="utf-8")
+            runtime_input = root / "input.json"
+            runtime_input.write_text(
+                json.dumps(
+                    {
+                        "execution_mode": "integrated",
+                        "input_type": "text",
+                        "conversation_id": "tts-voice-test",
+                        "system_prompt_path": str(prompt),
+                        "toolset": "test_tools",
+                        "max_turns": 2,
+                        "save_memory": "none",
+                        "user_input": "say hello",
+                        "enable_tts": True,
+                        "tts_voice": "bright",
+                    }
+                ),
+                encoding="utf-8",
+            )
+
+            with (
+                patch.dict(sys.modules, {"b3_tool_layer": b3, "b5_memory": b5}),
+                patch.object(
+                    b1_agent_runtime,
+                    "generate_agent_decision",
+                    return_value=direct_decision,
+                ),
+            ):
+                result = b1_agent_runtime.run_agent(
+                    str(runtime_input),
+                    str(root / "tools.yaml"),
+                    str(root / "memory.yaml"),
+                    str(root / "model.yaml"),
+                    str(root / "output"),
+                    "mock",
+                )
+
+        self.assertEqual(result["tts_output"]["voice_id"], "bright")
+        self.assertEqual(captured_calls[0]["name"], "text_to_speech")
+        self.assertEqual(captured_calls[0]["args"]["tts_voice"], "bright")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_b4_json_repair.py

@@ -0,0 +1,63 @@
+from __future__ import annotations
+
+import sys
+import unittest
+from pathlib import Path
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "code"))
+
+from b4_local_agent_llm import (
+    _extract_first_json_object,
+    _parse_execution_plan,
+    _parse_model_output,
+)
+
+
+class JsonRepairTests(unittest.TestCase):
+    def test_repairs_missing_outer_brace_in_direct_plan(self) -> None:
+        plan = _parse_execution_plan('{"use_plan":false,"steps":[]\n')
+
+        self.assertEqual(plan, {"use_plan": False, "steps": []})
+
+    def test_repairs_multiple_missing_container_closers(self) -> None:
+        value = _extract_first_json_object(
+            '{"outer":[{"value":1',
+            kind="test",
+        )
+
+        self.assertEqual(value, {"outer": [{"value": 1}]})
+
+    def test_does_not_guess_an_unterminated_string(self) -> None:
+        with self.assertRaisesRegex(ValueError, "contains no valid JSON object"):
+            _extract_first_json_object('{"value":"unfinished', kind="test")
+
+    def test_recovers_unescaped_quotes_inside_final_content(self) -> None:
+        raw_text = (
+            '{"content":"是国家"211工程"和"985工程"重点建设高校。\\n第二行。",'
+            '"tool_calls":[]}'
+        )
+
+        candidate, message = _parse_model_output(raw_text)
+
+        expected = '是国家"211工程"和"985工程"重点建设高校。\n第二行。'
+        self.assertEqual(candidate, {"content": expected, "tool_calls": []})
+        self.assertEqual(message["content"], expected)
+
+    def test_keeps_valid_escaped_quotes_unchanged(self) -> None:
+        candidate, _ = _parse_model_output(
+            '{"content":"国家\\\"211工程\\\"","tool_calls":[]}'
+        )
+
+        self.assertEqual(candidate["content"], '国家"211工程"')
+
+    def test_recovery_does_not_hide_invalid_tool_calls(self) -> None:
+        with self.assertRaises(ValueError):
+            _parse_model_output(
+                '{"content":"quoted "answer"",'
+                '"tool_calls":[{"id":"call_001","name":"calculator","args":{}}]}'
+            )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_b4_kv_cache.py

@@ -0,0 +1,498 @@
+from __future__ import annotations
+
+import json
+import sys
+import tempfile
+import time
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+import numpy as np
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "code"))
+
+from b4_local_agent_llm import (
+    KVCacheEntry,
+    PLAN_STEP_INSTRUCTION_MARKER,
+    _KV_CACHE,
+    _KV_CPU_CACHE,
+    _build_prompt_messages,
+    _classify_kv_hit,
+    _cache_prefix_snapshot,
+    _cache_requires_snapshot,
+    _clear_kv_cache,
+    _estimate_past_key_values_bytes,
+    _evict_kv_cache,
+    _generate_text_with_kv_cache,
+    _has_plan_step_instruction_suffix,
+    _kv_namespace,
+    _make_kv_cache_key,
+    _move_value_to_device,
+    _token_ids_are_prefix,
+    generate_ai_message,
+)
+
+
+class DummyTokenizer:
+    chat_template = "{{ messages }}"
+
+    @staticmethod
+    def decode(_tokens, **_kwargs) -> str:
+        return "decoded"
+
+
+class DummyNoGrad:
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *_args):
+        return False
+
+
+class DummyTorch:
+    @staticmethod
+    def no_grad() -> DummyNoGrad:
+        return DummyNoGrad()
+
+
+class StateTensor:
+    def __init__(self, value: int, count: int, item_bytes: int, device: str = "gpu"):
+        self.value = value
+        self.count = count
+        self.item_bytes = item_bytes
+        self.device = device
+
+    def numel(self) -> int:
+        return self.count
+
+    def element_size(self) -> int:
+        return self.item_bytes
+
+    def to(self, device: str) -> "StateTensor":
+        return StateTensor(self.value, self.count, self.item_bytes, str(device))
+
+
+class LinearCacheLayer:
+    def __init__(self):
+        self.conv_states = StateTensor(1, 2, 4)
+        self.recurrent_states = StateTensor(3, 3, 4)
+
+    @staticmethod
+    def crop(_token_count: int) -> None:
+        return None
+
+
+class FullCacheLayer:
+    def __init__(self):
+        self.keys = StateTensor(5, 3, 2)
+        self.values = StateTensor(7, 3, 2)
+
+    @staticmethod
+    def crop(_token_count: int) -> None:
+        return None
+
+
+class HybridCache:
+    def __init__(self):
+        self.layers = [LinearCacheLayer(), FullCacheLayer()]
+
+    def crop(self, token_count: int) -> None:
+        for layer in self.layers:
+            layer.crop(token_count)
+
+
+class MutatingModel:
+    def __init__(self):
+        self.generate_calls = 0
+
+    def generate(self, *, input_ids, past_key_values, **_kwargs):
+        self.generate_calls += 1
+        past_key_values.layers[0].recurrent_states.value += 100
+        driver = input_ids.reshape(-1).tolist()
+        return np.asarray([driver + [99]])
+
+
+class KVCacheTests(unittest.TestCase):
+    def setUp(self) -> None:
+        _clear_kv_cache()
+        self.model_identity = ("model", "tokenizer", "true", "true", "bf16", "auto", "none")
+        self.messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "hello"},
+        ]
+        self.tools = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "file_reader",
+                    "description": "read",
+                    "parameters": {"type": "object"},
+                },
+            }
+        ]
+
+    def test_token_prefix_hit_and_miss(self) -> None:
+        self.assertTrue(_token_ids_are_prefix((1, 2, 3), (1, 2, 3, 4)))
+        self.assertFalse(_token_ids_are_prefix((1, 2, 3), (1, 9, 3, 4)))
+
+    def test_system_prompt_change_changes_key(self) -> None:
+        first = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=self.tools,
+            prompt_messages=self.messages,
+            template_kwargs={},
+        )
+        changed_messages = [
+            {"role": "system", "content": "changed system"},
+            {"role": "user", "content": "hello"},
+        ]
+        second = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=self.tools,
+            prompt_messages=changed_messages,
+            template_kwargs={},
+        )
+
+        self.assertNotEqual(first, second)
+
+    def test_tools_schema_change_changes_key(self) -> None:
+        first = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=self.tools,
+            prompt_messages=self.messages,
+            template_kwargs={},
+        )
+        changed_tools = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "calculator",
+                    "description": "calculate",
+                    "parameters": {"type": "object"},
+                },
+            }
+        ]
+        second = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=changed_tools,
+            prompt_messages=self.messages,
+            template_kwargs={},
+        )
+
+        self.assertNotEqual(first, second)
+
+    def test_conversation_id_changes_key(self) -> None:
+        first = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=self.tools,
+            prompt_messages=self.messages,
+            template_kwargs={},
+            conversation_id="conversation_a",
+        )
+        second = _make_kv_cache_key(
+            model_identity=self.model_identity,
+            tokenizer=DummyTokenizer(),
+            backend="transformers",
+            namespace="ai_message.normal.prompt_schema",
+            schema_injection="prompt",
+            tools_schema=self.tools,
+            prompt_messages=self.messages,
+            template_kwargs={},
+            conversation_id="conversation_b",
+        )
+
+        self.assertNotEqual(first, second)
+
+    def test_plan_and_normal_namespaces_do_not_overlap(self) -> None:
+        self.assertEqual(_kv_namespace("plan_decision", "prompt"), "plan_decision")
+        self.assertEqual(
+            _kv_namespace("ai_message.normal", "prompt"),
+            "ai_message.normal.prompt_schema",
+        )
+        self.assertEqual(
+            _kv_namespace("ai_message.plan_step", "native"),
+            "ai_message.plan_step.native_tools",
+        )
+
+    def test_hybrid_cache_snapshot_survives_repeated_generation(self) -> None:
+        model = MutatingModel()
+        inputs = {
+            "input_ids": np.asarray([[10, 11, 12, 13]]),
+            "attention_mask": np.asarray([[1, 1, 1, 1]]),
+        }
+        cache_key = ("hybrid-cache",)
+        kv_config = {
+            "enabled": True,
+            "ttl_seconds": 300,
+            "max_entries": 8,
+            "max_tokens_per_entry": 8192,
+            "max_total_gpu_mb": 2048,
+            "cpu_enabled": True,
+            "cpu_max_entries": 32,
+            "cpu_max_total_mb": 8192,
+        }
+
+        def prefill(*, past_key_values, **_kwargs):
+            return past_key_values if past_key_values is not None else HybridCache()
+
+        with patch("b4_local_agent_llm._prefill_prompt_prefix", side_effect=prefill):
+            first_text, first_record = _generate_text_with_kv_cache(
+                torch_module=DummyTorch(),
+                tokenizer=DummyTokenizer(),
+                model=model,
+                inputs=inputs,
+                options={"max_new_tokens": 4},
+                kv_config=kv_config,
+                cache_key=cache_key,
+                namespace="ai_message.plan_step.prompt_schema",
+                cacheable_token_count=3,
+                source_messages=self.messages,
+            )
+            second_text, second_record = _generate_text_with_kv_cache(
+                torch_module=DummyTorch(),
+                tokenizer=DummyTokenizer(),
+                model=model,
+                inputs=inputs,
+                options={"max_new_tokens": 4},
+                kv_config=kv_config,
+                cache_key=cache_key,
+                namespace="ai_message.plan_step.prompt_schema",
+                cacheable_token_count=3,
+                source_messages=self.messages,
+            )
+
+        stored = _KV_CACHE[cache_key].past_key_values
+        self.assertEqual(first_text, "decoded")
+        self.assertEqual(second_text, "decoded")
+        self.assertEqual(first_record["event"], "miss")
+        self.assertEqual(second_record["event"], "hit")
+        self.assertEqual(first_record["rewind_strategy"], "snapshot")
+        self.assertEqual(second_record["rewind_strategy"], "snapshot")
+        self.assertEqual(stored.layers[0].recurrent_states.value, 3)
+        self.assertEqual(model.generate_calls, 2)
+
+    def test_layered_cache_size_move_and_snapshot_are_supported(self) -> None:
+        cache = HybridCache()
+
+        self.assertTrue(_cache_requires_snapshot(cache))
+        self.assertEqual(_estimate_past_key_values_bytes(cache), 32)
+
+        snapshot = _cache_prefix_snapshot(cache)
+        cache.layers[0].recurrent_states.value = 103
+        self.assertEqual(snapshot.layers[0].recurrent_states.value, 3)
+
+        moved = _move_value_to_device(snapshot, "cpu")
+        states = (
+            moved.layers[0].conv_states,
+            moved.layers[0].recurrent_states,
+            moved.layers[1].keys,
+            moved.layers[1].values,
+        )
+        self.assertTrue(all(state.device == "cpu" for state in states))
+
+    def test_hit_scope_distinguishes_internal_tool_loop_and_conversation(self) -> None:
+        entry = KVCacheEntry(
+            namespace="test",
+            input_ids_prefix=(1,),
+            attention_mask_prefix=(1,),
+            past_key_values=None,
+            token_count=1,
+            created_at=time.time(),
+            last_used_at=time.time(),
+            source_roles=("system", "user"),
+            source_message_count=2,
+        )
+        internal_messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "task"},
+            {"role": "assistant", "content": "", "tool_calls": [{"id": "call_001", "name": "file_reader", "args": {}}]},
+            {"role": "tool", "content": "{}", "tool_call_id": "call_001", "name": "file_reader", "status": "success"},
+        ]
+        conversation_messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "turn 1"},
+            {"role": "assistant", "content": "answer", "tool_calls": []},
+            {"role": "user", "content": "turn 2"},
+        ]
+
+        self.assertEqual(_classify_kv_hit(entry, internal_messages), "turn_internal")
+        self.assertEqual(_classify_kv_hit(entry, conversation_messages), "conversation")
+
+    def test_plan_step_suffix_is_detected_without_touching_normal_messages(self) -> None:
+        normal_messages = self.messages
+        plan_messages = [
+            *self.messages,
+            {
+                "role": "user",
+                "content": f"{PLAN_STEP_INSTRUCTION_MARKER}\nPLAN EXECUTION MODE\nCurrent step: 1",
+            },
+        ]
+
+        self.assertFalse(_has_plan_step_instruction_suffix(normal_messages))
+        self.assertTrue(_has_plan_step_instruction_suffix(plan_messages))
+
+    def test_plan_step_cacheable_base_excludes_transient_instruction(self) -> None:
+        plan_messages = [
+            *self.messages,
+            {
+                "role": "user",
+                "content": f"{PLAN_STEP_INSTRUCTION_MARKER}\nPLAN EXECUTION MODE\nCurrent step: 1",
+            },
+        ]
+        full_prompt = _build_prompt_messages(
+            plan_messages,
+            self.tools,
+            include_tools_schema=True,
+        )
+        cacheable_base = _build_prompt_messages(
+            plan_messages[:-1],
+            self.tools,
+            include_tools_schema=True,
+        )
+
+        self.assertIn(PLAN_STEP_INSTRUCTION_MARKER, str(full_prompt))
+        self.assertNotIn(PLAN_STEP_INSTRUCTION_MARKER, str(cacheable_base))
+
+    def test_prompt_keeps_latest_user_message_for_multi_turn_cache(self) -> None:
+        messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "turn one"},
+            {"role": "assistant", "content": "answer one", "tool_calls": []},
+            {"role": "user", "content": "turn two"},
+        ]
+
+        prompt = _build_prompt_messages(
+            messages,
+            self.tools,
+            include_tools_schema=True,
+        )
+        cacheable_base = _build_prompt_messages(
+            messages,
+            self.tools,
+            include_tools_schema=True,
+        )
+
+        self.assertEqual(prompt[-1], messages[-1])
+        self.assertEqual(prompt[1]["content"], "turn one")
+        self.assertEqual(prompt[3]["content"], "turn two")
+        self.assertEqual(prompt, cacheable_base)
+
+    def test_lru_and_ttl_eviction(self) -> None:
+        now = time.time()
+        _KV_CACHE[("old",)] = KVCacheEntry(
+            namespace="test",
+            input_ids_prefix=(1,),
+            attention_mask_prefix=(1,),
+            past_key_values=None,
+            token_count=1,
+            created_at=now - 10,
+            last_used_at=now - 10,
+        )
+        _KV_CACHE[("new",)] = KVCacheEntry(
+            namespace="test",
+            input_ids_prefix=(1, 2),
+            attention_mask_prefix=(1, 1),
+            past_key_values=None,
+            token_count=2,
+            created_at=now,
+            last_used_at=now,
+        )
+
+        _evict_kv_cache(
+            now,
+            {
+                "ttl_seconds": 1,
+                "max_entries": 8,
+                "max_tokens_per_entry": 8192,
+                "max_total_gpu_mb": 2048,
+            },
+        )
+
+        self.assertNotIn(("old",), _KV_CACHE)
+        self.assertIn(("new",), _KV_CACHE)
+        self.assertNotIn(("old",), _KV_CPU_CACHE)
+
+    def test_gpu_lru_eviction_offloads_to_cpu_cache(self) -> None:
+        now = time.time()
+        _KV_CACHE[("first",)] = KVCacheEntry(
+            namespace="test",
+            input_ids_prefix=(1,),
+            attention_mask_prefix=(1,),
+            past_key_values=None,
+            token_count=1,
+            created_at=now,
+            last_used_at=now,
+        )
+        _KV_CACHE[("second",)] = KVCacheEntry(
+            namespace="test",
+            input_ids_prefix=(2,),
+            attention_mask_prefix=(1,),
+            past_key_values=None,
+            token_count=1,
+            created_at=now,
+            last_used_at=now,
+        )
+
+        _evict_kv_cache(
+            now,
+            {
+                "ttl_seconds": 300,
+                "max_entries": 1,
+                "max_tokens_per_entry": 8192,
+                "max_total_gpu_mb": 2048,
+                "cpu_enabled": True,
+                "cpu_max_entries": 8,
+                "cpu_max_total_mb": 8192,
+            },
+        )
+
+        self.assertNotIn(("first",), _KV_CACHE)
+        self.assertIn(("first",), _KV_CPU_CACHE)
+        self.assertEqual(_KV_CPU_CACHE[("first",)].residency, "cpu")
+
+    def test_mock_ai_message_protocol_is_unchanged(self) -> None:
+        with tempfile.TemporaryDirectory() as temp_dir:
+            root = Path(temp_dir)
+            model_config = root / "model.yaml"
+            model_config.write_text("model:\n  backend: transformers\n", encoding="utf-8")
+            result = generate_ai_message(
+                str(model_config),
+                self.messages,
+                self.tools,
+                mode="mock",
+                artifact_dir=str(root),
+                artifact_stem="mock",
+            )
+
+            self.assertEqual(result["status"], "success")
+            self.assertEqual(set(result["ai_message"]), {"role", "content", "tool_calls"})
+            self.assertIn(result["ai_message"]["role"], {"assistant"})
+            raw = json.loads((root / "mock_raw_model_output.json").read_text(encoding="utf-8"))
+            self.assertEqual(raw["kv_cache"]["event"], "skipped")
+            self.assertEqual(raw["kv_cache"]["reason"], "mock_mode")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_b4_native_mode.py

@@ -0,0 +1,195 @@
+from __future__ import annotations
+
+import json
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "code"))
+
+from b4_local_agent_llm import (
+    _build_plan_prompt_messages,
+    _build_prompt_messages,
+    _messages_for_available_tools,
+    _parse_native_model_output,
+    _validate_tools_available,
+    generate_execution_plan,
+)
+
+
+class NativeModeTests(unittest.TestCase):
+    def setUp(self) -> None:
+        self.messages = [
+            {"role": "system", "content": "system"},
+            {"role": "user", "content": "read a file"},
+        ]
+        self.tools = [
+            {
+                "type": "function",
+                "function": {
+                    "name": "file_reader",
+                    "description": "read",
+                    "parameters": {
+                        "type": "object",
+                        "properties": {"path": {"type": "string"}},
+                        "required": ["path"],
+                    },
+                },
+            }
+        ]
+
+    def test_native_messages_have_no_json_envelope_instruction(self) -> None:
+        native_messages = _build_prompt_messages(
+            self.messages,
+            self.tools,
+            include_tools_schema=False,
+        )
+
+        self.assertEqual(native_messages, self.messages)
+        self.assertNotIn("IMPORTANT OUTPUT FORMAT", str(native_messages))
+        self.assertNotIn("Available tools JSON schema", str(native_messages))
+
+    def test_prompt_messages_keep_schema_and_json_instruction(self) -> None:
+        prompt_messages = _build_prompt_messages(
+            self.messages,
+            self.tools,
+            include_tools_schema=True,
+        )
+
+        rendered = str(prompt_messages)
+        self.assertIn("IMPORTANT OUTPUT FORMAT", rendered)
+        self.assertIn("Available tools JSON schema", rendered)
+
+    def test_prompt_schema_keeps_full_b3_tool_definition(self) -> None:
+        tools = [
+            {
+                **self.tools[0],
+                "function": {
+                    **self.tools[0]["function"],
+                    "x-returns": {"type": "object", "properties": {"content": {"type": "string"}}},
+                },
+            }
+        ]
+
+        prompt_messages = _build_prompt_messages(
+            self.messages,
+            tools,
+            include_tools_schema=True,
+        )
+
+        self.assertIn("x-returns", prompt_messages[0]["content"])
+
+    def test_native_xml_tool_call_is_normalized(self) -> None:
+        _, message = _parse_native_model_output(
+            """
+            <tool_call>
+            <function=file_reader>
+            <parameter=path>docs/agent_intro.txt</parameter>
+            </function>
+            </tool_call>
+            """
+        )
+
+        self.assertEqual(message["content"], "")
+        self.assertEqual(
+            message["tool_calls"],
+            [
+                {
+                    "id": "call_001",
+                    "name": "file_reader",
+                    "args": {"path": "docs/agent_intro.txt"},
+                }
+            ],
+        )
+
+    def test_native_plain_text_becomes_final_answer(self) -> None:
+        _, message = _parse_native_model_output("这是最终回答。")
+
+        self.assertEqual(message["content"], "这是最终回答。")
+        self.assertEqual(message["tool_calls"], [])
+
+    def test_unavailable_audio_bootstrap_is_hidden_from_reasoning_history(self) -> None:
+        messages = [
+            *self.messages,
+            {
+                "role": "assistant",
+                "content": "",
+                "tool_calls": [
+                    {"id": "runtime_audio", "name": "audio_preprocess", "args": {}}
+                ],
+            },
+            {
+                "role": "tool",
+                "tool_call_id": "runtime_audio",
+                "name": "audio_preprocess",
+                "content": "{}",
+            },
+            {"role": "user", "content": "[TRANSCRIBED_USER_REQUEST]\n介绍东北大学"},
+        ]
+
+        filtered = _messages_for_available_tools(messages, self.tools)
+
+        self.assertNotIn("audio_preprocess", str(filtered))
+        self.assertEqual(filtered[-1], messages[-1])
+
+        plan_messages = _build_plan_prompt_messages(messages, self.tools)
+        self.assertNotIn("audio_preprocess", str(plan_messages))
+
+    def test_native_call_must_exist_in_current_schema(self) -> None:
+        _, message = _parse_native_model_output(
+            "<tool_call><function=audio_preprocess></function></tool_call>"
+        )
+
+        with self.assertRaisesRegex(ValueError, "unavailable in this round"):
+            _validate_tools_available(message, self.tools)
+
+    def test_malformed_native_xml_is_not_treated_as_final_text(self) -> None:
+        with self.assertRaises(json.JSONDecodeError):
+            _parse_native_model_output("<tool_call><function=file_reader>")
+
+    def test_plan_decision_retries_native_tool_markup_as_json_only(self) -> None:
+        calls = []
+
+        def fake_plan_generate(*_args, **kwargs):
+            calls.append(kwargs)
+            if len(calls) == 1:
+                return "<tool_call>\n<function=local_file_search>\n</function>\n</tool_call>"
+            return '{"use_plan":false,"steps":[]}'
+
+        with tempfile.TemporaryDirectory() as temp_dir:
+            root = Path(temp_dir)
+            model_config = root / "model.yaml"
+            model_config.write_text("model:\n  backend: transformers\n", encoding="utf-8")
+
+            with patch(
+                "b4_local_agent_llm._plan_prompt_json_generate",
+                side_effect=fake_plan_generate,
+            ):
+                result = generate_execution_plan(
+                    str(model_config),
+                    self.messages,
+                    self.tools,
+                    mode="prompt_json",
+                    artifact_dir=str(root),
+                    artifact_stem="retry",
+                )
+
+            raw = json.loads(
+                (root / "retry_plan_raw_model_output.json").read_text(encoding="utf-8")
+            )
+
+        self.assertEqual(result["status"], "success")
+        self.assertEqual(result["execution_plan"], {"use_plan": False, "steps": []})
+        self.assertEqual(len(calls), 2)
+        self.assertIsNone(calls[0].get("format_retry"))
+        self.assertTrue(calls[1]["format_retry"])
+        self.assertEqual(raw["attempt_count"], 2)
+        self.assertIn("<tool_call>", raw["first_attempt_preview"])
+        self.assertEqual(raw["error"], None)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_local_file_search.py

@@ -0,0 +1,73 @@
+from __future__ import annotations
+
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+from skills.local_file_search import local_file_search
+
+
+class LocalFileSearchTests(unittest.TestCase):
+    def test_exact_phrase_still_ranks_first_with_hybrid_score(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            data_root = Path(directory)
+            docs = data_root / "docs"
+            docs.mkdir(parents=True, exist_ok=True)
+            (docs / "exact.md").write_text(
+                "Agent工具调用 is the central phrase. Agent工具调用 appears twice.",
+                encoding="utf-8",
+            )
+            (docs / "separated.md").write_text(
+                "Agent runtime records may include a later 工具调用 event.",
+                encoding="utf-8",
+            )
+
+            results = local_file_search(
+                "Agent 工具调用",
+                data_root=str(data_root),
+                root_dir="docs",
+                top_k=2,
+            )["results"]
+
+            self.assertEqual(results[0]["path"], "docs/exact.md")
+            self.assertTrue(results[0]["phrase_matched"])
+            self.assertIn("semantic_score", results[0])
+            self.assertIn("lexical_score", results[0])
+            self.assertIn("semantic_matched", results[0])
+
+    def test_redundant_data_root_prefix_is_normalized(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            data_root = Path(directory) / "data"
+            docs = data_root / "docs"
+            docs.mkdir(parents=True)
+            (docs / "campus.md").write_text(
+                "东北大学设有多个校区，并拥有优势学科。",
+                encoding="utf-8",
+            )
+
+            for root_dir in ("data", "data/docs"):
+                with self.subTest(root_dir=root_dir):
+                    results = local_file_search(
+                        "东北大学 校区 优势学科",
+                        data_root=str(data_root),
+                        root_dir=root_dir,
+                    )["results"]
+
+                    self.assertEqual(results[0]["path"], "docs/campus.md")
+
+    def test_unrelated_missing_directory_still_fails(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            with self.assertRaisesRegex(FileNotFoundError, "search directory not found"):
+                local_file_search(
+                    "东北大学",
+                    data_root=directory,
+                    root_dir="missing",
+                )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_tts_streaming.py

@@ -0,0 +1,125 @@
+from __future__ import annotations
+
+import os
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "code"))
+
+from skills import text_to_speech as tts
+from skills.text_to_speech import _markdown_to_speech_text, _split_text
+
+
+class TtsStreamingTests(unittest.TestCase):
+    def test_markdown_is_cleaned_before_stream_chunking(self) -> None:
+        text = """# 修复结果
+
+- **第一项**：缓存已经对齐。
+- [第二项](https://example.com)：流式播放正常。
+
+```python
+print("do not read this")
+```
+
+## 后续建议
+请重新测试语音模式。
+"""
+
+        spoken = _markdown_to_speech_text(text)
+        chunks = _split_text(text, max_chunk_chars=40, min_chunk_chars=15, hard_max_chunk_chars=60)
+
+        self.assertNotIn("#", spoken)
+        self.assertNotIn("**", spoken)
+        self.assertNotIn("https://", spoken)
+        self.assertNotIn("print", spoken)
+        self.assertIn("代码内容已省略。", spoken)
+        self.assertTrue(all(len(chunk) <= 60 for chunk in chunks))
+        self.assertTrue(all("#" not in chunk and "```" not in chunk for chunk in chunks))
+
+    def test_markdown_table_becomes_speakable_rows(self) -> None:
+        text = "| 状态 | 结果 |\n|---|---|\n| KV Cache | 正常 |"
+
+        spoken = _markdown_to_speech_text(text)
+
+        self.assertEqual(spoken, "状态，结果。\nKV Cache，正常。")
+
+    def test_split_text_prefers_short_natural_chunks(self) -> None:
+        text = (
+            "你好，这是第一阶段流式语音测试。"
+            "现在系统会按短句逐段合成，并在每一段完成后立刻返回音频。"
+            "后续再接入模型文本流。"
+            "这样用户不用等待整段回答全部合成完成，就可以先听到第一句内容。"
+        )
+
+        chunks = _split_text(text, max_chunk_chars=60, min_chunk_chars=20, hard_max_chunk_chars=80)
+
+        self.assertGreaterEqual(len(chunks), 2)
+        self.assertTrue(all(len(chunk) <= 80 for chunk in chunks))
+        self.assertIn("流式语音测试", chunks[0])
+
+
+    def test_load_tts_settings_selects_configured_voice_profile(self) -> None:
+        with tempfile.TemporaryDirectory() as temp_dir:
+            root = Path(temp_dir)
+            config_dir = root / "configs"
+            repo = root / "CosyVoice"
+            model = root / "CosyVoice2-0.5B"
+            prompt = root / "voices" / "bright.wav"
+            config_dir.mkdir()
+            repo.mkdir()
+            model.mkdir()
+            prompt.parent.mkdir()
+            prompt.write_bytes(b"RIFF")
+            config = config_dir / "tts.yaml"
+            config.write_text(
+                "\n".join(
+                    [
+                        "runtime:",
+                        f"  python_executable: {sys.executable!r}",
+                        "  timeout_sec: 600",
+                        "  max_total_chars: 1200",
+                        "  min_chunk_chars: 20",
+                        "  max_chunk_chars: 60",
+                        "  hard_max_chunk_chars: 80",
+                        "  inter_chunk_silence_sec: 0.08",
+                        "  persistent_worker: false",
+                        "  load_jit: false",
+                        "  fp16: false",
+                        "  disable_text_frontend: true",
+                        "voice:",
+                        "  default_voice_id: default",
+                        f"  cosyvoice_repo: {repo.as_posix()!r}",
+                        f"  model_dir: {model.as_posix()!r}",
+                        "  voices:",
+                        "    default:",
+                        f"      prompt_wav: {prompt.as_posix()!r}",
+                        "      prompt_text: default text",
+                        "    bright:",
+                        f"      prompt_wav: {prompt.as_posix()!r}",
+                        "      prompt_text: bright text",
+                    ]
+                )
+                + "\n",
+                encoding="utf-8",
+            )
+
+            env = {
+                key: value
+                for key, value in os.environ.items()
+                if not key.startswith("COSYVOICE_")
+            }
+            with patch.object(tts, "TTS_CONFIG_PATH", config), patch.dict(os.environ, env, clear=True):
+                settings = tts._load_tts_settings("bright")
+
+        self.assertEqual(settings["voice_id"], "bright")
+        self.assertEqual(settings["prompt_text"], "bright text")
+        self.assertEqual(settings["prompt_wav"], str(prompt.resolve()))
+
+
+if __name__ == "__main__":
+    unittest.main()