Merge pull request #1579 from A-Dawn/r-dev
refactor(core): 精细化异步类型并改进配置处理的相关逻辑
This commit is contained in:
@@ -5,7 +5,9 @@
|
||||
- A_Memorix 主线源码位于 `src/A_memorix`
|
||||
- 宿主接入层位于 `src/services/memory_service.py`、`src/webui/routers/memory.py` 与 dashboard 长期记忆页面
|
||||
- 运行配置位于 `config/a_memorix.toml`
|
||||
- 运行数据位于 `data/plugins/a-dawn.a-memorix/`
|
||||
- 运行数据由 `storage.data_dir` 决定(当前配置模板默认 `data/a-memorix/`)
|
||||
- 旧离线脚本默认目录仍可能落在 `data/plugins/a-dawn.a-memorix/`(见脚本注释/参数说明)
|
||||
- Web 上传暂存目录为 `data/memory_upload_staging/`
|
||||
- 上游同步方式固定为 `git subtree`
|
||||
|
||||
## 首次接入
|
||||
@@ -40,6 +42,7 @@ git subtree pull --prefix=src/A_memorix https://github.com/A-Dawn/A_memorix.git
|
||||
|
||||
## 同步后检查
|
||||
|
||||
1. 确认 `config/a_memorix.toml` 仍指向 `data/plugins/a-dawn.a-memorix`
|
||||
1. 确认 `config/a_memorix.toml` 中 `storage.data_dir` 与你的运行目录规划一致(默认模板为 `data/a-memorix`)
|
||||
2. 运行 `python src/A_memorix/scripts/runtime_self_check.py --help`
|
||||
3. 运行 A_memorix 相关测试或最少执行一次针对性导入验证
|
||||
3. 运行 `python -m pytest pytests/A_memorix_test/test_memory_service.py`
|
||||
4. 运行 `cd dashboard && npm run test -- src/routes/resource/__tests__/knowledge-base.test.tsx`
|
||||
|
||||
@@ -1,5 +1,15 @@
|
||||
# 更新日志 (Changelog)
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### 📚 文档口径同步
|
||||
|
||||
- 补充并统一“运行时目录”和“离线脚本默认目录”的说明:
|
||||
- 运行时主目录由 `storage.data_dir` 决定(当前模板默认 `data/a-memorix`);
|
||||
- 部分离线脚本仍以 `data/plugins/a-dawn.a-memorix` 作为默认处理目录。
|
||||
- 修正文档中的导入示例参数,`memory_import_admin.create_paste` 的 `input_mode` 示例统一为 `text`/`json`。
|
||||
- 更新 `README.md` 关于元数据 schema 的描述,和当前代码 `SCHEMA_VERSION = 9` 保持一致。
|
||||
|
||||
## [2.0.0] - 2026-03-18
|
||||
|
||||
本次 `2.0.0` 为架构收敛版本,主线是 **SDK Tool 接口统一**、**管理工具能力补齐**、**元数据 schema 升级到 v8** 与 **文档口径同步到 2.0.0**。
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
enabled = true
|
||||
|
||||
[storage]
|
||||
data_dir = "data/plugins/a-dawn.a-memorix"
|
||||
data_dir = "data/a-memorix"
|
||||
|
||||
[embedding]
|
||||
model_name = "auto"
|
||||
@@ -126,9 +126,14 @@ default_sample_size = 24
|
||||
|
||||
### `storage`
|
||||
|
||||
- `storage.data_dir` (代码默认 `./data`;当前内置配置推荐 `data/plugins/a-dawn.a-memorix`)
|
||||
- `storage.data_dir` (当前配置模板默认 `data/a-memorix`)
|
||||
: 数据目录。相对路径按 MaiBot 仓库根目录解析。
|
||||
|
||||
补充说明:
|
||||
|
||||
- 部分离线脚本若未显式覆盖路径,会回退到 `A_memorix.paths.default_data_dir()`(当前为 `data/plugins/a-dawn.a-memorix`)。
|
||||
- 建议在运维侧统一目录策略,避免“控制台写入目录”和“脚本处理目录”不一致。
|
||||
|
||||
### `embedding`
|
||||
|
||||
- `embedding.model_name` (默认 `auto`)
|
||||
|
||||
@@ -29,6 +29,11 @@ python src/A_memorix/scripts/runtime_self_check.py --json
|
||||
data/plugins/a-dawn.a-memorix/raw/
|
||||
```
|
||||
|
||||
说明:
|
||||
|
||||
- `process_knowledge.py` 当前默认扫描上述目录。
|
||||
- 若你的运行配置使用 `storage.data_dir = "data/a-memorix"`,请在执行脚本前统一目录,避免脚本导入目录与运行目录不一致。
|
||||
|
||||
执行:
|
||||
|
||||
```bash
|
||||
@@ -52,7 +57,7 @@ python src/A_memorix/scripts/import_lpmm_json.py <json文件或目录>
|
||||
## 2.3 LPMM 数据转换
|
||||
|
||||
```bash
|
||||
python src/A_memorix/scripts/convert_lpmm.py -i <lpmm数据目录> -o data/plugins/a-dawn.a-memorix
|
||||
python src/A_memorix/scripts/convert_lpmm.py -i <lpmm数据目录> -o data/a-memorix
|
||||
```
|
||||
|
||||
## 2.4 历史数据迁移
|
||||
@@ -115,7 +120,7 @@ python src/A_memorix/scripts/audit_vector_consistency.py --json
|
||||
"arguments": {
|
||||
"action": "create_paste",
|
||||
"content": "今天完成了检索调优回归。",
|
||||
"input_mode": "plain_text",
|
||||
"input_mode": "text",
|
||||
"source": "manual:worklog"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,9 +27,13 @@
|
||||
- `src/services/memory_service.py`
|
||||
- `src/webui/routers/memory.py`
|
||||
- `dashboard/src/routes/resource/knowledge-base.tsx`
|
||||
- `dashboard/src/routes/resource/__tests__/knowledge-base.test.tsx`
|
||||
- `dashboard/src/routes/resource/knowledge-graph/`
|
||||
- `dashboard/src/lib/memory-api.ts`
|
||||
- `config/a_memorix.toml`
|
||||
- `data/plugins/a-dawn.a-memorix/`
|
||||
- `data/a-memorix/`
|
||||
- `data/plugins/a-dawn.a-memorix/`(旧脚本默认路径)
|
||||
- `data/memory_upload_staging/`(Web 上传暂存)
|
||||
- `pytests/A_memorix_test/`
|
||||
- 同步脚本与同步文档,例如 `scripts/sync_a_memorix_subtree.sh`
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ pip install -r requirements.txt --upgrade
|
||||
enabled = true
|
||||
|
||||
[storage]
|
||||
data_dir = "data/plugins/a-dawn.a-memorix"
|
||||
data_dir = "data/a-memorix"
|
||||
|
||||
[embedding]
|
||||
model_name = "auto"
|
||||
@@ -157,12 +157,14 @@ python src/A_memorix/scripts/runtime_self_check.py --json
|
||||
|
||||
### 4.1 文本批量导入
|
||||
|
||||
把文本放到:
|
||||
`process_knowledge.py` 当前默认扫描目录为:
|
||||
|
||||
```text
|
||||
data/plugins/a-dawn.a-memorix/raw/
|
||||
```
|
||||
|
||||
若你当前运行目录使用 `storage.data_dir = "data/a-memorix"`,建议先把文本同步到脚本默认目录再执行,避免导入目录与运行目录不一致。
|
||||
|
||||
执行:
|
||||
|
||||
```bash
|
||||
@@ -181,7 +183,7 @@ python src/A_memorix/scripts/process_knowledge.py --chat-log --chat-reference-ti
|
||||
|
||||
```bash
|
||||
python src/A_memorix/scripts/import_lpmm_json.py <json文件或目录>
|
||||
python src/A_memorix/scripts/convert_lpmm.py -i <lpmm数据目录> -o data/plugins/a-dawn.a-memorix
|
||||
python src/A_memorix/scripts/convert_lpmm.py -i <lpmm数据目录> -o data/a-memorix
|
||||
python src/A_memorix/scripts/migrate_chat_history.py --help
|
||||
python src/A_memorix/scripts/migrate_maibot_memory.py --help
|
||||
python src/A_memorix/scripts/migrate_person_memory_points.py --help
|
||||
|
||||
@@ -21,7 +21,7 @@ A_Memorix 是 MaiBot 内置的长期记忆子系统。
|
||||
|
||||
- 旧 `components/commands/*`、`components/tools/*` 与 `server.py` 已移除。
|
||||
- 统一入口为宿主侧 host service + [`core/runtime/sdk_memory_kernel.py`](core/runtime/sdk_memory_kernel.py)。
|
||||
- 元数据 schema 为 `v8`,新增外部引用与运维操作记录(如 `external_memory_refs`、`memory_v5_operations`、`delete_operations`)。
|
||||
- 元数据 schema 为 `v9`,支持外部引用与运维操作记录(如 `external_memory_refs`、`memory_v5_operations`、`delete_operations`)。
|
||||
|
||||
如果你还在使用旧版 slash 命令(如 `/query`、`/memory`、`/visualize`),需要按本文的 Tool 接口迁移。
|
||||
|
||||
@@ -140,6 +140,7 @@ enabled = true
|
||||
### 配置方式
|
||||
|
||||
- 默认配置文件:`config/a_memorix.toml`
|
||||
- 运行目录主路径:`storage.data_dir`(当前模板默认 `data/a-memorix`)
|
||||
- 长期记忆控制台:适合修改常用高频项,如 embedding、检索、Episode、人物画像、导入与调优开关。
|
||||
- 原始 TOML:适合复制整份配置、批量粘贴参数,或编辑未在可视化表单中展示的高级项。
|
||||
- 配置参考:请结合 [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md) 查看各键的运行时语义与默认值。
|
||||
@@ -171,7 +172,7 @@ python src/A_memorix/scripts/process_knowledge.py
|
||||
- `web/import.html`(导入中心)
|
||||
- `web/tuning.html`(检索调优)
|
||||
|
||||
当前分支不再内置独立 `server.py`,页面路由与 API 暴露由宿主侧 React 页面和 `/api/webui/memory/*` 接口承接。
|
||||
当前分支不再内置独立 `server.py`,页面路由与 API 暴露由宿主侧 React 页面和 `/api/webui/memory/*` 接口承接(并保留 `/api/*` 兼容路由)。
|
||||
|
||||
### WebUI 验证脚本
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import Any, Callable, Coroutine, cast
|
||||
|
||||
from src.common.logger import get_logger
|
||||
|
||||
@@ -106,7 +106,8 @@ def start_background_tasks(plugin: Any) -> None:
|
||||
and bool(plugin.get_config("episode.generation_enabled", True))
|
||||
and (episode_task is None or episode_task.done())
|
||||
):
|
||||
plugin._episode_generation_task = asyncio.create_task(episode_loop())
|
||||
episode_loop_fn = cast(Callable[[], Coroutine[Any, Any, Any]], episode_loop)
|
||||
plugin._episode_generation_task = asyncio.create_task(episode_loop_fn())
|
||||
|
||||
|
||||
async def cancel_background_tasks(plugin: Any) -> None:
|
||||
|
||||
@@ -7,7 +7,7 @@ import time
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Awaitable, Callable, Dict, Iterable, List, Optional, Sequence
|
||||
from typing import Any, Callable, Coroutine, Dict, Iterable, List, Optional, Sequence
|
||||
|
||||
from src.common.logger import get_logger
|
||||
|
||||
@@ -83,7 +83,7 @@ class _KernelRuntimeFacade:
|
||||
async def execute_request_with_dedup(
|
||||
self,
|
||||
request_key: str,
|
||||
executor: Callable[[], Awaitable[Dict[str, Any]]],
|
||||
executor: Callable[[], Coroutine[Any, Any, Dict[str, Any]]],
|
||||
) -> tuple[bool, Dict[str, Any]]:
|
||||
return await self._kernel.execute_request_with_dedup(request_key, executor)
|
||||
|
||||
@@ -769,7 +769,7 @@ class SDKMemoryKernel:
|
||||
async def execute_request_with_dedup(
|
||||
self,
|
||||
request_key: str,
|
||||
executor: Callable[[], Awaitable[Dict[str, Any]]],
|
||||
executor: Callable[[], Coroutine[Any, Any, Dict[str, Any]]],
|
||||
) -> tuple[bool, Dict[str, Any]]:
|
||||
token = str(request_key or "").strip()
|
||||
if not token:
|
||||
@@ -1761,8 +1761,22 @@ class SDKMemoryKernel:
|
||||
profile = manager.get_profile_snapshot()
|
||||
return {"success": True, "profile": profile, "toml": manager.export_toml_snippet(profile)}
|
||||
if act == "apply_profile":
|
||||
profile = kwargs.get("profile") if isinstance(kwargs.get("profile"), dict) else kwargs
|
||||
return {"success": True, **await manager.apply_profile(profile, reason=str(kwargs.get("reason", "manual") or "manual"))}
|
||||
profile_raw = kwargs.get("profile")
|
||||
if isinstance(profile_raw, dict):
|
||||
profile_payload: Dict[str, Any] = dict(profile_raw)
|
||||
else:
|
||||
profile_payload = {
|
||||
key: value
|
||||
for key, value in kwargs.items()
|
||||
if key not in {"reason", "profile"}
|
||||
}
|
||||
return {
|
||||
"success": True,
|
||||
**await manager.apply_profile(
|
||||
profile_payload,
|
||||
reason=str(kwargs.get("reason", "manual") or "manual"),
|
||||
),
|
||||
}
|
||||
if act == "rollback_profile":
|
||||
return {"success": True, **await manager.rollback_profile()}
|
||||
if act == "export_profile":
|
||||
@@ -1999,7 +2013,11 @@ class SDKMemoryKernel:
|
||||
self._ensure_background_task("memory_maintenance", self._memory_maintenance_loop)
|
||||
self._ensure_background_task("person_profile_refresh", self._person_profile_refresh_loop)
|
||||
|
||||
def _ensure_background_task(self, name: str, factory: Callable[[], Awaitable[None]]) -> None:
|
||||
def _ensure_background_task(
|
||||
self,
|
||||
name: str,
|
||||
factory: Callable[[], Coroutine[Any, Any, None]],
|
||||
) -> None:
|
||||
task = self._background_tasks.get(name)
|
||||
if task is not None and not task.done():
|
||||
return
|
||||
|
||||
@@ -73,7 +73,7 @@ class GraphStore:
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
matrix_format: str = "csr",
|
||||
matrix_format: Union[str, SparseMatrixFormat] = "csr",
|
||||
data_dir: Optional[Union[str, Path]] = None,
|
||||
):
|
||||
"""
|
||||
|
||||
@@ -79,6 +79,7 @@ class VectorStore:
|
||||
self.quantization_type = QuantizationType.INT8
|
||||
self.index_type = "sq8"
|
||||
self.buffer_size = buffer_size
|
||||
self.min_train_threshold = self.DEFAULT_MIN_TRAIN
|
||||
|
||||
self._index: Optional[faiss.IndexIDMap2] = None
|
||||
self._init_index()
|
||||
|
||||
@@ -128,7 +128,7 @@ class RelationWriteService:
|
||||
predicate: str,
|
||||
obj: str,
|
||||
confidence: float = 1.0,
|
||||
source_paragraph: str = "",
|
||||
source_paragraph: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
*,
|
||||
write_vector: bool = True,
|
||||
|
||||
@@ -125,7 +125,8 @@ class DeletePurgeRequest(BaseModel):
|
||||
|
||||
|
||||
def _build_import_guide_markdown(settings: dict[str, Any]) -> str:
|
||||
path_aliases = settings.get("path_aliases") if isinstance(settings.get("path_aliases"), dict) else {}
|
||||
path_aliases_raw = settings.get("path_aliases")
|
||||
path_aliases = path_aliases_raw if isinstance(path_aliases_raw, dict) else {}
|
||||
alias_lines = [
|
||||
f"- `{name}` -> `{path}`"
|
||||
for name, path in sorted(path_aliases.items())
|
||||
@@ -394,15 +395,7 @@ async def _memory_config_get() -> dict:
|
||||
|
||||
|
||||
async def _memory_config_get_raw() -> dict:
|
||||
raw_payload_getter = getattr(a_memorix_host_service, "get_raw_config_with_meta", None)
|
||||
if callable(raw_payload_getter):
|
||||
raw_payload = raw_payload_getter()
|
||||
else:
|
||||
raw_payload = {
|
||||
"config": a_memorix_host_service.get_raw_config(),
|
||||
"exists": bool(a_memorix_host_service.get_config_path().exists()),
|
||||
"using_default": False,
|
||||
}
|
||||
raw_payload = a_memorix_host_service.get_raw_config_with_meta()
|
||||
return {
|
||||
"success": True,
|
||||
"config": str(raw_payload.get("config", "") or ""),
|
||||
@@ -628,8 +621,10 @@ async def _tuning_apply_best(task_id: str) -> dict:
|
||||
|
||||
|
||||
async def _tuning_report(task_id: str, fmt: str) -> dict:
|
||||
payload = await memory_service.tuning_admin(action="get_report", task_id=task_id, format=fmt)
|
||||
report = payload.get("report") if isinstance(payload.get("report"), dict) else {}
|
||||
payload_raw = await memory_service.tuning_admin(action="get_report", task_id=task_id, format=fmt)
|
||||
payload = payload_raw if isinstance(payload_raw, dict) else {}
|
||||
report_raw = payload.get("report")
|
||||
report = report_raw if isinstance(report_raw, dict) else {}
|
||||
return {
|
||||
"success": bool(payload.get("success", False)),
|
||||
"format": report.get("format", fmt),
|
||||
|
||||
Reference in New Issue
Block a user