1ed558b488
feat: 🏗️ 实现 Agent 消息可靠异步持久化（Outbox + Kafka） * 新增 Outbox 数据模型与消息载荷定义，位于 `backend/model/outbox.go` * 新增 Outbox DAO，支持创建、扫描、发布标记、失败重试与消费落库事务，位于 `backend/dao/outbox.go` * 新增 Kafka 基础封装，包含配置、生产者、消费者与消息包装，位于 `backend/kafka` 文件夹 * `config.go`：Kafka 配置文件 * `producer.go`：Kafka 生产者 * `consumer.go`：Kafka 消费者 * `envelope.go`：消息封装处理 * 新增异步管道服务，处理扫描投递与消费落库，位于 `backend/service/agent_async_pipeline.go` * 接入 Agent 聊天链路的可靠持久化，替换原有 goroutine 直接写库逻辑，位于 `backend/service/agent.go` * 启动流程接入管道初始化与启动，位于 `backend/cmd/start.go` * 增加 Kafka 配置项，更新 `backend/config.yaml` 与 `backend/config.example.yaml` * 引入 Kafka 依赖：`github.com/segmentio/kafka-go`（见 `backend/go.mod`, `backend/go.sum`） fix: 🐛 修复首启偶发 user 消息重复落库问题 * 解决因 Outbox 状态并发回写竞态，导致 `consumed` 被晚到的 `published` 覆盖的问题 * 在 `MarkPublished` 中增加条件，避免覆盖已标记为 `consumed` 或 `dead` 的消息，修复位置：`backend/dao/outbox.go` perf: ⚡ 更新 Docker Compose 配置与 Kafka 相关服务 * 更新 `docker-compose.yml` 文件，新增 Kafka 配置与服务 fix: 🧹 优化缓存删除逻辑 * 在 `cache deleter` 中忽略了 `model.AgentOutboxMessage`、`model.ChatHistory` 与 `model.AgentChat` 这三个结构体 * 防止这些结构体对应的表单删除缓存时，导致控制台消息爆炸
91 lines
3.9 KiB
Modula-2
91 lines
3.9 KiB
Modula-2
module github.com/LoveLosita/smartflow/backend
|
|
|
|
go 1.23.4
|
|
|
|
require (
|
|
github.com/cloudwego/eino-ext/components/model/ark v0.1.64
|
|
github.com/cloudwego/eino-ext/components/model/openai v0.1.8
|
|
github.com/gin-gonic/gin v1.11.0
|
|
github.com/go-redis/redis/v8 v8.11.5
|
|
github.com/golang-jwt/jwt/v4 v4.5.2
|
|
github.com/google/uuid v1.6.0
|
|
github.com/spf13/viper v1.21.0
|
|
golang.org/x/crypto v0.40.0
|
|
gorm.io/driver/mysql v1.6.0
|
|
gorm.io/gorm v1.31.1
|
|
)
|
|
|
|
require (
|
|
filippo.io/edwards25519 v1.1.0 // indirect
|
|
github.com/bahlo/generic-list-go v0.2.0 // indirect
|
|
github.com/buger/jsonparser v1.1.1 // indirect
|
|
github.com/bytedance/gopkg v0.1.3 // indirect
|
|
github.com/bytedance/sonic v1.14.1 // indirect
|
|
github.com/bytedance/sonic/loader v0.3.0 // indirect
|
|
github.com/cespare/xxhash/v2 v2.2.0 // indirect
|
|
github.com/cloudwego/base64x v0.1.6 // indirect
|
|
github.com/cloudwego/eino v0.7.13 // indirect
|
|
github.com/cloudwego/eino-ext/libs/acl/openai v0.1.13 // indirect
|
|
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
|
github.com/eino-contrib/jsonschema v1.0.3 // indirect
|
|
github.com/evanphx/json-patch v0.5.2 // indirect
|
|
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
|
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
|
|
github.com/gin-contrib/sse v1.1.0 // indirect
|
|
github.com/go-playground/locales v0.14.1 // indirect
|
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
|
github.com/go-playground/validator/v10 v10.27.0 // indirect
|
|
github.com/go-sql-driver/mysql v1.8.1 // indirect
|
|
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
|
github.com/goccy/go-json v0.10.2 // indirect
|
|
github.com/goccy/go-yaml v1.18.0 // indirect
|
|
github.com/goph/emperror v0.17.2 // indirect
|
|
github.com/jinzhu/inflection v1.0.0 // indirect
|
|
github.com/jinzhu/now v1.1.5 // indirect
|
|
github.com/jmespath/go-jmespath v0.4.0 // indirect
|
|
github.com/json-iterator/go v1.1.12 // indirect
|
|
github.com/klauspost/compress v1.15.9 // indirect
|
|
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
|
github.com/leodido/go-urn v1.4.0 // indirect
|
|
github.com/mailru/easyjson v0.7.7 // indirect
|
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
|
github.com/meguminnnnnnnnn/go-openai v0.1.1 // indirect
|
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
|
github.com/modern-go/reflect2 v1.0.2 // indirect
|
|
github.com/nikolalohinski/gonja v1.5.3 // indirect
|
|
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
|
github.com/pierrec/lz4/v4 v4.1.15 // indirect
|
|
github.com/pkg/errors v0.9.1 // indirect
|
|
github.com/quic-go/qpack v0.5.1 // indirect
|
|
github.com/quic-go/quic-go v0.54.0 // indirect
|
|
github.com/sagikazarmark/locafero v0.11.0 // indirect
|
|
github.com/segmentio/kafka-go v0.4.47 // indirect
|
|
github.com/sirupsen/logrus v1.9.3 // indirect
|
|
github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f // indirect
|
|
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
|
|
github.com/spf13/afero v1.15.0 // indirect
|
|
github.com/spf13/cast v1.10.0 // indirect
|
|
github.com/spf13/pflag v1.0.10 // indirect
|
|
github.com/subosito/gotenv v1.6.0 // indirect
|
|
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
|
github.com/ugorji/go/codec v1.3.0 // indirect
|
|
github.com/volcengine/volc-sdk-golang v1.0.23 // indirect
|
|
github.com/volcengine/volcengine-go-sdk v1.2.9 // indirect
|
|
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
|
|
github.com/yargevad/filepathx v1.0.0 // indirect
|
|
go.uber.org/mock v0.5.0 // indirect
|
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
|
golang.org/x/arch v0.20.0 // indirect
|
|
golang.org/x/exp v0.0.0-20230713183714-613f0c0eb8a1 // indirect
|
|
golang.org/x/mod v0.26.0 // indirect
|
|
golang.org/x/net v0.42.0 // indirect
|
|
golang.org/x/sync v0.16.0 // indirect
|
|
golang.org/x/sys v0.35.0 // indirect
|
|
golang.org/x/text v0.28.0 // indirect
|
|
golang.org/x/tools v0.35.0 // indirect
|
|
google.golang.org/protobuf v1.36.9 // indirect
|
|
gopkg.in/yaml.v2 v2.4.0 // indirect
|
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
|
)
|