merge: ARCH-004 BCE reranker alignment
This commit is contained in:
commit
4ce74604ba
@ -49,7 +49,7 @@ INDEX_PROFILE_ID=demo-bge-large-zh-structure-v1
|
||||
DENSE_TOP_N=20
|
||||
|
||||
RERANKER_PROVIDER=qianfan
|
||||
RERANKER_MODEL=qwen3-reranker-0.6b
|
||||
RERANKER_MODEL=bce-reranker-base
|
||||
RERANKER_ENDPOINT=https://qianfan.baidubce.com/v2/rerank
|
||||
RERANKER_API_KEY=
|
||||
RERANKER_TIMEOUT_SECONDS=30
|
||||
|
||||
@ -106,8 +106,8 @@ default is Top 20 with a configurable Mochow HNSW `ef`; the in-memory fake uses
|
||||
exact cosine search for deterministic offline regression. Returned objects are
|
||||
raw dense candidates only.
|
||||
|
||||
DEMO-008 adds a provider-neutral reranker, a validated Qianfan
|
||||
`qwen3-reranker-0.6b` adapter, and the final Top20-to-Top5 search pipeline. Final
|
||||
DEMO-008 adds a provider-neutral reranker, and ARCH-004 aligns its validated Qianfan
|
||||
adapter with the `bce-reranker-base` baseline. The final Top20-to-Top5 pipeline and
|
||||
source results are rehydrated from canonical PostgreSQL chunks and retain both
|
||||
Dense and reranker scores; stale vector content fails closed. See
|
||||
`docs/development/RERANKER_SETUP.md` for configuration and validation limits.
|
||||
|
||||
@ -146,7 +146,7 @@ def create_dense_retrieval_service(
|
||||
|
||||
|
||||
def create_reranker_provider(settings: Settings) -> QianfanRerankerProvider:
|
||||
"""Create the Demo Qwen3 reranker using an explicit or shared Qianfan key."""
|
||||
"""Create the Demo BCE reranker using an explicit or shared Qianfan key."""
|
||||
secret = settings.reranker_api_key or settings.baidu_qianfan_api_key
|
||||
if secret is None:
|
||||
raise ProviderConfigurationError("RERANKER_API_KEY or BAIDU_QIANFAN_API_KEY is required")
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# DEMO-008 千帆 Reranker 与 Source Result 配置
|
||||
# ARCH-004 千帆 BCE Reranker 与 Source Result 配置
|
||||
|
||||
本文记录 Knowledge Platform Demo 的重排序连接方式、请求边界和 canonical Source Result
|
||||
契约。真实 API Key 只允许由运行环境注入,不得写入代码、示例配置、日志或提交记录。
|
||||
@ -7,7 +7,7 @@
|
||||
|
||||
```dotenv
|
||||
RERANKER_PROVIDER=qianfan
|
||||
RERANKER_MODEL=qwen3-reranker-0.6b
|
||||
RERANKER_MODEL=bce-reranker-base
|
||||
RERANKER_ENDPOINT=https://qianfan.baidubce.com/v2/rerank
|
||||
RERANKER_API_KEY=<运行环境注入,可空>
|
||||
RERANKER_TIMEOUT_SECONDS=30
|
||||
@ -21,8 +21,8 @@ Provider 创建失败。凭证使用 `SecretStr`,HTTP/JSON 异常不会包含
|
||||
|
||||
## Reranker 请求契约
|
||||
|
||||
DEMO-008 固定使用千帆 `POST /v2/rerank`,模型基线为
|
||||
`qwen3-reranker-0.6b`:
|
||||
ARCH-004 固定使用千帆 `POST /v2/rerank`,模型基线为
|
||||
`bce-reranker-base`:
|
||||
|
||||
- Query 去除首尾空白后不能为空,最多 1600 字符;
|
||||
- documents 不能为空,最多 64 条,每条最多 4096 字符;
|
||||
@ -31,12 +31,17 @@ DEMO-008 固定使用千帆 `POST /v2/rerank`,模型基线为
|
||||
- `relevance_score` 必须是有限数值,并保持降序;
|
||||
- 响应若回显 document,必须与对应输入文本完全一致。
|
||||
|
||||
上述边界与千帆重排序接口当前约束一致。BCE 和原 Demo Qwen3 模型共享请求/响应结构,
|
||||
因此本次变更不修改 `RerankerProvider`、Search REST API 或最终 Top5 行为。Qwen3-Reranker
|
||||
保留为 PRD 中的后续 Benchmark 候选,不作为当前运行默认值。
|
||||
|
||||
Adapter 只向业务层返回原候选索引和 Reranker 分数,不暴露千帆响应结构。默认测试使用
|
||||
`FakeRerankerProvider` 的确定性字符覆盖率排序,不发起网络请求。
|
||||
|
||||
## Source Result 契约
|
||||
|
||||
Reranker 对 Mochow Dense Candidate 的 `content` 评分;选出 Top5 后,`SearchService` 使用
|
||||
Reranker 对 provider-neutral Dense Candidate 的 `content` 评分;选出 Top5 后,
|
||||
`SearchService` 使用
|
||||
`chunk_id` 一次性回查 PostgreSQL canonical `chunks` 表,并由 canonical row 生成:
|
||||
|
||||
```text
|
||||
@ -51,8 +56,8 @@ rerank_score
|
||||
```
|
||||
|
||||
其中 `score` 保留 Dense 分数,`rerank_score` 保存精排分数。若 Chunk 缺失、document_id
|
||||
不一致或 Mochow content 已与 canonical content 不一致,搜索会失败关闭,避免返回来源与
|
||||
评分文本不一致的结果。
|
||||
不一致或向量索引 content 已与 canonical content 不一致,搜索会失败关闭,避免返回来源
|
||||
与评分文本不一致的结果。
|
||||
|
||||
DEMO-009 已通过 `POST /api/v1/search` 提供序列化和错误映射,参见
|
||||
`docs/development/REST_API_SETUP.md`。Demo 仍不实现 RBAC/Document ACL,正式 V1 权限
|
||||
@ -67,3 +72,5 @@ Golden Dataset 验证预期文档和章节升至首位。真实千帆联调前
|
||||
- Endpoint 网络可达;
|
||||
- 模型 API 名称与控制台服务一致;
|
||||
- 真实中文 Query 的延迟、配额和 Top5 质量满足 Demo 演示需求。
|
||||
|
||||
官方接口参考:[千帆重排序 API](https://cloud.baidu.com/doc/qianfan-api/s/2m7u4zt74)。
|
||||
|
||||
@ -2,11 +2,11 @@
|
||||
|
||||
当前阶段:Provider 架构迁移
|
||||
|
||||
已完成:DEMO-001~DEMO-009、ARCH-001、ARCH-002、ARCH-003
|
||||
已完成:DEMO-001~DEMO-009、ARCH-001、ARCH-002、ARCH-003、ARCH-004
|
||||
进行中:无
|
||||
阻塞:无
|
||||
|
||||
下一任务:ARCH-004
|
||||
下一任务:ARCH-005
|
||||
|
||||
Provider 架构迁移按 `ARCH-001 → ARCH-002 → ARCH-003 → ARCH-004 → ARCH-005 → ARCH-006`
|
||||
执行;每个 Task 验证后独立推送并合并 `main`。全部迁移完成后再继续 DEMO-010。
|
||||
@ -106,7 +106,8 @@ Provider 架构迁移按 `ARCH-001 → ARCH-002 → ARCH-003 → ARCH-004 → AR
|
||||
|
||||
- 新增与厂商无关的 `RerankerProvider` / `RerankResult` 契约及确定性 Fake,使用原始
|
||||
candidate index 映射精排结果,业务服务不依赖千帆响应结构。
|
||||
- 新增千帆 v2 `qwen3-reranker-0.6b` Adapter,调用 `/v2/rerank`,严格限制 Query、
|
||||
- 新增千帆 v2 Reranker Adapter(初始为 `qwen3-reranker-0.6b`,ARCH-004 已迁移至
|
||||
`bce-reranker-base`),调用 `/v2/rerank`,严格限制 Query、
|
||||
documents 和 Top-N,校验响应数量、唯一索引、回显文本、有限分数与降序关系;错误不会
|
||||
回显请求内容或凭证。
|
||||
- 新增 `SearchService`,完成 Dense Top20 → Reranker → Top5。Reranker 选中候选后按
|
||||
@ -168,3 +169,14 @@ Provider 架构迁移按 `ARCH-001 → ARCH-002 → ARCH-003 → ARCH-004 → AR
|
||||
- 新增 `mochow|oceanbase` 类型化配置、SecretStr 密码和运行时工厂;Mochow 保持 ARCH-005
|
||||
前的默认实现。全部 Adapter、配置、Schema 和 wiring 测试均离线运行,真实数据库验收留在
|
||||
ARCH-006。
|
||||
|
||||
### ARCH-004
|
||||
|
||||
- 将千帆 Reranker 运行基线从 `qwen3-reranker-0.6b` 切换为已接受的
|
||||
`bce-reranker-base`;配置使用受限 Literal,业务 API、Provider 抽象和 Dense Top20 →
|
||||
Reranker Top5 行为保持不变。
|
||||
- 按千帆 `POST /v2/rerank` 官方约束复核并保留 Query 1600 字符、64 个候选、单文档
|
||||
4096 字符、Top-N 及响应排序/索引/分数失败关闭校验;真实 API Key 不进入代码和测试。
|
||||
- 更新 Adapter、环境示例、Provider wiring、配置与请求契约测试;Golden Dataset 继续使用
|
||||
确定性 Fake 离线评估,但显式绑定 `bce-reranker-base` 当前模型标识。真实 BCE 质量与延迟
|
||||
统一留在 ARCH-006 使用轮换凭据验收。
|
||||
|
||||
@ -5,7 +5,7 @@
|
||||
- [x] ARCH-001 Architecture decision, PRD, Scope and execution baseline
|
||||
- [x] ARCH-002 BOS ObjectStorage adapter and runtime configuration
|
||||
- [x] ARCH-003 OceanBase VectorStore adapter and index schema
|
||||
- [ ] ARCH-004 Qianfan `bce-reranker-base` alignment
|
||||
- [x] ARCH-004 Qianfan `bce-reranker-base` alignment
|
||||
- [ ] ARCH-005 Runtime cutover, shadow rebuild workflow and legacy configuration cleanup
|
||||
- [ ] ARCH-006 Controlled live integration and RAG acceptance
|
||||
|
||||
|
||||
@ -62,7 +62,7 @@ class Settings(BaseSettings):
|
||||
index_profile_id: str = "demo-bge-large-zh-structure-v1"
|
||||
dense_top_n: int = Field(default=20, ge=1, le=64)
|
||||
reranker_provider: Literal["qianfan"] = "qianfan"
|
||||
reranker_model: Literal["qwen3-reranker-0.6b"] = "qwen3-reranker-0.6b"
|
||||
reranker_model: Literal["bce-reranker-base"] = "bce-reranker-base"
|
||||
reranker_endpoint: str = "https://qianfan.baidubce.com/v2/rerank"
|
||||
reranker_api_key: SecretStr | None = None
|
||||
reranker_timeout_seconds: float = Field(default=30.0, gt=0)
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
"""Baidu Qianfan v2 adapter for the Demo Qwen3 reranker baseline."""
|
||||
"""Baidu Qianfan v2 adapter for the Demo BCE reranker baseline."""
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping, Sequence
|
||||
@ -10,7 +10,7 @@ from urllib.request import Request, urlopen
|
||||
|
||||
from ekp_reranker.base import RerankerProviderError, RerankResult
|
||||
|
||||
QWEN3_RERANKER_MODEL = "qwen3-reranker-0.6b"
|
||||
BCE_RERANKER_MODEL = "bce-reranker-base"
|
||||
QIANFAN_RERANK_ENDPOINT = "https://qianfan.baidubce.com/v2/rerank"
|
||||
QIANFAN_MAX_DOCUMENTS = 64
|
||||
QIANFAN_MAX_QUERY_CHARACTERS = 1600
|
||||
@ -65,7 +65,7 @@ class QianfanRerankerProvider:
|
||||
self,
|
||||
api_key: str,
|
||||
endpoint: str = QIANFAN_RERANK_ENDPOINT,
|
||||
model_name: str = QWEN3_RERANKER_MODEL,
|
||||
model_name: str = BCE_RERANKER_MODEL,
|
||||
timeout_seconds: float = 30.0,
|
||||
http_client: JsonHttpClient | None = None,
|
||||
) -> None:
|
||||
|
||||
@ -63,7 +63,11 @@ def test_reranker_promotes_expected_canonical_source_to_top_one(tmp_path: Path)
|
||||
indexing,
|
||||
)
|
||||
dense = DenseRetrievalService(embedding, vector_store, profile)
|
||||
search = SearchService(session_factory, dense, FakeRerankerProvider())
|
||||
# The fake keeps Golden Dataset evaluation deterministic while pinning the
|
||||
# active runtime model identifier selected by ARCH-004.
|
||||
reranker = FakeRerankerProvider(model_name="bce-reranker-base")
|
||||
search = SearchService(session_factory, dense, reranker)
|
||||
assert reranker.model_name == "bce-reranker-base"
|
||||
golden_records = [
|
||||
json.loads(line)
|
||||
for line in Path("fixtures/evaluation/golden.jsonl").read_text().splitlines()
|
||||
|
||||
@ -47,7 +47,7 @@ def test_settings_have_demo_defaults(isolated_settings_environment: None) -> Non
|
||||
assert settings.index_profile_id == "demo-bge-large-zh-structure-v1"
|
||||
assert settings.dense_top_n == 20
|
||||
assert settings.reranker_provider == "qianfan"
|
||||
assert settings.reranker_model == "qwen3-reranker-0.6b"
|
||||
assert settings.reranker_model == "bce-reranker-base"
|
||||
assert settings.reranker_endpoint.endswith("/v2/rerank")
|
||||
assert settings.reranker_api_key is None
|
||||
assert settings.search_top_k == 5
|
||||
|
||||
@ -141,7 +141,7 @@ def test_provider_wiring_reuses_qianfan_key_for_demo_reranker(
|
||||
|
||||
reranker = create_reranker_provider(settings)
|
||||
|
||||
assert reranker.model_name == "qwen3-reranker-0.6b"
|
||||
assert reranker.model_name == "bce-reranker-base"
|
||||
assert reranker.max_documents == 64
|
||||
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
"""Unit tests for the Qianfan Qwen3 reranker request/response contract."""
|
||||
"""Unit tests for the Qianfan BCE reranker request/response contract."""
|
||||
|
||||
from collections.abc import Mapping
|
||||
|
||||
@ -43,14 +43,14 @@ def test_qianfan_reranker_sends_top_n_and_preserves_ranked_indexes() -> None:
|
||||
|
||||
results = provider.rerank(" 产品A退款周期 ", ("住宿标准说明", "退款周期为七天"), 2)
|
||||
|
||||
assert provider.model_name == "qwen3-reranker-0.6b"
|
||||
assert provider.model_name == "bce-reranker-base"
|
||||
assert provider.max_documents == 64
|
||||
assert [result.candidate_index for result in results] == [1, 0]
|
||||
url, headers, payload, timeout = client.calls[0]
|
||||
assert url.endswith("/v2/rerank")
|
||||
assert headers["Authorization"] == "Bearer test-key"
|
||||
assert payload == {
|
||||
"model": "qwen3-reranker-0.6b",
|
||||
"model": "bce-reranker-base",
|
||||
"query": "产品A退款周期",
|
||||
"documents": ["住宿标准说明", "退款周期为七天"],
|
||||
"top_n": 2,
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user