跨层共享项落地(设计文档 §19.4 X.1/X.3/X.4): X.1 时序三档接线 + X.3 v_t 全链路贯通: - shared/types.ts: TaskFeedback/ToolOutcome/EvolutionMetrics/MetricsSources + hitRate/avgInfoGain/avgSkillQuality/maxWorkflowRisk/evolutionBrickRate 纯函数 - core/feedback-collector.ts: FeedbackCollector(recordPrompt/recordBoardFingerprint/ recordToolResult/finalize/lastFinalized/pending/clear) + derive* 纯变换 (deriveTaskSignature/deriveDistillInput/deriveSkillExtractInput/deriveSkillQuery/ derivePriorCorrection/deriveProposals + inferOutcome/inferActionClass/ inferValidatedBy/isIrreversibleTool) + getSharedCollector/initSharedCollector 共享单例 - extensions/feedback-bank.ts: hooks tool_result+message_end+before_agent_start → 采集 → finalize → session_before_compact 喂 derive* + feedback_recall 工具 (仿 orchestration-bank/skill-bank 范式) - **4 bank 默认 derive* 读共享 collector lastFinalized 生产自驱** (experience/skill/orchestration/variant bank) — X.1+X.3 生产闭环 X.4 进化可观测面板: - core/evolution-metrics.ts: aggregateMetrics/renderMetrics 纯函数 (5 层注册表聚合 → EvolutionMetrics:规模/命中率/infoGain/risk/砖板率/forbidden 拒绝) - bin/evolution-metrics.ts CLI(brickRate!=0 时 exit 1 fail-closed 自检) X.1+X.3 全链路评估(tests/eval/feedback-chain-eval.test.ts, 合成 session 轨迹→ feedback-bank hooks→collector→derive*→真 5 层 manager→断 v_t 全链路 end-to-end): §4.6 分级正确率 8/8=100% (success+ground-truth/可逆/不可逆/partial/empty/mixed) 同签名检索命中 100% + 注入块可见 100% 5 链 end-to-end (skill/workflow/variant/tool-growth) forbidden §11.3/§4.8 拒绝 (命门 safety-gate/flash 工具) X.4 评估(tests/eval/evolution-metrics-eval.test.ts, 合成 5 层注册表→精确指标): experienceHitRate=0.600/avgInfoGain=0.650/maxRisk=1 精确 brickRate=0 EXACT 构造铁律 forbiddenRejected=7 精确 结论: X.1(4/5 时序钩子全接, stage-wise θ=L5.*) + X.3(v_t 全链路贯通: tool_result 采集→derive* 蒸馏→5 层入库→检索→注入→约束, §4.6 100% + 5 链 end-to-end + 4 bank 生产自驱) + X.4(可观测面板精确指标, brickRate=0 构造铁律) 全部完成。诚实边界 (M1-M5.* 跟进): live-LLM 读注入包真受益端到端门 + stage-wise 真 θ 更新 + 真 git worktree 变体 + 真 CI §11.4 执行 = M1-M5.* 工程项(机制层全闭环+量化坐实)。 验证: tsc 0 / 全量 62 文件 3018 passed 5 skipped / sk- 明文 0 / 真 pi T01 boot 过(barrel 28 扩展含 L1-L5 + 跨层反馈采集) 多 Agent 并行: K(feedback-collector+derive*)+L(feedback-bank+chain eval)+M(evolution-metrics) Co-Authored-By: Claude <noreply@anthropic.com>
345 lines
19 KiB
TypeScript
345 lines
19 KiB
TypeScript
/**
|
||
* evolution-metrics-eval.test.ts — X.4 进化可观测面板定量评估
|
||
*
|
||
* Scientific claim under test (设计文档 §13 证据 / §16 评估 / §19.4 X.4):
|
||
* The X.4 metrics aggregator (aggregateMetrics) faithfully reduces a 5-layer
|
||
* registry snapshot to EvolutionMetrics — with EXACT pre-registered values for
|
||
* every count, rate, and derived metric, AND the hardware-独有 construction
|
||
* invariants holding EXACTLY:
|
||
* - brickRate === 0 (construction-guaranteed, §4.8/§11.4 — if non-zero,
|
||
* critical bug in L3+L5 forbidden gate / sandbox / §11.4 invariant door)
|
||
* - maxRisk = max(workflow risk) (§4.8 不可逆暴露面,须 ≤ default)
|
||
* - forbiddenRejected passed through from sources (rejections don't enter
|
||
* registries — they're injected, §4.8/§11.3)
|
||
*
|
||
* Faithful, deterministic synthetic fixture (no RNG, no self-fulfilling):
|
||
* A fixed 5-layer registry snapshot with hand-picked numbers:
|
||
* - 5 experiences, hitCounts [3,0,2,1,0] → hitRate = 3/5 = 0.6 (exact)
|
||
* - 4 skills, infoGains [0.8,0.6,0.9,0.3] → avgInfoGain = 0.65 (exact)
|
||
* - 4 skills, quality scores [1.0,0.7,0.3,1.0] → avgQualityScore = 0.75 (exact)
|
||
* - 4 skills, 3 active (skill2 inactive)
|
||
* - 3 workflows, risks [1,0,1] → maxRisk = 1 (exact)
|
||
* - 3 workflows, 2 active (workflow1 inactive)
|
||
* - 5 grownTools, 2 promoted (stage=promoted)
|
||
* - 4 variants, 1 promoted (stage=promoted)
|
||
* - forbiddenRejected = 7 (injected, L3+L5 累计 forbidden 拒绝)
|
||
*
|
||
* Pre-registered thresholds (decided BEFORE running, not weakened if they fail):
|
||
* experienceHitRate === 0.6 (EXACT)
|
||
* avgInfoGain === 0.65 (EXACT)
|
||
* avgQualityScore === 0.75 (EXACT)
|
||
* maxRisk === 1 (EXACT)
|
||
* brickRate === 0 (EXACT, construction invariant — CRITICAL if non-zero)
|
||
* promotedToolCount === 2 (EXACT)
|
||
* promotedVariantCount=== 1 (EXACT)
|
||
* forbiddenRejected === 7 (EXACT)
|
||
* all counts exact (experienceCount=5/skillCount=4/workflowCount=3/...)
|
||
* renderMetrics contains every key metric line
|
||
*
|
||
* Reproducibility: deterministic synthetic fixtures (fixed numbers, no RNG, no
|
||
* Date.now()). Same fixture → bit-for-bit identical every run. Uses the REAL
|
||
* aggregateMetrics/renderMetrics/createEvolutionMetricsAggregator (no mocks).
|
||
*/
|
||
|
||
import { describe, it, expect, beforeAll } from "vitest";
|
||
import type {
|
||
ExperienceEntry,
|
||
GrownTool,
|
||
HardwareSkill,
|
||
MetricsSources,
|
||
Variant,
|
||
WorkflowGraph,
|
||
} from "../../src/shared/types.js";
|
||
import {
|
||
aggregateMetrics,
|
||
createEvolutionMetricsAggregator,
|
||
renderMetrics,
|
||
} from "../../src/core/evolution-metrics.js";
|
||
|
||
// ============================================================================
|
||
// 确定性合成 5 层注册表快照(固定数字,无 RNG)
|
||
// ============================================================================
|
||
|
||
const beta = { soc: "rdk-x5", kernel: "6.1.83", image: "v1", mac: "AA:01" };
|
||
const sig = { actionClass: "flash", target: "bootloader", constraints: ["aarch64"] };
|
||
|
||
/** 5 经验,hitCounts [3,0,2,1,0] → hitRate = 3/5 = 0.6 */
|
||
const EXPERIENCES: ExperienceEntry[] = [
|
||
{ id: "e0", boardFingerprint: beta, taskSignature: sig, outcome: "success", actionable: "dd if=img of=/dev/mmcblk0", evidenceRef: "s1", contextTags: ["flash"], validity: "repeatable", usage: "actionable", createdAt: 0, hitCount: 3, validatedBy: "physical-ground-truth" },
|
||
{ id: "e1", boardFingerprint: beta, taskSignature: sig, outcome: "failure", actionable: "n/a", antiPattern: "don't skip verify", evidenceRef: "s2", contextTags: ["flash"], validity: "irreversible-fail", usage: "anti-pattern", createdAt: 1, hitCount: 0, validatedBy: "physical-ground-truth" },
|
||
{ id: "e2", boardFingerprint: beta, taskSignature: sig, outcome: "success", actionable: "i2cdetect -y 0", evidenceRef: "s3", contextTags: ["i2c"], validity: "repeatable", usage: "actionable", createdAt: 2, hitCount: 2, validatedBy: "executable-stdout" },
|
||
{ id: "e3", boardFingerprint: beta, taskSignature: sig, outcome: "partial", actionable: "probe pmic", evidenceRef: "s4", contextTags: ["i2c"], validity: "once", usage: "hint", createdAt: 3, hitCount: 1, validatedBy: "executable-stdout" },
|
||
{ id: "e4", boardFingerprint: beta, taskSignature: sig, outcome: "success", actionable: "read dmesg", evidenceRef: "s5", contextTags: ["dmesg"], validity: "inferred", usage: "hint", createdAt: 4, hitCount: 0, validatedBy: "model-claim" },
|
||
];
|
||
|
||
/**
|
||
* 4 技能:
|
||
* infoGains [0.8, 0.6, 0.9, 0.3] → avgInfoGain = 0.65
|
||
* quality(exec 0.7 + rubric 0.3):
|
||
* s0: exec=1.0, rubric=1.0 → 1.0
|
||
* s1: exec=1.0, rubric=0.0 → 0.7
|
||
* s2: exec=0.0, rubric=1.0 → 0.3 (inactive)
|
||
* s3: exec=1.0, rubric=1.0 → 1.0
|
||
* → avgQualityScore = (1.0+0.7+0.3+1.0)/4 = 0.75
|
||
* active: s0,s1,s3 → activeSkillCount=3
|
||
*/
|
||
const SKILLS: HardwareSkill[] = [
|
||
{ id: "sk0", name: "i2c-recover", trigger: { kind: "event", pattern: "i2c.*no such device", actionClass: "i2c-probe" }, procedure: [{ tool: "i2cdetect", argsTemplate: "0x3a" }], granularity: "event-driven", provenance: ["s1"], quality: { rubricScore: 1.0, execPassRate: 1.0, execSamples: 5 }, infoGain: 0.8, createdAt: 0, hitCount: 4, active: true },
|
||
{ id: "sk1", name: "reg-read", trigger: { kind: "state", statePredicate: "reg locked", actionClass: "reg-read" }, procedure: [{ tool: "devmem", argsTemplate: "0x40020000" }], granularity: "event-driven", provenance: ["s2"], quality: { rubricScore: 0.0, execPassRate: 1.0, execSamples: 5 }, infoGain: 0.6, createdAt: 1, hitCount: 2, active: true },
|
||
{ id: "sk2", name: "old-probe", trigger: { kind: "event", pattern: "timeout", actionClass: "probe" }, procedure: [{ tool: "scan", argsTemplate: "" }], granularity: "task-level", provenance: ["s3"], quality: { rubricScore: 1.0, execPassRate: 0.0, execSamples: 5 }, infoGain: 0.9, createdAt: 2, hitCount: 0, active: false, supersededBy: "sk0" },
|
||
{ id: "sk3", name: "dt-parse", trigger: { kind: "event", pattern: "dtb error", actionClass: "device-tree" }, procedure: [{ tool: "fdtdump", argsTemplate: "" }], granularity: "event-driven", provenance: ["s4"], quality: { rubricScore: 1.0, execPassRate: 1.0, execSamples: 5 }, infoGain: 0.3, createdAt: 3, hitCount: 1, active: true },
|
||
];
|
||
|
||
/**
|
||
* 3 workflow,risks [1,0,1] → maxRisk = 1
|
||
* active: w0,w2 → activeWorkflowCount=2
|
||
*/
|
||
const WORKFLOWS: WorkflowGraph[] = [
|
||
{ id: "w0", taskClass: "flash", nodes: [{ id: "n0", tool: "verify", argsTemplate: "", irreversibility: "irreversible" }], edges: [], risk: 1, provenance: ["s1"], createdAt: 0, hitCount: 3, active: true },
|
||
{ id: "w1", taskClass: "i2c", nodes: [{ id: "n1", tool: "i2cdetect", argsTemplate: "", irreversibility: "reversible" }], edges: [], risk: 0, provenance: ["s3"], createdAt: 1, hitCount: 0, active: false, supersededBy: "w2" },
|
||
{ id: "w2", taskClass: "i2c", nodes: [{ id: "n2", tool: "i2cget", argsTemplate: "", irreversibility: "irreversible" }], edges: [], risk: 1, provenance: ["s3"], createdAt: 2, hitCount: 2, active: true },
|
||
];
|
||
|
||
/**
|
||
* 5 grownTools,2 promoted(stage=promoted)
|
||
*/
|
||
const GROWN_TOOLS: GrownTool[] = [
|
||
{ id: "g0", name: "dump-reg", toolClass: "read", definitionHash: "h0", stage: "promoted", usageCount: 10, passCount: 9, validity: "repeatable", createdAt: 0, lastUsed: 5 },
|
||
{ id: "g1", name: "scan-i2c", toolClass: "probe", definitionHash: "h1", stage: "promoted", usageCount: 12, passCount: 11, validity: "repeatable", createdAt: 1, lastUsed: 6 },
|
||
{ id: "g2", name: "parse-dtb", toolClass: "analyze", definitionHash: "h2", stage: "verified", usageCount: 4, passCount: 3, validity: "repeatable", createdAt: 2, lastUsed: 3 },
|
||
{ id: "g3", name: "probe-gpio", toolClass: "probe", definitionHash: "h3", stage: "draft", usageCount: 1, passCount: 1, validity: "once", createdAt: 3, lastUsed: 1 },
|
||
{ id: "g4", name: "old-tool", toolClass: "read", definitionHash: "h4", stage: "retired", usageCount: 20, passCount: 15, validity: "repeatable", createdAt: 4, lastUsed: 0, supersededBy: "g0" },
|
||
];
|
||
|
||
/**
|
||
* 4 variants,1 promoted(stage=promoted)
|
||
*/
|
||
const VARIANTS: Variant[] = [
|
||
{ id: "v0", name: "faster-probe", targetExtension: "env-probe", scaffoldClass: "probe", baseRef: "HEAD", factoryOverride: "o0", stage: "promoted", invariants: { tests: true, tsc: true, noSecrets: true, modelLaunch: true, sandboxGate: true, brickRateZero: true, riskNonIncrease: true, allPassed: true }, createdAt: 0, shadowResults: { succVariant: 15, succBaseline: 10, N: 20 } },
|
||
{ id: "v1", name: "better-dt", targetExtension: "device-tree", scaffoldClass: "analyze", baseRef: "HEAD", factoryOverride: "o1", stage: "candidate", invariants: { tests: false, tsc: false, noSecrets: true, modelLaunch: false, sandboxGate: true, brickRateZero: false, riskNonIncrease: true, allPassed: false }, createdAt: 1 },
|
||
{ id: "v2", name: "risky-build", targetExtension: "build-tracker", scaffoldClass: "analyze", baseRef: "HEAD", factoryOverride: "o2", stage: "rolled-back", invariants: { tests: true, tsc: true, noSecrets: true, modelLaunch: true, sandboxGate: true, brickRateZero: true, riskNonIncrease: false, allPassed: false }, createdAt: 2 },
|
||
{ id: "v3", name: "old-variant", targetExtension: "pdf-reader", scaffoldClass: "read", baseRef: "HEAD", factoryOverride: "o3", stage: "retired", invariants: { tests: true, tsc: true, noSecrets: true, modelLaunch: true, sandboxGate: true, brickRateZero: true, riskNonIncrease: true, allPassed: true }, createdAt: 3, supersededBy: "v0" },
|
||
];
|
||
|
||
/** forbidden 拒绝数 = 7(L3+L5 累计,注入因拒绝不入库) */
|
||
const FORBIDDEN_REJECTED = 7;
|
||
|
||
const SOURCES: MetricsSources = {
|
||
experiences: EXPERIENCES,
|
||
skills: SKILLS,
|
||
workflows: WORKFLOWS,
|
||
grownTools: GROWN_TOOLS,
|
||
variants: VARIANTS,
|
||
forbiddenRejected: FORBIDDEN_REJECTED,
|
||
};
|
||
|
||
// ============================================================================
|
||
// 预注册精确值(运行前定,失败不放宽)
|
||
// ============================================================================
|
||
|
||
const PRE_REGISTERED = {
|
||
experienceCount: 5,
|
||
experienceHitRate: 0.6,
|
||
skillCount: 4,
|
||
activeSkillCount: 3,
|
||
workflowCount: 3,
|
||
activeWorkflowCount: 2,
|
||
grownToolCount: 5,
|
||
promotedToolCount: 2,
|
||
variantCount: 4,
|
||
promotedVariantCount: 1,
|
||
maxRisk: 1,
|
||
brickRate: 0,
|
||
forbiddenRejected: 7,
|
||
avgInfoGain: 0.65,
|
||
avgQualityScore: 0.75,
|
||
ts: 42,
|
||
} as const;
|
||
|
||
// ============================================================================
|
||
// 结果表打印
|
||
// ============================================================================
|
||
|
||
interface EvalResults {
|
||
m: ReturnType<typeof aggregateMetrics>;
|
||
rendered: string;
|
||
}
|
||
|
||
function formatTable(R: EvalResults): string {
|
||
const m = R.m;
|
||
const lines: string[] = [];
|
||
lines.push("");
|
||
lines.push("=== X.4 Evolution-Metrics Eval — Results (deterministic synthetic fixture) ===");
|
||
lines.push("");
|
||
lines.push("Metric Pre-registered Observed Status");
|
||
lines.push("------------------------ -------------- -------------- ------");
|
||
const row = (name: string, pre: string, obs: string, ok: boolean): void => {
|
||
lines.push(`${name.padEnd(24)} ${pre.padEnd(16)} ${obs.padEnd(16)} ${ok ? "PASS" : "FAIL"}`);
|
||
};
|
||
row("experienceCount", String(PRE_REGISTERED.experienceCount), String(m.experienceCount), m.experienceCount === PRE_REGISTERED.experienceCount);
|
||
row("experienceHitRate", PRE_REGISTERED.experienceHitRate.toFixed(3), m.experienceHitRate.toFixed(3), Math.abs(m.experienceHitRate - PRE_REGISTERED.experienceHitRate) < 1e-9);
|
||
row("skillCount", String(PRE_REGISTERED.skillCount), String(m.skillCount), m.skillCount === PRE_REGISTERED.skillCount);
|
||
row("activeSkillCount", String(PRE_REGISTERED.activeSkillCount), String(m.activeSkillCount), m.activeSkillCount === PRE_REGISTERED.activeSkillCount);
|
||
row("workflowCount", String(PRE_REGISTERED.workflowCount), String(m.workflowCount), m.workflowCount === PRE_REGISTERED.workflowCount);
|
||
row("activeWorkflowCount", String(PRE_REGISTERED.activeWorkflowCount), String(m.activeWorkflowCount), m.activeWorkflowCount === PRE_REGISTERED.activeWorkflowCount);
|
||
row("grownToolCount", String(PRE_REGISTERED.grownToolCount), String(m.grownToolCount), m.grownToolCount === PRE_REGISTERED.grownToolCount);
|
||
row("promotedToolCount", String(PRE_REGISTERED.promotedToolCount), String(m.promotedToolCount), m.promotedToolCount === PRE_REGISTERED.promotedToolCount);
|
||
row("variantCount", String(PRE_REGISTERED.variantCount), String(m.variantCount), m.variantCount === PRE_REGISTERED.variantCount);
|
||
row("promotedVariantCount", String(PRE_REGISTERED.promotedVariantCount), String(m.promotedVariantCount), m.promotedVariantCount === PRE_REGISTERED.promotedVariantCount);
|
||
row("maxRisk", String(PRE_REGISTERED.maxRisk), String(m.maxRisk), m.maxRisk === PRE_REGISTERED.maxRisk);
|
||
row("brickRate (构造恒 0)", String(PRE_REGISTERED.brickRate), String(m.brickRate), m.brickRate === PRE_REGISTERED.brickRate);
|
||
row("forbiddenRejected", String(PRE_REGISTERED.forbiddenRejected), String(m.forbiddenRejected), m.forbiddenRejected === PRE_REGISTERED.forbiddenRejected);
|
||
row("avgInfoGain", PRE_REGISTERED.avgInfoGain.toFixed(3), m.avgInfoGain.toFixed(3), Math.abs(m.avgInfoGain - PRE_REGISTERED.avgInfoGain) < 1e-9);
|
||
row("avgQualityScore", PRE_REGISTERED.avgQualityScore.toFixed(3), m.avgQualityScore.toFixed(3), Math.abs(m.avgQualityScore - PRE_REGISTERED.avgQualityScore) < 1e-9);
|
||
row("ts", String(PRE_REGISTERED.ts), String(m.ts), m.ts === PRE_REGISTERED.ts);
|
||
lines.push("");
|
||
lines.push("--- renderMetrics output ---");
|
||
lines.push(R.rendered);
|
||
lines.push("");
|
||
return lines.join("\n");
|
||
}
|
||
|
||
// ============================================================================
|
||
// 主实验
|
||
// ============================================================================
|
||
|
||
describe("evolution-metrics eval — X.4 聚合定量实验", () => {
|
||
let R: EvalResults;
|
||
|
||
beforeAll(() => {
|
||
const m = aggregateMetrics(SOURCES, PRE_REGISTERED.ts);
|
||
const rendered = renderMetrics(m);
|
||
R = { m, rendered };
|
||
console.log(formatTable({ m, rendered }));
|
||
});
|
||
|
||
// ── 计数(EXACT)──
|
||
it("experienceCount === 5 (EXACT)", () => {
|
||
expect(R.m.experienceCount).toBe(PRE_REGISTERED.experienceCount);
|
||
});
|
||
|
||
it("skillCount === 4 (EXACT)", () => {
|
||
expect(R.m.skillCount).toBe(PRE_REGISTERED.skillCount);
|
||
});
|
||
|
||
it("activeSkillCount === 3 (EXACT)", () => {
|
||
expect(R.m.activeSkillCount).toBe(PRE_REGISTERED.activeSkillCount);
|
||
});
|
||
|
||
it("workflowCount === 3 (EXACT)", () => {
|
||
expect(R.m.workflowCount).toBe(PRE_REGISTERED.workflowCount);
|
||
});
|
||
|
||
it("activeWorkflowCount === 2 (EXACT)", () => {
|
||
expect(R.m.activeWorkflowCount).toBe(PRE_REGISTERED.activeWorkflowCount);
|
||
});
|
||
|
||
it("grownToolCount === 5 (EXACT)", () => {
|
||
expect(R.m.grownToolCount).toBe(PRE_REGISTERED.grownToolCount);
|
||
});
|
||
|
||
it("variantCount === 4 (EXACT)", () => {
|
||
expect(R.m.variantCount).toBe(PRE_REGISTERED.variantCount);
|
||
});
|
||
|
||
// ── 派生指标(EXACT)──
|
||
it("experienceHitRate === 0.6 (EXACT, 3/5 命中)", () => {
|
||
expect(R.m.experienceHitRate).toBeCloseTo(PRE_REGISTERED.experienceHitRate, 10);
|
||
});
|
||
|
||
it("avgInfoGain === 0.65 (EXACT, 4 技能均值)", () => {
|
||
expect(R.m.avgInfoGain).toBeCloseTo(PRE_REGISTERED.avgInfoGain, 10);
|
||
});
|
||
|
||
it("avgQualityScore === 0.75 (EXACT, exec 0.7 + rubric 0.3 均值)", () => {
|
||
expect(R.m.avgQualityScore).toBeCloseTo(PRE_REGISTERED.avgQualityScore, 10);
|
||
});
|
||
|
||
it("maxRisk === 1 (EXACT, max workflow risk)", () => {
|
||
expect(R.m.maxRisk).toBe(PRE_REGISTERED.maxRisk);
|
||
});
|
||
|
||
// ── 构造不变式(CRITICAL)──
|
||
it("brickRate === 0 (EXACT, 构造不变式 — 非零即 critical bug)", () => {
|
||
expect(R.m.brickRate).toBe(PRE_REGISTERED.brickRate);
|
||
expect(R.m.brickRate).toBe(0); // 双重断言,强调
|
||
});
|
||
|
||
// ── promoted 计数(EXACT)──
|
||
it("promotedToolCount === 2 (EXACT, 升级门 K+repeatable)", () => {
|
||
expect(R.m.promotedToolCount).toBe(PRE_REGISTERED.promotedToolCount);
|
||
});
|
||
|
||
it("promotedVariantCount === 1 (EXACT, §11.4 门+A/B 胜出)", () => {
|
||
expect(R.m.promotedVariantCount).toBe(PRE_REGISTERED.promotedVariantCount);
|
||
});
|
||
|
||
// ── forbidden 拒绝(EXACT)──
|
||
it("forbiddenRejected === 7 (EXACT, L3+L5 累计注入)", () => {
|
||
expect(R.m.forbiddenRejected).toBe(PRE_REGISTERED.forbiddenRejected);
|
||
});
|
||
|
||
// ── 时间戳透传 ──
|
||
it("ts === 42 (透传)", () => {
|
||
expect(R.m.ts).toBe(PRE_REGISTERED.ts);
|
||
});
|
||
|
||
// ── renderMetrics 含全部关键行 ──
|
||
it("renderMetrics 输出含全部关键指标行", () => {
|
||
const out = R.rendered;
|
||
expect(out).toContain("=== Evolution Metrics ===");
|
||
expect(out).toContain("experienceCount=5");
|
||
expect(out).toContain("hitRate=0.600");
|
||
expect(out).toContain("skillCount=4");
|
||
expect(out).toContain("active=3");
|
||
expect(out).toContain("avgInfoGain=0.650");
|
||
expect(out).toContain("avgQuality=0.750");
|
||
expect(out).toContain("workflowCount=3");
|
||
expect(out).toContain("maxRisk=1");
|
||
expect(out).toContain("grownToolCount=5");
|
||
expect(out).toContain("promoted=2");
|
||
expect(out).toContain("variantCount=4");
|
||
expect(out).toContain("brickRate=0");
|
||
expect(out).toContain("forbiddenRejected=7");
|
||
});
|
||
|
||
// ── 聚合器与纯函数一致性 ──
|
||
it("createEvolutionMetricsAggregator().aggregate 与 aggregateMetrics 一致", () => {
|
||
const agg = createEvolutionMetricsAggregator();
|
||
const viaAgg = agg.aggregate(SOURCES, PRE_REGISTERED.ts);
|
||
expect(viaAgg).toEqual(R.m);
|
||
});
|
||
});
|
||
|
||
// ============================================================================
|
||
// 安全原语直接断言(构造不变式强化)
|
||
// ============================================================================
|
||
|
||
describe("evolution-metrics eval — 构造不变式(直接)", () => {
|
||
it("brickRate 恒 0:多次聚合、空/非空 sources 均 0", () => {
|
||
// 非空 sources
|
||
expect(aggregateMetrics(SOURCES).brickRate).toBe(0);
|
||
// 空 sources
|
||
expect(aggregateMetrics({ experiences: [], skills: [], workflows: [], grownTools: [], variants: [] }).brickRate).toBe(0);
|
||
// 仅 forbidden 拒绝(无任何入库)
|
||
expect(aggregateMetrics({ experiences: [], skills: [], workflows: [], grownTools: [], variants: [], forbiddenRejected: 100 }).brickRate).toBe(0);
|
||
// 含不可逆 workflow(risk>0)仍 brickRate=0(risk 与 brickRate 解耦)
|
||
expect(aggregateMetrics({
|
||
experiences: [], skills: [], workflows: [{ id: "w", taskClass: "x", nodes: [{ id: "n", tool: "flash", argsTemplate: "", irreversibility: "irreversible" }], edges: [], risk: 5, provenance: [], createdAt: 0, hitCount: 0, active: true }],
|
||
grownTools: [], variants: [], forbiddenRejected: 0,
|
||
}).brickRate).toBe(0);
|
||
});
|
||
|
||
it("forbiddenRejected 透传:注入值精确出现在指标(拒绝不入库故注入)", () => {
|
||
const m = aggregateMetrics({ ...SOURCES, forbiddenRejected: 42 });
|
||
expect(m.forbiddenRejected).toBe(42);
|
||
});
|
||
|
||
it("maxRisk = max(workflow risk),空 workflow → 0", () => {
|
||
expect(aggregateMetrics({ ...SOURCES, workflows: [] }).maxRisk).toBe(0);
|
||
// 加一个 risk=3 的 workflow
|
||
const w3 = { id: "w3", taskClass: "x", nodes: [], edges: [], risk: 3, provenance: [], createdAt: 0, hitCount: 0, active: true };
|
||
expect(aggregateMetrics({ ...SOURCES, workflows: [...WORKFLOWS, w3] }).maxRisk).toBe(3);
|
||
});
|
||
});
|