🧪 Hold the pause-resume body mid-stream so the pause lands in flight

The native pause-resume scenario failed with both old and new builds
because the harness could not reliably see an upload in flight. Pause
itself behaved correctly.

Client progress counts bytes handed to the socket, and the daemon writes
it to SQLite once per second. Loopback buffers absorb about 1-3 MiB of a
body (sometimes over 5 MiB on Linux), so a 4 MiB PUT read as 0 or as
complete in nearly every sample. The trigger fired on the 3rd or 4th PUT
at best, later or never under load, and never on Linux. With
FILE_CONCURRENCY=2 and PUT_CONCURRENCY=1, the file queued behind the
in-flight PUT is already dispatched and finishes too, so a late pause
left nothing paused.

The fixture now holds the first direct body after 64 KiB until the
harness has paused. pause-resume uses the 12 MiB default size, so a held
body still reads as partial. The scenario asserts that the held object
arrives intact and that every received object was reserved before the
pause. The daemon already follows the documented contract: pause stops
new dispatch and lets in-flight objects finish.

pause-resume: macOS arm64 10/10, Linux arm64 10/10 (old harness 0/5 on
Linux). Full native matrix on macOS: 26/26.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
eust-w 2026-10-30 15:57:53 +08:00
parent 143a24b1e9
commit 846b806ed5
2 changed files with 26 additions and 5 deletions

View File

@ -67,6 +67,8 @@ class Control(http.server.ThreadingHTTPServer):
self.uploads={}
self.part_paths={}
self.part_attempts={}
self.held=None
self.release_held=threading.Event()
def session(self, sid):
s = dict(self.sessions[sid])
@ -128,6 +130,16 @@ class Handler(http.server.BaseHTTPRequestHandler):
if spool:spool.write(part)
n += len(part)
time.sleep(0.001 if number else 0.015)
# Client progress counts bytes handed to loopback socket buffers and is
# sampled once per second, so a freely streaming body can read as 0 or
# complete in every sample. Park the first direct body mid-stream until
# the harness has paused, keeping that upload verifiably in flight. The
# body must exceed what buffers absorb while held: over 5 MiB on Linux.
if args.scenario=='pause-resume' and not number:
with c.lock:
hold=c.held is None
if hold:c.held=(iid,n)
if hold:c.release_held.wait(30)
if args.scenario=='put-503' and c.injections==0:
c.injections+=1
return self.reply({'code':'injected_storage_unavailable','message':'503 fixture','retryable':True},503)
@ -250,7 +262,7 @@ try:
if args.scenario.startswith('concurrency-'):names=[f'parallel_{n}.bin' for n in range(8)]
for name in names:
f = episode / name
f.write_bytes(b'page' if args.scenario=='pagination-1003' else (name.encode()*2000000)[:4*1024*1024 if args.scenario=='pause-resume' or args.scenario.startswith('concurrency-') else 12*1024*1024])
f.write_bytes(b'page' if args.scenario=='pagination-1003' else (name.encode()*2000000)[:4*1024*1024 if args.scenario.startswith('concurrency-') else 12*1024*1024])
if args.scenario in ['multipart-large','multipart-crash']:
block=hashlib.md5(name.encode()).digest()*32768;remaining=(257 if name=='large_257.bin' else 321)*1024*1024
with f.open('wb') as writer:
@ -286,10 +298,13 @@ try:
assert time.monotonic() < deadline, 'isolated upload exceeded 180 seconds'
with http.open(f'http://127.0.0.1:{port}/api/progress', timeout=5) as response:
progress=json.load(response);observed += progress['items']
if not triggered and args.scenario in ['pause-resume','abort-active','crash-resume','source-rewrite','source-delete','multipart-crash'] and (args.scenario!='multipart-crash' or any(server.uploads.values())) and any(0<x['uploadedBytes']<x['totalBytes'] for x in progress['items']):
inflight=[x for x in progress['items'] if 0<x['uploadedBytes']<x['totalBytes'] and (args.scenario!='pause-resume' or server.held and x['itemId']==server.held[0])]
if not triggered and args.scenario in ['pause-resume','abort-active','crash-resume','source-rewrite','source-delete','multipart-crash'] and (args.scenario!='multipart-crash' or any(server.uploads.values())) and inflight:
triggered=True
if args.scenario=='pause-resume':
cli('pause');evidence_actions.append('pause during body upload')
# Every object reserved by now was dispatched before the pause took effect.
cli('pause');reserved={iid for iid,item in server.items.items() if item['status']!='planned'};server.release_held.set()
evidence_actions.append(f"pause while the fixture held {server.held[0]} after {server.held[1]} bytes; Web reported {inflight[0]['uploadedBytes']}/{inflight[0]['totalBytes']}")
elif args.scenario=='abort-active':
sid=next(iter(server.sessions));cli('abort',sid);evidence_actions.append('abort active session')
elif args.scenario in ['crash-resume','multipart-crash']:
@ -347,9 +362,12 @@ try:
elif args.scenario in ['pause-resume','abort-active' ,'crash-resume','source-rewrite','source-delete','put-503','put-lost-response','pagination-1003','status-diagnostics','concurrency-4','concurrency-1','multipart-large','multipart-crash']:
first=json.loads(stdout)
if args.scenario=='pause-resume':
assert triggered, 'pause fixture failed to observe an in-flight upload'
assert triggered, f'pause fixture failed to observe an in-flight upload; held={server.held}, last Web row={[x for x in observed if server.held and x["itemId"]==server.held[0]][-1:]}'
assert first.get('paused') is True,first
assert 0<len(server.received)<len(names),server.received
# Pause must not cancel the held body, and must not dispatch anything new.
held=server.items[server.held[0]]
assert server.received.get(held['id'])==(held['sizeBytes'],held['md5sum']),server.received
assert 0<len(server.received)<len(names) and set(server.received)<=reserved,(reserved,server.received)
assert all((episode/name).exists() for name in names)
cli('resume');time.sleep(.2)
# The public resume entry may start the automatic worker. Wait for
@ -443,6 +461,7 @@ finally:
proc.terminate()
proc.wait(timeout=10)
log.close()
server.release_held.set()
server.shutdown()
server.server_close()
(OUT / 'calls.json').write_text(json.dumps(server.calls))

View File

@ -319,6 +319,8 @@ go test ./internal/api ./internal/objectsync ./internal/store -count=1
2026-10-30 同尺寸改写识别修复:macOS arm64 workspace 297 项、Linux arm64 workspace 300 项通过,真实清理链路 macOS 13/13、Linux 12/12 通过;本机 harness 的 `pause-resume` 场景修复前后同样失败。详见 [修复报告](../../doc/drosync-Linux同尺寸改写识别修复-2026-10-30.md)。
2026-10-30 本机 harness `pause-resume` 修复:失败来自 harness 观测时序,不是 daemon 暂停逻辑。客户端进度统计已交给 socket 的字节,每秒采样一次;回环缓冲可吸收数 MiB(Linux 偶尔超过 5 MiB),4 MiB 请求体在采样中常显示为 0 或已完成,暂停因此触发过晚或从未触发。fixture 现在让首个请求体在传输途中挂起,暂停生效后再放行;场景改用 12 MiB 文件,并断言在途对象完整到达、暂停后没有派发新对象。已派发、正在排队等待 PUT 名额的文件,按约定可以继续完成。macOS arm64 与 Linux arm64 各 10/10 通过,macOS 全部 26 个场景通过。
2026-10-28 清理修复的既有验证记录:macOS arm64 Rust workspace 280 项通过;Linux arm64 core/state 53 + 30 项通过;Go 相关包及变更组件 Clippy 通过。该记录使用的具体工具链、测试范围与原始日志见 [验证报告](../../doc/drosync-partial逐文件安全清理-2026-10-28.md),不应扩展为四平台发行验收或现场真实 BOS 删除验收。
### 发布包不是源码 ZIP