Correct restored device import context · 9334a29

Correct restored device import context
Browse files

bogdan committed a day ago

main

1 parent 92592cd

commit 9334a29

Showing 2 changed files with 31 additions and 22 deletions

+14 -11

HealthProbe/Doc/04-project/Development-Log.md

@@ -71,30 +71,33 @@ Another 2026-06-06 report from the restored device later imported both Blood
 Pressure Systolic and Diastolic successfully, `5,124` samples each. After the
 device unfroze enough for Settings to open, Blood Pressure appeared not
 authorized there, even though the user had selected all types in the HealthProbe
-authorization prompt. This indicates the restored device's Health authorization
-state is changing or being reconciled independently of HealthProbe's read-only
-queries.
+authorization prompt.
 
 Interpretation:
 - this is evidence of a device / iOS HealthKit authorization or Health database
   state change;
 - it is not evidence that HealthProbe cannot import Blood Pressure;
+- it is not evidence that HealthProbe changed Health data, because the app only
+  requests read access and runs read-only queries;
 - future runs should preserve these reports as before/after context.
 
-### 2026-06-06 - Suspected iCloud Health Reconciliation On Restored Device
+### 2026-06-06 - Restored Device Instability After Health Access
 
 After HealthProbe access requests, broad HealthKit queries, and retry attempts on
-the restored pre-loss reference device, the user observed that data from
-2026-05-24 appeared on the device. This suggests the device may have entered a
-Health/iCloud synchronization, reindexing, or aggregation phase.
+the restored pre-loss reference device, the user observed that entries dated
+2026-05-24 appeared on the device. This was later clarified as fresh data coming
+from an Apple Watch paired on 2026-05-24 in an attempt to unstuck Health
+synchronization, not as evidence that old data was pulled from or pushed to
+iCloud by HealthProbe activity.
 
 Interpretation:
 - the restored reference device should not be treated as a stable performance
-  benchmark while this reconciliation appears active;
-- snapshots from this period should be labelled as captured during suspected
-  Health/iCloud reconciliation;
+  benchmark while the Health UI, Health authorization UI, and HealthKit query
+  behavior remain unstable;
+- snapshots from this period should be labelled as captured during restored
+  device instability and Apple Watch pairing/sync-repair experiments;
 - the device remains important forensic evidence because HealthProbe operations
-  may have surfaced or triggered latent Health database synchronization work;
+  may surface latent Health database or authorization issues;
 - stable import performance comparisons should use another device until the
   Health UI, Health authorization UI, and repeated HealthProbe diagnostics become
   consistent again.


+17 -11

HealthProbe/Doc/04-project/Import-Optimization-Log.md

View

@@ -599,8 +599,8 @@ rows exist".
 | 2026-06-05 | `e7d45a2` | Count observation events from the event table first. | Confirmed on a full-profile repeated capture with `buildFingerprint: 1.0(1)-1780646299-92064`: wall clock `15.6s`, `127/127` complete, `CaptureModes: unchangedDelta=114, delta=13`, and `DeltaEvents: 99`. `SummedFinalizeEventCountElapsed` dropped from the post-`bf5a861` overnight baseline `2.5s` to `0.0s`; total finalize dropped to `1.7s`, while daily aggregate rebuild stayed `0.0s`. Remaining cost is processing: `9.1s` total, led by Heart Rate `4.6s` for `18` events, Active Energy `1.8s` for `11` events, and Basal Energy `1.5s` for `5` events. |
 | 2026-06-05 | `cfd9de8` | Split processing timing diagnostics. | Confirmed on a full-profile repeated capture with `buildFingerprint: 1.0(1)-1780683224-92064`: wall clock `14.7s`, `127/127` complete, `CaptureModes: unchangedDelta=120, delta=7`, and `DeltaEvents: 11`. `SummedProcessingElapsed` was `8.5s` and `SummedProcessingRecordArchiveRebuildElapsed` was also `8.5s`; delta apply, initial record processing, record archive finalization, and processing other all rounded to `0.0s`. Per-type rebuild cost dominated changed high-volume metrics: Heart Rate `4.4s`, Active Energy `1.7s`, Basal Energy `1.5s`, Steps `0.4s`, and Walking + Running Distance `0.4s`. Conclusion: the repeated-capture bottleneck is no longer SQLite finalization; it is the legacy compact `recordArchiveData` rebuild for changed types. |
 | 2026-06-05 | `0db2f5e` | Skip legacy compact archive rebuild for SQLite-backed deltas. | Confirmed on two full-profile repeated captures. First report, `buildFingerprint: 1.0(1)-1780689289-92064`, completed in `6.2s` with `127/127` complete, `CaptureModes: unchangedDelta=118, delta=9`, `DeltaEvents: 116`, `SummedProcessingElapsed: 0.0s`, and `SummedProcessingRecordArchiveRebuildElapsed: 0.0s`; Heart Rate and Active Energy each completed in `0.2s` with no rebuild. Second report, `buildFingerprint: 1.0(1)-1780689890-92064`, completed in `5.8s` with `CaptureModes: unchangedDelta=121, delta=6`, `DeltaEvents: 7`, fetch `2.0s`, insert `0.1s`, finalize `1.7s`, residual `1.1s`, and processing/rebuild still `0.0s`. Conclusion: the legacy compact archive rebuild was the repeated-capture bottleneck and is now removed from the normal SQLite-backed delta path; the repeated full-profile floor is now roughly `6s` on this device/database. |
-| 2026-06-06 | pending | Expose saved diagnostic reports in Settings. | Diagnostic reports were already persisted under Application Support at snapshot completion, but they were not discoverable from the app after dismissing the result sheet. Settings now lists the latest saved reports, opens them with the same chunked lazy diagnostics viewer, supports copy, and allows per-report deletion. Expected signal: future import analysis can recover the exact report text after the fact instead of depending on screenshots or manual copy at completion time. |
-| 2026-06-06 | pending | Stop HealthKit date-boundary queries on timeout cancellation. | A restored-device report with `buildFingerprint: 1.0(1)-1780695759-92064` completed as partial after `263m 4s`. `Zinc` was the only degraded metric and spent `262m 50s` in fetch, with both `earliest_sample` and `latest_sample` reported as timeout after `15770.00s` despite `timeoutConfigured: 15.0s`. Blood Pressure imported successfully in the same report, so this was not a general authorization failure. Diagnosis: the timeout task marked cancellation, but the underlying `HKSampleQuery` for earliest/latest date was not stopped, so the task group could not return until HealthKit eventually completed. Date-boundary queries now use the cancellable HealthKit continuation box and call `HKHealthStore.stop(_:)` when the timeout cancels the task. Expected signal: a wedged first metric should fail near its configured timeout, not after hours, and the remaining metrics should continue. |
+| 2026-06-06 | `fbbca81` | Expose saved diagnostic reports in Settings. | Diagnostic reports were already persisted under Application Support at snapshot completion, but they were not discoverable from the app after dismissing the result sheet. Settings now lists the latest saved reports, opens them with the same chunked lazy diagnostics viewer, supports copy, and allows per-report deletion. Expected signal: future import analysis can recover the exact report text after the fact instead of depending on screenshots or manual copy at completion time. |
+| 2026-06-06 | `92592cd` | Stop HealthKit date-boundary queries on timeout cancellation. | A restored-device report with `buildFingerprint: 1.0(1)-1780695759-92064` recorded `263m 4s` total and `262m 50s` fetch for `Zinc`, with both `earliest_sample` and `latest_sample` reported as timeout after `15770.00s` despite `timeoutConfigured: 15.0s`. This timing is not a clean import-duration measurement: the phone had already been unresponsive for roughly `4h22m`, while the app/snapshot attempt was started shortly before the report. The report still exposed a defensive gap: date-boundary queries used plain continuations, so cancellation could mark timeout without explicitly stopping the underlying `HKSampleQuery`. Date-boundary queries now use the cancellable HealthKit continuation box and call `HKHealthStore.stop(_:)` when the timeout cancels the task. Expected signal: if a future boundary query wedges, it should fail near its configured timeout and leave the rest of the capture able to continue; do not use the `263m` value as a throughput datapoint. |
 
 ## Current Diagnosis
 
@@ -764,10 +764,12 @@ The likely bottleneck is per-row SQLite work:
 - A 2026-06-06 screenshot from the large-database device showed a fast partial
   retry after first import: `127` metrics, `8,411,713` records fetched, `11.2s`
   duration, and `2` failed metrics (`Blood Pressure Diastolic` and
-  `Blood Pressure Systolic`). The device's Health database is known to be in a
-  frozen / non-syncing iCloud state from roughly one year earlier, except those
-  two blood-pressure values appear to still sync for unknown reasons. Treat the
-  two failed blood-pressure metrics as device/data-state evidence, not as an
+  `Blood Pressure Systolic`). The device's Health database is known to be a
+  restored pre-loss reference state with unstable Health/authorization behavior.
+  Entries dated 2026-05-24 were later clarified as fresh data from an Apple
+  Watch paired on that day in an attempt to repair synchronization, not evidence
+  that HealthProbe caused old data to be pulled from or pushed to iCloud. Treat
+  the two failed blood-pressure metrics as device/data-state evidence, not as an
   import-performance regression, unless a matching diagnostic report proves a
   HealthProbe error. The attached copied diagnostic text did not match that
   screenshot; it was the older `1.0(1)-1780689890-92064` repeated-capture report
@@ -802,8 +804,9 @@ The likely bottleneck is per-row SQLite work:
   the Health UI displayed different data.
 - This large-database device is not the untouched original source. It is a
   restored copy of the pre-data-loss database from a backup that still exists,
-  and it has already been used for multiple attempts to restart or force natural
-  iCloud synchronization. It can therefore be used for aggressive experiments
+  and it has already been used for multiple attempts to restart or repair Health
+  synchronization, including pairing an Apple Watch that later produced fresh
+  entries dated 2026-05-24. It can therefore be used for aggressive experiments
   when needed, but results must be labelled as coming from a restored
   pre-loss-reference copy rather than from the original live device state.
 
@@ -816,9 +819,12 @@ The likely bottleneck is per-row SQLite work:
   Diagnostics sheet is useful, but future comparisons should not depend on the
   operator remembering to copy the visible report. Saved reports are now exposed
   in Settings so the exact text can be recovered later.
-- A timeout report with elapsed time far above `timeoutConfigured` means the
-  underlying HealthKit query ignored task cancellation. Date-boundary queries now
-  explicitly stop their `HKSampleQuery` on cancellation.
+- A timeout report with elapsed time far above `timeoutConfigured` can indicate
+  either a stale/invalid timing baseline after device unresponsiveness or an
+  underlying HealthKit query that was not stopped on task cancellation.
+  Date-boundary queries now explicitly stop their `HKSampleQuery` on
+  cancellation; future reports should be judged by whether the elapsed time is
+  near the configured timeout.
 - After a completed import, the app may remain unresponsive or crash in legacy
   post-import cache work. A 2026-06-03 console log showed Heart Rate and Active
   Energy `TypeCount.detailCacheData` precompute immediately before a Core Data


	@@ -71,30 +71,33 @@ Another 2026-06-06 report from the restored device later imported both Blood
71	71	Pressure Systolic and Diastolic successfully, `5,124` samples each. After the
72	72	device unfroze enough for Settings to open, Blood Pressure appeared not
73	73	authorized there, even though the user had selected all types in the HealthProbe
74		-authorization prompt. This indicates the restored device's Health authorization
75		-state is changing or being reconciled independently of HealthProbe's read-only
76		-queries.
	74	+authorization prompt.
77	75
78	76	Interpretation:
79	77	- this is evidence of a device / iOS HealthKit authorization or Health database
80	78	state change;
81	79	- it is not evidence that HealthProbe cannot import Blood Pressure;
	80	+- it is not evidence that HealthProbe changed Health data, because the app only
	81	+ requests read access and runs read-only queries;
82	82	- future runs should preserve these reports as before/after context.
83	83
84		-### 2026-06-06 - Suspected iCloud Health Reconciliation On Restored Device
	84	+### 2026-06-06 - Restored Device Instability After Health Access
85	85
86	86	After HealthProbe access requests, broad HealthKit queries, and retry attempts on
87		-the restored pre-loss reference device, the user observed that data from
88		-2026-05-24 appeared on the device. This suggests the device may have entered a
89		-Health/iCloud synchronization, reindexing, or aggregation phase.
	87	+the restored pre-loss reference device, the user observed that entries dated
	88	+2026-05-24 appeared on the device. This was later clarified as fresh data coming
	89	+from an Apple Watch paired on 2026-05-24 in an attempt to unstuck Health
	90	+synchronization, not as evidence that old data was pulled from or pushed to
	91	+iCloud by HealthProbe activity.
90	92
91	93	Interpretation:
92	94	- the restored reference device should not be treated as a stable performance
93		- benchmark while this reconciliation appears active;
94		-- snapshots from this period should be labelled as captured during suspected
95		- Health/iCloud reconciliation;
	95	+ benchmark while the Health UI, Health authorization UI, and HealthKit query
	96	+ behavior remain unstable;
	97	+- snapshots from this period should be labelled as captured during restored
	98	+ device instability and Apple Watch pairing/sync-repair experiments;
96	99	- the device remains important forensic evidence because HealthProbe operations
97		- may have surfaced or triggered latent Health database synchronization work;
	100	+ may surface latent Health database or authorization issues;
98	101	- stable import performance comparisons should use another device until the
99	102	Health UI, Health authorization UI, and repeated HealthProbe diagnostics become
100	103	consistent again.

	@@ -599,8 +599,8 @@ rows exist".
599	599	\| 2026-06-05 \| `e7d45a2` \| Count observation events from the event table first. \| Confirmed on a full-profile repeated capture with `buildFingerprint: 1.0(1)-1780646299-92064`: wall clock `15.6s`, `127/127` complete, `CaptureModes: unchangedDelta=114, delta=13`, and `DeltaEvents: 99`. `SummedFinalizeEventCountElapsed` dropped from the post-`bf5a861` overnight baseline `2.5s` to `0.0s`; total finalize dropped to `1.7s`, while daily aggregate rebuild stayed `0.0s`. Remaining cost is processing: `9.1s` total, led by Heart Rate `4.6s` for `18` events, Active Energy `1.8s` for `11` events, and Basal Energy `1.5s` for `5` events. \|
600	600	\| 2026-06-05 \| `cfd9de8` \| Split processing timing diagnostics. \| Confirmed on a full-profile repeated capture with `buildFingerprint: 1.0(1)-1780683224-92064`: wall clock `14.7s`, `127/127` complete, `CaptureModes: unchangedDelta=120, delta=7`, and `DeltaEvents: 11`. `SummedProcessingElapsed` was `8.5s` and `SummedProcessingRecordArchiveRebuildElapsed` was also `8.5s`; delta apply, initial record processing, record archive finalization, and processing other all rounded to `0.0s`. Per-type rebuild cost dominated changed high-volume metrics: Heart Rate `4.4s`, Active Energy `1.7s`, Basal Energy `1.5s`, Steps `0.4s`, and Walking + Running Distance `0.4s`. Conclusion: the repeated-capture bottleneck is no longer SQLite finalization; it is the legacy compact `recordArchiveData` rebuild for changed types. \|
601	601	\| 2026-06-05 \| `0db2f5e` \| Skip legacy compact archive rebuild for SQLite-backed deltas. \| Confirmed on two full-profile repeated captures. First report, `buildFingerprint: 1.0(1)-1780689289-92064`, completed in `6.2s` with `127/127` complete, `CaptureModes: unchangedDelta=118, delta=9`, `DeltaEvents: 116`, `SummedProcessingElapsed: 0.0s`, and `SummedProcessingRecordArchiveRebuildElapsed: 0.0s`; Heart Rate and Active Energy each completed in `0.2s` with no rebuild. Second report, `buildFingerprint: 1.0(1)-1780689890-92064`, completed in `5.8s` with `CaptureModes: unchangedDelta=121, delta=6`, `DeltaEvents: 7`, fetch `2.0s`, insert `0.1s`, finalize `1.7s`, residual `1.1s`, and processing/rebuild still `0.0s`. Conclusion: the legacy compact archive rebuild was the repeated-capture bottleneck and is now removed from the normal SQLite-backed delta path; the repeated full-profile floor is now roughly `6s` on this device/database. \|
602		-\| 2026-06-06 \| pending \| Expose saved diagnostic reports in Settings. \| Diagnostic reports were already persisted under Application Support at snapshot completion, but they were not discoverable from the app after dismissing the result sheet. Settings now lists the latest saved reports, opens them with the same chunked lazy diagnostics viewer, supports copy, and allows per-report deletion. Expected signal: future import analysis can recover the exact report text after the fact instead of depending on screenshots or manual copy at completion time. \|
603		-\| 2026-06-06 \| pending \| Stop HealthKit date-boundary queries on timeout cancellation. \| A restored-device report with `buildFingerprint: 1.0(1)-1780695759-92064` completed as partial after `263m 4s`. `Zinc` was the only degraded metric and spent `262m 50s` in fetch, with both `earliest_sample` and `latest_sample` reported as timeout after `15770.00s` despite `timeoutConfigured: 15.0s`. Blood Pressure imported successfully in the same report, so this was not a general authorization failure. Diagnosis: the timeout task marked cancellation, but the underlying `HKSampleQuery` for earliest/latest date was not stopped, so the task group could not return until HealthKit eventually completed. Date-boundary queries now use the cancellable HealthKit continuation box and call `HKHealthStore.stop(_:)` when the timeout cancels the task. Expected signal: a wedged first metric should fail near its configured timeout, not after hours, and the remaining metrics should continue. \|
	602	+\| 2026-06-06 \| `fbbca81` \| Expose saved diagnostic reports in Settings. \| Diagnostic reports were already persisted under Application Support at snapshot completion, but they were not discoverable from the app after dismissing the result sheet. Settings now lists the latest saved reports, opens them with the same chunked lazy diagnostics viewer, supports copy, and allows per-report deletion. Expected signal: future import analysis can recover the exact report text after the fact instead of depending on screenshots or manual copy at completion time. \|
	603	+\| 2026-06-06 \| `92592cd` \| Stop HealthKit date-boundary queries on timeout cancellation. \| A restored-device report with `buildFingerprint: 1.0(1)-1780695759-92064` recorded `263m 4s` total and `262m 50s` fetch for `Zinc`, with both `earliest_sample` and `latest_sample` reported as timeout after `15770.00s` despite `timeoutConfigured: 15.0s`. This timing is not a clean import-duration measurement: the phone had already been unresponsive for roughly `4h22m`, while the app/snapshot attempt was started shortly before the report. The report still exposed a defensive gap: date-boundary queries used plain continuations, so cancellation could mark timeout without explicitly stopping the underlying `HKSampleQuery`. Date-boundary queries now use the cancellable HealthKit continuation box and call `HKHealthStore.stop(_:)` when the timeout cancels the task. Expected signal: if a future boundary query wedges, it should fail near its configured timeout and leave the rest of the capture able to continue; do not use the `263m` value as a throughput datapoint. \|
604	604
605	605	## Current Diagnosis
606	606
	@@ -764,10 +764,12 @@ The likely bottleneck is per-row SQLite work:
764	764	- A 2026-06-06 screenshot from the large-database device showed a fast partial
765	765	retry after first import: `127` metrics, `8,411,713` records fetched, `11.2s`
766	766	duration, and `2` failed metrics (`Blood Pressure Diastolic` and
767		- `Blood Pressure Systolic`). The device's Health database is known to be in a
768		- frozen / non-syncing iCloud state from roughly one year earlier, except those
769		- two blood-pressure values appear to still sync for unknown reasons. Treat the
770		- two failed blood-pressure metrics as device/data-state evidence, not as an
	767	+ `Blood Pressure Systolic`). The device's Health database is known to be a
	768	+ restored pre-loss reference state with unstable Health/authorization behavior.
	769	+ Entries dated 2026-05-24 were later clarified as fresh data from an Apple
	770	+ Watch paired on that day in an attempt to repair synchronization, not evidence
	771	+ that HealthProbe caused old data to be pulled from or pushed to iCloud. Treat
	772	+ the two failed blood-pressure metrics as device/data-state evidence, not as an
771	773	import-performance regression, unless a matching diagnostic report proves a
772	774	HealthProbe error. The attached copied diagnostic text did not match that
773	775	screenshot; it was the older `1.0(1)-1780689890-92064` repeated-capture report
	@@ -802,8 +804,9 @@ The likely bottleneck is per-row SQLite work:
802	804	the Health UI displayed different data.
803	805	- This large-database device is not the untouched original source. It is a
804	806	restored copy of the pre-data-loss database from a backup that still exists,
805		- and it has already been used for multiple attempts to restart or force natural
806		- iCloud synchronization. It can therefore be used for aggressive experiments
	807	+ and it has already been used for multiple attempts to restart or repair Health
	808	+ synchronization, including pairing an Apple Watch that later produced fresh
	809	+ entries dated 2026-05-24. It can therefore be used for aggressive experiments
807	810	when needed, but results must be labelled as coming from a restored
808	811	pre-loss-reference copy rather than from the original live device state.
809	812
	@@ -816,9 +819,12 @@ The likely bottleneck is per-row SQLite work:
816	819	Diagnostics sheet is useful, but future comparisons should not depend on the
817	820	operator remembering to copy the visible report. Saved reports are now exposed
818	821	in Settings so the exact text can be recovered later.
819		-- A timeout report with elapsed time far above `timeoutConfigured` means the
820		- underlying HealthKit query ignored task cancellation. Date-boundary queries now
821		- explicitly stop their `HKSampleQuery` on cancellation.
	822	+- A timeout report with elapsed time far above `timeoutConfigured` can indicate
	823	+ either a stale/invalid timing baseline after device unresponsiveness or an
	824	+ underlying HealthKit query that was not stopped on task cancellation.
	825	+ Date-boundary queries now explicitly stop their `HKSampleQuery` on
	826	+ cancellation; future reports should be judged by whether the elapsed time is
	827	+ near the configured timeout.
822	828	- After a completed import, the app may remain unresponsive or crash in legacy
823	829	post-import cache work. A 2026-06-03 console log showed Heart Rate and Active
824	830	Energy `TypeCount.detailCacheData` precompute immediately before a Core Data