|
Bogdan Timofte
authored
a month ago
|
1
|
# HealthProbe Implementation Status
|
|
|
2
|
|
|
|
3
|
## Overview
|
|
|
4
|
|
|
|
5
|
HealthProbe's comprehensive snapshot + delta system has been implemented according to the detailed plan. The project builds successfully with no compilation errors.
|
|
|
6
|
|
|
|
7
|
## Completed Components (100%)
|
|
|
8
|
|
|
|
9
|
### Models (Step 1-3)
|
|
|
10
|
✅ **SnapshotQuality.swift** — All quality states (complete, partial, unauthorized, loading, failed)
|
|
|
11
|
✅ **AnomalyType.swift** — All anomaly types + Severity + TypeTransition + TypeDeltaReason enums
|
|
|
12
|
✅ **HealthSnapshot.swift** — Chain metadata, quality, trigger context, registry fingerprinting, timezone context
|
|
|
13
|
✅ **TypeCount.swift** — Count with hash, date range, quality, yearly counts with cascade relationship
|
|
Bogdan Timofte
authored
2 weeks ago
|
14
|
✅ **SnapshotDelta.swift** — Local delta with checksums and cascade relationship to TypeDeltas
|
|
|
15
|
✅ **TypeDelta.swift** — Per-type local delta with transition, reason, quality before/after, yearly count note
|
|
Bogdan Timofte
authored
a month ago
|
16
|
✅ **AnomalyRecord.swift** — Anomaly record with deltaID set structurally by detector, never by caller
|
|
|
17
|
✅ **OperationLog.swift** — Audit trail for destructive operations with JSON-encoded affected snapshot IDs
|
|
|
18
|
✅ **YearlyCount.swift** — Per-year sample counts with approximation flag
|
|
|
19
|
|
|
|
20
|
### Services (Step 4-12)
|
|
|
21
|
|
|
|
22
|
#### Step 5: HashService ✅
|
|
|
23
|
- `typeHash()` — SHA256 of typeIdentifier|count|earliest|latest (ISO8601 with fractional seconds)
|
|
|
24
|
- `snapshotChecksum()` — Filters on quality==.complete (not hash!=""), concatenates type hashes
|
|
|
25
|
- `typeSetHash()` — SHA256 of sorted active typeIdentifiers (covers full intended registry)
|
|
|
26
|
|
|
|
27
|
#### Step 11 & 11b: HealthKitService & ObserverService ✅
|
|
|
28
|
- Per-type fetch with **15-second combined timeout** (distribution + earliestDate + latestDate)
|
|
|
29
|
- Concurrency capped at 6 simultaneous type fetches (prevents HealthKit resource exhaustion)
|
|
|
30
|
- Per-type quality detection (unauthorized, failed, complete)
|
|
|
31
|
- Real earliestDate/latestDate from separate HKSampleQuery (NOT from bin boundaries)
|
|
|
32
|
- YearlyCount population from distribution bins with isApproximate flag
|
|
|
33
|
- Snapshot quality aggregation (loading > unauthorized > partial > complete)
|
|
|
34
|
- Chain metadata set before save (previousSnapshotID, isChainStart, monitoredTypeSetHash)
|
|
|
35
|
- Auto-detect post-restore (full deny → complete transition, or chain start > 1000 records)
|
|
|
36
|
- **Post-save pipeline**: DeltaService → AnomalyDetector → OperationLog
|
|
|
37
|
- **ObserverService**: debounce (10 min), manual overlap suppression, all-monitored-types snapshot
|
|
|
38
|
- **Background delivery**: .immediate for heart rate/steps, .daily for others
|
|
|
39
|
|
|
|
40
|
#### Step 7: DeltaService ✅
|
|
|
41
|
- Computes and saves SnapshotDelta with TypeDeltas
|
|
|
42
|
- **Reason assignment with priority**: authorizationChanged > unsupported > registryChanged > unknown > normal
|
|
|
43
|
- **Unavailable count guard**: if either quality != .complete, countDelta = 0 (never from -1)
|
|
|
44
|
- **YearlyCount timezone guard**: if timezone changes, set countDelta = 0 and yearlyCountNote
|
|
|
45
|
- **Delta merge** (for intermediate deletion):
|
|
|
46
|
- Recomputes checksums from surrounding snapshots (never carries old checksums)
|
|
|
47
|
- Handles disappeared→appeared transition (remove from merged delta if type existed only in deleted snapshot)
|
|
|
48
|
- Applies unavailable count guard and reason priority to merged result
|
|
|
49
|
- Sets timezone note if either source had it
|
|
|
50
|
|
|
|
51
|
#### Step 8: AnomalyDetector ✅
|
|
|
52
|
- **Pure function**: no context mutation, receives TypeCount maps, returns DetectionResult
|
|
|
53
|
- **Quality gate**: both snapshots must be .complete (suppresses ALL detection including first auth after full deny)
|
|
|
54
|
- **Registry gate**: skips appeared/disappeared anomalies if reason != .normal
|
|
|
55
|
- **count = -1 guard**: skips any TypeDelta with qualityBefore or qualityAfter != .complete
|
|
|
56
|
- **Anomaly detection rules**:
|
|
|
57
|
- `historicalInsertion` — countDelta > 0 AND (earlier earliest date OR recent latest with increased count)
|
|
|
58
|
- `deletion` — countDelta < 0 (severity based on % loss)
|
|
|
59
|
- `duplication` — countDelta > 50% AND date ranges within 1 day
|
|
|
60
|
- `silentReplacement` — countDelta == 0 AND hash differs (best-effort, MVP limitation)
|
|
|
61
|
- `syncAnomaly` — ≥4 types with |delta| > 10% (critical severity)
|
|
|
62
|
- **isPostRestore suppression**:
|
|
|
63
|
- Suppresses syncAnomaly if previous.isPostRestore && previous.isPostRestoreSuppressedDeltaID == nil
|
|
|
64
|
- Suppression token consumed via DetectionResult, persisted by HealthKitService
|
|
|
65
|
- Forwarded past low-quality successors (quality gate prevents consumption on incomplete snapshots)
|
|
|
66
|
- **AnomalyRecord.deltaID**: set internally, structural guarantee (impossible to return record without deltaID)
|
|
|
67
|
|
|
|
68
|
#### Step 4: KeychainService ✅
|
|
|
69
|
- Stable device ID persisted in Keychain (service: "ro.xdev.healthprobe.deviceid", account: "stable_device_id")
|
|
|
70
|
- Detects DB reset: swiftDataStoreIsEmpty + existing keychain ID → recoveredDeviceID = true
|
|
|
71
|
- In-process cache for repeated lookups
|
|
|
72
|
|
|
|
73
|
#### Step 6 & 9: IntegrityService & Quality Aggregation ✅
|
|
|
74
|
- `validate()` — strict mode:
|
|
|
75
|
- Recomputes checksum from TypeCounts
|
|
|
76
|
- Compares with delta.checksumAfter
|
|
|
77
|
- Returns .valid or .checksumMismatch / .missingDelta / .corrupted
|
|
|
78
|
- `validateChain()` — walk backwards from latest via previousSnapshotID:
|
|
|
79
|
- **Fork detection**: asserts no duplicate previousSnapshotID (returns .corrupted immediately)
|
|
|
80
|
- Stops at first mismatch (no auto-repair, no skips)
|
|
|
81
|
- **Quality aggregation**: loading > unauthorized (only if ALL) > partial (any failed/unauthorized) > complete
|
|
|
82
|
|
|
|
83
|
#### Step 10: SnapshotLifecycleService ✅
|
|
|
84
|
- `previewDeletion()` — advisory integrity check, surfaces willBreakChain warning to UI
|
|
|
85
|
- `delete()` — handles all position cases (oldest, latest, intermediate):
|
|
|
86
|
- **Oldest**: set next as chain start
|
|
|
87
|
- **Latest**: just delete
|
|
|
88
|
- **Intermediate**: merge deltas, recompute checksums, update nextSnapshot.previousSnapshotID
|
|
|
89
|
- **OperationLog**: always written atomically with deletive changes
|
|
|
90
|
- **Post-save verification**: re-fetches log by ID, recovery re-insert if missing, logs critical error
|
|
|
91
|
|
|
Bogdan Timofte
authored
2 weeks ago
|
92
|
#### Step 12: Local-only storage refactor ✅
|
|
|
93
|
- Removed CloudKitSyncService and CloudKit-pending chain states
|
|
Bogdan Timofte
authored
a month ago
|
94
|
- **ModelContainer split**:
|
|
Bogdan Timofte
authored
2 weeks ago
|
95
|
- uiCacheConfig: HealthSnapshot, TypeCount, YearlyCount, SnapshotDelta, TypeDelta, AnomalyRecord (derived local UI/index data)
|
|
|
96
|
- localConfig: OperationLog, DeviceProfile, MetricTimeoutProfile (local-only settings and operation metadata)
|
|
|
97
|
- Added `HealthArchiveStore` protocol for the single local archive store source of truth
|
|
Bogdan Timofte
authored
2 weeks ago
|
98
|
- Added `SQLiteHealthArchiveStore`: actor-isolated SQLite archive with WAL, per-sample upsert, disappearance marking, verification timestamps, semantic fingerprints, metadata JSON, and scoped JSON report export
|
|
|
99
|
- HealthKit anchored-query pages now archive samples/deletions before SwiftData snapshot/index rows are built
|
|
Bogdan Timofte
authored
2 weeks ago
|
100
|
- Schema migration recovery: removes legacy SwiftData stores and retries once on failure
|
|
Bogdan Timofte
authored
a month ago
|
101
|
|
|
|
102
|
### UI (Step 13)
|
|
|
103
|
|
|
|
104
|
✅ **SnapshotRow** — Shows:
|
|
|
105
|
- Chain indicators: "Chain start" / "DB reset / recovered device ID" / "Post-restore baseline" / "Observer-triggered snapshot"
|
|
|
106
|
- Anomaly warning badge (exclamationmark.triangle) if anomalyFlags non-empty
|
|
|
107
|
- Incomplete snapshot warning if quality != .complete
|
|
|
108
|
|
|
|
109
|
✅ **SnapshotTypeCountRow** — Shows:
|
|
|
110
|
- "Unsupported" for isUnsupported = true (read directly, no delta needed)
|
|
|
111
|
- "Unavailable" for count = -1
|
|
|
112
|
- Numeric count with warning color if quality != .complete
|
|
|
113
|
- Delta badge vs. baseline (green/amber)
|
|
|
114
|
|
|
|
115
|
✅ **DashboardView** — Anomaly summary section:
|
|
|
116
|
- Counts unresolved anomalies by severity (critical/warning)
|
|
|
117
|
- Shows only if unresolved anomalies exist
|
|
|
118
|
|
|
|
119
|
✅ **Full feature coverage**:
|
|
|
120
|
- Snapshot creation with observer triggers
|
|
|
121
|
- Chain visualization and deletion with integrity warnings
|
|
|
122
|
- Quality badges and anomaly indicators
|
|
|
123
|
- Timezone/registry change awareness
|
|
|
124
|
- Baseline comparison across multiple devices
|
|
|
125
|
|
|
|
126
|
## Build Status
|
|
|
127
|
|
|
|
128
|
```
|
|
|
129
|
✅ BUILD SUCCEEDED
|
|
|
130
|
Target: HealthProbe (iOS 26.4)
|
|
|
131
|
No compilation errors or warnings
|
|
|
132
|
App signs successfully
|
|
|
133
|
```
|
|
|
134
|
|
|
|
135
|
## Verification Checklist (32 items from plan)
|
|
|
136
|
|
|
|
137
|
These tests should be run to ensure all backend functionality is correct:
|
|
|
138
|
|
|
|
139
|
### Basic Snapshot & Chain (1-3)
|
|
|
140
|
- [ ] 1. Build succeeds with no errors
|
|
|
141
|
- [ ] 2. First snapshot: isChainStart=true, previousSnapshotID=nil, no delta created
|
|
|
142
|
- [ ] 3. Second snapshot: SnapshotDelta created with correct checksumBefore/After
|
|
|
143
|
|
|
|
144
|
### Quality & Anomalies (4-7)
|
|
|
145
|
- [ ] 4. Revoke permission → type quality=.unauthorized, snapshot=.partial, no anomalies
|
|
|
146
|
- [ ] 5. All permissions revoked → snapshot=.unauthorized, no anomalies
|
|
|
147
|
- [ ] 6. Timeout simulation (1ms) → count=-1, quality=.failed, "Unavailable" in UI
|
|
|
148
|
- [ ] 7. Post-authorize after full deny → first delta suppressed, snapshot marked post-restore
|
|
|
149
|
|
|
|
150
|
### Chain Operations (8-10)
|
|
|
151
|
- [ ] 8. 3 snapshots A→B→C, delete B → single merged delta A→C, C.previousSnapshotID==A.id
|
|
|
152
|
- [ ] 9. Hash stability → no changes between snapshots = identical hashes/checksums
|
|
|
153
|
- [ ] 10. Integrity strict mode → corrupted checksum = validation stops, no auto-repair
|
|
|
154
|
|
|
|
155
|
### Advanced Features (11-20)
|
|
|
156
|
- [ ] 11. DB reset with Keychain survival → same deviceID, isChainStart=true, recoveredDeviceID=true
|
|
Bogdan Timofte
authored
2 weeks ago
|
157
|
- [ ] 12. Local-only launch → app functions without iCloud/CloudKit entitlements
|
|
Bogdan Timofte
authored
a month ago
|
158
|
- [ ] 13. Observer debounce → 10 rapid callbacks = exactly 1 snapshot (triggerReason=observerCallback)
|
|
|
159
|
- [ ] 14. Unsupported type → TypeCount(count=-1, quality=.failed, isUnsupported=true), "Unsupported" UI
|
|
|
160
|
- [ ] 15. YearlyCount timezone → Calendar.current used, isApproximate=true if bucket > day
|
|
|
161
|
- [ ] 16. Delta merge with unavailable counts → merged countDelta=0, impaired reason preserved
|
|
Bogdan Timofte
authored
2 weeks ago
|
162
|
- [ ] 17. Missing local delta/typeDeltas → integrity validation surfaces the fault, never hides it as sync latency
|
|
Bogdan Timofte
authored
a month ago
|
163
|
- [ ] 18. First auth after full deny (quality gate) → no anomalies, current.isPostRestore=true, isPostRestoreInferred=true
|
|
|
164
|
- [ ] 19. Chain fork → validateChain() returns .corrupted(reason: "chain fork detected"), stops
|
|
|
165
|
- [ ] 20. disappeared→appeared merge with -1 source → merged countDelta=0, reason != .normal
|
|
|
166
|
|
|
|
167
|
### Reason Priority & Suppression (21-26)
|
|
|
168
|
- [ ] 21. TypeDelta reason priority → .unauthorized wins over .registryChanged simultaneously
|
|
|
169
|
- [ ] 22. Debounce + manual overlap → no observer snapshot if manual created during debounce
|
|
|
170
|
- [ ] 23. completionHandler unconditional → called via defer, never gated on scheduling success
|
|
|
171
|
- [ ] 24. isPostRestore forwarding → suppression forwarded past low-quality, consumed on next .complete
|
|
Bogdan Timofte
authored
2 weeks ago
|
172
|
- [ ] 25. Missing delta → validateChain() returns .missingDelta and stops
|
|
Bogdan Timofte
authored
a month ago
|
173
|
- [ ] 26. OperationLog verification → recovery re-insert if missing after save, log critical error
|
|
|
174
|
|
|
|
175
|
### Coherence & Edge Cases (27-32)
|
|
|
176
|
- [ ] 27. Per-type query concurrency → max 6 simultaneous HK queries (not 3N at N=20)
|
|
|
177
|
- [ ] 28. YearlyCount timezone drift → countDelta=0, yearlyCountNote set, no anomalies
|
|
|
178
|
- [ ] 29. isUnsupported on TypeCount → UI shows "Unsupported" without delta context
|
|
|
179
|
- [ ] 30. count/quality coherence assert → debug assert fires, release corrects to -1
|
|
|
180
|
- [ ] 31. snapshotChecksum filter → uses quality==.complete, not hash!="" (determinism)
|
|
|
181
|
- [ ] 32. AnomalyRecord.deltaID structural → every record has deltaID==delta.id (no external setter)
|
|
|
182
|
|
|
|
183
|
## Architectural Highlights
|
|
|
184
|
|
|
|
185
|
### Purity & Immutability
|
|
|
186
|
- **AnomalyDetector** is pure: no SwiftData mutations, explicit TypeCount maps, DetectionResult metadata
|
|
|
187
|
- **DeltaService** never carries old checksums during merge (recomputes from surrounding snapshots)
|
|
|
188
|
- **OperationLog** atomicity: log + destructive changes in single context.save()
|
|
|
189
|
|
|
|
190
|
### Quality Gates
|
|
|
191
|
- **Snapshot quality** aggregation prevents false positives:
|
|
|
192
|
- All detection requires both snapshots .complete
|
|
|
193
|
- Covers first authorization after full deny (quality gate alone is complete suppression)
|
|
|
194
|
- isPostRestore suppression forwarded past low-quality successors
|
|
|
195
|
|
|
|
196
|
### Chain Integrity
|
|
|
197
|
- **previousSnapshotID** is the sole source of chain truth (not localSequenceNumber)
|
|
|
198
|
- **Fork detection** prevents chain divergence (asserts no duplicate previousSnapshotID)
|
|
|
199
|
- **Checksum validation** ensures data wasn't corrupted between snapshots
|
|
|
200
|
|
|
Bogdan Timofte
authored
2 weeks ago
|
201
|
### Local Archive Direction
|
|
|
202
|
- CloudKit/iCloud sync is not a product goal
|
|
|
203
|
- SwiftData rows are derived UI/index data and must be rebuildable from the local archive store
|
|
|
204
|
- Missing deltas or type deltas are treated as local integrity faults, not remote sync latency
|
|
Bogdan Timofte
authored
a month ago
|
205
|
|
|
|
206
|
### Observability
|
|
|
207
|
- **Reason priority** makes anomaly suppression deterministic
|
|
|
208
|
- authorizationChanged > unsupported > registryChanged > unknown > normal
|
|
|
209
|
- Prevents .registryChanged from masking .authorizationChanged
|
|
|
210
|
- **YearlyCount timezone guard** prevents false loss attribution across DST
|
|
|
211
|
- **TypeDelta.yearlyCountNote** signals unreliable year-level attribution
|
|
|
212
|
|
|
|
213
|
## Known Limitations (MVP)
|
|
|
214
|
|
|
|
215
|
1. **Hash** covers only count + date range, not distribution (silentReplacement is best-effort)
|
|
|
216
|
2. **YearlyCount** precision requires daily bucket granularity (noted if isApproximate)
|
|
Bogdan Timofte
authored
2 weeks ago
|
217
|
3. **Archive query/report UI is still pending** (store exists, UI still mostly reads SwiftData cache)
|
|
Bogdan Timofte
authored
2 weeks ago
|
218
|
4. **No automatic cross-device reconstruction**; cross-device analysis is future macOS/report work
|
|
Bogdan Timofte
authored
a month ago
|
219
|
|
|
|
220
|
## Next Steps
|
|
|
221
|
|
|
|
222
|
### Immediate (Testing)
|
|
|
223
|
1. Run all 32 verification checks against real HealthKit data
|
|
|
224
|
2. Create unit tests for delta merge, reason priority, anomaly detection
|
|
|
225
|
3. Test observer callback debounce with real HKObserverQuery
|
|
Bogdan Timofte
authored
2 weeks ago
|
226
|
4. Add archive status/report UI backed by `HealthArchiveStore`
|
|
Bogdan Timofte
authored
a month ago
|
227
|
|
|
|
228
|
### Post-MVP
|
|
|
229
|
1. Integrate actual BGTask expiration guard for observer snapshots (capture partial results)
|
|
|
230
|
2. Add delta comparison view showing TypeDelta reason and suppression explanations
|
|
|
231
|
3. Implement OperationLog viewer in UI (audit trail dashboard)
|
|
|
232
|
4. Add historical trend analysis (divergence detection, anomaly patterns)
|
|
|
233
|
|
|
|
234
|
---
|
|
|
235
|
|
|
Bogdan Timofte
authored
2 weeks ago
|
236
|
**Built with:** SwiftUI, SwiftData, HealthKit, CryptoKit
|
|
Bogdan Timofte
authored
a month ago
|
237
|
**Minimum iOS:** 17.0
|
|
|
238
|
**Target iOS:** 26.4
|
|
|
239
|
**Swift Version:** 5.9+
|