Refocus storage on local archive · 695070d

Refocus storage on local archive
Browse files

bogdan committed 2 weeks ago

main

1 parent e112cd0

commit 695070d

Showing 17 changed files with 326 additions and 259 deletions

+21 -17

AGENTS.md

@@ -5,6 +5,8 @@
 HealthProbe is built by multiple AI models, each owning a distinct domain.  
 This document defines boundaries, interfaces, and handoff contracts.
 
+**Agentic reality:** The repo is developed largely via agents (Codex CLI, Claude, and dedicated model sessions). When updating product scope, update docs first, then implement behind flags, and add tests for the new behavior.
+
 ---
 
 ## Model Allocation
@@ -12,10 +14,11 @@ This document defines boundaries, interfaces, and handoff contracts.
 | Domain | Owner | Tools |
 |--------|-------|-------|
 | **UI / SwiftUI Views** | Claude Code | Xcode, SwiftUI, CLAUDE.md |
-| **Data Models (SwiftData)** | Dedicated model session | Xcode, Swift |
+| **Archive Store** | Dedicated model session | SQLite/local archive format, HealthKit metadata mapping |
+| **Data Models (SwiftData)** | Dedicated model session | Xcode, Swift; derived UI/cache/settings/log models only |
 | **HealthKit Integration** | Dedicated model session | Xcode, HealthKit docs |
 | **Anomaly Detection Algorithms** | Dedicated model session | Swift, statistical references |
-| **Sync Monitoring** | Dedicated model session | Xcode, CloudKit docs |
+| **Context Monitoring** | Dedicated model session | Xcode; logs Health/iCloud state as context only |
 | **Documentation** | Claude Code + dedicated session | Markdown |
 | **Tests** | Dedicated model session | XCTest, Swift Testing |
 
@@ -28,14 +31,16 @@ HealthProbe/
 ├── Views/           ← Claude Code (UI)
 ├── ViewModels/      ← Claude Code (UI)
 ├── Utilities/       ← Claude Code (shared helpers, mocks)
-├── Models/          ← Models agent (SwiftData schemas)
-├── Services/        ← Services agent (HealthKit, anomaly, sync)
+├── Models/          ← Models agent (SwiftData UI/cache schemas)
+├── Services/        ← Services agent (HealthKit, archive store, anomaly, context)
 └── Tests/           ← Tests agent
 ```
 
 **Rule:** Each agent writes only within its owned directories.  
 Cross-boundary changes require an explicit interface contract (protocol) first.
 
+**Documentation scope:** `HealthProbe/Doc/` is shared. Keep it consistent with shipped behavior, and add a dated entry when objectives change.
+
 ---
 
 ## Interface Contracts
@@ -77,15 +82,15 @@ protocol AuditTrailProtocol {
 }
 ```
 
-### SyncMonitorProtocol
+### ContextMonitorProtocol
 
 ```swift
 /// Owned by: Services agent
-/// Consumed by: UI (SyncViewModel)
-protocol SyncMonitorProtocol {
+/// Consumed by: UI (ContextViewModel)
+protocol ContextMonitorProtocol {
     var iCloudEnabled: Bool { get }
-    var lastSyncDate: Date? { get }
-    var stateChanges: [SyncStateChange] { get }
+    var lastObservedAt: Date? { get }
+    var stateChanges: [ContextStateChange] { get }
 }
 ```
 
@@ -122,14 +127,11 @@ final class TypeDistributionBin {
 // Import uses a global anchored query per data type so follow-up snapshots fetch only
 // HealthKit deltas instead of scanning calendar blocks with fixed per-query latency.
 
-// Interface updated 2026-05-14 — see AGENTS.md
-// TypeCount.recordArchiveData is stored with SwiftData external storage.
-// The archive remains a complete per-record forensic snapshot, but large blobs should
-// not be kept inline with the TypeCount row because high-volume HealthKit stores can
-// exceed device memory during import or detail inspection.
-// Initial high-volume imports stream records directly into a compact binary archive
-// instead of building a full in-memory dictionary/array; plist archives remain readable
-// for backwards compatibility.
+// Interface updated 2026-05-18 — see AGENTS.md
+// SwiftData is not the forensic source of truth. TypeCount and related rows store
+// precomputed UI/index data only. Complete HealthKit samples and metadata belong
+// in the local archive store, in one schema that can preserve relationships across
+// data types, sources, devices, workouts, and metadata.
 
 // Interface updated 2026-05-17 — see AGENTS.md
 // Models/TypeCount.detailCacheData stores precomputed detail data for the current
@@ -230,6 +232,8 @@ The following modules involve non-trivial logic and should be reviewed carefully
 - No location data or patterns enabling re-identification
 - Synthetic data only in tests and previews
 
+**Clarification:** “No raw health values” applies to this repository’s contents. The app may optionally store a user's raw HealthKit samples *locally on-device* for forensic backup purposes, but such samples must never appear in source control, logs, or docs.
+
 ---
 
 ## Communication Between Agents


+13 -14

CLAUDE.md

View

@@ -6,7 +6,7 @@
 It detects anomalies: data loss, historical insertions, duplicates, divergence trends.  
 Full specification: `HealthProbe/Doc/HealthProbe – Complete Specification & Motivations.md`
 
-**Current state:** Xcode scaffold only. `ContentView.swift` and `Item.swift` are default templates — replace them entirely.
+**Current state:** SwiftUI + SwiftData app is active. Product direction changed on 2026-05-18: HealthProbe is a local audit/capture agent. Do not add HealthProbe CloudKit/iCloud sync.
 
 ---
 
@@ -21,7 +21,7 @@ Claude Code is responsible for:
 - **Accessibility** (VoiceOver, Dynamic Type)
 
 Claude Code does NOT own:
-- `Services/` — HealthKit queries, anomaly detection, sync monitoring (see AGENTS.md)
+- `Services/` — HealthKit queries, anomaly detection, archive store, context monitoring (see AGENTS.md)
 - `Models/` — SwiftData models (see AGENTS.md)
 - Entitlements, Info.plist, project configuration
 
@@ -36,7 +36,7 @@ App (TabView)
 ├── Tab 1: Dashboard          → DashboardView
 ├── Tab 2: Anomalies          → AnomalyListView → AnomalyDetailView
 ├── Tab 3: Audit Trail        → AuditTrailView
-├── Tab 4: Sync Status        → SyncStatusView
+├── Tab 4: Archive Status     → ArchiveStatusView
 └── Tab 5: Settings           → SettingsView
 ```
 
@@ -67,18 +67,17 @@ App (TabView)
 - Search/filter by event type
 - Export button → JSON
 
-### SyncStatusView
-- Current iCloud sync state (chip: enabled / local-only)
-- Last sync timestamp
-- List of `SyncStateChange` events (most recent first)
-- Visual indicator when sync is stalled (no event in > 48h)
+### ArchiveStatusView
+- Current local archive health
+- Last archive verification timestamp
+- Selected data types covered by forensic capture
+- Recent Health/iCloud context events (for correlation only; no HealthProbe sync)
 
 ### SettingsView
 - Check frequency: 2h / 6h / 12h / 24h (Picker)
 - Sample types to monitor (MultiSelect toggle list)
 - Alert thresholds (severity level for push notifications)
-- CloudKit sync toggle (off by default, with privacy disclaimer)
-- Export all data (JSON)
+- Point export/report actions for selected findings
 - Delete all audit data (destructive, confirm alert)
 
 ---
@@ -107,7 +106,7 @@ App (TabView)
 - `exclamationmark.triangle.fill` — warning
 - `xmark.shield.fill` — critical
 - `clock.arrow.circlepath` — audit trail
-- `antenna.radiowaves.left.and.right` — sync
+- `externaldrive.fill.badge.checkmark` — archive status
 - `waveform.path.ecg` — health data
 - `doc.text.magnifyingglass` — anomaly detail
 
@@ -202,14 +201,14 @@ HealthProbe/
 │   │   └── AnomalyDetailView.swift
 │   ├── AuditTrail/
 │   │   └── AuditTrailView.swift
-│   ├── Sync/
-│   │   └── SyncStatusView.swift
+│   ├── Archive/
+│   │   └── ArchiveStatusView.swift
 │   └── Settings/
 │       └── SettingsView.swift
 ├── ViewModels/
 │   ├── DashboardViewModel.swift
 │   ├── AnomalyListViewModel.swift
-│   └── SyncViewModel.swift
+│   └── ArchiveStatusViewModel.swift
 ├── Models/           ← NOT owned by Claude Code
 ├── Services/         ← NOT owned by Claude Code
 └── Utilities/


+0 -3

HealthProbe.xcodeproj/project.pbxproj

View

@@ -99,9 +99,6 @@
 							com.apple.HealthKit = {
 								enabled = 1;
 							};
-							com.apple.iCloud = {
-								enabled = 1;
-							};
 						};
 					};
 				};


+9 -6

HealthProbe/Doc/Forensics & Limitations.md

View

@@ -15,6 +15,7 @@
 | **Timing precision** | Anchored queries may batch multiple changes, lose granular timestamps | Store both sample timestamp AND observation timestamp |
 | **Private HealthKit types** | Some data types not accessible to third-party apps | Accept data available only to Health.app |
 | **Cross-device sync delays** | Watch-to-phone-to-cloud can take 24+ hours | Extend observation window, don't flag immediate after device sync |
+| **Consolidation / downsampling** | Apple Health/iCloud can rewrite high-frequency historical samples in-place (count decreases; values/intervals change) | Store fingerprints + optionally archive raw samples for selected types (local-only forensic backup mode) |
 
 ### 1.2 iOS Background Mode Limitations
 
@@ -42,6 +43,7 @@
 - Cannot reconstruct data that was never observed
 - Snapshot from 6 months ago may have samples no longer in HealthKit
 - Gap detection assumes continuous observation (may be false positive if app uninstalled then reinstalled)
+- If Apple consolidates history, **counts alone can be misleading** (a month can “lose” samples but keep the same totals via aggregation); value-level forensics are required for proof
 
 ---
 
@@ -53,7 +55,7 @@
 |------|--------|-----------|
 | **Raw health data exfiltration** | CRITICAL: user's personal health history exposed | ✅ Local-only storage, never sends raw samples |
 | **Device fingerprinting** | HIGH: tracking user across services | ✅ Salted hash of device ID, stored locally only |
-| **Timing attacks** (inferring behavior) | MEDIUM: sync patterns reveal habits | ✅ CloudKit digest is aggregated & anonymized |
+| **Timing attacks** (inferring behavior) | MEDIUM: archive/check patterns reveal habits | ✅ No automatic cloud sync; reports are explicit local exports |
 | **App crashes leaking data** | LOW: crash logs may contain HealthKit info | ✅ All logging is aggregated (counts, not values) |
 
 ### 2.2 Data Integrity Risks (Mitigation: Good ✅)
@@ -62,6 +64,7 @@
 |------|--------|-----------|
 | **Snapshot corruption** | HIGH: audit trail becomes unreliable | ✅ Use MD5 checksum of snapshots, detect corruption |
 | **Lost audit trail on uninstall** | HIGH: forensic data disappears | ⚠️ PARTIAL: Encourage export before uninstall; future: iCloud backup option |
+| **Silent rewrites (no deletion events)** | HIGH: history can change without HKDeletedObject evidence | ✅ Detect via fingerprint diffs; **Forensic Backup Mode** can preserve per-sample evidence locally for selected types |
 | **Clock skew** (device time wrong) | MEDIUM: timestamps inaccurate, anomaly detection confused | ⚠️ Log both device time + time since boot, detect skew |
 | **Concurrent modification** (app + Health.app) | LOW: race conditions during query | ✅ Anchored queries are atomic |
 
@@ -71,7 +74,7 @@
 |------|--------|-----------|
 | **Malicious apps accessing HealthKit** | MEDIUM: third-party apps can read our data | ✅ iOS sandboxing; ask user for HealthKit permission per app |
 | **Jailbroken device** | CRITICAL: all bets off | ⚠️ Not defendable; document assumption: standard iOS device |
-| **iCloud account compromise** | HIGH: CloudKit digest could be leaked | ✅ OPTIONAL: Users can disable CloudKit sync in settings |
+| **iCloud account compromise** | MEDIUM: Apple Health/iCloud state may still affect source data | ✅ HealthProbe archive remains local; no HealthProbe CloudKit sync |
 | **Local device theft** | MEDIUM: thief can see audit trail | ✅ Data encrypted by iOS, requires device unlock |
 
 ---
@@ -328,8 +331,8 @@ Post-incident:
 
 ```
 Setup:
-  1. Enable optional CloudKit sync (with privacy settings)
-  2. Opt into community pattern analysis
+  1. Export anonymized anomaly summaries manually
+  2. Keep raw archive local
   
 Analysis:
   1. Correlate own data loss with iOS release dates
@@ -386,9 +389,9 @@ Future: Alert before data is lost
 | Issue | Cause | Fix |
 |-------|-------|-----|
 | **No anomalies detected for weeks** | Background fetch disabled | Settings → HealthProbe → Background Refresh |
-| **Snapshots not being saved** | Insufficient storage | Free up space; SwiftData limited by device storage |
+| **Snapshots not being saved** | Insufficient storage or archive write failure | Free up space; verify local archive health and SwiftData cache rebuild |
 | **Sync state not updating** | iCloud token check failing | Sign out/in to iCloud; restart device |
-| **Old audit trail entries missing** | SwiftData old entries pruned | Expected after > 6 months; export before uninstall |
+| **Old audit trail entries missing** | SwiftData cache/log retention policy or migration issue | Rebuild derived views from archive where possible; export reports before uninstall |
 
 ---
 


+14 -5

HealthProbe/Doc/HealthProbe iOS – Specification (MVP).md

View

@@ -8,7 +8,7 @@ It detects anomalies such as:
 - duplicate records
 - divergence trends over time
 
-The application operates as a **local audit agent** and optionally synchronizes **aggregated snapshots (digests)** via CloudKit.
+The application operates as a **local audit and capture agent**. It does not sync HealthProbe data via CloudKit/iCloud; HealthKit databases can evolve differently per device, so the MVP keeps each device archive local and explicit.
 
 ⚠️ This document describes ONLY the iOS application (MVP phase).  
 A future macOS application will act as a visualization/analysis layer.
@@ -27,11 +27,12 @@ A future macOS application will act as a visualization/analysis layer.
 3. **Incremental observation**
    - Use anchored queries to track changes
 
-4. **Minimal data exfiltration**
-   - Only aggregated digests are synced (no raw Health data)
+4. **No app cloud sync**
+   - HealthProbe does not sync raw samples, digests, or reports through CloudKit/iCloud
 
-5. **Forward compatibility**
-   - CloudKit schema must be shared with a future macOS application
+5. **Robust local archive**
+   - Store captured HealthKit data in one local archive store, not per-data-type silos
+   - SwiftData is used for derived UI data, settings, logs, and history only
 
 ---
 
@@ -48,6 +49,14 @@ Track:
 - High Heart Rate Events
 - Other relevant samples (extensible)
 
+Persist:
+- sample values and units
+- source and source revision metadata
+- device metadata exposed by HealthKit
+- HealthKit metadata dictionaries
+- first-seen / last-seen / last-verified timestamps
+- fingerprints for matching against Apple Health XML exports and backup database extracts
+
 ---
 
 ### 2. Anomaly Detection


+103 -100

HealthProbe/Doc/HealthProbe – Complete Specification & Motivations.md

View

@@ -1,8 +1,8 @@
 # HealthProbe – Complete Specification & Motivations
 
-**Version:** 1.0  
+**Version:** 1.2  
 **Status:** MVP (iOS monitoring agent)  
-**Last Updated:** 2026-05-01  
+**Last Updated:** 2026-05-18  
 
 ---
 
@@ -12,7 +12,7 @@ HealthProbe is an **audit and integrity monitoring tool for Apple HealthKit**, d
 
 **Core Problem:** Apple Health data loss events (confirmed Sept 2025 incident, ongoing sporadic reports) lack detective mechanisms. Users do not know when data has been lost, corrupted, or silently modified.
 
-**Solution:** HealthProbe monitors HealthKit in real-time and maintains a tamper-proof audit trail, enabling post-incident forensic analysis and pattern detection.
+**Solution:** HealthProbe incrementally captures HealthKit data into a robust local archive and maintains an audit trail for post-incident forensic analysis. SwiftData is used for UI assistance, settings, logs, history, and precomputed values; it is not the source of truth. The local archive exists because Apple Health/iCloud can **rewrite / downsample / consolidate** historical samples in-place (not only add/delete), which cannot be proven by counts alone.
 
 ---
 
@@ -111,6 +111,19 @@ HealthProbe is an **audit and integrity monitoring tool for Apple HealthKit**, d
 
 ---
 
+#### E. Consolidation / Downsampling (Historical Rewrites)
+**Pattern:** For high-frequency data types, Apple Health/iCloud may rewrite historical samples such that:
+- total sample `count` decreases for older months
+- timestamps in a later snapshot/export become a **subset** of an earlier snapshot/export (thinning)
+- some samples become **interval-based** (`endDate > startDate`) and/or values become fractional
+- for cumulative quantities, `value_sum` can remain stable while per-sample `value_max` increases (consolidation)
+
+**Why it matters:** A “record counter” cannot distinguish discard vs consolidation. HealthProbe must support value-level forensics and (optionally) preserve complete evidence locally.
+
+**Detection method:** Snapshot comparison of fingerprints *plus* optional per-sample archives for selected data types.
+
+---
+
 ### 2.3 Why This Matters
 
 | Concern | Impact | HealthProbe Role |
@@ -130,9 +143,10 @@ HealthProbe is an **audit and integrity monitoring tool for Apple HealthKit**, d
 1. **Read-only operations** (never modify HealthKit data)
 2. **Local-first** (full functionality without network)
 3. **Incremental queries** (efficient, avoid repeating work)
-4. **Minimal exfiltration** (only digests, never raw samples)
+4. **Single archive store** (do not split the forensic store per data type; cross-type relationships and shared metadata matter)
 5. **Auditability** (every observation logged, timestamped, reproducible)
-6. **Privacy by default** (all aggregated, no PII stored)
+6. **Privacy by default** (no HealthProbe cloud sync; local storage remains under user control)
+7. **Forensic capture** (selected data types are archived locally as complete per-sample records with metadata to preserve evidence against silent rewrites)
 
 ### 3.2 Threading Model
 
@@ -157,49 +171,38 @@ HealthProbe is an **audit and integrity monitoring tool for Apple HealthKit**, d
                ├─ Write detected anomalies
                │
 ┌──────────────▼──────────────────────────┐
-│  Local Storage (SwiftData)              │
-│  - Audit trail (immutable append-only)  │
-│  - Snapshots (for forensics)            │
-│  - Anomaly records                      │
-│  - Metadata (versions, migrations)      │
+│  Local Archive Store                    │
+│  - Canonical HealthKit samples          │
+│  - Sources, devices, metadata           │
+│  - Cross-type relationships             │
+│  - Fingerprints and verification hashes │
+└──────────────┬──────────────────────────┘
+               │
+┌──────────────▼──────────────────────────┐
+│  SwiftData UI Store                     │
+│  - Precomputed counts/statistics        │
+│  - Visualization state and settings     │
+│  - Logs, history, report indexes        │
 └─────────────────────────────────────────┘
 ```
 
-### 3.3 Data Model (SwiftData)
+### 3.3 Storage Model
 
-```swift
-// Immutable audit log entry
-@Model
-final class AuditEntry {
-    var timestamp: Date
-    var eventType: String  // "snapshot", "deletion", "insertion", etc.
-    var samples: [HealthSample]
-    var anomalies: [AnomalyReport]
-    var deviceInfo: DeviceMetadata
-}
+**Local Archive Store (source of truth):**
+- one robust local database for all archived samples, not one archive per data type
+- normalized entities for samples, workouts, sources, source revisions, devices, metadata, relationships, and observations
+- multiple fingerprints per sample: HealthKit UUID, strict fingerprint, semantic fingerprint, and fuzzy matching keys for export/backup reconciliation
+- append-only observation history (`firstSeen`, `lastSeen`, `lastVerified`, disappearance evidence)
+- snapshot-level and table-level hashes for integrity checks
 
-// Snapshot for forensic comparison
-@Model
-final class HealthSnapshot {
-    var capturedAt: Date
-    var sampleType: String  // "HKWorkout", "HKQuantity:HeartRate", etc.
-    var sourceDevice: String
-    var recordCount: Int
-    var checksum: String  // MD5 of aggregated data
-    var digest: [String: Int]  // { "source": count, ... }
-}
+**SwiftData UI Store (derived/cache layer):**
+- settings and selected data types
+- import job state and progress
+- precomputed counts, temporal bins, display ranges, and summary statistics
+- audit log entries and report indexes
+- anomaly summaries and links into the archive store
 
-// Detected anomaly
-@Model
-final class AnomalyReport {
-    var type: String  // "historical_insertion", "deletion", "duplicate", "divergence"
-    var severity: String  // "info", "warning", "critical"
-    var detectedAt: Date
-    var description: String
-    var evidence: [String: Any]  // forensic data
-    var requiresAttention: Bool
-}
-```
+SwiftData rows must be rebuildable from the local archive store. If the two disagree, the archive store wins.
 
 ---
 
@@ -290,9 +293,13 @@ For each metric over time:
 
 ---
 
-## 5. Sync Monitoring (Separate Thread)
+## 5. Sync Context Logging
 
-### 5.1 Sync State Tracking
+HealthProbe does **not** sync its own archive through iCloud or CloudKit. Observed HealthKit databases can diverge between devices, so cross-device HealthProbe sync would increase complexity without providing a reliable forensic source of truth.
+
+Health/iCloud state is still useful as **context** for anomalies.
+
+### 5.1 Context Tracking
 
 **Observe HealthKit permission & sync state:**
 ```swift
@@ -302,23 +309,23 @@ HKHealthStore().requestAuthorization(...)
 // Monitor iCloud state
 FileManager.default.ubiquityIdentityToken
 // → Detects iCloud sign-in/sign-out
-// → Triggers re-baseline after sync state changes
+// → Logs context for later correlation
 ```
 
 **Capture lifecycle events:**
-- iCloud sign-in detected → re-snapshot everything
+- iCloud sign-in detected → log context and schedule a local archive verification pass
 - iCloud sign-out detected → note local-only mode
 - Device backup initiated → pre-backup snapshot
 - App backgrounded/foregrounded → check for sync activity
 
-### 5.2 Sync Documentation
+### 5.2 Context Documentation
 
 **Audit trail entries:**
 ```
 [2026-05-01 14:23:15] SYNC_STATE_CHANGE: iCloud enabled
   - Previous: local-only
-  - Action: re-snapshot all data
-  - Result: 5,432 samples captured
+  - Action: archive verification scheduled
+  - Result: no HealthProbe cloud sync performed
 
 [2026-05-01 14:24:02] SYNC_COMPLETE: iCloud data merged
   - Samples added: 87
@@ -334,52 +341,46 @@ FileManager.default.ubiquityIdentityToken
   - Severity: MEDIUM
 ```
 
-### 5.3 Background Sync Monitoring
+### 5.3 Background Monitoring
 
 **iOS Background Modes enabled:**
-- `background-fetch` — Periodic sync checks
-- `remote-notification` → For sync completion hints (optional)
+- `background-fetch` — periodic archive and context checks
+- `remote-notification` → not required for HealthProbe archive sync
 
-**Sync check frequency:**
+**Check frequency:**
 - Min: 2 hours
 - Max: 24 hours
 - Adapts based on anomaly detection frequency
 
 ---
 
-## 6. Data Export & Forensics
+## 6. Local Archive, Reports & Forensics
 
-### 6.1 Export Formats
+### 6.1 Local Archive Store
 
-**Option 1: Local Backup (Swift)**
-```
-Export structure:
-├── metadata.json          (device, iOS version, app version)
-├── snapshots/             (timestamped HealthKit snapshots)
-├── audit_trail.jsonl      (immutable log, one entry per line)
-├── anomalies.json         (detected anomalies)
-└── digests.json           (aggregated metrics for trending)
-```
+The main backup artifact is the on-device archive store. It is populated incrementally from HealthKit and is not dependent on Apple Health ZIP exports or full encrypted iPhone backups.
 
-**Option 2: CloudKit Digest (Optional)**
-```
-Send ONLY:
-  - Aggregated counts (not samples)
-  - Anomaly types & timestamps (not raw health data)
-  - Device fingerprint (not identifiable)
-  
-Schema:
-  zone: "HealthProbe_Public"
-  record: "MonthlyDigest"
-  fields:
-    - device_id (salted hash)
-    - month: YYYY-MM
-    - anomaly_counts: { type: count }
-    - data_loss_estimate: percent
-    - sample_types_tracked: [...]
-```
+The archive must preserve as much HealthKit information as the API exposes:
+- sample UUID, type, start/end date, value, unit, and metadata
+- source, source revision, bundle identifier, product type, version/build if available
+- device fields exposed by `HKDevice`
+- relationships between workouts, samples, events, and other linked records where available
+- first-seen / last-seen / last-verified observations
+- fingerprints suitable for matching against Apple Health XML exports and extracted backup databases
+
+The archive is selected by data type for performance and privacy, but it is stored in **one schema** so later analysis can follow relationships between types.
+
+### 6.2 Reports and Point Exports
 
-### 6.2 Forensic Query Examples
+HealthProbe does not need to optimize for routine complete exports. The local archive is the backup.
+
+Export is scoped to what the user is inspecting:
+- anomaly reports
+- tables of records shown in UI (e.g., “these 1,000 HK records disappeared”)
+- point-in-time manifests and hashes
+- selected record sets needed for external analysis
+
+### 6.3 Forensic Query Examples
 
 **"Has my step data been compromised?"**
 ```
@@ -391,8 +392,8 @@ Schema:
 
 **"Did iCloud sync break my data?"**
 ```
-1. Correlate anomalies with sync state changes
-2. Show timeline: before sync, during sync, after
+1. Correlate anomalies with observed Health/iCloud state changes
+2. Show timeline: before state change, during reconciliation, after
 3. Calculate: samples lost, duplicates introduced
 ```
 
@@ -420,27 +421,27 @@ Schema:
 - **Anomalies** — sortable list by date/severity
 - **Snapshots** — historical timeline of known-good snapshots
 - **Audit Trail** — complete immutable log
-- **Sync Status** — current iCloud state & last sync time
+- **Archive Status** — current local archive health, last verification, selected data types
 
 **Settings:**
 - Check frequency
 - Sample types to track
 - Alert thresholds
-- CloudKit sync opt-in
+- Local archive retention and report export options
 
 ### 7.2 Alerts
 
 **Push Notifications (opt-in):**
 - 🚨 "Critical data loss detected" (> 10% samples missing)
 - ⚠️ "Unexpected historical data inserted" (> 100 samples)
-- ℹ️ "Sync completed, 2 duplicates found"
+- ℹ️ "Archive check completed, 2 duplicates found"
 
 ---
 
 ## 8. Future Enhancements (Beyond MVP)
 
 ### 8.1 macOS Companion (Visualization Layer)
-- Aggregate data from multiple iOS devices
+- Open and analyze exported HealthProbe reports or archive copies
 - Long-term trend visualization (6-12 month history)
 - Cross-device anomaly correlation
 - Export to reproducible bug reports
@@ -466,17 +467,17 @@ Schema:
 
 ### 9.2 Permissions Required
 - `HealthKit` — read-only access to specified types
-- `iCloud CloudKit` — optional, for digest sync only
 - `Background Modes` — "Background Fetch"
 
 ### 9.3 Data Storage
-- **Local:** SwiftData (on-device, encrypted by iOS)
-- **Optional Cloud:** CloudKit (user-controlled, aggregated only)
+- **Local Archive Store:** canonical HealthKit sample archive (source of truth)
+- **SwiftData:** derived UI/cache/settings/log/history store
+- **No CloudKit sync:** HealthProbe data remains local unless the user exports a report or selected record table
 
 ### 9.4 Performance
 - Query time: < 5 seconds (anchored queries)
-- Snapshot size: ≈ 5-10 KB per type per snapshot
-- Storage: ≈ 100 MB per year of monitoring (aggregated)
+- Snapshot/index size: ≈ 5-10 KB per type per snapshot in SwiftData
+- Archive storage: depends on selected high-frequency data types; report per-type storage costs in settings
 
 ---
 
@@ -495,11 +496,14 @@ Schema:
 - ✅ Device model & iOS version (for context)
 - ✅ Anomaly types & severity
 
-### 10.3 Optional CloudKit Sync
-- Only aggregated digests (counts, not data)
-- Device identifier is salted & hashed
-- Deleted immediately when uninstalled
-- Can be disabled in settings
+**Local archive:**
+- ✅ Per-sample archive for user-selected types, stored on-device and exportable by user
+- ✅ Metadata needed for recognition in Apple Health XML exports, backup database extracts, and future datasets
+
+### 10.3 Cloud Policy
+- No HealthProbe CloudKit/iCloud sync
+- No automatic upload of raw samples, digests, reports, or device fingerprints
+- User-triggered exports are explicit, scoped, and local-file based
 
 ---
 
@@ -511,7 +515,7 @@ Schema:
 | **Forensic completeness** | % of anomalies with sufficient evidence | > 95% |
 | **False positives** | Alerts user shouldn't worry about | < 5% of total |
 | **Privacy** | % of users comfortable with data practices | > 90% |
-| **Performance** | Background sync battery impact | < 2% drain/day |
+| **Performance** | Background capture battery impact | < 2% drain/day |
 | **Adoption** | Users can reproduce bugs with HealthProbe data | High relevance in Apple feedback |
 
 ---
@@ -521,7 +525,6 @@ Schema:
 - [DearApple Issue #001](https://github.com/overbog/dear-apple/issues/0001-apple-health-mass-data-loss.md) — Sept 2025 mass data loss
 - [Apple HealthKit Documentation](https://developer.apple.com/documentation/healthkit/)
 - [HKAnchoredObjectQuery](https://developer.apple.com/documentation/healthkit/hkanchoredrobjectquery) — Efficient incremental queries
-- [CloudKit Best Practices](https://developer.apple.com/documentation/cloudkit/)
 
 ---
 


+70 -29

HealthProbe/Doc/Implementation Guide.md

View

@@ -14,7 +14,7 @@ The following rules apply to **all code, logs, examples, tests, and documentatio
 - **No personal data** — no names, email addresses, phone numbers, or dates of birth
 - **No device identifiers** — no UDIDs, serial numbers, advertising IDs, or device names
 - **No account identifiers** — no Apple IDs, iCloud account info, or CloudKit record IDs
-- **No raw health values** — no actual health records, measurements, or workout data
+- **No raw health values in the repository** — do not include real health records, measurements, or workouts in code, tests, logs, examples, or documentation. The app may optionally store a user's raw samples **locally on-device** for forensic backup, but nothing real belongs in this repo.
 - **No location data** — no GPS coordinates or location history
 - **No recognizable patterns** — no logs or exports where combining fields could identify a person or device
 
@@ -123,7 +123,22 @@ class HealthKitObserver {
 
 ---
 
-## 2. Data Model Implementation (SwiftData)
+## 2. Storage Implementation
+
+HealthProbe uses two storage layers:
+
+1. **Local Archive Store (source of truth)**
+   - Stores canonical HealthKit samples and all metadata exposed by the API
+   - Uses one schema for all selected data types, so workouts, samples, sources, devices, and metadata can be related later
+   - Maintains `firstSeen`, `lastSeen`, `lastVerified`, strict/semantic/fuzzy fingerprints, and integrity hashes
+   - Should be implemented with an explicit local database/archive format (not SwiftData model graphs for millions of samples)
+
+2. **SwiftData UI Store (derived/cache layer)**
+   - Stores settings, logs, import/check history, anomaly summaries, and precomputed values used by charts
+   - Can be rebuilt from the archive store
+   - Must not be treated as the only forensic copy
+
+### 2.1 SwiftData UI Models
 
 ```swift
 import SwiftData
@@ -225,7 +240,7 @@ final class DetectedAnomaly {
 }
 
 @Model
-final class SyncStateChange {
+final class ContextStateChange {
     @Attribute(.unique) var id: String = UUID().uuidString
     var timestamp: Date
     var previousState: String  // "local_only", "icloud_enabled", "icloud_sync_active"
@@ -247,7 +262,7 @@ func createModelContainer() throws -> ModelContainer {
         HealthSnapshot.self,
         AuditTrailEntry.self,
         DetectedAnomaly.self,
-        SyncStateChange.self,
+        ContextStateChange.self,
     ])
     
     let modelConfiguration = ModelConfiguration(
@@ -260,6 +275,29 @@ func createModelContainer() throws -> ModelContainer {
 }
 ```
 
+### 2.2 Local Archive Store Contract
+
+The archive store should expose a small service interface rather than leaking SQL/archive details into UI code:
+
+```swift
+protocol HealthArchiveStore {
+    func upsertSamples(_ samples: [HKSample], observedAt: Date) async throws
+    func markVerification(sampleType: HKSampleType, verifiedAt: Date) async throws
+    func recordDisappearance(fingerprint: String, observedMissingAt: Date) async throws
+    func records(for reportID: String) async throws -> [ArchivedHealthRecord]
+    func exportReport(_ reportID: String) async throws -> URL
+}
+```
+
+Archive rows should preserve:
+- HealthKit UUID where exposed
+- type identifier, start/end date, value, unit
+- source, source revision, bundle identifier, version/build/product type where available
+- `HKDevice` fields exposed by HealthKit
+- full metadata dictionary as structured data
+- relationship keys for workouts, events, and related samples where available
+- fingerprints for matching records across HealthProbe, Apple Health XML exports, and backup database extracts
+
 ---
 
 ## 3. Anomaly Detection Implementation
@@ -426,39 +464,41 @@ class AnomalyDetector {
 
 ---
 
-## 4. Sync Monitoring (Background Thread)
+## 4. Context Monitoring (Background Thread)
+
+HealthProbe does not sync its own database through iCloud/CloudKit. This service only logs Health/iCloud state as context for later forensic correlation.
 
 ```swift
-class SyncMonitor {
+class ContextMonitor {
     private let modelContext: ModelContext
     private let queue = DispatchQueue(label: "com.healthprobe.sync-monitor", qos: .background)
     
-    private var previousSyncState: String = "unknown"
+    private var previousHealthCloudState: String = "unknown"
     
     func startMonitoring() {
         queue.async {
-            self.monitorSyncState()
+            self.monitorContext()
         }
     }
     
-    private func monitorSyncState() {
+    private func monitorContext() {
         // Check iCloud state
         let iCloudToken = FileManager.default.ubiquityIdentityToken
         let currentState = iCloudToken != nil ? "icloud_enabled" : "local_only"
         
-        if currentState != previousSyncState {
-            logSyncStateChange(from: previousSyncState, to: currentState)
-            previousSyncState = currentState
+        if currentState != previousHealthCloudState {
+            logContextChange(from: previousHealthCloudState, to: currentState)
+            previousHealthCloudState = currentState
             
-            // Trigger re-snapshot on state change
+            // Schedule archive verification on state change
             DispatchQueue.main.async {
-                NotificationCenter.default.post(name: NSNotification.Name("SyncStateChanged"), object: nil)
+                NotificationCenter.default.post(name: NSNotification.Name("HealthContextChanged"), object: nil)
             }
         }
     }
     
-    private func logSyncStateChange(from: String, to: String) {
-        let change = SyncStateChange(
+    private func logContextChange(from: String, to: String) {
+        let change = ContextStateChange(
             timestamp: Date(),
             previousState: from,
             newState: to,
@@ -471,14 +511,14 @@ class SyncMonitor {
             
             let auditEntry = AuditTrailEntry(
                 timestamp: Date(),
-                eventType: "sync_state_change",
-                message: "Sync state: \(from) → \(to)",
+                eventType: "health_context_change",
+                message: "Health cloud context: \(from) → \(to)",
                 context: ["previous": from, "current": to]
             )
             modelContext.insert(auditEntry)
             try modelContext.save()
         } catch {
-            print("Error logging sync state change: \(error)")
+            print("Error logging context change: \(error)")
         }
     }
 }
@@ -492,14 +532,14 @@ class SyncMonitor {
 @main
 struct HealthProbeApp: App {
     @StateObject private var healthKitManager = HealthKitManager.shared
-    @StateObject private var syncMonitor: SyncMonitor
+    @StateObject private var contextMonitor: ContextMonitor
     let modelContainer: ModelContainer
     
     init() {
         do {
             modelContainer = try createModelContainer()
             let context = ModelContext(modelContainer)
-            _syncMonitor = StateObject(wrappedValue: SyncMonitor(modelContext: context))
+            _contextMonitor = StateObject(wrappedValue: ContextMonitor(modelContext: context))
         } catch {
             fatalError("Could not initialize model container: \(error)")
         }
@@ -513,9 +553,8 @@ struct HealthProbeApp: App {
                     // Request HealthKit permissions
                     healthKitManager.requestAuthorization { success, error in
                         if success {
-                            // Start monitoring
-                            syncMonitor.startMonitoring()
-                            // Initial snapshot
+                            // Start context monitoring and archive capture
+                            contextMonitor.startMonitoring()
                             captureInitialSnapshot()
                         }
                     }
@@ -565,8 +604,9 @@ class AnomalyDetectorTests: XCTestCase {
 
 ### Integration Tests
 - ✅ HealthKit query performance (anchor efficiency)
-- ✅ SwiftData persistence and recovery
-- ✅ Background sync monitoring accuracy
+- ✅ Local archive persistence and recovery
+- ✅ SwiftData cache rebuild from archive
+- ✅ Background context monitoring accuracy
 - ✅ Anomaly detection on real HealthKit data
 
 ---
@@ -577,8 +617,9 @@ class AnomalyDetectorTests: XCTestCase {
 |-----------|--------|-------|
 | Anchored query | < 5 sec | Background, user perceives delay > 2s |
 | Anomaly detection | < 2 sec | Should not block UI |
-| Snapshot creation | < 1 sec | Can run on main thread |
-| Background sync | < 30 sec | iOS allows 30 min for background fetch |
+| SwiftData cache update | < 1 sec | Can run on main thread only after archive work completes |
+| Archive write | Background | Stream large imports; never build full high-frequency datasets in memory |
+| Background check | < 30 sec | iOS allows 30 min for background fetch |
 
 ---
 
@@ -586,8 +627,8 @@ class AnomalyDetectorTests: XCTestCase {
 
 - [ ] HealthKit read permissions declared in Info.plist
 - [ ] Background Modes enabled ("Background Fetch")
-- [ ] Push notification entitlement (if using remote notifications)
 - [ ] SwiftData model migrations tested
+- [ ] Local archive schema migrations tested
 - [ ] Privacy Policy updated (what data is collected)
 - [ ] Accessibility review (VoiceOver, Dynamic Type)
 


+1 -1

HealthProbe/Doc/Open Source Publication Guidelines.md

View

@@ -201,7 +201,7 @@ Before release, verify:
 
 ### Privacy Compliance
 - [ ] No raw health sample data in examples
-- [ ] CloudKit sync described as optional, not default
+- [ ] HealthProbe CloudKit/iCloud sync is not described as a product goal
 - [ ] User consent documented
 - [ ] Data retention policy clear
 - [ ] No tracking/analytics hidden in code


+18 -9

HealthProbe/Doc/README.md

View

@@ -25,9 +25,10 @@
 | **Core Architecture** | 📋 Designed | See "Complete Specification" |
 | **HealthKit Integration** | ⏳ Pending | Implement anchored queries, observer queries |
 | **Anomaly Detection** | 📋 Designed | Logic documented, pending implementation |
-| **Sync Monitoring** | 📋 Designed | Background thread model defined |
+| **Sync Context Logging** | 📋 Designed | Log Health/iCloud state as forensic context; do not sync HealthProbe data via iCloud |
 | **UI Dashboard** | ⏳ Pending | Wireframes in Complete Specification |
-| **Data Export** | 📋 Designed | Format specs ready |
+| **Local Archive Store** | 📋 Designed | Robust on-device archive is the source of truth |
+| **Reports & Point Exports** | 📋 Designed | Export only selected reports/record tables, not a complete routine dump |
 | **macOS Companion** | 🔄 Future | Post-MVP enhancement |
 
 ---
@@ -59,10 +60,10 @@ See **Complete Specification § 2** for detailed observed cases and forensic imp
    - Duplicate fingerprinting
    - Divergence trend analysis
 
-3. **Implement Sync Monitoring** (`Sources/SyncMonitor.swift`)
-   - Background thread that tracks iCloud state changes
-   - Document sync events in audit trail
-   - Trigger re-baselining on sync state transitions
+3. **Implement Local Archive Store** (`Sources/ArchiveStore.swift`)
+   - Single robust local database for all archived samples
+   - Preserve cross-type relationships, sources, devices, metadata, and fingerprints
+   - Keep SwiftData as UI/cache/settings/history only
 
 4. **Create UI Dashboard** (`Views/HealthStatusView.swift`)
    - Show current health status
@@ -78,10 +79,11 @@ See **Complete Specification § 2** for detailed observed cases and forensic imp
 |----------|-----------|
 | **Read-only + HealthKit** | Never modify health data; pure observation only |
 | **Local-first storage** | Full functionality without internet; privacy-first |
-| **SwiftData** | Efficient local persistence, encrypted by iOS |
+| **Archive DB as truth** | Store HealthKit samples and metadata in a robust local database, not split per data type |
+| **SwiftData as UI cache** | Keep precomputed values, settings, logs, and history for visualization only |
 | **Anchored queries** | Minimize HealthKit load, reduce battery impact |
-| **Separate sync thread** | Non-blocking, responsive UI during background work |
-| **Aggregated digests only** | CloudKit sync respects privacy, no raw samples exported |
+| **No HealthProbe iCloud sync** | Device HealthKit databases evolve independently; CloudKit sync adds complexity without proven forensic benefit |
+| **Selective forensic capture** | When Health/iCloud rewrites or downsamples historical data, counts alone are insufficient; HealthProbe archives complete samples for selected types in one local store |
 
 ---
 
@@ -91,6 +93,13 @@ See **Complete Specification § 2** for detailed observed cases and forensic imp
   - Concrete cases from DearApple
   - Full technical architecture
   - MVP feature scope + future roadmap
+- **v1.1** — 2026-05-17 — Objective extension (post-findings)
+  - New observed behavior: Apple-side consolidation/downsampling can rewrite historical samples (not just add/delete)
+  - HealthProbe scope extended: from “counter of records” → “forensic backup agent” (local-only archives for selected types)
+- **v1.2** — 2026-05-18 — Storage direction update
+  - Robust single local archive store becomes the source of truth
+  - SwiftData is limited to precomputed UI data, settings, logs, and history
+  - CloudKit/iCloud sync removed from product goals; reports and point exports replace routine complete export ambitions
 
 ---
 


+14 -14

HealthProbe/HealthProbeApp.swift

View

@@ -21,12 +21,12 @@ struct HealthProbeApp: App {
         .modelContainer(sharedModelContainer)
     }
 
-    // Two separate ModelConfiguration instances:
-    //   cloudConfig - audit data
-    //   localConfig - local-only settings and operation metadata
+    // Two local ModelConfiguration instances:
+    //   uiCacheConfig - derived audit/index data for UI
+    //   localConfig   - local-only settings and operation metadata
     //
-    // ⚠️ DeviceProfile is kept local-only (not synced to CloudKit) since it's device-specific
-    //    cosmetic data (name, color tag) that should not cross devices.
+    // SwiftData is not the forensic source of truth. Complete HealthKit samples
+    // belong in the local archive store; these rows must remain rebuildable.
     private static func createModelContainer() throws -> ModelContainer {
         let fullSchema = Schema([
             HealthSnapshot.self, TypeCount.self, YearlyCount.self, HealthRecord.self, TypeDistributionBin.self,
@@ -36,18 +36,18 @@ struct HealthProbeApp: App {
 
         let appSupportURL = URL.applicationSupportDirectory
         try FileManager.default.createDirectory(at: appSupportURL, withIntermediateDirectories: true)
-        let cloudStoreURL = appSupportURL.appending(path: "HealthProbeRecords.store")
+        let uiCacheStoreURL = appSupportURL.appending(path: "HealthProbeRecords.store")
 
-        let cloudKitModels = Schema([
+        let uiCacheModels = Schema([
             HealthSnapshot.self, TypeCount.self, YearlyCount.self, HealthRecord.self, TypeDistributionBin.self,
             SnapshotDelta.self, TypeDelta.self, AnomalyRecord.self,
         ])
         let localModels = Schema([OperationLog.self, DeviceProfile.self, MetricTimeoutProfile.self])
 
-        let cloudConfig = ModelConfiguration(
-            "cloud",
-            schema: cloudKitModels,
-            url: cloudStoreURL,
+        let uiCacheConfig = ModelConfiguration(
+            "ui-cache",
+            schema: uiCacheModels,
+            url: uiCacheStoreURL,
             cloudKitDatabase: .none
         )
         let localConfig = ModelConfiguration(
@@ -58,10 +58,10 @@ struct HealthProbeApp: App {
         )
 
         do {
-            return try ModelContainer(for: fullSchema, configurations: [cloudConfig, localConfig])
+            return try ModelContainer(for: fullSchema, configurations: [uiCacheConfig, localConfig])
         } catch {
             let candidates: [URL] = [
-                cloudStoreURL,
+                uiCacheStoreURL,
                 appSupportURL.appending(path: "HealthProbeRecords.store.shm"),
                 appSupportURL.appending(path: "HealthProbeRecords.store.wal"),
                 appSupportURL.appending(path: "HealthProbeCloud.store"),
@@ -72,7 +72,7 @@ struct HealthProbeApp: App {
                 appSupportURL.appending(path: "HealthProbeLocal.store.wal"),
             ]
             for url in candidates { try? FileManager.default.removeItem(at: url) }
-            return try ModelContainer(for: fullSchema, configurations: [cloudConfig, localConfig])
+            return try ModelContainer(for: fullSchema, configurations: [uiCacheConfig, localConfig])
         }
     }
 }


+0 -1

HealthProbe/Models/SnapshotDelta.swift

View

@@ -14,7 +14,6 @@ import SwiftData
     var changedMetricCount: Int = 0
     var appearedMetricCount: Int = 0
     var disappearedMetricCount: Int = 0
-    var isCloudKitImported: Bool = false
     @Relationship(deleteRule: .cascade, inverse: \TypeDelta.delta)
     var typeDeltas: [TypeDelta]? = []
 


+0 -1

HealthProbe/Models/TypeDelta.swift

View

@@ -13,7 +13,6 @@ import SwiftData
     var transitionRaw: String = "unchanged"
     var reasonRaw: String = "normal"
     var yearlyCountNote: String = ""
-    var isCloudKitImported: Bool = false
     var delta: SnapshotDelta?
 
     init(typeIdentifier: String, displayName: String) {


+0 -25

HealthProbe/Services/CloudKitSyncService.swift

View

+                    -import CloudKit
+                    -import Foundation
+                    -import os.log
+                    -private let logger = Logger(subsystem: "ro.xdev.healthprobe", category: "CloudKitSyncService")
+                    -// CloudKit is treated as storage only — no ordering, no conflict resolution.
+                    -// Snapshots are identified by deviceID + snapshotID, not by chain position.
+                    -// On read, deduplicate by id BEFORE any chain traversal.
+                    -// Never assume chronological order from CloudKit fetch results.
+                    -final class CloudKitSyncService {
+                    -    static let shared = CloudKitSyncService()
+                    -    private let containerID = "iCloud.ro.xdev.healthprobe"
+                    -    func checkAvailability() async -> Bool {
+                    -        do {
+                    -            let status = try await CKContainer(identifier: containerID).accountStatus()
+                    -            return status == .available
+                    -        } catch {
+                    -            logger.error("CloudKit availability check failed: \(error)")
+                    -            return false
+                    -        }
+                    -    }
+                    -}

+0 -11

HealthProbe/Services/IntegrityService.swift

View

@@ -7,14 +7,11 @@ enum IntegrityService {
         case checksumMismatch(snapshotID: UUID, expected: String, actual: String)
         case missingDelta(fromID: UUID, toID: UUID)
         case corrupted(snapshotID: UUID, reason: String)
-        case pendingSync(deltaID: UUID)
     }
 
     // Strict mode: used by chain traversal and analysis.
     // Recomputes checksum from TypeCounts; compares with stored delta.checksumAfter.
     // Returns .valid only if they match exactly.
-    // Exception: if delta.isCloudKitImported == true AND delta.typeDeltas is nil/empty,
-    // returns .pendingSync(deltaID:) — UI shows "Syncing…".
     static func validate(snapshot: HealthSnapshot, delta: SnapshotDelta?) -> ValidationResult {
         guard let delta else {
             guard snapshot.isChainStart else {
@@ -23,11 +20,6 @@ enum IntegrityService {
             return .valid
         }
 
-        // CloudKit pending: delta arrived before typeDeltas
-        if delta.isCloudKitImported && (delta.typeDeltas?.isEmpty ?? true) {
-            return .pendingSync(deltaID: delta.id)
-        }
-
         let actual = HashService.snapshotChecksum(typeCounts: snapshot.typeCounts ?? [])
         guard actual == delta.checksumAfter else {
             return .checksumMismatch(snapshotID: snapshot.id, expected: delta.checksumAfter, actual: actual)
@@ -71,9 +63,6 @@ enum IntegrityService {
             switch result {
             case .valid:
                 break
-            case .pendingSync:
-                // CloudKit pending — emit but continue traversal
-                results.append(result)
             case .checksumMismatch, .missingDelta, .corrupted:
                 // Strict mode — stop immediately on any error
                 results.append(result)


+43 -0

HealthProbe/Services/Protocols/HealthArchiveStore.swift

View

+                    +import Foundation
+                    +import HealthKit
+                    +// Interface updated 2026-05-18 — see AGENTS.md
+                    +protocol HealthArchiveStore {
+                    +    func upsertSamples(_ samples: [HKSample], observedAt: Date) async throws -> HealthArchiveWriteSummary
+                    +    func markVerification(sampleType: HKSampleType, verifiedAt: Date) async throws
+                    +    func recordDisappearance(fingerprint: String, sampleTypeIdentifier: String, observedMissingAt: Date) async throws
+                    +    func records(for request: HealthArchiveRecordRequest) async throws -> [ArchivedHealthRecord]
+                    +    func exportReport(_ request: HealthArchiveReportRequest) async throws -> URL
+                    +}
+                    +struct HealthArchiveWriteSummary: Equatable, Sendable {
+                    +    let insertedCount: Int
+                    +    let updatedCount: Int
+                    +    let unchangedCount: Int
+                    +}
+                    +struct HealthArchiveRecordRequest: Equatable, Sendable {
+                    +    let sampleTypeIdentifier: String?
+                    +    let fingerprints: Set<String>
+                    +    let limit: Int?
+                    +}
+                    +struct HealthArchiveReportRequest: Equatable, Sendable {
+                    +    let reportID: UUID
+                    +    let title: String
+                    +    let includedFingerprints: Set<String>
+                    +}
+                    +struct ArchivedHealthRecord: Identifiable, Equatable, Sendable {
+                    +    let id: String
+                    +    let sampleTypeIdentifier: String
+                    +    let strictFingerprint: String
+                    +    let semanticFingerprint: String?
+                    +    let healthKitUUIDHash: String?
+                    +    let startDate: Date
+                    +    let endDate: Date
+                    +    let firstSeenAt: Date
+                    +    let lastSeenAt: Date?
+                    +    let lastVerifiedAt: Date?
+                    +    let disappearedAt: Date?
+                    +}

+1 -1

HealthProbe/Services/SnapshotLifecycleService.swift

View

@@ -22,7 +22,7 @@ enum SnapshotLifecycleService {
 
         let integrityResult = IntegrityService.validate(snapshot: snapshot, delta: incomingDelta)
         switch integrityResult {
-        case .valid, .pendingSync:
+        case .valid:
             break
         case .checksumMismatch(_, let expected, let actual):
             willBreakChain = true


+19 -22

IMPLEMENTATION_STATUS.md

View

@@ -11,8 +11,8 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
 ✅ **AnomalyType.swift** — All anomaly types + Severity + TypeTransition + TypeDeltaReason enums
 ✅ **HealthSnapshot.swift** — Chain metadata, quality, trigger context, registry fingerprinting, timezone context
 ✅ **TypeCount.swift** — Count with hash, date range, quality, yearly counts with cascade relationship
-✅ **SnapshotDelta.swift** — Delta with checksums, CloudKit import flag, cascade relationship to TypeDeltas
-✅ **TypeDelta.swift** — Per-type delta with transition, reason, quality before/after, yearly count note, CloudKit import flag
+✅ **SnapshotDelta.swift** — Local delta with checksums and cascade relationship to TypeDeltas
+✅ **TypeDelta.swift** — Per-type local delta with transition, reason, quality before/after, yearly count note
 ✅ **AnomalyRecord.swift** — Anomaly record with deltaID set structurally by detector, never by caller
 ✅ **OperationLog.swift** — Audit trail for destructive operations with JSON-encoded affected snapshot IDs
 ✅ **YearlyCount.swift** — Per-year sample counts with approximation flag
@@ -74,12 +74,10 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
 - `validate()` — strict mode:
   - Recomputes checksum from TypeCounts
   - Compares with delta.checksumAfter
-  - Returns .pendingSync if delta.isCloudKitImported && typeDeltas empty (not an error)
   - Returns .valid or .checksumMismatch / .missingDelta / .corrupted
 - `validateChain()` — walk backwards from latest via previousSnapshotID:
   - **Fork detection**: asserts no duplicate previousSnapshotID (returns .corrupted immediately)
   - Stops at first mismatch (no auto-repair, no skips)
-  - Emits .pendingSync for CloudKit-pending nodes, continues traversal
 - **Quality aggregation**: loading > unauthorized (only if ALL) > partial (any failed/unauthorized) > complete
 
 #### Step 10: SnapshotLifecycleService ✅
@@ -91,13 +89,13 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
 - **OperationLog**: always written atomically with deletive changes
 - **Post-save verification**: re-fetches log by ID, recovery re-insert if missing, logs critical error
 
-#### Step 12: CloudKitSyncService ✅
-- `checkAvailability()` — async account status check
+#### Step 12: Local-only storage refactor ✅
+- Removed CloudKitSyncService and CloudKit-pending chain states
 - **ModelContainer split**:
-  - cloudKitConfig: HealthSnapshot, TypeCount, YearlyCount, SnapshotDelta, TypeDelta, AnomalyRecord (synced to iCloud.ro.xdev.healthprobe)
-  - localConfig: OperationLog, DeviceProfile (local-only, never synced)
-- Simulator uses in-memory store; device uses CloudKit private database
-- Schema migration recovery: removes both stores and retries once on failure
+  - uiCacheConfig: HealthSnapshot, TypeCount, YearlyCount, SnapshotDelta, TypeDelta, AnomalyRecord (derived local UI/index data)
+  - localConfig: OperationLog, DeviceProfile, MetricTimeoutProfile (local-only settings and operation metadata)
+- Added `HealthArchiveStore` protocol for the single local archive store source of truth
+- Schema migration recovery: removes legacy SwiftData stores and retries once on failure
 
 ### UI (Step 13)
 
@@ -154,12 +152,12 @@ These tests should be run to ensure all backend functionality is correct:
 
 ### Advanced Features (11-20)
 - [ ] 11. DB reset with Keychain survival → same deviceID, isChainStart=true, recoveredDeviceID=true
-- [ ] 12. CloudKit unavailable → app functions (local-only fallback)
+- [ ] 12. Local-only launch → app functions without iCloud/CloudKit entitlements
 - [ ] 13. Observer debounce → 10 rapid callbacks = exactly 1 snapshot (triggerReason=observerCallback)
 - [ ] 14. Unsupported type → TypeCount(count=-1, quality=.failed, isUnsupported=true), "Unsupported" UI
 - [ ] 15. YearlyCount timezone → Calendar.current used, isApproximate=true if bucket > day
 - [ ] 16. Delta merge with unavailable counts → merged countDelta=0, impaired reason preserved
-- [ ] 17. CloudKit pending relationships → TypeDelta(delta=nil, isCloudKitImported=true) shows "Syncing…"
+- [ ] 17. Missing local delta/typeDeltas → integrity validation surfaces the fault, never hides it as sync latency
 - [ ] 18. First auth after full deny (quality gate) → no anomalies, current.isPostRestore=true, isPostRestoreInferred=true
 - [ ] 19. Chain fork → validateChain() returns .corrupted(reason: "chain fork detected"), stops
 - [ ] 20. disappeared→appeared merge with -1 source → merged countDelta=0, reason != .normal
@@ -169,7 +167,7 @@ These tests should be run to ensure all backend functionality is correct:
 - [ ] 22. Debounce + manual overlap → no observer snapshot if manual created during debounce
 - [ ] 23. completionHandler unconditional → called via defer, never gated on scheduling success
 - [ ] 24. isPostRestore forwarding → suppression forwarded past low-quality, consumed on next .complete
-- [ ] 25. CloudKit pending delta → validateChain() returns .pendingSync, UI shows "Syncing…"
+- [ ] 25. Missing delta → validateChain() returns .missingDelta and stops
 - [ ] 26. OperationLog verification → recovery re-insert if missing after save, log critical error
 
 ### Coherence & Edge Cases (27-32)
@@ -198,11 +196,10 @@ These tests should be run to ensure all backend functionality is correct:
 - **Fork detection** prevents chain divergence (asserts no duplicate previousSnapshotID)
 - **Checksum validation** ensures data wasn't corrupted between snapshots
 
-### CloudKit Readiness
-- **isCloudKitImported** distinguishes sync latency from corruption
-  - Pending: delta=nil, isCloudKitImported=true → "Syncing…" in UI
-  - Bug: delta=nil, isCloudKitImported=false → error log + validation stops
-- **ModelConfiguration split** keeps OperationLog local-only (audit trail never crosses devices)
+### Local Archive Direction
+- CloudKit/iCloud sync is not a product goal
+- SwiftData rows are derived UI/index data and must be rebuildable from the local archive store
+- Missing deltas or type deltas are treated as local integrity faults, not remote sync latency
 
 ### Observability
 - **Reason priority** makes anomaly suppression deterministic
@@ -215,8 +212,8 @@ These tests should be run to ensure all backend functionality is correct:
 
 1. **Hash** covers only count + date range, not distribution (silentReplacement is best-effort)
 2. **YearlyCount** precision requires daily bucket granularity (noted if isApproximate)
-3. **No server-side conflict resolution** for CloudKit (storage-only design, no ordering)
-4. **No transaction-level consistency** for cross-device chain reconstruction (local chains validated, CloudKit pending states flagged)
+3. **Local archive store implementation is still pending** (protocol boundary exists, SQLite/archive schema still needed)
+4. **No automatic cross-device reconstruction**; cross-device analysis is future macOS/report work
 
 ## Next Steps
 
@@ -224,7 +221,7 @@ These tests should be run to ensure all backend functionality is correct:
 1. Run all 32 verification checks against real HealthKit data
 2. Create unit tests for delta merge, reason priority, anomaly detection
 3. Test observer callback debounce with real HKObserverQuery
-4. Validate CloudKit sync with simulator and device
+4. Implement the local archive store behind `HealthArchiveStore`
 
 ### Post-MVP
 1. Integrate actual BGTask expiration guard for observer snapshots (capture partial results)
@@ -234,7 +231,7 @@ These tests should be run to ensure all backend functionality is correct:
 
 ---
 
-**Built with:** SwiftUI, SwiftData, HealthKit, CloudKit, CryptoKit  
+**Built with:** SwiftUI, SwiftData, HealthKit, CryptoKit  
 **Minimum iOS:** 17.0  
 **Target iOS:** 26.4  
 **Swift Version:** 5.9+


	@@ -5,6 +5,8 @@
5	5	HealthProbe is built by multiple AI models, each owning a distinct domain.
6	6	This document defines boundaries, interfaces, and handoff contracts.
7	7
	8	+Agentic reality: The repo is developed largely via agents (Codex CLI, Claude, and dedicated model sessions). When updating product scope, update docs first, then implement behind flags, and add tests for the new behavior.
	9	+
8	10	---
9	11
10	12	## Model Allocation
	@@ -12,10 +14,11 @@ This document defines boundaries, interfaces, and handoff contracts.
12	14	\| Domain \| Owner \| Tools \|
13	15	\|--------\|-------\|-------\|
14	16	\| UI / SwiftUI Views \| Claude Code \| Xcode, SwiftUI, CLAUDE.md \|
15		-\| Data Models (SwiftData) \| Dedicated model session \| Xcode, Swift \|
	17	+\| Archive Store \| Dedicated model session \| SQLite/local archive format, HealthKit metadata mapping \|
	18	+\| Data Models (SwiftData) \| Dedicated model session \| Xcode, Swift; derived UI/cache/settings/log models only \|
16	19	\| HealthKit Integration \| Dedicated model session \| Xcode, HealthKit docs \|
17	20	\| Anomaly Detection Algorithms \| Dedicated model session \| Swift, statistical references \|
18		-\| Sync Monitoring \| Dedicated model session \| Xcode, CloudKit docs \|
	21	+\| Context Monitoring \| Dedicated model session \| Xcode; logs Health/iCloud state as context only \|
19	22	\| Documentation \| Claude Code + dedicated session \| Markdown \|
20	23	\| Tests \| Dedicated model session \| XCTest, Swift Testing \|
21	24
	@@ -28,14 +31,16 @@ HealthProbe/
28	31	├── Views/ ← Claude Code (UI)
29	32	├── ViewModels/ ← Claude Code (UI)
30	33	├── Utilities/ ← Claude Code (shared helpers, mocks)
31		-├── Models/ ← Models agent (SwiftData schemas)
32		-├── Services/ ← Services agent (HealthKit, anomaly, sync)
	34	+├── Models/ ← Models agent (SwiftData UI/cache schemas)
	35	+├── Services/ ← Services agent (HealthKit, archive store, anomaly, context)
33	36	└── Tests/ ← Tests agent
34	37	```
35	38
36	39	Rule: Each agent writes only within its owned directories.
37	40	Cross-boundary changes require an explicit interface contract (protocol) first.
38	41
	42	+Documentation scope: `HealthProbe/Doc/` is shared. Keep it consistent with shipped behavior, and add a dated entry when objectives change.
	43	+
39	44	---
40	45
41	46	## Interface Contracts
	@@ -77,15 +82,15 @@ protocol AuditTrailProtocol {
77	82	}
78	83	```
79	84
80		-### SyncMonitorProtocol
	85	+### ContextMonitorProtocol
81	86
82	87	```swift
83	88	/// Owned by: Services agent
84		-/// Consumed by: UI (SyncViewModel)
85		-protocol SyncMonitorProtocol {
	89	+/// Consumed by: UI (ContextViewModel)
	90	+protocol ContextMonitorProtocol {
86	91	var iCloudEnabled: Bool { get }
87		- var lastSyncDate: Date? { get }
88		- var stateChanges: [SyncStateChange] { get }
	92	+ var lastObservedAt: Date? { get }
	93	+ var stateChanges: [ContextStateChange] { get }
89	94	}
90	95	```
91	96
	@@ -122,14 +127,11 @@ final class TypeDistributionBin {
122	127	// Import uses a global anchored query per data type so follow-up snapshots fetch only
123	128	// HealthKit deltas instead of scanning calendar blocks with fixed per-query latency.
124	129
125		-// Interface updated 2026-05-14 — see AGENTS.md
126		-// TypeCount.recordArchiveData is stored with SwiftData external storage.
127		-// The archive remains a complete per-record forensic snapshot, but large blobs should
128		-// not be kept inline with the TypeCount row because high-volume HealthKit stores can
129		-// exceed device memory during import or detail inspection.
130		-// Initial high-volume imports stream records directly into a compact binary archive
131		-// instead of building a full in-memory dictionary/array; plist archives remain readable
132		-// for backwards compatibility.
	130	+// Interface updated 2026-05-18 — see AGENTS.md
	131	+// SwiftData is not the forensic source of truth. TypeCount and related rows store
	132	+// precomputed UI/index data only. Complete HealthKit samples and metadata belong
	133	+// in the local archive store, in one schema that can preserve relationships across
	134	+// data types, sources, devices, workouts, and metadata.
133	135
134	136	// Interface updated 2026-05-17 — see AGENTS.md
135	137	// Models/TypeCount.detailCacheData stores precomputed detail data for the current
	@@ -230,6 +232,8 @@ The following modules involve non-trivial logic and should be reviewed carefully
230	232	- No location data or patterns enabling re-identification
231	233	- Synthetic data only in tests and previews
232	234
	235	+Clarification: “No raw health values” applies to this repository’s contents. The app may optionally store a user's raw HealthKit samples locally on-device for forensic backup purposes, but such samples must never appear in source control, logs, or docs.
	236	+
233	237	---
234	238
235	239	## Communication Between Agents

	@@ -6,7 +6,7 @@
6	6	It detects anomalies: data loss, historical insertions, duplicates, divergence trends.
7	7	Full specification: `HealthProbe/Doc/HealthProbe – Complete Specification & Motivations.md`
8	8
9		-Current state: Xcode scaffold only. `ContentView.swift` and `Item.swift` are default templates — replace them entirely.
	9	+Current state: SwiftUI + SwiftData app is active. Product direction changed on 2026-05-18: HealthProbe is a local audit/capture agent. Do not add HealthProbe CloudKit/iCloud sync.
10	10
11	11	---
12	12
	@@ -21,7 +21,7 @@ Claude Code is responsible for:
21	21	- Accessibility (VoiceOver, Dynamic Type)
22	22
23	23	Claude Code does NOT own:
24		-- `Services/` — HealthKit queries, anomaly detection, sync monitoring (see AGENTS.md)
	24	+- `Services/` — HealthKit queries, anomaly detection, archive store, context monitoring (see AGENTS.md)
25	25	- `Models/` — SwiftData models (see AGENTS.md)
26	26	- Entitlements, Info.plist, project configuration
27	27
	@@ -36,7 +36,7 @@ App (TabView)
36	36	├── Tab 1: Dashboard → DashboardView
37	37	├── Tab 2: Anomalies → AnomalyListView → AnomalyDetailView
38	38	├── Tab 3: Audit Trail → AuditTrailView
39		-├── Tab 4: Sync Status → SyncStatusView
	39	+├── Tab 4: Archive Status → ArchiveStatusView
40	40	└── Tab 5: Settings → SettingsView
41	41	```
42	42
	@@ -67,18 +67,17 @@ App (TabView)
67	67	- Search/filter by event type
68	68	- Export button → JSON
69	69
70		-### SyncStatusView
71		-- Current iCloud sync state (chip: enabled / local-only)
72		-- Last sync timestamp
73		-- List of `SyncStateChange` events (most recent first)
74		-- Visual indicator when sync is stalled (no event in > 48h)
	70	+### ArchiveStatusView
	71	+- Current local archive health
	72	+- Last archive verification timestamp
	73	+- Selected data types covered by forensic capture
	74	+- Recent Health/iCloud context events (for correlation only; no HealthProbe sync)
75	75
76	76	### SettingsView
77	77	- Check frequency: 2h / 6h / 12h / 24h (Picker)
78	78	- Sample types to monitor (MultiSelect toggle list)
79	79	- Alert thresholds (severity level for push notifications)
80		-- CloudKit sync toggle (off by default, with privacy disclaimer)
81		-- Export all data (JSON)
	80	+- Point export/report actions for selected findings
82	81	- Delete all audit data (destructive, confirm alert)
83	82
84	83	---
	@@ -107,7 +106,7 @@ App (TabView)
107	106	- `exclamationmark.triangle.fill` — warning
108	107	- `xmark.shield.fill` — critical
109	108	- `clock.arrow.circlepath` — audit trail
110		-- `antenna.radiowaves.left.and.right` — sync
	109	+- `externaldrive.fill.badge.checkmark` — archive status
111	110	- `waveform.path.ecg` — health data
112	111	- `doc.text.magnifyingglass` — anomaly detail
113	112
	@@ -202,14 +201,14 @@ HealthProbe/
202	201	│ │ └── AnomalyDetailView.swift
203	202	│ ├── AuditTrail/
204	203	│ │ └── AuditTrailView.swift
205		-│ ├── Sync/
206		-│ │ └── SyncStatusView.swift
	204	+│ ├── Archive/
	205	+│ │ └── ArchiveStatusView.swift
207	206	│ └── Settings/
208	207	│ └── SettingsView.swift
209	208	├── ViewModels/
210	209	│ ├── DashboardViewModel.swift
211	210	│ ├── AnomalyListViewModel.swift
212		-│ └── SyncViewModel.swift
	211	+│ └── ArchiveStatusViewModel.swift
213	212	├── Models/ ← NOT owned by Claude Code
214	213	├── Services/ ← NOT owned by Claude Code
215	214	└── Utilities/

	@@ -99,9 +99,6 @@
99	99	com.apple.HealthKit = {
100	100	enabled = 1;
101	101	};
102		- com.apple.iCloud = {
103		- enabled = 1;
104		- };
105	102	};
106	103	};
107	104	};

	@@ -15,6 +15,7 @@
15	15	\| Timing precision \| Anchored queries may batch multiple changes, lose granular timestamps \| Store both sample timestamp AND observation timestamp \|
16	16	\| Private HealthKit types \| Some data types not accessible to third-party apps \| Accept data available only to Health.app \|
17	17	\| Cross-device sync delays \| Watch-to-phone-to-cloud can take 24+ hours \| Extend observation window, don't flag immediate after device sync \|
	18	+\| Consolidation / downsampling \| Apple Health/iCloud can rewrite high-frequency historical samples in-place (count decreases; values/intervals change) \| Store fingerprints + optionally archive raw samples for selected types (local-only forensic backup mode) \|
18	19
19	20	### 1.2 iOS Background Mode Limitations
20	21
	@@ -42,6 +43,7 @@
42	43	- Cannot reconstruct data that was never observed
43	44	- Snapshot from 6 months ago may have samples no longer in HealthKit
44	45	- Gap detection assumes continuous observation (may be false positive if app uninstalled then reinstalled)
	46	+- If Apple consolidates history, counts alone can be misleading (a month can “lose” samples but keep the same totals via aggregation); value-level forensics are required for proof
45	47
46	48	---
47	49
	@@ -53,7 +55,7 @@
53	55	\|------\|--------\|-----------\|
54	56	\| Raw health data exfiltration \| CRITICAL: user's personal health history exposed \| ✅ Local-only storage, never sends raw samples \|
55	57	\| Device fingerprinting \| HIGH: tracking user across services \| ✅ Salted hash of device ID, stored locally only \|
56		-\| Timing attacks (inferring behavior) \| MEDIUM: sync patterns reveal habits \| ✅ CloudKit digest is aggregated & anonymized \|
	58	+\| Timing attacks (inferring behavior) \| MEDIUM: archive/check patterns reveal habits \| ✅ No automatic cloud sync; reports are explicit local exports \|
57	59	\| App crashes leaking data \| LOW: crash logs may contain HealthKit info \| ✅ All logging is aggregated (counts, not values) \|
58	60
59	61	### 2.2 Data Integrity Risks (Mitigation: Good ✅)
	@@ -62,6 +64,7 @@
62	64	\|------\|--------\|-----------\|
63	65	\| Snapshot corruption \| HIGH: audit trail becomes unreliable \| ✅ Use MD5 checksum of snapshots, detect corruption \|
64	66	\| Lost audit trail on uninstall \| HIGH: forensic data disappears \| ⚠️ PARTIAL: Encourage export before uninstall; future: iCloud backup option \|
	67	+\| Silent rewrites (no deletion events) \| HIGH: history can change without HKDeletedObject evidence \| ✅ Detect via fingerprint diffs; Forensic Backup Mode can preserve per-sample evidence locally for selected types \|
65	68	\| Clock skew (device time wrong) \| MEDIUM: timestamps inaccurate, anomaly detection confused \| ⚠️ Log both device time + time since boot, detect skew \|
66	69	\| Concurrent modification (app + Health.app) \| LOW: race conditions during query \| ✅ Anchored queries are atomic \|
67	70
	@@ -71,7 +74,7 @@
71	74	\|------\|--------\|-----------\|
72	75	\| Malicious apps accessing HealthKit \| MEDIUM: third-party apps can read our data \| ✅ iOS sandboxing; ask user for HealthKit permission per app \|
73	76	\| Jailbroken device \| CRITICAL: all bets off \| ⚠️ Not defendable; document assumption: standard iOS device \|
74		-\| iCloud account compromise \| HIGH: CloudKit digest could be leaked \| ✅ OPTIONAL: Users can disable CloudKit sync in settings \|
	77	+\| iCloud account compromise \| MEDIUM: Apple Health/iCloud state may still affect source data \| ✅ HealthProbe archive remains local; no HealthProbe CloudKit sync \|
75	78	\| Local device theft \| MEDIUM: thief can see audit trail \| ✅ Data encrypted by iOS, requires device unlock \|
76	79
77	80	---
	@@ -328,8 +331,8 @@ Post-incident:
328	331
329	332	```
330	333	Setup:
331		- 1. Enable optional CloudKit sync (with privacy settings)
332		- 2. Opt into community pattern analysis
	334	+ 1. Export anonymized anomaly summaries manually
	335	+ 2. Keep raw archive local
333	336
334	337	Analysis:
335	338	1. Correlate own data loss with iOS release dates
	@@ -386,9 +389,9 @@ Future: Alert before data is lost
386	389	\| Issue \| Cause \| Fix \|
387	390	\|-------\|-------\|-----\|
388	391	\| No anomalies detected for weeks \| Background fetch disabled \| Settings → HealthProbe → Background Refresh \|
389		-\| Snapshots not being saved \| Insufficient storage \| Free up space; SwiftData limited by device storage \|
	392	+\| Snapshots not being saved \| Insufficient storage or archive write failure \| Free up space; verify local archive health and SwiftData cache rebuild \|
390	393	\| Sync state not updating \| iCloud token check failing \| Sign out/in to iCloud; restart device \|
391		-\| Old audit trail entries missing \| SwiftData old entries pruned \| Expected after > 6 months; export before uninstall \|
	394	+\| Old audit trail entries missing \| SwiftData cache/log retention policy or migration issue \| Rebuild derived views from archive where possible; export reports before uninstall \|
392	395
393	396	---
394	397

	@@ -8,7 +8,7 @@ It detects anomalies such as:
8	8	- duplicate records
9	9	- divergence trends over time
10	10
11		-The application operates as a local audit agent and optionally synchronizes aggregated snapshots (digests) via CloudKit.
	11	+The application operates as a local audit and capture agent. It does not sync HealthProbe data via CloudKit/iCloud; HealthKit databases can evolve differently per device, so the MVP keeps each device archive local and explicit.
12	12
13	13	⚠️ This document describes ONLY the iOS application (MVP phase).
14	14	A future macOS application will act as a visualization/analysis layer.
	@@ -27,11 +27,12 @@ A future macOS application will act as a visualization/analysis layer.
27	27	3. Incremental observation
28	28	- Use anchored queries to track changes
29	29
30		-4. Minimal data exfiltration
31		- - Only aggregated digests are synced (no raw Health data)
	30	+4. No app cloud sync
	31	+ - HealthProbe does not sync raw samples, digests, or reports through CloudKit/iCloud
32	32
33		-5. Forward compatibility
34		- - CloudKit schema must be shared with a future macOS application
	33	+5. Robust local archive
	34	+ - Store captured HealthKit data in one local archive store, not per-data-type silos
	35	+ - SwiftData is used for derived UI data, settings, logs, and history only
35	36
36	37	---
37	38
	@@ -48,6 +49,14 @@ Track:
48	49	- High Heart Rate Events
49	50	- Other relevant samples (extensible)
50	51
	52	+Persist:
	53	+- sample values and units
	54	+- source and source revision metadata
	55	+- device metadata exposed by HealthKit
	56	+- HealthKit metadata dictionaries
	57	+- first-seen / last-seen / last-verified timestamps
	58	+- fingerprints for matching against Apple Health XML exports and backup database extracts
	59	+
51	60	---
52	61
53	62	### 2. Anomaly Detection

	@@ -1,8 +1,8 @@
1	1	# HealthProbe – Complete Specification & Motivations
2	2
3		-Version: 1.0
	3	+Version: 1.2
4	4	Status: MVP (iOS monitoring agent)
5		-Last Updated: 2026-05-01
	5	+Last Updated: 2026-05-18
6	6
7	7	---
8	8
	@@ -12,7 +12,7 @@ HealthProbe is an audit and integrity monitoring tool for Apple HealthKit, d
12	12
13	13	Core Problem: Apple Health data loss events (confirmed Sept 2025 incident, ongoing sporadic reports) lack detective mechanisms. Users do not know when data has been lost, corrupted, or silently modified.
14	14
15		-Solution: HealthProbe monitors HealthKit in real-time and maintains a tamper-proof audit trail, enabling post-incident forensic analysis and pattern detection.
	15	+Solution: HealthProbe incrementally captures HealthKit data into a robust local archive and maintains an audit trail for post-incident forensic analysis. SwiftData is used for UI assistance, settings, logs, history, and precomputed values; it is not the source of truth. The local archive exists because Apple Health/iCloud can rewrite / downsample / consolidate historical samples in-place (not only add/delete), which cannot be proven by counts alone.
16	16
17	17	---
18	18
	@@ -111,6 +111,19 @@ HealthProbe is an audit and integrity monitoring tool for Apple HealthKit, d
111	111
112	112	---
113	113
	114	+#### E. Consolidation / Downsampling (Historical Rewrites)
	115	+Pattern: For high-frequency data types, Apple Health/iCloud may rewrite historical samples such that:
	116	+- total sample `count` decreases for older months
	117	+- timestamps in a later snapshot/export become a subset of an earlier snapshot/export (thinning)
	118	+- some samples become interval-based (`endDate > startDate`) and/or values become fractional
	119	+- for cumulative quantities, `value_sum` can remain stable while per-sample `value_max` increases (consolidation)
	120	+
	121	+Why it matters: A “record counter” cannot distinguish discard vs consolidation. HealthProbe must support value-level forensics and (optionally) preserve complete evidence locally.
	122	+
	123	+Detection method: Snapshot comparison of fingerprints plus optional per-sample archives for selected data types.
	124	+
	125	+---
	126	+
114	127	### 2.3 Why This Matters
115	128
116	129	\| Concern \| Impact \| HealthProbe Role \|
	@@ -130,9 +143,10 @@ HealthProbe is an audit and integrity monitoring tool for Apple HealthKit, d
130	143	1. Read-only operations (never modify HealthKit data)
131	144	2. Local-first (full functionality without network)
132	145	3. Incremental queries (efficient, avoid repeating work)
133		-4. Minimal exfiltration (only digests, never raw samples)
	146	+4. Single archive store (do not split the forensic store per data type; cross-type relationships and shared metadata matter)
134	147	5. Auditability (every observation logged, timestamped, reproducible)
135		-6. Privacy by default (all aggregated, no PII stored)
	148	+6. Privacy by default (no HealthProbe cloud sync; local storage remains under user control)
	149	+7. Forensic capture (selected data types are archived locally as complete per-sample records with metadata to preserve evidence against silent rewrites)
136	150
137	151	### 3.2 Threading Model
138	152
	@@ -157,49 +171,38 @@ HealthProbe is an audit and integrity monitoring tool for Apple HealthKit, d
157	171	├─ Write detected anomalies
158	172	│
159	173	┌──────────────▼──────────────────────────┐
160		-│ Local Storage (SwiftData) │
161		-│ - Audit trail (immutable append-only) │
162		-│ - Snapshots (for forensics) │
163		-│ - Anomaly records │
164		-│ - Metadata (versions, migrations) │
	174	+│ Local Archive Store │
	175	+│ - Canonical HealthKit samples │
	176	+│ - Sources, devices, metadata │
	177	+│ - Cross-type relationships │
	178	+│ - Fingerprints and verification hashes │
	179	+└──────────────┬──────────────────────────┘
	180	+ │
	181	+┌──────────────▼──────────────────────────┐
	182	+│ SwiftData UI Store │
	183	+│ - Precomputed counts/statistics │
	184	+│ - Visualization state and settings │
	185	+│ - Logs, history, report indexes │
165	186	└─────────────────────────────────────────┘
166	187	```
167	188
168		-### 3.3 Data Model (SwiftData)
	189	+### 3.3 Storage Model
169	190
170		-```swift
171		-// Immutable audit log entry
172		-@Model
173		-final class AuditEntry {
174		- var timestamp: Date
175		- var eventType: String // "snapshot", "deletion", "insertion", etc.
176		- var samples: [HealthSample]
177		- var anomalies: [AnomalyReport]
178		- var deviceInfo: DeviceMetadata
179		-}
	191	+Local Archive Store (source of truth):
	192	+- one robust local database for all archived samples, not one archive per data type
	193	+- normalized entities for samples, workouts, sources, source revisions, devices, metadata, relationships, and observations
	194	+- multiple fingerprints per sample: HealthKit UUID, strict fingerprint, semantic fingerprint, and fuzzy matching keys for export/backup reconciliation
	195	+- append-only observation history (`firstSeen`, `lastSeen`, `lastVerified`, disappearance evidence)
	196	+- snapshot-level and table-level hashes for integrity checks
180	197
181		-// Snapshot for forensic comparison
182		-@Model
183		-final class HealthSnapshot {
184		- var capturedAt: Date
185		- var sampleType: String // "HKWorkout", "HKQuantity:HeartRate", etc.
186		- var sourceDevice: String
187		- var recordCount: Int
188		- var checksum: String // MD5 of aggregated data
189		- var digest: [String: Int] // { "source": count, ... }
190		-}
	198	+SwiftData UI Store (derived/cache layer):
	199	+- settings and selected data types
	200	+- import job state and progress
	201	+- precomputed counts, temporal bins, display ranges, and summary statistics
	202	+- audit log entries and report indexes
	203	+- anomaly summaries and links into the archive store
191	204
192		-// Detected anomaly
193		-@Model
194		-final class AnomalyReport {
195		- var type: String // "historical_insertion", "deletion", "duplicate", "divergence"
196		- var severity: String // "info", "warning", "critical"
197		- var detectedAt: Date
198		- var description: String
199		- var evidence: [String: Any] // forensic data
200		- var requiresAttention: Bool
201		-}
202		-```
	205	+SwiftData rows must be rebuildable from the local archive store. If the two disagree, the archive store wins.
203	206
204	207	---
205	208
	@@ -290,9 +293,13 @@ For each metric over time:
290	293
291	294	---
292	295
293		-## 5. Sync Monitoring (Separate Thread)
	296	+## 5. Sync Context Logging
294	297
295		-### 5.1 Sync State Tracking
	298	+HealthProbe does not sync its own archive through iCloud or CloudKit. Observed HealthKit databases can diverge between devices, so cross-device HealthProbe sync would increase complexity without providing a reliable forensic source of truth.
	299	+
	300	+Health/iCloud state is still useful as context for anomalies.
	301	+
	302	+### 5.1 Context Tracking
296	303
297	304	Observe HealthKit permission & sync state:
298	305	```swift
	@@ -302,23 +309,23 @@ HKHealthStore().requestAuthorization(...)
302	309	// Monitor iCloud state
303	310	FileManager.default.ubiquityIdentityToken
304	311	// → Detects iCloud sign-in/sign-out
305		-// → Triggers re-baseline after sync state changes
	312	+// → Logs context for later correlation
306	313	```
307	314
308	315	Capture lifecycle events:
309		-- iCloud sign-in detected → re-snapshot everything
	316	+- iCloud sign-in detected → log context and schedule a local archive verification pass
310	317	- iCloud sign-out detected → note local-only mode
311	318	- Device backup initiated → pre-backup snapshot
312	319	- App backgrounded/foregrounded → check for sync activity
313	320
314		-### 5.2 Sync Documentation
	321	+### 5.2 Context Documentation
315	322
316	323	Audit trail entries:
317	324	```
318	325	[2026-05-01 14:23:15] SYNC_STATE_CHANGE: iCloud enabled
319	326	- Previous: local-only
320		- - Action: re-snapshot all data
321		- - Result: 5,432 samples captured
	327	+ - Action: archive verification scheduled
	328	+ - Result: no HealthProbe cloud sync performed
322	329
323	330	[2026-05-01 14:24:02] SYNC_COMPLETE: iCloud data merged
324	331	- Samples added: 87
	@@ -334,52 +341,46 @@ FileManager.default.ubiquityIdentityToken
334	341	- Severity: MEDIUM
335	342	```
336	343
337		-### 5.3 Background Sync Monitoring
	344	+### 5.3 Background Monitoring
338	345
339	346	iOS Background Modes enabled:
340		-- `background-fetch` — Periodic sync checks
341		-- `remote-notification` → For sync completion hints (optional)
	347	+- `background-fetch` — periodic archive and context checks
	348	+- `remote-notification` → not required for HealthProbe archive sync
342	349
343		-Sync check frequency:
	350	+Check frequency:
344	351	- Min: 2 hours
345	352	- Max: 24 hours
346	353	- Adapts based on anomaly detection frequency
347	354
348	355	---
349	356
350		-## 6. Data Export & Forensics
	357	+## 6. Local Archive, Reports & Forensics
351	358
352		-### 6.1 Export Formats
	359	+### 6.1 Local Archive Store
353	360
354		-Option 1: Local Backup (Swift)
355		-```
356		-Export structure:
357		-├── metadata.json (device, iOS version, app version)
358		-├── snapshots/ (timestamped HealthKit snapshots)
359		-├── audit_trail.jsonl (immutable log, one entry per line)
360		-├── anomalies.json (detected anomalies)
361		-└── digests.json (aggregated metrics for trending)
362		-```
	361	+The main backup artifact is the on-device archive store. It is populated incrementally from HealthKit and is not dependent on Apple Health ZIP exports or full encrypted iPhone backups.
363	362
364		-Option 2: CloudKit Digest (Optional)
365		-```
366		-Send ONLY:
367		- - Aggregated counts (not samples)
368		- - Anomaly types & timestamps (not raw health data)
369		- - Device fingerprint (not identifiable)
370		-
371		-Schema:
372		- zone: "HealthProbe_Public"
373		- record: "MonthlyDigest"
374		- fields:
375		- - device_id (salted hash)
376		- - month: YYYY-MM
377		- - anomaly_counts: { type: count }
378		- - data_loss_estimate: percent
379		- - sample_types_tracked: [...]
380		-```
	363	+The archive must preserve as much HealthKit information as the API exposes:
	364	+- sample UUID, type, start/end date, value, unit, and metadata
	365	+- source, source revision, bundle identifier, product type, version/build if available
	366	+- device fields exposed by `HKDevice`
	367	+- relationships between workouts, samples, events, and other linked records where available
	368	+- first-seen / last-seen / last-verified observations
	369	+- fingerprints suitable for matching against Apple Health XML exports and extracted backup databases
	370	+
	371	+The archive is selected by data type for performance and privacy, but it is stored in one schema so later analysis can follow relationships between types.
	372	+
	373	+### 6.2 Reports and Point Exports
381	374
382		-### 6.2 Forensic Query Examples
	375	+HealthProbe does not need to optimize for routine complete exports. The local archive is the backup.
	376	+
	377	+Export is scoped to what the user is inspecting:
	378	+- anomaly reports
	379	+- tables of records shown in UI (e.g., “these 1,000 HK records disappeared”)
	380	+- point-in-time manifests and hashes
	381	+- selected record sets needed for external analysis
	382	+
	383	+### 6.3 Forensic Query Examples
383	384
384	385	"Has my step data been compromised?"
385	386	```
	@@ -391,8 +392,8 @@ Schema:
391	392
392	393	"Did iCloud sync break my data?"
393	394	```
394		-1. Correlate anomalies with sync state changes
395		-2. Show timeline: before sync, during sync, after
	395	+1. Correlate anomalies with observed Health/iCloud state changes
	396	+2. Show timeline: before state change, during reconciliation, after
396	397	3. Calculate: samples lost, duplicates introduced
397	398	```
398	399
	@@ -420,27 +421,27 @@ Schema:
420	421	- Anomalies — sortable list by date/severity
421	422	- Snapshots — historical timeline of known-good snapshots
422	423	- Audit Trail — complete immutable log
423		-- Sync Status — current iCloud state & last sync time
	424	+- Archive Status — current local archive health, last verification, selected data types
424	425
425	426	Settings:
426	427	- Check frequency
427	428	- Sample types to track
428	429	- Alert thresholds
429		-- CloudKit sync opt-in
	430	+- Local archive retention and report export options
430	431
431	432	### 7.2 Alerts
432	433
433	434	Push Notifications (opt-in):
434	435	- 🚨 "Critical data loss detected" (> 10% samples missing)
435	436	- ⚠️ "Unexpected historical data inserted" (> 100 samples)
436		-- ℹ️ "Sync completed, 2 duplicates found"
	437	+- ℹ️ "Archive check completed, 2 duplicates found"
437	438
438	439	---
439	440
440	441	## 8. Future Enhancements (Beyond MVP)
441	442
442	443	### 8.1 macOS Companion (Visualization Layer)
443		-- Aggregate data from multiple iOS devices
	444	+- Open and analyze exported HealthProbe reports or archive copies
444	445	- Long-term trend visualization (6-12 month history)
445	446	- Cross-device anomaly correlation
446	447	- Export to reproducible bug reports
	@@ -466,17 +467,17 @@ Schema:
466	467
467	468	### 9.2 Permissions Required
468	469	- `HealthKit` — read-only access to specified types
469		-- `iCloud CloudKit` — optional, for digest sync only
470	470	- `Background Modes` — "Background Fetch"
471	471
472	472	### 9.3 Data Storage
473		-- Local: SwiftData (on-device, encrypted by iOS)
474		-- Optional Cloud: CloudKit (user-controlled, aggregated only)
	473	+- Local Archive Store: canonical HealthKit sample archive (source of truth)
	474	+- SwiftData: derived UI/cache/settings/log/history store
	475	+- No CloudKit sync: HealthProbe data remains local unless the user exports a report or selected record table
475	476
476	477	### 9.4 Performance
477	478	- Query time: < 5 seconds (anchored queries)
478		-- Snapshot size: ≈ 5-10 KB per type per snapshot
479		-- Storage: ≈ 100 MB per year of monitoring (aggregated)
	479	+- Snapshot/index size: ≈ 5-10 KB per type per snapshot in SwiftData
	480	+- Archive storage: depends on selected high-frequency data types; report per-type storage costs in settings
480	481
481	482	---
482	483
	@@ -495,11 +496,14 @@ Schema:
495	496	- ✅ Device model & iOS version (for context)
496	497	- ✅ Anomaly types & severity
497	498
498		-### 10.3 Optional CloudKit Sync
499		-- Only aggregated digests (counts, not data)
500		-- Device identifier is salted & hashed
501		-- Deleted immediately when uninstalled
502		-- Can be disabled in settings
	499	+Local archive:
	500	+- ✅ Per-sample archive for user-selected types, stored on-device and exportable by user
	501	+- ✅ Metadata needed for recognition in Apple Health XML exports, backup database extracts, and future datasets
	502	+
	503	+### 10.3 Cloud Policy
	504	+- No HealthProbe CloudKit/iCloud sync
	505	+- No automatic upload of raw samples, digests, reports, or device fingerprints
	506	+- User-triggered exports are explicit, scoped, and local-file based
503	507
504	508	---
505	509
	@@ -511,7 +515,7 @@ Schema:
511	515	\| Forensic completeness \| % of anomalies with sufficient evidence \| > 95% \|
512	516	\| False positives \| Alerts user shouldn't worry about \| < 5% of total \|
513	517	\| Privacy \| % of users comfortable with data practices \| > 90% \|
514		-\| Performance \| Background sync battery impact \| < 2% drain/day \|
	518	+\| Performance \| Background capture battery impact \| < 2% drain/day \|
515	519	\| Adoption \| Users can reproduce bugs with HealthProbe data \| High relevance in Apple feedback \|
516	520
517	521	---
	@@ -521,7 +525,6 @@ Schema:
521	525	- [DearApple Issue #001](https://github.com/overbog/dear-apple/issues/0001-apple-health-mass-data-loss.md) — Sept 2025 mass data loss
522	526	- [Apple HealthKit Documentation](https://developer.apple.com/documentation/healthkit/)
523	527	- [HKAnchoredObjectQuery](https://developer.apple.com/documentation/healthkit/hkanchoredrobjectquery) — Efficient incremental queries
524		-- [CloudKit Best Practices](https://developer.apple.com/documentation/cloudkit/)
525	528
526	529	---
527	530

	@@ -14,7 +14,7 @@ The following rules apply to **all code, logs, examples, tests, and documentatio
14	14	- No personal data — no names, email addresses, phone numbers, or dates of birth
15	15	- No device identifiers — no UDIDs, serial numbers, advertising IDs, or device names
16	16	- No account identifiers — no Apple IDs, iCloud account info, or CloudKit record IDs
17		-- No raw health values — no actual health records, measurements, or workout data
	17	+- No raw health values in the repository — do not include real health records, measurements, or workouts in code, tests, logs, examples, or documentation. The app may optionally store a user's raw samples locally on-device for forensic backup, but nothing real belongs in this repo.
18	18	- No location data — no GPS coordinates or location history
19	19	- No recognizable patterns — no logs or exports where combining fields could identify a person or device
20	20
	@@ -123,7 +123,22 @@ class HealthKitObserver {
123	123
124	124	---
125	125
126		-## 2. Data Model Implementation (SwiftData)
	126	+## 2. Storage Implementation
	127	+
	128	+HealthProbe uses two storage layers:
	129	+
	130	+1. Local Archive Store (source of truth)
	131	+ - Stores canonical HealthKit samples and all metadata exposed by the API
	132	+ - Uses one schema for all selected data types, so workouts, samples, sources, devices, and metadata can be related later
	133	+ - Maintains `firstSeen`, `lastSeen`, `lastVerified`, strict/semantic/fuzzy fingerprints, and integrity hashes
	134	+ - Should be implemented with an explicit local database/archive format (not SwiftData model graphs for millions of samples)
	135	+
	136	+2. SwiftData UI Store (derived/cache layer)
	137	+ - Stores settings, logs, import/check history, anomaly summaries, and precomputed values used by charts
	138	+ - Can be rebuilt from the archive store
	139	+ - Must not be treated as the only forensic copy
	140	+
	141	+### 2.1 SwiftData UI Models
127	142
128	143	```swift
129	144	import SwiftData
	@@ -225,7 +240,7 @@ final class DetectedAnomaly {
225	240	}
226	241
227	242	@Model
228		-final class SyncStateChange {
	243	+final class ContextStateChange {
229	244	@Attribute(.unique) var id: String = UUID().uuidString
230	245	var timestamp: Date
231	246	var previousState: String // "local_only", "icloud_enabled", "icloud_sync_active"
	@@ -247,7 +262,7 @@ func createModelContainer() throws -> ModelContainer {
247	262	HealthSnapshot.self,
248	263	AuditTrailEntry.self,
249	264	DetectedAnomaly.self,
250		- SyncStateChange.self,
	265	+ ContextStateChange.self,
251	266	])
252	267
253	268	let modelConfiguration = ModelConfiguration(
	@@ -260,6 +275,29 @@ func createModelContainer() throws -> ModelContainer {
260	275	}
261	276	```
262	277
	278	+### 2.2 Local Archive Store Contract
	279	+
	280	+The archive store should expose a small service interface rather than leaking SQL/archive details into UI code:
	281	+
	282	+```swift
	283	+protocol HealthArchiveStore {
	284	+ func upsertSamples(_ samples: [HKSample], observedAt: Date) async throws
	285	+ func markVerification(sampleType: HKSampleType, verifiedAt: Date) async throws
	286	+ func recordDisappearance(fingerprint: String, observedMissingAt: Date) async throws
	287	+ func records(for reportID: String) async throws -> [ArchivedHealthRecord]
	288	+ func exportReport(_ reportID: String) async throws -> URL
	289	+}
	290	+```
	291	+
	292	+Archive rows should preserve:
	293	+- HealthKit UUID where exposed
	294	+- type identifier, start/end date, value, unit
	295	+- source, source revision, bundle identifier, version/build/product type where available
	296	+- `HKDevice` fields exposed by HealthKit
	297	+- full metadata dictionary as structured data
	298	+- relationship keys for workouts, events, and related samples where available
	299	+- fingerprints for matching records across HealthProbe, Apple Health XML exports, and backup database extracts
	300	+
263	301	---
264	302
265	303	## 3. Anomaly Detection Implementation
	@@ -426,39 +464,41 @@ class AnomalyDetector {
426	464
427	465	---
428	466
429		-## 4. Sync Monitoring (Background Thread)
	467	+## 4. Context Monitoring (Background Thread)
	468	+
	469	+HealthProbe does not sync its own database through iCloud/CloudKit. This service only logs Health/iCloud state as context for later forensic correlation.
430	470
431	471	```swift
432		-class SyncMonitor {
	472	+class ContextMonitor {
433	473	private let modelContext: ModelContext
434	474	private let queue = DispatchQueue(label: "com.healthprobe.sync-monitor", qos: .background)
435	475
436		- private var previousSyncState: String = "unknown"
	476	+ private var previousHealthCloudState: String = "unknown"
437	477
438	478	func startMonitoring() {
439	479	queue.async {
440		- self.monitorSyncState()
	480	+ self.monitorContext()
441	481	}
442	482	}
443	483
444		- private func monitorSyncState() {
	484	+ private func monitorContext() {
445	485	// Check iCloud state
446	486	let iCloudToken = FileManager.default.ubiquityIdentityToken
447	487	let currentState = iCloudToken != nil ? "icloud_enabled" : "local_only"
448	488
449		- if currentState != previousSyncState {
450		- logSyncStateChange(from: previousSyncState, to: currentState)
451		- previousSyncState = currentState
	489	+ if currentState != previousHealthCloudState {
	490	+ logContextChange(from: previousHealthCloudState, to: currentState)
	491	+ previousHealthCloudState = currentState
452	492
453		- // Trigger re-snapshot on state change
	493	+ // Schedule archive verification on state change
454	494	DispatchQueue.main.async {
455		- NotificationCenter.default.post(name: NSNotification.Name("SyncStateChanged"), object: nil)
	495	+ NotificationCenter.default.post(name: NSNotification.Name("HealthContextChanged"), object: nil)
456	496	}
457	497	}
458	498	}
459	499
460		- private func logSyncStateChange(from: String, to: String) {
461		- let change = SyncStateChange(
	500	+ private func logContextChange(from: String, to: String) {
	501	+ let change = ContextStateChange(
462	502	timestamp: Date(),
463	503	previousState: from,
464	504	newState: to,
	@@ -471,14 +511,14 @@ class SyncMonitor {
471	511
472	512	let auditEntry = AuditTrailEntry(
473	513	timestamp: Date(),
474		- eventType: "sync_state_change",
475		- message: "Sync state: \(from) → \(to)",
	514	+ eventType: "health_context_change",
	515	+ message: "Health cloud context: \(from) → \(to)",
476	516	context: ["previous": from, "current": to]
477	517	)
478	518	modelContext.insert(auditEntry)
479	519	try modelContext.save()
480	520	} catch {
481		- print("Error logging sync state change: \(error)")
	521	+ print("Error logging context change: \(error)")
482	522	}
483	523	}
484	524	}
	@@ -492,14 +532,14 @@ class SyncMonitor {
492	532	@main
493	533	struct HealthProbeApp: App {
494	534	@StateObject private var healthKitManager = HealthKitManager.shared
495		- @StateObject private var syncMonitor: SyncMonitor
	535	+ @StateObject private var contextMonitor: ContextMonitor
496	536	let modelContainer: ModelContainer
497	537
498	538	init() {
499	539	do {
500	540	modelContainer = try createModelContainer()
501	541	let context = ModelContext(modelContainer)
502		- _syncMonitor = StateObject(wrappedValue: SyncMonitor(modelContext: context))
	542	+ _contextMonitor = StateObject(wrappedValue: ContextMonitor(modelContext: context))
503	543	} catch {
504	544	fatalError("Could not initialize model container: \(error)")
505	545	}
	@@ -513,9 +553,8 @@ struct HealthProbeApp: App {
513	553	// Request HealthKit permissions
514	554	healthKitManager.requestAuthorization { success, error in
515	555	if success {
516		- // Start monitoring
517		- syncMonitor.startMonitoring()
518		- // Initial snapshot
	556	+ // Start context monitoring and archive capture
	557	+ contextMonitor.startMonitoring()
519	558	captureInitialSnapshot()
520	559	}
521	560	}
	@@ -565,8 +604,9 @@ class AnomalyDetectorTests: XCTestCase {
565	604
566	605	### Integration Tests
567	606	- ✅ HealthKit query performance (anchor efficiency)
568		-- ✅ SwiftData persistence and recovery
569		-- ✅ Background sync monitoring accuracy
	607	+- ✅ Local archive persistence and recovery
	608	+- ✅ SwiftData cache rebuild from archive
	609	+- ✅ Background context monitoring accuracy
570	610	- ✅ Anomaly detection on real HealthKit data
571	611
572	612	---
	@@ -577,8 +617,9 @@ class AnomalyDetectorTests: XCTestCase {
577	617	\|-----------\|--------\|-------\|
578	618	\| Anchored query \| < 5 sec \| Background, user perceives delay > 2s \|
579	619	\| Anomaly detection \| < 2 sec \| Should not block UI \|
580		-\| Snapshot creation \| < 1 sec \| Can run on main thread \|
581		-\| Background sync \| < 30 sec \| iOS allows 30 min for background fetch \|
	620	+\| SwiftData cache update \| < 1 sec \| Can run on main thread only after archive work completes \|
	621	+\| Archive write \| Background \| Stream large imports; never build full high-frequency datasets in memory \|
	622	+\| Background check \| < 30 sec \| iOS allows 30 min for background fetch \|
582	623
583	624	---
584	625
	@@ -586,8 +627,8 @@ class AnomalyDetectorTests: XCTestCase {
586	627
587	628	- [ ] HealthKit read permissions declared in Info.plist
588	629	- [ ] Background Modes enabled ("Background Fetch")
589		-- [ ] Push notification entitlement (if using remote notifications)
590	630	- [ ] SwiftData model migrations tested
	631	+- [ ] Local archive schema migrations tested
591	632	- [ ] Privacy Policy updated (what data is collected)
592	633	- [ ] Accessibility review (VoiceOver, Dynamic Type)
593	634

	@@ -201,7 +201,7 @@ Before release, verify:
201	201
202	202	### Privacy Compliance
203	203	- [ ] No raw health sample data in examples
204		-- [ ] CloudKit sync described as optional, not default
	204	+- [ ] HealthProbe CloudKit/iCloud sync is not described as a product goal
205	205	- [ ] User consent documented
206	206	- [ ] Data retention policy clear
207	207	- [ ] No tracking/analytics hidden in code

	@@ -25,9 +25,10 @@
25	25	\| Core Architecture \| 📋 Designed \| See "Complete Specification" \|
26	26	\| HealthKit Integration \| ⏳ Pending \| Implement anchored queries, observer queries \|
27	27	\| Anomaly Detection \| 📋 Designed \| Logic documented, pending implementation \|
28		-\| Sync Monitoring \| 📋 Designed \| Background thread model defined \|
	28	+\| Sync Context Logging \| 📋 Designed \| Log Health/iCloud state as forensic context; do not sync HealthProbe data via iCloud \|
29	29	\| UI Dashboard \| ⏳ Pending \| Wireframes in Complete Specification \|
30		-\| Data Export \| 📋 Designed \| Format specs ready \|
	30	+\| Local Archive Store \| 📋 Designed \| Robust on-device archive is the source of truth \|
	31	+\| Reports & Point Exports \| 📋 Designed \| Export only selected reports/record tables, not a complete routine dump \|
31	32	\| macOS Companion \| 🔄 Future \| Post-MVP enhancement \|
32	33
33	34	---
	@@ -59,10 +60,10 @@ See Complete Specification § 2 for detailed observed cases and forensic imp
59	60	- Duplicate fingerprinting
60	61	- Divergence trend analysis
61	62
62		-3. Implement Sync Monitoring (`Sources/SyncMonitor.swift`)
63		- - Background thread that tracks iCloud state changes
64		- - Document sync events in audit trail
65		- - Trigger re-baselining on sync state transitions
	63	+3. Implement Local Archive Store (`Sources/ArchiveStore.swift`)
	64	+ - Single robust local database for all archived samples
	65	+ - Preserve cross-type relationships, sources, devices, metadata, and fingerprints
	66	+ - Keep SwiftData as UI/cache/settings/history only
66	67
67	68	4. Create UI Dashboard (`Views/HealthStatusView.swift`)
68	69	- Show current health status
	@@ -78,10 +79,11 @@ See Complete Specification § 2 for detailed observed cases and forensic imp
78	79	\|----------\|-----------\|
79	80	\| Read-only + HealthKit \| Never modify health data; pure observation only \|
80	81	\| Local-first storage \| Full functionality without internet; privacy-first \|
81		-\| SwiftData \| Efficient local persistence, encrypted by iOS \|
	82	+\| Archive DB as truth \| Store HealthKit samples and metadata in a robust local database, not split per data type \|
	83	+\| SwiftData as UI cache \| Keep precomputed values, settings, logs, and history for visualization only \|
82	84	\| Anchored queries \| Minimize HealthKit load, reduce battery impact \|
83		-\| Separate sync thread \| Non-blocking, responsive UI during background work \|
84		-\| Aggregated digests only \| CloudKit sync respects privacy, no raw samples exported \|
	85	+\| No HealthProbe iCloud sync \| Device HealthKit databases evolve independently; CloudKit sync adds complexity without proven forensic benefit \|
	86	+\| Selective forensic capture \| When Health/iCloud rewrites or downsamples historical data, counts alone are insufficient; HealthProbe archives complete samples for selected types in one local store \|
85	87
86	88	---
87	89
	@@ -91,6 +93,13 @@ See Complete Specification § 2 for detailed observed cases and forensic imp
91	93	- Concrete cases from DearApple
92	94	- Full technical architecture
93	95	- MVP feature scope + future roadmap
	96	+- v1.1 — 2026-05-17 — Objective extension (post-findings)
	97	+ - New observed behavior: Apple-side consolidation/downsampling can rewrite historical samples (not just add/delete)
	98	+ - HealthProbe scope extended: from “counter of records” → “forensic backup agent” (local-only archives for selected types)
	99	+- v1.2 — 2026-05-18 — Storage direction update
	100	+ - Robust single local archive store becomes the source of truth
	101	+ - SwiftData is limited to precomputed UI data, settings, logs, and history
	102	+ - CloudKit/iCloud sync removed from product goals; reports and point exports replace routine complete export ambitions
94	103
95	104	---
96	105

	@@ -21,12 +21,12 @@ struct HealthProbeApp: App {
21	21	.modelContainer(sharedModelContainer)
22	22	}
23	23
24		- // Two separate ModelConfiguration instances:
25		- // cloudConfig - audit data
26		- // localConfig - local-only settings and operation metadata
	24	+ // Two local ModelConfiguration instances:
	25	+ // uiCacheConfig - derived audit/index data for UI
	26	+ // localConfig - local-only settings and operation metadata
27	27	//
28		- // ⚠️ DeviceProfile is kept local-only (not synced to CloudKit) since it's device-specific
29		- // cosmetic data (name, color tag) that should not cross devices.
	28	+ // SwiftData is not the forensic source of truth. Complete HealthKit samples
	29	+ // belong in the local archive store; these rows must remain rebuildable.
30	30	private static func createModelContainer() throws -> ModelContainer {
31	31	let fullSchema = Schema([
32	32	HealthSnapshot.self, TypeCount.self, YearlyCount.self, HealthRecord.self, TypeDistributionBin.self,
	@@ -36,18 +36,18 @@ struct HealthProbeApp: App {
36	36
37	37	let appSupportURL = URL.applicationSupportDirectory
38	38	try FileManager.default.createDirectory(at: appSupportURL, withIntermediateDirectories: true)
39		- let cloudStoreURL = appSupportURL.appending(path: "HealthProbeRecords.store")
	39	+ let uiCacheStoreURL = appSupportURL.appending(path: "HealthProbeRecords.store")
40	40
41		- let cloudKitModels = Schema([
	41	+ let uiCacheModels = Schema([
42	42	HealthSnapshot.self, TypeCount.self, YearlyCount.self, HealthRecord.self, TypeDistributionBin.self,
43	43	SnapshotDelta.self, TypeDelta.self, AnomalyRecord.self,
44	44	])
45	45	let localModels = Schema([OperationLog.self, DeviceProfile.self, MetricTimeoutProfile.self])
46	46
47		- let cloudConfig = ModelConfiguration(
48		- "cloud",
49		- schema: cloudKitModels,
50		- url: cloudStoreURL,
	47	+ let uiCacheConfig = ModelConfiguration(
	48	+ "ui-cache",
	49	+ schema: uiCacheModels,
	50	+ url: uiCacheStoreURL,
51	51	cloudKitDatabase: .none
52	52	)
53	53	let localConfig = ModelConfiguration(
	@@ -58,10 +58,10 @@ struct HealthProbeApp: App {
58	58	)
59	59
60	60	do {
61		- return try ModelContainer(for: fullSchema, configurations: [cloudConfig, localConfig])
	61	+ return try ModelContainer(for: fullSchema, configurations: [uiCacheConfig, localConfig])
62	62	} catch {
63	63	let candidates: [URL] = [
64		- cloudStoreURL,
	64	+ uiCacheStoreURL,
65	65	appSupportURL.appending(path: "HealthProbeRecords.store.shm"),
66	66	appSupportURL.appending(path: "HealthProbeRecords.store.wal"),
67	67	appSupportURL.appending(path: "HealthProbeCloud.store"),
	@@ -72,7 +72,7 @@ struct HealthProbeApp: App {
72	72	appSupportURL.appending(path: "HealthProbeLocal.store.wal"),
73	73	]
74	74	for url in candidates { try? FileManager.default.removeItem(at: url) }
75		- return try ModelContainer(for: fullSchema, configurations: [cloudConfig, localConfig])
	75	+ return try ModelContainer(for: fullSchema, configurations: [uiCacheConfig, localConfig])
76	76	}
77	77	}
78	78	}

	@@ -14,7 +14,6 @@ import SwiftData
14	14	var changedMetricCount: Int = 0
15	15	var appearedMetricCount: Int = 0
16	16	var disappearedMetricCount: Int = 0
17		- var isCloudKitImported: Bool = false
18	17	@Relationship(deleteRule: .cascade, inverse: \TypeDelta.delta)
19	18	var typeDeltas: [TypeDelta]? = []
20	19

	@@ -13,7 +13,6 @@ import SwiftData
13	13	var transitionRaw: String = "unchanged"
14	14	var reasonRaw: String = "normal"
15	15	var yearlyCountNote: String = ""
16		- var isCloudKitImported: Bool = false
17	16	var delta: SnapshotDelta?
18	17
19	18	init(typeIdentifier: String, displayName: String) {

	@@ -7,14 +7,11 @@ enum IntegrityService {
7	7	case checksumMismatch(snapshotID: UUID, expected: String, actual: String)
8	8	case missingDelta(fromID: UUID, toID: UUID)
9	9	case corrupted(snapshotID: UUID, reason: String)
10		- case pendingSync(deltaID: UUID)
11	10	}
12	11
13	12	// Strict mode: used by chain traversal and analysis.
14	13	// Recomputes checksum from TypeCounts; compares with stored delta.checksumAfter.
15	14	// Returns .valid only if they match exactly.
16		- // Exception: if delta.isCloudKitImported == true AND delta.typeDeltas is nil/empty,
17		- // returns .pendingSync(deltaID:) — UI shows "Syncing…".
18	15	static func validate(snapshot: HealthSnapshot, delta: SnapshotDelta?) -> ValidationResult {
19	16	guard let delta else {
20	17	guard snapshot.isChainStart else {
	@@ -23,11 +20,6 @@ enum IntegrityService {
23	20	return .valid
24	21	}
25	22
26		- // CloudKit pending: delta arrived before typeDeltas
27		- if delta.isCloudKitImported && (delta.typeDeltas?.isEmpty ?? true) {
28		- return .pendingSync(deltaID: delta.id)
29		- }
30		-
31	23	let actual = HashService.snapshotChecksum(typeCounts: snapshot.typeCounts ?? [])
32	24	guard actual == delta.checksumAfter else {
33	25	return .checksumMismatch(snapshotID: snapshot.id, expected: delta.checksumAfter, actual: actual)
	@@ -71,9 +63,6 @@ enum IntegrityService {
71	63	switch result {
72	64	case .valid:
73	65	break
74		- case .pendingSync:
75		- // CloudKit pending — emit but continue traversal
76		- results.append(result)
77	66	case .checksumMismatch, .missingDelta, .corrupted:
78	67	// Strict mode — stop immediately on any error
79	68	results.append(result)

	@@ -22,7 +22,7 @@ enum SnapshotLifecycleService {
22	22
23	23	let integrityResult = IntegrityService.validate(snapshot: snapshot, delta: incomingDelta)
24	24	switch integrityResult {
25		- case .valid, .pendingSync:
	25	+ case .valid:
26	26	break
27	27	case .checksumMismatch(_, let expected, let actual):
28	28	willBreakChain = true

	@@ -11,8 +11,8 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
11	11	✅ AnomalyType.swift — All anomaly types + Severity + TypeTransition + TypeDeltaReason enums
12	12	✅ HealthSnapshot.swift — Chain metadata, quality, trigger context, registry fingerprinting, timezone context
13	13	✅ TypeCount.swift — Count with hash, date range, quality, yearly counts with cascade relationship
14		-✅ SnapshotDelta.swift — Delta with checksums, CloudKit import flag, cascade relationship to TypeDeltas
15		-✅ TypeDelta.swift — Per-type delta with transition, reason, quality before/after, yearly count note, CloudKit import flag
	14	+✅ SnapshotDelta.swift — Local delta with checksums and cascade relationship to TypeDeltas
	15	+✅ TypeDelta.swift — Per-type local delta with transition, reason, quality before/after, yearly count note
16	16	✅ AnomalyRecord.swift — Anomaly record with deltaID set structurally by detector, never by caller
17	17	✅ OperationLog.swift — Audit trail for destructive operations with JSON-encoded affected snapshot IDs
18	18	✅ YearlyCount.swift — Per-year sample counts with approximation flag
	@@ -74,12 +74,10 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
74	74	- `validate()` — strict mode:
75	75	- Recomputes checksum from TypeCounts
76	76	- Compares with delta.checksumAfter
77		- - Returns .pendingSync if delta.isCloudKitImported && typeDeltas empty (not an error)
78	77	- Returns .valid or .checksumMismatch / .missingDelta / .corrupted
79	78	- `validateChain()` — walk backwards from latest via previousSnapshotID:
80	79	- Fork detection: asserts no duplicate previousSnapshotID (returns .corrupted immediately)
81	80	- Stops at first mismatch (no auto-repair, no skips)
82		- - Emits .pendingSync for CloudKit-pending nodes, continues traversal
83	81	- Quality aggregation: loading > unauthorized (only if ALL) > partial (any failed/unauthorized) > complete
84	82
85	83	#### Step 10: SnapshotLifecycleService ✅
	@@ -91,13 +89,13 @@ HealthProbe's comprehensive snapshot + delta system has been implemented accordi
91	89	- OperationLog: always written atomically with deletive changes
92	90	- Post-save verification: re-fetches log by ID, recovery re-insert if missing, logs critical error
93	91
94		-#### Step 12: CloudKitSyncService ✅
95		-- `checkAvailability()` — async account status check
	92	+#### Step 12: Local-only storage refactor ✅
	93	+- Removed CloudKitSyncService and CloudKit-pending chain states
96	94	- ModelContainer split:
97		- - cloudKitConfig: HealthSnapshot, TypeCount, YearlyCount, SnapshotDelta, TypeDelta, AnomalyRecord (synced to iCloud.ro.xdev.healthprobe)
98		- - localConfig: OperationLog, DeviceProfile (local-only, never synced)
99		-- Simulator uses in-memory store; device uses CloudKit private database
100		-- Schema migration recovery: removes both stores and retries once on failure
	95	+ - uiCacheConfig: HealthSnapshot, TypeCount, YearlyCount, SnapshotDelta, TypeDelta, AnomalyRecord (derived local UI/index data)
	96	+ - localConfig: OperationLog, DeviceProfile, MetricTimeoutProfile (local-only settings and operation metadata)
	97	+- Added `HealthArchiveStore` protocol for the single local archive store source of truth
	98	+- Schema migration recovery: removes legacy SwiftData stores and retries once on failure
101	99
102	100	### UI (Step 13)
103	101
	@@ -154,12 +152,12 @@ These tests should be run to ensure all backend functionality is correct:
154	152
155	153	### Advanced Features (11-20)
156	154	- [ ] 11. DB reset with Keychain survival → same deviceID, isChainStart=true, recoveredDeviceID=true
157		-- [ ] 12. CloudKit unavailable → app functions (local-only fallback)
	155	+- [ ] 12. Local-only launch → app functions without iCloud/CloudKit entitlements
158	156	- [ ] 13. Observer debounce → 10 rapid callbacks = exactly 1 snapshot (triggerReason=observerCallback)
159	157	- [ ] 14. Unsupported type → TypeCount(count=-1, quality=.failed, isUnsupported=true), "Unsupported" UI
160	158	- [ ] 15. YearlyCount timezone → Calendar.current used, isApproximate=true if bucket > day
161	159	- [ ] 16. Delta merge with unavailable counts → merged countDelta=0, impaired reason preserved
162		-- [ ] 17. CloudKit pending relationships → TypeDelta(delta=nil, isCloudKitImported=true) shows "Syncing…"
	160	+- [ ] 17. Missing local delta/typeDeltas → integrity validation surfaces the fault, never hides it as sync latency
163	161	- [ ] 18. First auth after full deny (quality gate) → no anomalies, current.isPostRestore=true, isPostRestoreInferred=true
164	162	- [ ] 19. Chain fork → validateChain() returns .corrupted(reason: "chain fork detected"), stops
165	163	- [ ] 20. disappeared→appeared merge with -1 source → merged countDelta=0, reason != .normal
	@@ -169,7 +167,7 @@ These tests should be run to ensure all backend functionality is correct:
169	167	- [ ] 22. Debounce + manual overlap → no observer snapshot if manual created during debounce
170	168	- [ ] 23. completionHandler unconditional → called via defer, never gated on scheduling success
171	169	- [ ] 24. isPostRestore forwarding → suppression forwarded past low-quality, consumed on next .complete
172		-- [ ] 25. CloudKit pending delta → validateChain() returns .pendingSync, UI shows "Syncing…"
	170	+- [ ] 25. Missing delta → validateChain() returns .missingDelta and stops
173	171	- [ ] 26. OperationLog verification → recovery re-insert if missing after save, log critical error
174	172
175	173	### Coherence & Edge Cases (27-32)
	@@ -198,11 +196,10 @@ These tests should be run to ensure all backend functionality is correct:
198	196	- Fork detection prevents chain divergence (asserts no duplicate previousSnapshotID)
199	197	- Checksum validation ensures data wasn't corrupted between snapshots
200	198
201		-### CloudKit Readiness
202		-- isCloudKitImported distinguishes sync latency from corruption
203		- - Pending: delta=nil, isCloudKitImported=true → "Syncing…" in UI
204		- - Bug: delta=nil, isCloudKitImported=false → error log + validation stops
205		-- ModelConfiguration split keeps OperationLog local-only (audit trail never crosses devices)
	199	+### Local Archive Direction
	200	+- CloudKit/iCloud sync is not a product goal
	201	+- SwiftData rows are derived UI/index data and must be rebuildable from the local archive store
	202	+- Missing deltas or type deltas are treated as local integrity faults, not remote sync latency
206	203
207	204	### Observability
208	205	- Reason priority makes anomaly suppression deterministic
	@@ -215,8 +212,8 @@ These tests should be run to ensure all backend functionality is correct:
215	212
216	213	1. Hash covers only count + date range, not distribution (silentReplacement is best-effort)
217	214	2. YearlyCount precision requires daily bucket granularity (noted if isApproximate)
218		-3. No server-side conflict resolution for CloudKit (storage-only design, no ordering)
219		-4. No transaction-level consistency for cross-device chain reconstruction (local chains validated, CloudKit pending states flagged)
	215	+3. Local archive store implementation is still pending (protocol boundary exists, SQLite/archive schema still needed)
	216	+4. No automatic cross-device reconstruction; cross-device analysis is future macOS/report work
220	217
221	218	## Next Steps
222	219
	@@ -224,7 +221,7 @@ These tests should be run to ensure all backend functionality is correct:
224	221	1. Run all 32 verification checks against real HealthKit data
225	222	2. Create unit tests for delta merge, reason priority, anomaly detection
226	223	3. Test observer callback debounce with real HKObserverQuery
227		-4. Validate CloudKit sync with simulator and device
	224	+4. Implement the local archive store behind `HealthArchiveStore`
228	225
229	226	### Post-MVP
230	227	1. Integrate actual BGTask expiration guard for observer snapshots (capture partial results)
	@@ -234,7 +231,7 @@ These tests should be run to ensure all backend functionality is correct:
234	231
235	232	---
236	233
237		-Built with: SwiftUI, SwiftData, HealthKit, CloudKit, CryptoKit
	234	+Built with: SwiftUI, SwiftData, HealthKit, CryptoKit
238	235	Minimum iOS: 17.0
239	236	Target iOS: 26.4
240	237	Swift Version: 5.9+