# HealthProbe – Complete Specification & Motivations

**Version:** 1.0  
**Status:** MVP (iOS monitoring agent)  
**Last Updated:** 2026-05-01  

---

## 1. Executive Summary

HealthProbe is an **audit and integrity monitoring tool for Apple HealthKit**, designed to detect and document anomalies in health data that would otherwise go unnoticed. It serves as a **local sentinel** — read-only observation with forensic-grade logging for later analysis.

**Core Problem:** Apple Health data loss events (confirmed Sept 2025 incident, ongoing sporadic reports) lack detective mechanisms. Users do not know when data has been lost, corrupted, or silently modified.

**Solution:** HealthProbe monitors HealthKit in real-time and maintains a tamper-proof audit trail, enabling post-incident forensic analysis and pattern detection.

---

## 2. Motivations & Concrete Observed Cases

### 2.1 The September 2025 Mass Data Loss Event

**What happened:**
- Large-scale loss of Apple Health records reported across multiple devices and iOS versions
- Timeframe: September 2025 (correlated with iOS 26 release)
- Suspected triggers:
  - Device migration (iCloud sync state transitions)
  - OS upgrade/downgrade cycles
  - Backup restore operations
  - HealthKit database re-indexing
  - iCloud sync divergence

**Why undetected:**
- No notification from Apple Health
- Users discover loss retrospectively (weeks/months later)
- No audit trail to identify exactly *when* or *what* was lost
- No differentiation between: user deletion, sync loss, corruption, or app bug

**HealthProbe's answer:**
- Continuous monitoring detects *first occurrence* of loss
- Timestamped snapshots enable forensic reconstruction
- Pattern detection identifies trends (gradual loss vs. sudden wipe)
- Allows users to file reproducible bug reports with evidence

### 2.2 Observed Anomalies

#### A. Historical Insertions (Backdated Data)
**Pattern:** HealthKit receives samples with `startDate` far in the past, but `firstSeen` ≈ now
- **Examples:**
  - Workout from Jan 2023 suddenly appears in Feb 2026
  - Step count "corrected" retroactively without user action
  - Heart rate baseline recalibration affecting past months
  
**Root cause theories:**
  - iCloud sync restoring from outdated backup
  - Third-party fitness app injecting historical reconstructions
  - HealthKit recovery logic applying retroactive corrections
  - Cross-device sync desynchronization

**Detection method:** Anchored queries + timestamp comparison

---

#### B. Silent Deletions
**Pattern:** Samples present in previous snapshot, absent in current, no `HKDeletedObject` notification
- **Examples:**
  - 2-week gap of step data (no deletion events logged)
  - Entire workout history from old iPhone missing post-restore
  - Selective loss (e.g., only workouts, heart rate preserved)

**Root cause theories:**
  - Incomplete restore from backup
  - Selective iCloud sync pruning based on storage limits
  - Corrupted local database during indexing
  - Race condition during multi-device sync

**Detection method:** Snapshot comparison + gap detection

---

#### C. Duplicate Records
**Pattern:** Identical samples (same type, time, value) appearing multiple times
- **Examples:**
  - Duplicate step counts from watch syncing (15 min apart)
  - Duplicate workouts after iCloud re-sync
  - Conflicting HR readings within 30-second window

**Root cause theories:**
  - Multi-device sync collision (watch + phone + iPad)
  - Retry logic without deduplication
  - Backup restore merging with live data

**Detection method:** Fingerprinting (type + date + value + source)

---

#### D. Divergence Trends
**Pattern:** Measurable drift in aggregated metrics over time
- **Examples:**
  - Active energy expenditure trending down 30% without behavior change
  - Sleep records shifting systematically earlier/later
  - Heart rate variability calculation method changing unexpectedly

**Root cause theories:**
  - Algorithm updates not backfilled uniformly
  - Device calibration drift
  - Source priority shifting (watch → phone)
  - Health app recalculation without user visibility

**Detection method:** Time-series aggregation + statistical outlier detection

---

### 2.3 Why This Matters

| Concern | Impact | HealthProbe Role |
|---------|--------|-----------------|
| **Data loss undetected** | Users lose personal health history with no notification | Immediate detection & alert |
| **No forensic trail** | Impossible to reproduce for bug reports | Audit trail enables Apple debugging |
| **Blame uncertainty** | "Is it sync? Backup? A bug?" | Precise classification of anomaly type |
| **Third-party apps** | Apps assume data is trustworthy, may make wrong decisions | Detect corruption before downstream use |
| **Privacy of monitoring** | Users fear data exfiltration by health apps | Local-only observation, no cloud upload |

---

## 3. Core Architecture

### 3.1 Design Principles

1. **Read-only operations** (never modify HealthKit data)
2. **Local-first** (full functionality without network)
3. **Incremental queries** (efficient, avoid repeating work)
4. **Minimal exfiltration** (only digests, never raw samples)
5. **Auditability** (every observation logged, timestamped, reproducible)
6. **Privacy by default** (all aggregated, no PII stored)

### 3.2 Threading Model

```
┌─────────────────────────────────────────┐
│  Main Thread (UI)                       │
│  - Display current health status        │
│  - Show alerts & anomalies              │
│  - User interaction                     │
└──────────────┬──────────────────────────┘
               │
               ├─ Delegate query results
               │
┌──────────────▼──────────────────────────┐
│  Background Queue (HealthKit Queries)   │
│  - HKAnchoredObjectQuery (efficient)    │
│  - HKObserverQuery (reactive)           │
│  - Snapshot comparisons                 │
│  - Anomaly detection logic              │
└──────────────┬──────────────────────────┘
               │
               ├─ Write detected anomalies
               │
┌──────────────▼──────────────────────────┐
│  Local Storage (SwiftData)              │
│  - Audit trail (immutable append-only)  │
│  - Snapshots (for forensics)            │
│  - Anomaly records                      │
│  - Metadata (versions, migrations)      │
└─────────────────────────────────────────┘
```

### 3.3 Data Model (SwiftData)

```swift
// Immutable audit log entry
@Model
final class AuditEntry {
    var timestamp: Date
    var eventType: String  // "snapshot", "deletion", "insertion", etc.
    var samples: [HealthSample]
    var anomalies: [AnomalyReport]
    var deviceInfo: DeviceMetadata
}

// Snapshot for forensic comparison
@Model
final class HealthSnapshot {
    var capturedAt: Date
    var sampleType: String  // "HKWorkout", "HKQuantity:HeartRate", etc.
    var sourceDevice: String
    var recordCount: Int
    var checksum: String  // MD5 of aggregated data
    var digest: [String: Int]  // { "source": count, ... }
}

// Detected anomaly
@Model
final class AnomalyReport {
    var type: String  // "historical_insertion", "deletion", "duplicate", "divergence"
    var severity: String  // "info", "warning", "critical"
    var detectedAt: Date
    var description: String
    var evidence: [String: Any]  // forensic data
    var requiresAttention: Bool
}
```

---

## 4. Monitoring Features (MVP)

### 4.1 Incremental Change Detection

**Using `HKAnchoredObjectQuery`:**
```
Query pattern:
├─ Initial query: anchor = 0 → captures all existing data
├─ Store anchor locally
├─ Periodic queries: anchor = stored → captures only new/modified samples
└─ Update anchor → efficient incremental updates
```

**What triggers a query:**
- App launch
- Background refresh (iOS allows periodic background queries)
- User manually triggers "Check Now"
- Every 12-24 hours (configurable)

### 4.2 Tracked Sample Types (Extensible)

| Type | Why Monitored | Anomaly Signal |
|------|---------------|----------------|
| **Workouts** | High-value data, often synced from watch | Historical insertions, duplicates |
| **Heart Rate** | Continuous stream, high modification risk | Gaps, divergence |
| **Activity Summary** | Auto-computed, depends on other types | Recalculation without notice |
| **Steps** | Cross-device (watch/phone), sync-heavy | Duplicate from retries |
| **Sleep** | Frequently "corrected" post-recording | Backdated entries, loss |
| **Blood Pressure** | Manual entry, sync state-dependent | Divergence trends |
| **Audio Exposure** | Often device-specific | Selective loss |

### 4.3 Anomaly Detection Logic

#### A. Historical Insertion Detection
```
For each sample:
  Δt = now - startDate  (age of sample)
  Δt_observed = now - firstSeen  (how long we've known about it)
  
  IF Δt >> Δt_observed (e.g., Δt ≈ 6 months, Δt_observed ≈ 5 minutes):
    → Flag as "historical insertion"
    → Severity: MEDIUM (might be legitimate correction)
```

#### B. Deletion Detection
```
Current snapshot: S_now
Previous snapshot: S_prev

Missing = S_prev - S_now
  (samples present before, absent now)

IF |Missing| > 0 AND no HKDeletedObject notification:
  → Flag as "silent deletion"
  → Severity: CRITICAL
  → Record gap_duration = time between last observation and absence
```

#### C. Duplicate Detection
```
Fingerprint = (sampleType, startDate, value, unit, source)

IF count(fingerprint) > 1:
  → Flag as "duplicate record"
  → Severity: LOW (data integrity risk)
  → Calculate time between duplicates
```

#### D. Divergence Detection
```
Track aggregated metrics:
  total_steps_per_day[date]
  active_energy_per_day[date]
  hr_average_per_day[date]

For each metric over time:
  σ_expected = standard deviation (normal range)
  σ_observed = recent variance
  
  IF σ_observed > 2 * σ_expected:
    → Flag as "divergence trend"
    → Severity: MEDIUM
    → Record trend direction & magnitude
```

---

## 5. Sync Monitoring (Separate Thread)

### 5.1 Sync State Tracking

**Observe HealthKit permission & sync state:**
```swift
HKHealthStore().requestAuthorization(...)
// → Detect when user grants/revokes permissions

// Monitor iCloud state
FileManager.default.ubiquityIdentityToken
// → Detects iCloud sign-in/sign-out
// → Triggers re-baseline after sync state changes
```

**Capture lifecycle events:**
- iCloud sign-in detected → re-snapshot everything
- iCloud sign-out detected → note local-only mode
- Device backup initiated → pre-backup snapshot
- App backgrounded/foregrounded → check for sync activity

### 5.2 Sync Documentation

**Audit trail entries:**
```
[2026-05-01 14:23:15] SYNC_STATE_CHANGE: iCloud enabled
  - Previous: local-only
  - Action: re-snapshot all data
  - Result: 5,432 samples captured

[2026-05-01 14:24:02] SYNC_COMPLETE: iCloud data merged
  - Samples added: 87
  - Samples deleted: 3
  - Duplicates found: 2
  - Divergence detected: NO

[2026-05-01 16:15:00] ANOMALY_DETECTED: Historical insertion
  - Sample: Workout "Running" 
  - Original date: 2024-03-15
  - First observed: 2026-05-01
  - Age: 778 days
  - Severity: MEDIUM
```

### 5.3 Background Sync Monitoring

**iOS Background Modes enabled:**
- `background-fetch` — Periodic sync checks
- `remote-notification` → For sync completion hints (optional)

**Sync check frequency:**
- Min: 2 hours
- Max: 24 hours
- Adapts based on anomaly detection frequency

---

## 6. Data Export & Forensics

### 6.1 Export Formats

**Option 1: Local Backup (Swift)**
```
Export structure:
├── metadata.json          (device, iOS version, app version)
├── snapshots/             (timestamped HealthKit snapshots)
├── audit_trail.jsonl      (immutable log, one entry per line)
├── anomalies.json         (detected anomalies)
└── digests.json           (aggregated metrics for trending)
```

**Option 2: CloudKit Digest (Optional)**
```
Send ONLY:
  - Aggregated counts (not samples)
  - Anomaly types & timestamps (not raw health data)
  - Device fingerprint (not identifiable)
  
Schema:
  zone: "HealthProbe_Public"
  record: "MonthlyDigest"
  fields:
    - device_id (salted hash)
    - month: YYYY-MM
    - anomaly_counts: { type: count }
    - data_loss_estimate: percent
    - sample_types_tracked: [...]
```

### 6.2 Forensic Query Examples

**"Has my step data been compromised?"**
```
1. Load all snapshots for "Steps" type
2. Plot sample count over time
3. Identify gaps > 6 hours
4. Report: when, how many missing, context
```

**"Did iCloud sync break my data?"**
```
1. Correlate anomalies with sync state changes
2. Show timeline: before sync, during sync, after
3. Calculate: samples lost, duplicates introduced
```

**"Is my health data drifting?"**
```
1. Compute daily aggregates (steps, energy, HR)
2. Fit trend line over 30-90 days
3. Report: slope (drift direction), R² (confidence)
4. Compare to device baseline
```

---

## 7. User-Facing Features

### 7.1 Dashboard (iOS App)

**Home Screen:**
- **Health Status** — "✅ Healthy" / "⚠️ Check" / "🚨 Critical"
- **Last Check** — timestamp of last monitoring run
- **Quick Stats** — samples tracked, anomalies found (all-time)
- **Active Alerts** — up to 3 most recent anomalies

**Detail Views:**
- **Anomalies** — sortable list by date/severity
- **Snapshots** — historical timeline of known-good snapshots
- **Audit Trail** — complete immutable log
- **Sync Status** — current iCloud state & last sync time

**Settings:**
- Check frequency
- Sample types to track
- Alert thresholds
- CloudKit sync opt-in

### 7.2 Alerts

**Push Notifications (opt-in):**
- 🚨 "Critical data loss detected" (> 10% samples missing)
- ⚠️ "Unexpected historical data inserted" (> 100 samples)
- ℹ️ "Sync completed, 2 duplicates found"

---

## 8. Future Enhancements (Beyond MVP)

### 8.1 macOS Companion (Visualization Layer)
- Aggregate data from multiple iOS devices
- Long-term trend visualization (6-12 month history)
- Cross-device anomaly correlation
- Export to reproducible bug reports

### 8.2 Machine Learning
- Personalized baseline generation
- Anomaly confidence scoring
- Predictive detection (flag drift before threshold hit)

### 8.3 Community Patterns
- Anonymized digest sharing → identify systemic issues
- Detect if data loss correlates with: iOS version, device model, iCloud region, etc.
- Contribute to DearApple bug reports with statistical evidence

---

## 9. Technical Specifications

### 9.1 Platform
- **iOS 15.0+** (HealthKit framework support)
- **watchOS 8.0+** (future sync awareness)
- **macOS 12.0+** (visualization, analysis)

### 9.2 Permissions Required
- `HealthKit` — read-only access to specified types
- `iCloud CloudKit` — optional, for digest sync only
- `Background Modes` — "Background Fetch"

### 9.3 Data Storage
- **Local:** SwiftData (on-device, encrypted by iOS)
- **Optional Cloud:** CloudKit (user-controlled, aggregated only)

### 9.4 Performance
- Query time: < 5 seconds (anchored queries)
- Snapshot size: ≈ 5-10 KB per type per snapshot
- Storage: ≈ 100 MB per year of monitoring (aggregated)

---

## 10. Privacy & Security

### 10.1 What HealthProbe Never Does
- ❌ Exports raw health samples to cloud
- ❌ Identifies users by name/account
- ❌ Shares device location or personal context
- ❌ Modifies any HealthKit data
- ❌ Sells or shares data with third parties

### 10.2 What HealthProbe Collects (Local Only)
- ✅ Aggregated counts (not samples)
- ✅ Timestamps of anomalies
- ✅ Device model & iOS version (for context)
- ✅ Anomaly types & severity

### 10.3 Optional CloudKit Sync
- Only aggregated digests (counts, not data)
- Device identifier is salted & hashed
- Deleted immediately when uninstalled
- Can be disabled in settings

---

## 11. Success Criteria

| Objective | Metric | Target |
|-----------|--------|--------|
| **Detect loss** | Time to detection after loss occurs | < 24 hours |
| **Forensic completeness** | % of anomalies with sufficient evidence | > 95% |
| **False positives** | Alerts user shouldn't worry about | < 5% of total |
| **Privacy** | % of users comfortable with data practices | > 90% |
| **Performance** | Background sync battery impact | < 2% drain/day |
| **Adoption** | Users can reproduce bugs with HealthProbe data | High relevance in Apple feedback |

---

## 12. References & Related Work

- [DearApple Issue #001](https://github.com/overbog/dear-apple/issues/0001-apple-health-mass-data-loss.md) — Sept 2025 mass data loss
- [Apple HealthKit Documentation](https://developer.apple.com/documentation/healthkit/)
- [HKAnchoredObjectQuery](https://developer.apple.com/documentation/healthkit/hkanchoredrobjectquery) — Efficient incremental queries
- [CloudKit Best Practices](https://developer.apple.com/documentation/cloudkit/)

---

## Appendix A: Example Anomaly Report

```json
{
  "anomaly_id": "ANML_20260501_001",
  "type": "historical_insertion",
  "timestamp_detected": "2026-05-01T14:35:22Z",
  "severity": "MEDIUM",
  "evidence": {
    "sample_type": "HKWorkout",
    "workout_type": "Running",
    "start_date": "2025-01-15T07:30:00Z",
    "end_date": "2025-01-15T08:15:00Z",
    "duration_minutes": 45,
    "calories": 420,
    "first_observed": "2026-05-01T14:35:00Z",
    "age_days": 106,
    "source": "Health.app",
    "context": "iCloud sync completed 2 hours prior"
  },
  "classification": "Likely data recovery from cloud",
  "recommended_action": "Monitor for similar patterns"
}
```

---

*HealthProbe — Guarding the integrity of your health data.*
