Write Health archive samples into v2 tables · 8f40c05

Write Health archive samples into v2 tables
Browse files

bogdan committed 2 weeks ago

main

1 parent 5474149

commit 8f40c05

Showing 5 changed files with 847 additions and 80 deletions

+9 -8

HealthProbe/Doc/04-project/IMPLEMENTATION_STATUS.md

@@ -25,7 +25,7 @@ There are no real deployments, only test installations. Existing prototype datab
 |------|----------------|--------------------|
 | Product docs | Updated | Keep `HealthProbe/Doc/README.md` as canonical index |
 | HealthKit capture | Prototype exists | Adapt capture to write differential SQLite observations first |
-| SQLite archive | Archive v2 schema bootstrap exists; legacy write table still active | Move write path from `archive_samples` to observations/samples/versions/events/ranges |
+| SQLite archive | Archive v2 schema and differential write path partially implemented; legacy read table still active | Add daily aggregates, SQL reads, integrity tests, then retire `archive_samples` |
 | Core Data cache | Not implemented | Add rebuildable cache for expensive counts, summaries, report metadata, UI state |
 | SwiftData cache | Exists | Treat as disposable prototype data; reset/ignore during v2 transition |
 | UI | Prototype exists | Reframe screens around observations, diffs, export, archive status |
@@ -38,14 +38,15 @@ There are no real deployments, only test installations. Existing prototype datab
 
 Detailed checkable milestones live in [`Refactoring-Plan.md`](Refactoring-Plan.md).
 
-1. Implement differential write path: observations, samples, payload versions, events/ranges, aggregates.
+1. Finish differential write path: daily aggregates and stricter retry/idempotency tests.
 2. Add SQLite integrity/open/schema-version tests.
-3. Move large diffs/counts into SQL queries with indexes/temp tables/paged results.
-4. Add Core Data UI/report cache and rebuild pipeline.
-5. Replace SwiftData UI dependencies with Core Data/cache DTOs.
-6. Update UI language from anomaly/status to observation/diff/export.
-7. Add streaming exports with manifests.
-8. Validate on low-memory/legacy-class devices.
+3. Move archive reads from `archive_samples` to SQL over visibility ranges and sample versions.
+4. Move large diffs/counts into SQL queries with indexes/temp tables/paged results.
+5. Add Core Data UI/report cache and rebuild pipeline.
+6. Replace SwiftData UI dependencies with Core Data/cache DTOs.
+7. Update UI language from anomaly/status to observation/diff/export.
+8. Add streaming exports with manifests.
+9. Validate on low-memory/legacy-class devices.
 
 ## Known Prototype Mismatches
 


+14 -14

HealthProbe/Doc/04-project/Refactoring-Plan.md

View

+                     **Purpose:** Write observations without storing full recurring snapshots.
+                     Checklist:
+                    -- [ ] Create observation transaction wrapper.
+                    -- [ ] Upsert sample types.
+                    -- [ ] Upsert source/source revision/device/metadata rows.
+                    -- [ ] Upsert sample identity.
+                    -- [ ] Upsert sample payload version only when payload changes.
+                    -- [ ] Insert appeared/verified/representationChanged events.
+                    -- [ ] Record `HKDeletedObject` evidence by UUID hash.
+                    -- [ ] Close visibility ranges for disappeared/deleted samples.
+                    -- [ ] Maintain open visibility ranges for visible samples.
+                    +- [x] Create observation transaction wrapper.
+                    +- [x] Upsert sample types.
+                    +- [x] Upsert source/source revision/device/metadata rows.
+                    +- [x] Upsert sample identity.
+                    +- [x] Upsert sample payload version only when payload changes.
+                    +- [x] Insert appeared/verified/representationChanged events.
+                    +- [x] Record `HKDeletedObject` evidence by UUID hash.
+                    +- [x] Close visibility ranges for disappeared/deleted samples.
+                    +- [x] Maintain open visibility ranges for visible samples.
+                     - [ ] Rebuild/update affected aggregates after capture.
+                    -- [ ] Commit SQLite before Core Data/cache work.
+                    +- [x] Commit SQLite before Core Data/cache work.
+                     - [ ] Make repeated capture page writes idempotent.
+                     Acceptance:
+                    -- [ ] Initial import stores identities and versions once.
+                    -- [ ] Re-running same page does not duplicate records.
+                    -- [ ] Representation change creates a new version, not a new logical sample.
+                    -- [ ] Disappearance closes visibility range.
+                    +- [x] Initial import stores identities and versions once.
+                    +- [x] Re-running same page does not duplicate sample identities or payload versions.
+                    +- [x] Representation change creates a new version, not a new logical sample.
+                    +- [x] Disappearance closes visibility range.
+                     - [ ] No full observation copy table is written.
+                     ## Milestone 5 - SQL Analysis Layer

+14 -8

HealthProbe/Services/HashService.swift

View

@@ -20,17 +20,23 @@ enum HashService {
         }
     }
 
-    private static let iso8601Formatter: ISO8601DateFormatter = {
+    nonisolated(unsafe) private static let iso8601Formatter: ISO8601DateFormatter = {
         let f = ISO8601DateFormatter()
         f.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
         return f
     }()
 
+    nonisolated static func archiveContentHash(domain: String, parts: [String?]) -> String {
+        let input = ([domain] + parts.map { $0 ?? "null" }).joined(separator: "\u{1f}")
+        let digest = SHA256.hash(data: Data(input.utf8))
+        return digest.map { String(format: "%02x", $0) }.joined()
+    }
+
     // SHA256 of "typeIdentifier|totalCount|earliestDateISO|latestDateISO"
     // ⚠️ MVP limitation: hash covers only count + date range, not value distribution.
     // silentReplacement detection based on this hash is best-effort only —
     // it will miss replacements that preserve total count and date boundaries.
-    static func typeHash(
+    nonisolated static func typeHash(
         typeIdentifier: String,
         totalCount: Int,
         earliestDate: Date?,
@@ -43,7 +49,7 @@ enum HashService {
         return digest.map { String(format: "%02x", $0) }.joined()
     }
 
-    static func typeHash(typeIdentifier: String, recordFingerprints: [String]) -> String {
+    nonisolated static func typeHash(typeIdentifier: String, recordFingerprints: [String]) -> String {
         var hasher = SHA256()
         hasher.update(data: Data(typeIdentifier.utf8))
         for fingerprint in recordFingerprints.sorted() {
@@ -54,7 +60,7 @@ enum HashService {
         return digest.map { String(format: "%02x", $0) }.joined()
     }
 
-    static func typeHashInRecordedOrder(typeIdentifier: String, recordFingerprints: [String]) -> String {
+    nonisolated static func typeHashInRecordedOrder(typeIdentifier: String, recordFingerprints: [String]) -> String {
         var builder = TypeHashBuilder(typeIdentifier: typeIdentifier)
         for fingerprint in recordFingerprints {
             builder.append(recordFingerprint: fingerprint)
@@ -62,7 +68,7 @@ enum HashService {
         return builder.finalize()
     }
 
-    static func sampleFingerprint(
+    nonisolated static func sampleFingerprint(
         typeIdentifier: String,
         sampleUUID: String,
         startDate: Date,
@@ -78,12 +84,12 @@ enum HashService {
         return digest.map { String(format: "%02x", $0) }.joined()
     }
 
-    static func sampleUUIDHash(_ sampleUUID: String) -> String {
+    nonisolated static func sampleUUIDHash(_ sampleUUID: String) -> String {
         let digest = SHA256.hash(data: Data(sampleUUID.utf8))
         return digest.map { String(format: "%02x", $0) }.joined()
     }
 
-    static func archiveSemanticFingerprint(
+    nonisolated static func archiveSemanticFingerprint(
         typeIdentifier: String,
         startDate: Date,
         endDate: Date,
@@ -124,7 +130,7 @@ enum HashService {
     // ⚠️ Covers the FULL intended registry (selectedTypeIDs), including types that may have
     // failed, timed out, or been unauthorized — never filter down to only the successfully-
     // fetched subset. A query failure must not silently change the registry hash.
-    static func typeSetHash(typeIDs: [String]) -> String {
+    nonisolated static func typeSetHash(typeIDs: [String]) -> String {
         let sorted = typeIDs.sorted().joined(separator: "|")
         let digest = SHA256.hash(data: Data(sorted.utf8))
         return digest.map { String(format: "%02x", $0) }.joined()


+6 -6

HealthProbe/Services/KeychainService.swift

View

@@ -5,9 +5,9 @@ import os.log
 private let logger = Logger(subsystem: "ro.xdev.healthprobe", category: "KeychainService")
 
 enum KeychainService {
-    private static let service = "ro.xdev.healthprobe.deviceid"
-    private static let account = "stable_device_id"
-    private static var cached: String?
+    nonisolated private static let service = "ro.xdev.healthprobe.deviceid"
+    nonisolated private static let account = "stable_device_id"
+    nonisolated(unsafe) private static var cached: String?
 
     struct Resolution {
         let id: String
@@ -17,7 +17,7 @@ enum KeychainService {
     // If swiftDataStoreIsEmpty is true and Keychain has an existing ID, the DB was wiped
     // (reinstall or manual reset) but Keychain survived. We keep the same deviceID but signal
     // that a new chain must start and is marked recovered.
-    static func resolveDeviceID(swiftDataStoreIsEmpty: Bool) -> Resolution {
+    nonisolated static func resolveDeviceID(swiftDataStoreIsEmpty: Bool) -> Resolution {
         if let existing = readFromKeychain() {
             if swiftDataStoreIsEmpty {
                 // DB was wiped but Keychain survived — recovered device ID
@@ -30,7 +30,7 @@ enum KeychainService {
         return Resolution(id: new, isRecovered: false)
     }
 
-    private static func readFromKeychain() -> String? {
+    nonisolated private static func readFromKeychain() -> String? {
         if let cached { return cached }
         let query: [String: Any] = [
             kSecClass as String:            kSecClassGenericPassword,
@@ -50,7 +50,7 @@ enum KeychainService {
         return string
     }
 
-    private static func writeToKeychain(_ value: String) {
+    nonisolated private static func writeToKeychain(_ value: String) {
         guard let data = value.data(using: .utf8) else { return }
         let attributes: [String: Any] = [
             kSecClass as String:        kSecClassGenericPassword,


+804 -44

HealthProbe/Services/SQLiteHealthArchiveStore.swift

View

+                             try execute("BEGIN IMMEDIATE TRANSACTION", db: db)
+                             do {
+                                 let summary = try upsertSamples(samples, observedAt: observedAt, db: db)
+                    +            try execute("PRAGMA foreign_key_check", db: db)
+                                 try execute("COMMIT", db: db)
+                                 return summary
+                             } catch {
+                             let db = try openDatabase()
+                             defer { sqlite3_close(db) }
+                             try prepareSchemaIfNeeded(db)
+                    +        try execute("BEGIN IMMEDIATE TRANSACTION", db: db)
+                    +        do {
+                    +            let observationID = try createObservation(
+                    +                observedAt: observedMissingAt,
+                    +                triggerReason: "deleted_object",
+                    +                status: "completed",
+                    +                db: db
+                    +            )
+                    +            if let sampleTypeID = try sampleTypeID(typeIdentifier: sampleTypeIdentifier, db: db),
+                    +               let sampleID = try sampleID(sampleUUIDHash: sampleUUIDHash, sampleTypeID: sampleTypeID, db: db) {
+                    +                try insertObservationEvent(
+                    +                    observationID: observationID,
+                    +                    sampleID: sampleID,
+                    +                    versionID: nil,
+                    +                    eventKind: "disappeared",
+                    +                    evidenceKind: "deleted_object",
+                    +                    observedAt: observedMissingAt,
+                    +                    db: db
+                    +                )
+                    +                try closeOpenVisibilityRanges(
+                    +                    sampleID: sampleID,
+                    +                    excludingVersionID: nil,
+                    +                    closedAtObservationID: observationID,
+                    +                    observedAt: observedMissingAt,
+                    +                    db: db
+                    +                )
+                    +                try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
+                    +            }
+                    -        let sql = """
+                    -        UPDATE archive_samples
+                    -        SET disappeared_at = ?, last_verified_at = ?
+                    -        WHERE sample_uuid_hash = ? AND type_identifier = ?
+                    -        """
+                    -        try withStatement(sql, db: db) { statement in
+                    -            sqlite3_bind_double(statement, 1, observedMissingAt.timeIntervalSinceReferenceDate)
+                    -            sqlite3_bind_double(statement, 2, observedMissingAt.timeIntervalSinceReferenceDate)
+                    -            bindText(sampleUUIDHash, to: 3, in: statement)
+                    -            bindText(sampleTypeIdentifier, to: 4, in: statement)
+                    -            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    -                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            let sql = """
+                    +            UPDATE archive_samples
+                    +            SET disappeared_at = ?, last_verified_at = ?
+                    +            WHERE sample_uuid_hash = ? AND type_identifier = ?
+                    +            """
+                    +            try withStatement(sql, db: db) { statement in
+                    +                sqlite3_bind_double(statement, 1, observedMissingAt.timeIntervalSinceReferenceDate)
+                    +                sqlite3_bind_double(statement, 2, observedMissingAt.timeIntervalSinceReferenceDate)
+                    +                bindText(sampleUUIDHash, to: 3, in: statement)
+                    +                bindText(sampleTypeIdentifier, to: 4, in: statement)
+                    +                guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                    throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +                }
+                    +            try execute("COMMIT", db: db)
+                    +        } catch {
+                    +            try? execute("ROLLBACK", db: db)
+                    +            throw error
+                         private func upsertSamples(_ samples: [HKSample], observedAt: Date, db: OpaquePointer?) throws -> HealthArchiveWriteSummary {
+                    +        let rows = samples.map { ArchiveSampleRow(sample: $0, observedAt: observedAt) }
+                    +        let observationID = try createObservation(
+                    +            observedAt: observedAt,
+                    +            triggerReason: "anchored_page",
+                    +            status: "completed",
+                    +            db: db
+                    +        )
+                    +        var inserted = 0
+                    +        var updated = 0
+                    +        var unchanged = 0
+                    +        var touchedTypeIDs = Set<Int64>()
+                    +        try withLegacyArchiveSampleStatement(db: db) { legacyStatement in
+                    +            for row in rows {
+                    +                let result = try upsertArchiveV2Sample(row, observationID: observationID, db: db)
+                    +                touchedTypeIDs.insert(result.sampleTypeID)
+                    +                switch result.kind {
+                    +                case .inserted:
+                    +                    inserted += 1
+                    +                case .updated:
+                    +                    updated += 1
+                    +                case .unchanged:
+                    +                    unchanged += 1
+                    +                }
+                    +                sqlite3_reset(legacyStatement)
+                    +                sqlite3_clear_bindings(legacyStatement)
+                    +                bind(row, to: legacyStatement)
+                    +                guard sqlite3_step(legacyStatement) == SQLITE_DONE else {
+                    +                    throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +                }
+                    +            }
+                    +        }
+                    +        for sampleTypeID in touchedTypeIDs {
+                    +            try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
+                    +        }
+                    +        return HealthArchiveWriteSummary(
+                    +            insertedCount: inserted,
+                    +            updatedCount: updated,
+                    +            unchangedCount: unchanged
+                    +        )
+                    +    }
+                    +    private func withLegacyArchiveSampleStatement<T>(
+                    +        db: OpaquePointer?,
+                    +        body: (OpaquePointer?) throws -> T
+                    +    ) throws -> T {
+                             let sql = """
+                             INSERT INTO archive_samples (
+                                 sample_uuid_hash, type_identifier, strict_fingerprint, semantic_fingerprint,
+                                 metadata_json = excluded.metadata_json,
+                                 archived_at = excluded.archived_at
+                             """
+                    +        return try withStatement(sql, db: db, body: body)
+                    +    }
+                    +    private func upsertArchiveV2Sample(
+                    +        _ row: ArchiveSampleRow,
+                    +        observationID: Int64,
+                    +        db: OpaquePointer?
+                    +    ) throws -> ArchiveV2SampleWriteResult {
+                    +        let sampleTypeID = try upsertSampleType(typeIdentifier: row.typeIdentifier, db: db)
+                    +        let sourceRevisionID = try upsertSourceRevision(row, db: db)
+                    +        let deviceID = try upsertDevice(row, db: db)
+                    +        let metadataID = try upsertMetadataBlob(row, db: db)
+                    +        let sampleResult = try upsertSample(row, sampleTypeID: sampleTypeID, observationID: observationID, db: db)
+                    +        let versionResult = try upsertSampleVersion(
+                    +            row,
+                    +            sampleID: sampleResult.id,
+                    +            sourceRevisionID: sourceRevisionID,
+                    +            deviceID: deviceID,
+                    +            metadataID: metadataID,
+                    +            observationID: observationID,
+                    +            db: db
+                    +        )
+                    +        let writeKind: ArchiveV2SampleWriteKind
+                    +        let eventKind: String
+                    +        if sampleResult.inserted {
+                    +            writeKind = .inserted
+                    +            eventKind = "appeared"
+                    +        } else if versionResult.inserted {
+                    +            writeKind = .updated
+                    +            eventKind = "representationChanged"
+                    +        } else {
+                    +            writeKind = .unchanged
+                    +            eventKind = "verified"
+                    +        }
+                    +        try insertObservationEvent(
+                    +            observationID: observationID,
+                    +            sampleID: sampleResult.id,
+                    +            versionID: versionResult.id,
+                    +            eventKind: eventKind,
+                    +            evidenceKind: "healthkit_sample",
+                    +            observedAt: row.observedAt,
+                    +            db: db
+                    +        )
+                    +        try closeOpenVisibilityRanges(
+                    +            sampleID: sampleResult.id,
+                    +            excludingVersionID: versionResult.id,
+                    +            closedAtObservationID: observationID,
+                    +            observedAt: row.observedAt,
+                    +            db: db
+                    +        )
+                    +        try insertOpenVisibilityRangeIfNeeded(
+                    +            sampleID: sampleResult.id,
+                    +            versionID: versionResult.id,
+                    +            observationID: observationID,
+                    +            observedAt: row.observedAt,
+                    +            db: db
+                    +        )
+                    +        return ArchiveV2SampleWriteResult(sampleTypeID: sampleTypeID, kind: writeKind)
+                    +    }
+                    +    private func createObservation(
+                    +        observedAt: Date,
+                    +        triggerReason: String,
+                    +        status: String,
+                    +        db: OpaquePointer?
+                    +    ) throws -> Int64 {
+                    +        let deviceChainID = try upsertCurrentDeviceChain(db)
+                    +        let timeZone = TimeZone.current
+                    +        let sql = """
+                    +        INSERT INTO observations (
+                    +            device_chain_id, observed_at, started_at, ended_at, status, trigger_reason,
+                    +            app_version, os_version, time_zone_identifier, time_zone_seconds_from_gmt,
+                    +            schema_version, selected_type_set_hash, notes
+                    +        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, NULL, NULL)
+                    +        """
+                    +        try withStatement(sql, db: db) { statement in
+                    +            bindInt64(deviceChainID, to: 1, in: statement)
+                    +            sqlite3_bind_double(statement, 2, observedAt.timeIntervalSince1970)
+                    +            sqlite3_bind_double(statement, 3, observedAt.timeIntervalSince1970)
+                    +            sqlite3_bind_double(statement, 4, Date().timeIntervalSince1970)
+                    +            bindText(status, to: 5, in: statement)
+                    +            bindText(triggerReason, to: 6, in: statement)
+                    +            bindText(Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String, to: 7, in: statement)
+                    +            bindText(ProcessInfo.processInfo.operatingSystemVersionString, to: 8, in: statement)
+                    +            bindText(timeZone.identifier, to: 9, in: statement)
+                    +            sqlite3_bind_int(statement, 10, Int32(timeZone.secondsFromGMT(for: observedAt)))
+                    +            bindInt(Self.archiveSchemaVersion, to: 11, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return sqlite3_last_insert_rowid(db)
+                    +    }
+                    +    private func upsertCurrentDeviceChain(_ db: OpaquePointer?) throws -> Int64 {
+                    +        let resolution = KeychainService.resolveDeviceID(swiftDataStoreIsEmpty: false)
+                    +        let chainHash = HashService.archiveContentHash(domain: "hp:v2:device_chain", parts: [resolution.id])
+                    +        try withStatement(
+                    +            "INSERT OR IGNORE INTO device_chains (device_chain_hash, created_at, recovered_from_keychain) VALUES (?, ?, ?)",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(chainHash, to: 1, in: statement)
+                    +            sqlite3_bind_double(statement, 2, Date().timeIntervalSince1970)
+                    +            sqlite3_bind_int(statement, 3, resolution.isRecovered ? 1 : 0)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return try requiredInt64(
+                    +            "SELECT id FROM device_chains WHERE device_chain_hash = ? LIMIT 1",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(chainHash, to: 1, in: statement)
+                    +        }
+                    +    }
+                    +    private func upsertSampleType(typeIdentifier: String, db: OpaquePointer?) throws -> Int64 {
+                    +        try withStatement(
+                    +            "INSERT OR IGNORE INTO sample_types (type_identifier, display_name, category) VALUES (?, NULL, NULL)",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(typeIdentifier, to: 1, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return try requiredInt64("SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1", db: db) { statement in
+                    +            bindText(typeIdentifier, to: 1, in: statement)
+                    +        }
+                    +    }
+                    +    private func sampleTypeID(typeIdentifier: String, db: OpaquePointer?) throws -> Int64? {
+                    +        try optionalInt64("SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1", db: db) { statement in
+                    +            bindText(typeIdentifier, to: 1, in: statement)
+                    +        }
+                    +    }
+                    +    private func upsertSourceRevision(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
+                    +        guard row.sourceName != nil || row.sourceBundleIdentifier != nil else { return nil }
+                    +        let sourceNameHash = row.sourceName.map { HashService.archiveContentHash(domain: "hp:v2:source_name", parts: [$0]) }
+                    +        let sourceID = try upsertSource(sourceNameHash: sourceNameHash, bundleIdentifier: row.sourceBundleIdentifier, db: db)
+                    +        try withStatement(
+                    +            "INSERT OR IGNORE INTO source_revisions (source_id, product_type, version, operating_system_version) VALUES (?, ?, ?, ?)",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sourceID, to: 1, in: statement)
+                    +            bindText(row.sourceProductType, to: 2, in: statement)
+                    +            bindText(row.sourceVersion, to: 3, in: statement)
+                    +            bindText(row.sourceOperatingSystemVersion, to: 4, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return try requiredInt64(
+                    +            """
+                    +            SELECT id FROM source_revisions
+                    +            WHERE source_id = ?
+                    +              AND (product_type = ? OR (product_type IS NULL AND ? IS NULL))
+                    +              AND (version = ? OR (version IS NULL AND ? IS NULL))
+                    +              AND (operating_system_version = ? OR (operating_system_version IS NULL AND ? IS NULL))
+                    +            LIMIT 1
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sourceID, to: 1, in: statement)
+                    +            bindText(row.sourceProductType, to: 2, in: statement)
+                    +            bindText(row.sourceProductType, to: 3, in: statement)
+                    +            bindText(row.sourceVersion, to: 4, in: statement)
+                    +            bindText(row.sourceVersion, to: 5, in: statement)
+                    +            bindText(row.sourceOperatingSystemVersion, to: 6, in: statement)
+                    +            bindText(row.sourceOperatingSystemVersion, to: 7, in: statement)
+                    +        }
+                    +    }
+                    +    private func upsertSource(sourceNameHash: String?, bundleIdentifier: String?, db: OpaquePointer?) throws -> Int64 {
+                    +        if let existing = try optionalInt64(
+                    +            """
+                    +            SELECT id FROM sources
+                    +            WHERE (source_name_hash = ? OR (source_name_hash IS NULL AND ? IS NULL))
+                    +              AND (bundle_identifier = ? OR (bundle_identifier IS NULL AND ? IS NULL))
+                    +            LIMIT 1
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(sourceNameHash, to: 1, in: statement)
+                    +            bindText(sourceNameHash, to: 2, in: statement)
+                    +            bindText(bundleIdentifier, to: 3, in: statement)
+                    +            bindText(bundleIdentifier, to: 4, in: statement)
+                    +        } {
+                    +            return existing
+                    +        }
+                    +        try withStatement("INSERT INTO sources (source_name_hash, bundle_identifier) VALUES (?, ?)", db: db) { statement in
+                    +            bindText(sourceNameHash, to: 1, in: statement)
+                    +            bindText(bundleIdentifier, to: 2, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return sqlite3_last_insert_rowid(db)
+                    +    }
+                    +    private func upsertDevice(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
+                    +        guard row.hasDeviceProvenance else { return nil }
+                    +        let deviceHash = row.deviceName.map { HashService.archiveContentHash(domain: "hp:v2:device_name", parts: [$0]) }
+                    +        let manufacturerHash = row.deviceManufacturer.map { HashService.archiveContentHash(domain: "hp:v2:device_manufacturer", parts: [$0]) }
+                    +        let localIdentifierHash = row.deviceLocalIdentifier.map { HashService.archiveContentHash(domain: "hp:v2:device_local_id", parts: [$0]) }
+                    +        let udiHash = row.deviceUDI.map { HashService.archiveContentHash(domain: "hp:v2:device_udi", parts: [$0]) }
+                    +        if let existing = try optionalInt64(
+                    +            """
+                    +            SELECT id FROM hk_devices
+                    +            WHERE (device_hash = ? OR (device_hash IS NULL AND ? IS NULL))
+                    +              AND (local_identifier_hash = ? OR (local_identifier_hash IS NULL AND ? IS NULL))
+                    +              AND (udi_hash = ? OR (udi_hash IS NULL AND ? IS NULL))
+                    +              AND (model = ? OR (model IS NULL AND ? IS NULL))
+                    +            LIMIT 1
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(deviceHash, to: 1, in: statement)
+                    +            bindText(deviceHash, to: 2, in: statement)
+                    +            bindText(localIdentifierHash, to: 3, in: statement)
+                    +            bindText(localIdentifierHash, to: 4, in: statement)
+                    +            bindText(udiHash, to: 5, in: statement)
+                    +            bindText(udiHash, to: 6, in: statement)
+                    +            bindText(row.deviceModel, to: 7, in: statement)
+                    +            bindText(row.deviceModel, to: 8, in: statement)
+                    +        } {
+                    +            return existing
+                    +        }
+                    +        try withStatement(
+                    +            """
+                    +            INSERT INTO hk_devices (
+                    +                device_hash, manufacturer_hash, model, hardware_version, firmware_version,
+                    +                software_version, local_identifier_hash, udi_hash
+                    +            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(deviceHash, to: 1, in: statement)
+                    +            bindText(manufacturerHash, to: 2, in: statement)
+                    +            bindText(row.deviceModel, to: 3, in: statement)
+                    +            bindText(row.deviceHardwareVersion, to: 4, in: statement)
+                    +            bindText(row.deviceFirmwareVersion, to: 5, in: statement)
+                    +            bindText(row.deviceSoftwareVersion, to: 6, in: statement)
+                    +            bindText(localIdentifierHash, to: 7, in: statement)
+                    +            bindText(udiHash, to: 8, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return sqlite3_last_insert_rowid(db)
+                    +    }
+                    +    private func upsertMetadataBlob(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
+                    +        guard let metadataHash = row.metadataHash, let metadataJSON = row.metadataJSON else { return nil }
+                    +        try withStatement(
+                    +            "INSERT OR IGNORE INTO metadata_blobs (metadata_hash, metadata_json) VALUES (?, ?)",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindText(metadataHash, to: 1, in: statement)
+                    +            bindText(metadataJSON, to: 2, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        return try requiredInt64("SELECT id FROM metadata_blobs WHERE metadata_hash = ? LIMIT 1", db: db) { statement in
+                    +            bindText(metadataHash, to: 1, in: statement)
+                    +        }
+                    +    }
+                    +    private func upsertSample(
+                    +        _ row: ArchiveSampleRow,
+                    +        sampleTypeID: Int64,
+                    +        observationID: Int64,
+                    +        db: OpaquePointer?
+                    +    ) throws -> (id: Int64, inserted: Bool) {
+                    +        try withStatement(
+                    +            """
+                    +            INSERT OR IGNORE INTO samples (
+                    +                sample_type_id, sample_uuid_hash, strict_fingerprint, semantic_fingerprint,
+                    +                fuzzy_key, first_seen_observation_id, first_seen_at
+                    +            ) VALUES (?, ?, ?, ?, NULL, ?, ?)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleTypeID, to: 1, in: statement)
+                    +            bindText(row.sampleUUIDHash, to: 2, in: statement)
+                    +            bindText(row.strictFingerprint, to: 3, in: statement)
+                    +            bindText(row.semanticFingerprint, to: 4, in: statement)
+                    +            bindInt64(observationID, to: 5, in: statement)
+                    +            sqlite3_bind_double(statement, 6, row.observedAt.timeIntervalSince1970)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        let inserted = sqlite3_changes(db) > 0
+                    +        let id = try requiredInt64(
+                    +            "SELECT id FROM samples WHERE sample_type_id = ? AND strict_fingerprint = ? LIMIT 1",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleTypeID, to: 1, in: statement)
+                    +            bindText(row.strictFingerprint, to: 2, in: statement)
+                    +        }
+                    +        return (id, inserted)
+                    +    }
+                    +    private func sampleID(sampleUUIDHash: String, sampleTypeID: Int64, db: OpaquePointer?) throws -> Int64? {
+                    +        try optionalInt64(
+                    +            "SELECT id FROM samples WHERE sample_type_id = ? AND sample_uuid_hash = ? LIMIT 1",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleTypeID, to: 1, in: statement)
+                    +            bindText(sampleUUIDHash, to: 2, in: statement)
+                    +        }
+                    +    }
+                    +    private func upsertSampleVersion(
+                    +        _ row: ArchiveSampleRow,
+                    +        sampleID: Int64,
+                    +        sourceRevisionID: Int64?,
+                    +        deviceID: Int64?,
+                    +        metadataID: Int64?,
+                    +        observationID: Int64,
+                    +        db: OpaquePointer?
+                    +    ) throws -> (id: Int64, inserted: Bool) {
+                    +        try withStatement(
+                    +            """
+                    +            INSERT OR IGNORE INTO sample_versions (
+                    +                sample_id, payload_hash, start_date, end_date, value_kind, numeric_value,
+                    +                unit, category_value, workout_activity_type, duration_seconds,
+                    +                source_revision_id, hk_device_id, metadata_id, created_observation_id
+                    +            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleID, to: 1, in: statement)
+                    +            bindText(row.payloadHash, to: 2, in: statement)
+                    +            sqlite3_bind_double(statement, 3, row.startDate.timeIntervalSince1970)
+                    +            sqlite3_bind_double(statement, 4, row.endDate.timeIntervalSince1970)
+                    +            bindText(row.valueKind, to: 5, in: statement)
+                    +            bindDouble(row.value, to: 6, in: statement)
+                    +            bindText(row.unit, to: 7, in: statement)
+                    +            bindInt(row.categoryValue, to: 8, in: statement)
+                    +            bindInt(row.workoutActivityType, to: 9, in: statement)
+                    +            bindDouble(row.durationSeconds, to: 10, in: statement)
+                    +            bindInt64(sourceRevisionID, to: 11, in: statement)
+                    +            bindInt64(deviceID, to: 12, in: statement)
+                    +            bindInt64(metadataID, to: 13, in: statement)
+                    +            bindInt64(observationID, to: 14, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +        let inserted = sqlite3_changes(db) > 0
+                    +        let id = try requiredInt64(
+                    +            "SELECT id FROM sample_versions WHERE sample_id = ? AND payload_hash = ? LIMIT 1",
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleID, to: 1, in: statement)
+                    +            bindText(row.payloadHash, to: 2, in: statement)
+                    +        }
+                    +        return (id, inserted)
+                    +    }
+                    +    private func insertObservationEvent(
+                    +        observationID: Int64,
+                    +        sampleID: Int64,
+                    +        versionID: Int64?,
+                    +        eventKind: String,
+                    +        evidenceKind: String,
+                    +        observedAt: Date,
+                    +        db: OpaquePointer?
+                    +    ) throws {
+                    +        try withStatement(
+                    +            """
+                    +            INSERT OR IGNORE INTO sample_observation_events (
+                    +                observation_id, sample_id, version_id, event_kind, observed_at, evidence_kind
+                    +            ) VALUES (?, ?, ?, ?, ?, ?)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(observationID, to: 1, in: statement)
+                    +            bindInt64(sampleID, to: 2, in: statement)
+                    +            bindInt64(versionID, to: 3, in: statement)
+                    +            bindText(eventKind, to: 4, in: statement)
+                    +            sqlite3_bind_double(statement, 5, observedAt.timeIntervalSince1970)
+                    +            bindText(evidenceKind, to: 6, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +    }
+                    +    private func closeOpenVisibilityRanges(
+                    +        sampleID: Int64,
+                    +        excludingVersionID: Int64?,
+                    +        closedAtObservationID: Int64,
+                    +        observedAt: Date,
+                    +        db: OpaquePointer?
+                    +    ) throws {
+                    +        let versionPredicate = excludingVersionID == nil
+                    +            ? ""
+                    +            : "AND (version_id IS NULL OR version_id != ?)"
+                    +        let sql = """
+                    +        UPDATE sample_visibility_ranges
+                    +        SET last_observation_id = ?, last_seen_at = ?
+                    +        WHERE sample_id = ? AND last_observation_id IS NULL \(versionPredicate)
+                    +        """
+                    +        try withStatement(sql, db: db) { statement in
+                    +            bindInt64(closedAtObservationID, to: 1, in: statement)
+                    +            sqlite3_bind_double(statement, 2, observedAt.timeIntervalSince1970)
+                    +            bindInt64(sampleID, to: 3, in: statement)
+                    +            if let excludingVersionID {
+                    +                bindInt64(excludingVersionID, to: 4, in: statement)
+                    +            }
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +    }
+                    +    private func insertOpenVisibilityRangeIfNeeded(
+                    +        sampleID: Int64,
+                    +        versionID: Int64,
+                    +        observationID: Int64,
+                    +        observedAt: Date,
+                    +        db: OpaquePointer?
+                    +    ) throws {
+                    +        let existing = try optionalInt64(
+                    +            """
+                    +            SELECT first_observation_id
+                    +            FROM sample_visibility_ranges
+                    +            WHERE sample_id = ? AND version_id = ? AND last_observation_id IS NULL
+                    +            LIMIT 1
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleID, to: 1, in: statement)
+                    +            bindInt64(versionID, to: 2, in: statement)
+                    +        }
+                    +        guard existing == nil else { return }
+                    +        try withStatement(
+                    +            """
+                    +            INSERT OR IGNORE INTO sample_visibility_ranges (
+                    +                sample_id, version_id, first_observation_id, last_observation_id, first_seen_at, last_seen_at
+                    +            ) VALUES (?, ?, ?, NULL, ?, NULL)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(sampleID, to: 1, in: statement)
+                    +            bindInt64(versionID, to: 2, in: statement)
+                    +            bindInt64(observationID, to: 3, in: statement)
+                    +            sqlite3_bind_double(statement, 4, observedAt.timeIntervalSince1970)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +    }
+                    +    private func rebuildTypeSummary(observationID: Int64, sampleTypeID: Int64, db: OpaquePointer?) throws {
+                    +        let summary = try typeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
+                    +        let aggregateParts: [String?] = [
+                    +            String(observationID),
+                    +            String(sampleTypeID),
+                    +            String(summary.visibleRecordCount),
+                    +            String(summary.appearedCount),
+                    +            String(summary.disappearedCount),
+                    +            String(summary.representationChangedCount),
+                    +            summary.earliestStartDate.map { String($0) },
+                    +            summary.latestEndDate.map { String($0) },
+                    +            summary.valueSum.map { String(format: "%.17g", $0) },
+                    +            summary.valueMax.map { String(format: "%.17g", $0) }
+                    +        ]
+                    +        let aggregateHash = HashService.archiveContentHash(
+                    +            domain: "hp:v2:type_summary",
+                    +            parts: aggregateParts
+                    +        )
+                    +        try withStatement(
+                    +            """
+                    +            INSERT OR REPLACE INTO observation_type_summaries (
+                    +                observation_id, sample_type_id, visible_record_count, appeared_count,
+                    +                disappeared_count, representation_changed_count, earliest_start_date,
+                    +                latest_end_date, value_sum, value_max, aggregate_hash
+                    +            ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                    +            """,
+                    +            db: db
+                    +        ) { statement in
+                    +            bindInt64(observationID, to: 1, in: statement)
+                    +            bindInt64(sampleTypeID, to: 2, in: statement)
+                    +            bindInt(summary.visibleRecordCount, to: 3, in: statement)
+                    +            bindInt(summary.appearedCount, to: 4, in: statement)
+                    +            bindInt(summary.disappearedCount, to: 5, in: statement)
+                    +            bindInt(summary.representationChangedCount, to: 6, in: statement)
+                    +            bindDouble(summary.earliestStartDate, to: 7, in: statement)
+                    +            bindDouble(summary.latestEndDate, to: 8, in: statement)
+                    +            bindDouble(summary.valueSum, to: 9, in: statement)
+                    +            bindDouble(summary.valueMax, to: 10, in: statement)
+                    +            bindText(aggregateHash, to: 11, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_DONE else {
+                    +                throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    +            }
+                    +        }
+                    +    }
+                    +    private func typeSummary(observationID: Int64, sampleTypeID: Int64, db: OpaquePointer?) throws -> ArchiveV2TypeSummary {
+                    +        let counts = try eventCounts(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
+                    +        let aggregate = try visibleAggregate(sampleTypeID: sampleTypeID, db: db)
+                    +        return ArchiveV2TypeSummary(
+                    +            visibleRecordCount: aggregate.visibleRecordCount,
+                    +            appearedCount: counts.appeared,
+                    +            disappearedCount: counts.disappeared,
+                    +            representationChangedCount: counts.representationChanged,
+                    +            earliestStartDate: aggregate.earliestStartDate,
+                    +            latestEndDate: aggregate.latestEndDate,
+                    +            valueSum: aggregate.valueSum,
+                    +            valueMax: aggregate.valueMax
+                    +        )
+                    +    }
+                    +    private func eventCounts(
+                    +        observationID: Int64,
+                    +        sampleTypeID: Int64,
+                    +        db: OpaquePointer?
+                    +    ) throws -> (appeared: Int, disappeared: Int, representationChanged: Int) {
+                    +        let sql = """
+                    +        SELECT event_kind, COUNT(*)
+                    +        FROM sample_observation_events e
+                    +        JOIN samples s ON s.id = e.sample_id
+                    +        WHERE e.observation_id = ? AND s.sample_type_id = ?
+                    +        GROUP BY event_kind
+                    +        """
+                             return try withStatement(sql, db: db) { statement in
+                    -            var inserted = 0
+                    -            var updated = 0
+                    -            for sample in samples {
+                    -                sqlite3_reset(statement)
+                    -                sqlite3_clear_bindings(statement)
+                    -                let row = ArchiveSampleRow(sample: sample, observedAt: observedAt)
+                    -                bind(row, to: statement)
+                    -                guard sqlite3_step(statement) == SQLITE_DONE else {
+                    -                    throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                    -                }
+                    -                if sqlite3_changes(db) == 1 {
+                    -                    inserted += 1
+                    -                } else {
+                    -                    updated += 1
+                    +            bindInt64(observationID, to: 1, in: statement)
+                    +            bindInt64(sampleTypeID, to: 2, in: statement)
+                    +            var appeared = 0
+                    +            var disappeared = 0
+                    +            var representationChanged = 0
+                    +            while sqlite3_step(statement) == SQLITE_ROW {
+                    +                let kind = columnText(statement, 0)
+                    +                let count = columnInt(statement, 1) ?? 0
+                    +                switch kind {
+                    +                case "appeared":
+                    +                    appeared = count
+                    +                case "disappeared":
+                    +                    disappeared = count
+                    +                case "representationChanged":
+                    +                    representationChanged = count
+                    +                default:
+                    +                    break
+                    -            return HealthArchiveWriteSummary(
+                    -                insertedCount: inserted,
+                    -                updatedCount: updated,
+                    -                unchangedCount: max(0, samples.count - inserted - updated)
+                    +            return (appeared, disappeared, representationChanged)
+                    +        }
+                    +    }
+                    +    private func visibleAggregate(sampleTypeID: Int64, db: OpaquePointer?) throws -> ArchiveV2VisibleAggregate {
+                    +        let sql = """
+                    +        SELECT COUNT(*), MIN(v.start_date), MAX(v.end_date), SUM(v.numeric_value), MAX(v.numeric_value)
+                    +        FROM sample_visibility_ranges r
+                    +        JOIN samples s ON s.id = r.sample_id
+                    +        JOIN sample_versions v ON v.id = r.version_id
+                    +        WHERE s.sample_type_id = ? AND r.last_observation_id IS NULL
+                    +        """
+                    +        return try withStatement(sql, db: db) { statement in
+                    +            bindInt64(sampleTypeID, to: 1, in: statement)
+                    +            guard sqlite3_step(statement) == SQLITE_ROW else {
+                    +                return ArchiveV2VisibleAggregate(
+                    +                    visibleRecordCount: 0,
+                    +                    earliestStartDate: nil,
+                    +                    latestEndDate: nil,
+                    +                    valueSum: nil,
+                    +                    valueMax: nil
+                    +                )
+                    +            }
+                    +            return ArchiveV2VisibleAggregate(
+                    +                visibleRecordCount: columnInt(statement, 0) ?? 0,
+                    +                earliestStartDate: columnDouble(statement, 1),
+                    +                latestEndDate: columnDouble(statement, 2),
+                    +                valueSum: columnDouble(statement, 3),
+                    +                valueMax: columnDouble(statement, 4)
+                    +    private func requiredInt64(
+                    +        _ sql: String,
+                    +        db: OpaquePointer?,
+                    +        bind: (OpaquePointer?) throws -> Void
+                    +    ) throws -> Int64 {
+                    +        guard let value = try optionalInt64(sql, db: db, bind: bind) else {
+                    +            throw SQLiteHealthArchiveStoreError.stepFailed("missing required row")
+                    +        }
+                    +        return value
+                    +    }
+                    +    private func optionalInt64(
+                    +        _ sql: String,
+                    +        db: OpaquePointer?,
+                    +        bind: (OpaquePointer?) throws -> Void
+                    +    ) throws -> Int64? {
+                    +        try withStatement(sql, db: db) { statement in
+                    +            try bind(statement)
+                    +            guard sqlite3_step(statement) == SQLITE_ROW else { return nil }
+                    +            return sqlite3_column_int64(statement, 0)
+                    +        }
+                    +    }
+                         private func execute(_ sql: String, db: OpaquePointer?) throws {
+                             guard sqlite3_exec(db, sql, nil, nil, nil) == SQLITE_OK else {
+                                 throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
+                         let typeIdentifier: String
+                         let strictFingerprint: String
+                         let semanticFingerprint: String
+                    +    let payloadHash: String
+                         let startDate: Date
+                         let endDate: Date
+                         let observedAt: Date
+                         let deviceLocalIdentifier: String?
+                         let deviceUDI: String?
+                         let metadataJSON: String?
+                    +    let metadataHash: String?
+                    +    nonisolated var hasDeviceProvenance: Bool {
+                    +        deviceName != nil ||
+                    +        deviceManufacturer != nil ||
+                    +        deviceModel != nil ||
+                    +        deviceHardwareVersion != nil ||
+                    +        deviceFirmwareVersion != nil ||
+                    +        deviceSoftwareVersion != nil ||
+                    +        deviceLocalIdentifier != nil ||
+                    +        deviceUDI != nil
+                    +    }
+                         nonisolated init(sample: HKSample, observedAt: Date) {
+                             let sampleUUID = sample.uuid.uuidString
+                             let workout = sample as? HKWorkout
+                             let sourceRevision = sample.sourceRevision
+                             let device = sample.device
+                    +        let valueKind = quantity?.kind ?? (category == nil ? (workout == nil ? nil : "workout") : "category")
+                    +        let numericValue = quantity?.value
+                    +        let unit = quantity?.unit
+                    +        let categoryValue = category?.value
+                    +        let workoutActivityType = workout.map { Int($0.workoutActivityType.rawValue) }
+                    +        let durationSeconds = workout?.duration
+                    +        let sourceOperatingSystemVersion = ArchiveSampleRow.operatingSystemVersionString(sourceRevision.operatingSystemVersion)
+                    +        let metadataJSON = ArchiveSampleRow.metadataJSONString(sample.metadata)
+                    +        let metadataHash = metadataJSON.map { HashService.archiveContentHash(domain: "hp:v2:metadata", parts: [$0]) }
+                             self.sampleUUIDHash = HashService.sampleUUIDHash(sampleUUID)
+                             self.typeIdentifier = typeIdentifier
+                                 typeIdentifier: typeIdentifier,
+                                 startDate: sample.startDate,
+                                 endDate: sample.endDate,
+                    -            value: quantity?.value,
+                    -            unit: quantity?.unit,
+                    -            categoryValue: category?.value,
+                    +            value: numericValue,
+                    +            unit: unit,
+                    +            categoryValue: categoryValue,
+                                 workoutActivityType: workout?.workoutActivityType.rawValue,
+                                 sourceBundleIdentifier: sourceRevision.source.bundleIdentifier
+                    -        self.startDate = sample.startDate
+                    -        self.endDate = sample.endDate
+                    -        self.observedAt = observedAt
+                    -        self.valueKind = quantity?.kind ?? (category == nil ? (workout == nil ? nil : "workout") : "category")
+                    -        self.value = quantity?.value
+                    -        self.unit = quantity?.unit
+                    -        self.categoryValue = category?.value
+                    -        self.workoutActivityType = workout.map { Int($0.workoutActivityType.rawValue) }
+                    -        self.durationSeconds = workout?.duration
+                    +        self.valueKind = valueKind
+                    +        self.value = numericValue
+                    +        self.unit = unit
+                    +        self.categoryValue = categoryValue
+                    +        self.workoutActivityType = workoutActivityType
+                    +        self.durationSeconds = durationSeconds
+                             self.sourceName = sourceRevision.source.name
+                             self.sourceBundleIdentifier = sourceRevision.source.bundleIdentifier
+                             self.sourceProductType = sourceRevision.productType
+                             self.sourceVersion = sourceRevision.version
+                    -        self.sourceOperatingSystemVersion = ArchiveSampleRow.operatingSystemVersionString(sourceRevision.operatingSystemVersion)
+                    +        self.sourceOperatingSystemVersion = sourceOperatingSystemVersion
+                             self.deviceName = device?.name
+                             self.deviceManufacturer = device?.manufacturer
+                             self.deviceModel = device?.model
+                             self.deviceSoftwareVersion = device?.softwareVersion
+                             self.deviceLocalIdentifier = device?.localIdentifier
+                             self.deviceUDI = device?.udiDeviceIdentifier
+                    -        self.metadataJSON = ArchiveSampleRow.metadataJSONString(sample.metadata)
+                    +        self.metadataJSON = metadataJSON
+                    +        self.metadataHash = metadataHash
+                    +        self.payloadHash = HashService.archiveContentHash(
+                    +            domain: "hp:v2:payload",
+                    +            parts: [
+                    +                typeIdentifier,
+                    +                ArchiveSampleRow.timestampString(sample.startDate),
+                    +                ArchiveSampleRow.timestampString(sample.endDate),
+                    +                valueKind,
+                    +                numericValue.map { String(format: "%.17g", $0) },
+                    +                unit,
+                    +                categoryValue.map(String.init),
+                    +                workoutActivityType.map(String.init),
+                    +                durationSeconds.map { String(format: "%.17g", $0) },
+                    +                sourceRevision.source.bundleIdentifier,
+                    +                sourceRevision.productType,
+                    +                sourceRevision.version,
+                    +                ArchiveSampleRow.operatingSystemVersionString(sourceRevision.operatingSystemVersion),
+                    +                device?.model,
+                    +                device?.hardwareVersion,
+                    +                device?.firmwareVersion,
+                    +                device?.softwareVersion,
+                    +                device?.localIdentifier,
+                    +                device?.udiDeviceIdentifier,
+                    +                metadataHash
+                    +            ]
+                    +        )
+                    +        self.startDate = sample.startDate
+                    +        self.endDate = sample.endDate
+                    +        self.observedAt = observedAt
+                    +    }
+                    +    nonisolated private static func timestampString(_ date: Date) -> String {
+                    +        String(format: "%.6f", date.timeIntervalSince1970)
+                         nonisolated private static func quantityPayload(_ sample: HKSample) -> (kind: String, value: Double, unit: String)? {
+                    +private enum ArchiveV2SampleWriteKind {
+                    +    case inserted
+                    +    case updated
+                    +    case unchanged
+                    +}
+                    +private struct ArchiveV2SampleWriteResult {
+                    +    let sampleTypeID: Int64
+                    +    let kind: ArchiveV2SampleWriteKind
+                    +}
+                    +private struct ArchiveV2TypeSummary {
+                    +    let visibleRecordCount: Int
+                    +    let appearedCount: Int
+                    +    let disappearedCount: Int
+                    +    let representationChangedCount: Int
+                    +    let earliestStartDate: Double?
+                    +    let latestEndDate: Double?
+                    +    let valueSum: Double?
+                    +    let valueMax: Double?
+                    +}
+                    +private struct ArchiveV2VisibleAggregate {
+                    +    let visibleRecordCount: Int
+                    +    let earliestStartDate: Double?
+                    +    let latestEndDate: Double?
+                    +    let valueSum: Double?
+                    +    let valueMax: Double?
+                    +}
+                     nonisolated private struct HealthArchiveReportPayload: Encodable {
+                         let reportID: UUID
+                         let title: String
+                         sqlite3_bind_int64(statement, index, sqlite3_int64(value))
+                    +nonisolated private func bindInt64(_ value: Int64?, to index: Int32, in statement: OpaquePointer?) {
+                    +    guard let value else {
+                    +        sqlite3_bind_null(statement, index)
+                    +        return
+                    +    }
+                    +    sqlite3_bind_int64(statement, index, sqlite3_int64(value))
+                    +}
+                     nonisolated private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? {
+                         guard sqlite3_column_type(statement, index) != SQLITE_NULL,
+                               let pointer = sqlite3_column_text(statement, index) else {

	@@ -25,7 +25,7 @@ There are no real deployments, only test installations. Existing prototype datab
25	25	\|------\|----------------\|--------------------\|
26	26	\| Product docs \| Updated \| Keep `HealthProbe/Doc/README.md` as canonical index \|
27	27	\| HealthKit capture \| Prototype exists \| Adapt capture to write differential SQLite observations first \|
28		-\| SQLite archive \| Archive v2 schema bootstrap exists; legacy write table still active \| Move write path from `archive_samples` to observations/samples/versions/events/ranges \|
	28	+\| SQLite archive \| Archive v2 schema and differential write path partially implemented; legacy read table still active \| Add daily aggregates, SQL reads, integrity tests, then retire `archive_samples` \|
29	29	\| Core Data cache \| Not implemented \| Add rebuildable cache for expensive counts, summaries, report metadata, UI state \|
30	30	\| SwiftData cache \| Exists \| Treat as disposable prototype data; reset/ignore during v2 transition \|
31	31	\| UI \| Prototype exists \| Reframe screens around observations, diffs, export, archive status \|
	@@ -38,14 +38,15 @@ There are no real deployments, only test installations. Existing prototype datab
38	38
39	39	Detailed checkable milestones live in [`Refactoring-Plan.md`](Refactoring-Plan.md).
40	40
41		-1. Implement differential write path: observations, samples, payload versions, events/ranges, aggregates.
	41	+1. Finish differential write path: daily aggregates and stricter retry/idempotency tests.
42	42	2. Add SQLite integrity/open/schema-version tests.
43		-3. Move large diffs/counts into SQL queries with indexes/temp tables/paged results.
44		-4. Add Core Data UI/report cache and rebuild pipeline.
45		-5. Replace SwiftData UI dependencies with Core Data/cache DTOs.
46		-6. Update UI language from anomaly/status to observation/diff/export.
47		-7. Add streaming exports with manifests.
48		-8. Validate on low-memory/legacy-class devices.
	43	+3. Move archive reads from `archive_samples` to SQL over visibility ranges and sample versions.
	44	+4. Move large diffs/counts into SQL queries with indexes/temp tables/paged results.
	45	+5. Add Core Data UI/report cache and rebuild pipeline.
	46	+6. Replace SwiftData UI dependencies with Core Data/cache DTOs.
	47	+7. Update UI language from anomaly/status to observation/diff/export.
	48	+8. Add streaming exports with manifests.
	49	+9. Validate on low-memory/legacy-class devices.
49	50
50	51	## Known Prototype Mismatches
51	52

	@@ -20,17 +20,23 @@ enum HashService {
20	20	}
21	21	}
22	22
23		- private static let iso8601Formatter: ISO8601DateFormatter = {
	23	+ nonisolated(unsafe) private static let iso8601Formatter: ISO8601DateFormatter = {
24	24	let f = ISO8601DateFormatter()
25	25	f.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
26	26	return f
27	27	}()
28	28
	29	+ nonisolated static func archiveContentHash(domain: String, parts: [String?]) -> String {
	30	+ let input = ([domain] + parts.map { $0 ?? "null" }).joined(separator: "\u{1f}")
	31	+ let digest = SHA256.hash(data: Data(input.utf8))
	32	+ return digest.map { String(format: "%02x", $0) }.joined()
	33	+ }
	34	+
29	35	// SHA256 of "typeIdentifier\|totalCount\|earliestDateISO\|latestDateISO"
30	36	// ⚠️ MVP limitation: hash covers only count + date range, not value distribution.
31	37	// silentReplacement detection based on this hash is best-effort only —
32	38	// it will miss replacements that preserve total count and date boundaries.
33		- static func typeHash(
	39	+ nonisolated static func typeHash(
34	40	typeIdentifier: String,
35	41	totalCount: Int,
36	42	earliestDate: Date?,
	@@ -43,7 +49,7 @@ enum HashService {
43	49	return digest.map { String(format: "%02x", $0) }.joined()
44	50	}
45	51
46		- static func typeHash(typeIdentifier: String, recordFingerprints: [String]) -> String {
	52	+ nonisolated static func typeHash(typeIdentifier: String, recordFingerprints: [String]) -> String {
47	53	var hasher = SHA256()
48	54	hasher.update(data: Data(typeIdentifier.utf8))
49	55	for fingerprint in recordFingerprints.sorted() {
	@@ -54,7 +60,7 @@ enum HashService {
54	60	return digest.map { String(format: "%02x", $0) }.joined()
55	61	}
56	62
57		- static func typeHashInRecordedOrder(typeIdentifier: String, recordFingerprints: [String]) -> String {
	63	+ nonisolated static func typeHashInRecordedOrder(typeIdentifier: String, recordFingerprints: [String]) -> String {
58	64	var builder = TypeHashBuilder(typeIdentifier: typeIdentifier)
59	65	for fingerprint in recordFingerprints {
60	66	builder.append(recordFingerprint: fingerprint)
	@@ -62,7 +68,7 @@ enum HashService {
62	68	return builder.finalize()
63	69	}
64	70
65		- static func sampleFingerprint(
	71	+ nonisolated static func sampleFingerprint(
66	72	typeIdentifier: String,
67	73	sampleUUID: String,
68	74	startDate: Date,
	@@ -78,12 +84,12 @@ enum HashService {
78	84	return digest.map { String(format: "%02x", $0) }.joined()
79	85	}
80	86
81		- static func sampleUUIDHash(_ sampleUUID: String) -> String {
	87	+ nonisolated static func sampleUUIDHash(_ sampleUUID: String) -> String {
82	88	let digest = SHA256.hash(data: Data(sampleUUID.utf8))
83	89	return digest.map { String(format: "%02x", $0) }.joined()
84	90	}
85	91
86		- static func archiveSemanticFingerprint(
	92	+ nonisolated static func archiveSemanticFingerprint(
87	93	typeIdentifier: String,
88	94	startDate: Date,
89	95	endDate: Date,
	@@ -124,7 +130,7 @@ enum HashService {
124	130	// ⚠️ Covers the FULL intended registry (selectedTypeIDs), including types that may have
125	131	// failed, timed out, or been unauthorized — never filter down to only the successfully-
126	132	// fetched subset. A query failure must not silently change the registry hash.
127		- static func typeSetHash(typeIDs: [String]) -> String {
	133	+ nonisolated static func typeSetHash(typeIDs: [String]) -> String {
128	134	let sorted = typeIDs.sorted().joined(separator: "\|")
129	135	let digest = SHA256.hash(data: Data(sorted.utf8))
130	136	return digest.map { String(format: "%02x", $0) }.joined()

	@@ -5,9 +5,9 @@ import os.log
5	5	private let logger = Logger(subsystem: "ro.xdev.healthprobe", category: "KeychainService")
6	6
7	7	enum KeychainService {
8		- private static let service = "ro.xdev.healthprobe.deviceid"
9		- private static let account = "stable_device_id"
10		- private static var cached: String?
	8	+ nonisolated private static let service = "ro.xdev.healthprobe.deviceid"
	9	+ nonisolated private static let account = "stable_device_id"
	10	+ nonisolated(unsafe) private static var cached: String?
11	11
12	12	struct Resolution {
13	13	let id: String
	@@ -17,7 +17,7 @@ enum KeychainService {
17	17	// If swiftDataStoreIsEmpty is true and Keychain has an existing ID, the DB was wiped
18	18	// (reinstall or manual reset) but Keychain survived. We keep the same deviceID but signal
19	19	// that a new chain must start and is marked recovered.
20		- static func resolveDeviceID(swiftDataStoreIsEmpty: Bool) -> Resolution {
	20	+ nonisolated static func resolveDeviceID(swiftDataStoreIsEmpty: Bool) -> Resolution {
21	21	if let existing = readFromKeychain() {
22	22	if swiftDataStoreIsEmpty {
23	23	// DB was wiped but Keychain survived — recovered device ID
	@@ -30,7 +30,7 @@ enum KeychainService {
30	30	return Resolution(id: new, isRecovered: false)
31	31	}
32	32
33		- private static func readFromKeychain() -> String? {
	33	+ nonisolated private static func readFromKeychain() -> String? {
34	34	if let cached { return cached }
35	35	let query: [String: Any] = [
36	36	kSecClass as String: kSecClassGenericPassword,
	@@ -50,7 +50,7 @@ enum KeychainService {
50	50	return string
51	51	}
52	52
53		- private static func writeToKeychain(_ value: String) {
	53	+ nonisolated private static func writeToKeychain(_ value: String) {
54	54	guard let data = value.data(using: .utf8) else { return }
55	55	let attributes: [String: Any] = [
56	56	kSecClass as String: kSecClassGenericPassword,