Showing 2 changed files with 197 additions and 62 deletions
+2 -1
HealthProbe/Doc/04-project/IMPLEMENTATION_STATUS.md
@@ -25,7 +25,7 @@ There are no real deployments, only test installations. Existing prototype datab
25 25
 |------|----------------|--------------------|
26 26
 | Product docs | Updated | Keep `HealthProbe/Doc/README.md` as canonical index |
27 27
 | HealthKit capture | Capture now opens one archive observation per user-visible snapshot, attaches HealthKit pages, deleted-object evidence, and type verification to that observation id before finishing it, no longer aborts initial full-history imports after a fixed 30-minute wall-clock cap while page-level HealthKit timeouts remain in place, defers grouped observation summary/daily aggregate rebuilds until per-type verification instead of rebuilding after every imported page, and persists large HealthKit pages in smaller archive chunks while using a smaller anchored-query page size to avoid long monolithic SQLite stalls | Continue moving UI/cache reads to archive-backed observation ids and revisit fully adaptive paging/background collection separately |
28
-| SQLite archive | Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed | Continue moving capture/Dashboard actions to archive/cache DTOs |
28
+| SQLite archive | Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed, and the hot write path now reuses prepared SQLite statements within grouped page writes instead of reparsing the same SQL for every sample | Continue moving capture/Dashboard actions to archive/cache DTOs |
29 29
 | Core Data cache | Initial programmatic Core Data model, full-cache rebuild service, read DTOs for observation/type/diff/health rows, and Dashboard archive-cache status wiring are in place | Move remaining export/report paths to cache DTOs and add targeted partial invalidation |
30 30
 | SwiftData cache | Exists; test builds now reset legacy prototype UI/archive/cache stores once for archive v2 so old SwiftData-only snapshots are not treated as backed-up observations. Metric timeout calibration, local device profile settings, operation logging, ContentView preview, Settings data maintenance, legacy detail/PDF views, unused legacy repair/observer services, Dashboard view/view-model access, and legacy anomaly/count-drop review have moved outside SwiftData or been removed. Remaining SwiftData imports are inventoried in [`SwiftData-Retirement-Inventory.md`](SwiftData-Retirement-Inventory.md) | Treat as disposable prototype data; stop returning/storing `HealthSnapshot` bridge handles before removing `ModelContainer` |
31 31
 | UI | Prototype exists; Dashboard status reads archive/cache observation rows and shows cache health, and Dashboard view/view-model code no longer imports SwiftData or reads `ModelContext`; capture/review actions now route through DTOs and snapshot ids, with the remaining legacy bridge isolated in `HealthKitService`. Snapshots and Data Types tab roots no longer import SwiftData, load Core Data cached observation rows, and open archive/cache-backed detail rows; `SnapshotArchiveDetailView` and `DataTypeArchiveDetailView` read Core Data type/diff summaries and page record drill-down through SQLite; unused legacy SwiftData snapshot/type detail and PDF views have been deleted; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language | Stop writing prototype `HealthSnapshot` bridge rows during capture/review |
@@ -52,6 +52,7 @@ Detailed checkable milestones live in [`Refactoring-Plan.md`](Refactoring-Plan.m
52 52
 - Snapshots timeline/detail rows, Data Types list/detail rows, and record drill-down are archive/cache-backed for new archive v2 observations when cache rows exist.
53 53
 - Legacy SwiftData-only snapshots are reset for archive v2 test installs rather than migrated.
54 54
 - Capture strategy and some legacy SwiftData transition paths may still decode or cache too much data for low-end devices.
55
+- Very large first-run HealthKit imports may still require adaptive paging, retryable partial progress, and background-friendly collection beyond the current smaller pages, chunked persistence, and prepared-statement reuse.
55 56
 - Old prototype database compatibility is no longer required.
56 57
 - Initial SQLite archive tests cover open/init/reset/idempotency, snapshot-level observation grouping, legacy mirror removal, small observation diffs, large synthetic diff pagination, formal timing/memory metrics, materialized aggregate comparison, source/provenance breakdowns, consolidation-evidence labels, export preview, paged JSON output, and manifest row persistence.
57 58
 - Initial Core Data cache tests cover full rebuild from SQLite and delete-cache-then-rebuild without losing archive data.
+195 -61
HealthProbe/Services/SQLiteHealthArchiveStore.swift
@@ -1808,6 +1808,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1808 1808
         db: OpaquePointer?,
1809 1809
         rebuildDerivedState: Bool
1810 1810
     ) throws -> HealthArchiveWriteSummary {
1811
+        let statementCache = SQLiteStatementCache(db: db)
1812
+        defer { statementCache.finalizeAll() }
1811 1813
         let rows = samples.map { ArchiveSampleRow(sample: $0, observedAt: observedAt) }
1812 1814
         var inserted = 0
1813 1815
         var updated = 0
@@ -1815,7 +1817,12 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1815 1817
         var typeEventCounts: [Int64: (inserted: Int, deleted: Int)] = [:]
1816 1818
 
1817 1819
         for row in rows {
1818
-            let result = try upsertArchiveV2Sample(row, observationID: observationID, db: db)
1820
+            let result = try upsertArchiveV2Sample(
1821
+                row,
1822
+                observationID: observationID,
1823
+                db: db,
1824
+                statementCache: statementCache
1825
+            )
1819 1826
             var counts = typeEventCounts[result.sampleTypeID, default: (inserted: 0, deleted: 0)]
1820 1827
             switch result.kind {
1821 1828
             case .inserted:
@@ -1839,7 +1846,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1839 1846
                 insertedEventCount: eventCounts.inserted,
1840 1847
                 deletedEventCount: eventCounts.deleted,
1841 1848
                 verifiedVisibleCount: nil,
1842
-                db: db
1849
+                db: db,
1850
+                statementCache: statementCache
1843 1851
             )
1844 1852
             if rebuildDerivedState {
1845 1853
                 try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
@@ -1865,7 +1873,13 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1865 1873
         observationID: Int64,
1866 1874
         db: OpaquePointer?
1867 1875
     ) throws {
1868
-        let sampleTypeID = try upsertSampleType(typeIdentifier: sampleType.identifier, db: db)
1876
+        let statementCache = SQLiteStatementCache(db: db)
1877
+        defer { statementCache.finalizeAll() }
1878
+        let sampleTypeID = try upsertSampleType(
1879
+            typeIdentifier: sampleType.identifier,
1880
+            db: db,
1881
+            statementCache: statementCache
1882
+        )
1869 1883
         let visibleCount = try visibleAggregate(sampleTypeID: sampleTypeID, db: db).visibleRecordCount
1870 1884
         try insertObservationTypeRun(
1871 1885
             observationID: observationID,
@@ -1875,7 +1889,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1875 1889
             insertedEventCount: 0,
1876 1890
             deletedEventCount: 0,
1877 1891
             verifiedVisibleCount: visibleCount,
1878
-            db: db
1892
+            db: db,
1893
+            statementCache: statementCache
1879 1894
         )
1880 1895
         try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
1881 1896
         try rebuildDailyAggregates(
@@ -1894,8 +1909,19 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1894 1909
         db: OpaquePointer?,
1895 1910
         rebuildDerivedState: Bool
1896 1911
     ) throws {
1897
-        guard let sampleTypeID = try sampleTypeID(typeIdentifier: sampleTypeIdentifier, db: db),
1898
-              let sampleID = try sampleID(sampleUUIDHash: sampleUUIDHash, sampleTypeID: sampleTypeID, db: db) else {
1912
+        let statementCache = SQLiteStatementCache(db: db)
1913
+        defer { statementCache.finalizeAll() }
1914
+        guard let sampleTypeID = try sampleTypeID(
1915
+                typeIdentifier: sampleTypeIdentifier,
1916
+                db: db,
1917
+                statementCache: statementCache
1918
+              ),
1919
+              let sampleID = try sampleID(
1920
+                sampleUUIDHash: sampleUUIDHash,
1921
+                sampleTypeID: sampleTypeID,
1922
+                db: db,
1923
+                statementCache: statementCache
1924
+              ) else {
1899 1925
             return
1900 1926
         }
1901 1927
 
@@ -1906,14 +1932,16 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1906 1932
             eventKind: "disappeared",
1907 1933
             evidenceKind: "deleted_object",
1908 1934
             observedAt: observedMissingAt,
1909
-            db: db
1935
+            db: db,
1936
+            statementCache: statementCache
1910 1937
         )
1911 1938
         try closeOpenVisibilityRanges(
1912 1939
             sampleID: sampleID,
1913 1940
             excludingVersionID: nil,
1914 1941
             closedAtObservationID: observationID,
1915 1942
             observedAt: observedMissingAt,
1916
-            db: db
1943
+            db: db,
1944
+            statementCache: statementCache
1917 1945
         )
1918 1946
         try insertObservationTypeRun(
1919 1947
             observationID: observationID,
@@ -1923,7 +1951,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1923 1951
             insertedEventCount: 0,
1924 1952
             deletedEventCount: 1,
1925 1953
             verifiedVisibleCount: nil,
1926
-            db: db
1954
+            db: db,
1955
+            statementCache: statementCache
1927 1956
         )
1928 1957
         if rebuildDerivedState {
1929 1958
             try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
@@ -1939,13 +1968,20 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1939 1968
     private func upsertArchiveV2Sample(
1940 1969
         _ row: ArchiveSampleRow,
1941 1970
         observationID: Int64,
1942
-        db: OpaquePointer?
1971
+        db: OpaquePointer?,
1972
+        statementCache: SQLiteStatementCache
1943 1973
     ) throws -> ArchiveV2SampleWriteResult {
1944
-        let sampleTypeID = try upsertSampleType(typeIdentifier: row.typeIdentifier, db: db)
1945
-        let sourceRevisionID = try upsertSourceRevision(row, db: db)
1946
-        let deviceID = try upsertDevice(row, db: db)
1947
-        let metadataID = try upsertMetadataBlob(row, db: db)
1948
-        let sampleResult = try upsertSample(row, sampleTypeID: sampleTypeID, observationID: observationID, db: db)
1974
+        let sampleTypeID = try upsertSampleType(typeIdentifier: row.typeIdentifier, db: db, statementCache: statementCache)
1975
+        let sourceRevisionID = try upsertSourceRevision(row, db: db, statementCache: statementCache)
1976
+        let deviceID = try upsertDevice(row, db: db, statementCache: statementCache)
1977
+        let metadataID = try upsertMetadataBlob(row, db: db, statementCache: statementCache)
1978
+        let sampleResult = try upsertSample(
1979
+            row,
1980
+            sampleTypeID: sampleTypeID,
1981
+            observationID: observationID,
1982
+            db: db,
1983
+            statementCache: statementCache
1984
+        )
1949 1985
         let versionResult = try upsertSampleVersion(
1950 1986
             row,
1951 1987
             sampleID: sampleResult.id,
@@ -1953,7 +1989,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1953 1989
             deviceID: deviceID,
1954 1990
             metadataID: metadataID,
1955 1991
             observationID: observationID,
1956
-            db: db
1992
+            db: db,
1993
+            statementCache: statementCache
1957 1994
         )
1958 1995
 
1959 1996
         let writeKind: ArchiveV2SampleWriteKind
@@ -1976,21 +2013,24 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1976 2013
             eventKind: eventKind,
1977 2014
             evidenceKind: "healthkit_sample",
1978 2015
             observedAt: row.observedAt,
1979
-            db: db
2016
+            db: db,
2017
+            statementCache: statementCache
1980 2018
         )
1981 2019
         try closeOpenVisibilityRanges(
1982 2020
             sampleID: sampleResult.id,
1983 2021
             excludingVersionID: versionResult.id,
1984 2022
             closedAtObservationID: observationID,
1985 2023
             observedAt: row.observedAt,
1986
-            db: db
2024
+            db: db,
2025
+            statementCache: statementCache
1987 2026
         )
1988 2027
         try insertOpenVisibilityRangeIfNeeded(
1989 2028
             sampleID: sampleResult.id,
1990 2029
             versionID: versionResult.id,
1991 2030
             observationID: observationID,
1992 2031
             observedAt: row.observedAt,
1993
-            db: db
2032
+            db: db,
2033
+            statementCache: statementCache
1994 2034
         )
1995 2035
 
1996 2036
         return ArchiveV2SampleWriteResult(sampleTypeID: sampleTypeID, kind: writeKind)
@@ -2059,12 +2099,14 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2059 2099
         insertedEventCount: Int,
2060 2100
         deletedEventCount: Int,
2061 2101
         verifiedVisibleCount: Int?,
2062
-        db: OpaquePointer?
2102
+        db: OpaquePointer?,
2103
+        statementCache: SQLiteStatementCache? = nil
2063 2104
     ) throws {
2064 2105
         if let existing = try observationTypeRunCounts(
2065 2106
             observationID: observationID,
2066 2107
             sampleTypeID: sampleTypeID,
2067
-            db: db
2108
+            db: db,
2109
+            statementCache: statementCache
2068 2110
         ) {
2069 2111
             try withStatement(
2070 2112
                 """
@@ -2078,7 +2120,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2078 2120
                 WHERE observation_id = ? AND sample_type_id = ?
2079 2121
                 """,
2080 2122
                 db: db
2081
-            ) { statement in
2123
+            , statementCache: statementCache) { statement in
2082 2124
                 bindText(status, to: 1, in: statement)
2083 2125
                 sqlite3_bind_double(statement, 2, observedAt.timeIntervalSince1970)
2084 2126
                 sqlite3_bind_double(statement, 3, observedAt.timeIntervalSince1970)
@@ -2098,7 +2140,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2098 2140
                 inserted_event_count, deleted_event_count, verified_visible_count
2099 2141
             ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
2100 2142
             """
2101
-            try withStatement(sql, db: db) { statement in
2143
+            try withStatement(sql, db: db, statementCache: statementCache) { statement in
2102 2144
                 bindInt64(observationID, to: 1, in: statement)
2103 2145
                 bindInt64(sampleTypeID, to: 2, in: statement)
2104 2146
                 bindText(status, to: 3, in: statement)
@@ -2117,7 +2159,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2117 2159
     private func observationTypeRunCounts(
2118 2160
         observationID: Int64,
2119 2161
         sampleTypeID: Int64,
2120
-        db: OpaquePointer?
2162
+        db: OpaquePointer?,
2163
+        statementCache: SQLiteStatementCache? = nil
2121 2164
     ) throws -> (inserted: Int, deleted: Int)? {
2122 2165
         try withStatement(
2123 2166
             """
@@ -2126,7 +2169,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2126 2169
             WHERE observation_id = ? AND sample_type_id = ?
2127 2170
             LIMIT 1
2128 2171
             """,
2129
-            db: db
2172
+            db: db,
2173
+            statementCache: statementCache
2130 2174
         ) { statement in
2131 2175
             bindInt64(observationID, to: 1, in: statement)
2132 2176
             bindInt64(sampleTypeID, to: 2, in: statement)
@@ -2157,44 +2201,60 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2157 2201
         }
2158 2202
     }
2159 2203
 
2160
-    private func upsertSampleType(typeIdentifier: String, db: OpaquePointer?) throws -> Int64 {
2204
+    private func upsertSampleType(typeIdentifier: String, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64 {
2161 2205
         try withStatement(
2162 2206
             "INSERT OR IGNORE INTO sample_types (type_identifier, display_name, category) VALUES (?, NULL, NULL)",
2163
-            db: db
2207
+            db: db,
2208
+            statementCache: statementCache
2164 2209
         ) { statement in
2165 2210
             bindText(typeIdentifier, to: 1, in: statement)
2166 2211
             guard sqlite3_step(statement) == SQLITE_DONE else {
2167 2212
                 throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
2168 2213
             }
2169 2214
         }
2170
-        return try requiredInt64("SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1", db: db) { statement in
2215
+        return try requiredInt64(
2216
+            "SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1",
2217
+            db: db,
2218
+            statementCache: statementCache
2219
+        ) { statement in
2171 2220
             bindText(typeIdentifier, to: 1, in: statement)
2172 2221
         }
2173 2222
     }
2174 2223
 
2175
-    private func sampleTypeID(typeIdentifier: String, db: OpaquePointer?) throws -> Int64? {
2176
-        try optionalInt64("SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1", db: db) { statement in
2224
+    private func sampleTypeID(typeIdentifier: String, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64? {
2225
+        try optionalInt64(
2226
+            "SELECT id FROM sample_types WHERE type_identifier = ? LIMIT 1",
2227
+            db: db,
2228
+            statementCache: statementCache
2229
+        ) { statement in
2177 2230
             bindText(typeIdentifier, to: 1, in: statement)
2178 2231
         }
2179 2232
     }
2180 2233
 
2181
-    private func upsertSourceRevision(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
2234
+    private func upsertSourceRevision(_ row: ArchiveSampleRow, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64? {
2182 2235
         guard row.sourceName != nil || row.sourceBundleIdentifier != nil else { return nil }
2183 2236
         let sourceNameHash = row.sourceName.map { HashService.archiveContentHash(domain: "hp:v2:source_name", parts: [$0]) }
2184
-        let sourceID = try upsertSource(sourceNameHash: sourceNameHash, bundleIdentifier: row.sourceBundleIdentifier, db: db)
2237
+        let sourceID = try upsertSource(
2238
+            sourceNameHash: sourceNameHash,
2239
+            bundleIdentifier: row.sourceBundleIdentifier,
2240
+            db: db,
2241
+            statementCache: statementCache
2242
+        )
2185 2243
         if let existing = try sourceRevisionID(
2186 2244
             sourceID: sourceID,
2187 2245
             productType: row.sourceProductType,
2188 2246
             version: row.sourceVersion,
2189 2247
             operatingSystemVersion: row.sourceOperatingSystemVersion,
2190
-            db: db
2248
+            db: db,
2249
+            statementCache: statementCache
2191 2250
         ) {
2192 2251
             return existing
2193 2252
         }
2194 2253
 
2195 2254
         try withStatement(
2196 2255
             "INSERT INTO source_revisions (source_id, product_type, version, operating_system_version) VALUES (?, ?, ?, ?)",
2197
-            db: db
2256
+            db: db,
2257
+            statementCache: statementCache
2198 2258
         ) { statement in
2199 2259
             bindInt64(sourceID, to: 1, in: statement)
2200 2260
             bindText(row.sourceProductType, to: 2, in: statement)
@@ -2212,7 +2272,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2212 2272
         productType: String?,
2213 2273
         version: String?,
2214 2274
         operatingSystemVersion: String?,
2215
-        db: OpaquePointer?
2275
+        db: OpaquePointer?,
2276
+        statementCache: SQLiteStatementCache? = nil
2216 2277
     ) throws -> Int64? {
2217 2278
         try optionalInt64(
2218 2279
             """
@@ -2223,7 +2284,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2223 2284
               AND (operating_system_version = ? OR (operating_system_version IS NULL AND ? IS NULL))
2224 2285
             LIMIT 1
2225 2286
             """,
2226
-            db: db
2287
+            db: db,
2288
+            statementCache: statementCache
2227 2289
         ) { statement in
2228 2290
             bindInt64(sourceID, to: 1, in: statement)
2229 2291
             bindText(productType, to: 2, in: statement)
@@ -2235,7 +2297,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2235 2297
         }
2236 2298
     }
2237 2299
 
2238
-    private func upsertSource(sourceNameHash: String?, bundleIdentifier: String?, db: OpaquePointer?) throws -> Int64 {
2300
+    private func upsertSource(sourceNameHash: String?, bundleIdentifier: String?, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64 {
2239 2301
         if let existing = try optionalInt64(
2240 2302
             """
2241 2303
             SELECT id FROM sources
@@ -2244,6 +2306,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2244 2306
             LIMIT 1
2245 2307
             """,
2246 2308
             db: db,
2309
+            statementCache: statementCache,
2247 2310
             bind: { statement in
2248 2311
                 bindText(sourceNameHash, to: 1, in: statement)
2249 2312
                 bindText(sourceNameHash, to: 2, in: statement)
@@ -2253,7 +2316,11 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2253 2316
         ) {
2254 2317
             return existing
2255 2318
         }
2256
-        try withStatement("INSERT INTO sources (source_name_hash, bundle_identifier) VALUES (?, ?)", db: db) { statement in
2319
+        try withStatement(
2320
+            "INSERT INTO sources (source_name_hash, bundle_identifier) VALUES (?, ?)",
2321
+            db: db,
2322
+            statementCache: statementCache
2323
+        ) { statement in
2257 2324
             bindText(sourceNameHash, to: 1, in: statement)
2258 2325
             bindText(bundleIdentifier, to: 2, in: statement)
2259 2326
             guard sqlite3_step(statement) == SQLITE_DONE else {
@@ -2263,7 +2330,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2263 2330
         return sqlite3_last_insert_rowid(db)
2264 2331
     }
2265 2332
 
2266
-    private func upsertDevice(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
2333
+    private func upsertDevice(_ row: ArchiveSampleRow, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64? {
2267 2334
         guard row.hasDeviceProvenance else { return nil }
2268 2335
         let deviceHash = row.deviceName.map { HashService.archiveContentHash(domain: "hp:v2:device_name", parts: [$0]) }
2269 2336
         let manufacturerHash = row.deviceManufacturer.map { HashService.archiveContentHash(domain: "hp:v2:device_manufacturer", parts: [$0]) }
@@ -2280,6 +2347,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2280 2347
             LIMIT 1
2281 2348
             """,
2282 2349
             db: db,
2350
+            statementCache: statementCache,
2283 2351
             bind: { statement in
2284 2352
                 bindText(deviceHash, to: 1, in: statement)
2285 2353
                 bindText(deviceHash, to: 2, in: statement)
@@ -2301,7 +2369,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2301 2369
                 software_version, local_identifier_hash, udi_hash
2302 2370
             ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
2303 2371
             """,
2304
-            db: db
2372
+            db: db,
2373
+            statementCache: statementCache
2305 2374
         ) { statement in
2306 2375
             bindText(deviceHash, to: 1, in: statement)
2307 2376
             bindText(manufacturerHash, to: 2, in: statement)
@@ -2318,11 +2387,12 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2318 2387
         return sqlite3_last_insert_rowid(db)
2319 2388
     }
2320 2389
 
2321
-    private func upsertMetadataBlob(_ row: ArchiveSampleRow, db: OpaquePointer?) throws -> Int64? {
2390
+    private func upsertMetadataBlob(_ row: ArchiveSampleRow, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64? {
2322 2391
         guard let metadataHash = row.metadataHash, let metadataJSON = row.metadataJSON else { return nil }
2323 2392
         try withStatement(
2324 2393
             "INSERT OR IGNORE INTO metadata_blobs (metadata_hash, metadata_json) VALUES (?, ?)",
2325
-            db: db
2394
+            db: db,
2395
+            statementCache: statementCache
2326 2396
         ) { statement in
2327 2397
             bindText(metadataHash, to: 1, in: statement)
2328 2398
             bindText(metadataJSON, to: 2, in: statement)
@@ -2330,7 +2400,11 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2330 2400
                 throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
2331 2401
             }
2332 2402
         }
2333
-        return try requiredInt64("SELECT id FROM metadata_blobs WHERE metadata_hash = ? LIMIT 1", db: db) { statement in
2403
+        return try requiredInt64(
2404
+            "SELECT id FROM metadata_blobs WHERE metadata_hash = ? LIMIT 1",
2405
+            db: db,
2406
+            statementCache: statementCache
2407
+        ) { statement in
2334 2408
             bindText(metadataHash, to: 1, in: statement)
2335 2409
         }
2336 2410
     }
@@ -2339,7 +2413,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2339 2413
         _ row: ArchiveSampleRow,
2340 2414
         sampleTypeID: Int64,
2341 2415
         observationID: Int64,
2342
-        db: OpaquePointer?
2416
+        db: OpaquePointer?,
2417
+        statementCache: SQLiteStatementCache? = nil
2343 2418
     ) throws -> (id: Int64, inserted: Bool) {
2344 2419
         try withStatement(
2345 2420
             """
@@ -2348,7 +2423,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2348 2423
                 fuzzy_key, first_seen_observation_id, first_seen_at
2349 2424
             ) VALUES (?, ?, ?, ?, NULL, ?, ?)
2350 2425
             """,
2351
-            db: db
2426
+            db: db,
2427
+            statementCache: statementCache
2352 2428
         ) { statement in
2353 2429
             bindInt64(sampleTypeID, to: 1, in: statement)
2354 2430
             bindText(row.sampleUUIDHash, to: 2, in: statement)
@@ -2363,7 +2439,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2363 2439
         let inserted = sqlite3_changes(db) > 0
2364 2440
         let id = try requiredInt64(
2365 2441
             "SELECT id FROM samples WHERE sample_type_id = ? AND strict_fingerprint = ? LIMIT 1",
2366
-            db: db
2442
+            db: db,
2443
+            statementCache: statementCache
2367 2444
         ) { statement in
2368 2445
             bindInt64(sampleTypeID, to: 1, in: statement)
2369 2446
             bindText(row.strictFingerprint, to: 2, in: statement)
@@ -2371,10 +2448,11 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2371 2448
         return (id, inserted)
2372 2449
     }
2373 2450
 
2374
-    private func sampleID(sampleUUIDHash: String, sampleTypeID: Int64, db: OpaquePointer?) throws -> Int64? {
2451
+    private func sampleID(sampleUUIDHash: String, sampleTypeID: Int64, db: OpaquePointer?, statementCache: SQLiteStatementCache? = nil) throws -> Int64? {
2375 2452
         try optionalInt64(
2376 2453
             "SELECT id FROM samples WHERE sample_type_id = ? AND sample_uuid_hash = ? LIMIT 1",
2377
-            db: db
2454
+            db: db,
2455
+            statementCache: statementCache
2378 2456
         ) { statement in
2379 2457
             bindInt64(sampleTypeID, to: 1, in: statement)
2380 2458
             bindText(sampleUUIDHash, to: 2, in: statement)
@@ -2388,7 +2466,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2388 2466
         deviceID: Int64?,
2389 2467
         metadataID: Int64?,
2390 2468
         observationID: Int64,
2391
-        db: OpaquePointer?
2469
+        db: OpaquePointer?,
2470
+        statementCache: SQLiteStatementCache? = nil
2392 2471
     ) throws -> (id: Int64, inserted: Bool) {
2393 2472
         try withStatement(
2394 2473
             """
@@ -2398,7 +2477,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2398 2477
                 source_revision_id, hk_device_id, metadata_id, created_observation_id
2399 2478
             ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2400 2479
             """,
2401
-            db: db
2480
+            db: db,
2481
+            statementCache: statementCache
2402 2482
         ) { statement in
2403 2483
             bindInt64(sampleID, to: 1, in: statement)
2404 2484
             bindText(row.payloadHash, to: 2, in: statement)
@@ -2421,7 +2501,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2421 2501
         let inserted = sqlite3_changes(db) > 0
2422 2502
         let id = try requiredInt64(
2423 2503
             "SELECT id FROM sample_versions WHERE sample_id = ? AND payload_hash = ? LIMIT 1",
2424
-            db: db
2504
+            db: db,
2505
+            statementCache: statementCache
2425 2506
         ) { statement in
2426 2507
             bindInt64(sampleID, to: 1, in: statement)
2427 2508
             bindText(row.payloadHash, to: 2, in: statement)
@@ -2436,7 +2517,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2436 2517
         eventKind: String,
2437 2518
         evidenceKind: String,
2438 2519
         observedAt: Date,
2439
-        db: OpaquePointer?
2520
+        db: OpaquePointer?,
2521
+        statementCache: SQLiteStatementCache? = nil
2440 2522
     ) throws {
2441 2523
         try withStatement(
2442 2524
             """
@@ -2444,7 +2526,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2444 2526
                 observation_id, sample_id, version_id, event_kind, observed_at, evidence_kind
2445 2527
             ) VALUES (?, ?, ?, ?, ?, ?)
2446 2528
             """,
2447
-            db: db
2529
+            db: db,
2530
+            statementCache: statementCache
2448 2531
         ) { statement in
2449 2532
             bindInt64(observationID, to: 1, in: statement)
2450 2533
             bindInt64(sampleID, to: 2, in: statement)
@@ -2463,7 +2546,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2463 2546
         excludingVersionID: Int64?,
2464 2547
         closedAtObservationID: Int64,
2465 2548
         observedAt: Date,
2466
-        db: OpaquePointer?
2549
+        db: OpaquePointer?,
2550
+        statementCache: SQLiteStatementCache? = nil
2467 2551
     ) throws {
2468 2552
         let versionPredicate = excludingVersionID == nil
2469 2553
             ? ""
@@ -2473,7 +2557,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2473 2557
         SET last_observation_id = ?, last_seen_at = ?
2474 2558
         WHERE sample_id = ? AND last_observation_id IS NULL \(versionPredicate)
2475 2559
         """
2476
-        try withStatement(sql, db: db) { statement in
2560
+        try withStatement(sql, db: db, statementCache: statementCache) { statement in
2477 2561
             bindInt64(closedAtObservationID, to: 1, in: statement)
2478 2562
             sqlite3_bind_double(statement, 2, observedAt.timeIntervalSince1970)
2479 2563
             bindInt64(sampleID, to: 3, in: statement)
@@ -2491,7 +2575,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2491 2575
         versionID: Int64,
2492 2576
         observationID: Int64,
2493 2577
         observedAt: Date,
2494
-        db: OpaquePointer?
2578
+        db: OpaquePointer?,
2579
+        statementCache: SQLiteStatementCache? = nil
2495 2580
     ) throws {
2496 2581
         let existing = try optionalInt64(
2497 2582
             """
@@ -2500,7 +2585,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2500 2585
             WHERE sample_id = ? AND version_id = ? AND last_observation_id IS NULL
2501 2586
             LIMIT 1
2502 2587
             """,
2503
-            db: db
2588
+            db: db,
2589
+            statementCache: statementCache
2504 2590
         ) { statement in
2505 2591
             bindInt64(sampleID, to: 1, in: statement)
2506 2592
             bindInt64(versionID, to: 2, in: statement)
@@ -2512,7 +2598,8 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2512 2598
                 sample_id, version_id, first_observation_id, last_observation_id, first_seen_at, last_seen_at
2513 2599
             ) VALUES (?, ?, ?, NULL, ?, NULL)
2514 2600
             """,
2515
-            db: db
2601
+            db: db,
2602
+            statementCache: statementCache
2516 2603
         ) { statement in
2517 2604
             bindInt64(sampleID, to: 1, in: statement)
2518 2605
             bindInt64(versionID, to: 2, in: statement)
@@ -2786,9 +2873,10 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2786 2873
     private func requiredInt64(
2787 2874
         _ sql: String,
2788 2875
         db: OpaquePointer?,
2876
+        statementCache: SQLiteStatementCache? = nil,
2789 2877
         bind: (OpaquePointer?) throws -> Void
2790 2878
     ) throws -> Int64 {
2791
-        guard let value = try optionalInt64(sql, db: db, bind: bind) else {
2879
+        guard let value = try optionalInt64(sql, db: db, statementCache: statementCache, bind: bind) else {
2792 2880
             throw SQLiteHealthArchiveStoreError.stepFailed("missing required row")
2793 2881
         }
2794 2882
         return value
@@ -2797,9 +2885,10 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2797 2885
     private func optionalInt64(
2798 2886
         _ sql: String,
2799 2887
         db: OpaquePointer?,
2888
+        statementCache: SQLiteStatementCache? = nil,
2800 2889
         bind: (OpaquePointer?) throws -> Void
2801 2890
     ) throws -> Int64? {
2802
-        try withStatement(sql, db: db) { statement in
2891
+        try withStatement(sql, db: db, statementCache: statementCache) { statement in
2803 2892
             try bind(statement)
2804 2893
             guard sqlite3_step(statement) == SQLITE_ROW else { return nil }
2805 2894
             return sqlite3_column_int64(statement, 0)
@@ -2829,7 +2918,15 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2829 2918
         }
2830 2919
     }
2831 2920
 
2832
-    private func withStatement<T>(_ sql: String, db: OpaquePointer?, body: (OpaquePointer?) throws -> T) throws -> T {
2921
+    private func withStatement<T>(
2922
+        _ sql: String,
2923
+        db: OpaquePointer?,
2924
+        statementCache: SQLiteStatementCache? = nil,
2925
+        body: (OpaquePointer?) throws -> T
2926
+    ) throws -> T {
2927
+        if let statementCache {
2928
+            return try statementCache.withStatement(sql, body: body)
2929
+        }
2833 2930
         var statement: OpaquePointer?
2834 2931
         guard sqlite3_prepare_v2(db, sql, -1, &statement, nil) == SQLITE_OK else {
2835 2932
             throw SQLiteHealthArchiveStoreError.prepareFailed(lastErrorMessage(db))
@@ -2840,6 +2937,43 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2840 2937
 
2841 2938
 }
2842 2939
 
2940
+private final class SQLiteStatementCache: @unchecked Sendable {
2941
+    private let db: OpaquePointer?
2942
+    private var statements: [String: OpaquePointer?] = [:]
2943
+
2944
+    nonisolated init(db: OpaquePointer?) {
2945
+        self.db = db
2946
+    }
2947
+
2948
+    deinit {
2949
+        finalizeAll()
2950
+    }
2951
+
2952
+    nonisolated(unsafe) func withStatement<T>(_ sql: String, body: (OpaquePointer?) throws -> T) throws -> T {
2953
+        let statement: OpaquePointer?
2954
+        if let cached = statements[sql] {
2955
+            statement = cached
2956
+            sqlite3_reset(statement)
2957
+            sqlite3_clear_bindings(statement)
2958
+        } else {
2959
+            var prepared: OpaquePointer?
2960
+            guard sqlite3_prepare_v2(db, sql, -1, &prepared, nil) == SQLITE_OK else {
2961
+                throw SQLiteHealthArchiveStoreError.prepareFailed(lastErrorMessage(db))
2962
+            }
2963
+            statements[sql] = prepared
2964
+            statement = prepared
2965
+        }
2966
+        return try body(statement)
2967
+    }
2968
+
2969
+    nonisolated(unsafe) func finalizeAll() {
2970
+        for statement in statements.values {
2971
+            sqlite3_finalize(statement)
2972
+        }
2973
+        statements.removeAll()
2974
+    }
2975
+}
2976
+
2843 2977
 private struct ArchiveSampleRow {
2844 2978
     let sampleUUIDHash: String
2845 2979
     let typeIdentifier: String