Showing 3 changed files with 66 additions and 24 deletions
+32 -3
HealthProbe/Doc/04-project/Import-Optimization-Log.md
@@ -266,6 +266,32 @@ import performance. The small differences are within expected run-to-run noise.
266 266
 The larger first-import gain remains attributable to the earlier hot `samples`
267 267
 index removal plus clean reset conditions.
268 268
 
269
+### 2026-06-02 Incremental No-Delta Snapshot With Post-Import UI Freeze
270
+
271
+Commit context: after `06ee6be`. Source: user-provided diagnostic report with
272
+`previousSnapshotID` present, `isChainStart: false`, and the same total record
273
+count as the preceding first snapshot.
274
+
275
+| Metric | Value |
276
+|--------|-------|
277
+| Wall clock | 9.2s |
278
+| Summed metric total | 8.9s |
279
+| Summed fetch | 0.2s |
280
+| Summed processing | 0.0s |
281
+| Summed insert | 0.0s |
282
+| Summed finalize | 8.5s |
283
+| Total records | 1,579,168 |
284
+| Heart Rate finalize | 4.8s |
285
+| Active Energy finalize | 1.8s |
286
+
287
+Conclusion: repeated no-delta capture is now fast and does not write unchanged
288
+records. The user still observed roughly one minute of app unresponsiveness
289
+after finalization, so the remaining issue is outside the measured import
290
+phases. The most likely culprit is synchronous post-import Core Data cache
291
+rebuild / dashboard refresh work. The dashboard cache refresh was moved to a
292
+background task after this report; next reports should distinguish import time
293
+from post-import UI recovery.
294
+
269 295
 ## Optimization Iterations
270 296
 
271 297
 | Date | Commit | Change | Result / Status |
@@ -287,6 +313,7 @@ index removal plus clean reset conditions.
287 313
 | 2026-06-02 | `a281c51` | Stopped writing `verified` observation events for unchanged existing samples. | Awaiting comparable non-chain-start/full-scan report. Expected signal is lower `SummedInsertElapsed` and especially lower Heart Rate insert time when most rows are unchanged. |
288 314
 | 2026-06-02 | `3dd5f48` | Fortified scheduled test database reset with a disk marker and extra SQLite sidecar cleanup. | Real-device report confirmed reset produced `previousSnapshotID: none`, `isChainStart: true`, and a clean first-snapshot timeline. |
289 315
 | 2026-06-02 | `06ee6be` | Removed unused `sample_versions(start_date, end_date)` and redundant `sample_visibility_ranges(sample_id, last_observation_id)` indexes. | Comparable first-import report was flat: wall clock 12m43s -> 12m39s and summed insert 10m11s -> 10m07s. Treat as no material performance change. |
316
+| 2026-06-02 | pending | Moved Dashboard archive cache refresh/rebuild off the UI path after snapshot completion. | Awaiting real-device confirmation that the app no longer stays unresponsive for roughly one minute after a completed snapshot. |
290 317
 
291 318
 ## Current Diagnosis
292 319
 
@@ -302,14 +329,16 @@ The likely bottleneck is per-row SQLite work:
302 329
 - index maintenance while importing high-volume rows;
303 330
 - multiple dependent writes per sample;
304 331
 - commit / transaction shape;
305
-- Core Data or UI refresh work after SQLite completes, if the app remains unresponsive after import.
332
+- Core Data or UI refresh work after SQLite completes. This became the active
333
+  bottleneck after a no-delta incremental snapshot completed in 9.2s but the UI
334
+  remained unresponsive for roughly one minute.
306 335
 
307 336
 ## Open Issues / Observations
308 337
 
309 338
 - Very small pages reduced freeze risk but introduced visible overhead.
310 339
 - Some progress timing displayed in the UI did not include overhead, so elapsed time and rates looked better than the real operation.
311 340
 - A previous Heart Rate import appeared to stall for long periods around roughly 900k records, but later progress resumed; avoid classifying this as a hard timeout without report evidence.
312
-- After a completed import, the app may remain unresponsive for more than one minute. This needs separate timing around post-import cache rebuild, UI refresh, report generation, and main-thread work.
341
+- After a completed import, the app may remain unresponsive for more than one minute. A no-delta incremental report shows the import itself completed in 9.2s, so post-import cache rebuild / UI refresh is the likely cause.
313 342
 - Partial / old imported observations can pollute comparisons. Fresh first-snapshot performance comparisons should use a confirmed reset database.
314 343
 - Non-chain-start full scans can be slower than first imports if unchanged existing samples still write per-sample archive evidence.
315 344
 
@@ -317,7 +346,7 @@ The likely bottleneck is per-row SQLite work:
317 346
 
318 347
 Prioritize experiments in this order:
319 348
 
320
-1. Add explicit post-import timings if the app is still unresponsive after the operation reports success.
349
+1. Confirm whether the background dashboard cache refresh removes the post-import UI freeze. If not, add explicit timings around cache rebuild, dashboard refresh, diagnostic report generation, and sheet dismissal.
321 350
 2. Run a non-chain-start/full-scan benchmark after skipping unchanged `verified` events and compare `SummedInsertElapsed`, `Heart Rate insertElapsed`, `Steps insertElapsed`, and `Walking + Running Distance insertElapsed`.
322 351
 3. Reduce remaining per-sample SQLite writes for unchanged existing samples during non-chain-start full scans, especially open visibility-range existence checks.
323 352
 4. Profile whether index maintenance dominates first-import insert cost.
+9 -9
HealthProbe/Services/CoreDataArchiveCacheStore.swift
@@ -82,7 +82,7 @@ struct CachedArchiveHealthStatus: Equatable, Sendable {
82 82
 }
83 83
 
84 84
 // Interface updated 2026-05-24 — see AGENTS.md
85
-final class CoreDataArchiveCacheStore {
85
+nonisolated final class CoreDataArchiveCacheStore {
86 86
     static let cacheSchemaVersion = 1
87 87
 
88 88
     let container: NSPersistentContainer
@@ -497,7 +497,7 @@ final class CoreDataArchiveCacheStore {
497 497
     }
498 498
 }
499 499
 
500
-private extension CoreDataArchiveCacheStore {
500
+nonisolated private extension CoreDataArchiveCacheStore {
501 501
     nonisolated static func observationRow(_ object: NSManagedObject) -> CachedArchiveObservationRow {
502 502
         CachedArchiveObservationRow(
503 503
             observationID: object.value(forKey: "observationID") as? Int64 ?? 0,
@@ -562,7 +562,7 @@ private extension CoreDataArchiveCacheStore {
562 562
     }
563 563
 }
564 564
 
565
-private extension CoreDataArchiveCacheStore {
565
+nonisolated private extension CoreDataArchiveCacheStore {
566 566
     static let cacheEntityNames = [
567 567
         "CachedObservationRow",
568 568
         "CachedTypeSummary",
@@ -703,7 +703,7 @@ private extension CoreDataArchiveCacheStore {
703 703
     }
704 704
 }
705 705
 
706
-private extension CoreDataArchiveCacheStore {
706
+nonisolated private extension CoreDataArchiveCacheStore {
707 707
     func openArchive(at archiveURL: URL) throws -> OpaquePointer? {
708 708
         var db: OpaquePointer?
709 709
         guard sqlite3_open_v2(archiveURL.path, &db, SQLITE_OPEN_READONLY | SQLITE_OPEN_FULLMUTEX, nil) == SQLITE_OK else {
@@ -745,12 +745,12 @@ private extension CoreDataArchiveCacheStore {
745 745
     }
746 746
 }
747 747
 
748
-private func lastErrorMessage(_ db: OpaquePointer?) -> String {
748
+nonisolated private func lastErrorMessage(_ db: OpaquePointer?) -> String {
749 749
     guard let message = sqlite3_errmsg(db) else { return "unknown SQLite error" }
750 750
     return String(cString: message)
751 751
 }
752 752
 
753
-private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? {
753
+nonisolated private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? {
754 754
     guard sqlite3_column_type(statement, index) != SQLITE_NULL,
755 755
           let pointer = sqlite3_column_text(statement, index) else {
756 756
         return nil
@@ -758,16 +758,16 @@ private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String?
758 758
     return String(cString: pointer)
759 759
 }
760 760
 
761
-private func columnDouble(_ statement: OpaquePointer?, _ index: Int32) -> Double? {
761
+nonisolated private func columnDouble(_ statement: OpaquePointer?, _ index: Int32) -> Double? {
762 762
     guard sqlite3_column_type(statement, index) != SQLITE_NULL else { return nil }
763 763
     return sqlite3_column_double(statement, index)
764 764
 }
765 765
 
766
-private func columnInt64(_ statement: OpaquePointer?, _ index: Int32) -> Int64? {
766
+nonisolated private func columnInt64(_ statement: OpaquePointer?, _ index: Int32) -> Int64? {
767 767
     guard sqlite3_column_type(statement, index) != SQLITE_NULL else { return nil }
768 768
     return sqlite3_column_int64(statement, index)
769 769
 }
770 770
 
771
-private func columnUnixDate(_ statement: OpaquePointer?, _ index: Int32) -> Date? {
771
+nonisolated private func columnUnixDate(_ statement: OpaquePointer?, _ index: Int32) -> Date? {
772 772
     columnDouble(statement, index).map { Date(timeIntervalSince1970: $0) }
773 773
 }
+25 -12
HealthProbe/ViewModels/DashboardViewModel.swift
@@ -31,6 +31,11 @@ final class DashboardViewModel {
31 31
     private var pendingPartialSnapshotID: UUID?
32 32
     private var pendingAmbiguousSnapshotID: UUID?
33 33
 
34
+    private struct ArchiveCacheRefreshResult: Sendable {
35
+        let observationRows: [CachedArchiveObservationRow]
36
+        let healthStatus: CachedArchiveHealthStatus?
37
+    }
38
+
34 39
     func requestAuthorization() async {
35 40
         isRequestingAuth = true
36 41
         authError = nil
@@ -186,7 +191,7 @@ final class DashboardViewModel {
186 191
             }
187 192
 
188 193
             snapshotProgress = .complete
189
-            refreshArchiveCache()
194
+            await refreshArchiveCache()
190 195
         } catch is CancellationError {
191 196
             snapshotError = "Snapshot creation exceeded the operation timeout. Individual metric timeouts are adaptive; retry failed metrics with an extended timeout when available."
192 197
             snapshotProgress = .idle
@@ -208,7 +213,7 @@ final class DashboardViewModel {
208 213
                 typeIdentifiers: ambiguousIDs,
209 214
                 using: healthKit
210 215
             )
211
-            finishSavedReviewedSnapshot(outcome)
216
+            await finishSavedReviewedSnapshot(outcome)
212 217
         } catch {
213 218
             snapshotError = "Failed to save reviewed snapshot: \(error.localizedDescription)"
214 219
             showProgressSheet = true
@@ -220,7 +225,7 @@ final class DashboardViewModel {
220 225
 
221 226
         do {
222 227
             let outcome = try await LegacySwiftDataBridge.saveReviewedCompleteSnapshot(id: snapshotID, using: healthKit)
223
-            finishSavedReviewedSnapshot(outcome)
228
+            await finishSavedReviewedSnapshot(outcome)
224 229
         } catch {
225 230
             snapshotError = "Failed to save reviewed snapshot: \(error.localizedDescription)"
226 231
             showProgressSheet = true
@@ -280,7 +285,7 @@ final class DashboardViewModel {
280 285
                 monitoredTypeSetHash: outcome.monitoredTypeSetHash,
281 286
                 monitoredRegistryVersion: outcome.monitoredRegistryVersion
282 287
             )
283
-            refreshArchiveCache()
288
+            await refreshArchiveCache()
284 289
         } catch {
285 290
             snapshotError = "Failed to save partial snapshot: \(error.localizedDescription)"
286 291
             showProgressSheet = true
@@ -348,7 +353,7 @@ final class DashboardViewModel {
348 353
         }
349 354
     }
350 355
 
351
-    private func finishSavedReviewedSnapshot(_ outcome: LegacySnapshotOutcome) {
356
+    private func finishSavedReviewedSnapshot(_ outcome: LegacySnapshotOutcome) async {
352 357
         applyOutcome(outcome)
353 358
         pendingAmbiguousSnapshotID = nil
354 359
         pendingPartialSnapshotID = nil
@@ -363,7 +368,7 @@ final class DashboardViewModel {
363 368
         fetchProgress = nil
364 369
         showProgressSheet = false
365 370
         snapshotProgress = .idle
366
-        refreshArchiveCache()
371
+        await refreshArchiveCache()
367 372
     }
368 373
 
369 374
     private func applyOutcome(_ outcome: LegacySnapshotOutcome) {
@@ -374,13 +379,21 @@ final class DashboardViewModel {
374 379
         completedSnapshotRetryOfSnapshotID = outcome.retryOfSnapshotID
375 380
     }
376 381
 
377
-    private func refreshArchiveCache() {
382
+    private func refreshArchiveCache() async {
378 383
         do {
379
-            let cache = try CoreDataArchiveCacheStore()
380
-            _ = try cache.rebuild(fromArchiveAt: SQLiteHealthArchiveStore.defaultDatabaseURL)
381
-            archiveObservationRows = try cache.observationRows(limit: 2)
382
-            latestArchiveObservation = archiveObservationRows.first
383
-            archiveHealthStatus = try cache.latestArchiveHealthStatus()
384
+            let result = try await Task.detached(priority: .utility) {
385
+                let cache = try CoreDataArchiveCacheStore()
386
+                _ = try cache.rebuild(fromArchiveAt: SQLiteHealthArchiveStore.defaultDatabaseURL)
387
+                let rows = try cache.observationRows(limit: 2)
388
+                let healthStatus = try cache.latestArchiveHealthStatus()
389
+                return ArchiveCacheRefreshResult(
390
+                    observationRows: rows,
391
+                    healthStatus: healthStatus
392
+                )
393
+            }.value
394
+            archiveObservationRows = result.observationRows
395
+            latestArchiveObservation = result.observationRows.first
396
+            archiveHealthStatus = result.healthStatus
384 397
             archiveCacheError = nil
385 398
         } catch {
386 399
             archiveObservationRows = []