@@ -266,6 +266,32 @@ import performance. The small differences are within expected run-to-run noise. |
||
| 266 | 266 |
The larger first-import gain remains attributable to the earlier hot `samples` |
| 267 | 267 |
index removal plus clean reset conditions. |
| 268 | 268 |
|
| 269 |
+### 2026-06-02 Incremental No-Delta Snapshot With Post-Import UI Freeze |
|
| 270 |
+ |
|
| 271 |
+Commit context: after `06ee6be`. Source: user-provided diagnostic report with |
|
| 272 |
+`previousSnapshotID` present, `isChainStart: false`, and the same total record |
|
| 273 |
+count as the preceding first snapshot. |
|
| 274 |
+ |
|
| 275 |
+| Metric | Value | |
|
| 276 |
+|--------|-------| |
|
| 277 |
+| Wall clock | 9.2s | |
|
| 278 |
+| Summed metric total | 8.9s | |
|
| 279 |
+| Summed fetch | 0.2s | |
|
| 280 |
+| Summed processing | 0.0s | |
|
| 281 |
+| Summed insert | 0.0s | |
|
| 282 |
+| Summed finalize | 8.5s | |
|
| 283 |
+| Total records | 1,579,168 | |
|
| 284 |
+| Heart Rate finalize | 4.8s | |
|
| 285 |
+| Active Energy finalize | 1.8s | |
|
| 286 |
+ |
|
| 287 |
+Conclusion: repeated no-delta capture is now fast and does not write unchanged |
|
| 288 |
+records. The user still observed roughly one minute of app unresponsiveness |
|
| 289 |
+after finalization, so the remaining issue is outside the measured import |
|
| 290 |
+phases. The most likely culprit is synchronous post-import Core Data cache |
|
| 291 |
+rebuild / dashboard refresh work. The dashboard cache refresh was moved to a |
|
| 292 |
+background task after this report; next reports should distinguish import time |
|
| 293 |
+from post-import UI recovery. |
|
| 294 |
+ |
|
| 269 | 295 |
## Optimization Iterations |
| 270 | 296 |
|
| 271 | 297 |
| Date | Commit | Change | Result / Status | |
@@ -287,6 +313,7 @@ index removal plus clean reset conditions. |
||
| 287 | 313 |
| 2026-06-02 | `a281c51` | Stopped writing `verified` observation events for unchanged existing samples. | Awaiting comparable non-chain-start/full-scan report. Expected signal is lower `SummedInsertElapsed` and especially lower Heart Rate insert time when most rows are unchanged. | |
| 288 | 314 |
| 2026-06-02 | `3dd5f48` | Fortified scheduled test database reset with a disk marker and extra SQLite sidecar cleanup. | Real-device report confirmed reset produced `previousSnapshotID: none`, `isChainStart: true`, and a clean first-snapshot timeline. | |
| 289 | 315 |
| 2026-06-02 | `06ee6be` | Removed unused `sample_versions(start_date, end_date)` and redundant `sample_visibility_ranges(sample_id, last_observation_id)` indexes. | Comparable first-import report was flat: wall clock 12m43s -> 12m39s and summed insert 10m11s -> 10m07s. Treat as no material performance change. | |
| 316 |
+| 2026-06-02 | pending | Moved Dashboard archive cache refresh/rebuild off the UI path after snapshot completion. | Awaiting real-device confirmation that the app no longer stays unresponsive for roughly one minute after a completed snapshot. | |
|
| 290 | 317 |
|
| 291 | 318 |
## Current Diagnosis |
| 292 | 319 |
|
@@ -302,14 +329,16 @@ The likely bottleneck is per-row SQLite work: |
||
| 302 | 329 |
- index maintenance while importing high-volume rows; |
| 303 | 330 |
- multiple dependent writes per sample; |
| 304 | 331 |
- commit / transaction shape; |
| 305 |
-- Core Data or UI refresh work after SQLite completes, if the app remains unresponsive after import. |
|
| 332 |
+- Core Data or UI refresh work after SQLite completes. This became the active |
|
| 333 |
+ bottleneck after a no-delta incremental snapshot completed in 9.2s but the UI |
|
| 334 |
+ remained unresponsive for roughly one minute. |
|
| 306 | 335 |
|
| 307 | 336 |
## Open Issues / Observations |
| 308 | 337 |
|
| 309 | 338 |
- Very small pages reduced freeze risk but introduced visible overhead. |
| 310 | 339 |
- Some progress timing displayed in the UI did not include overhead, so elapsed time and rates looked better than the real operation. |
| 311 | 340 |
- A previous Heart Rate import appeared to stall for long periods around roughly 900k records, but later progress resumed; avoid classifying this as a hard timeout without report evidence. |
| 312 |
-- After a completed import, the app may remain unresponsive for more than one minute. This needs separate timing around post-import cache rebuild, UI refresh, report generation, and main-thread work. |
|
| 341 |
+- After a completed import, the app may remain unresponsive for more than one minute. A no-delta incremental report shows the import itself completed in 9.2s, so post-import cache rebuild / UI refresh is the likely cause. |
|
| 313 | 342 |
- Partial / old imported observations can pollute comparisons. Fresh first-snapshot performance comparisons should use a confirmed reset database. |
| 314 | 343 |
- Non-chain-start full scans can be slower than first imports if unchanged existing samples still write per-sample archive evidence. |
| 315 | 344 |
|
@@ -317,7 +346,7 @@ The likely bottleneck is per-row SQLite work: |
||
| 317 | 346 |
|
| 318 | 347 |
Prioritize experiments in this order: |
| 319 | 348 |
|
| 320 |
-1. Add explicit post-import timings if the app is still unresponsive after the operation reports success. |
|
| 349 |
+1. Confirm whether the background dashboard cache refresh removes the post-import UI freeze. If not, add explicit timings around cache rebuild, dashboard refresh, diagnostic report generation, and sheet dismissal. |
|
| 321 | 350 |
2. Run a non-chain-start/full-scan benchmark after skipping unchanged `verified` events and compare `SummedInsertElapsed`, `Heart Rate insertElapsed`, `Steps insertElapsed`, and `Walking + Running Distance insertElapsed`. |
| 322 | 351 |
3. Reduce remaining per-sample SQLite writes for unchanged existing samples during non-chain-start full scans, especially open visibility-range existence checks. |
| 323 | 352 |
4. Profile whether index maintenance dominates first-import insert cost. |
@@ -82,7 +82,7 @@ struct CachedArchiveHealthStatus: Equatable, Sendable {
|
||
| 82 | 82 |
} |
| 83 | 83 |
|
| 84 | 84 |
// Interface updated 2026-05-24 — see AGENTS.md |
| 85 |
-final class CoreDataArchiveCacheStore {
|
|
| 85 |
+nonisolated final class CoreDataArchiveCacheStore {
|
|
| 86 | 86 |
static let cacheSchemaVersion = 1 |
| 87 | 87 |
|
| 88 | 88 |
let container: NSPersistentContainer |
@@ -497,7 +497,7 @@ final class CoreDataArchiveCacheStore {
|
||
| 497 | 497 |
} |
| 498 | 498 |
} |
| 499 | 499 |
|
| 500 |
-private extension CoreDataArchiveCacheStore {
|
|
| 500 |
+nonisolated private extension CoreDataArchiveCacheStore {
|
|
| 501 | 501 |
nonisolated static func observationRow(_ object: NSManagedObject) -> CachedArchiveObservationRow {
|
| 502 | 502 |
CachedArchiveObservationRow( |
| 503 | 503 |
observationID: object.value(forKey: "observationID") as? Int64 ?? 0, |
@@ -562,7 +562,7 @@ private extension CoreDataArchiveCacheStore {
|
||
| 562 | 562 |
} |
| 563 | 563 |
} |
| 564 | 564 |
|
| 565 |
-private extension CoreDataArchiveCacheStore {
|
|
| 565 |
+nonisolated private extension CoreDataArchiveCacheStore {
|
|
| 566 | 566 |
static let cacheEntityNames = [ |
| 567 | 567 |
"CachedObservationRow", |
| 568 | 568 |
"CachedTypeSummary", |
@@ -703,7 +703,7 @@ private extension CoreDataArchiveCacheStore {
|
||
| 703 | 703 |
} |
| 704 | 704 |
} |
| 705 | 705 |
|
| 706 |
-private extension CoreDataArchiveCacheStore {
|
|
| 706 |
+nonisolated private extension CoreDataArchiveCacheStore {
|
|
| 707 | 707 |
func openArchive(at archiveURL: URL) throws -> OpaquePointer? {
|
| 708 | 708 |
var db: OpaquePointer? |
| 709 | 709 |
guard sqlite3_open_v2(archiveURL.path, &db, SQLITE_OPEN_READONLY | SQLITE_OPEN_FULLMUTEX, nil) == SQLITE_OK else {
|
@@ -745,12 +745,12 @@ private extension CoreDataArchiveCacheStore {
|
||
| 745 | 745 |
} |
| 746 | 746 |
} |
| 747 | 747 |
|
| 748 |
-private func lastErrorMessage(_ db: OpaquePointer?) -> String {
|
|
| 748 |
+nonisolated private func lastErrorMessage(_ db: OpaquePointer?) -> String {
|
|
| 749 | 749 |
guard let message = sqlite3_errmsg(db) else { return "unknown SQLite error" }
|
| 750 | 750 |
return String(cString: message) |
| 751 | 751 |
} |
| 752 | 752 |
|
| 753 |
-private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? {
|
|
| 753 |
+nonisolated private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? {
|
|
| 754 | 754 |
guard sqlite3_column_type(statement, index) != SQLITE_NULL, |
| 755 | 755 |
let pointer = sqlite3_column_text(statement, index) else {
|
| 756 | 756 |
return nil |
@@ -758,16 +758,16 @@ private func columnText(_ statement: OpaquePointer?, _ index: Int32) -> String? |
||
| 758 | 758 |
return String(cString: pointer) |
| 759 | 759 |
} |
| 760 | 760 |
|
| 761 |
-private func columnDouble(_ statement: OpaquePointer?, _ index: Int32) -> Double? {
|
|
| 761 |
+nonisolated private func columnDouble(_ statement: OpaquePointer?, _ index: Int32) -> Double? {
|
|
| 762 | 762 |
guard sqlite3_column_type(statement, index) != SQLITE_NULL else { return nil }
|
| 763 | 763 |
return sqlite3_column_double(statement, index) |
| 764 | 764 |
} |
| 765 | 765 |
|
| 766 |
-private func columnInt64(_ statement: OpaquePointer?, _ index: Int32) -> Int64? {
|
|
| 766 |
+nonisolated private func columnInt64(_ statement: OpaquePointer?, _ index: Int32) -> Int64? {
|
|
| 767 | 767 |
guard sqlite3_column_type(statement, index) != SQLITE_NULL else { return nil }
|
| 768 | 768 |
return sqlite3_column_int64(statement, index) |
| 769 | 769 |
} |
| 770 | 770 |
|
| 771 |
-private func columnUnixDate(_ statement: OpaquePointer?, _ index: Int32) -> Date? {
|
|
| 771 |
+nonisolated private func columnUnixDate(_ statement: OpaquePointer?, _ index: Int32) -> Date? {
|
|
| 772 | 772 |
columnDouble(statement, index).map { Date(timeIntervalSince1970: $0) }
|
| 773 | 773 |
} |
@@ -31,6 +31,11 @@ final class DashboardViewModel {
|
||
| 31 | 31 |
private var pendingPartialSnapshotID: UUID? |
| 32 | 32 |
private var pendingAmbiguousSnapshotID: UUID? |
| 33 | 33 |
|
| 34 |
+ private struct ArchiveCacheRefreshResult: Sendable {
|
|
| 35 |
+ let observationRows: [CachedArchiveObservationRow] |
|
| 36 |
+ let healthStatus: CachedArchiveHealthStatus? |
|
| 37 |
+ } |
|
| 38 |
+ |
|
| 34 | 39 |
func requestAuthorization() async {
|
| 35 | 40 |
isRequestingAuth = true |
| 36 | 41 |
authError = nil |
@@ -186,7 +191,7 @@ final class DashboardViewModel {
|
||
| 186 | 191 |
} |
| 187 | 192 |
|
| 188 | 193 |
snapshotProgress = .complete |
| 189 |
- refreshArchiveCache() |
|
| 194 |
+ await refreshArchiveCache() |
|
| 190 | 195 |
} catch is CancellationError {
|
| 191 | 196 |
snapshotError = "Snapshot creation exceeded the operation timeout. Individual metric timeouts are adaptive; retry failed metrics with an extended timeout when available." |
| 192 | 197 |
snapshotProgress = .idle |
@@ -208,7 +213,7 @@ final class DashboardViewModel {
|
||
| 208 | 213 |
typeIdentifiers: ambiguousIDs, |
| 209 | 214 |
using: healthKit |
| 210 | 215 |
) |
| 211 |
- finishSavedReviewedSnapshot(outcome) |
|
| 216 |
+ await finishSavedReviewedSnapshot(outcome) |
|
| 212 | 217 |
} catch {
|
| 213 | 218 |
snapshotError = "Failed to save reviewed snapshot: \(error.localizedDescription)" |
| 214 | 219 |
showProgressSheet = true |
@@ -220,7 +225,7 @@ final class DashboardViewModel {
|
||
| 220 | 225 |
|
| 221 | 226 |
do {
|
| 222 | 227 |
let outcome = try await LegacySwiftDataBridge.saveReviewedCompleteSnapshot(id: snapshotID, using: healthKit) |
| 223 |
- finishSavedReviewedSnapshot(outcome) |
|
| 228 |
+ await finishSavedReviewedSnapshot(outcome) |
|
| 224 | 229 |
} catch {
|
| 225 | 230 |
snapshotError = "Failed to save reviewed snapshot: \(error.localizedDescription)" |
| 226 | 231 |
showProgressSheet = true |
@@ -280,7 +285,7 @@ final class DashboardViewModel {
|
||
| 280 | 285 |
monitoredTypeSetHash: outcome.monitoredTypeSetHash, |
| 281 | 286 |
monitoredRegistryVersion: outcome.monitoredRegistryVersion |
| 282 | 287 |
) |
| 283 |
- refreshArchiveCache() |
|
| 288 |
+ await refreshArchiveCache() |
|
| 284 | 289 |
} catch {
|
| 285 | 290 |
snapshotError = "Failed to save partial snapshot: \(error.localizedDescription)" |
| 286 | 291 |
showProgressSheet = true |
@@ -348,7 +353,7 @@ final class DashboardViewModel {
|
||
| 348 | 353 |
} |
| 349 | 354 |
} |
| 350 | 355 |
|
| 351 |
- private func finishSavedReviewedSnapshot(_ outcome: LegacySnapshotOutcome) {
|
|
| 356 |
+ private func finishSavedReviewedSnapshot(_ outcome: LegacySnapshotOutcome) async {
|
|
| 352 | 357 |
applyOutcome(outcome) |
| 353 | 358 |
pendingAmbiguousSnapshotID = nil |
| 354 | 359 |
pendingPartialSnapshotID = nil |
@@ -363,7 +368,7 @@ final class DashboardViewModel {
|
||
| 363 | 368 |
fetchProgress = nil |
| 364 | 369 |
showProgressSheet = false |
| 365 | 370 |
snapshotProgress = .idle |
| 366 |
- refreshArchiveCache() |
|
| 371 |
+ await refreshArchiveCache() |
|
| 367 | 372 |
} |
| 368 | 373 |
|
| 369 | 374 |
private func applyOutcome(_ outcome: LegacySnapshotOutcome) {
|
@@ -374,13 +379,21 @@ final class DashboardViewModel {
|
||
| 374 | 379 |
completedSnapshotRetryOfSnapshotID = outcome.retryOfSnapshotID |
| 375 | 380 |
} |
| 376 | 381 |
|
| 377 |
- private func refreshArchiveCache() {
|
|
| 382 |
+ private func refreshArchiveCache() async {
|
|
| 378 | 383 |
do {
|
| 379 |
- let cache = try CoreDataArchiveCacheStore() |
|
| 380 |
- _ = try cache.rebuild(fromArchiveAt: SQLiteHealthArchiveStore.defaultDatabaseURL) |
|
| 381 |
- archiveObservationRows = try cache.observationRows(limit: 2) |
|
| 382 |
- latestArchiveObservation = archiveObservationRows.first |
|
| 383 |
- archiveHealthStatus = try cache.latestArchiveHealthStatus() |
|
| 384 |
+ let result = try await Task.detached(priority: .utility) {
|
|
| 385 |
+ let cache = try CoreDataArchiveCacheStore() |
|
| 386 |
+ _ = try cache.rebuild(fromArchiveAt: SQLiteHealthArchiveStore.defaultDatabaseURL) |
|
| 387 |
+ let rows = try cache.observationRows(limit: 2) |
|
| 388 |
+ let healthStatus = try cache.latestArchiveHealthStatus() |
|
| 389 |
+ return ArchiveCacheRefreshResult( |
|
| 390 |
+ observationRows: rows, |
|
| 391 |
+ healthStatus: healthStatus |
|
| 392 |
+ ) |
|
| 393 |
+ }.value |
|
| 394 |
+ archiveObservationRows = result.observationRows |
|
| 395 |
+ latestArchiveObservation = result.observationRows.first |
|
| 396 |
+ archiveHealthStatus = result.healthStatus |
|
| 384 | 397 |
archiveCacheError = nil |
| 385 | 398 |
} catch {
|
| 386 | 399 |
archiveObservationRows = [] |