Fast-fail snapshot capture when protected data locks · a6d7479

Fast-fail snapshot capture when protected data locks
Browse files

bogdan committed 14 hours ago

main

1 parent 01414eb

commit a6d7479

Showing 4 changed files with 164 additions and 8 deletions

+31 -0

HealthProbe/Doc/04-project/Import-Optimization-Log.md

+                     ## Real-Device Results
+                    +### 2026-06-07 Overnight Initial Imports On Both Large Devices
+                    +Source: two user-provided overnight first-import reports from large devices,
+                    +both on build fingerprint `1.0(1)-1780767682-92064`.
+                    +Observed behavior:
+                    +- report A: `264,844` records, `80 complete / 47 degraded`, wall clock
+                    +  `405m 54s`, summed fetch `157m 3s`, summed insert `56.6s`;
+                    +- report B: `783,190` records, `51 complete / 76 degraded`, wall clock
+                    +  `250m 47s`, summed fetch `237m 43s`, summed insert `9m 5s`;
+                    +- the dominant regression was not SQLite write time; it was fetch time spent on
+                    +  failed or stalled type-boundary queries;
+                    +- one report showed explicit `HKErrorDatabaseInaccessible` (`Protected health
+                    +  data is inaccessible`) for many failed types, while the other showed the same
+                    +  failure mode amplified by adaptive timeout-based retries.
+                    +Interpretation:
+                    +- this is primarily a HealthKit availability / device-lock regression path, not
+                    +  an archive insert regression;
+                    +- debug binaries may add overhead, but they do not explain multi-hour fetch-only
+                    +  inflation while insert remains in seconds or single-digit minutes;
+                    +- once protected health data becomes inaccessible mid-run, continuing to probe
+                    +  all remaining types wastes wall-clock time and pollutes timeout learning.
+                    +Action taken:
+                    +- detect `HKErrorDatabaseInaccessible` explicitly;
+                    +- stop inferring that protected-data failures should keep probing the remaining
+                    +  types;
+                    +- fast-fail all remaining types in the same snapshot once protected data becomes
+                    +  unavailable, instead of paying one timeout/error cycle per metric.
+                     ### 2026-06-02 Baseline Before Latest Batch/Chunk Work
+                     Source: user-provided diagnostic report.

+90 -8

HealthProbe/Services/HealthKitService.swift

View

+                         ) async -> [TypeCount] {
+                             var typeCounts: [TypeCount] = []
+                             typeCounts.reserveCapacity(active.count)
+                    +        var protectedDataBecameUnavailable = !UIApplication.shared.isProtectedDataAvailable
+                             for monitoredType in active {
+                                 var profile = timeoutProfile(for: monitoredType)
+                                     adaptiveTimeoutsEnabled: adaptiveTimeoutsEnabled,
+                                     timeoutMultiplier: timeoutMultiplier
+                    -            var result = await fetchTypeCountData(
+                    -                for: monitoredType,
+                    -                timeoutProfile: profile,
+                    -                timeoutSeconds: timeout,
+                    -                previousTypeCount: previousSnapshot?.typeCounts?.first { $0.typeIdentifier == monitoredType.id },
+                    -                archiveObservationID: archiveObservationID,
+                    -                progress: progress
+                    -            )
+                    +            var result: TypeCountFetchResult
+                    +            if protectedDataBecameUnavailable {
+                    +                result = protectedDataUnavailableResult(
+                    +                    for: monitoredType,
+                    +                    timeoutProfile: profile,
+                    +                    timeoutSeconds: timeout,
+                    +                    progress: progress
+                    +                )
+                    +            } else {
+                    +                result = await fetchTypeCountData(
+                    +                    for: monitoredType,
+                    +                    timeoutProfile: profile,
+                    +                    timeoutSeconds: timeout,
+                    +                    previousTypeCount: previousSnapshot?.typeCounts?.first { $0.typeIdentifier == monitoredType.id },
+                    +                    archiveObservationID: archiveObservationID,
+                    +                    progress: progress
+                    +                )
+                    +                if result.indicatesProtectedDataInaccessible {
+                    +                    protectedDataBecameUnavailable = true
+                    +                }
+                    +            }
+                                 updateTimeoutProfile(&profile, with: result, monitoredType: monitoredType)
+                                 LocalMetricTimeoutProfileStore.save(profile)
+                                 result.applyTimeoutProfile(profile)
+                             return typeCounts
+                    +    private func protectedDataUnavailableResult(
+                    +        for monitoredType: MonitoredType,
+                    +        timeoutProfile: LocalMetricTimeoutProfile,
+                    +        timeoutSeconds: TimeInterval,
+                    +        progress: SnapshotFetchProgress?
+                    +    ) -> TypeCountFetchResult {
+                    +        let message = "Protected health data is inaccessible because the device is locked."
+                    +        let call = HealthKitAPICallResult(
+                    +            queryType: "earliest_sample",
+                    +            status: .failed,
+                    +            elapsedSeconds: 0,
+                    +            resultValue: nil,
+                    +            errorCode: "\(HKError.Code.errorDatabaseInaccessible.rawValue)",
+                    +            errorDomain: HKError.errorDomain,
+                    +            errorDescription: message,
+                    +            failureKind: "HealthKit error",
+                    +            cancellationReason: nil
+                    +        )
+                    +        let companionCall = HealthKitAPICallResult(
+                    +            queryType: "latest_sample",
+                    +            status: .failed,
+                    +            elapsedSeconds: 0,
+                    +            resultValue: nil,
+                    +            errorCode: "\(HKError.Code.errorDatabaseInaccessible.rawValue)",
+                    +            errorDomain: HKError.errorDomain,
+                    +            errorDescription: message,
+                    +            failureKind: "HealthKit error",
+                    +            cancellationReason: nil
+                    +        )
+                    +        var result = TypeCountFetchResult(
+                    +            typeIdentifier: monitoredType.id,
+                    +            displayName: monitoredType.displayName,
+                    +            count: -1,
+                    +            contentHash: "",
+                    +            earliestDate: nil,
+                    +            latestDate: nil,
+                    +            quality: .failed,
+                    +            diagnosticQuality: HealthKitAPICallResult.Status.failed.rawValue,
+                    +            isUnsupported: false,
+                    +            authorizationStatus: "unavailable",
+                    +            apiCalls: [
+                    +                Self.placeholderAPICall(
+                    +                    queryType: "record_import",
+                    +                    status: .unknown,
+                    +                    message: "Skipped because protected health data became inaccessible earlier in this run."
+                    +                ),
+                    +                call,
+                    +                companionCall
+                    +            ],
+                    +            yearlyCounts: [],
+                    +            distributionBins: [],
+                    +            records: [],
+                    +            recordArchiveData: nil
+                    +        )
+                    +        result.timeoutConfiguredSeconds = timeoutSeconds
+                    +        result.applyTimeoutProfile(timeoutProfile)
+                    +        progress?.updateDetails(from: result)
+                    +        progress?.updateStatus(monitoredType.id, status: .failed("Protected data unavailable"))
+                    +        return result
+                    +    }
+                         private func fetchTypeCountData(
+                             for monitoredType: MonitoredType,
+                             timeoutProfile: LocalMetricTimeoutProfile,
+                             if apiCalls.contains(where: { $0.status == .unauthorized }) {
+                                 return "denied"
+                    +        if apiCalls.contains(where: \.indicatesProtectedDataInaccessible) {
+                    +            return "unavailable"
+                    +        }
+                             return HealthKitAPICallResult.Status.unknown.rawValue
+                         var successCount: Int = 0
+                         var timingBreakdown: ImportTimingBreakdown = .zero
+                    +    var indicatesProtectedDataInaccessible: Bool {
+                    +        apiCalls.contains(where: \.indicatesProtectedDataInaccessible)
+                    +    }
+                         mutating func applyTimeoutProfile(_ profile: LocalMetricTimeoutProfile) {
+                             timeoutMode = profile.timeoutMode
+                             lastSuccessfulElapsed = profile.lastSuccessfulElapsed

+6 -0

HealthProbe/Utilities/SnapshotFetchProgress.swift

View

@@ -1,4 +1,5 @@
 import Foundation
+import HealthKit
 
 struct ImportTimingBreakdown: Sendable, Equatable {
     var fetchElapsedSeconds: TimeInterval = 0
@@ -108,6 +109,11 @@ struct HealthKitAPICallResult: Sendable, Equatable {
         case .unsupported: return "unsupported"
         }
     }
+
+    var indicatesProtectedDataInaccessible: Bool {
+        errorDomain == HKError.errorDomain
+            && errorCode == "\(HKError.Code.errorDatabaseInaccessible.rawValue)"
+    }
 }
 
 @Observable


+37 -0

HealthProbeTests/HealthKitAPICallResultTests.swift

View

+                    +import HealthKit
+                    +import XCTest
+                    +@testable import HealthProbe
+                    +final class HealthKitAPICallResultTests: XCTestCase {
+                    +    func testProtectedDataInaccessibleDetectionRecognizesHealthKitDatabaseLockError() {
+                    +        let result = HealthKitAPICallResult(
+                    +            queryType: "earliest_sample",
+                    +            status: .failed,
+                    +            elapsedSeconds: 0,
+                    +            resultValue: nil,
+                    +            errorCode: "\(HKError.Code.errorDatabaseInaccessible.rawValue)",
+                    +            errorDomain: HKError.errorDomain,
+                    +            errorDescription: "Protected health data is inaccessible because the device is locked.",
+                    +            failureKind: "HealthKit error",
+                    +            cancellationReason: nil
+                    +        )
+                    +        XCTAssertTrue(result.indicatesProtectedDataInaccessible)
+                    +    }
+                    +    func testProtectedDataInaccessibleDetectionIgnoresOtherHealthKitFailures() {
+                    +        let result = HealthKitAPICallResult(
+                    +            queryType: "earliest_sample",
+                    +            status: .failed,
+                    +            elapsedSeconds: 0,
+                    +            resultValue: nil,
+                    +            errorCode: "\(HKError.Code.errorAuthorizationDenied.rawValue)",
+                    +            errorDomain: HKError.errorDomain,
+                    +            errorDescription: "Authorization denied.",
+                    +            failureKind: "HealthKit error",
+                    +            cancellationReason: nil
+                    +        )
+                    +        XCTAssertFalse(result.indicatesProtectedDataInaccessible)
+                    +    }
+                    +}

	@@ -1,4 +1,5 @@
1	1	import Foundation
	2	+import HealthKit
2	3
3	4	struct ImportTimingBreakdown: Sendable, Equatable {
4	5	var fetchElapsedSeconds: TimeInterval = 0
	@@ -108,6 +109,11 @@ struct HealthKitAPICallResult: Sendable, Equatable {
108	109	case .unsupported: return "unsupported"
109	110	}
110	111	}
	112	+
	113	+ var indicatesProtectedDataInaccessible: Bool {
	114	+ errorDomain == HKError.errorDomain
	115	+ && errorCode == "\(HKError.Code.errorDatabaseInaccessible.rawValue)"
	116	+ }
111	117	}
112	118
113	119	@Observable