Persist import diagnostic reports · dac6e77

Persist import diagnostic reports
Browse files

bogdan committed 4 days ago

main

1 parent e121b0b

commit dac6e77

Showing 2 changed files with 80 additions and 4 deletions

+8 -0

HealthProbe/Doc/04-project/Import-Optimization-Log.md

@@ -577,6 +577,7 @@ rows exist".
 | 2026-06-03 | committed | Add explicit capture profile controls for full-dataset discovery. | A real-device report after registry expansion still showed `Types: 15/15 processed` and the old monitored type-set hash because `selectedTypeIDs` persisted the v1 core profile in `UserDefaults`. Settings now exposes `Select All Available Types`, `Select Core Profile`, and selected/available counts so the next real-device run can deliberately switch from the v1 sample set to the expanded supported registry. |
 | 2026-06-03 | pending | Migrate legacy core-profile selections to full available capture by default. | A follow-up real-device report still showed the old `4907...` monitored type-set hash and `Types: 15/15 processed`, proving the running app still used the old persisted selected type set. New installs and pre-profile settings that exactly match the old core profile now migrate to `All available`; only an explicit `Select Core Profile` action persists the core subset. Settings also shows the active profile label (`All available`, `Core`, or `Custom`) for quick verification before capture. |
 | 2026-06-03 | pending | Harden quantity unit conversion for full-profile imports. | The first 127-metric run crashed while archiving `HKQuantityTypeIdentifierDietaryWater`: the previous fallback converted unknown quantities with `.count()`, and HealthKit raised `NSInvalidArgumentException` for incompatible `mL` to `count`. The archive now maps known extended units, stores quantity rows with nil numeric/unit when a future type is unmapped, and removes the unsafe display fallback. Next run should pass Water and reveal any remaining type-specific unit gaps. |
+| 2026-06-03 | pending | Preserve completed import diagnostics and render large diagnostic reports lazily. | A copied full-profile reimport report completed successfully: `Types: 127/127 processed`, `Records: 2,646,527`, `WallClock: 40.3s`, `SummedProcessingElapsed: 21.7s`, `SummedFinalizeElapsed: 15.1s`, `SummedFetchElapsed: 1.7s`, `SummedInsertElapsed: 0.1s`, and `0` degraded metrics. The Diagnostics sheet copied the report text but displayed a blank body for the huge report; completed/partial/review reports are now persisted under Application Support and displayed in chunked lazy text blocks. |
 
 ## Current Diagnosis
 
@@ -611,12 +612,19 @@ The likely bottleneck is per-row SQLite work:
 - The validated import metrics are based on the original 15-type profile. The
   next correctness/performance question is full-dataset coverage and volume, not
   further confidence from the restricted sample alone.
+- Full-profile reimport after unit hardening completed successfully across 127
+  metrics and 2.6M archived rows. This is not comparable to clean first-import
+  timing, but it confirms that the expanded registry can pass the previous Water
+  unit crash point.
 
 ## Open Issues / Observations
 
 - Very small pages reduced freeze risk but introduced visible overhead.
 - Some progress timing displayed in the UI did not include overhead, so elapsed time and rates looked better than the real operation.
 - A previous Heart Rate import appeared to stall for long periods around roughly 900k records, but later progress resumed; avoid classifying this as a hard timeout without report evidence.
+- Diagnostic reports must remain available after each import. Copying from the
+  Diagnostics sheet is useful, but future comparisons should not depend on the
+  operator remembering to copy the visible report.
 - After a completed import, the app may remain unresponsive or crash in legacy
   post-import cache work. A 2026-06-03 console log showed Heart Rate and Active
   Energy `TypeCount.detailCacheData` precompute immediately before a Core Data


+72 -4

HealthProbe/Views/Dashboard/DashboardView.swift

View

@@ -12,6 +12,7 @@ struct DashboardView: View {
     @State private var expandedIssueIDs: Set<String> = []
     @State private var idleTimerWasDisabledBeforeSnapshot = false
     @State private var snapshotIdleTimerOverrideActive = false
+    @State private var savedDiagnosticSnapshotIDs: Set<UUID> = []
 
     private var latestArchiveObservation: CachedArchiveObservationRow? {
         viewModel.latestArchiveObservation
@@ -1037,12 +1038,34 @@ struct DashboardView: View {
         .onChange(of: viewModel.snapshotProgress) { _, newProgress in
             if newProgress == .incomplete || newProgress == .requiresResolution {
                 snapshotSheetTab = .report
+                persistCompletedDiagnosticReportIfNeeded()
             } else if newProgress == .fetching {
                 snapshotSheetTab = .progress
+            } else if newProgress == .complete {
+                persistCompletedDiagnosticReportIfNeeded()
             }
         }
     }
 
+    private func persistCompletedDiagnosticReportIfNeeded() {
+        guard let progress = viewModel.fetchProgress,
+              let snapshotID = viewModel.completedSnapshotID,
+              !savedDiagnosticSnapshotIDs.contains(snapshotID) else {
+            return
+        }
+        let text = buildDiagnosticText(progress, mode: .full)
+        do {
+            try DiagnosticReportStore.persist(
+                text: text,
+                snapshotID: snapshotID,
+                operationID: viewModel.operationID
+            )
+            savedDiagnosticSnapshotIDs.insert(snapshotID)
+        } catch {
+            print("Failed to persist diagnostic report: \(error.localizedDescription)")
+        }
+    }
+
     @ViewBuilder
     private var snapshotResultActionsSection: some View {
         let hasAuthorizationIssue = hasAuthorizationFailures(viewModel.fetchProgress)
@@ -1259,6 +1282,28 @@ private struct DiagnosticReport: Identifiable {
     let text: String
 }
 
+private enum DiagnosticReportStore {
+    static func persist(text: String, snapshotID: UUID, operationID: UUID?) throws {
+        let directory = URL.applicationSupportDirectory
+            .appending(path: "HealthProbeDiagnostics", directoryHint: .isDirectory)
+        try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
+
+        let timestamp = Self.filenameTimestamp(Date())
+        let operationPart = operationID.map { "_operation-\($0.uuidString)" } ?? ""
+        let filename = "\(timestamp)_snapshot-\(snapshotID.uuidString)\(operationPart).txt"
+        let fileURL = directory.appending(path: filename, directoryHint: .notDirectory)
+        try text.write(to: fileURL, atomically: true, encoding: .utf8)
+    }
+
+    private static func filenameTimestamp(_ date: Date) -> String {
+        let formatter = DateFormatter()
+        formatter.locale = Locale(identifier: "en_US_POSIX")
+        formatter.timeZone = TimeZone(secondsFromGMT: 0)
+        formatter.dateFormat = "yyyyMMdd_HHmmss"
+        return formatter.string(from: date)
+    }
+}
+
 // Avoids LabeledContent's TableRowContent ambiguity in List/Section contexts.
 private struct InfoRow<Content: View>: View {
     let label: String
@@ -1331,11 +1376,21 @@ private struct DiagnosticReportSheet: View {
     var body: some View {
         NavigationStack {
             ScrollView {
-                Text(reportText)
-                    .font(.system(.caption, design: .monospaced))
-                    .textSelection(.enabled)
-                    .frame(maxWidth: .infinity, alignment: .leading)
+                if reportText.isEmpty {
+                    ContentUnavailableView("No diagnostics available", systemImage: "doc.text.magnifyingglass")
+                        .padding(24)
+                } else {
+                    LazyVStack(alignment: .leading, spacing: 0) {
+                        ForEach(Array(Self.textChunks(reportText).enumerated()), id: \.offset) { _, chunk in
+                            Text(chunk)
+                                .font(.system(.caption, design: .monospaced))
+                                .foregroundStyle(.primary)
+                                .textSelection(.enabled)
+                                .frame(maxWidth: .infinity, alignment: .leading)
+                        }
+                    }
                     .padding(16)
+                }
             }
             .background(Color(.systemGroupedBackground))
             .navigationTitle("Diagnostics")
@@ -1371,6 +1426,19 @@ private struct DiagnosticReportSheet: View {
             }
         }
     }
+
+    private static func textChunks(_ text: String, chunkSize: Int = 4_000) -> [String] {
+        guard !text.isEmpty else { return [] }
+        var chunks: [String] = []
+        chunks.reserveCapacity(max(1, text.count / chunkSize))
+        var start = text.startIndex
+        while start < text.endIndex {
+            let end = text.index(start, offsetBy: chunkSize, limitedBy: text.endIndex) ?? text.endIndex
+            chunks.append(String(text[start..<end]))
+            start = end
+        }
+        return chunks
+    }
 }
 
 extension Bundle {


	@@ -577,6 +577,7 @@ rows exist".
577	577	\| 2026-06-03 \| committed \| Add explicit capture profile controls for full-dataset discovery. \| A real-device report after registry expansion still showed `Types: 15/15 processed` and the old monitored type-set hash because `selectedTypeIDs` persisted the v1 core profile in `UserDefaults`. Settings now exposes `Select All Available Types`, `Select Core Profile`, and selected/available counts so the next real-device run can deliberately switch from the v1 sample set to the expanded supported registry. \|
578	578	\| 2026-06-03 \| pending \| Migrate legacy core-profile selections to full available capture by default. \| A follow-up real-device report still showed the old `4907...` monitored type-set hash and `Types: 15/15 processed`, proving the running app still used the old persisted selected type set. New installs and pre-profile settings that exactly match the old core profile now migrate to `All available`; only an explicit `Select Core Profile` action persists the core subset. Settings also shows the active profile label (`All available`, `Core`, or `Custom`) for quick verification before capture. \|
579	579	\| 2026-06-03 \| pending \| Harden quantity unit conversion for full-profile imports. \| The first 127-metric run crashed while archiving `HKQuantityTypeIdentifierDietaryWater`: the previous fallback converted unknown quantities with `.count()`, and HealthKit raised `NSInvalidArgumentException` for incompatible `mL` to `count`. The archive now maps known extended units, stores quantity rows with nil numeric/unit when a future type is unmapped, and removes the unsafe display fallback. Next run should pass Water and reveal any remaining type-specific unit gaps. \|
	580	+\| 2026-06-03 \| pending \| Preserve completed import diagnostics and render large diagnostic reports lazily. \| A copied full-profile reimport report completed successfully: `Types: 127/127 processed`, `Records: 2,646,527`, `WallClock: 40.3s`, `SummedProcessingElapsed: 21.7s`, `SummedFinalizeElapsed: 15.1s`, `SummedFetchElapsed: 1.7s`, `SummedInsertElapsed: 0.1s`, and `0` degraded metrics. The Diagnostics sheet copied the report text but displayed a blank body for the huge report; completed/partial/review reports are now persisted under Application Support and displayed in chunked lazy text blocks. \|
580	581
581	582	## Current Diagnosis
582	583
	@@ -611,12 +612,19 @@ The likely bottleneck is per-row SQLite work:
611	612	- The validated import metrics are based on the original 15-type profile. The
612	613	next correctness/performance question is full-dataset coverage and volume, not
613	614	further confidence from the restricted sample alone.
	615	+- Full-profile reimport after unit hardening completed successfully across 127
	616	+ metrics and 2.6M archived rows. This is not comparable to clean first-import
	617	+ timing, but it confirms that the expanded registry can pass the previous Water
	618	+ unit crash point.
614	619
615	620	## Open Issues / Observations
616	621
617	622	- Very small pages reduced freeze risk but introduced visible overhead.
618	623	- Some progress timing displayed in the UI did not include overhead, so elapsed time and rates looked better than the real operation.
619	624	- A previous Heart Rate import appeared to stall for long periods around roughly 900k records, but later progress resumed; avoid classifying this as a hard timeout without report evidence.
	625	+- Diagnostic reports must remain available after each import. Copying from the
	626	+ Diagnostics sheet is useful, but future comparisons should not depend on the
	627	+ operator remembering to copy the visible report.
620	628	- After a completed import, the app may remain unresponsive or crash in legacy
621	629	post-import cache work. A 2026-06-03 console log showed Heart Rate and Active
622	630	Energy `TypeCount.detailCacheData` precompute immediately before a Core Data

	@@ -12,6 +12,7 @@ struct DashboardView: View {
12	12	@State private var expandedIssueIDs: Set<String> = []
13	13	@State private var idleTimerWasDisabledBeforeSnapshot = false
14	14	@State private var snapshotIdleTimerOverrideActive = false
	15	+ @State private var savedDiagnosticSnapshotIDs: Set<UUID> = []
15	16
16	17	private var latestArchiveObservation: CachedArchiveObservationRow? {
17	18	viewModel.latestArchiveObservation
	@@ -1037,12 +1038,34 @@ struct DashboardView: View {
1037	1038	.onChange(of: viewModel.snapshotProgress) { _, newProgress in
1038	1039	if newProgress == .incomplete \|\| newProgress == .requiresResolution {
1039	1040	snapshotSheetTab = .report
	1041	+ persistCompletedDiagnosticReportIfNeeded()
1040	1042	} else if newProgress == .fetching {
1041	1043	snapshotSheetTab = .progress
	1044	+ } else if newProgress == .complete {
	1045	+ persistCompletedDiagnosticReportIfNeeded()
1042	1046	}
1043	1047	}
1044	1048	}
1045	1049
	1050	+ private func persistCompletedDiagnosticReportIfNeeded() {
	1051	+ guard let progress = viewModel.fetchProgress,
	1052	+ let snapshotID = viewModel.completedSnapshotID,
	1053	+ !savedDiagnosticSnapshotIDs.contains(snapshotID) else {
	1054	+ return
	1055	+ }
	1056	+ let text = buildDiagnosticText(progress, mode: .full)
	1057	+ do {
	1058	+ try DiagnosticReportStore.persist(
	1059	+ text: text,
	1060	+ snapshotID: snapshotID,
	1061	+ operationID: viewModel.operationID
	1062	+ )
	1063	+ savedDiagnosticSnapshotIDs.insert(snapshotID)
	1064	+ } catch {
	1065	+ print("Failed to persist diagnostic report: \(error.localizedDescription)")
	1066	+ }
	1067	+ }
	1068	+
1046	1069	@ViewBuilder
1047	1070	private var snapshotResultActionsSection: some View {
1048	1071	let hasAuthorizationIssue = hasAuthorizationFailures(viewModel.fetchProgress)
	@@ -1259,6 +1282,28 @@ private struct DiagnosticReport: Identifiable {
1259	1282	let text: String
1260	1283	}
1261	1284
	1285	+private enum DiagnosticReportStore {
	1286	+ static func persist(text: String, snapshotID: UUID, operationID: UUID?) throws {
	1287	+ let directory = URL.applicationSupportDirectory
	1288	+ .appending(path: "HealthProbeDiagnostics", directoryHint: .isDirectory)
	1289	+ try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
	1290	+
	1291	+ let timestamp = Self.filenameTimestamp(Date())
	1292	+ let operationPart = operationID.map { "_operation-\($0.uuidString)" } ?? ""
	1293	+ let filename = "\(timestamp)_snapshot-\(snapshotID.uuidString)\(operationPart).txt"
	1294	+ let fileURL = directory.appending(path: filename, directoryHint: .notDirectory)
	1295	+ try text.write(to: fileURL, atomically: true, encoding: .utf8)
	1296	+ }
	1297	+
	1298	+ private static func filenameTimestamp(_ date: Date) -> String {
	1299	+ let formatter = DateFormatter()
	1300	+ formatter.locale = Locale(identifier: "en_US_POSIX")
	1301	+ formatter.timeZone = TimeZone(secondsFromGMT: 0)
	1302	+ formatter.dateFormat = "yyyyMMdd_HHmmss"
	1303	+ return formatter.string(from: date)
	1304	+ }
	1305	+}
	1306	+
1262	1307	// Avoids LabeledContent's TableRowContent ambiguity in List/Section contexts.
1263	1308	private struct InfoRow<Content: View>: View {
1264	1309	let label: String
	@@ -1331,11 +1376,21 @@ private struct DiagnosticReportSheet: View {
1331	1376	var body: some View {
1332	1377	NavigationStack {
1333	1378	ScrollView {
1334		- Text(reportText)
1335		- .font(.system(.caption, design: .monospaced))
1336		- .textSelection(.enabled)
1337		- .frame(maxWidth: .infinity, alignment: .leading)
	1379	+ if reportText.isEmpty {
	1380	+ ContentUnavailableView("No diagnostics available", systemImage: "doc.text.magnifyingglass")
	1381	+ .padding(24)
	1382	+ } else {
	1383	+ LazyVStack(alignment: .leading, spacing: 0) {
	1384	+ ForEach(Array(Self.textChunks(reportText).enumerated()), id: \.offset) { _, chunk in
	1385	+ Text(chunk)
	1386	+ .font(.system(.caption, design: .monospaced))
	1387	+ .foregroundStyle(.primary)
	1388	+ .textSelection(.enabled)
	1389	+ .frame(maxWidth: .infinity, alignment: .leading)
	1390	+ }
	1391	+ }
1338	1392	.padding(16)
	1393	+ }
1339	1394	}
1340	1395	.background(Color(.systemGroupedBackground))
1341	1396	.navigationTitle("Diagnostics")
	@@ -1371,6 +1426,19 @@ private struct DiagnosticReportSheet: View {
1371	1426	}
1372	1427	}
1373	1428	}
	1429	+
	1430	+ private static func textChunks(_ text: String, chunkSize: Int = 4_000) -> [String] {
	1431	+ guard !text.isEmpty else { return [] }
	1432	+ var chunks: [String] = []
	1433	+ chunks.reserveCapacity(max(1, text.count / chunkSize))
	1434	+ var start = text.startIndex
	1435	+ while start < text.endIndex {
	1436	+ let end = text.index(start, offsetBy: chunkSize, limitedBy: text.endIndex) ?? text.endIndex
	1437	+ chunks.append(String(text[start..<end]))
	1438	+ start = end
	1439	+ }
	1440	+ return chunks
	1441	+ }
1374	1442	}
1375	1443
1376	1444	extension Bundle {