Move data type diffs off SwiftData fallback · 2e39fc3

Move data type diffs off SwiftData fallback
Browse files

bogdan committed a week ago

main

1 parent e725581

commit 2e39fc3

Showing 5 changed files with 62 additions and 25 deletions

+1 -1

HealthProbe/Doc/04-project/IMPLEMENTATION_STATUS.md

@@ -28,7 +28,7 @@ There are no real deployments, only test installations. Existing prototype datab
 | SQLite archive | Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed | Move Snapshots/Data Types from SwiftData previews to archive/cache DTOs |
 | Core Data cache | Initial programmatic Core Data model, full-cache rebuild service, read DTOs for observation/type/diff/health rows, and Dashboard archive-cache status wiring are in place | Move remaining export/report paths to cache DTOs and add targeted partial invalidation |
 | SwiftData cache | Exists; test builds now reset legacy prototype UI/archive/cache stores once for archive v2 so old SwiftData-only snapshots are not treated as backed-up observations. Metric timeout calibration, local device profile settings, operation logging, ContentView preview, Settings data maintenance, and legacy anomaly/count-drop review have moved outside SwiftData or been removed. Remaining SwiftData imports are inventoried in [`SwiftData-Retirement-Inventory.md`](SwiftData-Retirement-Inventory.md) | Treat as disposable prototype data; replace capture review actions and navigation handles before removing `ModelContainer` |
-| UI | Prototype exists; Snapshots/Data Types now default to the local device timeline instead of a multi-device picker. Dashboard status reads archive/cache observation rows and shows cache health, with SwiftData retained only for capture/review actions; Snapshots timeline reads archive/cache rows when available and no longer queries `SnapshotDelta` for list summaries; snapshot detail summaries/type rows and Data Types list prefer Core Data cache rows when archive observation ids exist; data type detail reads Core Data type/diff summaries and uses SQLite `diffRecords` for paged drill-down; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language, with SwiftData detail cache as transition fallback | Remove remaining SwiftData navigation handles |
+| UI | Prototype exists; Snapshots/Data Types now default to the local device timeline instead of a multi-device picker. Dashboard status reads archive/cache observation rows and shows cache health, with SwiftData retained only for capture/review actions; Snapshots timeline reads archive/cache rows when available and no longer queries `SnapshotDelta` for list summaries; snapshot detail summaries/type rows and Data Types list prefer Core Data cache rows when archive observation ids exist, with Data Types diff rows no longer falling back to SwiftData `TypeCount` traversal; data type detail reads Core Data type/diff summaries and uses SQLite `diffRecords` for paged drill-down; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language, with SwiftData detail cache as transition fallback | Remove remaining SwiftData navigation handles |
 | Diff/change explanation | SQL diff summaries, paged diff records, aggregate comparisons, and consolidation-evidence labels exist; legacy anomaly/count-drop review has been removed from active flows | Continue moving remaining SwiftData fallback detail paths to archive/cache DTOs |
 | Export | SQLite export preview, paged JSON writing, SHA256 manifest hashing, and `export_manifests` rows are in place for selected records and observation diffs | Fill remaining recovery-compatible envelope metadata, CSV export, relationship preservation, and reproducibility checks |
 | Legacy device support | Simplified detail UI mode is implemented for small/accessibility layouts and as a Settings toggle | Remove SwiftData dependency and validate lower deployment targets |


+1 -1

HealthProbe/Doc/04-project/Refactoring-Plan.md

View

@@ -227,7 +227,7 @@ Checklist:
   longer query `SnapshotDelta` list summaries, while retaining SwiftData handles
   for detail navigation during transition.
 - [x] Observation detail uses cached summary/type rows plus SQLite diff summaries when archive observation ids exist.
-- [x] Data Types list rows prefer Core Data cached counts plus SQLite `diffSummary` when archive observation ids exist.
+- [x] Data Types list rows use Core Data cached counts plus SQLite `diffSummary` and no longer fall back to SwiftData `TypeCount` traversal.
 - [x] Data type detail uses SQLite `diffSummary` when archive observation ids exist.
 - [x] Data type new/missing drill-down pages through SQLite `diffRecords` when archive observation ids exist.
 - [x] Diff detail fully uses cached summary plus paged SQLite DTOs.


+4 -0

HealthProbe/Doc/04-project/SwiftData-Retirement-Inventory.md

View

@@ -119,6 +119,10 @@ The following SwiftData dependencies were removed from active flows:
   `SnapshotDelta` or runs `DeltaService` list-summary repair. Timeline change
   summaries come from archive/cache rows when available; SwiftData remains there
   only for temporary snapshot navigation/deletion handles.
+- `HealthProbe/ViewModels/DataTypesViewModel.swift` now resolves baselines from
+  small observation contexts and builds rows from Core Data cache + SQLite
+  archive diff APIs. It no longer falls back to `SnapshotDiffService.diff(...)`
+  over SwiftData `TypeCount` relationships.
 - `HealthProbe/Models/AnomalyRecord.swift`,
   `HealthProbe/Models/AnomalyType.swift`, and
   `HealthProbe/Services/AnomalyDetector.swift` were deleted. The app no longer


+30 -17

HealthProbe/ViewModels/DataTypesViewModel.swift

View

@@ -1,5 +1,11 @@
 import Foundation
 
+struct DataTypeSnapshotContext: Equatable, Sendable {
+    let id: UUID
+    let observedAt: Date
+    let archiveObservationID: Int64?
+}
+
 @Observable
 final class DataTypesViewModel {
     var filter: DiffFilter = .all
@@ -7,19 +13,12 @@ final class DataTypesViewModel {
     var archiveDiffs: [TypeDiff]?
     var archiveDiffError: String?
 
-    private let diffService = SnapshotDiffService.shared
-
-    func diffs(current: HealthSnapshot?, snapshots: [HealthSnapshot]) -> [TypeDiff] {
-        if let archiveDiffs {
-            return diffService.apply(filter: filter, to: archiveDiffs)
-        }
-        guard let current else { return [] }
-        guard let baseline = resolveBaseline(for: current, in: snapshots) else { return [] }
-        let all = diffService.diff(current: current, baseline: baseline)
-        return diffService.apply(filter: filter, to: all)
+    func diffs() -> [TypeDiff] {
+        guard let archiveDiffs else { return [] }
+        return apply(filter: filter, to: archiveDiffs)
     }
 
-    func baseline(for snapshot: HealthSnapshot, in snapshots: [HealthSnapshot]) -> HealthSnapshot? {
+    func baseline(for snapshot: DataTypeSnapshotContext, in snapshots: [DataTypeSnapshotContext]) -> DataTypeSnapshotContext? {
         resolveBaseline(for: snapshot, in: snapshots)
     }
 
@@ -29,7 +28,7 @@ final class DataTypesViewModel {
     }
 
     @MainActor
-    func loadArchiveDiffs(current: HealthSnapshot?, snapshots: [HealthSnapshot]) async {
+    func loadArchiveDiffs(current: DataTypeSnapshotContext?, snapshots: [DataTypeSnapshotContext]) async {
         guard let current,
               let baseline = resolveBaseline(for: current, in: snapshots),
               let currentObservationID = current.archiveObservationID,
@@ -80,16 +79,30 @@ final class DataTypesViewModel {
         }
     }
 
-    private func resolveBaseline(for snapshot: HealthSnapshot, in snapshots: [HealthSnapshot]) -> HealthSnapshot? {
-        let sorted = snapshots.sorted { $0.timestamp > $1.timestamp }
+    private func resolveBaseline(
+        for snapshot: DataTypeSnapshotContext,
+        in snapshots: [DataTypeSnapshotContext]
+    ) -> DataTypeSnapshotContext? {
+        let sorted = snapshots.sorted { $0.observedAt > $1.observedAt }
         switch comparisonMode {
         case .previous:
-            return sorted.first { $0.timestamp < snapshot.timestamp }
+            return sorted.first { $0.observedAt < snapshot.observedAt }
         case .selected:
             return nil  // DataTypesView uses .previous by default; selection lives in SnapshotsTab
         case .relativeTime(let interval):
-            let target = snapshot.timestamp.addingTimeInterval(-interval)
-            return diffService.nearest(to: target, in: snapshots)
+            let target = snapshot.observedAt.addingTimeInterval(-interval)
+            return snapshots
+                .filter { $0.observedAt <= target }
+                .max { $0.observedAt < $1.observedAt }
+        }
+    }
+
+    private func apply(filter: DiffFilter, to diffs: [TypeDiff]) -> [TypeDiff] {
+        switch filter {
+        case .all:       return diffs
+        case .changed:   return diffs.filter { $0.previousTracked && $0.hasChanges }
+        case .increased: return diffs.filter { $0.previousTracked && $0.delta > 0 }
+        case .decreased: return diffs.filter { $0.previousTracked && $0.delta < 0 }
         }
     }
 }


+26 -6

HealthProbe/Views/DataTypes/DataTypesView.swift

View

@@ -12,14 +12,34 @@ struct DataTypesView: View {
 
     private var latest: HealthSnapshot? { displayedSnapshots.first }
 
-    private var currentBaseline: HealthSnapshot? {
-        guard let latest else { return nil }
-        return viewModel.baseline(for: latest, in: displayedSnapshots)
+    private var displayedSnapshotContexts: [DataTypeSnapshotContext] {
+        displayedSnapshots.map {
+            DataTypeSnapshotContext(
+                id: $0.id,
+                observedAt: $0.timestamp,
+                archiveObservationID: $0.archiveObservationID
+            )
+        }
+    }
+
+    private var latestContext: DataTypeSnapshotContext? {
+        latest.map {
+            DataTypeSnapshotContext(
+                id: $0.id,
+                observedAt: $0.timestamp,
+                archiveObservationID: $0.archiveObservationID
+            )
+        }
+    }
+
+    private var currentBaseline: DataTypeSnapshotContext? {
+        guard let latestContext else { return nil }
+        return viewModel.baseline(for: latestContext, in: displayedSnapshotContexts)
     }
 
     private var archiveDiffTaskID: String {
         [
-            latest?.id.uuidString ?? "none",
+            latestContext?.id.uuidString ?? "none",
             currentBaseline?.id.uuidString ?? "none",
             String(describing: viewModel.comparisonMode)
         ].joined(separator: "|")
@@ -50,7 +70,7 @@ struct DataTypesView: View {
             .navigationTitle("Data Types")
             .toolbar { filterPicker }
             .task(id: archiveDiffTaskID) {
-                await viewModel.loadArchiveDiffs(current: latest, snapshots: displayedSnapshots)
+                await viewModel.loadArchiveDiffs(current: latestContext, snapshots: displayedSnapshotContexts)
             }
         }
     }
@@ -58,7 +78,7 @@ struct DataTypesView: View {
     // MARK: - List
 
     private var typeList: some View {
-        let diffs = viewModel.diffs(current: latest, snapshots: displayedSnapshots)
+        let diffs = viewModel.diffs()
         return List {
             comparisonModeHeader
             if diffs.isEmpty {


	@@ -227,7 +227,7 @@ Checklist:
227	227	longer query `SnapshotDelta` list summaries, while retaining SwiftData handles
228	228	for detail navigation during transition.
229	229	- [x] Observation detail uses cached summary/type rows plus SQLite diff summaries when archive observation ids exist.
230		-- [x] Data Types list rows prefer Core Data cached counts plus SQLite `diffSummary` when archive observation ids exist.
	230	+- [x] Data Types list rows use Core Data cached counts plus SQLite `diffSummary` and no longer fall back to SwiftData `TypeCount` traversal.
231	231	- [x] Data type detail uses SQLite `diffSummary` when archive observation ids exist.
232	232	- [x] Data type new/missing drill-down pages through SQLite `diffRecords` when archive observation ids exist.
233	233	- [x] Diff detail fully uses cached summary plus paged SQLite DTOs.

	@@ -119,6 +119,10 @@ The following SwiftData dependencies were removed from active flows:
119	119	`SnapshotDelta` or runs `DeltaService` list-summary repair. Timeline change
120	120	summaries come from archive/cache rows when available; SwiftData remains there
121	121	only for temporary snapshot navigation/deletion handles.
	122	+- `HealthProbe/ViewModels/DataTypesViewModel.swift` now resolves baselines from
	123	+ small observation contexts and builds rows from Core Data cache + SQLite
	124	+ archive diff APIs. It no longer falls back to `SnapshotDiffService.diff(...)`
	125	+ over SwiftData `TypeCount` relationships.
122	126	- `HealthProbe/Models/AnomalyRecord.swift`,
123	127	`HealthProbe/Models/AnomalyType.swift`, and
124	128	`HealthProbe/Services/AnomalyDetector.swift` were deleted. The app no longer

	@@ -28,7 +28,7 @@ There are no real deployments, only test installations. Existing prototype datab
28	28	\| SQLite archive \| Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed \| Move Snapshots/Data Types from SwiftData previews to archive/cache DTOs \|
29	29	\| Core Data cache \| Initial programmatic Core Data model, full-cache rebuild service, read DTOs for observation/type/diff/health rows, and Dashboard archive-cache status wiring are in place \| Move remaining export/report paths to cache DTOs and add targeted partial invalidation \|
30	30	\| SwiftData cache \| Exists; test builds now reset legacy prototype UI/archive/cache stores once for archive v2 so old SwiftData-only snapshots are not treated as backed-up observations. Metric timeout calibration, local device profile settings, operation logging, ContentView preview, Settings data maintenance, and legacy anomaly/count-drop review have moved outside SwiftData or been removed. Remaining SwiftData imports are inventoried in [`SwiftData-Retirement-Inventory.md`](SwiftData-Retirement-Inventory.md) \| Treat as disposable prototype data; replace capture review actions and navigation handles before removing `ModelContainer` \|
31		-\| UI \| Prototype exists; Snapshots/Data Types now default to the local device timeline instead of a multi-device picker. Dashboard status reads archive/cache observation rows and shows cache health, with SwiftData retained only for capture/review actions; Snapshots timeline reads archive/cache rows when available and no longer queries `SnapshotDelta` for list summaries; snapshot detail summaries/type rows and Data Types list prefer Core Data cache rows when archive observation ids exist; data type detail reads Core Data type/diff summaries and uses SQLite `diffRecords` for paged drill-down; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language, with SwiftData detail cache as transition fallback \| Remove remaining SwiftData navigation handles \|
	31	+\| UI \| Prototype exists; Snapshots/Data Types now default to the local device timeline instead of a multi-device picker. Dashboard status reads archive/cache observation rows and shows cache health, with SwiftData retained only for capture/review actions; Snapshots timeline reads archive/cache rows when available and no longer queries `SnapshotDelta` for list summaries; snapshot detail summaries/type rows and Data Types list prefer Core Data cache rows when archive observation ids exist, with Data Types diff rows no longer falling back to SwiftData `TypeCount` traversal; data type detail reads Core Data type/diff summaries and uses SQLite `diffRecords` for paged drill-down; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language, with SwiftData detail cache as transition fallback \| Remove remaining SwiftData navigation handles \|
32	32	\| Diff/change explanation \| SQL diff summaries, paged diff records, aggregate comparisons, and consolidation-evidence labels exist; legacy anomaly/count-drop review has been removed from active flows \| Continue moving remaining SwiftData fallback detail paths to archive/cache DTOs \|
33	33	\| Export \| SQLite export preview, paged JSON writing, SHA256 manifest hashing, and `export_manifests` rows are in place for selected records and observation diffs \| Fill remaining recovery-compatible envelope metadata, CSV export, relationship preservation, and reproducibility checks \|
34	34	\| Legacy device support \| Simplified detail UI mode is implemented for small/accessibility layouts and as a Settings toggle \| Remove SwiftData dependency and validate lower deployment targets \|

	@@ -1,5 +1,11 @@
1	1	import Foundation
2	2
	3	+struct DataTypeSnapshotContext: Equatable, Sendable {
	4	+ let id: UUID
	5	+ let observedAt: Date
	6	+ let archiveObservationID: Int64?
	7	+}
	8	+
3	9	@Observable
4	10	final class DataTypesViewModel {
5	11	var filter: DiffFilter = .all
	@@ -7,19 +13,12 @@ final class DataTypesViewModel {
7	13	var archiveDiffs: [TypeDiff]?
8	14	var archiveDiffError: String?
9	15
10		- private let diffService = SnapshotDiffService.shared
11		-
12		- func diffs(current: HealthSnapshot?, snapshots: [HealthSnapshot]) -> [TypeDiff] {
13		- if let archiveDiffs {
14		- return diffService.apply(filter: filter, to: archiveDiffs)
15		- }
16		- guard let current else { return [] }
17		- guard let baseline = resolveBaseline(for: current, in: snapshots) else { return [] }
18		- let all = diffService.diff(current: current, baseline: baseline)
19		- return diffService.apply(filter: filter, to: all)
	16	+ func diffs() -> [TypeDiff] {
	17	+ guard let archiveDiffs else { return [] }
	18	+ return apply(filter: filter, to: archiveDiffs)
20	19	}
21	20
22		- func baseline(for snapshot: HealthSnapshot, in snapshots: [HealthSnapshot]) -> HealthSnapshot? {
	21	+ func baseline(for snapshot: DataTypeSnapshotContext, in snapshots: [DataTypeSnapshotContext]) -> DataTypeSnapshotContext? {
23	22	resolveBaseline(for: snapshot, in: snapshots)
24	23	}
25	24
	@@ -29,7 +28,7 @@ final class DataTypesViewModel {
29	28	}
30	29
31	30	@MainActor
32		- func loadArchiveDiffs(current: HealthSnapshot?, snapshots: [HealthSnapshot]) async {
	31	+ func loadArchiveDiffs(current: DataTypeSnapshotContext?, snapshots: [DataTypeSnapshotContext]) async {
33	32	guard let current,
34	33	let baseline = resolveBaseline(for: current, in: snapshots),
35	34	let currentObservationID = current.archiveObservationID,
	@@ -80,16 +79,30 @@ final class DataTypesViewModel {
80	79	}
81	80	}
82	81
83		- private func resolveBaseline(for snapshot: HealthSnapshot, in snapshots: [HealthSnapshot]) -> HealthSnapshot? {
84		- let sorted = snapshots.sorted { $0.timestamp > $1.timestamp }
	82	+ private func resolveBaseline(
	83	+ for snapshot: DataTypeSnapshotContext,
	84	+ in snapshots: [DataTypeSnapshotContext]
	85	+ ) -> DataTypeSnapshotContext? {
	86	+ let sorted = snapshots.sorted { $0.observedAt > $1.observedAt }
85	87	switch comparisonMode {
86	88	case .previous:
87		- return sorted.first { $0.timestamp < snapshot.timestamp }
	89	+ return sorted.first { $0.observedAt < snapshot.observedAt }
88	90	case .selected:
89	91	return nil // DataTypesView uses .previous by default; selection lives in SnapshotsTab
90	92	case .relativeTime(let interval):
91		- let target = snapshot.timestamp.addingTimeInterval(-interval)
92		- return diffService.nearest(to: target, in: snapshots)
	93	+ let target = snapshot.observedAt.addingTimeInterval(-interval)
	94	+ return snapshots
	95	+ .filter { $0.observedAt <= target }
	96	+ .max { $0.observedAt < $1.observedAt }
	97	+ }
	98	+ }
	99	+
	100	+ private func apply(filter: DiffFilter, to diffs: [TypeDiff]) -> [TypeDiff] {
	101	+ switch filter {
	102	+ case .all: return diffs
	103	+ case .changed: return diffs.filter { $0.previousTracked && $0.hasChanges }
	104	+ case .increased: return diffs.filter { $0.previousTracked && $0.delta > 0 }
	105	+ case .decreased: return diffs.filter { $0.previousTracked && $0.delta < 0 }
93	106	}
94	107	}
95	108	}

	@@ -12,14 +12,34 @@ struct DataTypesView: View {
12	12
13	13	private var latest: HealthSnapshot? { displayedSnapshots.first }
14	14
15		- private var currentBaseline: HealthSnapshot? {
16		- guard let latest else { return nil }
17		- return viewModel.baseline(for: latest, in: displayedSnapshots)
	15	+ private var displayedSnapshotContexts: [DataTypeSnapshotContext] {
	16	+ displayedSnapshots.map {
	17	+ DataTypeSnapshotContext(
	18	+ id: $0.id,
	19	+ observedAt: $0.timestamp,
	20	+ archiveObservationID: $0.archiveObservationID
	21	+ )
	22	+ }
	23	+ }
	24	+
	25	+ private var latestContext: DataTypeSnapshotContext? {
	26	+ latest.map {
	27	+ DataTypeSnapshotContext(
	28	+ id: $0.id,
	29	+ observedAt: $0.timestamp,
	30	+ archiveObservationID: $0.archiveObservationID
	31	+ )
	32	+ }
	33	+ }
	34	+
	35	+ private var currentBaseline: DataTypeSnapshotContext? {
	36	+ guard let latestContext else { return nil }
	37	+ return viewModel.baseline(for: latestContext, in: displayedSnapshotContexts)
18	38	}
19	39
20	40	private var archiveDiffTaskID: String {
21	41	[
22		- latest?.id.uuidString ?? "none",
	42	+ latestContext?.id.uuidString ?? "none",
23	43	currentBaseline?.id.uuidString ?? "none",
24	44	String(describing: viewModel.comparisonMode)
25	45	].joined(separator: "\|")
	@@ -50,7 +70,7 @@ struct DataTypesView: View {
50	70	.navigationTitle("Data Types")
51	71	.toolbar { filterPicker }
52	72	.task(id: archiveDiffTaskID) {
53		- await viewModel.loadArchiveDiffs(current: latest, snapshots: displayedSnapshots)
	73	+ await viewModel.loadArchiveDiffs(current: latestContext, snapshots: displayedSnapshotContexts)
54	74	}
55	75	}
56	76	}
	@@ -58,7 +78,7 @@ struct DataTypesView: View {
58	78	// MARK: - List
59	79
60	80	private var typeList: some View {
61		- let diffs = viewModel.diffs(current: latest, snapshots: displayedSnapshots)
	81	+ let diffs = viewModel.diffs()
62	82	return List {
63	83	comparisonModeHeader
64	84	if diffs.isEmpty {