Showing 2 changed files with 64 additions and 25 deletions
+1 -1
HealthProbe/Doc/04-project/IMPLEMENTATION_STATUS.md
@@ -25,7 +25,7 @@ There are no real deployments, only test installations. Existing prototype datab
25 25
 |------|----------------|--------------------|
26 26
 | Product docs | Updated | Keep `HealthProbe/Doc/README.md` as canonical index |
27 27
 | HealthKit capture | Capture now opens one archive observation per user-visible snapshot, attaches HealthKit pages, deleted-object evidence, and type verification to that observation id before finishing it, no longer aborts initial full-history imports after a fixed 30-minute wall-clock cap while page-level HealthKit timeouts remain in place, defers grouped observation summary/daily aggregate rebuilds until per-type verification instead of rebuilding after every imported page, and persists large HealthKit pages in smaller archive chunks while using type-specific import strategies: conservative paging for the heaviest metrics, more aggressive pages/chunks for ordinary metrics, adaptive write chunk sizing, batched deleted-object persistence, explicit task yields, and lower-allocation streaming loops to avoid long monolithic SQLite stalls | Continue moving UI/cache reads to archive-backed observation ids and revisit full checkpoint/resume and background collection separately |
28
-| SQLite archive | Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed, the hot write path now reuses prepared SQLite statements within grouped page writes instead of reparsing the same SQL for every sample, caches repeated sample-type/source/source-revision/device/metadata id lookups within grouped writes, skips redundant visibility close/existence checks when grouped imports create a brand-new sample or payload version, processes sample rows in a lower-allocation streaming loop, batches same-page deleted-object evidence in one transaction, adds composite indexes for visibility-range and sample-uuid hot lookups, and opens SQLite connections with import-friendly busy timeout / synchronous / temp-store pragmas | Continue moving capture/Dashboard actions to archive/cache DTOs |
28
+| SQLite archive | Archive v2 schema, snapshot-level observation grouping, differential write path, v2 verification/delete bookkeeping, daily aggregate rebuilds, integrity report, v2 record reads, SQL diff/count/aggregate/provenance/consolidation-evidence APIs, large synthetic diff pagination coverage, formal timing/memory metrics, and XCTest coverage are in place; the legacy `archive_samples` mirror has been removed, the hot write path now reuses prepared SQLite statements within grouped page writes instead of reparsing the same SQL for every sample, caches repeated sample-type/source/source-revision/device/metadata id lookups within grouped writes, skips redundant visibility close/existence checks when grouped imports create a brand-new sample or payload version, reuses verification aggregates instead of rescanning them twice, drives per-type finalize queries from sample-type-filtered sample ids, processes sample rows in a lower-allocation streaming loop, batches same-page deleted-object evidence in one transaction, adds composite indexes for visibility-range and sample-uuid hot lookups, and opens SQLite connections with import-friendly busy timeout / synchronous / temp-store pragmas | Continue moving capture/Dashboard actions to archive/cache DTOs |
29 29
 | Core Data cache | Initial programmatic Core Data model, full-cache rebuild service, read DTOs for observation/type/diff/health rows, and Dashboard archive-cache status wiring are in place | Move remaining export/report paths to cache DTOs and add targeted partial invalidation |
30 30
 | SwiftData cache | Exists; test builds now reset legacy prototype UI/archive/cache stores once for archive v2 so old SwiftData-only snapshots are not treated as backed-up observations. Metric timeout calibration, local device profile settings, operation logging, ContentView preview, Settings data maintenance, legacy detail/PDF views, unused legacy repair/observer services, Dashboard view/view-model access, and legacy anomaly/count-drop review have moved outside SwiftData or been removed. Remaining SwiftData imports are inventoried in [`SwiftData-Retirement-Inventory.md`](SwiftData-Retirement-Inventory.md) | Treat as disposable prototype data; stop returning/storing `HealthSnapshot` bridge handles before removing `ModelContainer` |
31 31
 | UI | Prototype exists; Dashboard status reads archive/cache observation rows and shows cache health, and Dashboard view/view-model code no longer imports SwiftData or reads `ModelContext`; capture/review actions now route through DTOs and snapshot ids, with the remaining legacy bridge isolated in `HealthKitService`. Snapshots and Data Types tab roots no longer import SwiftData, load Core Data cached observation rows, and open archive/cache-backed detail rows; `SnapshotArchiveDetailView` and `DataTypeArchiveDetailView` read Core Data type/diff summaries and page record drill-down through SQLite; unused legacy SwiftData snapshot/type detail and PDF views have been deleted; record-change evolution and temporal distribution screens now receive DTO rows/cache input instead of querying SwiftData directly; export preview reads the archive export API before showing/exporting JSON; simplified detail mode replaces heavy charts with summary rows on small/accessibility layouts or when enabled in Settings; visible change labels now use neutral new/missing/change-review language; Settings can now schedule a full test-database reset for the next app launch | Stop writing prototype `HealthSnapshot` bridge rows during capture/review |
+63 -24
HealthProbe/Services/SQLiteHealthArchiveStore.swift
@@ -1660,6 +1660,7 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1660 1660
         )
1661 1661
         """, db: db)
1662 1662
         try execute("CREATE INDEX IF NOT EXISTS idx_samples_uuid_hash ON samples(sample_uuid_hash)", db: db)
1663
+        try execute("CREATE INDEX IF NOT EXISTS idx_samples_type_id ON samples(sample_type_id, id)", db: db)
1663 1664
         try execute("CREATE INDEX IF NOT EXISTS idx_samples_type_uuid_hash ON samples(sample_type_id, sample_uuid_hash)", db: db)
1664 1665
         try execute("CREATE INDEX IF NOT EXISTS idx_samples_type_semantic ON samples(sample_type_id, semantic_fingerprint)", db: db)
1665 1666
         try execute("""
@@ -1927,7 +1928,9 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1927 1928
             statementCache: statementCache,
1928 1929
             lookupCache: lookupCache
1929 1930
         )
1930
-        let visibleCount = try visibleAggregate(sampleTypeID: sampleTypeID, db: db).visibleRecordCount
1931
+        let counts = try eventCounts(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
1932
+        let aggregate = try visibleAggregate(sampleTypeID: sampleTypeID, db: db)
1933
+        let visibleCount = aggregate.visibleRecordCount
1931 1934
         try insertObservationTypeRun(
1932 1935
             observationID: observationID,
1933 1936
             sampleTypeID: sampleTypeID,
@@ -1939,7 +1942,13 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
1939 1942
             db: db,
1940 1943
             statementCache: statementCache
1941 1944
         )
1942
-        try rebuildTypeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
1945
+        try upsertTypeSummary(
1946
+            observationID: observationID,
1947
+            sampleTypeID: sampleTypeID,
1948
+            counts: counts,
1949
+            aggregate: aggregate,
1950
+            db: db
1951
+        )
1943 1952
         try rebuildDailyAggregates(
1944 1953
             observationID: observationID,
1945 1954
             sampleTypeID: sampleTypeID,
@@ -2844,17 +2853,43 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2844 2853
 
2845 2854
     private func rebuildTypeSummary(observationID: Int64, sampleTypeID: Int64, db: OpaquePointer?) throws {
2846 2855
         let summary = try typeSummary(observationID: observationID, sampleTypeID: sampleTypeID, db: db)
2856
+        try upsertTypeSummary(
2857
+            observationID: observationID,
2858
+            sampleTypeID: sampleTypeID,
2859
+            counts: (
2860
+                appeared: summary.appearedCount,
2861
+                disappeared: summary.disappearedCount,
2862
+                representationChanged: summary.representationChangedCount
2863
+            ),
2864
+            aggregate: ArchiveV2VisibleAggregate(
2865
+                visibleRecordCount: summary.visibleRecordCount,
2866
+                earliestStartDate: summary.earliestStartDate,
2867
+                latestEndDate: summary.latestEndDate,
2868
+                valueSum: summary.valueSum,
2869
+                valueMax: summary.valueMax
2870
+            ),
2871
+            db: db
2872
+        )
2873
+    }
2874
+
2875
+    private func upsertTypeSummary(
2876
+        observationID: Int64,
2877
+        sampleTypeID: Int64,
2878
+        counts: (appeared: Int, disappeared: Int, representationChanged: Int),
2879
+        aggregate: ArchiveV2VisibleAggregate,
2880
+        db: OpaquePointer?
2881
+    ) throws {
2847 2882
         let aggregateParts: [String?] = [
2848 2883
             String(observationID),
2849 2884
             String(sampleTypeID),
2850
-            String(summary.visibleRecordCount),
2851
-            String(summary.appearedCount),
2852
-            String(summary.disappearedCount),
2853
-            String(summary.representationChangedCount),
2854
-            summary.earliestStartDate.map { String($0) },
2855
-            summary.latestEndDate.map { String($0) },
2856
-            summary.valueSum.map { String(format: "%.17g", $0) },
2857
-            summary.valueMax.map { String(format: "%.17g", $0) }
2885
+            String(aggregate.visibleRecordCount),
2886
+            String(counts.appeared),
2887
+            String(counts.disappeared),
2888
+            String(counts.representationChanged),
2889
+            aggregate.earliestStartDate.map { String($0) },
2890
+            aggregate.latestEndDate.map { String($0) },
2891
+            aggregate.valueSum.map { String(format: "%.17g", $0) },
2892
+            aggregate.valueMax.map { String(format: "%.17g", $0) }
2858 2893
         ]
2859 2894
         let aggregateHash = HashService.archiveContentHash(
2860 2895
             domain: "hp:v2:type_summary",
@@ -2872,14 +2907,14 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2872 2907
         ) { statement in
2873 2908
             bindInt64(observationID, to: 1, in: statement)
2874 2909
             bindInt64(sampleTypeID, to: 2, in: statement)
2875
-            bindInt(summary.visibleRecordCount, to: 3, in: statement)
2876
-            bindInt(summary.appearedCount, to: 4, in: statement)
2877
-            bindInt(summary.disappearedCount, to: 5, in: statement)
2878
-            bindInt(summary.representationChangedCount, to: 6, in: statement)
2879
-            bindDouble(summary.earliestStartDate, to: 7, in: statement)
2880
-            bindDouble(summary.latestEndDate, to: 8, in: statement)
2881
-            bindDouble(summary.valueSum, to: 9, in: statement)
2882
-            bindDouble(summary.valueMax, to: 10, in: statement)
2910
+            bindInt(aggregate.visibleRecordCount, to: 3, in: statement)
2911
+            bindInt(counts.appeared, to: 4, in: statement)
2912
+            bindInt(counts.disappeared, to: 5, in: statement)
2913
+            bindInt(counts.representationChanged, to: 6, in: statement)
2914
+            bindDouble(aggregate.earliestStartDate, to: 7, in: statement)
2915
+            bindDouble(aggregate.latestEndDate, to: 8, in: statement)
2916
+            bindDouble(aggregate.valueSum, to: 9, in: statement)
2917
+            bindDouble(aggregate.valueMax, to: 10, in: statement)
2883 2918
             bindText(aggregateHash, to: 11, in: statement)
2884 2919
             guard sqlite3_step(statement) == SQLITE_DONE else {
2885 2920
                 throw SQLiteHealthArchiveStoreError.stepFailed(lastErrorMessage(db))
@@ -2951,10 +2986,12 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
2951 2986
             SUM(v.numeric_value),
2952 2987
             MAX(v.numeric_value),
2953 2988
             v.source_revision_id
2954
-        FROM sample_visibility_ranges r
2955
-        JOIN samples s ON s.id = r.sample_id
2989
+        FROM samples s
2990
+        JOIN sample_visibility_ranges r
2991
+          ON r.sample_id = s.id
2992
+         AND r.last_observation_id IS NULL
2956 2993
         JOIN sample_versions v ON v.id = r.version_id
2957
-        WHERE s.sample_type_id = ? AND r.last_observation_id IS NULL
2994
+        WHERE s.sample_type_id = ?
2958 2995
         GROUP BY bucket_start, bucket_end, v.source_revision_id
2959 2996
         ORDER BY bucket_start ASC, v.source_revision_id ASC
2960 2997
         """
@@ -3034,10 +3071,12 @@ actor SQLiteHealthArchiveStore: HealthArchiveStore {
3034 3071
     private func visibleAggregate(sampleTypeID: Int64, db: OpaquePointer?) throws -> ArchiveV2VisibleAggregate {
3035 3072
         let sql = """
3036 3073
         SELECT COUNT(*), MIN(v.start_date), MAX(v.end_date), SUM(v.numeric_value), MAX(v.numeric_value)
3037
-        FROM sample_visibility_ranges r
3038
-        JOIN samples s ON s.id = r.sample_id
3074
+        FROM samples s
3075
+        JOIN sample_visibility_ranges r
3076
+          ON r.sample_id = s.id
3077
+         AND r.last_observation_id IS NULL
3039 3078
         JOIN sample_versions v ON v.id = r.version_id
3040
-        WHERE s.sample_type_id = ? AND r.last_observation_id IS NULL
3079
+        WHERE s.sample_type_id = ?
3041 3080
         """
3042 3081
         return try withStatement(sql, db: db) { statement in
3043 3082
             bindInt64(sampleTypeID, to: 1, in: statement)