Harden GoPro import safety and date fallback · 8b9540d

Harden GoPro import safety and date fallback
Browse files

bogdan committed 2 weeks ago

main

1 parent 334a207

commit 8b9540d

Showing 5 changed files with 1157 additions and 78 deletions

+106 -7

DEVELOPMENT.md

@@ -2,11 +2,10 @@
 
 ## 1. Objectives
 
-The Media Importer project aims to provide a robust, efficient solution for organizing media files by date with proper timezone handling and conflict - **Naming Convention**: `{s/f}_{YYYYMMDD_HHMMSS}_{TestName}.md` format where:
-  - `s` = success, `f` = failure
-  - Followed by timestamp and test name (spaces converted to underscores)esolution.
+The Media Importer project aims to provide a robust, efficient solution for organizing media files by date with proper timezone handling and conflict resolution.
 
 Key objectives:
+
 - Reliable EXIF/metadata extraction and date parsing
 - Proper UTC time conversion for QuickTime/Apple media files
 - Flexible organization patterns (year/month/day/hour)
@@ -23,14 +22,15 @@ When making changes to the project, follow this structured approach:
 
 All changelog entries should follow this format:
 
-```
+```text
 - Date time
 - Bug/Feature description  
 - Changes made
 ```
 
 Example:
-```
+
+```text
 - 2025-09-07 10:30
 - Fixed data loss issue when processing already-sorted folders
 - Added exclusion patterns for sorted/organized/processed folders
@@ -40,10 +40,19 @@ Example:
 ### File Move Confirmation
 
 Every file move operation should be confirmed:
+
 - After moving a file, the script must check that the file exists at the destination.
 - If the file is not present at the destination after the move operation, the script should immediately stop and report an error.
 - This ensures data integrity and prevents silent data loss.
 
+Implementation note (2026-05):
+
+- Move flow now uses `copy -> verify -> delete source`.
+- Verification is controlled by `--verify-mode` with modes `size` (default), `strict`, and `none`.
+- `size` validates destination existence, size match, and metadata/date consistency.
+- `strict` adds byte-to-byte validation (`cmp`) on top of `size` checks.
+- Source deletion is allowed only after successful verification.
+
 ### Destination Inside Source Handling
 
 - Given the script's purpose, the destination folder may be inside the source folder.
@@ -56,18 +65,21 @@ Every file move operation should be confirmed:
 Testing is essential to ensure the script's reliability and data safety. The following methodology should be used:
 
 #### Test Environment Setup
+
 - The `samples` directory contains a variety of media files for testing.
 - Create a dedicated working directory named `test` for each test run.
 - Copy selected files from `samples` into the `test` directory to simulate real-world scenarios.
 - Perform import operations using the script, targeting the `test` directory as the source and a subdirectory (e.g., `test/sorted`) as the destination.
 
 #### Test Execution and Documentation
+
 - Before and after each import operation, run `find` on both the source and destination directories to capture the file structure:
   - Example: `find ./test > test/source_before.txt`
   - Example: `find ./test/sorted > test/dest_before.txt`
 - Log all results, including script output and directory listings, into a dedicated log file for each test.
 
 #### Test Report Format
+
 Each test must generate a comprehensive Markdown report in `test/test_report.md` with the following structure:
 
 ```markdown
@@ -161,6 +173,7 @@ Each test must generate a comprehensive Markdown report in `test/test_report.md`
 ```
 
 #### Automated Test Runner
+
 A comprehensive test runner script (`test_runner.sh`) is available to automate the testing process:
 
 ```bash
@@ -168,6 +181,7 @@ A comprehensive test runner script (`test_runner.sh`) is available to automate t
 ```
 
 The script provides:
+
 - **Pre-configured test scenarios** for common use cases
 - **Automatic report generation** in Markdown format
 - **State capture** before and after test execution
@@ -175,6 +189,7 @@ The script provides:
 - **Custom test support** for specific scenarios
 
 #### Test Categories
+
 The test runner provides the following pre-configured test scenarios:
 
 1. **Basic Functionality Test**: Tests processing of files with valid EXIF data to verify correct sorting and organization
@@ -183,13 +198,20 @@ The test runner provides the following pre-configured test scenarios:
 4. **Safety Protections Test**: Tests destination exclusion and move confirmation mechanisms to prevent data loss
 5. **UTC Conversion Test**: Tests UTC timestamp conversion for QuickTime/Apple EXIF data
 6. **Subdirectory Processing Test**: Tests processing of files in nested subdirectories to ensure recursive file discovery
-7. **Custom Test**: Allows user-defined test scenarios with custom file sets and commands
+7. **Source Only Test**: Tests automatic `source/sorted` destination behavior
+8. **Destination Inside Source Test**: Verifies destination exclusion and prevents `sorted/sorted` recursion
+9. **Verify Mode Test**: Verifies default `size` verification and explicit `strict` mode
+10. **Timestamp Collision No-Overwrite Test**: Reproduces GoPro-style chapters with identical `CreateDate` values and verifies unique destination filenames with no overwrite collapse
+11. **GoPro Sidecar Metadata Sync Test**: Verifies GoPro MP4 imports use THM start times and automatically write the corrected timestamp into destination metadata
 
 #### Test Result Persistence
+
 The test runner includes automatic result persistence:
 
 - **Archival Location**: Test results are saved as individual Markdown files in `test_reports/` directory
-- **Naming Convention**: `{TestName}_{YYYYMMDD_HHMMSS}.md` format for easy identification
+- **Naming Convention**: `{s/f}_{YYYYMMDD_HHMMSS}_{TestName}.md` format where:
+  - `s` = success, `f` = failure
+  - Followed by timestamp and test name (spaces converted to underscores)
 - **Contents Preserved**: Single self-contained Markdown file with:
   - Complete test information and directory structures
   - Full script execution output embedded inline (ANSI codes stripped for readability)
@@ -198,13 +220,82 @@ The test runner includes automatic result persistence:
 - **Historical Tracking**: Maintains complete test history for debugging and regression testing
 
 #### Cleanup
+
 - Review the test report and verify all aspects are documented
 - Clean up the `test` directory after each test run to ensure a fresh environment for subsequent tests
 - Archive important test reports in a `test_reports/` directory for future reference
 
 ## 3. Changelog
 
+### 2026-05-16 00:10 - GoPro Chapter Overwrite Postmortem
+
+- Documented the 2026-05-15 GoPro import data-loss incident in `INCIDENTS.md`.
+- Root cause: multiple chaptered GoPro MP4 files shared the same timestamp-derived destination filename; the script warned but continued, allowing each copy to overwrite the previous destination before deleting the current source after verification.
+- Clarified that QuickTime UTC-to-local conversion likely explains the visible `19:20` vs `22:20` timestamp difference, but was not the data-loss mechanism.
+- Recorded surviving evidence: only the final full-resolution chapter remained in destination; `.THM` and `.LRV` sidecars remained on the card and preserved the chapter timeline.
+- Added operational guidance for future GoPro imports: start with `--date-source filesystem --sync-metadata --dry-run -v` and verify that GoPro MP4 dates come from matching `.THM` sidecars.
+- Added regression rule: future changes to date extraction, naming, conflict handling, copy/move, or GoPro/Garmin/Varia behavior must run `./test_runner.sh timestamp-collision`.
+
+---
+
+### 2026-05-17 08:35 - Automatic GoPro Metadata Timestamp Correction
+
+- Changed GoPro import behavior so `GX/GH/GP*.MP4` files with matching `.THM` sidecars prefer the THM filesystem timestamp even in default `auto` date mode.
+- GoPro imports that use THM/filesystem dates now automatically sync destination metadata to the corrected clip start time; `--sync-metadata` is no longer required for this GoPro path.
+- Changed move flow for metadata-corrected imports to copy and verify first, write destination metadata, verify the metadata timestamp, and only then delete the source file.
+- Added `GoPro_Sidecar_Metadata_Sync` regression coverage in `test_runner.sh`.
+- Verified the real `/Volumes/GOPRO` dry-run maps the current files to `19:03:20`, `19:15:08`, `19:26:56`, and `19:38:44`, all from matching THM sidecars.
+
+---
+
+### 2026-05-17 09:20 - Explicit Destination Conflict Policy
+
+- Added `--unattended`; unattended/non-interactive runs never prompt and resolve existing destinations with numeric suffixes (`_1`, `_2`, ...).
+- Interactive runs now ask on destination conflicts and offer suffix once/all, skip once/all, or abort, so long imports do not require repeated decisions.
+- Dry-run reserves planned destinations, so collisions within the same run are visible before writing.
+- Updated `Timestamp_Collision_No_Overwrite` to assert numeric suffixes and reject the legacy `__GX...` suffix form.
+
+---
+
+### 2026-05-17 12:05 - Long Import UX Feedback
+
+- Added per-file progress lines (`Processing [n/total]`) before long copy/move work starts.
+- Replaced integer `files/sec` reporting with decimal throughput that shows `files/min` for large slow imports and keeps `MB/sec` precision.
+- Added average time per processed file to the final report.
+
+---
+
+### 2026-05-17 12:20 - GoPro Filesystem Date Fallback
+
+- GoPro auto date extraction now stays on filesystem timestamps even when THM is missing.
+- Fallback order is matching `THM`, matching `LRV`, then the MP4 file's own filesystem mtime.
+- Automatic metadata sync now applies to all GoPro filesystem date sources, including the MP4 fallback.
+- Extended `GoPro_Sidecar_Metadata_Sync` to cover THM, LRV-only, and no-sidecar GoPro imports.
+- Added `GoPro_No_Sidecar_Reimport` to verify no-sidecar MP4 fallback plus safe numeric suffixing when the same GoPro file is imported again.
+- Excluded macOS AppleDouble `._*` files from source discovery so NAS metadata forks are not treated as importable media.
+
+---
+
+### 2026-05-13 21:10 - Critical Data Loss Incident + Hardening
+
+- Observed incident: running with explicit source on a problematic/near-full card led to writes on source media and source content removal without strong post-write confirmation.
+- Impact: major data loss risk on unstable storage media (including read-only/IO edge cases).
+- Clarified behavior kept as valid: destination may be inside source (`source/sorted`) and must be excluded from scanning.
+- Reworked move safety in `media-importer.sh`: direct `mv` path replaced by verified flow `copy -> verify -> delete source`.
+- Added verification modes via `--verify-mode`:
+  - `size` (default): destination exists + size match + metadata/date validation
+  - `strict`: adds byte-to-byte content validation (`cmp`)
+  - `none`: disables verification (explicit opt-out)
+- Applied the same verified move logic for unsortable collection path.
+- Added regression coverage in `test_runner.sh`:
+  - Destination-inside-source test (`-s source -d source/sorted`) to verify destination exclusion and no `sorted/sorted` recursion.
+  - Verify-mode test to confirm default `size` and explicit `strict` behavior.
+- Updated ignore/staging hygiene to avoid committing generated test artifacts and `.DS_Store`.
+
+---
+
 ### 2025-09-07 21:15 - Test 2 and 3 Enhancements
+
 - Updated Test 2 (Unimportable Files Test) to include files in both root and subfolder
 - Removed --collect-unimportable flag from Test 2 to test default behavior
 - Updated Test 3 (Mixed Content Test) to use separate folders for sortable vs unimportable files
@@ -215,6 +306,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 21:25 - Documentation Enhancement
+
 - Added comprehensive documentation for --collect-unimportable flag in README.md
 - Added Example 4 showing how to use --collect-unimportable flag
 - Updated Features section to mention unimportable files handling
@@ -224,6 +316,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 21:30 - Git Ignore Enhancement
+
 - Added test_reports/ to .gitignore to exclude generated test reports from version control
 - Test reports are generated files that don't need to be tracked in Git
 - Prevents large numbers of timestamped report files from cluttering the repository
@@ -232,6 +325,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 20:40 - Source Only Test Addition
+
 - Added Test 8: Source Only Test to test runner
 - Tests processing with only source parameter (creates sorted subdirectory automatically)
 - Verifies that when no destination is specified, files are sorted into source/sorted/
@@ -240,6 +334,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 20:45 - Test 7 Refinement
+
 - Updated Test 7 to test --keep-empty-dirs functionality instead of cleanup
 - Since cleanup is now default behavior, Test 7 now verifies empty directory preservation
 - Renamed from "Cleanup Empty Directories Test" to "Keep Empty Directories Test"
@@ -249,6 +344,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 19:30 - Test Runner Directory Separation
+
 - Adapted test runner to use separate source and destination directories
 - Changed from test/ as source to test/source/ and test/destination/
 - Updated all test functions to use proper directory separation
@@ -257,6 +353,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 19:00 - Default Cleanup Behavior
+
 - Made --cleanup-empty-dirs the default behavior (implicit option)
 - Added --keep-empty-dirs flag to disable cleanup if needed
 - Updated help text and configuration display to reflect new default
@@ -265,6 +362,7 @@ The test runner includes automatic result persistence:
 ---
 
 ### 2025-09-07 18:56 - Cleanup Empty Directories Feature
+
 - Added --cleanup-empty-dirs option to remove empty directories from source after processing
 - Added cleanup_empty_directories() function with safe empty directory detection
 - Updated final report to show cleanup status
@@ -274,6 +372,7 @@ The test runner includes automatic result persistence:
 ## 4. Todo
 
 Key areas for future development:
+
 - GPS metadata integration for timezone detection
 - Enhanced duplicate detection
 - Performance optimizations for large file sets


+206 -0

INCIDENTS.md

View

+                    +# Incident Reports
+                    +## 2026-05-15 - GoPro Chapter Import Overwrite
+                    +### Summary
+                    +An import from `/Volumes/NO NAME` into `~/Autofs/xdev/autonas/ext01/@Camera/GoPro`
+                    +processed 9 GoPro MP4 files and reported success for all of them, but the import
+                    +collapsed multiple chapter files onto one destination filename. Because the script
+                    +continued after detecting the collision, each later copy overwrote the previous
+                    +destination file and the source file was then deleted after verification.
+                    +The surviving destination file appears to be the last chapter (`GX091621.MP4`),
+                    +stored as:
+                    +```text
+                    +/Users/bogdan/Autofs/xdev/autonas/ext01/@Camera/GoPro/2026-05-15/2026-05-15_22-20-09.mp4
+                    +```
+                    +### User Command
+                    +```bash
+                    +media-importer.sh -s "/Volumes/NO NAME" -d ~/Autofs/xdev/autonas/ext01/@Camera/GoPro
+                    +```
+                    +Effective settings from the log:
+                    +```text
+                    +Organization pattern: ymd
+                    +Destination:         /Users/bogdan/Autofs/xdev/autonas/ext01/@Camera/GoPro
+                    +Keep originals:      No
+                    +Verify mode:         size
+                    +Dry run:             No
+                    +Keep empty dirs:     No
+                    +Source patterns:
+                    +  - /Volumes/NO NAME
+                    +```
+                    +### Impact
+                    +- Total files found: 9
+                    +- Total size found: 30GB
+                    +- Script reported: 9 successfully processed, 0 errors
+                    +- Actual observed result: only one full-resolution MP4 remained in destination
+                    +- Source MP4 files were removed from the card
+                    +- Remaining card artifacts included `.LRV` proxy files and `.THM` thumbnails
+                    +The original full-resolution data for the first 8 chapters was not recoverable
+                    +from the mounted filesystem at the time of inspection. No recovery tool was run.
+                    +### Evidence
+                    +The import log repeatedly mapped each source chapter to the same destination:
+                    +```text
+                    +GX011621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX021621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX031621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX041621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX051621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX061621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX071621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX081621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +GX091621.MP4 -> 2026-05-15_22-20-09.mp4
+                    +```
+                    +The script emitted warnings like:
+                    +```text
+                    +Destination already exists: .../2026-05-15_22-20-09.mp4 - proceeding to move/copy and letting external tools handle conflicts
+                    +```
+                    +That warning was the bug signal. The script should have chosen a unique destination
+                    +or failed closed. Instead it proceeded.
+                    +Post-incident inspection found:
+                    +```text
+                    +/Users/bogdan/Autofs/xdev/autonas/ext01/@Camera/GoPro/2026-05-15/2026-05-15_22-20-09.mp4
+                    +size: 586M
+                    +duration: 0:01:49
+                    +QuickTime CreateDate: 2026:05:15 19:20:09
+                    +```
+                    +The duration matched the surviving low-resolution proxy:
+                    +```text
+                    +GL091621.LRV duration: 108.58s
+                    +```
+                    +This strongly indicates that the surviving full-resolution file was the last
+                    +chapter (`GX091621.MP4`).
+                    +### Timeline Recovered From Card Artifacts
+                    +The full-resolution MP4 files were gone from `/Volumes/NO NAME/DCIM/100GOPRO`,
+                    +but GoPro sidecar/proxy files remained. The `.THM` timestamps preserved a useful
+                    +chapter timeline:
+                    +```text
+                    +GX011621.THM  2026-05-15 19:20:10
+                    +GX021621.THM  2026-05-15 19:31:58
+                    +GX031621.THM  2026-05-15 19:43:44
+                    +GX041621.THM  2026-05-15 19:55:32
+                    +GX051621.THM  2026-05-15 20:07:20
+                    +GX061621.THM  2026-05-15 20:19:08
+                    +GX071621.THM  2026-05-15 20:30:54
+                    +GX081621.THM  2026-05-15 20:42:42
+                    +GX091621.THM  2026-05-15 20:54:30
+                    +```
+                    +The `.LRV` proxy files also remained and can preserve low-resolution visual
+                    +content when full-resolution MP4 recovery is not possible.
+                    +### Root Cause
+                    +There were two separate issues:
+                    +1. Filename collision handling was unsafe.
+                    +   - Multiple GoPro chapters had the same metadata timestamp.
+                    +   - `generate_destination_path` generated the same destination filename for all
+                    +     files.
+                    +   - `process_file` detected the existing destination but continued.
+                    +   - The copy operation overwrote the destination.
+                    +   - Verification then validated the newly overwritten destination.
+                    +   - The source file was deleted.
+                    +2. Timestamp interpretation was easy to misunderstand.
+                    +   - The script converts QuickTime `CreateDate` values from UTC to local time.
+                    +   - The accident log used `22:20:09`, while surviving `.THM/.LRV` filesystem
+                    +     timestamps showed `19:20`.
+                    +   - This looks like QuickTime UTC conversion rather than a GoPro firmware
+                    +     regression.
+                    +   - This was not the data-loss mechanism, but it affected the visible names.
+                    +### Corrective Actions
+                    +Implemented in `media-importer.sh`:
+                    +- Added explicit no-overwrite checks in `safe_cp` and `safe_mv`.
+                    +- Changed copy/move flow to copy into a temporary file, verify the temporary
+                    +  file, then move it into place only if the final destination does not exist.
+                    +- Added explicit destination conflict handling. Unattended runs append numeric
+                    +  suffixes; interactive runs ask the user and can apply the decision to all
+                    +  similar conflicts during long imports.
+                    +```text
+                    +2026-05-15_19-20-09.mp4
+                    +2026-05-15_19-20-09_1.mp4
+                    +2026-05-15_19-20-09_2.mp4
+                    +```
+                    +- Sorted discovered files before processing to make chapter order stable.
+                    +- Added `--date-source filesystem` for imports where filesystem timestamps are
+                    +  the source of truth.
+                    +- GoPro `GX/GH/GP*.MP4` files now prefer filesystem timestamps in default
+                    +  `auto` mode, using matching `THM`, matching `LRV`, then MP4 mtime as fallback.
+                    +- GoPro imports that use filesystem dates automatically write the corrected clip
+                    +  start time into destination metadata before deleting the source.
+                    +- Added `--sync-metadata` to force the same metadata write for non-GoPro cases.
+                    +- Prevented accidental execution when the script is sourced rather than run.
+                    +Implemented in `test_runner.sh`:
+                    +- Added `timestamp-collision` regression test.
+                    +- The test creates multiple MP4 files with identical `CreateDate` values.
+                    +- It runs move mode and verifies that all files arrive at unique destination
+                    +  paths, with no overwrite collapse.
+                    +Verification command:
+                    +```bash
+                    +./test_runner.sh timestamp-collision
+                    +```
+                    +### Operational Guidance
+                    +For the next GoPro import, start with dry-run:
+                    +```bash
+                    +./media-importer.sh -s "/Volumes/GOPRO" -d "$HOME/Autofs/xdev/autonas/ext01/@Camera/GoPro" --dry-run -v
+                    +```
+                    +Review the dry-run output before removing `--dry-run`. Expected healthy output:
+                    +- each `GX*.MP4` maps to a distinct destination path;
+                    +- date source says `Filesystem:GX*.THM`, `Filesystem:GX*.LRV`, or the MP4
+                    +  basename depending on the best available GoPro filesystem source;
+                    +- each GoPro line says metadata date would be synced;
+                    +- no line says that an existing destination will be overwritten;
+                    +- any unavoidable conflict is either a deliberate numeric suffix or an explicit
+                    +  user choice.
+                    +After dry-run review, run the same command without `--dry-run`.
+                    +### Regression Rule
+                    +Any future change to date extraction, filename generation, verification, copy,
+                    +move, conflict handling, or GoPro/Garmin/Varia media behavior must run:
+                    +```bash
+                    +./test_runner.sh timestamp-collision
+                    +```
+                    +The test must fail if two source media files can collapse into one destination
+                    +file while the script still reports success.

+2 -0

README.md

View

@@ -5,6 +5,7 @@ This repository contains the development and testing resources for the `media-im
 ## Project Structure
 
 - `media-importer.sh`: The primary script under development and testing.
+- `INCIDENTS.md`: Postmortems for data-loss or near-data-loss incidents and the regression rules derived from them.
 - `RPI/`: Raspberry Pi integration prototype. This area is currently frozen because Raspberry Pi 3B+ hardware with 1 GB RAM has not provided reliable validation capacity and may also be affected by hardware defects.
 - `samples/code/autonas-media-importer.sh`: The original script that serves as inspiration for this project.
 - `samples/`: Contains code samples, example files, and other resources used for development and testing. **Note:** Scripts and functions are not imported from this directory; it is strictly for reference and testing purposes.
@@ -24,6 +25,7 @@ samples/
 - All development should focus on `media-importer.sh`.
 - Do not import or source scripts/functions from the `samples/` directory.
 - Use resources in `samples/` only for testing and development reference.
+- Destination conflicts must never be delegated to copy/move tools. In unattended runs the importer appends numeric suffixes (`_1`, `_2`, ...); in interactive runs it asks the user and supports applying the choice to all similar conflicts.
 
 ## License
 


+539 -66

media-importer.sh

View

@@ -17,9 +17,15 @@ SOURCE_PATTERNS=()
 DESTINATION=""
 KEEP_ORIGINALS=0
 VERIFY_MODE="size"  # options: size, strict, none
+DATE_SOURCE="auto"  # options: auto, exif, filesystem
+SYNC_METADATA=0     # when 1, write reconstructed date into destination metadata
+UNATTENDED=0        # when 1, never prompt; destination conflicts get numeric suffixes
 DRY_RUN=0
 VERBOSE=0
 CLEANUP_EMPTY_DIRS=1
+CONFLICT_APPLY_ALL=""  # suffix|skip after an interactive "all similar" choice
+RESOLVED_DESTINATION_PATH=""
+RESERVED_DESTINATION_PATHS=()
 
 # Counters and statistics
 TOTAL_FILES=0
@@ -29,6 +35,7 @@ ERROR_FILES=0
 TOTAL_SIZE=0
 PROCESSED_SIZE=0
 START_TIME=$(date +%s)
+CURRENT_FILE_INDEX=0
 
 # Colors for output
 RED='\033[0;31m'
@@ -98,6 +105,9 @@ Options:
     -d, --destination PATH       Destination folder. Required when multiple -s are given.
     -k, --keep-originals         Copy files instead of moving
     --verify-mode MODE           size|strict|none (default: size)
+    --date-source SOURCE         auto|exif|filesystem (default: auto)
+    --sync-metadata              Write chosen date into destination metadata (automatic for GoPro filesystem dates)
+    --unattended                 Never prompt; resolve destination conflicts with numeric suffixes
     --collect-unsortable         Put files without dates into DEST/unsortable
     --keep-empty-dirs            Keep empty directories after processing
     --dry-run                    Show actions without changing files
@@ -165,7 +175,6 @@ check_dependencies() {
 
 # Determine filesystem/device ID for a path (portable between Linux and macOS)
 get_dev() {
-    # Return a device identifier for the supplied path (portable between GNU stat and BSD stat)
     local path="$1"
     if [[ -z "$path" ]]; then
         path="."
@@ -180,30 +189,6 @@ get_dev() {
     fi
 }
 
-# Determine the mount root for a path by walking up until device changes
-get_mountpoint() {
-    local path="$1"
-    if [[ -z "$path" ]]; then path="."; fi
-    # Resolve to absolute
-    local abs
-    abs=$(cd "$path" 2>/dev/null && pwd) || abs="$path"
-    # Walk up until device differs or we reach /
-    local parent="$abs"
-    local root_dev
-    root_dev=$(get_dev "$parent")
-    while [[ "$parent" != "/" ]]; do
-        local next_parent
-        next_parent=$(dirname "$parent")
-        local next_dev
-        next_dev=$(get_dev "$next_parent")
-        if [[ "$next_dev" != "$root_dev" ]]; then
-            break
-        fi
-        parent="$next_parent"
-    done
-    echo "$parent"
-}
-
 # Function to get file size in bytes (portable between Linux and macOS)
 get_file_size() {
     local file="$1"
@@ -234,12 +219,291 @@ safe_mv() {
     return $?
 }
 
-# Use simple safe_mv/safe_cp for moving/copying files. Removed atomic installer to let exiftool or filesystem handle renames.
+destination_path_reserved() {
+    local candidate="$1"
+    local reserved_path
+    for reserved_path in "${RESERVED_DESTINATION_PATHS[@]}"; do
+        if [[ "$reserved_path" == "$candidate" ]]; then
+            return 0
+        fi
+    done
+    return 1
+}
 
-safe_cp() {
-    local src="$1" dst="$2"
-    cp "$src" "$dst" 2> >(grep -v "set flags (was:" >&2)
-    return $?
+destination_path_unavailable() {
+    local candidate="$1"
+    [[ -e "$candidate" ]] || destination_path_reserved "$candidate"
+}
+
+reserve_destination_path() {
+    local candidate="$1"
+    if [[ -n "$candidate" ]] && ! destination_path_reserved "$candidate"; then
+        RESERVED_DESTINATION_PATHS+=("$candidate")
+    fi
+}
+
+prompt_destination_conflict_choice() {
+    local source_file="$1"
+    local desired_path="$2"
+    local choice
+
+    if [[ ! -r /dev/tty || ! -w /dev/tty ]]; then
+        return 1
+    fi
+
+    {
+        print_color "$YELLOW" "Destination already exists:"
+        echo "  Source:      $source_file"
+        echo "  Destination: $desired_path"
+        echo ""
+        echo "Choose conflict action:"
+        echo "  [s] suffix once"
+        echo "  [S] suffix for all similar conflicts"
+        echo "  [k] skip once"
+        echo "  [K] skip all similar conflicts"
+        echo "  [a] abort import"
+    } > /dev/tty
+
+    while true; do
+        printf "Action [s/S/k/K/a]: " > /dev/tty
+        IFS= read -r choice < /dev/tty || return 1
+        case "$choice" in
+            s|"")
+                echo "suffix"
+                return 0
+                ;;
+            S)
+                echo "suffix_all"
+                return 0
+                ;;
+            k)
+                echo "skip"
+                return 0
+                ;;
+            K)
+                echo "skip_all"
+                return 0
+                ;;
+            a|A)
+                echo "abort"
+                return 0
+                ;;
+            *)
+                print_color "$YELLOW" "Please choose s, S, k, K, or a." > /dev/tty
+                ;;
+        esac
+    done
+}
+
+resolve_destination_conflict() {
+    local desired_path="$1"
+    local source_file="$2"
+    local resolved_path choice
+    RESOLVED_DESTINATION_PATH=""
+
+    if [[ -z "$desired_path" ]]; then
+        return 1
+    fi
+
+    if ! destination_path_unavailable "$desired_path"; then
+        RESOLVED_DESTINATION_PATH="$desired_path"
+        reserve_destination_path "$RESOLVED_DESTINATION_PATH"
+        return 0
+    fi
+
+    if [[ "$CONFLICT_APPLY_ALL" == "skip" ]]; then
+        return 3
+    fi
+
+    if [[ "$CONFLICT_APPLY_ALL" == "suffix" || $UNATTENDED -eq 1 ]]; then
+        resolved_path=$(ensure_unique_destination_path "$desired_path")
+        if [[ -z "$resolved_path" ]]; then
+            return 1
+        fi
+        RESOLVED_DESTINATION_PATH="$resolved_path"
+        reserve_destination_path "$RESOLVED_DESTINATION_PATH"
+        return 0
+    fi
+
+    choice=$(prompt_destination_conflict_choice "$source_file" "$desired_path")
+    if [[ $? -ne 0 || -z "$choice" ]]; then
+        log_message "Cannot prompt for destination conflict; using unattended numeric suffix mode" "WARNING"
+        resolved_path=$(ensure_unique_destination_path "$desired_path")
+        if [[ -z "$resolved_path" ]]; then
+            return 1
+        fi
+        RESOLVED_DESTINATION_PATH="$resolved_path"
+        reserve_destination_path "$RESOLVED_DESTINATION_PATH"
+        return 0
+    fi
+
+    case "$choice" in
+        suffix)
+            resolved_path=$(ensure_unique_destination_path "$desired_path")
+            ;;
+        suffix_all)
+            CONFLICT_APPLY_ALL="suffix"
+            resolved_path=$(ensure_unique_destination_path "$desired_path")
+            ;;
+        skip)
+            return 3
+            ;;
+        skip_all)
+            CONFLICT_APPLY_ALL="skip"
+            return 3
+            ;;
+        abort)
+            return 4
+            ;;
+        *)
+            return 1
+            ;;
+    esac
+
+    if [[ -z "$resolved_path" ]]; then
+        return 1
+    fi
+
+    RESOLVED_DESTINATION_PATH="$resolved_path"
+    reserve_destination_path "$RESOLVED_DESTINATION_PATH"
+    return 0
+}
+
+extract_filesystem_date() {
+    # Returns yyyy-mm-dd hh:mm:ss based on filesystem mtime.
+    # We intentionally use mtime (not birthtime) because birthtime isn't preserved by copies
+    # across filesystems, while mtime can be preserved via `cp -p`.
+    local file="$1"
+    if [[ ! -e "$file" ]]; then
+        return 2
+    fi
+
+    local epoch=""
+
+    if [[ "$OSTYPE" == "darwin"* ]]; then
+        epoch=$(stat -f %m "$file" 2>/dev/null || echo "")
+        [[ -n "$epoch" ]] || return 2
+        date -j -r "$epoch" "+%Y-%m-%d %H:%M:%S" 2>/dev/null || return 2
+        return 0
+    else
+        epoch=$(stat -c %Y "$file" 2>/dev/null || echo "")
+        [[ -n "$epoch" ]] || return 2
+        date -d "@$epoch" "+%Y-%m-%d %H:%M:%S" 2>/dev/null || return 2
+        return 0
+    fi
+}
+
+filesystem_date_reference() {
+    local file="$1"
+    local dir base stem ext sidecar_ext sidecar
+    dir=$(dirname "$file")
+    base=$(basename "$file")
+    stem="${base%.*}"
+    ext="${base##*.}"
+
+    if [[ "$ext" =~ ^([Mm][Pp]4)$ ]]; then
+        for sidecar_ext in THM thm LRV lrv; do
+            sidecar="$dir/$stem.$sidecar_ext"
+            if [[ -f "$sidecar" ]]; then
+                echo "$sidecar"
+                return 0
+            fi
+        done
+    fi
+
+    echo "$file"
+}
+
+is_gopro_media_file() {
+    local filename
+    filename=$(basename "$1")
+    [[ "$filename" =~ ^G[HPX][0-9]{6}\.[Mm][Pp]4$ ]]
+}
+
+should_prefer_gopro_filesystem_date() {
+    local file="$1"
+
+    is_gopro_media_file "$file"
+}
+
+filesystem_date_source_label() {
+    local file="$1"
+    local reference="$2"
+
+    if is_gopro_media_file "$file"; then
+        echo "Filesystem:$(basename "$reference")"
+    elif [[ "$reference" != "$file" ]]; then
+        echo "Filesystem:$(basename "$reference")"
+    else
+        echo "Filesystem"
+    fi
+}
+
+date_to_exiftool_format() {
+    # yyyy-mm-dd hh:mm:ss -> yyyy:mm:dd hh:mm:ss
+    local s="$1"
+    if [[ "$s" =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})[[:space:]]+([0-9]{2}):([0-9]{2}):([0-9]{2})$ ]]; then
+        echo "${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]} ${BASH_REMATCH[4]}:${BASH_REMATCH[5]}:${BASH_REMATCH[6]}"
+        return 0
+    fi
+    return 1
+}
+
+sync_destination_metadata_to_date() {
+    local file="$1"
+    local date_str="$2" # yyyy-mm-dd hh:mm:ss
+
+    local exif_dt
+    exif_dt=$(date_to_exiftool_format "$date_str") || return 1
+
+    exiftool -overwrite_original \
+        "-CreateDate=$exif_dt" \
+        "-DateTimeOriginal=$exif_dt" \
+        "-DateTime=$exif_dt" \
+        "-ModifyDate=$exif_dt" \
+        "-MediaCreateDate=$exif_dt" \
+        "-TrackCreateDate=$exif_dt" \
+        "-QuickTime:CreateDate=$exif_dt" \
+        "-QuickTime:ModifyDate=$exif_dt" \
+        "$file" >/dev/null 2>&1
+    return 0
+}
+
+verify_synced_metadata_date() {
+    local file="$1"
+    local expected_date="$2"
+
+    local metadata_date
+    metadata_date=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$file" 2>/dev/null | head -1)
+    if [[ -z "$metadata_date" ]]; then
+        metadata_date=$(exiftool -api QuickTimeUTC=0 -s -s -s -CreateDate "$file" 2>/dev/null | head -1)
+    fi
+
+    if [[ "$metadata_date" =~ ^([0-9]{4}):([0-9]{2}):([0-9]{2})[[:space:]]+([0-9]{2}):([0-9]{2}):([0-9]{2}) ]]; then
+        metadata_date="${BASH_REMATCH[1]}-${BASH_REMATCH[2]}-${BASH_REMATCH[3]} ${BASH_REMATCH[4]}:${BASH_REMATCH[5]}:${BASH_REMATCH[6]}"
+    fi
+
+    if [[ "$metadata_date" != "$expected_date" ]]; then
+        log_message "Destination metadata sync mismatch: expected $expected_date, got ${metadata_date:-none} for $file" "ERROR"
+        return 1
+    fi
+
+    return 0
+}
+
+should_sync_imported_metadata() {
+    local original_filename="$1"
+    local date_source="$2"
+
+    if [[ $SYNC_METADATA -eq 1 ]]; then
+        return 0
+    fi
+
+    if [[ "$date_source" == Filesystem* ]] && is_gopro_media_file "$original_filename"; then
+        return 0
+    fi
+
+    return 1
 }
 
 verify_copied_file() {
@@ -298,12 +562,39 @@ copy_with_verification() {
     local dst="$2"
     local expected_date="$3"
 
-    if ! safe_cp "$src" "$dst"; then
+    if [[ -e "$dst" ]]; then
+        log_message "Refusing to overwrite existing destination: $dst" "ERROR"
         return 1
     fi
 
-    if ! verify_copied_file "$src" "$dst" "$expected_date"; then
-        rm -f "$dst"
+    local dst_dir tmp
+    dst_dir=$(dirname "$dst")
+    tmp=$(mktemp "$dst_dir/.media-importer.$(basename "$dst").tmp.XXXXXX") || return 1
+    rm -f "$tmp"
+
+    if ! safe_cp "$src" "$tmp"; then
+        rm -f "$tmp"
+        return 1
+    fi
+
+    if ! verify_copied_file "$src" "$tmp" "$expected_date"; then
+        rm -f "$tmp"
+        return 1
+    fi
+
+    if [[ -e "$dst" ]]; then
+        log_message "Destination appeared during copy, refusing to overwrite: $dst" "ERROR"
+        rm -f "$tmp"
+        return 1
+    fi
+
+    if ! safe_mv "$tmp" "$dst"; then
+        rm -f "$tmp"
+        return 1
+    fi
+
+    if [[ ! -f "$dst" ]]; then
+        log_message "Copied file missing after final move: $dst" "ERROR"
         return 1
     fi
 
@@ -341,12 +632,62 @@ format_size() {
     fi
 }
 
+format_duration() {
+    local total_seconds=$1
+    local hours=$((total_seconds / 3600))
+    local minutes=$(((total_seconds % 3600) / 60))
+    local seconds=$((total_seconds % 60))
+
+    if (( hours > 0 )); then
+        printf "%dh %02dm %02ds" "$hours" "$minutes" "$seconds"
+    elif (( minutes > 0 )); then
+        printf "%dm %02ds" "$minutes" "$seconds"
+    else
+        printf "%ds" "$seconds"
+    fi
+}
+
+format_processing_rate() {
+    local files_count="$1"
+    local bytes_count="$2"
+    local elapsed_seconds="$3"
+
+    awk -v files="$files_count" -v bytes="$bytes_count" -v seconds="$elapsed_seconds" '
+        BEGIN {
+            if (seconds <= 0 || files <= 0) {
+                exit
+            }
+
+            files_per_second = files / seconds
+            files_per_minute = files * 60 / seconds
+            mb_per_second = bytes / seconds / 1048576
+
+            if (files_per_second >= 1) {
+                printf "%.2f files/sec, %.2f MB/sec", files_per_second, mb_per_second
+            } else {
+                printf "%.2f files/min, %.2f MB/sec", files_per_minute, mb_per_second
+            }
+        }
+    '
+}
+
 # Function to extract date from file
 extract_file_date() {
     local file="$1"
     local create_date=""
     local date_source=""
     local exif_found=0
+
+    # Filesystem authoritative mode, and GoPro media in auto mode.
+    # GoPro fallback order is THM, LRV, then the MP4 filesystem timestamp.
+    if [[ "$DATE_SOURCE" == "filesystem" ]] || { [[ "$DATE_SOURCE" == "auto" ]] && should_prefer_gopro_filesystem_date "$file"; }; then
+        local filesystem_reference
+        filesystem_reference=$(filesystem_date_reference "$file")
+        create_date=$(extract_filesystem_date "$filesystem_reference") || return 2
+        echo "$create_date|$(filesystem_date_source_label "$file" "$filesystem_reference")"
+        return 0
+    fi
+
     # Try to get creation date from EXIF data
     local exif_output=$(exiftool -G1 -s -CreateDate -DateTimeOriginal -DateTime "$file" 2>/dev/null)
     if [[ -n "$exif_output" ]]; then
@@ -383,6 +724,15 @@ extract_file_date() {
             date_source="MediaInfo:Recorded_Date"
         fi
     fi
+
+    # In auto mode, if metadata is missing/unreliable, fall back to filesystem timestamps
+    if [[ -z "$create_date" && "$DATE_SOURCE" == "auto" ]]; then
+        local filesystem_reference
+        filesystem_reference=$(filesystem_date_reference "$file")
+        create_date=$(extract_filesystem_date "$filesystem_reference") || return 2
+        date_source=$(filesystem_date_source_label "$file" "$filesystem_reference")
+    fi
+
     # If no EXIF or mediainfo date found, return failure
     if [[ -z "$create_date" ]]; then
         return 2  # No date metadata found
@@ -570,11 +920,14 @@ find_source_files() {
         fi
         # Add expression
         # shellcheck disable=SC2068
-        "${find_cmd[@]}" \( $ext_expr \) 2>/dev/null || true
+        "${find_cmd[@]}" ! -name '._*' \( $ext_expr \) 2>/dev/null || true
     else
         # Scan each provided source
         for src in "${SOURCE_PATTERNS[@]}"; do
             if [[ -f "$src" ]]; then
+                if [[ "$(basename "$src")" == ._* ]]; then
+                    continue
+                fi
                 # single file - skip if it's inside dest
                 local abs_file
                 abs_file=$(cd "$(dirname "$src")" 2>/dev/null && pwd)/$(basename "$src")
@@ -587,9 +940,9 @@ find_source_files() {
                 abs_src=$(cd "$src" 2>/dev/null && pwd)
                 if [[ -n "$abs_src" ]]; then
                     if [[ -n "$abs_dest" && "$abs_dest" == "$abs_src"* ]]; then
-                        find -L "$abs_src" -type f \( $ext_expr \) ! -path "$abs_dest/*" ! -path "$abs_dest" 2>/dev/null || true
+                        find -L "$abs_src" -type f ! -name '._*' \( $ext_expr \) ! -path "$abs_dest/*" ! -path "$abs_dest" 2>/dev/null || true
                     else
-                        find -L "$abs_src" -type f \( $ext_expr \) 2>/dev/null || true
+                        find -L "$abs_src" -type f ! -name '._*' \( $ext_expr \) 2>/dev/null || true
                     fi
                 else
                     print_color "$YELLOW" "Warning: Could not resolve source directory: $src"
@@ -605,8 +958,15 @@ find_source_files() {
 process_file() {
     local file="$1"
     local file_size=$(get_file_size "$file")
+    local file_label
+    file_label="$(basename "$file")"
     TOTAL_SIZE=$((TOTAL_SIZE + file_size))
     
+    if [[ $TOTAL_FILES -gt 0 && $CURRENT_FILE_INDEX -gt 0 ]]; then
+        print_color "$BLUE" "Processing [$CURRENT_FILE_INDEX/$TOTAL_FILES]: $file_label ($(format_size "$file_size"))"
+    else
+        print_color "$BLUE" "Processing: $file_label ($(format_size "$file_size"))"
+    fi
     log_message "Processing: $file" "INFO"
     
     # Extract date information
@@ -617,6 +977,29 @@ process_file() {
                 local unsortable_dir="$DESTINATION/unsortable"
                 mkdir -p "$unsortable_dir"
                 local unsortable_path="$unsortable_dir/$(basename "$file")"
+                local desired_unsortable_path="$unsortable_path"
+                local unsortable_conflict_status
+                resolve_destination_conflict "$unsortable_path" "$file"
+                unsortable_conflict_status=$?
+                if [[ $unsortable_conflict_status -eq 0 ]]; then
+                    unsortable_path="$RESOLVED_DESTINATION_PATH"
+                    if [[ "$unsortable_path" != "$desired_unsortable_path" ]]; then
+                        log_message "Destination already exists or is already planned: $desired_unsortable_path - using: $unsortable_path" "WARNING"
+                    fi
+                elif [[ $unsortable_conflict_status -eq 3 ]]; then
+                    log_message "Destination conflict skipped: $desired_unsortable_path" "WARNING"
+                    SKIPPED_FILES=$((SKIPPED_FILES + 1))
+                    return 1
+                elif [[ $unsortable_conflict_status -eq 4 ]]; then
+                    log_message "Import aborted by user at destination conflict: $desired_unsortable_path" "ERROR"
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    FATAL_ERROR=1
+                    return 2
+                else
+                    log_message "Could not resolve a unique destination path for $file (wanted: $desired_unsortable_path)" "ERROR"
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                fi
                 if [[ $DRY_RUN -eq 1 ]]; then
                     print_color "$BLUE" "Would move unsortable: $file -> $unsortable_path"
                 else
@@ -641,21 +1024,43 @@ process_file() {
         local date_source="${date_info#*|}"
         log_message "Date: $date_str (from $date_source)" "INFO"
         # Generate destination path
-        local dest_path=$(generate_destination_path "$date_str" "$(basename "$file")" "$DESTINATION")
+        local original_basename
+        original_basename="$(basename "$file")"
+        local dest_path
+        dest_path=$(generate_destination_path "$date_str" "$original_basename" "$DESTINATION")
         if [[ $? -ne 0 ]] || [[ -z "$dest_path" ]]; then
             log_message "Could not generate destination path for $file" "ERROR"
             ERROR_FILES=$((ERROR_FILES + 1))
             FATAL_ERROR=1
             return 2
     fi
-    
-    # If destination exists, do not attempt complex conflict resolution here.
-    # Let external tools (exiftool) or filesystem semantics handle renames/overwrites.
-    if [[ -f "$dest_path" ]]; then
-        log_message "Destination already exists: $dest_path - proceeding to move/copy and letting external tools handle conflicts" "WARNING"
+
+    local desired_dest_path="$dest_path"
+    local conflict_status
+    resolve_destination_conflict "$dest_path" "$file"
+    conflict_status=$?
+    if [[ $conflict_status -eq 0 ]]; then
+        dest_path="$RESOLVED_DESTINATION_PATH"
+    elif [[ $conflict_status -eq 3 ]]; then
+        log_message "Destination conflict skipped: $desired_dest_path" "WARNING"
+        SKIPPED_FILES=$((SKIPPED_FILES + 1))
+        return 1
+    elif [[ $conflict_status -eq 4 ]]; then
+        log_message "Import aborted by user at destination conflict: $desired_dest_path" "ERROR"
+        ERROR_FILES=$((ERROR_FILES + 1))
+        FATAL_ERROR=1
+        return 2
+    else
+        log_message "Could not resolve a unique destination path for $file (wanted: $desired_dest_path)" "ERROR"
+        ERROR_FILES=$((ERROR_FILES + 1))
+        return 1
     fi
-    
-    local dest_dir=$(dirname "$dest_path")
+    if [[ "$dest_path" != "$desired_dest_path" ]]; then
+        log_message "Destination already exists or is already planned: $desired_dest_path - using: $dest_path" "WARNING"
+    fi
+
+    local dest_dir
+    dest_dir=$(dirname "$dest_path")
     
     if [[ $DRY_RUN -eq 1 ]]; then
         if [[ $KEEP_ORIGINALS -eq 1 ]]; then
@@ -663,6 +1068,9 @@ process_file() {
         else
             print_color "$BLUE" "Would move: $file -> $dest_path"
         fi
+        if should_sync_imported_metadata "$original_basename" "$date_source"; then
+            print_color "$BLUE" "Would sync metadata date: $dest_path -> $date_str"
+        fi
         PROCESSED_FILES=$((PROCESSED_FILES + 1))
         PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
         return 0
@@ -675,9 +1083,26 @@ process_file() {
         return 1
     fi
     
-    # Copy or move file using safe helpers (filter benign stderr). Let external tools handle renaming conflicts.
+    local sync_metadata_after_copy=0
+    local verification_date="$date_str"
+    if should_sync_imported_metadata "$original_basename" "$date_source"; then
+        sync_metadata_after_copy=1
+        verification_date=""
+    fi
+
+    # Copy or move file using safe helpers after destination conflicts are resolved.
     if [[ $KEEP_ORIGINALS -eq 1 ]]; then
-        if copy_with_verification "$file" "$dest_path" "$date_str"; then
+        if copy_with_verification "$file" "$dest_path" "$verification_date"; then
+            if [[ $sync_metadata_after_copy -eq 1 ]]; then
+                if ! sync_destination_metadata_to_date "$dest_path" "$date_str"; then
+                    log_message "Failed to sync destination metadata timestamps: $dest_path" "ERROR"
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                elif ! verify_synced_metadata_date "$dest_path" "$date_str"; then
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                fi
+            fi
             log_message "Copied: $file -> $dest_path" "SUCCESS"
             PROCESSED_FILES=$((PROCESSED_FILES + 1))
             PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
@@ -688,8 +1113,40 @@ process_file() {
             return 1
         fi
     else
-        if verified_move_file "$file" "$dest_path" "$date_str"; then
+        if [[ $sync_metadata_after_copy -eq 1 ]]; then
+            if copy_with_verification "$file" "$dest_path" "$verification_date"; then
+                if ! sync_destination_metadata_to_date "$dest_path" "$date_str"; then
+                    log_message "Failed to sync destination metadata timestamps: $dest_path" "ERROR"
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                fi
+                if ! verify_synced_metadata_date "$dest_path" "$date_str"; then
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                fi
+                if ! remove_source_file "$file"; then
+                    log_message "Copied, verified, and synced destination, but failed to remove source: $file" "ERROR"
+                    ERROR_FILES=$((ERROR_FILES + 1))
+                    return 1
+                fi
+                log_message "Moved: $file -> $dest_path" "SUCCESS"
+                PROCESSED_FILES=$((PROCESSED_FILES + 1))
+                PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
+                return 0
+            else
+                log_message "Failed to move after copy verification: $file -> $dest_path" "ERROR"
+                ERROR_FILES=$((ERROR_FILES + 1))
+                return 1
+            fi
+        elif verified_move_file "$file" "$dest_path" "$date_str"; then
             log_message "Moved: $file -> $dest_path" "SUCCESS"
+            if [[ $sync_metadata_after_copy -eq 1 ]]; then
+                if ! sync_destination_metadata_to_date "$dest_path" "$date_str"; then
+                    log_message "Failed to sync destination metadata timestamps: $dest_path" "WARNING"
+                elif ! verify_synced_metadata_date "$dest_path" "$date_str"; then
+                    log_message "Failed to verify synced destination metadata timestamps: $dest_path" "WARNING"
+                fi
+            fi
             PROCESSED_FILES=$((PROCESSED_FILES + 1))
             PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
             return 0
@@ -715,26 +1172,9 @@ show_report() {
     print_color "$GREEN" "=========================================="
     echo ""
     
-    echo "Files Summary:"
-    echo "  Total files found:     $TOTAL_FILES"
-    echo "  Successfully processed: $PROCESSED_FILES"
-    echo "  Skipped (no date):     $SKIPPED_FILES"
-    echo "  Errors:                $ERROR_FILES"
-    echo ""
-    
     echo "Size Summary:"
-    echo "  Total size found:      $(format_size $TOTAL_SIZE)"
-    echo "  Successfully processed: $(format_size $PROCESSED_SIZE)"
-    echo ""
-    
-    echo "Time Summary:"
-    printf "  Time elapsed:          %02d:%02d:%02d\n" $hours $minutes $seconds
-    
-    if [[ $elapsed_time -gt 0 && $PROCESSED_FILES -gt 0 ]]; then
-        local files_per_second=$((PROCESSED_FILES / elapsed_time))
-        local mb_per_second=$((PROCESSED_SIZE / elapsed_time / 1048576))
-        echo "  Speed:                 $files_per_second files/sec, ${mb_per_second}MB/sec"
-    fi
+    printf "  %-22s %s\n" "Total size found:" "$(format_size $TOTAL_SIZE)"
+    printf "  %-22s %s\n" "Successfully processed:" "$(format_size $PROCESSED_SIZE)"
     
     echo ""
     
@@ -750,6 +1190,10 @@ show_report() {
     print_color "$GREEN" "=========================================="
 }
 
+if [[ "${BASH_SOURCE[0]}" != "$0" ]]; then
+    return 0
+fi
+
 # Parse command line arguments
 while [[ $# -gt 0 ]]; do
     case $1 in
@@ -800,6 +1244,22 @@ while [[ $# -gt 0 ]]; do
             fi
             shift 2
             ;;
+        --date-source)
+            DATE_SOURCE="$2"
+            if [[ ! "$DATE_SOURCE" =~ ^(auto|exif|filesystem)$ ]]; then
+                print_color "$RED" "Error: Invalid date source. Must be one of: auto, exif, filesystem"
+                exit 1
+            fi
+            shift 2
+            ;;
+        --sync-metadata)
+            SYNC_METADATA=1
+            shift
+            ;;
+        --unattended)
+            UNATTENDED=1
+            shift
+            ;;
         --dry-run)
             DRY_RUN=1
             shift
@@ -824,6 +1284,11 @@ while [[ $# -gt 0 ]]; do
     esac
 done
 
+# Non-interactive execution cannot safely ask conflict questions.
+if [[ $UNATTENDED -eq 0 && ( ! -t 0 || ! -t 1 ) ]]; then
+    UNATTENDED=1
+fi
+
 # If no organization is provided, leave ORGANIZATION empty and filename mode will decide naming
 
 # If no source specified, default to current directory. Refuse to run when cwd is unsafe.
@@ -877,6 +1342,9 @@ echo "  Organization pattern: $ORGANIZATION"
 echo "  Destination:         $DESTINATION"
 echo "  Keep originals:      $([ $KEEP_ORIGINALS -eq 1 ] && echo "Yes" || echo "No")"
 echo "  Verify mode:         $VERIFY_MODE"
+echo "  Date source:         $DATE_SOURCE"
+echo "  Sync metadata:       $([ $SYNC_METADATA -eq 1 ] && echo "Yes" || echo "No")"
+echo "  Unattended:          $([ $UNATTENDED -eq 1 ] && echo "Yes" || echo "No")"
 echo "  Dry run:             $([ $DRY_RUN -eq 1 ] && echo "Yes" || echo "No")"
 echo "  Keep empty dirs:     $([ $CLEANUP_EMPTY_DIRS -eq 0 ] && echo "Yes" || echo "No")"
 echo "  Verbose:             $([ $VERBOSE -eq 1 ] && echo "Yes" || echo "No")"
@@ -910,6 +1378,10 @@ files=()
 while IFS= read -r file; do
     files+=("$file")
 done < <(find_source_files)
+if [[ ${#files[@]} -gt 0 ]]; then
+    IFS=$'\n' files=($(printf '%s\n' "${files[@]}" | sort))
+    unset IFS
+fi
 TOTAL_FILES=${#files[@]}
 
 if [[ $TOTAL_FILES -eq 0 ]]; then
@@ -925,6 +1397,7 @@ echo ""
 FATAL_ERROR=0
 for file in "${files[@]}"; do
     if [[ -f "$file" ]]; then
+        CURRENT_FILE_INDEX=$((CURRENT_FILE_INDEX + 1))
         process_file "$file"
         if [[ $FATAL_ERROR -eq 1 ]]; then
             print_color "$RED" "Fatal error encountered. Stopping further processing."


+304 -5

test_runner.sh

View

+                             "$result"
+                    +# Test Case 10: Timestamp Collision No-Overwrite Test
+                    +run_timestamp_collision_no_overwrite_test() {
+                    +    print_color "$GREEN" "=== Running Test 10: Timestamp Collision No-Overwrite Test ==="
+                    +    clean_test_dir
+                    +    create_test_dirs
+                    +    local log_file="$TEST_DIR/import_log.txt"
+                    +    local result=0
+                    +    : > "$log_file"
+                    +    if ! command -v ffmpeg >/dev/null 2>&1; then
+                    +        echo "ffmpeg is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    elif ! command -v exiftool >/dev/null 2>&1; then
+                    +        echo "exiftool is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    else
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=blue:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX011621.MP4" >> "$log_file" 2>&1 || result=1
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=red:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX021621.MP4" >> "$log_file" 2>&1 || result=1
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=green:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX031621.MP4" >> "$log_file" 2>&1 || result=1
+                    +        exiftool -overwrite_original -CreateDate="2026:05:15 19:20:09" "$SOURCE_DIR/GX011621.MP4" "$SOURCE_DIR/GX021621.MP4" "$SOURCE_DIR/GX031621.MP4" >> "$log_file" 2>&1 || result=1
+                    +        touch -t 202605151920.09 "$SOURCE_DIR/GX011621.MP4" "$SOURCE_DIR/GX021621.MP4" "$SOURCE_DIR/GX031621.MP4" || result=1
+                    +    fi
+                    +    # Capture pre-test state
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_before.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_before.txt" "destination"
+                    +    local command="\"$MEDIA_IMPORTER\" -s \"$SOURCE_DIR\" -d \"$DEST_DIR\" --verify-mode size -v"
+                    +    if [[ $result -eq 0 ]]; then
+                    +        run_test_command "$command" "$log_file" || result=$?
+                    +    fi
+                    +    local dest_count
+                    +    dest_count=$(find "$DEST_DIR" -type f -iname "*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$dest_count" != "3" ]]; then
+                    +        echo "Expected 3 destination MP4 files after timestamp collision import, found $dest_count" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local source_count
+                    +    source_count=$(find "$SOURCE_DIR" -type f -iname "*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$source_count" != "0" ]]; then
+                    +        echo "Expected source MP4 files to be moved, found $source_count remaining" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local unique_names
+                    +    unique_names=$(find "$DEST_DIR" -type f -iname "*.mp4" -exec basename {} \; | sort -u | wc -l | tr -d ' ')
+                    +    if [[ "$unique_names" != "3" ]]; then
+                    +        echo "Expected 3 unique destination filenames, found $unique_names" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local expected_base="$DEST_DIR/2026-05-15/2026-05-15_19-20-09.mp4"
+                    +    local expected_suffix_1="$DEST_DIR/2026-05-15/2026-05-15_19-20-09_1.mp4"
+                    +    local expected_suffix_2="$DEST_DIR/2026-05-15/2026-05-15_19-20-09_2.mp4"
+                    +    if [[ ! -f "$expected_base" || ! -f "$expected_suffix_1" || ! -f "$expected_suffix_2" ]]; then
+                    +        echo "Expected unattended numeric conflict suffixes _1 and _2 were not created" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local legacy_suffix_count
+                    +    legacy_suffix_count=$(find "$DEST_DIR" -type f -name "*__GX*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$legacy_suffix_count" != "0" ]]; then
+                    +        echo "Expected no legacy original-name conflict suffixes, found $legacy_suffix_count" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    # Capture post-test state
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_after.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_after.txt" "destination"
+                    +    # Generate report
+                    +    generate_test_report \
+                    +        "Timestamp_Collision_No_Overwrite" \
+                    +        "Processing multiple GoPro-style chapters with identical CreateDate values" \
+                    +        "Verify timestamp filename collisions do not overwrite prior imports or delete sources" \
+                    +        "Synthetic MP4 files with identical QuickTime CreateDate metadata" \
+                    +        "$command" \
+                    +        "$result"
+                    +}
+                    +# Test Case 11: GoPro Sidecar Metadata Sync Test
+                    +run_gopro_sidecar_metadata_sync_test() {
+                    +    print_color "$GREEN" "=== Running Test 11: GoPro Sidecar Metadata Sync Test ==="
+                    +    clean_test_dir
+                    +    create_test_dirs
+                    +    local log_file="$TEST_DIR/import_log.txt"
+                    +    local result=0
+                    +    : > "$log_file"
+                    +    if ! command -v ffmpeg >/dev/null 2>&1; then
+                    +        echo "ffmpeg is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    elif ! command -v exiftool >/dev/null 2>&1; then
+                    +        echo "exiftool is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    else
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=blue:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX011622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=red:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX021622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=green:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX031622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=yellow:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX041622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        exiftool -overwrite_original -QuickTime:CreateDate="2026:05:16 19:03:19" "$SOURCE_DIR/GX011622.MP4" "$SOURCE_DIR/GX021622.MP4" "$SOURCE_DIR/GX031622.MP4" "$SOURCE_DIR/GX041622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        touch -t 202605161903.20 "$SOURCE_DIR/GX011622.THM" || result=1
+                    +        touch -t 202605161915.08 "$SOURCE_DIR/GX021622.THM" || result=1
+                    +        touch -t 202605161926.56 "$SOURCE_DIR/GX031622.LRV" || result=1
+                    +        touch -t 202605161938.44 "$SOURCE_DIR/GX041622.MP4" || result=1
+                    +    fi
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_before.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_before.txt" "destination"
+                    +    local command="\"$MEDIA_IMPORTER\" -s \"$SOURCE_DIR\" -d \"$DEST_DIR\" --verify-mode size -v"
+                    +    if [[ $result -eq 0 ]]; then
+                    +        run_test_command "$command" "$log_file" || result=$?
+                    +    fi
+                    +    local first_dest="$DEST_DIR/2026-05-16/2026-05-16_19-03-20.mp4"
+                    +    local second_dest="$DEST_DIR/2026-05-16/2026-05-16_19-15-08.mp4"
+                    +    local third_dest="$DEST_DIR/2026-05-16/2026-05-16_19-26-56.mp4"
+                    +    local fourth_dest="$DEST_DIR/2026-05-16/2026-05-16_19-38-44.mp4"
+                    +    if [[ ! -f "$first_dest" || ! -f "$second_dest" || ! -f "$third_dest" || ! -f "$fourth_dest" ]]; then
+                    +        echo "Expected GoPro filesystem-based destination filenames were not created" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local first_metadata second_metadata third_metadata fourth_metadata
+                    +    if [[ -f "$first_dest" ]]; then
+                    +        first_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$first_dest" 2>/dev/null | head -1)
+                    +        if [[ "$first_metadata" != "2026:05:16 19:03:20" ]]; then
+                    +            echo "Expected first imported metadata CreateDate 2026:05:16 19:03:20, got ${first_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    if [[ -f "$third_dest" ]]; then
+                    +        third_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$third_dest" 2>/dev/null | head -1)
+                    +        if [[ "$third_metadata" != "2026:05:16 19:26:56" ]]; then
+                    +            echo "Expected third imported metadata CreateDate 2026:05:16 19:26:56, got ${third_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    if [[ -f "$fourth_dest" ]]; then
+                    +        fourth_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$fourth_dest" 2>/dev/null | head -1)
+                    +        if [[ "$fourth_metadata" != "2026:05:16 19:38:44" ]]; then
+                    +            echo "Expected fourth imported metadata CreateDate 2026:05:16 19:38:44, got ${fourth_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    if [[ -f "$second_dest" ]]; then
+                    +        second_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$second_dest" 2>/dev/null | head -1)
+                    +        if [[ "$second_metadata" != "2026:05:16 19:15:08" ]]; then
+                    +            echo "Expected second imported metadata CreateDate 2026:05:16 19:15:08, got ${second_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    local source_count
+                    +    source_count=$(find "$SOURCE_DIR" -type f -iname "*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$source_count" != "0" ]]; then
+                    +        echo "Expected GoPro source MP4 files to be moved, found $source_count remaining" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_after.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_after.txt" "destination"
+                    +    generate_test_report \
+                    +        "GoPro_Sidecar_Metadata_Sync" \
+                    +        "Processing GoPro-style MP4 chapters with THM, LRV, and MP4 filesystem timestamps" \
+                    +        "Verify GoPro imports use filesystem fallback dates and sync destination metadata automatically" \
+                    +        "Synthetic GoPro MP4 files with shared QuickTime CreateDate plus THM/LRV/no-sidecar mtime variants" \
+                    +        "$command" \
+                    +        "$result"
+                    +}
+                    +# Test Case 12: GoPro No-Sidecar Reimport Test
+                    +run_gopro_no_sidecar_reimport_test() {
+                    +    print_color "$GREEN" "=== Running Test 12: GoPro No-Sidecar Reimport Test ==="
+                    +    clean_test_dir
+                    +    create_test_dirs
+                    +    local log_file="$TEST_DIR/import_log.txt"
+                    +    local result=0
+                    +    : > "$log_file"
+                    +    if ! command -v ffmpeg >/dev/null 2>&1; then
+                    +        echo "ffmpeg is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    elif ! command -v exiftool >/dev/null 2>&1; then
+                    +        echo "exiftool is required for this regression test" >> "$log_file"
+                    +        result=1
+                    +    else
+                    +        ffmpeg -hide_banner -loglevel error -f lavfi -i color=c=purple:s=160x120:d=1 -c:v libx264 -pix_fmt yuv420p "$SOURCE_DIR/GX051622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        exiftool -overwrite_original -QuickTime:CreateDate="2026:05:16 19:03:19" "$SOURCE_DIR/GX051622.MP4" >> "$log_file" 2>&1 || result=1
+                    +        rm -f "$SOURCE_DIR/GX051622.THM" "$SOURCE_DIR/GX051622.thm" "$SOURCE_DIR/GX051622.LRV" "$SOURCE_DIR/GX051622.lrv"
+                    +        touch -t 202605161950.32 "$SOURCE_DIR/GX051622.MP4" || result=1
+                    +    fi
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_before.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_before.txt" "destination"
+                    +    local command="\"$MEDIA_IMPORTER\" -s \"$SOURCE_DIR\" -d \"$DEST_DIR\" --verify-mode size -k -v && \"$MEDIA_IMPORTER\" -s \"$SOURCE_DIR\" -d \"$DEST_DIR\" --verify-mode size -k -v"
+                    +    if [[ $result -eq 0 ]]; then
+                    +        run_test_command "$command" "$log_file" || result=$?
+                    +    fi
+                    +    local first_dest="$DEST_DIR/2026-05-16/2026-05-16_19-50-32.mp4"
+                    +    local reimport_dest="$DEST_DIR/2026-05-16/2026-05-16_19-50-32_1.mp4"
+                    +    if [[ ! -f "$first_dest" || ! -f "$reimport_dest" ]]; then
+                    +        echo "Expected no-sidecar GoPro reimport destinations were not created" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local dest_count
+                    +    dest_count=$(find "$DEST_DIR" -type f -iname "*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$dest_count" != "2" ]]; then
+                    +        echo "Expected 2 destination MP4 files after reimport, found $dest_count" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local source_count
+                    +    source_count=$(find "$SOURCE_DIR" -type f -iname "*.mp4" | wc -l | tr -d ' ')
+                    +    if [[ "$source_count" != "1" ]]; then
+                    +        echo "Expected source MP4 to remain in keep-originals reimport test, found $source_count" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local sidecar_count
+                    +    sidecar_count=$(find "$SOURCE_DIR" -type f \( -iname "*.thm" -o -iname "*.lrv" \) | wc -l | tr -d ' ')
+                    +    if [[ "$sidecar_count" != "0" ]]; then
+                    +        echo "Expected no THM/LRV sidecars in no-sidecar test, found $sidecar_count" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    if ! grep -q "Filesystem:GX051622.MP4" "$log_file"; then
+                    +        echo "Expected date source fallback to Filesystem:GX051622.MP4" >> "$log_file"
+                    +        result=1
+                    +    fi
+                    +    local first_metadata reimport_metadata
+                    +    if [[ -f "$first_dest" ]]; then
+                    +        first_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$first_dest" 2>/dev/null | head -1)
+                    +        if [[ "$first_metadata" != "2026:05:16 19:50:32" ]]; then
+                    +            echo "Expected first import metadata CreateDate 2026:05:16 19:50:32, got ${first_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    if [[ -f "$reimport_dest" ]]; then
+                    +        reimport_metadata=$(exiftool -api QuickTimeUTC=0 -s -s -s -QuickTime:CreateDate "$reimport_dest" 2>/dev/null | head -1)
+                    +        if [[ "$reimport_metadata" != "2026:05:16 19:50:32" ]]; then
+                    +            echo "Expected reimport metadata CreateDate 2026:05:16 19:50:32, got ${reimport_metadata:-none}" >> "$log_file"
+                    +            result=1
+                    +        fi
+                    +    fi
+                    +    capture_state "$SOURCE_DIR" "$TEST_DIR/source_after.txt" "source"
+                    +    capture_state "$DEST_DIR" "$TEST_DIR/dest_after.txt" "destination"
+                    +    generate_test_report \
+                    +        "GoPro_No_Sidecar_Reimport" \
+                    +        "Importing and reimporting a GoPro MP4 with no THM/LRV sidecars" \
+                    +        "Verify MP4 filesystem mtime fallback, metadata sync, and numeric suffix on reimport" \
+                    +        "Synthetic GoPro MP4 with no sidecars and wrong QuickTime metadata" \
+                    +        "$command" \
+                    +        "$result"
+                    +}
+                     # Function to show menu
+                     show_menu() {
+                         echo ""
+                         echo "7. Source Only Test"
+                         echo "8. Destination Inside Source Test"
+                         echo "9. Verify Mode Test"
+                    -    echo "10. Run All Tests"
+                    +    echo "10. Timestamp Collision No-Overwrite Test"
+                    +    echo "11. GoPro Sidecar Metadata Sync Test"
+                    +    echo "12. GoPro No-Sidecar Reimport Test"
+                    +    echo "13. Run All Tests"
+                         echo "q. Quit"
+                         echo ""
+                    -    echo -n "Select test to run (0-10, q to quit): "
+                    +    echo -n "Select test to run (0-13, q to quit): "
+                     # Function to run all tests
+                         run_destination_inside_source_test
+                         echo ""
+                         run_verify_mode_test
+                    +    echo ""
+                    +    run_timestamp_collision_no_overwrite_test
+                    +    echo ""
+                    +    run_gopro_sidecar_metadata_sync_test
+                    +    echo ""
+                    +    run_gopro_no_sidecar_reimport_test
+                         print_color "$GREEN" "All tests completed!"
+                                     run_verify_mode_test
+                                     ;;
+)
+                    +                run_timestamp_collision_no_overwrite_test
+                    +                ;;
+                    +            11)
+                    +                run_gopro_sidecar_metadata_sync_test
+                    +                ;;
+                    +            12)
+                    +                run_gopro_no_sidecar_reimport_test
+                    +                ;;
+                    +            13)
+                                     run_all_tests
+                                     ;;
+                                 q|Q)
+                                     exit 0
+                                     ;;
+                                 *)
+                    -                print_color "$RED" "Invalid choice. Please select 0-10 or q to quit."
+                    +                print_color "$RED" "Invalid choice. Please select 0-13 or q to quit."
+                                     ;;
+                             esac
+                             "verify-mode"|"9")
+                                 run_verify_mode_test
+                                 ;;
+                    -        "all"|"10")
+                    +        "timestamp-collision"|"collision"|"10")
+                    +            run_timestamp_collision_no_overwrite_test
+                    +            ;;
+                    +        "gopro-sidecar-sync"|"gopro-sync"|"11")
+                    +            run_gopro_sidecar_metadata_sync_test
+                    +            ;;
+                    +        "gopro-no-sidecar-reimport"|"gopro-reimport"|"12")
+                    +            run_gopro_no_sidecar_reimport_test
+                    +            ;;
+                    +        "all"|"13")
+                                 run_all_tests
+                                 ;;
+                             "clean")
+                                 ;;
+                             *)
+                                 print_color "$RED" "Unknown test: $1"
+                    -            echo "Usage: $0 [basic|unimportable|mixed|safety|utc|subdir|keep-empty|source-only|dest-in-source|verify-mode|all|clean|setup] or [0-10]"
+                    +            echo "Usage: $0 [basic|unimportable|mixed|safety|utc|subdir|keep-empty|source-only|dest-in-source|verify-mode|timestamp-collision|gopro-sidecar-sync|gopro-no-sidecar-reimport|all|clean|setup] or [0-13]"
+                                 exit 1
+                                 ;;
+                         esac

	@@ -2,11 +2,10 @@
2	2
3	3	## 1. Objectives
4	4
5		-The Media Importer project aims to provide a robust, efficient solution for organizing media files by date with proper timezone handling and conflict - Naming Convention: `{s/f}_{YYYYMMDD_HHMMSS}_{TestName}.md` format where:
6		- - `s` = success, `f` = failure
7		- - Followed by timestamp and test name (spaces converted to underscores)esolution.
	5	+The Media Importer project aims to provide a robust, efficient solution for organizing media files by date with proper timezone handling and conflict resolution.
8	6
9	7	Key objectives:
	8	+
10	9	- Reliable EXIF/metadata extraction and date parsing
11	10	- Proper UTC time conversion for QuickTime/Apple media files
12	11	- Flexible organization patterns (year/month/day/hour)
	@@ -23,14 +22,15 @@ When making changes to the project, follow this structured approach:
23	22
24	23	All changelog entries should follow this format:
25	24
26		-```
	25	+```text
27	26	- Date time
28	27	- Bug/Feature description
29	28	- Changes made
30	29	```
31	30
32	31	Example:
33		-```
	32	+
	33	+```text
34	34	- 2025-09-07 10:30
35	35	- Fixed data loss issue when processing already-sorted folders
36	36	- Added exclusion patterns for sorted/organized/processed folders
	@@ -40,10 +40,19 @@ Example:
40	40	### File Move Confirmation
41	41
42	42	Every file move operation should be confirmed:
	43	+
43	44	- After moving a file, the script must check that the file exists at the destination.
44	45	- If the file is not present at the destination after the move operation, the script should immediately stop and report an error.
45	46	- This ensures data integrity and prevents silent data loss.
46	47
	48	+Implementation note (2026-05):
	49	+
	50	+- Move flow now uses `copy -> verify -> delete source`.
	51	+- Verification is controlled by `--verify-mode` with modes `size` (default), `strict`, and `none`.
	52	+- `size` validates destination existence, size match, and metadata/date consistency.
	53	+- `strict` adds byte-to-byte validation (`cmp`) on top of `size` checks.
	54	+- Source deletion is allowed only after successful verification.
	55	+
47	56	### Destination Inside Source Handling
48	57
49	58	- Given the script's purpose, the destination folder may be inside the source folder.
	@@ -56,18 +65,21 @@ Every file move operation should be confirmed:
56	65	Testing is essential to ensure the script's reliability and data safety. The following methodology should be used:
57	66
58	67	#### Test Environment Setup
	68	+
59	69	- The `samples` directory contains a variety of media files for testing.
60	70	- Create a dedicated working directory named `test` for each test run.
61	71	- Copy selected files from `samples` into the `test` directory to simulate real-world scenarios.
62	72	- Perform import operations using the script, targeting the `test` directory as the source and a subdirectory (e.g., `test/sorted`) as the destination.
63	73
64	74	#### Test Execution and Documentation
	75	+
65	76	- Before and after each import operation, run `find` on both the source and destination directories to capture the file structure:
66	77	- Example: `find ./test > test/source_before.txt`
67	78	- Example: `find ./test/sorted > test/dest_before.txt`
68	79	- Log all results, including script output and directory listings, into a dedicated log file for each test.
69	80
70	81	#### Test Report Format
	82	+
71	83	Each test must generate a comprehensive Markdown report in `test/test_report.md` with the following structure:
72	84
73	85	```markdown
	@@ -161,6 +173,7 @@ Each test must generate a comprehensive Markdown report in `test/test_report.md`
161	173	```
162	174
163	175	#### Automated Test Runner
	176	+
164	177	A comprehensive test runner script (`test_runner.sh`) is available to automate the testing process:
165	178
166	179	```bash
	@@ -168,6 +181,7 @@ A comprehensive test runner script (`test_runner.sh`) is available to automate t
168	181	```
169	182
170	183	The script provides:
	184	+
171	185	- Pre-configured test scenarios for common use cases
172	186	- Automatic report generation in Markdown format
173	187	- State capture before and after test execution
	@@ -175,6 +189,7 @@ The script provides:
175	189	- Custom test support for specific scenarios
176	190
177	191	#### Test Categories
	192	+
178	193	The test runner provides the following pre-configured test scenarios:
179	194
180	195	1. Basic Functionality Test: Tests processing of files with valid EXIF data to verify correct sorting and organization
	@@ -183,13 +198,20 @@ The test runner provides the following pre-configured test scenarios:
183	198	4. Safety Protections Test: Tests destination exclusion and move confirmation mechanisms to prevent data loss
184	199	5. UTC Conversion Test: Tests UTC timestamp conversion for QuickTime/Apple EXIF data
185	200	6. Subdirectory Processing Test: Tests processing of files in nested subdirectories to ensure recursive file discovery
186		-7. Custom Test: Allows user-defined test scenarios with custom file sets and commands
	201	+7. Source Only Test: Tests automatic `source/sorted` destination behavior
	202	+8. Destination Inside Source Test: Verifies destination exclusion and prevents `sorted/sorted` recursion
	203	+9. Verify Mode Test: Verifies default `size` verification and explicit `strict` mode
	204	+10. Timestamp Collision No-Overwrite Test: Reproduces GoPro-style chapters with identical `CreateDate` values and verifies unique destination filenames with no overwrite collapse
	205	+11. GoPro Sidecar Metadata Sync Test: Verifies GoPro MP4 imports use THM start times and automatically write the corrected timestamp into destination metadata
187	206
188	207	#### Test Result Persistence
	208	+
189	209	The test runner includes automatic result persistence:
190	210
191	211	- Archival Location: Test results are saved as individual Markdown files in `test_reports/` directory
192		-- Naming Convention: `{TestName}_{YYYYMMDD_HHMMSS}.md` format for easy identification
	212	+- Naming Convention: `{s/f}_{YYYYMMDD_HHMMSS}_{TestName}.md` format where:
	213	+ - `s` = success, `f` = failure
	214	+ - Followed by timestamp and test name (spaces converted to underscores)
193	215	- Contents Preserved: Single self-contained Markdown file with:
194	216	- Complete test information and directory structures
195	217	- Full script execution output embedded inline (ANSI codes stripped for readability)
	@@ -198,13 +220,82 @@ The test runner includes automatic result persistence:
198	220	- Historical Tracking: Maintains complete test history for debugging and regression testing
199	221
200	222	#### Cleanup
	223	+
201	224	- Review the test report and verify all aspects are documented
202	225	- Clean up the `test` directory after each test run to ensure a fresh environment for subsequent tests
203	226	- Archive important test reports in a `test_reports/` directory for future reference
204	227
205	228	## 3. Changelog
206	229
	230	+### 2026-05-16 00:10 - GoPro Chapter Overwrite Postmortem
	231	+
	232	+- Documented the 2026-05-15 GoPro import data-loss incident in `INCIDENTS.md`.
	233	+- Root cause: multiple chaptered GoPro MP4 files shared the same timestamp-derived destination filename; the script warned but continued, allowing each copy to overwrite the previous destination before deleting the current source after verification.
	234	+- Clarified that QuickTime UTC-to-local conversion likely explains the visible `19:20` vs `22:20` timestamp difference, but was not the data-loss mechanism.
	235	+- Recorded surviving evidence: only the final full-resolution chapter remained in destination; `.THM` and `.LRV` sidecars remained on the card and preserved the chapter timeline.
	236	+- Added operational guidance for future GoPro imports: start with `--date-source filesystem --sync-metadata --dry-run -v` and verify that GoPro MP4 dates come from matching `.THM` sidecars.
	237	+- Added regression rule: future changes to date extraction, naming, conflict handling, copy/move, or GoPro/Garmin/Varia behavior must run `./test_runner.sh timestamp-collision`.
	238	+
	239	+---
	240	+
	241	+### 2026-05-17 08:35 - Automatic GoPro Metadata Timestamp Correction
	242	+
	243	+- Changed GoPro import behavior so `GX/GH/GP*.MP4` files with matching `.THM` sidecars prefer the THM filesystem timestamp even in default `auto` date mode.
	244	+- GoPro imports that use THM/filesystem dates now automatically sync destination metadata to the corrected clip start time; `--sync-metadata` is no longer required for this GoPro path.
	245	+- Changed move flow for metadata-corrected imports to copy and verify first, write destination metadata, verify the metadata timestamp, and only then delete the source file.
	246	+- Added `GoPro_Sidecar_Metadata_Sync` regression coverage in `test_runner.sh`.
	247	+- Verified the real `/Volumes/GOPRO` dry-run maps the current files to `19:03:20`, `19:15:08`, `19:26:56`, and `19:38:44`, all from matching THM sidecars.
	248	+
	249	+---
	250	+
	251	+### 2026-05-17 09:20 - Explicit Destination Conflict Policy
	252	+
	253	+- Added `--unattended`; unattended/non-interactive runs never prompt and resolve existing destinations with numeric suffixes (`_1`, `_2`, ...).
	254	+- Interactive runs now ask on destination conflicts and offer suffix once/all, skip once/all, or abort, so long imports do not require repeated decisions.
	255	+- Dry-run reserves planned destinations, so collisions within the same run are visible before writing.
	256	+- Updated `Timestamp_Collision_No_Overwrite` to assert numeric suffixes and reject the legacy `__GX...` suffix form.
	257	+
	258	+---
	259	+
	260	+### 2026-05-17 12:05 - Long Import UX Feedback
	261	+
	262	+- Added per-file progress lines (`Processing [n/total]`) before long copy/move work starts.
	263	+- Replaced integer `files/sec` reporting with decimal throughput that shows `files/min` for large slow imports and keeps `MB/sec` precision.
	264	+- Added average time per processed file to the final report.
	265	+
	266	+---
	267	+
	268	+### 2026-05-17 12:20 - GoPro Filesystem Date Fallback
	269	+
	270	+- GoPro auto date extraction now stays on filesystem timestamps even when THM is missing.
	271	+- Fallback order is matching `THM`, matching `LRV`, then the MP4 file's own filesystem mtime.
	272	+- Automatic metadata sync now applies to all GoPro filesystem date sources, including the MP4 fallback.
	273	+- Extended `GoPro_Sidecar_Metadata_Sync` to cover THM, LRV-only, and no-sidecar GoPro imports.
	274	+- Added `GoPro_No_Sidecar_Reimport` to verify no-sidecar MP4 fallback plus safe numeric suffixing when the same GoPro file is imported again.
	275	+- Excluded macOS AppleDouble `._*` files from source discovery so NAS metadata forks are not treated as importable media.
	276	+
	277	+---
	278	+
	279	+### 2026-05-13 21:10 - Critical Data Loss Incident + Hardening
	280	+
	281	+- Observed incident: running with explicit source on a problematic/near-full card led to writes on source media and source content removal without strong post-write confirmation.
	282	+- Impact: major data loss risk on unstable storage media (including read-only/IO edge cases).
	283	+- Clarified behavior kept as valid: destination may be inside source (`source/sorted`) and must be excluded from scanning.
	284	+- Reworked move safety in `media-importer.sh`: direct `mv` path replaced by verified flow `copy -> verify -> delete source`.
	285	+- Added verification modes via `--verify-mode`:
	286	+ - `size` (default): destination exists + size match + metadata/date validation
	287	+ - `strict`: adds byte-to-byte content validation (`cmp`)
	288	+ - `none`: disables verification (explicit opt-out)
	289	+- Applied the same verified move logic for unsortable collection path.
	290	+- Added regression coverage in `test_runner.sh`:
	291	+ - Destination-inside-source test (`-s source -d source/sorted`) to verify destination exclusion and no `sorted/sorted` recursion.
	292	+ - Verify-mode test to confirm default `size` and explicit `strict` behavior.
	293	+- Updated ignore/staging hygiene to avoid committing generated test artifacts and `.DS_Store`.
	294	+
	295	+---
	296	+
207	297	### 2025-09-07 21:15 - Test 2 and 3 Enhancements
	298	+
208	299	- Updated Test 2 (Unimportable Files Test) to include files in both root and subfolder
209	300	- Removed --collect-unimportable flag from Test 2 to test default behavior
210	301	- Updated Test 3 (Mixed Content Test) to use separate folders for sortable vs unimportable files
	@@ -215,6 +306,7 @@ The test runner includes automatic result persistence:
215	306	---
216	307
217	308	### 2025-09-07 21:25 - Documentation Enhancement
	309	+
218	310	- Added comprehensive documentation for --collect-unimportable flag in README.md
219	311	- Added Example 4 showing how to use --collect-unimportable flag
220	312	- Updated Features section to mention unimportable files handling
	@@ -224,6 +316,7 @@ The test runner includes automatic result persistence:
224	316	---
225	317
226	318	### 2025-09-07 21:30 - Git Ignore Enhancement
	319	+
227	320	- Added test_reports/ to .gitignore to exclude generated test reports from version control
228	321	- Test reports are generated files that don't need to be tracked in Git
229	322	- Prevents large numbers of timestamped report files from cluttering the repository
	@@ -232,6 +325,7 @@ The test runner includes automatic result persistence:
232	325	---
233	326
234	327	### 2025-09-07 20:40 - Source Only Test Addition
	328	+
235	329	- Added Test 8: Source Only Test to test runner
236	330	- Tests processing with only source parameter (creates sorted subdirectory automatically)
237	331	- Verifies that when no destination is specified, files are sorted into source/sorted/
	@@ -240,6 +334,7 @@ The test runner includes automatic result persistence:
240	334	---
241	335
242	336	### 2025-09-07 20:45 - Test 7 Refinement
	337	+
243	338	- Updated Test 7 to test --keep-empty-dirs functionality instead of cleanup
244	339	- Since cleanup is now default behavior, Test 7 now verifies empty directory preservation
245	340	- Renamed from "Cleanup Empty Directories Test" to "Keep Empty Directories Test"
	@@ -249,6 +344,7 @@ The test runner includes automatic result persistence:
249	344	---
250	345
251	346	### 2025-09-07 19:30 - Test Runner Directory Separation
	347	+
252	348	- Adapted test runner to use separate source and destination directories
253	349	- Changed from test/ as source to test/source/ and test/destination/
254	350	- Updated all test functions to use proper directory separation
	@@ -257,6 +353,7 @@ The test runner includes automatic result persistence:
257	353	---
258	354
259	355	### 2025-09-07 19:00 - Default Cleanup Behavior
	356	+
260	357	- Made --cleanup-empty-dirs the default behavior (implicit option)
261	358	- Added --keep-empty-dirs flag to disable cleanup if needed
262	359	- Updated help text and configuration display to reflect new default
	@@ -265,6 +362,7 @@ The test runner includes automatic result persistence:
265	362	---
266	363
267	364	### 2025-09-07 18:56 - Cleanup Empty Directories Feature
	365	+
268	366	- Added --cleanup-empty-dirs option to remove empty directories from source after processing
269	367	- Added cleanup_empty_directories() function with safe empty directory detection
270	368	- Updated final report to show cleanup status
	@@ -274,6 +372,7 @@ The test runner includes automatic result persistence:
274	372	## 4. Todo
275	373
276	374	Key areas for future development:
	375	+
277	376	- GPS metadata integration for timezone detection
278	377	- Enhanced duplicate detection
279	378	- Performance optimizations for large file sets

	@@ -5,6 +5,7 @@ This repository contains the development and testing resources for the `media-im
5	5	## Project Structure
6	6
7	7	- `media-importer.sh`: The primary script under development and testing.
	8	+- `INCIDENTS.md`: Postmortems for data-loss or near-data-loss incidents and the regression rules derived from them.
8	9	- `RPI/`: Raspberry Pi integration prototype. This area is currently frozen because Raspberry Pi 3B+ hardware with 1 GB RAM has not provided reliable validation capacity and may also be affected by hardware defects.
9	10	- `samples/code/autonas-media-importer.sh`: The original script that serves as inspiration for this project.
10	11	- `samples/`: Contains code samples, example files, and other resources used for development and testing. Note: Scripts and functions are not imported from this directory; it is strictly for reference and testing purposes.
	@@ -24,6 +25,7 @@ samples/
24	25	- All development should focus on `media-importer.sh`.
25	26	- Do not import or source scripts/functions from the `samples/` directory.
26	27	- Use resources in `samples/` only for testing and development reference.
	28	+- Destination conflicts must never be delegated to copy/move tools. In unattended runs the importer appends numeric suffixes (`_1`, `_2`, ...); in interactive runs it asks the user and supports applying the choice to all similar conflicts.
27	29
28	30	## License
29	31