bogdan feat: add media cleanup script and improve transcoder resilience

4093ab8 a month ago History

# VariaReEncoder

Batch transcoder for Garmin Varia dashcam footage. Converts H.264 source clips to HEVC MP4,
optimized for Apple Photos / QuickTime compatibility, with sidecar JSON copy and a telemetry
manifest placeholder for a future FIT sync pipeline.

## Requirements

- macOS or Linux
- `ffmpeg` and `ffprobe` in `PATH` (any recent version with libx265/libx264; videotoolbox on macOS)
- Bash 3.2+

## Quick Start

```bash
# Transcode everything in SampleFootage → Output (hardware encoder on macOS)
./garmin_varia_transcode.sh -s SampleFootage -d Output

# Preview what would happen without writing any files
./garmin_varia_transcode.sh -s SampleFootage -d Output --dry-run

# Single file, verbose output
./garmin_varia_transcode.sh -s SampleFootage/Day/clip.mp4 -d Output --verbose

# Transcode + delete originals after validation
./garmin_varia_transcode.sh -s SampleFootage -d Output --delete-source

# Long unattended run preset (delete-source + keep-going)
./garmin_varia_transcode.sh -s SampleFootage -d Output --unattended

# Media cleanup helper (Apple artifacts, zero-byte MP4, duplicate suffix normalization)
./cleanup_garmin_varia_media_folder.sh --dry-run ~/Autofs/xdev/autonas/ext01/@Camera/import
```

## Encoding Modes

| Mode       | Encoder            | Speed (30s clip) | Power\*   | Use when |
|------------|--------------------|-----------------|-----------|----------|
| `hardware` | hevc_videotoolbox  | ~4-5s           | ~35W      | Default — battery, large libraries, dashcam footage |
| `auto`     | videotoolbox → x265 → x264 | varies | varies   | Cross-platform or unknown machine |
| `quality`  | libx265 (CRF 20)   | ~50s            | ~80W      | Archival, cinema, complex sources |
| `compat`   | libx264 (CRF 19)   | ~30s            | ~70W      | Older TVs, players without HEVC support |

\* Measured on Apple Silicon MacBook Pro.

## CLI Reference

```
Options:
  -s, --source, --input DIR       Source directory or single file (default: current directory)
  -d, --destination, --output DIR Destination directory (default: current directory)
  --mode MODE                     hardware|auto|quality|compat (default: hardware)
  --crf N                         CRF for quality/compat modes (lower = better; default: 20/19)
  --no-overwrite                  Skip files that already exist at destination
  --dry-run                       Print actions without writing files
  --no-recursive                  Process only the top-level source directory
  --extensions LIST               Comma-separated extensions (default: mp4,mov,avi,m4v)
  --verbose                       Full per-operation logs + ffmpeg/ffprobe output
  --delete-source                 Delete source after strict post-encode validation
  --keep-going                    Continue after source-file failures (default: stop)
  --unattended                    Preset for long runs: --delete-source + --keep-going
  --no-apple-repack-fallback      Disable macOS avconvert fallback for unreadable MP4/MOV sources
  --apple-repack-fallback         Enable macOS avconvert fallback (default)
  -h, --help                      Show full help with encoding mode details
```

## Output Conventions

- Output is always `.mp4`
- HEVC outputs tagged `hvc1` (required for Apple Photos / QuickTime)
- Directory structure under source is preserved at destination
- File timestamps copied from source (`touch -r`)
- JSON sidecars copied 1:1 to destination
- `telemetry_manifest.json` written to destination as a placeholder for a future FIT sync pipeline

## Output Format (quiet mode, default)

```
2026-05-04 12:08:00 : Transcoding SampleFootage/Day/2026-04-04_18-10-36.mp4 ... done in 5s
2026-05-04 12:08:04 : Transcoding SampleFootage/Day/2026-04-04_18-10-05.mp4 ... done in 4s
[2026-05-04 12:08:35] [INFO] Summary: videos_processed=10 videos_skipped=0 errors=0
[2026-05-04 12:08:35] [INFO] Timing: total_elapsed=44s, video_encode_avg=4s
```

## Safety

- Destination inside source is rejected with an error
- `--delete-source` only deletes source after codec + duration validation passes
- `--dry-run` never writes files
- `Ctrl+C` behavior during long runs:
  first press requests stop after current file; second press force-stops current encode
- On macOS, unreadable MP4/MOV sources automatically try `avconvert --preset PresetPassthrough`
  as fallback before being marked as unreadable

Backward-compatible aliases:
- `--move-source` maps to `--delete-source`
- `--continue-on-error` maps to `--keep-going`

## Media Cleanup Utility

`cleanup_garmin_varia_media_folder.sh` is a companion utility for import and transcoded media folders.

What it does:
- Removes AppleDouble artifacts (`._*`) up to 4096 bytes
- Removes zero-size `.mp4` files
- Normalizes duplicate timestamp files:
  if `YYYY-MM-DD_HH-MM-SS.mp4` is missing and exactly one
  `YYYY-MM-DD_HH-MM-SS_<n>.mp4` exists, it renames it to base name
- Reports blocked duplicate groups when base exists or multiple suffixed duplicates exist

Usage:

```bash
# Preview only
./cleanup_garmin_varia_media_folder.sh --dry-run ~/Autofs/xdev/autonas/ext01/@Camera/import

# Apply changes
./cleanup_garmin_varia_media_folder.sh ~/Autofs/xdev/autonas/ext01/@Camera/import

# Example on transcoded output folder
./cleanup_garmin_varia_media_folder.sh --dry-run ~/Autofs/xdev/is-baobab/nvme0n1/@backup/Garmin
```

Exit codes:
- `0`: cleanup completed with no blocked duplicate groups
- `1`: cleanup completed but blocked duplicate groups require manual review
- `2`: invalid invocation or runtime error

## Source Files

```
SampleFootage/          Original clips (gitignored)
Output/                 Transcoded output (gitignored)
garmin_varia_transcode.sh   Main script
cleanup_garmin_varia_media_folder.sh   Media cleanup utility (import + transcoded folders)
CHANGELOG.md            Version history
DEVLOG.md               Development decisions and rationale
```