bogdan / MediaImporter
Showing 8 changed files with 951 additions and 0 deletions
BIN
.DS_Store
View
Binary file not shown.
+3 -0
.gitignore
View 
@@ -0,0 +1,3 @@
1 
+# Exclude samples directory from version control
2 
+samples/
3 
+tests/
+18 -0
CHANGELOG.md
View 
@@ -0,0 +1,18 @@
1 
+# Changelog
2 
+
3 
+## [Unreleased]
4 
+### Added
5 
+- `--collect-unsortable` argument: Files with no EXIF or media date can be moved to an `unsortable` folder in the destination.
6 
+
7 
+### Changed
8 
+- The script now **requires** EXIF or media date for processing files. If no date is found, files are skipped and left in place by default.
9 
+- No fallback to filesystem modification date or other heuristics; only EXIF or media date is used for sorting and renaming.
10 
+
11 
+### Fixed
12 
+- Improved argument parsing to support new options.
13 
+- Improved warnings and debug output handling.
14 
+
15 
+---
16 
+
17 
+## [Earlier versions]
18 
+- See previous commits for details.
+8 -0
Media Importer.code-workspace
View 
@@ -0,0 +1,8 @@
1 
+{
2 
+	"folders": [
3 
+		{
4 
+			"path": "."
5 
+		}
6 
+	],
7 
+	"settings": {}
8 
+}
+37 -0
README.md
View 
@@ -0,0 +1,37 @@
1 
+# Media Importer Project
2 
+
3 
+This repository contains the development and testing resources for the `media-importer.sh` script, which is the main product of this project.
4 
+
5 
+
6 
+## Project Structure
7 
+
8 
+- `media-importer.sh`: The primary script under development and testing.
9 
+- `samples/code/autonas-media-importer.sh`: The original script that serves as inspiration for this project.
10 
+- `samples/`: Contains code samples, example files, and other resources used for development and testing. **Note:** Scripts and functions are not imported from this directory; it is strictly for reference and testing purposes.
11 
+
12 
+## Version Control
13 
+
14 
+Version control is implemented for all files except the `samples/` directory. The `samples/` directory is excluded from version control to avoid tracking test resources and sample files.
15 
+
16 
+To exclude the `samples/` directory, the following entry should be added to the `.gitignore` file:
17 
+
18 
+```
19 
+samples/
20 
+```
21 
+
22 
+## Contribution Guidelines
23 
+
24 
+- All development should focus on `media-importer.sh`.
25 
+- Do not import or source scripts/functions from the `samples/` directory.
26 
+- Use resources in `samples/` only for testing and development reference.
27 
+
28 
+## License
29 
+
30 
+Specify your license here.
31 
+
32 
+---
33 
+
34 
+
35 
+For details on recent changes, including the removal of fallback date handling, see [CHANGELOG.md](./CHANGELOG.md).
36 
+
37 
+This file is intended for GitHub Copilot and contributors to understand the project structure and guidelines.
+140 -0
TEST_PLAN.md
View 
@@ -0,0 +1,140 @@
1 
+# Media Importer Test Plan
2 
+
3 
+## Purpose
4 
+This test plan ensures the reliability and correctness of `media-importer.sh` by covering key scenarios and edge cases.
5 
+
6 
+## Test Areas
7 
+1. **Basic Functionality**
8 
+   - Organize media files by date using all supported organization patterns (`y`, `m`, `d`, `h`).
9 
+   - Move and copy modes (with and without `--keep-originals`).
10 
+   - Dry run mode (`--dry-run`).
11 
+
12 
+2. **Source and Destination Handling**
13 
+   - Single and multiple source patterns (folders, globs, files).
14 
+   - Destination folder creation and error handling.
15 
+   - Exclusion of destination from source search.
16 
+   - Symlinked source directories.
17 
+
18 
+3. **Error Handling**
19 
+   - Missing or invalid organization pattern.
20 
+   - Missing required dependencies (e.g., exiftool).
21 
+   - Unwritable destination directory.
22 
+   - Files with no EXIF/media date metadata (should be skipped or collected as unsortable).
23 
+   - Fatal errors (e.g., cannot generate destination path).
24 
+   - Recoverable errors (e.g., failed copy/move).
25 
+
26 
+4. **Performance and Edge Cases**
27 
+   - Large number of files (10k+).
28 
+   - Files with conflicting names (should be renamed).
29 
+   - Mixed media types and extensions.
30 
+   - Files with unusual or missing extensions.
31 
+
32 
+5. **Reporting**
33 
+   - Correct summary of processed, skipped, and error files.
34 
+   - Accurate size and speed reporting.
35 
+   - Verbose and non-verbose output modes.
36 
+
37 
+
38 
+
39 
+## Sample Data Note
40 
+The `samples/media` directory contains files both with and without EXIF date metadata. Any test that does not correctly detect and handle files lacking EXIF dates (i.e., skips or collects them as unsortable) is considered failed.
41 
+
42 
+## Test Directory Usage
43 
+To facilitate testing, the `tests` directory will be used as the working directory for all test scenarios. Before each test, relevant files will be copied from `samples/media` into `tests`. After inspecting the results, the `tests` directory will be cleaned to ensure a fresh state for the next test.
44 
+
45 
+## Specific Test Cases
46 
+
47 
+### TC01: Basic Organization Patterns
48 
+- **Objective**: Test all organization patterns (y, m, d, h)
49 
+- **Setup**: Copy sample files from `samples/media` to `tests/media`
50 
+- **Commands**:
51 
+  - `./media-importer.sh -s ./tests/media -d ./tests/sorted_y -o y`
52 
+  - `./media-importer.sh -s ./tests/media -d ./tests/sorted_m -o m`
53 
+  - `./media-importer.sh -s ./tests/media -d ./tests/sorted_d -o d`
54 
+  - `./media-importer.sh -s ./tests/media -d ./tests/sorted_h -o h`
55 
+- **Expected**: Files organized according to each pattern
56 
+- **Cleanup**: `rm -rf ./tests/*`
57 
+
58 
+### TC02: Copy vs Move Mode
59 
+- **Objective**: Test --keep-originals flag
60 
+- **Setup**: Copy sample files to `tests/media`
61 
+- **Commands**:
62 
+  - `./media-importer.sh -s ./tests/media -d ./tests/copied -k`
63 
+  - `./media-importer.sh -s ./tests/media -d ./tests/moved`
64 
+- **Expected**: First command copies files, second moves them
65 
+- **Cleanup**: `rm -rf ./tests/*`
66 
+
67 
+### TC03: Dry Run Mode
68 
+- **Objective**: Test --dry-run functionality
69 
+- **Setup**: Copy sample files to `tests/media`
70 
+- **Command**: `./media-importer.sh -s ./tests/media -d ./tests/dry --dry-run`
71 
+- **Expected**: Shows what would be done without moving files
72 
+- **Cleanup**: `rm -rf ./tests/*`
73 
+
74 
+### TC04: Symlinked Source
75 
+- **Objective**: Test symlinked source directory
76 
+- **Setup**:
77 
+  - `mkdir -p tests/media`
78 
+  - `ln -sf ../samples/media tests/linked_media`
79 
+- **Command**: `./media-importer.sh -s ./tests/linked_media -d ./tests/from_symlink`
80 
+- **Expected**: Files from symlinked directory are processed
81 
+- **Cleanup**: `rm -rf ./tests/*`
82 
+
83 
+### TC05: Error Handling - Invalid Pattern
84 
+- **Objective**: Test invalid organization pattern
85 
+- **Command**: `./media-importer.sh -o z`
86 
+- **Expected**: Error message and exit code 1
87 
+- **Cleanup**: None needed
88 
+
89 
+### TC06: Same Source and Destination (Fatal Error)
90 
+- **Objective**: Test fatal error handling
91 
+- **Setup**: Copy sample files to `tests/media`
92 
+- **Command**: `./media-importer.sh -s ./tests -d ./tests`
93 
+- **Expected**: Fatal error stops processing immediately
94 
+- **Cleanup**: `rm -rf ./tests/*`
95 
+
96 
+## New Feature: --full-date Argument & Default Flat Naming
97 
+
98 
+### TC07: --full-date Argument (EXIF Date Required)
99 
+- **Objective**: Verify that only files with EXIF/media dates are processed and named with full date format.
100 
+- **Setup**: Copy sample files to `tests/media`
101 
+- **Command**: `./media-importer.sh --full-date -s ./tests/media -d ./tests/flat`
102 
+- **Expected**: Only files with EXIF dates in `tests/flat` named as `yyyy-mm-dd_hh-mm-ss.ext` (no subfolders). Files without EXIF dates remain in source.
103 
+- **Cleanup**: `rm -rf ./tests/*`
104 
+
105 
+### TC08: No Organization Directive (EXIF Date Required)
106 
+- **Objective**: Verify that omitting the organization directive results in flat full-date naming for files with EXIF dates only.
107 
+- **Setup**: Copy sample files to `tests/media`
108 
+- **Command**: `./media-importer.sh -s ./tests/media -d ./tests/flat_default`
109 
+- **Expected**: Only files with EXIF dates in `tests/flat_default` named as `yyyy-mm-dd_hh-mm-ss.ext` (no subfolders). Files without EXIF dates remain in source.
110 
+- **Cleanup**: `rm -rf ./tests/*`
111 
+
112 
+### TC09: Organization Directive with --full-date (EXIF Date Required)
113 
+- **Objective**: Verify that --full-date overrides any organization directive and processes only files with EXIF dates.
114 
+- **Setup**: Copy sample files to `tests/media`
115 
+- **Command**: `./media-importer.sh --full-date -o y -s ./tests/media -d ./tests/flat_override`
116 
+- **Expected**: Only files with EXIF dates in `tests/flat_override` named as `yyyy-mm-dd_hh-mm-ss.ext` (no subfolders). Files without EXIF dates remain in source.
117 
+- **Cleanup**: `rm -rf ./tests/*`
118 
+
119 
+### TC10: --collect-unsortable Flag
120 
+- **Objective**: Verify that files without EXIF dates are moved to unsortable folder when flag is used.
121 
+- **Setup**: Copy sample files to `tests/media`
122 
+- **Command**: `./media-importer.sh --full-date --collect-unsortable -s ./tests/media -d ./tests/flat`
123 
+- **Expected**: Files with EXIF dates in `tests/flat` named as `yyyy-mm-dd_hh-mm-ss.ext`. Files without EXIF dates in `tests/flat/unsortable/`.
124 
+- **Cleanup**: `rm -rf ./tests/*`
125 
+
126 
+### TC11: Basic Organization with EXIF Requirement
127 
+- **Objective**: Test that organization patterns only process files with EXIF dates.
128 
+- **Setup**: Copy sample files to `tests/media`
129 
+- **Command**: `./media-importer.sh -o y -s ./tests/media -d ./tests/sorted_y`
130 
+- **Expected**: Only files with EXIF dates organized by year. Files without EXIF dates remain in source.
131 
+- **Cleanup**: `rm -rf ./tests/*`
132 
+
133 
+## Test Execution Framework
134 
+- Before each test: Prepare test data from `samples/media`
135 
+- Run test command and capture output
136 
+- Verify expected behavior and results
137 
+- After each test: Clean `tests` directory with `rm -rf ./tests/*`
138 
+
139 
+---
140 
+This plan should be updated as new features or bug fixes are added.
+16 -0
TODO.md
View 
@@ -0,0 +1,16 @@
1 
+# TODO
2 
+
3 
+## Tasks
4 
+
5 
+1. **QuickTime Local Time from GPS Metadata**
6 
+   - When processing QuickTime files, use GPS metadata (if available) to determine the local time zone and convert the UTC timestamp to local time accordingly.
7 
+   - Fallback to system timezone if GPS data is not present.
8 
+
9 
+2. **Implicit Behavior Selection**
10 
+   - Define and document the script's default behavior when no explicit directives/arguments are provided.
11 
+   - Allow users to override defaults via command-line arguments.
12 
+   - Ensure stability and predictability in all cases.
13 
+
14 
+---
15 
+
16 
+(Feel free to expand with subtasks or implementation notes.)
+729 -0
media-importer.sh
View 
@@ -0,0 +1,729 @@
1 
+#!/bin/bash
2 
+
3 
+# Standalone Media Importer
4 
+# Version: 1.0
5 
+# A comprehensive media file organizer that sorts photos and videos by date
6 
+# with various organization patterns and timezone handling
7 
+
8 
+VERSION="1.0"
9 
+SCRIPT_NAME="Standalone Media Importer"
10 
+
11 
+# Default values
12 
+ORGANIZATION=""
13 
+FORCE_FULL_DATE=0
14 
+COLLECT_UNSORTABLE=0
15 
+SOURCE_PATTERNS=()
16 
+DESTINATION=""
17 
+KEEP_ORIGINALS=0
18 
+DRY_RUN=0
19 
+VERBOSE=0
20 
+
21 
+# Counters and statistics
22 
+TOTAL_FILES=0
23 
+PROCESSED_FILES=0
24 
+SKIPPED_FILES=0
25 
+ERROR_FILES=0
26 
+TOTAL_SIZE=0
27 
+PROCESSED_SIZE=0
28 
+START_TIME=$(date +%s)
29 
+
30 
+# Colors for output
31 
+RED='\033[0;31m'
32 
+GREEN='\033[0;32m'
33 
+YELLOW='\033[1;33m'
34 
+BLUE='\033[0;34m'
35 
+NC='\033[0m' # No Color
36 
+
37 
+# Function to print colored output
38 
+print_color() {
39 
+    local color="$1"
40 
+    local message="$2"
41 
+    echo -e "${color}${message}${NC}"
42 
+}
43 
+
44 
+# Function to log messages with timestamp
45 
+log_message() {
46 
+    local message="$1"
47 
+    local level="${2:-INFO}"
48 
+    local timestamp=$(date '+%Y-%m-%d %H:%M:%S')
49 
+
50 
+    case "$level" in
51 
+        "ERROR")
52 
+            print_color "$RED" "[$timestamp] ERROR: $message" >&2
53 
+            ;;
54 
+        "WARNING")
55 
+            print_color "$YELLOW" "[$timestamp] WARNING: $message"
56 
+            ;;
57 
+        "SUCCESS")
58 
+            print_color "$GREEN" "[$timestamp] SUCCESS: $message"
59 
+            ;;
60 
+        "INFO")
61 
+            if [[ $VERBOSE -eq 1 ]]; then
62 
+                print_color "$BLUE" "[$timestamp] INFO: $message"
63 
+            fi
64 
+            ;;
65 
+        *)
66 
+            echo "[$timestamp] $message"
67 
+            ;;
68 
+    esac
69 
+}
70 
+
71 
+# Function to display help
72 
+show_help() {
73 
+    cat << EOF
74 
+$SCRIPT_NAME v$VERSION
75 
+
76 
+USAGE:
77 
+    $0 [OPTIONS]
78 
+
79 
+DESCRIPTION:
80 
+    Organizes media files (photos and videos) by date with various naming patterns.
81 
+    if [[ $exif_found -eq 0 ]]; then
82 
+        log_message "Warning: No EXIF date found for $file. Using filesystem modification time."
83 
+    -o, --organization PATTERN
84 
+        Organization pattern:
85 
+        y -> target/yyyy/mm-dd_hh-mm-ss.orig_ext
86 
+        m -> target/yyyy/mm/dd_hh-mm-ss.orig_ext
87 
+        d -> target/yyyy/mm/dd/mm-dd_hh-mm-ss.orig_ext
88 
+        h -> target/yyyy/mm/dd/hh/mm-ss.orig_ext
89 
+
90 
+    --full-date
91 
+        Force all files to be named with full date (yyyy-mm-dd_hh-mm-ss.ext) in the destination folder, regardless of organization pattern.
92 
+
93 
+    -s, --source PATTERN
94 
+        Source folder pattern(s) with simple regex support (*^$)
95 
+        Can be specified multiple times
96 
+        Examples:
97 
+            -s "/DCIM/*Video$"
98 
+            -s "/path/to/photos"
99 
+            -s "*.jpg"
100 
+        Default: all subfolders in current directory except destination
101 
+
102 
+    -d, --destination PATH
103 
+        Destination folder (default: ./sorted)
104 
+
105 
+    -k, --keep-originals
106 
+        Keep original files (copy instead of move)
107 
+
108 
+    --dry-run
109 
+        Show what would be done without actually doing it
110 
+
111 
+    -v, --verbose
112 
+        Enable verbose output
113 
+
114 
+    -h, --help
115 
+        Show this help message
116 
+
117 
+    --version
118 
+        Show version information
119 
+
120 
+EXAMPLES:
121 
+    # Basic usage - organize all media in current directory
122 
+    $0
123 
+
124 
+    # Organize with monthly folders, keep originals
125 
+    $0 -o m -k
126 
+
127 
+    # Process specific folders with hourly organization
128 
+    # Return date and source (no warnings or debug output)
129 
+    echo "$create_date|$date_source"
130 
+    # Dry run with verbose output
131 
+    $0 --dry-run -v -s "*.mov" -d "/tmp/test"
132 
+
133 
+DEPENDENCIES:
134 
+    Required: exiftool
135 
+    Optional: mediainfo, file (for enhanced metadata detection)
136 
+
137 
+EOF
138 
+}
139 
+
140 
+# Function to show version
141 
+show_version() {
142 
+    echo "$SCRIPT_NAME v$VERSION"
143 
+    echo "A comprehensive media file organizer with timezone support"
144 
+}
145 
+
146 
+# Function to check dependencies
147 
+check_dependencies() {
148 
+    local missing_deps=()
149 
+
150 
+    # Check for required dependencies
151 
+    if ! command -v exiftool &> /dev/null; then
152 
+        missing_deps+=("exiftool")
153 
+    fi
154 
+
155 
+    # Check for optional dependencies
156 
+    local optional_missing=()
157 
+    if ! command -v mediainfo &> /dev/null; then
158 
+        optional_missing+=("mediainfo")
159 
+    fi
160 
+
161 
+    if ! command -v file &> /dev/null; then
162 
+        optional_missing+=("file")
163 
+    fi
164 
+
165 
+    if [[ ${#missing_deps[@]} -gt 0 ]]; then
166 
+        print_color "$RED" "ERROR: Missing required dependencies:"
167 
+        for dep in "${missing_deps[@]}"; do
168 
+            echo "  - $dep"
169 
+        done
170 
+        echo ""
171 
+        echo "Installation instructions:"
172 
+        echo "  macOS: brew install exiftool"
173 
+        echo "  Ubuntu/Debian: sudo apt-get install libimage-exiftool-perl"
174 
+        echo "  CentOS/RHEL: sudo yum install perl-Image-ExifTool"
175 
+        echo "  Arch: sudo pacman -S perl-image-exiftool"
176 
+        exit 1
177 
+    fi
178 
+
179 
+    if [[ ${#optional_missing[@]} -gt 0 && $VERBOSE -eq 1 ]]; then
180 
+        log_message "Optional dependencies not found (functionality may be limited): ${optional_missing[*]}" "WARNING"
181 
+    fi
182 
+
183 
+    log_message "All required dependencies found" "SUCCESS"
184 
+}
185 
+
186 
+# Function to get file size in bytes
187 
+get_file_size() {
188 
+    local file="$1"
189 
+    if [[ -f "$file" ]]; then
190 
+        if command -v stat &> /dev/null; then
191 
+            # Try GNU stat first (Linux)
192 
+            stat -c%s "$file" 2>/dev/null || stat -f%z "$file" 2>/dev/null
193 
+        else
194 
+            # Fallback to ls
195 
+            ls -l "$file" | awk '{print $5}'
196 
+        fi
197 
+    else
198 
+        echo "0"
199 
+    fi
200 
+}
201 
+
202 
+# Function to format file size
203 
+format_size() {
204 
+    local size=$1
205 
+    if (( size < 1024 )); then
206 
+        echo "${size}B"
207 
+    elif (( size < 1048576 )); then
208 
+        echo "$(( size / 1024 ))KB"
209 
+    elif (( size < 1073741824 )); then
210 
+        echo "$(( size / 1048576 ))MB"
211 
+    else
212 
+        echo "$(( size / 1073741824 ))GB"
213 
+    fi
214 
+}
215 
+
216 
+# Function to extract date from file
217 
+extract_file_date() {
218 
+    local file="$1"
219 
+    local create_date=""
220 
+    local date_source=""
221 
+    local exif_found=0
222 
+    # Try to get creation date from EXIF data
223 
+    local exif_output=$(exiftool -G1 -s -CreateDate -DateTimeOriginal -DateTime "$file" 2>/dev/null)
224 
+    if [[ -n "$exif_output" ]]; then
225 
+        # Parse the exiftool output to find the best date
226 
+        while IFS= read -r line; do
227 
+            if [[ "$line" =~ ^\[([^\]]+)\][[:space:]]*([^:]+)[[:space:]]*:[[:space:]]*(.+)$ ]]; then
228 
+                local group="${BASH_REMATCH[1]}"
229 
+                local tag="${BASH_REMATCH[2]}"
230 
+                local value="${BASH_REMATCH[3]}"
231 
+                # Trim spaces from tag name
232 
+                tag=$(echo "$tag" | sed 's/[[:space:]]*$//')
233 
+                # Prefer DateTimeOriginal, then CreateDate, then DateTime
234 
+                if [[ "$tag" == "DateTimeOriginal" && -z "$create_date" ]]; then
235 
+                    create_date="$value"
236 
+                    date_source="$group:$tag"
237 
+                    exif_found=1
238 
+                elif [[ "$tag" == "CreateDate" && "$date_source" != *"DateTimeOriginal"* ]]; then
239 
+                    create_date="$value"
240 
+                    date_source="$group:$tag"
241 
+                    exif_found=1
242 
+                elif [[ "$tag" == "DateTime" && -z "$create_date" ]]; then
243 
+                    create_date="$value"
244 
+                    date_source="$group:$tag"
245 
+                    exif_found=1
246 
+                fi
247 
+            fi
248 
+        done <<< "$exif_output"
249 
+    fi
250 
+    # If no EXIF date found, try mediainfo for video files
251 
+    if [[ -z "$create_date" ]] && command -v mediainfo &> /dev/null; then
252 
+        local media_date=$(mediainfo --Inform="General;%Recorded_Date%" "$file" 2>/dev/null)
253 
+        if [[ -n "$media_date" && "$media_date" != "0000-00-00 00:00:00" ]]; then
254 
+            create_date="$media_date"
255 
+            date_source="MediaInfo:Recorded_Date"
256 
+        fi
257 
+    fi
258 
+    # If no EXIF or mediainfo date found, return failure
259 
+    if [[ -z "$create_date" ]]; then
260 
+        return 2  # No date metadata found
261 
+    fi
262 
+
263 
+    # Convert EXIF format (YYYY:MM:DD HH:MM:SS) to standard format
264 
+    # Always output as yyyy-mm-dd hh:mm:ss (pad single digits)
265 
+    if [[ "$create_date" =~ ^([0-9]{4}):([0-9]{1,2}):([0-9]{1,2})[[:space:]]*([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2})$ ]]; then
266 
+        year="${BASH_REMATCH[1]}"
267 
+        month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
268 
+        day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
269 
+        hour=$(printf "%02d" "$((10#${BASH_REMATCH[4]}))")
270 
+        minute=$(printf "%02d" "$((10#${BASH_REMATCH[5]}))")
271 
+        second=$(printf "%02d" "$((10#${BASH_REMATCH[6]}))")
272 
+        create_date="$year-$month-$day $hour:$minute:$second"
273 
+    else
274 
+        # Try to convert yyyy-mm-dd hh:mm:ss (already correct)
275 
+        if [[ "$create_date" =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})[[:space:]]*([0-9]{2}):([0-9]{2}):([0-9]{2})$ ]]; then
276 
+            # Already correct
277 
+            :
278 
+        else
279 
+            print_color "$RED" "Error: Cannot parse date '$create_date'" >&2
280 
+            return 2
281 
+        fi
282 
+    fi
283 
+
284 
+    # For QuickTime files, the CreateDate is in UTC and needs conversion to local time
285 
+    if [[ "$date_source" == *"QuickTime"* ]]; then
286 
+        # Convert UTC time to local time
287 
+        if [[ "$OSTYPE" == "darwin"* ]]; then
288 
+            # On macOS, use TZ=UTC to interpret the input time as UTC
289 
+            local utc_timestamp=$(TZ=UTC date -j -f "%Y-%m-%d %H:%M:%S" "$create_date" "+%s" 2>/dev/null)
290 
+            if [[ -n "$utc_timestamp" ]]; then
291 
+                create_date=$(date -j -r "$utc_timestamp" "+%Y-%m-%d %H:%M:%S" 2>/dev/null)
292 
+                date_source="$date_source (converted from UTC)"
293 
+            fi
294 
+        else
295 
+            local utc_timestamp=$(date -d "$create_date UTC" "+%s" 2>/dev/null)
296 
+            if [[ -n "$utc_timestamp" ]]; then
297 
+                create_date=$(date -d "@$utc_timestamp" "+%Y-%m-%d %H:%M:%S" 2>/dev/null)
298 
+                date_source="$date_source (converted from UTC)"
299 
+            fi
300 
+        fi
301 
+    fi
302 
+
303 
+    echo "$create_date|$date_source"
304 
+    return 0
305 
+}
306 
+
307 
+# Function to generate destination path based on organization pattern
308 
+generate_destination_path() {
309 
+    local date_str="$1"
310 
+    local original_filename="$2"
311 
+    local base_destination="$3"
312 
+
313 
+    # Extract date components - handle both GNU and BSD date
314 
+    local year month day hour minute second
315 
+    if [[ "$OSTYPE" == "darwin"* ]]; then
316 
+        # macOS (BSD date) - use more robust regex (allow single-digit month/day, tolerate extra spaces)
317 
+        if [[ "$date_str" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2})[[:space:]]*([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2})$ ]]; then
318 
+            year="${BASH_REMATCH[1]}"
319 
+            month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
320 
+            day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
321 
+            hour=$(printf "%02d" "$((10#${BASH_REMATCH[4]}))")
322 
+            minute=$(printf "%02d" "$((10#${BASH_REMATCH[5]}))")
323 
+            second=$(printf "%02d" "$((10#${BASH_REMATCH[6]}))")
324 
+        else
325 
+            return 1
326 
+        fi
327 
+    else
328 
+        # Linux (GNU date)
329 
+        year=$(date -d "$date_str" "+%Y" 2>/dev/null)
330 
+        month=$(date -d "$date_str" "+%m" 2>/dev/null)
331 
+        day=$(date -d "$date_str" "+%d" 2>/dev/null)
332 
+        hour=$(date -d "$date_str" "+%H" 2>/dev/null)
333 
+        minute=$(date -d "$date_str" "+%M" 2>/dev/null)
334 
+        second=$(date -d "$date_str" "+%S" 2>/dev/null)
335 
+    fi
336 
+
337 
+    if [[ -z "$year" || -z "$month" || -z "$day" ]]; then
338 
+        return 1
339 
+    fi
340 
+
341 
+    # Get file extension
342 
+    local extension="${original_filename##*.}"
343 
+    local lowercase_ext=$(echo "$extension" | tr '[:upper:]' '[:lower:]')
344 
+
345 
+    # Generate path and filename based on organization pattern
346 
+    local dir_path=""
347 
+    local filename=""
348 
+    if [[ $FORCE_FULL_DATE -eq 1 ]]; then
349 
+        dir_path="$base_destination"
350 
+        filename="${year}-${month}-${day}_${hour}-${minute}-${second}.${lowercase_ext}"
351 
+    else
352 
+        case "$ORGANIZATION" in
353 
+            "y")
354 
+                dir_path="$base_destination/$year"
355 
+                filename="${month}-${day}_${hour}-${minute}-${second}.${lowercase_ext}"
356 
+                ;;
357 
+            "m")
358 
+                dir_path="$base_destination/$year/$month"
359 
+                filename="${day}_${hour}-${minute}-${second}.${lowercase_ext}"
360 
+                ;;
361 
+            "d")
362 
+                dir_path="$base_destination/$year/$month/$day"
363 
+                filename="${month}-${day}_${hour}-${minute}-${second}.${lowercase_ext}"
364 
+                ;;
365 
+            "h")
366 
+                dir_path="$base_destination/$year/$month/$day/$hour"
367 
+                filename="${minute}-${second}.${lowercase_ext}"
368 
+                ;;
369 
+            *)
370 
+                log_message "Invalid organization pattern: $ORGANIZATION" "ERROR"
371 
+                return 1
372 
+                ;;
373 
+        esac
374 
+    fi
375 
+    echo "$dir_path/$filename"
376 
+    return 0
377 
+}
378 
+
379 
+# Function to find files matching patterns
380 
+find_source_files() {
381 
+    local files=()
382 
+
383 
+    if [[ ${#SOURCE_PATTERNS[@]} -eq 0 ]]; then
384 
+        # Default: find all media files in current directory and subdirectories
385 
+        # Exclude destination directory if it's a subdirectory of current directory
386 
+    local find_cmd="find -L . -type f"
387 
+
388 
+        # Add exclusion for destination if it's relative to current directory
389 
+        if [[ "$DESTINATION" =~ ^\./.*$ ]] || [[ "$DESTINATION" =~ ^[^/].*$ ]]; then
390 
+            local abs_dest=$(cd "$DESTINATION" 2>/dev/null && pwd)
391 
+            local abs_current=$(pwd)
392 
+            if [[ "$abs_dest" == "$abs_current"* ]]; then
393 
+                find_cmd="$find_cmd ! -path \"$DESTINATION/*\""
394 
+            fi
395 
+        fi
396 
+
397 
+        # Add media file extensions
398 
+        local extensions=("*.jpg" "*.jpeg" "*.png" "*.tiff" "*.tif" "*.cr2" "*.nef" "*.arw" "*.dng" "*.raw" "*.mp4" "*.mov" "*.avi" "*.mts" "*.m2ts" "*.mkv" "*.wmv" "*.3gp" "*.m4v")
399 
+        local ext_pattern=""
400 
+        for ext in "${extensions[@]}"; do
401 
+            if [[ -n "$ext_pattern" ]]; then
402 
+                ext_pattern="$ext_pattern -o"
403 
+            fi
404 
+            ext_pattern="$ext_pattern -iname $ext"
405 
+        done
406 
+
407 
+    eval "$find_cmd \\( $ext_pattern \\)" | while IFS= read -r file; do
408 
+            echo "$file"
409 
+        done
410 
+    else
411 
+        # Use specified patterns
412 
+        for pattern in "${SOURCE_PATTERNS[@]}"; do
413 
+            # Handle different pattern types
414 
+            if [[ -d "$pattern" ]]; then
415 
+                # Directory pattern
416 
+                find -L "$pattern" -type f \( -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.png" -o -iname "*.tiff" -o -iname "*.tif" -o -iname "*.cr2" -o -iname "*.nef" -o -iname "*.arw" -o -iname "*.dng" -o -iname "*.raw" -o -iname "*.mp4" -o -iname "*.mov" -o -iname "*.avi" -o -iname "*.mts" -o -iname "*.m2ts" -o -iname "*.mkv" -o -iname "*.wmv" -o -iname "*.3gp" -o -iname "*.m4v" \) 2>/dev/null
417 
+            elif [[ "$pattern" == *"*"* ]] || [[ "$pattern" == *"?"* ]]; then
418 
+                # Glob pattern
419 
+                for file in $pattern; do
420 
+                    if [[ -f "$file" ]]; then
421 
+                        echo "$file"
422 
+                    fi
423 
+                done
424 
+            else
425 
+                # Exact file or directory
426 
+                if [[ -f "$pattern" ]]; then
427 
+                    echo "$pattern"
428 
+                elif [[ -d "$pattern" ]]; then
429 
+                    find -L "$pattern" -type f \( -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.png" -o -iname "*.tiff" -o -iname "*.tif" -o -iname "*.cr2" -o -iname "*.nef" -o -iname "*.arw" -o -iname "*.dng" -o -iname "*.raw" -o -iname "*.mp4" -o -iname "*.mov" -o -iname "*.avi" -o -iname "*.mts" -o -iname "*.m2ts" -o -iname "*.mkv" -o -iname "*.wmv" -o -iname "*.3gp" -o -iname "*.m4v" \) 2>/dev/null
430 
+                fi
431 
+            fi
432 
+        done
433 
+    fi
434 
+}
435 
+
436 
+# Function to process a single file
437 
+process_file() {
438 
+    local file="$1"
439 
+    local file_size=$(get_file_size "$file")
440 
+    TOTAL_SIZE=$((TOTAL_SIZE + file_size))
441 
+
442 
+    log_message "Processing: $file" "INFO"
443 
+
444 
+    # Extract date information
445 
+    local date_info=$(extract_file_date "$file")
446 
+        local extract_status=$?
447 
+        if [[ $extract_status -eq 2 ]]; then
448 
+            if [[ $COLLECT_UNSORTABLE -eq 1 ]]; then
449 
+                local unsortable_dir="$DESTINATION/unsortable"
450 
+                mkdir -p "$unsortable_dir"
451 
+                local unsortable_path="$unsortable_dir/$(basename "$file")"
452 
+                if [[ $DRY_RUN -eq 1 ]]; then
453 
+                    print_color "$BLUE" "Would move unsortable: $file -> $unsortable_path"
454 
+                else
455 
+                    if mv "$file" "$unsortable_path"; then
456 
+                        log_message "Unsortable: $file -> $unsortable_path" "SUCCESS"
457 
+                    else
458 
+                        log_message "Failed to move unsortable file: $file" "ERROR"
459 
+                    fi
460 
+                fi
461 
+                SKIPPED_FILES=$((SKIPPED_FILES + 1))
462 
+            else
463 
+                log_message "Could not extract date from $file - skipping" "WARNING"
464 
+                SKIPPED_FILES=$((SKIPPED_FILES + 1))
465 
+            fi
466 
+            return 1
467 
+        elif [[ $extract_status -ne 0 ]] || [[ -z "$date_info" ]]; then
468 
+            log_message "Could not extract date from $file - skipping" "WARNING"
469 
+            SKIPPED_FILES=$((SKIPPED_FILES + 1))
470 
+            return 1
471 
+        fi
472 
+        local date_str="${date_info%|*}"
473 
+        local date_source="${date_info#*|}"
474 
+        log_message "Date: $date_str (from $date_source)" "INFO"
475 
+        # Generate destination path
476 
+        local dest_path=$(generate_destination_path "$date_str" "$(basename "$file")" "$DESTINATION")
477 
+        if [[ $? -ne 0 ]] || [[ -z "$dest_path" ]]; then
478 
+            log_message "Could not generate destination path for $file" "ERROR"
479 
+            ERROR_FILES=$((ERROR_FILES + 1))
480 
+            FATAL_ERROR=1
481 
+            return 2
482 
+    fi
483 
+
484 
+    # Handle filename conflicts
485 
+    local counter=1
486 
+    local original_dest_path="$dest_path"
487 
+    while [[ -f "$dest_path" ]]; do
488 
+        local dir_path=$(dirname "$original_dest_path")
489 
+        local filename=$(basename "$original_dest_path")
490 
+        local name_without_ext="${filename%.*}"
491 
+        local ext="${filename##*.}"
492 
+        dest_path="$dir_path/${name_without_ext}_${counter}.${ext}"
493 
+        counter=$((counter + 1))
494 
+    done
495 
+
496 
+    local dest_dir=$(dirname "$dest_path")
497 
+
498 
+    if [[ $DRY_RUN -eq 1 ]]; then
499 
+        if [[ $KEEP_ORIGINALS -eq 1 ]]; then
500 
+            print_color "$BLUE" "Would copy: $file -> $dest_path"
501 
+        else
502 
+            print_color "$BLUE" "Would move: $file -> $dest_path"
503 
+        fi
504 
+        PROCESSED_FILES=$((PROCESSED_FILES + 1))
505 
+        PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
506 
+        return 0
507 
+    fi
508 
+
509 
+    # Create destination directory
510 
+    if ! mkdir -p "$dest_dir"; then
511 
+        log_message "Could not create directory: $dest_dir" "ERROR"
512 
+        ERROR_FILES=$((ERROR_FILES + 1))
513 
+        return 1
514 
+    fi
515 
+
516 
+    # Copy or move file
517 
+    if [[ $KEEP_ORIGINALS -eq 1 ]]; then
518 
+        if cp "$file" "$dest_path"; then
519 
+            log_message "Copied: $file -> $dest_path" "SUCCESS"
520 
+            PROCESSED_FILES=$((PROCESSED_FILES + 1))
521 
+            PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
522 
+            return 0
523 
+        else
524 
+            log_message "Failed to copy: $file" "ERROR"
525 
+            ERROR_FILES=$((ERROR_FILES + 1))
526 
+            return 1
527 
+        fi
528 
+    else
529 
+        if mv "$file" "$dest_path"; then
530 
+            log_message "Moved: $file -> $dest_path" "SUCCESS"
531 
+            PROCESSED_FILES=$((PROCESSED_FILES + 1))
532 
+            PROCESSED_SIZE=$((PROCESSED_SIZE + file_size))
533 
+            return 0
534 
+        else
535 
+            log_message "Failed to move: $file" "ERROR"
536 
+            ERROR_FILES=$((ERROR_FILES + 1))
537 
+            return 1
538 
+        fi
539 
+    fi
540 
+}
541 
+
542 
+# Function to display final report
543 
+show_report() {
544 
+    local end_time=$(date +%s)
545 
+    local elapsed_time=$((end_time - START_TIME))
546 
+    local hours=$((elapsed_time / 3600))
547 
+    local minutes=$(((elapsed_time % 3600) / 60))
548 
+    local seconds=$((elapsed_time % 60))
549 
+
550 
+    echo ""
551 
+    print_color "$GREEN" "=========================================="
552 
+    print_color "$GREEN" "           PROCESSING REPORT"
553 
+    print_color "$GREEN" "=========================================="
554 
+    echo ""
555 
+
556 
+    echo "Files Summary:"
557 
+    echo "  Total files found:     $TOTAL_FILES"
558 
+    echo "  Successfully processed: $PROCESSED_FILES"
559 
+    echo "  Skipped (no date):     $SKIPPED_FILES"
560 
+    echo "  Errors:                $ERROR_FILES"
561 
+    echo ""
562 
+
563 
+    echo "Size Summary:"
564 
+    echo "  Total size found:      $(format_size $TOTAL_SIZE)"
565 
+    echo "  Successfully processed: $(format_size $PROCESSED_SIZE)"
566 
+    echo ""
567 
+
568 
+    echo "Time Summary:"
569 
+    printf "  Time elapsed:          %02d:%02d:%02d\n" $hours $minutes $seconds
570 
+
571 
+    if [[ $elapsed_time -gt 0 && $PROCESSED_FILES -gt 0 ]]; then
572 
+        local files_per_second=$((PROCESSED_FILES / elapsed_time))
573 
+        local mb_per_second=$((PROCESSED_SIZE / elapsed_time / 1048576))
574 
+        echo "  Speed:                 $files_per_second files/sec, ${mb_per_second}MB/sec"
575 
+    fi
576 
+
577 
+    echo ""
578 
+
579 
+    if [[ $DRY_RUN -eq 1 ]]; then
580 
+        print_color "$YELLOW" "DRY RUN MODE - No files were actually moved/copied"
581 
+    elif [[ $KEEP_ORIGINALS -eq 1 ]]; then
582 
+        print_color "$BLUE" "COPY MODE - Original files were preserved"
583 
+    else
584 
+        print_color "$GREEN" "MOVE MODE - Files were moved to destination"
585 
+    fi
586 
+
587 
+    echo ""
588 
+    print_color "$GREEN" "=========================================="
589 
+}
590 
+
591 
+# Parse command line arguments
592 
+while [[ $# -gt 0 ]]; do
593 
+    case $1 in
594 
+        -o|--organization)
595 
+            ORGANIZATION="$2"
596 
+            if [[ ! "$ORGANIZATION" =~ ^[ymdh]$ ]]; then
597 
+                print_color "$RED" "Error: Invalid organization pattern. Must be one of: y, m, d, h"
598 
+                exit 1
599 
+            fi
600 
+            shift 2
601 
+            ;;
602 
+        --full-date)
603 
+            FORCE_FULL_DATE=1
604 
+            shift
605 
+            ;;
606 
+        --collect-unsortable)
607 
+            COLLECT_UNSORTABLE=1
608 
+            shift
609 
+            ;;
610 
+        -s|--source)
611 
+            SOURCE_PATTERNS+=("$2")
612 
+            shift 2
613 
+            ;;
614 
+        -d|--destination)
615 
+            DESTINATION="$2"
616 
+            shift 2
617 
+            ;;
618 
+        -k|--keep-originals)
619 
+            KEEP_ORIGINALS=1
620 
+            shift
621 
+            ;;
622 
+        --dry-run)
623 
+            DRY_RUN=1
624 
+            shift
625 
+            ;;
626 
+        -v|--verbose)
627 
+            VERBOSE=1
628 
+            shift
629 
+            ;;
630 
+        -h|--help)
631 
+            show_help
632 
+            exit 0
633 
+            ;;
634 
+        --version)
635 
+            show_version
636 
+            exit 0
637 
+            ;;
638 
+        *)
639 
+            print_color "$RED" "Error: Unknown option: $1"
640 
+            echo "Use -h or --help for usage information."
641 
+            exit 1
642 
+            ;;
643 
+    esac
644 
+done
645 
+
646 
+# Set default destination if not specified
647 
+if [[ -z "$DESTINATION" ]]; then
648 
+    DESTINATION="./sorted"
649 
+fi
650 
+
651 
+# If no organization is provided, default to flat full-date naming
652 
+if [[ -z "$ORGANIZATION" ]]; then
653 
+    FORCE_FULL_DATE=1
654 
+fi
655 
+
656 
+# Convert destination to absolute path
657 
+DESTINATION=$(cd "$(dirname "$DESTINATION")" 2>/dev/null && pwd)/$(basename "$DESTINATION") || DESTINATION=$(realpath "$DESTINATION" 2>/dev/null) || DESTINATION="$DESTINATION"
658 
+
659 
+# Display configuration
660 
+print_color "$GREEN" "$SCRIPT_NAME v$VERSION"
661 
+echo ""
662 
+echo "Configuration:"
663 
+echo "  Organization pattern: $ORGANIZATION"
664 
+echo "  Destination:         $DESTINATION"
665 
+echo "  Keep originals:      $([ $KEEP_ORIGINALS -eq 1 ] && echo "Yes" || echo "No")"
666 
+echo "  Dry run:             $([ $DRY_RUN -eq 1 ] && echo "Yes" || echo "No")"
667 
+echo "  Verbose:             $([ $VERBOSE -eq 1 ] && echo "Yes" || echo "No")"
668 
+
669 
+if [[ ${#SOURCE_PATTERNS[@]} -gt 0 ]]; then
670 
+    echo "  Source patterns:"
671 
+    for pattern in "${SOURCE_PATTERNS[@]}"; do
672 
+        echo "    - $pattern"
673 
+    done
674 
+else
675 
+    echo "  Source patterns:     All media files in current directory"
676 
+fi
677 
+
678 
+echo ""
679 
+
680 
+# Check dependencies
681 
+check_dependencies
682 
+
683 
+# Create destination directory if it doesn't exist (unless dry run)
684 
+if [[ $DRY_RUN -eq 0 ]]; then
685 
+    if ! mkdir -p "$DESTINATION"; then
686 
+        print_color "$RED" "Error: Cannot create destination directory: $DESTINATION"
687 
+        exit 1
688 
+    fi
689 
+fi
690 
+
691 
+# Find all source files
692 
+
693 
+print_color "$BLUE" "Scanning for media files..."
694 
+files=()
695 
+while IFS= read -r file; do
696 
+    files+=("$file")
697 
+done < <(find_source_files)
698 
+TOTAL_FILES=${#files[@]}
699 
+
700 
+if [[ $TOTAL_FILES -eq 0 ]]; then
701 
+    print_color "$YELLOW" "No media files found matching the specified patterns."
702 
+    exit 0
703 
+fi
704 
+
705 
+print_color "$BLUE" "Found $TOTAL_FILES media files to process"
706 
+echo ""
707 
+
708 
+# Process each file
709 
+
710 
+FATAL_ERROR=0
711 
+for file in "${files[@]}"; do
712 
+    if [[ -f "$file" ]]; then
713 
+        process_file "$file"
714 
+        if [[ $FATAL_ERROR -eq 1 ]]; then
715 
+            print_color "$RED" "Fatal error encountered. Stopping further processing."
716 
+            break
717 
+        fi
718 
+    fi
719 
+done
720 
+
721 
+# Show final report
722 
+show_report
723 
+
724 
+# Exit with appropriate code
725 
+if [[ $ERROR_FILES -gt 0 ]]; then
726 
+    exit 1
727 
+else
728 
+    exit 0
729 
+fi
Gitprep Version v2.6.2 build on Mojolicious
DBViewer