TIP Organizing digital photos
From Gentoo Linux Wiki
Contents |
[edit] Intro
From one obsessive compulsive pack rat to another, here's a little trick / convention that I use to organize / manage my digital photos:
First I just copy them to the hard drive from my camera or download them from my e-mail. Then put them in folders. Then on occasion I run a script that moves all of the photos into a permanent chronologically ordered directory but leaves symlinks behind for me to muck around with. On top of that I use a web-based program to share them with family and friends.
So for this project we'll use mainly three tools - picdir.sh, Gallery2, and RenRot
[edit] picdir.sh
This script will by default sift through all files in your home directory, move any JPEG / MP4 / AVI / 3GP files into a special directory which is ordered chronologically while leaving symlinks in place of the originals and also deletes any duplicates (but again leaves symlinks).
The script supports images and videos from my mobile phone (also older Nokia phones which store the date and time in comments). AVI video files from digital cameras can be sorted according to the video creation time (I didn't find that AVI files have the creation timestamp included in the header). The script also asks user to input a date and time for images which don't have timestamps and optionally to create EXIF header and save the information there.
| Code: Get what you need |
emerge jhead mpeg4ip exiftool
mkdir ${HOME}/bin
nano -w ${HOME}/bin/picdir.sh # insert script below
sudo ln -s ${HOME}/bin/picdir.sh /usr/local/bin/
picdir.sh /media/HP733_CARD/ /home/USER/Pictures/tmp/
|
| File: picdir.sh |
#!/bin/bash
# Picdir v0.9 rc2
# Copyleft 2005 Gentoo-Wiki.com - This software is OpenSource
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# Guaranteed to be my personal best!
# ...not guaranteed to work...
# What will this do?
# Search for timestamps in all JPEG, 3GP, MP4 and AVI files within a directory
# Use the timestamp to relocate and rename the file
# Using checksums, determine and remove duplicate files
# Uniquely name files which have the same timestamp but different data
# needs BASH FIND GREP CUT FILE DATE JHEAD MP4DUMP MD5SUM MV SED WC EXIFTOOL
TIMEZONE="0"
OPERATION="mv"
if [ ! -n "${1}" ]; then
echo "USAGE: ${0} [OPTIONS] SOURCE [DESTINATION]"
echo "Try ${0} -h for help"
exit
fi
while [ -n "${1}" ]; do
case $1 in
-h | --help)
echo "Organizes pictures and videos with EXIF/MP4/3GP/AVI timestamps"
echo ""
echo "USAGE: ${0} [OPTIONS] /my/pictures/unsorted/ [/my/pictures/sorted/]"
echo ""
echo "Optional arguments:"
echo " -m, --move (default) moves files to destination"
echo " -L, --no-ln just move, don't leave symlinks"
echo " -c, --copy copies files to destination instead of moving (implies -L)"
echo " -h, --help shows this help"
echo " -e, --exif creates EXIF date for images which don't have it"
echo " -q, --quiet be quiet and don't bother the user"
echo ""
echo "If no DESTINATION directory is given, files will be moved/copied"
echo "${HOME}/.Pictures/"
echo "(Note that it's a hidden directory)"
echo ""
echo "EXAMPLE: ${0} -c ~/Desktop/pics/unsorted/ ~/Pictures/chrono/"
echo ""
exit 1;;
-c | --copy)
COPYOPTION=${1}
OPERATION="cp -p"
LINKOPTION="no"
shift 1;;
-m | --move)
shift 1;;
-L | --no-ln)
LINKOPTION="no"
shift 1;;
-e | --exif)
EXIFOPTION=${1}
shift 1;;
-q | --quiet)
QUIETOPTION=${1}
shift 1;;
-*)
echo "Error: no such option $1. -h for help"
exit 1;;
*)
break;;
esac
done
if [ ! -n "${2}" ]; then
mkdir -p "${HOME}/.Pictures/"
MOVETO="${HOME}/.Pictures/"
else
# Add trailing "/" character if it is missing
LASTCHAR=$(echo -n "${2}" | cut -b $(echo -n "${2}" | wc -c))
if [ "$LASTCHAR" != "/" ]; then
MOVETO="${2}/"
else
MOVETO="${2}"
fi
fi
LOGDIR="${MOVETO}"
# Call self recursively
if [ ! "${3}" = "RECUR" ]; then
echo "Finding and organizing all pictures with EXIF/MP4/3GP/AVI timestamps..."
find "${1}" -type f -exec "${0}" ${COPYOPTION} ${EXIFOPTION} ${QUIETOPTION} {} "${2}" "RECUR" \;
echo "Done!"
else
FILE="${1}"
IS_JPEG=$(file "${FILE}" | grep 'JPEG')
if [ -n "${IS_JPEG}" ]; then
SUFFIX=jpg
TIMESTAMP=`jhead "${FILE}" 2>/dev/null | grep 'Date/Time' | cut -d':' -f2-6`
if [ ! -n "${TIMESTAMP}" ]; then
# Older Nokia phones (3650 6600 etc.) have the timestamp in the comment - dig it out.
COMMENTDATE=$(jhead "${FILE}" 2>/dev/null | grep '^Comment' | grep '[0-9]\{2\}/[0-9]\{2\}/[0-9]\{4\}' | sed 's/Comment : //g')
COMMENTTIME=$(jhead "${FILE}" 2>/dev/null | grep '^Comment' | grep '[0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}' | sed 's/Comment : //g')
if [ -n "${COMMENTDATE}" -a -n "${COMMENTTIME}" ]; then
COMMENTYEAR=`echo ${COMMENTDATE} | cut -d'/' -f3`
COMMENTMONTH=`echo ${COMMENTDATE} | cut -d'/' -f2`
COMMENTDAY=`echo ${COMMENTDATE} | cut -d'/' -f1`
TIMESTAMP="$COMMENTYEAR:$COMMENTMONTH:$COMMENTDAY $COMMENTTIME"
else
# Get initial timestamp from file modification time and ask if it's ok
TIMESTAMP=$(ls -l --full-time "${FILE}" | cut -d ' ' -f6-7 | cut -d '.' -f1 | sed 's/-/:/g')
while [ ! -n "$VALID" ]; do
if [ ! -n "${QUIETOPTION}" ]; then
echo -n "Enter new date and time for ${FILE} or press ENTER to accept [$TIMESTAMP]:"
read READDATE READTIME
if [ ! -n $READDATE -o -n $READTIME ]; then
TIMESTAMP="$READDATE $READTIME"
fi
fi
VALID=$(echo $TIMESTAMP | grep '[1-2][0-9][0-9][0-9]:[0-1][0-9]:[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]')
done
fi
fi
fi
# MP4/3GP files have timestamp but it is stored as seconds from 1904-01-01 00:00:00.
IS_MP4=$(file "${FILE}" | grep 'MPEG v4')
if [ -n "${IS_MP4}" ]; then
SUFFIX=mp4
IS_3GP=$(file "${FILE}" | grep '3GPP')
if [ -n "${IS_3GP}" ]; then
SUFFIX=3gp
fi
TIMESTAMP=$(date -d "1970-01-01 00:00:00 $(($(date -d "1904-01-01 00:00:00" +%s) + $(mp4dump "${FILE}" | grep creationTime | cut -d '=' -f2 | cut -d ' ' -f2 | head -n1))) sec + $TIMEZONE hours" "+ %Y:%m:%d %H:%M:%S")
if [ ! -n "${TIMESTAMP}" ]; then
# '${FILE}' has no creation timestamp!"
exit
fi
fi
# AVI doesn't have timestamp so trust the creation time.
IS_AVI=$(file "${FILE}" | grep 'AVI')
if [ -n "${IS_AVI}" ]; then
SUFFIX=avi
TIMESTAMP=$(ls -l --full-time "${FILE}" | cut -d ' ' -f6-7 | cut -d '.' -f1 | sed 's/-/:/g')
fi
if [ -n "${TIMESTAMP}" ]; then
# Deciding path
DATE=`echo ${TIMESTAMP} | cut -d' ' -f1`
YEAR=`echo ${DATE} | cut -d':' -f1`
MONTH=`echo ${DATE} | cut -d':' -f2`
DAY=`echo ${DATE} | cut -d':' -f3`
MOVETO="${MOVETO}${YEAR}/${MONTH}/" #Directory Structure
mkdir -p ${MOVETO}
# Deciding filename
TIME=`echo ${TIMESTAMP} | cut -d' ' -f2`
HOUR=`echo ${TIME} | cut -d':' -f1`
MINUTE=`echo ${TIME} | cut -d':' -f2`
SECOND=`echo ${TIME} | cut -d':' -f3`
NEWFILE="${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.$SUFFIX"
# Complete path
ABSPATH="${MOVETO}${NEWFILE}"
# Complete path
if [ -f "${ABSPATH}" ]; then
# 1) File exists with that name"
if [ ! "${ABSPATH}" = "${FILE}" ]; then
# 2) The file isn't itself ... yeah ... that makes sense"
SUM0=`/usr/bin/md5sum "${FILE}" | cut -d' ' -f1`
SUM1=`/usr/bin/md5sum "${ABSPATH}" | cut -d' ' -f1`
if [ ! "${SUM0}" = "${SUM1}" ]; then
# 3) It isn't a duplicate of the same picture"
i=0
ABSPATH_1="d${i}-${ABSPATH}"
ABSPATH_1="${MOVETO}d${i}-${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.${SUFFIX}"
until [ ! -f ${ABSPATH_1} ]; do
(( i++ ))
ABSPATH_1="${MOVETO}d${i}-${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}${SECOND}.${SUFFIX}"
done
# Giving it a unique name 'd#-FILE'"
if [ ! -n "${QUIETOPTION}" ]; then
echo "Appending 'd${i}-' to ${ABSPATH}: file exists."
fi
ABSPATH="${ABSPATH_1}"
# 3) non-duplicate of same name was renamed"
else
if [ ! -n "${QUIETOPTION}" ]; then
echo "Checksum match: Duplicate file overwritten"
fi
fi
fi
# 2) if the file is itself, pass-along, mv/cp will handle it"
fi
# 1) All that settled, should be safe to move/copy the file
if [ ! "${FILE}" = "${ABSPATH}" ]; then
if [ ! -n "${QUIETOPTION}" ]; then
if [ "$OPERATION" == "mv" ]; then
echo "Moving ${FILE} to ${ABSPATH}"
else
echo "Copying ${FILE} to ${ABSPATH}"
fi
if [ ! -n "$LINKOPTION" ]; then
echo "Creating symbolic link"
fi
fi
$OPERATION -f "${FILE}" ${ABSPATH}
if [ ! -n "${LINKOPTION}" ]; then
ln -s ${ABSPATH} "${FILE}"
fi
echo "${ABSPATH}:::::${FILE}" > ${LOGDIR}catalog.log
# Create Date/Time EXIF tag for image
if [ -n "${EXIFOPTION}" ]; then
exiftool -overwrite_original "-dateTimeOriginal=$TIMESTAMP" "${ABSPATH}" >/dev/null
fi
fi
fi
# 0) Not supported file."
fi
|
gphoto2 might have some tools that could further enhance this? Anyone with know-how, please report.
[edit] Gallery2
- Download Gallery2
- Follow the instructions
- Create an upload directory in the g2data folder
- As admin, specify that folder as the upload folder
- Link that folder to your ${HOME}/pictures/tmp
- Use the local server upload page when uploading pictures
- Delete the pictures from ${HOME}/pictures/tmp
- Link the g2data/${USER} folder to your ${HOME}/pictures/public
- Keep your non-public pictures in ${HOME}/pictures/private
[edit] Convention
- GIMP your photos before you upload them. Although you can do this afterwards, it mean that you have to recreate thumbnails and such as well.
I upload all of my photos via gallery2 and then organize in albums in chronological order. These are my 'permalinks', if you will. Most of my pictures I don't actually rename from 200506261265.jpg to meaningfull names, like Lake_Placid.jpg - I use the 'Title' feature in gallery2, but for the sake of illistration, I'll pretend that I do.
username/life/chrono/2005/06/Lake_Placid.jpg username/life/chrono/2005/06/John_Bytheway.jpg username/life/chrono/2005/06/Painting.jpg
Additionally, I have a misc and tech album for things that most people wouldn't be interested in, but are important to me.
username/tech/wallpaper/gentoo_1024x768.png username/tech/screenshots/gaim_fullscreen.png username/misc/road_signs/dead-end_natures-way.jpg username/misc/bumper_stickers/MEN_we_re_just_better.jpg
But that's not really great for keeping track of recent events, or currrent friends, etc - just for permalinks. So as temporary albums (people, places, things, events - basically) that I leave up for a few months and then delete, I use the 'create link' functionality of gallery2. So these would actually be symbolic links:
username/life/recent/bbq_at_johns/MEN_we_re_just_better.jpg username/life/recent/bbq_at_johns/John_Bytheway.jpg
username/life/recent/vacation_vt/dead-end_natures-way.jpg
username/life/friends/john_bytheway/John_Bytheway.jpg
Just thought I'd share that. Hope you find it useful.
[edit] RenRot
RenRot is a program to rename and lossless rotate (for now only JPEG format) files according to their EXIF tags values.
RenRot is intended to work with files of --extension extension containing EXIF date and can do two things with them - rename and rotate. It runs in batch mode in current or set with --work-directory directory as well as selective mode for separate files given as arguments in command line.
Template ideology was implemented. It includes flexibility in file name construction. The template can contain different data, from direct name to EXIF data (the date, id or shooting details such as WhiteBalance, ISO, e.t.c.). For further information, please, see applied manual.
RenRot rotates file and its thumbnail, according to EXIF tag Orientation. If the tag is absent or miss set, than the script allows to rotate the file as well as it's thumbnail "by hands".
Furthermore, the script put a comment to: - Comment tag from comment file; - UserComment tag from configuration variable or command line option.
Personal data could be specified via XMP tags defined in configuration file.
In addition RenRot can aggregate all files in directories according the given date/time pattern template, set with --aggr-template.
| Code: rename each file according to the given template |
renrot --name-template="01.%c.%Y%m%d%H%M%S.%E%F%W%I" --extension JPG |
| Code: rename each file according to the given template and aggregate according the date |
renrot --name-template="%y%m%d%H%M%S.%i" --aggr-mode="template" --aggr-template="%Y%m%d" *.JPG |
| Code: aggregate files by yymmddHHMM |
renrot --aggr-mode="template" --aggr-template="%y%m%d%H%M" --extension jpg |
| Code: rotate each file and their thumbnail by 90CW in specified directory |
renrot --rotate-angle 90 --work-directory="/tmp/images" --extension jpg |
| Code: rotate thumbnails, included to EXIF, for each file by 270CW (same as 90CCW) |
renrot --rotate-thumb 270 --extension jpg |
| Code: rotate given files by Orientation tag (no real rotation will be done) |
renrot --no-rename --mtime --rotate-angle=90 --only-orientation *.JPEG |
| Code: fix file mtime according to its EXIF tags or current time stamp, when tags are invalid |
renrot --no-rotate --no-rename --mtime --extension jpeg |
| Code: leave mtime untouched for couple of files |
renrot --no-mtime *.jpg |
[edit] See Also
CoolAJ86's Review of Gallery2 In version 0.16 and later the previous behaviour still present with default NameTemplate.
