WhatsApp images have an invalid creation date

[harshjk]

The bug

[BUG] I got some forwarded photos and videos in Whatsapp and I see that the year of these media are more than 4 digits. So It'll not appear in the timeline. I attached photos here.

The OS that Immich Server is running on

Debian

Version of Immich Server

v1.70

Version of Immich Mobile App

v1.71

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

Same as v1.70 docker-compose.yml

Your .env content

Same as v1.70 example.env and just update the upload path

Reproduction steps

1. try to upload photos with year more than 4 digits in that like 123456

Additional information

No response

[uhthomas]

Yeah, this is a bit weird but I don't think we have any reason to support photos from the year 10,000 or beyond.

[uhthomas]

I guess thinking further we should try to understand why the year on that photo was so wrong, and maybe handle that better? Was the image uploaded from the app?

[uhthomas]

Maybe #3547?

[harshjk]

I guess thinking further we should try to understand why the year on that photo was so wrong, and maybe handle that better? Was the image uploaded from the app?

Most are forwarded photos and videos. No, It doesn't upload from the app. I used bulk upload docker CLI.

[gmag11]

Whatsapp pictures management is shit. In this case date is indicated on filename. For instance IMG-20200210-WA0000.jpg corresponds with 2020/feb/10. All metadata into file, if any, should be ignored.

[samip5]

Whatsapp seems to be scrampling the EXIF on the file, so yes, metadata should be ignored if there is any.
I think parsing the filename would be an option.

[uhthomas]

Would anyone be kind enough to show the full output of exiftool on a WhatsApp image?

[fritzip]

exiftool IMG-20230618-WA0001.jpg

ExifTool Version Number         : 12.60
File Name                       : IMG-20230618-WA0001.jpg
Directory                       : .
File Size                       : 138 kB
File Modification Date/Time     : 2023:06:18 14:52:50+02:00
File Access Date/Time           : 2023:06:18 14:52:50+02:00
File Inode Change Date/Time     : 2023:09:07 12:01:26+02:00
File Permissions                : -rw-r--r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Image Width                     : 1200
Image Height                    : 1600
Encoding Process                : Progressive DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Image Size                      : 1200x1600
Megapixels                      : 1.9

[erikvanoosten]

The same happens with photos that are shared via Signal: all exif data is gone.
My image import pipeline corrects for this by parsing the filename 🤷

[gmag11]

Just for information.

Librephotos did a hacky solution for pictures with no metadata https://docs.librephotos.com/docs/user-guide/settings/date-rules

It has a list of defined rules to extract dates from different places (standar exif tag, alternate tags, filename regex, etc). User may add some of that rules from a list and may assign different priority to each rule.

[icecore2]

The issue still there, the parsing not works properly or at all.
Librephotos solution sounds nice, let the user build own parsing methods with built-in ones.

[ach-raf]

if I edit an image manualy or using a script, to modify and fix the date for whatsapp images, how can I make immich take the new changes on considiration, I even moved the image to the correct folder structure in my immich library, but it still shows as the old date.

[moseshewetson]

I have just discovered this issue after uploading around 50,000 images to a library. Maybe 20% of them are Whatsapp images that have the dates all wrong! Not I need to somehow bulk delete them from my immich library if anyone has any ideas.

[alextran1502]

Moving to feature request

[icecore2]

I have made this script to perform the extraction and setting the date to exif:
https://github.com/icecore2/whatsapp-media-date-to-exif

[sugar012]

do you think it can be:

• automated with a cronjob
• able to scan subfolders
• overwrite the existing file?
  So that the future upload in the immich folder structure remain intact

[icecore2]

1. Yes it can
2. It is
3. This step is overwrites all existing files. This flag should be used for the first time.

[bjoli]

A batch-edit feature for file dates would solve something like 99% of the issues people describe with bad date metadata. I just looked at the file names of my local signal exports and they follow the pattern signal-YYYY-MM-DDHHmmSS.png. I could export them from signal, move them to my computer, batch fix them with exiftool and reimport them, but fixing it in the app would be a lot simpler.

Immich knows how it wants the dates stored, and it already has support for editing dates and for setting a date for multiple files at once. Letting people shoot themselves in the foot using regexps (or something simplified) would be pretty simple.

I expect to get some more computer time when my newborn gets a bit older, but I am not counting on it. When I do I can have a look at implementing it if there is any interest in having it mainstream. Even having a guesser would be simple enough.

[icecore2]

Hi @bjoli, I'd like to resolve this for you and maybe others.
Can you please upload to 'Drive' or other place a sample file?
I'll take a look into this and let you know with updates.

[bjoli]

Signal nukes the metadata. What I am asking for would be a simple batch-edit-date utility that could match against a file name and set the metadata. Say my file is called signal-2023-09-14120005.jpg i could write a match string like "signal-[YYYY]-[MM]-[DD][HH][mm][SS].jpg and have immich batch-set-the-date.

Or maybe declare named matches and set them as metadata fields.

I can probably write a proof of concept for the web UI, but that will happen first in a couple of months.