Replies: 9 comments 32 replies
-
|
If you get a chance would you mind trying the latest osxphotos release against a test library in Sequoia? I don't have the beta installed but I've updated osxphotos to use the new schema based on sample data shared by other users. None of the new Sequoia features are supported but basic osxphotos functionality should work. |
Beta Was this translation helpful? Give feedback.
-
|
Well, that sure lit up XprotectService for a hot minute, but in the end got: Verson: 5001 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. That should be enough to let me fix it for this beta at least. The schema keeps changing with the betas so I've had to make some guesses. I'm traveling and don't have access to a Mac that I can test on with the beta. |
Beta Was this translation helpful? Give feedback.
-
|
...and now a .1 beta with (some) AI. I haven't upgraded on MacOS yet, perhaps today. |
Beta Was this translation helpful? Give feedback.
-
|
https://9to5mac.com/2024/07/30/apple-intelligence-macos-external-drive/ |
Beta Was this translation helpful? Give feedback.
-
|
Sorry I should have explained this cryptic comment further. There are two issues at hand: 15.1 at this moment can't be scratch installed on an external disk, and I'm unwilling to upgrade my current Sonoma MBP to Sequoia. The second issue is that Apple has made the decision that AI will only work on data stored internally to your Mac. We've seen this before with Photos, while it still runs photoanalysisd and mediaanalysisd on libraries on external drives, they (somehow) back on delivering those results. Unknown if this is going to be permanent or if it's just a side effect of the beta. There's some idea that your AI results will be more secure if contained completely within your Mac. I think that's got some validity but oh boy this could be a blow to people with bigger libraries on external drives. And it will force me to either pony up for bigger internal drives at foolish markup or just bypass Apple's AI enhancements. Side note, clearly this decision is fine for iPhones. |
Beta Was this translation helpful? Give feedback.
-
|
Release v0.68.3 should restore functionality for Sequoia beta and macOS 14.6 however I don't have access to these to test. Please let me know if it works or if you encounter any issues. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
OK, I think this is fixed for real now in v0.68.4. A couple other users provided test libraries so I was able to actually test the code against 14.6 and 15.0/15.1 betas. Unfortunately I don't currently have access to my M1 Mac so I cannot produce the Apple Silicon binary but pipx should work. Until I upgrade my machine to Sequoia (which will be a while) I cannot work on the issues with the executable (I am convinced Apple is on a path to make it impossible to run code not downloaded via App Store...they are slowly boiling the frog). Perhaps notarizing the binary will help and I'll experiment with that eventually,. |
Beta Was this translation helpful? Give feedback.
-
|
Yep, this version works! I believe Apple will indeed make it harder and harder, but they will not completely shut the door on non-App store code. Did you get the test library results already or should I give that a whirl? |
Beta Was this translation helpful? Give feedback.
-
|
I was able to get a test library and the test suite seems to be passing with the new version. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Cool... and MacRumors has also noted the Gatekeeper "enhancements": https://www.macrumors.com/2024/08/06/macos-sequoia-gatekeeper-security-change/ |
Beta Was this translation helpful? Give feedback.
-
|
Well hot diggity, HDR in both Lightroom Classic and macOS Sequoia is making massive progress over the last couple of releases. The HDR tag - which I've never ever seen on a non-iPhone photo - also appears on my AVIF files. Well, at least for a little bit, the tag goes away for some reason. Perhaps Apple doesn't want the vast majority of pictures to have the HDR label? Dunno. Regardless, the HDR is there and now when I create a smart album of HDR photos, the exports from LrC now show up - they didn't before and it was a pain. To be clear, the entire ecosystem isn't covered yet. I tested both screensaver on my M1 MBP (with HDR display) and it shows SDR. Additionally, when viewing the results on the AppleTV via the poorly-named "Computers" app it's all SDR there now as well. But now the images at least show up, previously entire albums were ignored. Progress! I'll see if I can't edit a RAW into an HDR image in Photos. I haven't been able to but I just realized I haven't tried on this latest release. |
Beta Was this translation helpful? Give feedback.
-
|
A few things:
HDR is still a mess, and there's politics behind it at the same time. It doesn't help that definitions for things like ISO had to be based on weird absolute measurements like "18% brightness on a piece of white paper in a room with windows" and so on. ISO has been around for a long time and pre-dates much electronics, much less TV screens, pixels, image processing and so on. Same with HDR, there appears to be a faction who want absolute image fidelity down to the nit and I think that's an issue. I just want better HDR and better fidelity, more like the real world, I don't care if the absolute emitted nits are not perfect. Sorry got ranty at the end again :^) |
Beta Was this translation helpful? Give feedback.
-
|
Photos finally have support for .jxl now. It works really well. Does osxphotos have the ability to use --jpeg to convert specific file-formats to normal jpegs? E.g convert only .jxl and .avif to .jpeg for better compatibility? |
Beta Was this translation helpful? Give feedback.
-
OSXPhotos does not have a way to specify the conversion of only specific file formats. Using It would be helpful to test this by importing a JXL then select it and export it using the conversion:
and see if that works. |
Beta Was this translation helpful? Give feedback.
-
|
Just tested the command with a JXL and it works fine! Without |
Beta Was this translation helpful? Give feedback.
-
|
Some moderately interesting things to mention: In the past, I've been able to drag-and-drop my main .photolibrary bundle to external SSDs which I subsequently attach to other Mac servers as part of a 3-2-1 backup process (and also serving slideshows to Apple TVs). This still works - nobody panic! - but oddly enough the other 2 servers no longer run photoanalysisd and mediaanalysisd the way they used to. The exported libraries also have different duplicates detected and almost no photos wind up in the "featured" tab. I haven't really dug into this yet, but it would at least seem like whatever work photoanalysisd and mediaanalysisd did on my main Mac are wiped out when that library is copied and then opened on another machine... and neither analysis daemon provides new results. Temporary bug? New feature? Hard to say. To be completely clear, all the assets are still there, and it looks like the edits are there as well - but some or all of the analysis is removed. Now, Apple did say that analysis will only function on libraries stored on internal disks, but it seems a bit radical to (possibly) remove those analysis results on other machines for transferred libraries. Especially if this is the Photos backup process published by Apple. Perhaps related - got a new iPhone. Exported all my favorites from my main Mac library to the iPhone. iPhone also ignores photos that were not taken by the iPhone. No featured photos at all, no faces recognized, no trips, no nothing. After I took ~500 photos with the iPhone directly some of these started filling in, but also very predominantly for photos taken on the iPhone itself. Odd behavior. Let's see if it persists. |
Beta Was this translation helpful? Give feedback.
-
|
@cfc62 a question.
|
Beta Was this translation helpful? Give feedback.
-
|
Well, that used to work just fine. But it's clearly not working for me any longer, and I see others with similar complaints. To be clear, here's my setup - note no iCloud Photos for me. M1 Max MBP stores entire library I would detach external TB or USB SSDs from the Mac minis, take them over to the MBP and put the current .photoslibrary bundle on each external SSD and then place them back onto the minis. These two mini servers don't have photoanalysisd or medianalysisd fire up on them any more, not since Sonoma came out. Spotlight and Finder still work like crazy though, but I see no actual benefit from it yet. Only my MBP is logged into an Apple account; perhaps that has something to do with it. One the 15.2 beta becomes public I'll install that on the M1 mini and then also log into my Apple account on that machine to see if that makes any difference. My hope is that this is a bug that will be fixed at some point. My based-in-reality worry is that this is all by design, as Apple is "worried about the security of AI" and so on. Then again, if all of these AI features really don't add anything to the experience, then possibly keeping your photos externally is a benefit... After all, if I want to make an album with person X in it, I can do that easily enough on the MBP and it will carry through to the minis. The drawback is that I have to do that ahead of time, I can't do it dynamically. |
Beta Was this translation helpful? Give feedback.
-
|
A quick note: I realized the other day that I've never tried keeping my originals separate from the .photoslibrary bundle. So far I've taken the easy and likely safer route of having Photos manage everything, originals included. The whole non-indexing thing has been bugging me, so since I had already copied my originals to a SSD for use in Lightroom, I decided to try to basically start over and re-import every original onto my M1 mini, but this time not have Photos slurp up the originals and have them referenced to the external SSD instead. This is taking some time, and will likely take days to complete, but preliminary results are good with some caveats: Yes, the library seems to be indexed now, I'm getting new memories, featured photos and so on. It's really early to draw too many conclusions yet but things look good. On the negative side, it appears that there's still quite a bit of disk space usage with this approach, with 2+ renders of many photos being generated. I'm already up to 50Gb for the .photoslibrary bundle with only about 35% complete. That could mean a total photos library size of 150-175Gb, even with originals stored elsewhere. But if this holds, a fast TB4 or even TB5 disk could hold the originals while the internal SSD of an Apple Silicon Mac could hold the library and still have it indexed and all the AI stuff working as well, on a reasonably priced system. Lots to look at yet. How I did this was also pretty basic, really just copied out the "originals" part of the .photoslibrary so all the edits and so forth won't come over - but the purpose of the test is to see if this would work at all. One thing is that the RAW and JPEG pairing is lost, but I was expecting that to happen. |
Beta Was this translation helpful? Give feedback.
-
|
You could export the raw+jpeg using osxphotos then import those into the new library. I'm curious how this works. I recommend against using referenced libraries (the Apple term for what you're doing) because historically they get broken if you ever change Macs or disks (due to security-scoped bookmarks). I'm not using Sequoia yet so cant' test to see if they've fixed this. |
Beta Was this translation helpful? Give feedback.
-
|
Oh I for sure share your concerns over using referenced files! Mainly trying this to understand where Apple draws the line on AI features. I would never do this to a primary library, too much at stake at this point. I've started to run into some issues with importing, it's getting petulant about further imports even though I'm less than 40% through importing my originals. Also, it at least seems that it stopped generating new memories/featured photos so if that craps out, none of the rest of this matters. |
Beta Was this translation helpful? Give feedback.
-
|
I've always used referenced library. Up to Monterey. Main issue is the odd "Find original..." prompt it comes up when, for example, I select Edit or to play some videos; to which I reply by confirming the folder location of the file which is presented by Photos. But now as I'll need to move Library+Photos into another Mac I'm going the path of
Hence my interest on how the AI analysis works on external disk library (non-referenced) But also curious if the internal disk Library with external disk referenced photos combo would do the work. @cfc62 i have a sequoia laptop apple silicon to run some tests.
|
Beta Was this translation helpful? Give feedback.
-
|
Making progress with the import w/referenced assets. Looks like leaving it alone overnight and letting it chew on the assets for a while let it do further analysis, it had looked like it was stuck which is what happens when I move the library bundle to a new machine. To see if analysis is working, I mainly look at growth in:
and other items in Utilities like Handwriting, Receipts and so on. Of course keeping an eye on the processes running on the Mac is helpful as well, look for photoanalysisd and mediaanalysisd to be running quite a lot. I think Apple has hidden more analysis in Finder and Spotlight as well - it's not as clear-cut as it used to be in Sonoma. NOTE: I just saw that my old iMac (Intel based) has a truncated list of items under Utilities. That is, there are no Handwriting, Receipts or Duplicates on that Photos library. Perhaps it only adds those when there's a certain number of them? |
Beta Was this translation helpful? Give feedback.
-
|
Also, I'm a big fan of iStat Menus to easily see CPU, memory and disk r/w activity without having to use too much effort. I do use other utilities like Process Monitor and log but for high level "what's going on" investigation iStat Menus is perfect. |
Beta Was this translation helpful? Give feedback.
-
|
At this point, I'm calling the experiment a success. To recap, I wanted to see if analysis would take place on a library that's on the internal SSD of the Mac, but with the originals stored on an external drive. This appears to be the case. The reasons I wanted to test this were manifold but the primary interest was because my old method of making a copy of my master .photolibrary bundle was no longer working. The assets and edits would come over, but the "extras" that come with analysis were either frozen in time (e.g., not updating) or were even wiped out completely. Yes, it's debatable whether or not these features are worthwhile but my line of thought is that they will be useful at some point. Secondary to that, I wanted to see:
Plus I found some other interesting things along the way. Quick answers:
It will take some time to see how equivalent the results are as Featured Photos takes a lot of time to build, but overall results look quite similar. If this diverges in any meaningful way I'll post it.
This means even a quite large collection of referenced assets can fit into a small SSD. Hopefully this means M4 Mac purchases can remain economical and not pay the ransom Apple wants for internal SSD capacity. At the same time, care needs to be taken with referenced libraries.
I had postulated that perhaps this was a reason for analysis not working, as I was moving a library from a Mac where I am logged in to 2 other Macs where I am not - no one is logged in. It's perhaps still the case that moving a library that was built by someone else's Apple Account to a Mac without that account is a problem, I'll be able to test that in a week or so. Some observations on the rebuilding process:
All in all, not a process that I'd recommend to the average user. It takes a long time and can fail silently. But, it's certainly doable. |
Beta Was this translation helpful? Give feedback.
-
|
Great news. Even on Monterey I've noticed that in one situation Import silently fails as you said. Out of 3500 pics it failed to load 5.
Hence my question: what log do you check for Photos import errors? macOS console with some filter? Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
Back a few releases ago, Apple removed a lot of logging tools that I had relied on. The old sysadmin in me loved to watch /dev/console and stderr and all that olde-time stuff. What I didn't realize was that Apple actually went a lot further in providing tools to watch things. I feel dumb that I didn't know about "log" and that's a go-to CLI tool for me now. So However, I do like watching when I'm trying something out of the ordinary. I normally run this without root permission as I feel it's certainly verbose enough as it is, but I will be trying that going forward as well. Suggest just looking for Error to pare things down a bit. I don't know how long logs stick around but perhaps the errors for your import issue are still around. That's the main one but |
Beta Was this translation helpful? Give feedback.
-
Great info! Using:
Just FYI...
Error was:
I understood the issue was with the AAE file. I've used this code to decode the AAE file to view the contents. The diff from failed file to working file is the following: Diff Non-Working.AAE to Working.AAE2,7d1
< "versionInfo": {
< "buildNumber": "21C52",
< "appVersion": "421.0.100",
< "schemaRevision": 1,
< "platform": "OSX"
< },
9,11c3,4
< "masterWidth": 4000,
< "pipelineVersion": "OSX.3",
< "masterHeight": 3000,
---
> "masterHeight": 2052,
> "masterWidth": 1539,
14a8,13
> "versionInfo": {
> "buildNumber": "21H1320",
> "appVersion": "460.0.120",
> "schemaRevision": 0,
> "platform": "macOS"
> },
18c17
< "enabled": true,
---
> "enabled": 1,
20,26c19,30
< "height": "2052",
< "straightenAngle": "-0",
< "constraintWidth": "3000",
< "width": "1539",
< "xOrigin": "1072",
< "yOrigin": "284",
< "constraintHeight": "4000"
---
> "yaw": 0,
> "originalCrop": 0,
> "xOrigin": 949,
> "smart": 0,
> "width": 1835,
> "yOrigin": 22,
> "straightenAngle": 0,
> "auto": 0,
> "height": 2447,
> "constraintHeight": 4000,
> "constraintWidth": 3000,
> "pitch": 0
28c32,33
< "identifier": "Crop"
---
> "identifier": "Crop",
> "formatIdentifier": "com.apple.photo"Final thought: May have to go back and find all "older" Edited pics with incompatible AAE's and re-edited them! |
Beta Was this translation helpful? Give feedback.
-
|
I see others are running into this "I moved my .Photoslibrary on an external disk to a new Mac and things aren't updating" issue: https://old.reddit.com/r/ApplePhotos/comments/1gxb6df/problem_with_location_info_photos_on_my_new_mac/ for one, but I've seen others. While I suspect this is a bug, I know Apple bug fix cycle times can be very very long - multiple releases sometimes. I still have a couple of experiments to try:
The last experiment will be fodder for opening a case with Apple. To circle around, if this needs to be the method for moving a library, I'd like to maintain my edits and also simplify the importation process. It was a pain to move files into sub-sub-subdirectories to get Photos to import folders at the end of the process. I would also not want to have to keep pointing to a mass of folders one by one to import either, there's probably some middle ground here. |
Beta Was this translation helpful? Give feedback.
-
|
I promise that this line of thought and work will wind up with some I did place my originals on the aforementioned RAID 0 TB SSD pair to see If that impacted anything. Interestingly enough, this time I was able to import entire directories recursively. This either failed silently or wasn't offered (the "review files for import" never became active) when I was using the USB SSD. I do explicitly remember trying this several times with no luck - I was thinking "I can do this with Lightroom Classic, I should be able to do this with Photos." In my prior USB-based attempt I would up only being able to import a directory at a time with no subdirectories involved. I did free up some space on my M1 mini's internal SSD before starting the second attempt, so perhaps that has something to do with it. Because I was so sure I had to import directory by directory I did not try to import everything in one go, but rather one import for each So, did the RAID TB SSD device help? Well, very possibly if it indeed unlocked easier importation, but I will say the conversion of RAW into JPEG still took 20 hours plus - while it appear multi-threaded and works on multiple cores, it only works on one file at a time and is fairly slow - perhaps 1 file per second. The CPU utilization is fairly low while it works through the conversions. Nicely enough, it doesn't take breaks in this process. As before, I am getting good results after this point, with analysis taking place. Much but not all of the analysis happens right on import. I can see Faces, Memories, Trips and the like populate during importation - but the results are clearly not complete until the RAW images are done - then it seems to make another pass through the assets. At this point I do think the results are different. Face recognition is better for example. But I need to let it chew for a while, some parts of the analysis (clearly Featured Photos being one, Duplicates being another) are done over days and perhaps even weeks. I can definitely envision a scenario where running all of the originals through analysis gives you a fresh start with better algorithms and AI/ML models. Part of me wanting to do this is also figuring out what parts of the .photoslibrary are needed and which are obsolete. As I moved from iPhoto to Aperture to Photos, a lot of cruft has built up in the bundle and I wanted to either streamline that or just start over. So, time will tell. With the holiday weekend approaching, I'll just let it soak for a week or so and see what happens. Other miscellaneous notes:
|
Beta Was this translation helpful? Give feedback.
-
So this is the most change at one time I can remember in Photos for a while now, figured I'd start a thread to cover it.
Honestly, I don't know what to think yet. The presentation of photos is clearly different, and while it might wind up being better, I'm not "feeling it" at the moment. That is, it's not clearly better, it's not addressing any problem that I personally had with Photos. I didn't open it for the first time and say "yessssss" or "oh baby this is nice."
I suspect there aren't a lot of AI-based improvements yet. Apple's been clear that those are coming down the road.
There are items that are gone and supplanted elsewhere in the App (both iOS and MacOS). I for one really liked the Library view by day. It did a great job in showing you (somehow) curated photos and videos in a chronological grid that you could scroll through to your hearts content (and I would). Now each day is a separate grid, and the photos are much smaller and video seems to be downplayed quite a bit.
It would appear Apple is trying to make it easier for people to find specific photos. That's fine, but I don't really consume my content like that.
It groups by days, recent days, trip, person, memories, albums and Featured. No idea how something gets featured, but it's very picky. I am happy to see a "remove from featured album" selection but it's ineffective at the moment.
I do like that Photos - on both platforms - now gives you an idea of what it's working on and how far along it is. Finally. Plus it tells you more explicitly when it will work on some things. It told me that it would only work on XYZ (sorry forgot what the task was) when Photos was closed and it would probably take overnight. Another task told me it was 78% complete. Nice.
It also pushes slideshows, which is potentially OK but I hate the fact that it insists on playing music. There's no option for silent. I will file feedback on that for sure.
I have Sequoia installed on a external boot drive so I don't run it all the time, but happy to run experiments there.
Beta Was this translation helpful? Give feedback.
All reactions