Migrating from iCloud Drive > Pixelmator Photo > Linked Files to an external Files workflow (v1.5.2)

Discuss Photomator and photo editing.
User avatar

2021-05-24 17:18:30

Hi guys!

I want to free up some iCloud Drive space and go for an archive somewhere else.

Device: iPad Pro 12.9” 2018
OS: 14.5
App Version: 1.5.2


Here’s my original workflow:

1. Importing original RAW+JPG images to iCloud Photo (camera’s SD card) or retrieving some RAW+JPG shot iPhone images through iCloud sync (3rd party camera app)
2. Editing the RAW part within Pixelmator Photo nondestructively
3. A linked file will be generated here: iCloud Drive > Pixelmator Photo > Linked Files
4. No further export, just keeping the Pixelmator Photo edits and having the results synced on both iPad an iPhone following ‘a nondestructive but replacing approach’. Considering that there’s an original RAW + original JPG + edited RAW only the final result will be exposed which is neat.

2.083 linked files occupy 88.3GB: iCloud Drive > Pixelmator Photo > Linked Files

Now you get the point why I want to migrate.

Ok so what were the things I tried on my own?

I went to iPad’s settings > Pixelmator Photo > Store location > and switched from iCloud Drive to My iPad.
As a result, a brand new iPad-local folder Pixelmator Photo appears but no equivalent Linked Files subfolder. OK, let’s try anyway.

I captured a new RAW+JPG, imported both to iCloud Photos and applied a simple Pixelmotor Photo edit just to see if it puts the new linked file to the new Pixelmator Photo iPad-local folder. But it didn’t, the new linked file becomes only visible within the iCloud Drive location.

So I tried something different by creating a similar folder structure on my own:

My iPad > Pixelmator Photo > Linked Files

Tried again with a new RAW+JPG and Pixelmator photo edit, but again, no linked file there. It’s only within the iCloud Drive folder where I actually want to migrate from.

So what do you think guys, what does the trick?

Is it dangerous to delete these 2.083 linked files (88.3GB) from iCloud Drive > Pixelmator Photo > Linked Files?
And is it smart to proceed further with the Pixelmator Photo’s batch export function after getting rid of those 2.083 linked files to create finalized 85% JPEG quality versions or is it dangerous? How would Pixelmator Photo behave?

Or is it smarter to cut out these 2.083 linked files (88.3GB) from iCloud Drive > Pixelmator Photo > Linked Files and paste these to an external hard drive as a backup?
Meanwhile I noticed that I have no writing access to iCloud Drive > Pixelmator Photo > Linked Files so reverting to the old state for recovery purposes seems to be impossible, true story?

Last question to you guys:

If I want to keep working with Pixelmator Photo but doing so with the app’s folder-based editing feature, can I migrate these 2.083 edits as well or will I have to start off from zero once again (and having the original RAW’s+JPG’s stored elsewhere + the finalized edit as JPEG’s 85% quality files BUT NO WAY TO recreate the previous linked files relationship now to be used within the folder-based environment)? Losing the access to the edited sliders of each image would be not too bad but it would be interesting to know about it beforehand.
User avatar

2021-06-01 09:21:46

Hey there! The linked files can be migrated from iCloud to your iPad storage and vice versa via the Pixelmator Photo Editing menu. There, you should see the option "Use iCloud". That's the only correct way to migrate the edits and still have them associated with the corresponding images.
Is it dangerous to delete these 2.083 linked files (88.3GB) from iCloud Drive > Pixelmator Photo > Linked Files?
Let's first break down how Pixelmator Photo works:

1. It opens an image from the Photos library and lets you edit it.
2. When saving/modifying an image, Pixelmator Photo saves a modified JPEG in the Photos library and a linked file with your edits in your chosen location (iCloud or iPad).
3. When you reopen the same photo, Pixelmator Photo checks if there's a linked file associated with it. If there is — the image will be opened with all of the nondestructive edits. If not — the modified JPEG will be opened instead, without the nondestructive edits.
4. If you delete or move the linked file, Pixelmator Photo will no longer be able to find the edits and will only open the modified JPEG from Photos.
5. It's important to keep in mind that Photos always saves the original RAWs and you can always revert the modified images to original RAWs if need be.

Now, to answer your questions:

If you delete the linked files or move them out of the designated folders (to an external drive, for instance) — the edits will no longer be available but the Photos library will still have the original RAW+JPEG saved, plus the last modified JPEG. So even after you delete the linked files, you will still have all your edited photos available (only without the nondestructive edits).
And is it smart to proceed further with the Pixelmator Photo’s batch export function after getting rid of those 2.083 linked files to create finalized 85% JPEG quality versions or is it dangerous? How would Pixelmator Photo behave?
In such a case, Pixelmator Photo would take the modified JPEG from Photos and create a file in the format you choose (JPEG, HEIF, etc.). If you're looking to export images at the best possible quality, though, it's best to do it before deleting the linked files. Linked files still have nondestructive data which you can use to create a lossless image, like TIFF, for instance.
User avatar

2022-03-01 12:53:05

Can I ask, why there is a linked file created anyway?
Like why does an actual photo need to be saved in that folder. It basically duplicates the images, which is a bit overshot I think.
If Pixelmator photo would only save text file that basically tells the app what edits where applied, wouldn’t that be 1. much faster to sync and 2. much more Storage efficient?
User avatar

2022-03-01 20:29:24

I agree, although I can see the process involved. Many DAMs that operate a non-destructive system do create a set of various images alongside the original. For example Lightroom can produce anything from nothing, to a basic set of thumbnails and a small Preview, right up to a full res high quality Preview, as well as their 'Smart' Previews.

In most cases some kind of metadata sidecar file, or embedded data file (in the case of a DNG usually) is necessary to hold any editing data.

The additional image files are needed to display the edits on software outside the source editor (whatever it is) as it's normal for such data to be proprietary to that software. So any editing applied within Pixelmator, for example, cannot be transferred directly to another viewer, whether it's Photos or Lightroom, or anything else, as each uses its own parameters for editing data, and do not interpret Raw data in the same way (in some rare cases, such as with darktable or RawTherapee, it's possible to reverse engineer this, but in practice, it doesn't work very well).

However, I agree that Pixelmator is very wasteful of storage space in this respect, as it seems to store data in a fully uncompressed format, so even edits from a small JPEG can result in a huge sidecar file, some Raws can be ridiculously large (some of my NEF 24MP raws start off at 27MB, and the Pixelmator sidecar file can be nearly 200MB). Yet other editors can manage at least a reasonable size of sidecar data and previews.

But, as Pixelmator seems to be the best option for working alongside Photos on an iPad, I too find the above workaround is the best way for now. So I carry out all my edits, usually working on a folder/album of images, ensure I've finished editing, then I simply delete all the Pixelmator sidecar files on my iCloud Drive.

For me, I don't need an uncompressed version to work from, and if I need to change any edits, I'd just revert to original and start again. I'm not a pro, just a hobbyist, to this is sufficient for my needs here.

I would like to see at least a user option to somehow control the sidecar data being stored though, and maybe choose to only keep basic editing data, or a set of full size previews (something like Lightroom does, where these can be created on demand as needed, rather that using up large amounts of storage permanently).

Just beware though, that this issue isn't uncommon, many others also use this system , such as Affinity Photo and Skylum Luminar, all create a large sidecar editing file if you sue the non-destructive options.

If integrating with Photos, the only one I know of that uses sensible data usage is Raw Power (by Gentlemen Coders), which embeds its data inside the Photos library, and uses negligible storage space. However, that's also less well featured than Pixelmator (although an excellent basic image editor).

Cheers

Andy
User avatar

2022-03-01 21:17:51

by Aurelija 2021-06-01 07:21:46 Hey there! The linked files can be migrated from iCloud to your iPad storage and vice versa via the Pixelmator Photo Editing menu. There, you should see the option "Use iCloud". That's the only correct way to migrate the edits and still have them associated with the corresponding images.



Let's first break down how Pixelmator Photo works:

1. It opens an image from the Photos library and lets you edit it.
2. When saving/modifying an image, Pixelmator Photo saves a modified JPEG in the Photos library and a linked file with your edits in your chosen location (iCloud or iPad).
3. When you reopen the same photo, Pixelmator Photo checks if there's a linked file associated with it. If there is — the image will be opened with all of the nondestructive edits. If not — the modified JPEG will be opened instead, without the nondestructive edits.
4. If you delete or move the linked file, Pixelmator Photo will no longer be able to find the edits and will only open the modified JPEG from Photos.
5. It's important to keep in mind that Photos always saves the original RAWs and you can always revert the modified images to original RAWs if need be.

Now, to answer your questions:

If you delete the linked files or move them out of the designated folders (to an external drive, for instance) — the edits will no longer be available but the Photos library will still have the original RAW+JPEG saved, plus the last modified JPEG. So even after you delete the linked files, you will still have all your edited photos available (only without the nondestructive edits).



In such a case, Pixelmator Photo would take the modified JPEG from Photos and create a file in the format you choose (JPEG, HEIF, etc.). If you're looking to export images at the best possible quality, though, it's best to do it before deleting the linked files. Linked files still have nondestructive data which you can use to create a lossless image, like TIFF, for instance.
Can you talk to your team about what I and Andy just mentioned? I am sure there is room for improvement :)
User avatar

2022-03-08 05:43:27

I've had a look inside one of the Pixelmator Photo edit files, and it does seem to be full of unnecessary cruft.

I started with a 10MB Raw file.

The 'edit' file for Pixelmator contains what seems to be a bunch of re-encoded files, so that nothing else can open them. One looks like a copy of the original Raw, judging by the naming, and the size, which is identical to the original, so already storage requirements have doubled.

Then there's some more that seem to be about the editing workflow. One appears to be a huge TIIFF type file, with the word 'bitmap' in the title, that's about 100MB (which would be about right for a TIFF.

Then there's the 'backup' file, at about 28MB.

Finally a few data files of a few KB each.

Now looking into the Photos library, it seems it does use standard xml/xmp sidecar files for editing data for each image. I suspect Photos and Raw Power uses these for their editing data.

As Photos already keeps a copy of the original, and never overwrites it, why not use the integrated system, rather than such a wasteful proprietary system. If I edited all my images nondestructively with Pixelmator, I would need 15x more storage capacity (not even possible with an iCloud account, or locally on an i-Device).

As it is, with a large library (for a hobby photographer), the increase in storage requirements just means I can not make continued use of non-destructive editing in Pixelmator, and have to delete the edit files as I go about editing my images.

Cheers.
User avatar

2022-03-08 15:58:07

Can the Pixelmator Team look into this? I think there plenty of room for improvement about this :)
User avatar

2022-03-10 14:07:08

Thanks for the feedback everyone! Passing on the word from the devs:

"We need all the data for all the non-destructive features we provide. It’s Repair, Super Resolution, and Denoise that require an especially big amount of space. However, we are aware of the fact that our sidecar files can sometimes take a lot of disk space and we are constantly looking for ways to make them smaller."
User avatar

2022-03-10 16:59:56

Thank you for the efforts Aurelija, that is appreciated, although of course it doesn't solve anything, it does help to have some feedback for sure.

I understand that of course, but there must be ways around it, as not all editors have the same huge sidecar files. Of course I know many do also have very large sidecar files, so I understand it must be a tricky thing to get right. However, it must be possible to minimise the amount of data somewhere.

I know from using Lightroom in the past they do everything in xml data, and then the user can choose what level of Preview image is used, an entire Catalogue for Lightroom for me was about 100GB for a 600GB library. I think they relied more on using Previews to display the data, and re-render the images from the data if needed. This does slow things down a little (not that I ever really noticed to be fair), but does make the sidecar data more manageable.

Indeed, if I decide to save as an uncompressed TIFF, or PSD, they ended up at 2-300MB each too, but it was all down to user choice.

I realise we're not paying the premium cost commanded by the features supplied by Adobe software, so if this is something that simply requires ridiculous dev resources, then of course that's understandable too. In which case I'm just happy to continue as I am, and use the non-destructive files as a temporary working solution in my workflow.

Cheers.
User avatar

2022-05-28 02:38:28

I am having a similar problem, but slightly different. I preserve many edits using the iCloud option and aren’t concerned about storage. However, I am having a problem with my iPad’s System Data growing exponentially. On a 512Gb iPad, it’s grown from about 8Gb to now be at about 130Gb. The major app I use is Pixelmator. Could this be the reason for the System Data being so large?
User avatar

2022-06-30 13:04:46

Hi David! I realize I'm a bit late to the party but for what it's worth, that doesn't sound like a Pixelmator Photo issue since your edits are stored on iCloud, not your device, and more importantly, Pixelmator Photo files wouldn't be categorized as System Data anyway. I'd suggest getting in touch with Apple Support if you're still dealing with this issue.
User avatar

2023-04-08 20:06:12

by Aurelija Thanks for the feedback everyone! Passing on the word from the devs:

"We need all the data for all the non-destructive features we provide. It’s Repair, Super Resolution, and Denoise that require an especially big amount of space. However, we are aware of the fact that our sidecar files can sometimes take a lot of disk space and we are constantly looking for ways to make them smaller."
Thanks for noting this. These don't exactly seem like sidecar files. They're just full blown copies of the photos. I recently discovered a large chunk of my iCloud Drive space being taken up by the Pixelmator Photo Linked Files. This kind of gunks up my iCloud storage considering I'm already syncing the photos themselves, not to mention requires a fair amount of electricity to constantly sync this data between my devices and some server in Cali. I would love to put my 2 cents in for a solution that actually stores "sidecar" files, even if there were a few performance compromises.
User avatar

2023-09-22 19:08:25

Same issue, but with Photomator. For me, the problem is moving these big files off of iCloud into the local storage.

I edited a batch of photos and there's now 11GB in my iCloud -> Photomator -> linked files folder.

I switched Photomator to use local storage in the Pictures folder instead. It didn't move any of the files by itself. So after restarting the program a couple of times with no better luck, I moved all the linked files into the local pictures folder - but Photomator can't see them. It flattens all the images and can't adjust the edits. Layers have disappeared.

I didn't realize Photomator would eat up all that iCloud storage with giant files. And that's fine, but now I want to move it local. But it won't let me!
User avatar

2023-09-22 19:11:46

Follw up: As a test, I tried opening one of the moved "linked files" (.photo-edit file) into Photomator directly. It... makes another copy of itself, and loses the JPEG to create a RAW-only version in Photos. I can edit again but it's now wasting double the space.

Hmmmm
User avatar

2023-10-24 02:29:45

One more test today. Photomator on MacOS does NOT have the option to move the sidecar files off of iCloud. Very confusing. And my new iCloud subscription is already full, which is very frustrating.

I tried to be clever - I moved all the Sidecar files into user->pictures->Linked files and then went into my iCloud folder and replaced the "Linked Files" folder there with an alias that points to my local pictures->linked files, where the local Photomator files are stored.

Guess what! Photomator does NOT find the sidecar files there. Somehow it doesn't believe it should look through the alias link into the target folder.

So if I ever want to go back and edit these files I moved off iCloud: I have to open the pic and see if the non-destrcutive edits show up in Photomator. If they don't. I have to close the app, find the associated sidecar file in my local drive. Drag it into the proper iCloud folder. THEN re-open Photomator and start editing. When I'm done, I have to move the files off iCloud again.

I wish there was a less clunky way to archive projects. I will fill up my MacBook Pro hard drive in a while.