February 11, 2020

Pixelmator Pro update improves ML Super Resolution and ML Denoise

Howdy! Pixelmator Pro 1.5.5 has just been released and it brings some significant improvements to ML Super Resolution and ML Denoise as well as two new Automator actions for those two features. Because seeing is believing, we thought we’d whip up a blog post with some examples to highlight the changes. Let’s start with ML Super Resolution.

ML Super Resolution

Even though we released it just about two months ago, we’ve made some pretty big improvements to ML Super Resolution in that time. In this update to it (a version 2.0, if you will) we’ve focused on four things in particular:

1. Removing compression blocks in heavily compressed images

2. Improving how portrait photos are upscaled

3. Preserving sharp edges and details in illustrations

4. Making the new, smarter algorithm faster and more efficient

Compression block removal

The previous algorithm already did a great job at removing the small compression artifactsCompression Artifacts Compression artifacts often look like small dots and appear at edge areas — where contrasting colors meet in images. that usually appear around object edges in compressed JPEG images. But JPEGs, especially ones that have been compressed quite a lot, will also feature visible compression blocksCompression Blocks JPEG compression blocks appear when the compression algorithm reduces subtly differently colored pixels (for example, in gradients) to one color. They’re most visible in photos that include the sky or other gradients.. This new version of ML Super Resolution does an even better job with the small artifacts and it takes care of the blocks!

Original Image

ML Super Resolution(Old Algorithm)

ML Super Resolution(New Algorithm)

ML Super Resolution (Old Algorithm)

ML Super Resolution (New Algorithm)

200%

Download Image

We had actually been working on this since before the original release and wanted to include it when ML Super Resolution first came out. However, adding the block removal code would cause some pretty crazy blurring issues and, honestly, we couldn’t work out why. After some digging, it became clear that certain sizes of text in our training dataset were being interpreted as compression blocks! Once we addressed that, we were good to go.

Upscaling portrait photos

How ML Super Resolution upscales extremely pixelated faces is kind of magical. We made some improvements to make sure facial features are recreated in as natural a way as possible, even from a few pixels.

Original Image

ML Super Resolution

Improved quality around edges and small details

Another thing we focused on is edge areas — especially between colors that have different hues and levels of saturation but the same lightness. These can appear in any image but are especially common in illustrations. Edges such as this weren’t quite as sharp as we would have liked and there was some slight ringing that we really wanted to get rid of, too.

Original Image

ML Super Resolution(Old Algorithm)

ML Super Resolution(New Algorithm)


Smarter and more efficient

In order to make these improvements, we retrained our algorithm and slightly increased the size of the Core ML model included in Pixelmator Pro — from 5 MB to around 7 MB. Despite being a little bigger, this version of the model is actually faster than the previous one thanks to some additional optimizations. This is also a good place for a reminder that when you use any of our machine learning features, everything is processed securely on your device by a trained model integrated using Core ML.

Algorithm improvements

50%

more intelligent1

20%

more efficient2

1. Trainable neural network parameters (more is better): 1,692,928 (new algorithm) vs. 1,185,408 (old algorithm).

2. Floating point operations per pixel (fewer is better): 903,870 (new algorithm) vs. 1,128,062 (old algorithm).

ML Denoise

It’s probably fair to say that ML Denoise, our machine learning-powered noise removal feature, has been in a kind of beta stage until today. When we create any automatic feature, one of the main goals is for it to never make any image worse. With noise removal, that’s very difficult. So we took a conservative approach and tried to avoid this. This meant that the algorithm would sometimes do nothing at all, which is a pretty frustrating experience. What’s more, sometimes some loss of quality or sharpness is worth it because there are ways to recover it. With today’s update, ML Denoise is much better at removing heavy camera noise and is a little more confident in general.

ML Denoise in action

You can see ML Denoise in action with a few example images below. Your feedback about this feature over the last few months was a great help, by the way!

Original Image

ML Denoise

200%

Original Image

ML Denoise

200%

Original Image

ML Denoise

200%

Seeing as this is a machine learning-based algorithm and requires lots of processing power, for now, this will continue to be a one-click action. Adding a slider is potentially possible but getting real-time feedback and good performance requires us to do some more research. It is still in our plans for the future, though.

Download All Sample Images

Automator actions for ML Denoise and ML Super Resolution

And to round off the update, we’ve added two new Automator actions: Increase Resolution of Images and Denoise Images, bringing the total number of Pixelmator Pro actions to 9.

  • Increase Resolution of Images

  • Denoise Images

To learn more about using Automator, check out our tutorial on the topic. Thanks to these new actions, doing awesome things like this will now be much faster and easier.

As usual, this update also includes a number of smaller improvements and fixes. It’s free to download for every existing Pixelmator Pro user and you’ve probably already received it automatically. But, just in case you haven’t, head down to the Mac App Store to make sure you’re all up to date.

Download Now

We’ll be back soon with some more news about upcoming Pixelmator Pro updates — we’ve got some new tools in the works (not ML-based this time) and some great improvements to the color picking UX that we just cannot wait to share with you. Until next time!