17 days ago

Pixelmator Pro update improves ML Super Resolution and ML Denoise

Howdy! Pixelmator Pro 1.5.5 has just been released and it brings some significant improvements to ML Super Resolution and ML Denoise as well as two new Automator actions for those two features. Because seeing is believing, we thought we’d whip up a blog post with some examples to highlight the changes. Let’s start with ML Super Resolution.

ML Super Resolution

Even though we released it just about two months ago, we’ve made some pretty big improvements to ML Super Resolution in that time. In this update to it (a version 2.0, if you will) we’ve focused on four things in particular:

1. Removing compression blocks in heavily compressed images

2. Improving how portrait photos are upscaled

3. Preserving sharp edges and details in illustrations

4. Making the new, smarter algorithm faster and more efficient

Compression block removal

The previous algorithm already did a great job at removing the small compression artifactsCompression Artifacts Compression artifacts often look like small dots and appear at edge areas — where contrasting colors meet in images. that usually appear around object edges in compressed JPEG images. But JPEGs, especially ones that have been compressed quite a lot, will also feature visible compression blocksCompression Blocks JPEG compression blocks appear when the compression algorithm reduces subtly differently colored pixels (for example, in gradients) to one color. They’re most visible in photos that include the sky or other gradients.. This new version of ML Super Resolution does an even better job with the small artifacts and it takes care of the blocks!

Original Image

ML Super Resolution(Old Algorithm)

ML Super Resolution(New Algorithm)

ML Super Resolution (Old Algorithm)

ML Super Resolution (New Algorithm)

200%

Download Image

We had actually been working on this since before the original release and wanted to include it when ML Super Resolution first came out. However, adding the block removal code would cause some pretty crazy blurring issues and, honestly, we couldn’t work out why. After some digging, it became clear that certain sizes of text in our training dataset were being interpreted as compression blocks! Once we addressed that, we were good to go.

Upscaling portrait photos

How ML Super Resolution upscales extremely pixelated faces is kind of magical. We made some improvements to make sure facial features are recreated in as natural a way as possible, even from a few pixels.

Original Image

ML Super Resolution

Improved quality around edges and small details

Another thing we focused on is edge areas — especially between colors that have different hues and levels of saturation but the same lightness. These can appear in any image but are especially common in illustrations. Edges such as this weren’t quite as sharp as we would have liked and there was some slight ringing that we really wanted to get rid of, too.

Original Image

ML Super Resolution(Old Algorithm)

ML Super Resolution(New Algorithm)


Smarter and more efficient

In order to make these improvements, we retrained our algorithm and slightly increased the size of the Core ML model included in Pixelmator Pro — from 5 MB to around 7 MB. Despite being a little bigger, this version of the model is actually faster than the previous one thanks to some additional optimizations. This is also a good place for a reminder that when you use any of our machine learning features, everything is processed securely on your device by a trained model integrated using Core ML.

Algorithm improvements

50%

more intelligent1

20%

more efficient2

1. Trainable neural network parameters (more is better): 1,692,928 (new algorithm) vs. 1,185,408 (old algorithm).

2. Floating point operations per pixel (fewer is better): 903,870 (new algorithm) vs. 1,128,062 (old algorithm).

ML Denoise

It’s probably fair to say that ML Denoise, our machine learning-powered noise removal feature, has been in a kind of beta stage until today. When we create any automatic feature, one of the main goals is for it to never make any image worse. With noise removal, that’s very difficult. So we took a conservative approach and tried to avoid this. This meant that the algorithm would sometimes do nothing at all, which is a pretty frustrating experience. What’s more, sometimes some loss of quality or sharpness is worth it because there are ways to recover it. With today’s update, ML Denoise is much better at removing heavy camera noise and is a little more confident in general.

ML Denoise in action

You can see ML Denoise in action with a few example images below. Your feedback about this feature over the last few months was a great help, by the way!

Original Image

ML Denoise

200%

Original Image

ML Denoise

200%

Original Image

ML Denoise

200%

Seeing as this is a machine learning-based algorithm and requires lots of processing power, for now, this will continue to be a one-click action. Adding a slider is potentially possible but getting real-time feedback and good performance requires us to do some more research. It is still in our plans for the future, though.

Download All Sample Images

Automator actions for ML Denoise and ML Super Resolution

And to round off the update, we’ve added two new Automator actions: Increase Resolution of Images and Denoise Images, bringing the total number of Pixelmator Pro actions to 9.

  • Increase Resolution of Images

  • Denoise Images

To learn more about using Automator, check out our tutorial on the topic. Thanks to these new actions, doing awesome things like this will now be much faster and easier.

As usual, this update also includes a number of smaller improvements and fixes. It’s free to download for every existing Pixelmator Pro user and you’ve probably already received it automatically. But, just in case you haven’t, head down to the Mac App Store to make sure you’re all up to date.

Download Now

We’ll be back soon with some more news about upcoming Pixelmator Pro updates — we’ve got some new tools in the works (not ML-based this time) and some great improvements to the color picking UX that we just cannot wait to share with you. Until next time!

December 17, 2019

All about the new ML Super Resolution feature in Pixelmator Pro

It’s no secret that we’re pretty big fans of machine learning and we love thinking of new and exciting ways to use it in Pixelmator Pro. Our latest ML-powered feature is called ML Super Resolution, released in today’s update, and it makes it possible to increase the resolution of images while keeping them stunningly sharp and detailed. Yes, zooming and enhancing images like they do in all those cheesy police dramas is now a reality!

Let’s see some examples

Before we get into the nitty-gritty technical stuff, let’s get right to the point and take a look at some examples of what ML Super Resolution can do. Until now, if you had opened up the Image menu and chosen Image Size, you would’ve found three image scaling algorithms — Bilinear, Lanczos (lan-tsosh, for anyone curious), and Nearest Neighbor, so we’ll compare our new algorithm to those three.

Note that the images below are zoomed in to 200% to make the changes easier to see, but you can zoom out to 100% by clicking the magnifying glass button.

Bilinear

Lanczos

Nearest Neighbor

ML Super Resolution

200%

Download Image

Bilinear

Lanczos

Nearest Neighbor

ML Super Resolution

200%

Download Image

Bilinear

Lanczos

Nearest Neighbor

ML Super Resolution

200%

Download Image

Pretty incredible, right? Until now, if an image was too small to be used at its original resolution, either on the web or in print, there was no way to scale it up without introducing visible image defects like pixelation, blurriness, or ringing artifacts. Now, with ML Super Resolution, scaling up an image to three times its original resolution is no problem at all.

How does it all work?

As computers get ever more powerful, the additional power opens up new possibilities. One of the uses of machine learning, on a very fundamental level, is to make predictions about things. In this case, we gathered a set of images, scaled them down, and then ‘taught’ the algorithm to go from the scaled-down version to the original resolution, high-quality image, predicting the values of each new pixel. The algorithm can’t recreate detail that is too small to be visible but it can make amazing predictions about edges, shapes, contours, and patterns that traditional algorithms simply cannot.

Traditional approaches

Traditional approaches use (relatively) simple mathematics to interpolate the values of pixels when scaling images.

Nearest Neighbor

When adding new pixels, the most basic algorithm, Nearest Neighbor, simply takes the color of the closest neighboring pixel. This results in the classic blocky appearance because the previously imperceptibly small pixels are now big enough to be seen.

Bilinear

The Bilinear algorithm is a little more advanced. A texture map of the image is created according to an algorithm and the values of the 4 closest texels (texture elements) are used when recreating each pixel in the new image. The goal of this approach is to make the transition between pixels much smoother. However, when upscaling quite significantly (or upscaling small images) this algorithm creates the familiar blurry appearance.

Lanczos

Lanczos is yet more advanced, using a complicated mathematical formula to interpolate (another word for predict) the value of any newly created pixels while keeping edges as sharp as possible. Its main disadvantage is that, in its attempts to retain sharpness, the algorithm can sometimes create ringing artifacts. So, ultimately, it’s useful in certain specialized situations, but not much more.

The machine learning way

So, how does the machine learning approach work? Put simply, it takes into account the actual content of every image, attempting to recognize edges, patterns, and textures, recreating detail based on our dataset and extensive training. When upscaling, it can make much better predictions because a red pixel next to a blue pixel can be a completely different type of texture or edge in different images even though, to the primitive approaches, they’re always the same.

The ML Super Resolution convolutional neural network

To create the ML Super Resolution feature, we used a convolutional neural network. This type of deep neural network reduces raster images and their complex inter-pixel dependencies into a form that is easier to process (i.e. requires less computation) without losing important features (edges, patterns, colors, textures, gradients, and so on). The ML Super Resolution network includes 29 convolutional layers which scan the image and create an over-100-channel-deep version of it that contains a range of identified features. This is then upscaled, post-processed and turned back into a raster image. Below is a simplified representation of the neural network.

First, the input image is passed through a high pass filter for basic edge detection. Then, the first convolutional layer reduces the size of these features and pools the data. In the Descriptor Fusion block, the image is scanned to find any JPEG compression blocks within it and this is fused with the other features identified so far.

The next convolutional layers and residual blocks are where the magic happens — these detect the features (edges, patterns, colors, textures, gradients, and so on) in the image, building them up into a complex representation that is over 100 channels deep. In a convolutional neural network, more layers mean better accuracy but with a large enough number of layers, a network becomes near-impossible to train. Residual blocks are designed to increase the complexity and accuracy of networks without making them impossible to train.

Finally, all the features identified by the neural network are enlarged in the Enlarge block. After this, the two residual blocks and the final convolutional layer post-process the data and turn the features back into an image. It’s also important to note that all this happens on-device and the entire trained machine learning model is included inside the Pixelmator Pro app package.

Dealing with noise and artifacts

Small images often contain compression artifacts and noise. If we want our upscaled images to be usable, artifacts and noise shouldn’t be scaled up together with the actual contents of the image. In fact, if possible, they should be removed altogether. And, as mentioned above, ML Super Resolution is designed to do just that, borrowing some of the technologies we developed for ML Denoise to remove both camera noise and JPEG compression artifacts. By the way, in this update, ML Denoise has also been improved, bringing noise removal that is between 2 to 4 times better than before.

Nearest Neighbor

ML Super Resolution

200%

Download Image

Processing power required

Naturally, the machine learning way requires a lot more processing power than the primitive approaches — between 8 to 62 thousand times more, in fact.

Algorithm
Total Floating Point Operations (FLOPs) per pixel*
Nearest Neighbor
18 FLOPs
Bilinear
45 FLOPs
Lanczos
130 FLOPs
ML Super Resolution
1,128,062 FLOPs (1.1 megaFLOPs)

* When upscaling 1 pixel by 300%, creating 9 pixels.

Making this available in an app like Pixelmator Pro has only become possible in the last couple of years — even on Mac computers from 5 or so years ago, ML Super Resolution can take minutes to process a single image due to slower performance and less available memory. On the latest hardware, however, images are processing in a few seconds, and even faster on iMac Pro, Mac Pro, or any Mac with multiple GPUs thanks to our use of Core ML 3 and its multi-GPU support. For the same reasons, the performance of ML Super Resolution is also significantly improved when using an eGPU.

Mac
Compute time1
Compute time with eGPU2
MacBook Pro (13-inch, Mid 2012)
61.6s
N/A3
MacBook Air (13-inch, 2018)
13.7s
0.58s
MacBook Pro (16-inch, 2019)
2.4s
0.4s
iMac Pro (2017)
0.56s
0.31s

1. For this test, a 300,000 pixel image was upscaled to three times its original size.

2. Tested using an AMD Radeon RX 5700 XT eGPU.

3. External GPUs require a Thunderbolt 3-equipped Mac.

Using the 2012 MacBook Pro as a baseline, the latest devices are up to 200x faster!

We’re incredibly excited about ML Super Resolution and we honestly hope you’re going to love it too. If you’d like to, you can download all the images in this blog post using the link below and test everything in today’s update out for yourself.

Download All Sample Images

Pixelmator Pro 1.5.4 is now available from the Mac App Store, so head on down there and make sure you’re up to date. The trial version has also been updated so if you don’t yet have a copy, you’re welcome to try it out. That’s it for now, but we hope to surprise you with one more cool new feature before the year is up — stay tuned!

Note

ML Super Resolution requires macOS Mojave or later.

Download Now

November 26, 2019

Black Friday 2019 — special offers on Pixelmator apps (sale ended)

Black Friday, which usually marks the start of the winter holiday and gift-giving season, is fast approaching and we thought we’d join the fun with two fantastic offers.

Black Friday Offers

Pixelmator Pro

$39.99$29.99

Offer Ended

Pixelmator Photo

$4.99Free

Offer Ended

Starting today, you can get the amazing Pixelmator Pro for 25% off. This discount will be available for a week, until December 3rd. In addition, Pixelmator Photo, our incredible photo editor for iPad, is completely free for 24 hours until 9am ET, November 27th. Spread the word!

November 14, 2019

Pixelmator 3.9 Classic now available

Did you know the original Pixelmator turned 12 years old a couple of months ago? Crazy! The fact that lots of people still love and use the app every day is a real testament to its quality. While Pixelmator Pro is its successor (Pixelmator 2, if you will), we’re planning to support the original Pixelmator for a while longer. And today, we’ve released Pixelmator 3.9 (codenamed Classic) with macOS Catalina support, including support for Sidecar and Apple Pencil.

macOS Catalina

Sidecar

Apple Pencil

Along with some optimizations for macOS Catalina and a few fixes, we’ve also added support for Sidecar, so you can extend your Pixelmator workspace using your iPad as a second display. And with support for Apple Pencil, you can paint, retouch, and illustrate with ultimate precision!

This free update is out now on the Mac App Store, so head on down there and make sure you’re all up to date.

Download Now

October 25, 2019

CMYK proofing comes to Pixelmator Pro

Good news, everyone! We’ve just shipped a smaller Pixelmator Pro update with a big new feature called Soft Proof Colors. This one’s for all you print designers out there.

CMYK Proofing

Soft proofing lets you see what an image would look like when reproduced on a different output device — for example on a different monitor or when printed. And for most Pixelmator Pro users, the most important thing is being able to soft proof images with CMYK color profiles when working on designs that will be printed. As Pixelmator Pro already had the ability to convert to CMYK when exporting, with soft proofing, it becomes an even more powerful image editor for those of you creating designs for print.

A few days ago, we also started the Pixelmator Pro Upgrade Sale to encourage any original Pixelmator users to give Pixelmator Pro a chance and it’s been a great couple of days in many different ways! More on that later, but the sale ends on Tuesday, October 29th, so you have a few more days to take advantage of it — it really is a fantastic offer.

While we get back to working on Pixelmator Pro 2.0 (oh yes, we’ve already started and it’s going to be an amazing free upgrade), feel free to check out the full release notes for this update by visiting our What’s New page or just go ahead and view Pixelmator Pro 1.5.1 on the Mac App Store.

Download Now

October 22, 2019

Pixelmator Pro is on sale for 50% off (sale ended)

We’ve said (quite a few times now) that Pixelmator Pro is Pixelmator v2 and the future of Pixelmator. It’s not a more ‘pro-oriented’ app — it’s pro image editing for everyone.

And we think Pixelmator Pro is an incredible app that every user of the original Pixelmator would absolutely love using. That’s one of the reasons why we recently created the upgrade bundle on the Mac App Store and an upgrade page comparing the two apps. Though for those of you who bought Pixelmator on sale (and quite a few did), the discount was very small or, in certain regions, there was no discount at all.

For that reason, today, we’re starting a limited-time upgrade sale letting everyone get Pixelmator Pro for just $19.99. This is much lower than our usual sale price, so you’d be crazy not to take advantage of it!

Edit: The sale has now ended and Pixelmator Pro is back to its usual price of $39.99.

Buy Now

October 10, 2019

Pixelmator Pro 1.5 Avalon now available

Pixelmator Pro just got an awesome major update! Version 1.5 (codenamed Avalon) brings full support for macOS Catalina, the upcoming Mac Pro and Pro Display XDR, introduces intelligent, machine-learning powered noise removal, big performance improvements, and a whole lot more. Here’s a quick rundown of what we’ve added.

macOS Catalina Support

Pixelmator Pro is now fully compatible with macOS Catalina, including support for Sidecar and Apple Pencil, which means you can extend your desktop workspace using your iPad as a second Mac display! And with Apple Pencil support, you can paint, sketch, create graphic designs, and retouch your photos with pressure-sensitivity, acceleration, and tilt support as well as support for the double-tap gesture. Plus, as Pixelmator Pro fully supports Touch Bar, handy controls appear at the bottom of your iPad when using Sidecar — even if your Mac doesn’t have a Touch Bar.

Mac Pro

Thanks to its Metal-based architecture and GPU-powered editing engine, Pixelmator Pro absolutely flies on the upcoming Mac Pro. We’ve been able to optimize multi-GPU support so that with multiple graphics processors in Mac Pro, you’ll see big performance boosts in Pixelmator Pro. And performance increases even more with each additional GPU. For example, Pixelmator Pro applies the new Core ML-powered ML Denoise up to 2.5x faster and effects up to 2x faster on Mac Pro with 2 GPUs compared to iMac Pro.

EDR Mode

We’ve also added support for the upcoming Pro Display XDR. The display’s true 10-bit color depth and P3 wide color gamut are natively supported in Pixelmator Pro. And the new Extended Dynamic Range Mode lets you display clipped details in RAW photos. To do this, we use the display’s 1600-nit peak brightness to bring out details in the highlights without compressing the dynamic range of the rest of the photo, bringing a never-before-possible RAW editing and viewing experience.

ML Denoise

ML Denoise is an amazing machine learning-powered noise removal tool. Integrated via Core ML, ML Denoise effortlessly removes luminance and color noise created by cameras in low-light photos. What’s more, it can even reduce artifacts caused by image compression algorithms, improving image quality while preserving important details!

Performance Improvements

Pixelmator Pro now has an end-to-end Metal pipeline for rendering and editing and a new asynchronous zoom engine, which brings some big performance improvements. Zooming and scrolling is now at least 10x faster and always responsive, effects are up to 2.7x faster, and painting is up to 2.4x faster!

SF Symbols

SF Symbols, the new set of over 1,500 configurable vector icons designed by Apple, is fully supported in Pixelmator Pro. So you can easily open SF Symbols templates, customize them to create your own symbols, and even drag and drop symbols right into existing documents!

Download Now

This fantastic fifth major update to Pixelmator Pro is free for all existing users, so download it, try it, and let us know how we did. As you might guess, we’re already working on some more really incredible additions to Pixelmator Pro and we can’t wait to share them with you. Until next time!

October 1, 2019

First major Pixelmator Photo update out now

The Pixelmator Photo 1.1 major update is here! And it really is a big one. We’ve added full compatibility with iPadOS 13, full-featured batch editing, an all-new and improved workflow, export resizing, and some smaller improvements and fixes.

iPadOS 13

Everything seems really cool in its own right so it’s difficult to pick just one headline feature but let’s start with iPadOS. iPad OS 13 support is a very big part of this update and with our Files-based design, you can now take full advantage of support for external drives and new external locations.

Batch Editing

Batch editing is huge! We’ve brought a really full-featured batch editing experience to iPad that’s, let’s face it, even better than what we currently have on the Mac with Pixelmator Pro. A lot of that is down to Pixelmator Photo being a dedicated photo editor, of course, but it’s amazing to see all the incredible machine learning features now being available for batch editing on iPad.

All-New Workflow

The one new feature that might not be as flashy but will directly affect every single current and future Pixelmator Photo user is the all-new workflow and direct integration with your iCloud Photos library. Gone are the days of having to import photos and manage separate Pixelmator Photo files, everything is now simple and intuitive.If you’re editing in your Photos library, edits are automatically saved to the same images you open. Nondestructive edits are preserved too! And if you’re editing in Files, Pixelmator Photo does some magic to save changes back to the same image while preserving nondestructive edits in a linked file.

Export Sizes

Finally, we’ve also added the ability to export images at different sizes, which is a nice little extra that we found time to squeeze in between all the other huge things and foundational changes to Pixelmator Photo.

Download Now

We really hope you’ll enjoy the update and we can’t wait to surprise you with more awesomeness in the future. Until next time!

September 26, 2019

Announcing the Pixelmator Pro Upgrade Program

The time has come. It’s time for us to give all of you still using the original Pixelmator a friendly nudge to check out Pixelmator Pro, the successor to the image editor you know and love. And to do that, we’ve created a special Upgrade page outlining all the goodness of Pixelmator Pro.

The new page includes an extremely in-depth comparison of the two apps, some FAQs about the upgrade, and a little roadmap that even existing Pixelmator Pro users will find very interesting. Naturally, there’s an upgrade discount of up to 50% for all of you who purchased the original Pixelmator from the Mac App Store.

If you think it took us a while to get this page together, you’d definitely be right. But, to be honest, we’ve just been focusing like crazy on Pixelmator Pro itself and making the app as good as it can be. When (nearly two years ago now) we launched Pixelmator Pro, its average worldwide rating was stuck at 3.7 for about a year and that told us there was a whole bunch of things, large and small, that need to be improved. So we focused on doing just that. With lots of free updates (27 and counting) since then, we’re incredibly proud to see the app now rated 4.7 worldwide with over 4,000 ratings and over 3,300 five-star ratings. And we’re also incredibly proud of the Mac App of the Year award it received from Apple as well.

What’s more, when Pixelmator Pro launched, app bundles (and, by extension, upgrade bundles) were not yet available on the Mac App Store. So, now that we can provide an upgrade discount, we think it’s a great time to give you existing Pixelmator users a handy page to answer all the questions you might have about upgrading to Pixelmator Pro and some more information on the app itself.

Visit the new Upgrade to Pixelmator Pro page to see why so many people have already made the switch. And if you have any other questions, we’re always here to answer them either in the comments or via email at support@pixelmator.com.

September 20, 2019

Pixelmator Pro 1.5 beta out now

A few months ago, WWDC 2019 bought some really incredible Mac software and hardware announcements. A brand new, ultra-powerful Mac Pro, the stunning Pro Display XDR, and of course the awesome software news with macOS Catalina and all its new features.

Now, after several months of hard work, we’re almost ready to release a major update to Pixelmator Pro, adding support for all this. And today, we excited to share a public beta version of Pixelmator Pro 1.5 (codenamed Avalon) so you can test it out before the update hits the Mac App Store.

In our blog post after WWDC, we focused mostly on Mac Pro, Pro Display XDR, and Sidecar, but this update has a whole lot more than that. For example, there’s ML Denoise, which removes noise from photos while intelligently preserving details using, you guessed it, advanced machine learning! What’s more, ML Denoise can even remove JPEG compression artifacts. There’s also some major performance improvements, including a completely redesigned and always responsive zoom engine, painting tool improvements, and faster effect rendering. We’ve even squeezed in full support for SF Symbols!

This is going to be another awesome update and if you’d like to get an early look at it before everyone else, all you need to do is sign up for our public beta. Feedback is always welcome at beta@pixelmator.com!

Join Beta