Add PhotoSauce.NativeCodecs.Mozjpeg

[georg-jung]

Thanks for developing this great image processing pipeline!

I'm interested in compressing jpegs with a focus on storage efficiency. Thus, I created a simple .Net MozJpeg wrapper library based on pre-existing libjpeg-turbo wrappers. In the recent weeks I tried to get a deeper understanding of state-of-the-art .Net image processing and started to read parts of ImageSharp's and PhotoSauce's source code. I think it's fascinating what you and the others did both in terms of performance and thought that went into the API interface. I don't think I could neither would it make sense to compete in the context of my MozJpeg wrapper. Providing a rather thin wrapper around MozJpeg probably has its use-cases in the .Net world - from my application developer's perspective it would probably be nicer to use a more easy and complete image processing pipeline like PhotoSauce (with support for metadata copying, resizing, ...) and have MozJpeg as encoder at the end of it, though.

I've seen there already is support for libjpeg-turbo. Thus, I thought it might be not too hard to provide a packaged version of MozJpeg as an alternative that focuses more on storage efficiency than on CPU time efficiency. What do you think?

I evaluated providing a PR, but I don't think I'd get that right, given I'm not that experienced with native interop. Also I wasn't sure what this

Standard releases of libjpeg and libjpeg-turbo are not usable reliably from .NET; a custom build is required.

means.

Mozjpeg is already supported in vcpkg.

Edit: afaik MozJpeg does provide a switch to disable its size for performance trade. Thus it might even be possible to switch the backend and use that switch. I'm not sure however how accessible that switch is using the libjpeg API and if that'd be the better option overall.

[saucecontrol]

Howdy! Thanks for logging this. I've had MozJPEG on my internal to-do list for a while, and it's good to see some interest.

I haven't fully reviewed the diffs between libjpeg-turbo and MozJPEG to see whether it's more appropriate to make it an encode-only plugin or whether it's suitable as a full substitute JPEG codec. One challenge is that the maintainer of MozJPEG has been slow to pick up the upstream changes from libjpeg-turbo, so maybe encode-only is best.

Also I wasn't sure what this

Standard releases of libjpeg and libjpeg-turbo are not usable reliably from .NET; a custom build is required.

means.

That's just a reference to the fact I'm using the classic libjpeg API, which uses C runtime setjmp/longjmp-based error handling (with a default behavior of terminating the process on error), so it requires a C wrapper to use safely. The TurboJPEG API that your wrapper uses is perfectly fine to use cross-language, but it doesn't give enough granularity/control for my use.

[georg-jung]

If there's a way I could help making this happen let me know 👍🏻

[saucecontrol]

Just spent the entire weekend+ updating to the latest versions of all the currently-used native codecs, which has decreased my appetite for adding more 😅

I took the opportunity to go over the diffs between MozJPEG and libjpeg-turbo since they're only one release apart currently. MozJPEG does, in fact, only differ on the encode side, and quite a few of the differences exist only in the cmdline tools -- not in the library itself. Summarizing the core differences here for reference:

1. MozJPEG enables progressive encoding and optimized huffman tables by default. These options are already available (just not default) in libjpeg-turbo.
2. Additional quantization tables are available. These could be implemented outside of libjpeg-turbo since it allows quantization tables to be explicitly provided. In fact, it has been on my to-do list to add Adobe's quantization tables from Photoshop's "Save For Web", which tend to give better results than the IJG standard tables.
3. Overshoot deringing, which really only applies when you use JPEG on images you shouldn't use JPEG on.
4. Trellis quantization, which is a very expensive iterative approach to determining progressive scans and quantization tables. This may be where the most benefit from MozJPEG comes from, but it's worth closer investigation. If this turns out to be important, the question becomes one of maintenance cost of patching libjpeg-turbo with these (unchanged in many years) enhancements or maintaining two separate but mostly identical encoder libraries).

One other option is to say that if you don't care about the perf cost of trellis quantization, you probably also don't care about the efficiency benefits of using the lower-level libjpeg API, so binding against the simpler TurboJPEG API would suffice for a MozJPEG encoder plugin.

[saucecontrol]

Another option I'm keeping an eye on is jpegli, which is based on some of the guetzli work combined with parts from JXL. Seems that it has advantages over MozJPEG, while still producing images readable by standard JPEG decoders.

[georg-jung]

maintenance cost of patching libjpeg-turbo with these

Wouldn't it be an option to replace libjpeg-turbo with mozjpeg altogether? Isn't mozjpeg just libjpeg-turbo with those optimizations? If you have to create a custom fork, reapply the optimizations, and then keep the fork up to date with upstream, wouldn't it be easier to just use mozjpeg in the first place, and if it ever becomes too outdated again, use a then-created fork of mozjpeg, or even send a PR, instead of creating a very similar but "competing" library?

don't care about the efficiency benefits of using the lower-level libjpeg API

That would be true for my use cases. I don't really care if it takes 10 or 20% more time in cases I'm leaning towards space efficiency.

which is based on some of the guetzli work

I gave guetzli a try some years ago and just remeber that it was too compute intense. I intend to use mozjpeg for e.g. my family photo archive too and mozjpeg has a great perf/space trade-off for use cases like this imho. "about 1 minute of CPU per 1 MPix" would be a deal breaker though. Also, for ImageSharp.Web-like applications I can imagine using mozjpeg with caching but guetzli probably wouldn't be an option. Don't get me wrong, I always found it quite fascinating, I just think the possibilities to use it are narrower. Not sure to which extend this applies to jpegli too though.

Reading that article it seems like a pitty chromium doesn't support jxl.

[saucecontrol]

If you have to create a custom fork, reapply the optimizations, and then keep the fork up to date with upstream, wouldn't it be easier to just use mozjpeg in the first place, and if it ever becomes too outdated again, use a then-created fork of mozjpeg, or even send a PR, instead of creating a very similar but "competing" library?

I've tried the PR route in the past, but it was ignored for a month before the maintainer did the work himself and closed the PR 🤷‍♂️ (mozilla/mozjpeg#366)

The issue here is that I have made customizations to libjpeg-turbo (both for performance and compatibility) that have to be maintained, plus MozJPEG and libjpeg-turbo are rarely in sync. libjpeg-turbo is actively maintained while MozJPEG maintenance consists almost entirely of merging upstream changes. So either way, it would be a 3-way merge on my end. It most likely would be easier to manage 2 relatively static change sets on top of libjpeg-turbo than to approach it from the other direction.

I gave guetzli a try some years ago and just remeber that it was too compute intense

Same here. That post didn't mention it, but jpegli is about 1000x faster than Guetzli. Guetzli used a brute-force approach to optimization, but jpegli uses some dead-zone logic from JXL that solves the same problem. It's actually quite exciting.

The post was also comparing jpegli to MozJPEG using arithmetic encoding in the 'tuned' case, which isn't practical since most decoders still don't support that. So the best case MozJPEG numbers shown there are actually a better-than-best case.