Skip to content

Jpeg encoder complete rewrite - #2120

Merged
JimBobSquarePants merged 41 commits into
SixLabors:mainfrom
br3aker:dp/jpeg-encoder-color-conversion
Aug 11, 2022
Merged

Jpeg encoder complete rewrite#2120
JimBobSquarePants merged 41 commits into
SixLabors:mainfrom
br3aker:dp/jpeg-encoder-color-conversion

Conversation

@br3aker

@br3akerbr3aker commented May 15, 2022

Copy link
Copy Markdown
Contributor

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
  • I have provided test coverage for my change (where applicable)

New architecture

Jpeg encoder now have a brand new architecture which allows us to define encoding 'configs' like this:

// YCbCr 4:1:0newJpegFrameConfig(JpegColorSpace.YCbCr,JpegEncodingColor.YCbCrRatio410,newJpegComponentConfig[]{newJpegComponentConfig(id:1,hsf:4,vsf:2,quantIndex:0,dcIndex:0,acIndex:0),newJpegComponentConfig(id:2,hsf:1,vsf:1,quantIndex:1,dcIndex:1,acIndex:1),newJpegComponentConfig(id:3,hsf:1,vsf:1,quantIndex:1,dcIndex:1,acIndex:1),},yCbCrHuffmanConfigs,yCbCrQuantTableConfigs)

Which allow us to define any allowed subsampling setup or even a 'custom' scenario with any number of components if user needs to. This API is internal for now - user can only choose encoding mode via enum which we had a long time ago. Maybe one day we can make it public - who knows.

Due to this change: #1713 is resolved and #1476 is resolved.

It would also be fairly easy to implement optimized huffman tables or progressive encoding with current architecture.

API changes

  1. All of these encoding modes are implemented now:
YCbCrRatio420=0,YCbCrRatio444=1,YCbCrRatio422=2,YCbCrRatio411=3,YCbCrRatio410=4,Luminance=5,Rgb=6,Cmyk=7,Ycck=8,
  1. Grayscale encoding now has a vectorized color conversion which gained a lot of performance.

3. I've removed Ycck encoding simply because it's not popular and we didn't support it anyway (we still can decode YccK though). This issue is partially done: #808

  1. Added non-interleaved encoding for any color type.

Performance

Encoder got a bit slower but I think new architecture's worth it - we were really fast before, now we are still fast enough.

Main branch

MethodTargetColorSpaceQualityMean
BenchmarkLuminance756.923 ms
BenchmarkRgb7512.116 ms
BenchmarkYCbCrRatio420756.484 ms
BenchmarkYCbCrRatio444758.263 ms
BenchmarkLuminance907.046 ms
BenchmarkRgb9013.349 ms
BenchmarkYCbCrRatio420906.714 ms
BenchmarkYCbCrRatio444909.198 ms
BenchmarkLuminance1009.057 ms
BenchmarkRgb10019.099 ms
BenchmarkYCbCrRatio42010010.194 ms
BenchmarkYCbCrRatio44410015.372 ms

PR

MethodTargetColorSpaceQualityMean
BenchmarkLuminance754.618 ms
BenchmarkRgb7512.543 ms
BenchmarkYCbCrRatio420756.639 ms
BenchmarkYCbCrRatio444758.590 ms
BenchmarkLuminance905.456 ms
BenchmarkRgb9013.447 ms
BenchmarkYCbCrRatio420907.218 ms
BenchmarkYCbCrRatio444909.150 ms
BenchmarkLuminance1006.731 ms
BenchmarkRgb10019.831 ms
BenchmarkYCbCrRatio42010010.541 ms
BenchmarkYCbCrRatio44410015.345 ms

@br3aker

Copy link
Copy Markdown
ContributorAuthor

Need to write a ton of tests now....

MultiplyToAverage(sourceRow, averageMultiplier);

// copy to the first 8 slots
sourceRow.Slice(0, packedWidth).CopyTo(this.ColorBuffer.DangerousGetRowSpan(i / factors.Height));

@br3akerbr3akerMay 15, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the reasons 420 subsampling encoding got slower - we absolutely need to move ready-to-encode color strides to the first 8 indices. This potentially can be resolved at the TPixel -> RGB conversion step - we can rearrange color strides before doing subsampling but it's a very tricky thing to do, I've decided to make this PR as small as possible.

@gfoidlgfoidl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half way viewed (have to leave now, will continue later).

Comment threadsrc/ImageSharp/Common/Helpers/SimdUtils.HwIntrinsics.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Block8x8F.ScaledCopy.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Block8x8F.ScaledCopy.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Block8x8F.ScaledCopy.cs Outdated

@gfoidlgfoidl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished my review -- didn't look at test code (due to missing knowledge there).

Looks good 👍🏻

Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/Component.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/Component.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/Component.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/ComponentProcessor.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/HuffmanScanEncoder.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/HuffmanScanEncoder.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/HuffmanScanEncoder.cs Outdated
Comment threadsrc/ImageSharp/Formats/Jpeg/Components/Encoder/HuffmanScanEncoder.cs Outdated
Comment threadsrc/ImageSharp/PixelFormats/PixelOperations{TPixel}.cs
@br3aker

br3aker commented Aug 10, 2022

Copy link
Copy Markdown
ContributorAuthor

Looks like I've messed up linked issues, here's the complete list of what's resolved by this PR:

  1. Speed Up Jpeg Encoder Color Conversion #1476 - all color conversions now support scalar/vector/avx paths
  2. Feature request: allow Color Space choices in JpegEncoder #808 - all output color types are supported
  3. Jpeg encoder writes zeroed chrominance quantization table for grayscale images and RGB encoding #1713 - only required quantization table(s) is(are) written
  4. Opening and saving jpeg using CMYK color space drastically changes image's color #1238 - CMYK input jpeg now should be resaved to CMYK output jpeg if not explicitely overriden in encoder settings

@JimBobSquarePantsJimBobSquarePants left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks awesome. Thanks again for such a fantastic addition to the library!

@JimBobSquarePants
JimBobSquarePants merged commit 13897ae into SixLabors:mainAug 11, 2022
@br3aker
br3aker deleted the dp/jpeg-encoder-color-conversion branch August 11, 2022 14:16
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@br3aker@brianpopow@JimBobSquarePants@gfoidl