[rmkit] Pixmap alpha rendering

[rexxar-tc]

It is presently impossible to render a PNG with transparency. For example, shading a button with an image to show that it's disabled.

Expected:

Actual:

There is a bug in the image resizing function.

rmkit/src/rmkit/util/image.cpy

Lines 17 to 21 in fc372b2

	for (int i=0; i < resize_len; i++):
	rgba_buf[j++] = resize_buffer[i]
	rgba_buf[j++] = resize_buffer[i]
	rgba_buf[j++] = resize_buffer[i]
	rgba_buf[j++] = resize_buffer[i]

All four bytes are set to the grayscale value. This means that if you enable alpha when drawing the bitmap, fully black pixels will be fully transparent (and are skipped)

rmkit/src/rmkit/fb/fb.cpy

Lines 355 to 357 in fc372b2

	// 4th bit is alpha -- if it's 0, skip drawing
	if ((char*)src)[i*image.channels+3] != 0:
	self._set_pixel(&ptr[i], i, j, pack_pixel((char *) src, i*image.channels))

This function should probably set the 4th byte to 0xFF instead of the color value.

Confoundingly, Pixmap::alpha is also broken. The alpha value is a remarkable_color(uint16), which makes no sense when passed in as the pseudo alpha. In draw_bitmap, we compare the full uint32 pixel to a uint16, which means it's impossible to set the pseudo alpha to white (0xFFFFFFFF). Additionally the default value of 97 doesn't make sense to me. I would expect it to be WHITE if not disabled by default.

rmkit/src/rmkit/ui/pixmap.cpy

Line 87 in fc372b2

remarkable_color alpha = 97

If I fix both Pixmap::alpha and util::resize_image I can get the above result by setting the alpha to WHITE. However, this is obviously not true alpha, if you zoom in all the way you can see some pixels around the edge that are lighter than the background.

Forcibly converting all images to a 1-channel grayscale and then back up to a 4-channel RGBA doesn't make a ton of sense. If you want to convert all images down to grayscale before resizing, that's fine, but why bother with converting it to RGBA? render_bitmap has logic for 1-channel bitmaps.

One more thing:

rmkit/src/rmkit/fb/fb.cpy

Lines 313 to 317 in fc372b2

	inline void grayscale_to_rgb32(uint8_t src, char *dst):
	uint32_t color = (src * 0x00010101);
	dst[0] = color & 0x00FF
	dst[1] = color & 0x0000FF
	dst[2] = color & 0x000000FF

While this technically still produces the correct result (and likely gets optimized away), the code is not doing what you appear to have intended.

I could make a PR just fixing pseudo alpha on Pixmaps, but that feels like a hack on top of some structural issues in bitmap rendering. I think that CachedIcons and Pixmap rendering need to be rethought. Alpha blending does not appear to be supported at all in the render process, so why are we forcing all icons into RGBA? There are two separate pseudo alpha calculations happening here, which gives you a situation where both full white and full black are transparent.

My recommendation is to rework cached icons to either explicitly convert to grayscale or RGB, and ensuring bitmap rendering works for 1 and 3 channel bitmaps. Since alpha is not actually supported, we just shouldn't support RGBA bitmaps at all. In that case, a simple pseudo alpha is fine.

While it would be extremely nice to properly support alpha blending, I think that might be a much bigger change.

[raisjn]

thank you, this is great feedback - appreciate you digging in!

My recommendation is to rework cached icons to either explicitly convert to grayscale or RGB, and ensuring bitmap rendering works for 1 and 3 channel bitmaps. Since alpha is not actually supported, we just shouldn't support RGBA bitmaps at all. In that case, a simple pseudo alpha is fine.

this seems reasonable to me. it sounds like the main thing you don't like is the RGBA (instead of just RGB) and that A is mis-leading?

[rexxar-tc]

Well, it's not only misleading, it's just plain incorrect. Copying a grayscale value into all four bytes of an RGBA color is not the right way to inflate a 1 channel image into 4. This gives you a gradient of colors from fully opaque white to fully transparent black. The more correct way would be to fix the alpha byte at 0xFF for fully opaque.

I'm most confused about the re-encoding of the input image. I really don't understand why you'd compress all images down to a single channel before manually inflating it back to four channels. Why not just ask STB for a four channel image? Why not just leave the original colors and let the render figure out how to draw it? The hardware clearly supports full RGB, so I really don't get why you'd touch color at all here.

So yes, if you want to enforce Pixmaps to only display grayscale, that needs to be explicitly communicated, and the Pixmap should probably not present a full RGBA bitmap. However, my expectation of a Pixmap type widget is that it will simply display the image I give it. At most it should resize the image. Having it strip all color and present a mangled grayscale RGBA bitmap is so unexpected that I must assume it's unintentional.

[rexxar-tc]

Well, this was a bit involved, but I made some changes to the renderer: master...rexxar-tc:rmkit:feature/alpha

With these changes, I get the following:

So, render_bitmap appeared to break on anything other than a 3 or 4 channel RGBA image. I assume this is why text and Pixmaps inflate grayscale bitmaps to RGBA, otherwise you read out of bounds and draw a corrupt image. Also the text rendering improperly puts a remarkable_color into the image buffer, which is expected to be in RGBA format instead. I think this is why there was some artifacting over the text.

At this point, it seems clear that the renderer needs an overhaul. There's a bunch of opportunities for optimization and lots of tiny bugs and edge cases. I'm gonna keep working on this refactor and clean up everything I find.

[rexxar-tc]

I've rewritten the bitmap renderer to support alpha blending.

The previous implementation did not correctly handle 3-channel images and just renders garbage. Now we properly handle 1, 3, and 4 channel images with and without transparency.

This has vastly improved the appearance of text and bitmaps.

Text now renders correctly over a background:

I've also added a transparent draw_rect, which allows me to shade text and buttons.

Same text rendering code, just with a transparent gray rectangle overtop instead of on the background.

Here's some comparison images. Stock rmkit:

With alpha:

I think it's a huge improvement and allows for much more complex layouts. The actual code is pretty light, just a bit of integer arithmetic. I'll do some benchmarks in the next few days, but I don't expect it to be that much slower.

[raisjn]

this is amazing, great work - looking fwd to the pr 👏 thank you for continually improving rmkit 😀