Discussion:
Make QImage colorspace aware
Niels Ole Salscheider
2011-09-22 10:45:35 UTC
Permalink
Hello,

it might be a good idea to make QImage colorspace aware and add some other
color formats (like 30 bit or floating point formats).

This might be useful for a fully color managed desktop environment, too. I am
not sure how this will work in wayland. Either the compositor applies the
color conversion (what happens if the client uses several QImage with different
colorspaces?) or it has to be done by the client (it should use a shader to do
so nevertheless).
How does it work on other platforms?

Regards,

Ole
l***@nokia.com
2011-09-23 20:25:26 UTC
Permalink
Hi Niels,

while I very much would like to see some code to do advanced color space
management in Qt, I am unsure whether this should go into QtGui or into an
add-on module.

The reason is that adding this and other image formats can easily blow up
the size of the Gui library a lot.

In addition, I'd also prefer to see how color management on a desktop
environment would be done. Simply adding this to QImage is not enough,
esp. since many things (QML, QPixmap, QWidget) don't even go through
images.

Cheers,
Lars

On 9/22/11 12:45 PM, "ext Niels Ole Salscheider"
Post by Niels Ole Salscheider
Hello,
it might be a good idea to make QImage colorspace aware and add some other
color formats (like 30 bit or floating point formats).
This might be useful for a fully color managed desktop environment, too. I am
not sure how this will work in wayland. Either the compositor applies the
color conversion (what happens if the client uses several QImage with different
colorspaces?) or it has to be done by the client (it should use a shader to do
so nevertheless).
How does it work on other platforms?
Regards,
Ole_______________________________________________
Qt5-feedback mailing list
http://lists.qt.nokia.com/mailman/listinfo/qt5-feedback
Zack Rusin
2011-09-26 04:19:22 UTC
Permalink
Post by l***@nokia.com
Hi Niels,
while I very much would like to see some code to do advanced color space
management in Qt, I am unsure whether this should go into QtGui or into an
add-on module.
The reason is that adding this and other image formats can easily blow up
the size of the Gui library a lot.
It would have to be both if we wanted proper support, with QtGui, depending on
the addon module. We could get away with a very limited support in QtGui and
it could probably be enough for most use cases. The most prevelant use case
for Qt will be displaying images, most of them coming from the web. Cameras
now embedded pretty accurate color profiles with pictures. Granted that WebKit
would need to be fixed first but Qt would just need support for telling some
external entity to interpret the given screen estate as the following color
profile.
It's probably one of the two things I'd like to fix. In general we don't do a
very good job or respecting color spaces, we mix linear rgb with srgb pretty
freely right now.
Post by l***@nokia.com
In addition, I'd also prefer to see how color management on a desktop
environment would be done. Simply adding this to QImage is not enough,
esp. since many things (QML, QPixmap, QWidget) don't even go through
images.
Yea, btw, it would be nice to eliminate the QPixmap/QImage split for Qt5.
Just do a QImage/QImageData split with the latter letting people access the
actual data and synchronous functions used to fetch the QImageData from
QImage. Maybe with wrapper classes to imitate the old behavior (so maybe call
it QSurface/QSurfaceData or something to not take over the QImage name) we
could stay source compatitible and eliminate that pretty arbitrary, for most
people, split.

z
g***@nokia.com
2011-09-26 07:06:38 UTC
Permalink
Post by Zack Rusin
Post by l***@nokia.com
Hi Niels,
while I very much would like to see some code to do advanced color space
management in Qt, I am unsure whether this should go into QtGui or into an
add-on module.
The reason is that adding this and other image formats can easily blow up
the size of the Gui library a lot.
It would have to be both if we wanted proper support, with QtGui, depending on
the addon module. We could get away with a very limited support in QtGui and
it could probably be enough for most use cases. The most prevelant use case
for Qt will be displaying images, most of them coming from the web. Cameras
now embedded pretty accurate color profiles with pictures. Granted that WebKit
would need to be fixed first but Qt would just need support for telling some
external entity to interpret the given screen estate as the following color
profile.
It's probably one of the two things I'd like to fix. In general we don't do a
very good job or respecting color spaces, we mix linear rgb with srgb pretty
freely right now.
Yeah, I would eventually like to us to have proper color space support. The problem with putting it into QImage is that QImage comes with a lot of baggage. Integration with QPainter, existing code that relies on bits()/bytesPerLine() access. It is not a simple fix to restructure QImage into a pluggable imagedata class, and it would not plug directly into existing code in a source-wise nice manner.

The primary usecase for non sRGB color data is camera data and if you are looking at camera data, then you probably also want to add other things like timestamps, geo-tags and other metadata, which would make QImage an even uglier mixture of concepts.

So because of this, I would prefer if we stared the such an camera-data image class in an addon. If it becomes a good and lean class, then maybe we deprecate QImage/QPixmap over time and replace it with this and migrate this class into QtGui. I wouldn't have a problem with that.
Post by Zack Rusin
Post by l***@nokia.com
In addition, I'd also prefer to see how color management on a desktop
environment would be done. Simply adding this to QImage is not enough,
esp. since many things (QML, QPixmap, QWidget) don't even go through
images.
Yea, btw, it would be nice to eliminate the QPixmap/QImage split for Qt5.
Just do a QImage/QImageData split with the latter letting people access the
actual data and synchronous functions used to fetch the QImageData from
QImage. Maybe with wrapper classes to imitate the old behavior (so maybe call
it QSurface/QSurfaceData or something to not take over the QImage name) we
could stay source compatitible and eliminate that pretty arbitrary, for most
people, split.
z
Yup, I was never happy with how the QPixmap / QImage split turned out. In QML 2 we at least stopped using the QPixmap class.

-
Gunnar
Zack Rusin
2011-09-26 22:18:32 UTC
Permalink
Post by g***@nokia.com
The primary usecase for non sRGB color data is camera data and if you are
looking at camera data, then you probably also want to add other things
like timestamps, geo-tags and other metadata, which would make QImage an
even uglier mixture of concepts.
That metadata isn't terribly interesting and there are other ways of embedding
it, so I'm not interested or worried about it. The issue is that Qt is flat out
wrong when it comes to displaying the images. E.g.

http://www.gballard.net/psd/srgbforwww.html

The snafu is that pretty much all of the flicker images will be coming with a
color profile, which we're going to consistently ignore, meaning that more and
more images on all Qt constructed browsers will look at best wrong (and often
just shitty). So it's not a question of extra features, it's a question of
correctness.
Even just a way of preserving the color profile identifier would be probably
enough to let the compositor/windowing system know what format the given
rect/window is in, to let it do the right thing. Or we just recommend that
people don't use Qt image loaders and use their own stuff that converts the
format to whatever QImage wants. Either way it has to be decided before Qt5
because I think it's going to be very hard to shoehorn color managment in, in
a completely BC way after Qt5.

z
Niels Ole Salscheider
2011-09-27 07:01:50 UTC
Permalink
Hello,
Post by Zack Rusin
Even just a way of preserving the color profile identifier would be
probably enough to let the compositor/windowing system know what format
the given rect/window is in, to let it do the right thing. Or we just
recommend that people don't use Qt image loaders and use their own stuff
that converts the format to whatever QImage wants. Either way it has to be
decided before Qt5 because I think it's going to be very hard to shoehorn
color managment in, in a completely BC way after Qt5.
Passing one colorspace per window is not enough if it contains several
different images.
X Color Management
(http://oyranos.org/scm?p=xcolor.git;a=blob;f=docs/X_Color_Management.txt,
formerly net-color spec) does it per rect, but this might be a problem if you
use some effects (blur the border of the image, transparency, ...).

Maybe it is worth to have a look how Gnome does it. In
http://libregraphicsworld.org/articles.php?article_id=42, Richard Hughes says:

I've considered using some of the existing net-color spec by Kai-Uwe, but I
think a lot of the existing spec is just wrong and adding things like network
transparency means we introduce a huge number of issues that I don't think we
want to support. I also want to make it super easy to opt-in or opt-out
widgets (and windows) from this FSCM and the net-color spec doesn't exactly
make that easy without defining manual opt-out regions that doesn't work so
well with moving, resizing, reshaped widgets in modern toolkits.

If the windowing system only has access to the completely rendered client
window (this is how it works with wayland, isn't it?), I think color
management has to be done by the client toolkit. Could this deal with multi-
head setups?
Otherwise, passing the color space information to the windowing system might
be better.

Regards,

Ole
Samuel Rødal
2011-09-27 13:56:14 UTC
Permalink
Post by Niels Ole Salscheider
Hello,
Post by Zack Rusin
Even just a way of preserving the color profile identifier would be
probably enough to let the compositor/windowing system know what format
the given rect/window is in, to let it do the right thing. Or we just
recommend that people don't use Qt image loaders and use their own stuff
that converts the format to whatever QImage wants. Either way it has to be
decided before Qt5 because I think it's going to be very hard to shoehorn
color managment in, in a completely BC way after Qt5.
Passing one colorspace per window is not enough if it contains several
different images.
X Color Management
(http://oyranos.org/scm?p=xcolor.git;a=blob;f=docs/X_Color_Management.txt,
formerly net-color spec) does it per rect, but this might be a problem if you
use some effects (blur the border of the image, transparency, ...).
Maybe it is worth to have a look how Gnome does it. In
I've considered using some of the existing net-color spec by Kai-Uwe, but I
think a lot of the existing spec is just wrong and adding things like network
transparency means we introduce a huge number of issues that I don't think we
want to support. I also want to make it super easy to opt-in or opt-out
widgets (and windows) from this FSCM and the net-color spec doesn't exactly
make that easy without defining manual opt-out regions that doesn't work so
well with moving, resizing, reshaped widgets in modern toolkits.
If the windowing system only has access to the completely rendered client
window (this is how it works with wayland, isn't it?), I think color
management has to be done by the client toolkit. Could this deal with multi-
head setups?
Otherwise, passing the color space information to the windowing system might
be better.
Agreed, it does sound like color management needs to be done by the
client toolkit.

From my understanding there are several aspects to correctly handling
color spaces. The first aspect is storage / representation. Eventually
we'll probably need image formats with more than 8-bits per color
channel and potentially floating point as well to support Deep Color and
extended gamut color spaces like xvYCC. QImage's Format enum kind breaks
down once we want to support variable bit depths and floating point
formats, so a new approach with generic storage and color space
descriptors might be useful there. A question is whether we want these
image formats to be paintable with QPainter.

Another aspect is presentation of this content on a screen visible
surface, which is abstracted in Qt 5 by the QWindow class. We now have
two approaches for getting content onto a QWindow, QBackingStore and
QOpenGLContext. QWindow has a QSurfaceFormat, which might already be
able to support bit depths beyond 8 for the color channels, but which
doesn't have a say about color space. Currently the assumption is
typically that the final QWindow content should be in the sRGB color space.

The third aspect is doing color space aware blending, antialiasing,
sampling, conversions etc. OpenGL already has some relevant extensions
for sRGB content, namely GL_ARB_framebuffer_sRGB and
GL_EXT_texture_sRGB. Maybe scene graph / QML 2 should be using them
behind the scenes on desktop when appropriate. It might be possible to
handle other color spaces than sRGB with fragment shaders. For QPainter
on a QImage this might be too computationally expensive however (we
already ignore sRGB and treat everything as linear in our current
rendering pipelines, both raster and OpenGL).

The first aspect, storage, seems to me to be the trickiest one, as it
would require API changes in QImage or a new image class (which would
then not play as nicely with existing Qt APIs like QPainter).

--
Samuel
Zack Rusin
2011-09-27 15:59:55 UTC
Permalink
Post by Niels Ole Salscheider
Passing one colorspace per window is not enough if it contains several
different images.
X Color Management
(http://oyranos.org/scm?p=xcolor.git;a=blob;f=docs/X_Color_Management.txt
formerly net-color spec) does it per rect, but this might be a problem if
you use some effects (blur the border of the image, transparency, ...).
It's not enough for tons of stuff besides multiple-color-spaces, which includes
any operations doing blending of color-spaced objects but that's not the
point. It's a question of benefits vs implementation complexity and they don't
have a linear relationship. It'd be bonkers to do a full color space managment
just because you wouldn't want someone who wants to do gaussian blur on an
image with an embedded color space suffer loss of color quality on the blurred
image.
What's important for Qt is simply correctly displaying images, everything else
and that includes any manipulation of those images, is a great extra but is
non-essential.

z
Kai-Uwe Behrmann
2011-09-27 16:32:09 UTC
Permalink
Hello,

I am the author of the X Color Management alias net-color spec.

I want to clearify its intention. The desktop has a range of regions
originating from a heterogenous set of toolkits. Some might do colour
correction some not. It is the goal of colour management to correct all
regions. So all regions need to get a source ICC profile assigned.
This will be implicitely sRGB. On the screen are as well colour management
aware windows, which do already colour correction on their own. But they
do that only in certain regions, and all toolkit widgets are typical not
colour corrected like in Krita, Scribus and so on. For these type of
applications is a opt-out of colour management a good solution. They can
continue with their custom colour management and render directly to the
screen under X11 but in a homogenous environment. The assignment of
colour spaces to window regions through atoms is indirect. But with X11
there is no other means like offloading the colour correction of all
undefined regions to a compositing window manager. Thats a late colour
binding with some small artefacts, but generally a big improvement for
most systems.


kind regards
Kai-Uwe Behrmann
--
developing for colour management
oyranos-cms.blogspot.com ***@freenode#openicc
Kai-Uwe Behrmann
2011-09-27 16:47:01 UTC
Permalink
Great that you discuss colour management in Qt.

Niels Ole Salscheider niels_ole at salscheider-online.de
Post by Niels Ole Salscheider
it might be a good idea to make QImage colorspace aware and add some
other color formats (like 30 bit or floating point formats).
A 30-bit format will be from the point of keeping a 32-bit per pixel
format ideal. But for linear images and colour management it might be too
less. It is not the best editing format, as applications might want to
route 30-bit to monitors but need headroom for compositing to avoid
banding. Correct would be a full 16-bit per channel format, not only
10-bit per channel. Floating point is of course appreciated.
Post by Niels Ole Salscheider
This might be useful for a fully color managed desktop environment, too.
I am not sure how this will work in wayland. Either the compositor
applies the color conversion (what happens if the client uses several
QImage with different colorspaces?) or it has to be done by the client
(it should use a shader to do so nevertheless).
How does it work on other platforms?
The correct time for colour conversion is at blending time in the
decided blending space. This can be sRGB or a linear blending space or
whatever, but idealy not a device dependent colour space.
For instance, compositing of a transparent CIE*Lab image over printer Cmyk
or video Yuv and colour converting afterward blending from, I do not know
which space to select, to a actual monitor space will result in garbage.

Note the X Color Mangagement spec works because it assumes there is no
blending involved. However the window manager can do compositing in
monitor space like for transparent windows, which is kind of ok with very
minor artefacts only.

kind regards
Kai-Uwe Behrmann
--
developing for colour management
oyranos-cms.blogspot.com ***@freenode#openicc
Kai-Uwe Behrmann
2011-09-27 16:55:31 UTC
Permalink
Niels Ole Salscheider niels_ole at salscheider-online.de
Post by Niels Ole Salscheider
If the windowing system only has access to the completely rendered
client window (this is how it works with wayland, isn't it?), I think
color management has to be done by the client toolkit. Could this deal
If you consider compositing as part of the process then absolutely yes.
Post by Niels Ole Salscheider
with multi-head setups?
It depends. If a appliciation has to be aware of all kind of
window distortions, that can become a callange.
Post by Niels Ole Salscheider
Otherwise, passing the color space information to the windowing system
might be better.
Yes. A strategy along this path would be, to render the whole window to a
certian intermediate space ad pass that together with the according ICC
profile to the windowing system for the final to monitor colour
correction. You need to decide about a blending space anyway, so that
intermediate step is likely not much more effort.

kind regards
Kai-Uwe Behrmann
--
developing for colour management
oyranos-cms.blogspot.com ***@freenode#openicc
Niels Ole Salscheider
2011-10-07 11:52:59 UTC
Permalink
Hello,
Post by Kai-Uwe Behrmann
Yes. A strategy along this path would be, to render the whole window to a
certian intermediate space ad pass that together with the according ICC
profile to the windowing system for the final to monitor colour
correction. You need to decide about a blending space anyway, so that
intermediate step is likely not much more effort.
We could use L*a*b* color space with 16 bits per pixel as intermediate space
if the compositor supports it. This can be translated to the device color
space with a LUT from the according icc profile by the compositor.

If L*a*b* / color management is not supported by the compositor or if it is
not that important (e. g. on some embedded devices), we can convert the image
to srgb and pass this.

Regards,

Ole

Loading...