Color histograms are useful to get a sense of what a given image looks like. If you work on deep learning with PyTorch, you might sometimes want to calculate color histograms of image tensors. In this article, I will show you two ways to do it: OpenCV-based and PyTorch-based. As of August 2022, he PyTorch solution does not work on GPU.

Setups

First of all, I summarize the setups.

In the following part, all the code was run on Colaboratory with a Tesla T4 GPU.

To calculate color historgram, you should be aware of which color space you are using. It can be RGB, HSV, Lab, or anything else, but here I chose Lab color space because it was designed to be perceptually uniform. The image below visualizes the Lab color space when the perceptual lightness L* is 127 (out of 255).

As a sample image, I used the following image generated by DALL-E (query I meant: “An oil painting by Mattise of cockatoos chatting over tea in Morocco”).

We can create an image tensor represented in Lab color space with the following snippet:

import cv2
import torch
from torchvision import transforms
from PIL import Image

from matplotlib import pyplot as plt

img = cv2.imread('sample.png')
img = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)
X = torch.Tensor(img)

Once given a histogram algorithm, we can plot the result on the a*-b* plane like below:

###
# Calculate color histogram here
# histr = my_histogram(X)
###

fig, ax = plt.subplots(1,1)
im = ax.imshow(histr, extent=[0, 255, 0, 255])
ax.set_xlabel("b*")
ax.set_ylabel("a*")
fig.colorbar(im)
fig.show()

That’s it for the setups. Let’s move on to the calculating color histograms!

OpenCV

OpenCV supports the very solution. With the function calcHist, calculating color histogram is straightforward.

histr = cv2.calcHist([X.numpy()], channels=[1, 2], mask=None, histSize=[32, 32], ranges=[0,256, 0, 256])
histr /= histr.sum() # normalize sum to 1

The above snippet was executed in 5.25 ms. You can also pass a mask to the function.

PyTorch

PyTorch supports a function for calculating multi-dimensional histogram, which has the same interface as NumPy. You can pass an image tensor to the function histogramdd after flattening the spatial dimensions.

histr, _ = torch.histogramdd(X[:,:,1:].view(-1, 2), bins=[32,32], range=[0,256, 0, 256])
histr /= histr.sum() # normalize sum to 1

The above snippet was executed in 23.1 ms. OpenCV is superior in terms of speed, simplicity, and usefulness.

PyTorch on GPU

Can we expect a significant speed-up for the PyTorch solution on GPU? Unfortunately, it does not work at the time of this writing because the function torch.histogramdd for the CUDA backend is not supported by PyTorch v1.12 (the latest). If you call the function on a GPU, it will throw a NotImplementedError:

There are already some feature requests to implement this function for the CUDA backend (here and here). I don’t know how to write CUDA codes, but if it can be extended from the GPU version of torch.histc function, it might not be too hard.

References

[1] torch.histogramdd — PyTorch 1.12 documentation
[2] torch.histc — PyTorch 1.12 documentation
[3] ヒストグラム その1: 計算して，プロットして，解析する !!! — OpenCV-Python Tutorials 1 documentation
[4] Lab* 色空間 | OpenCV を用いて Lab* 色空間を取り扱う方法