SOCR ≫ DSPA ≫ Topics ≫

In this DSPA Appendix, we present examples of image processing, spectral manipulation, and filtering.

## Warning: package 'knitr' was built under R version 4.1.2

1 Getting started

r Biocpkg(“EBImage”) is an R package distributed as part of the Bioconductor project. To install the package, start R and enter:

install.packages("BiocManager")
BiocManager::install("EBImage")

Once the R package(“EBImage”) is installed, it can be loaded by the following command.

library("EBImage")
  • Note: If you can’t install the EBImage package for any reason (e.g., operating system or version incompatibility, this is not a problem as you should be able to use the jpeg package instead.
# install.packages("jpeg")
library(jpeg)

2 Reading, displaying and writing images

Basic R package(“EBImage”) functionality includes reading, writing, and displaying of images. Images are read using the function readImage, which takes as input a filename or an URL. To start off, let us load a sample picture distributed with the package.

f = system.file("images", "sample.png", package="EBImage")
img = readImage(f)

The R package (“EBImage”) currently supports three image file formats: jpeg, png and tiff. Additional image formats can be imported using the R GitHubPkg (“aoles/RBioFormats”), which adds support for a wide range of file formats including proprietary microscopy image data and metadata.

Imported images can be visualized by the function display().

display(img, method="browser")

In interactive R sessions, display opens the image in a JavaScript viewer in a web browser tab. Mouse or keyboard shortcuts allow zooming in and out of the image, panning, and cycling through multiple image frames. Images can also be displayed using core R plotting methods, which allows for combining images with other plotting functionality, e.g., adding text labels on top of the image.

display(img, method="raster")
text(x = 20, y = 20, label = "Colorful Parrots", adj = c(0,1), col = "orange", cex = 2)

The graphics displayed in an R device can be saved using R package(“base”) R functions dev.print or dev.copy. For example, let’s save our annotated image as a JPEG file and verify its size on disk.

filename = "parrots.jpg"
dev.print(jpeg, filename = filename , width = dim(img)[1], height = dim(img)[2])
png 
  2 
file.info(filename)$size
[1] 34311

The default behavior of display can be globally changed by setting the "options("EBImage.display") to either“browser”or“raster”`. This is useful, for example, to preview images inside RStudio.

It is also possible to read and view color images,

imgcol = readImage(system.file("images", "sample-color.png", package="EBImage"))
display(imgcol)

Images may contain multiple frames, in which case they can be displayed all at once in a grid arrangement by specifying the function argument all = TRUE.

nuc = readImage(system.file("images", "nuclei.tif", package="EBImage"))
display(nuc, method = "raster", all = TRUE)

Alternatively, single frames can be displayed.

In addition to importing images, images can be exported (saved) to files using the EBImage::writeImage(). The image that we loaded was a PNG file. To save this image as a JPEG file, the JPEG format allows setting a quality value between 1 and 100 to reflect the desired level of image compression. The default value of the quality argument of writeImage is 100. Smaller values yield images of smaller size, but worse resolution (less detail).

writeImage(imgcol, "sample.jpeg", quality = 85)

Similarly, we could have saved the image as a TIFF file and set which compression algorithm we want to use. For a complete list of available parameters see ?writeImage.

3 Image data representation

EBImage uses a package-specific class Image to store and process images. It extends the R base class array, and all R package(“EBImage”) functions can also be called directly on matrices and arrays. You can find out more about this class by typing ?Image. Let us peek into the internal structure of an Image object.

str(img)
Formal class 'Image' [package "EBImage"] with 2 slots
  ..@ .Data    : num [1:768, 1:512] 0.447 0.451 0.463 0.455 0.463 ...
  ..@ colormode: int 0
  ..$ dim: int [1:2] 768 512

The .Data slot contains a numeric array of pixel intensities. We see that in this case the array is two-dimensional, with 768 times 512 elements, and corresponds to the pixel width and height of the image. These dimensions can be accessed using the dim function, just like for regular arrays.

dim(img)
[1] 768 512

Image data can be accessed as a plain R array using the imageData accessor.

imageData(img)[1:3, 1:6]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.4470588 0.4627451 0.4784314 0.4980392 0.5137255 0.5294118
[2,] 0.4509804 0.4627451 0.4784314 0.4823529 0.5058824 0.5215686
[3,] 0.4627451 0.4666667 0.4823529 0.4980392 0.5137255 0.5137255

The as.array() method can be used to coerce an Image to an array.

is.Image( as.array(img) )
[1] FALSE

The distribution of pixel intensities can be plotted in a histogram, and their range inspected using the range function.

hist(img)

range(img)
[1] 0 1

A useful summary of Image objects is also provided by the show method, which is invoked if we simply type the object’s name.

img
Image 
  colorMode    : Grayscale 
  storage.mode : double 
  dim          : 768 512 
  frames.total : 1 
  frames.render: 1 

imageData(object)[1:5,1:6]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.4470588 0.4627451 0.4784314 0.4980392 0.5137255 0.5294118
[2,] 0.4509804 0.4627451 0.4784314 0.4823529 0.5058824 0.5215686
[3,] 0.4627451 0.4666667 0.4823529 0.4980392 0.5137255 0.5137255
[4,] 0.4549020 0.4666667 0.4862745 0.4980392 0.5176471 0.5411765
[5,] 0.4627451 0.4627451 0.4823529 0.4980392 0.5137255 0.5411765

For a more compact representation without the preview of the intensities array use the print method with the argument short set to TRUE.

print(img, short=TRUE)
Image 
  colorMode    : Grayscale 
  storage.mode : double 
  dim          : 768 512 
  frames.total : 1 
  frames.render: 1

Color images are based on 3 channels (RBG).

print(imgcol, short=TRUE)
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 768 512 3 
  frames.total : 3 
  frames.render: 1

They differ from gray-scale images by the property colorMode and the number of dimensions, 3 (B&W) vs. 4 (color). The colorMode slot turns out to be convenient when dealing with stacks of images. If it is set to gray-scale, then the third and all higher dimensions of the array are considered as separate image frames corresponding, for instance, to different z-positions, time points, replicates, etc. On the other hand, if colorMode is Color, then the third dimension is assumed to hold different color channels, and only the fourth and higher dimensions are used for multiple image frames. imgcol contains three color channels, which correspond to the red, green and blue intensities of the photograph. However, this does not necessarily need to be the case, and the number of color channels is arbitrary.

The “frames.total” and “frames.render” fields shown by the object summary correspond to the total number of frames contained in the image, and to the number of rendered frames. These numbers can be accessed using the function numberOfFrames by specifying the type argument.

numberOfFrames(imgcol, type = "render")
[1] 1
numberOfFrames(imgcol, type = "total")
[1] 3

Image frames can be extracted using getFrame and getFrames. getFrame returns the i-th frame contained in the image y. If type is "total", the function is unaware of the color mode and returns an xy-plane. For type="render" the function returns the i-th image as shown by the display function. While getFrame returns just a single frame, getFrames retrieves a list of frames which can serve as input to lapply-family functions. See the “Global thresholding” section for an illustration of this approach.

Let’s look at nuclear/cellular imaging data, which contains 4 total frames that correspond to the 4 separate gray-scale images, as indicated by “frames.render”.

nuc
Image 
  colorMode    : Grayscale 
  storage.mode : double 
  dim          : 510 510 4 
  frames.total : 4 
  frames.render: 4 

imageData(object)[1:5,1:6,1]
           [,1]       [,2]       [,3]       [,4]       [,5]       [,6]
[1,] 0.06274510 0.07450980 0.07058824 0.08235294 0.10588235 0.09803922
[2,] 0.06274510 0.05882353 0.07843137 0.09019608 0.09019608 0.10588235
[3,] 0.06666667 0.06666667 0.08235294 0.07843137 0.09411765 0.09411765
[4,] 0.06666667 0.06666667 0.07058824 0.08627451 0.08627451 0.09803922
[5,] 0.05882353 0.06666667 0.07058824 0.08235294 0.09411765 0.10588235

4 Color management

As described in the previous section, the class Image extends the base class array and uses colorMode to store how the color information of the multi-dimensional data should be handled.

The function colorMode can be used to access and change this property, modifying the rendering mode of an image. For example, if we take a Color image and change its mode to gray-scale, then the image won’t display as a single color image anymore but rather as three separate gray-scale frames corresponding to the red, green and blue channels. The function colorMode does not change the actual content of the image but only changes the way the image is rendered by R package(“EBImage”).

colorMode(imgcol) = Grayscale
display(imgcol, all=TRUE)

Color space conversions between gray-scale and Color images are performed using the function channel. It has a flexible interface which allows to convert either way between the modes, and can be used to extract color channels. Unlike colorMode, channel changes the pixel intensity values of the image.

Color to gray-scale conversion modes include taking a uniform average across the RGB channels, and a weighted luminescence preserving conversion mode better suited for display purposes.

The asred, asgreen and asblue modes convert a gray-scale image or array into a color image of the specified hue.

The convenience function toRGB promotes a gray-scale image to RGB color space by replicating it across the red, green and blue channels, which is equivalent to calling channel with mode set to rgb. When displayed, this image doesn’t look different from its gray-scale origin, which is expected because the information between the color channels is the same. To combine three gray-scale images into a single rgb image use the function rgbImage.

The function Image can be used to construct a color image from a character vector or array of named R colors (as listed by colors()) and/or hexadecimal strings of the form “#rrggbb” or “#rrggbbaa”.

colorMat = matrix(rep(c("red","green", "#0000ff"), 25), 5, 5)
colorImg = Image(colorMat)
colorImg
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 5 5 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:5,1]
     [,1] [,2] [,3] [,4] [,5]
[1,]    1    0    0    1    0
[2,]    0    1    0    0    1
[3,]    0    0    1    0    0
[4,]    1    0    0    1    0
[5,]    0    1    0    0    1
display(colorImg, interpolate=FALSE)

5 Manipulating images

Being numeric arrays, images can be conveniently manipulated by any of R’s arithmetic operators. For example, we can produce a negative image by simply subtracting the image from its maximum value.

img_neg = max(img) - img
display( img_neg )

We can also increase the brightness of an image through addition, adjust the contrast through multiplication, and apply gamma correction through exponentiation.

img_comb = combine(
  img,
  img + 0.3,
  img * 2,
  img ^ 0.5
)

display(img_comb, all=TRUE)