Today’s Class

1. All about Pixels
2. Colour and Colourspaces
3. 2D Transformations
4. HTML Canvas

Pixels

• Computers represent images with discrete “picture elements” known as pixels
• Each pixel can only show one colour and can be tiny
• Most screens use 24-bit colours, where red green and blue are given 8 bits (0 to 255 value)

Pixel Coordinates

• Images are represented as tensors (generalization of matrices)
• Each pixel has 2D coordinates and is a vector
  • For example 24-bit RGB colour screen each pixel would be $I(u,v) \in \{0,\ldots,255\}^3$
• Different ways of accessing pixels
  • Where should the origin $(0,0)$ be?
  • OpenGL defines origin at bottom-left by default
  • It can be set arbitrarily: a linear transformation is sufficient
• Pixel coordinates can floating point $c$ or integer $d$ and are related by

   $$\begin{align}
  d &= \text{floor}(c) \\
  c &= d + 0.5
  \end{align}$$ 

Drawing Lines

• Idea: write equation of the line segment and then solve it for integer values of $y$ given $x$

   $$\frac{y - y_0}{y_1-y_0} = \frac{x-x_0}{x_1-x_0}$$ 

• Solve for $y$ with $y = \frac{y_1-y_0}{x_1-x_0} (x-x_0) + y_0$

Integer-Only Bresenham Algorithm

plotLine(x0, y0, x1, y1)
    dx = x1 - x0
    dy = y1 - y0
    D = 2*dy - dx
    y = y0

    for x from x0 to x1
        draw(x,y)
        if D > 0
            y = y + 1
            D = D - 2*dx
        end if
        D = D + 2*dy

Rasterization

• How to determine the pixel values from complex 2D or 3D objects?
  • May not be a one to one pixel mapping!
• Rasterization is the process of finding pixel values for all the primitives (e.g. triangles) being rendered
  • Converts 2D vertices (projected onto the screen) with $z$-values (depth) and various shading information into pixels on screen

Rasterizing Triangles

1. Triangle Setup
   • Differentials, edge equations, and other data for triangles are computed
2. Triangle Traversal
   • Find what pixels centers are contained in each triangle
   • Perspective-corrected interpolation
3. Pixel Shading
   • Per-pixel shading is done, including texturing
4. Merging
   • Pixel information is stored in a colour buffer
   • Visibility usually resolved with $z$-buffer (depth)

Visible Spectrum

• Light is a wave
• Visible light (by humans) has a wavelength between roughly 400 nm and 700 nm (depends on individual)
• Colour is dependent on wavelength

What is colour anyway?

 $$\text{illumination} \cdot \text{reflection} = \text{stimulus}$$ 

• Colour is qualia (individual instance of subjective concious experience)
• Basic mechanics
  1. Light hits an object (illumination)
  2. Light is then reflected by the object (reflectance)
  3. Reflected light is then seen by the observer (stimulus)
• Colour depends on illumination and reflectance!
• Stimulus is processed by the eye
  • Dedicated colour cone cells react to light waves

The Human Eye

Colour Cone Cells

Colour Cone Spectrum

• Three types of colour cone cells (Short, Medium, and Long wavelengths)
  • Colourblind people have fewer!
• Cone response is $\int_\lambda \text{stimulus}(\lambda)\,\text{response}(\lambda)\, d\lambda$
• (There also are rod cells for nightvision)

Trichromatic Theory

• Idea that we have three colour receptors is known as Trichromatic Theory
• Proposed by Thomas Young in 1802
• Further developed by Hermann von Helmholtz in 1850
• Proven by James Maxwell in 1857 with linear algebra
• Existence of cone cells first shown in 1956 and validated in 1983

Colour Reproduction

• Three primary colours are enough to reproduce all colours

CIE

• Commission Internationale de l’éclairage (CIE) or International Commission on Illumination
• Establish in 1913
• Objective to try to establish standards regarding colour and illumination
• For colours, do human experiments to create new standards (CIE Standard Observer)
  • Choose 3 primary colours (453.8 nm, 546.1 nm, 700 nm)
  • Have to reproduce reference colour with 3 primary colours
  • Create colour matching functions
• Notable standards
  • CIE 1931 XYZ
  • CIE 1931 RGB
  • CIE 1976 LAB

CIE 1931 colourspace

CIE 1931 XYZ

• Created in 1931, before knowing how human cone cells respond
• Designed so that Y is luminance (brightness)
• X and Z are chrominance (colour)

 $$\begin{bmatrix} X \\ Y \\ Z \end{bmatrix} =
\begin{bmatrix}
1.91020 & -1.11212 & 0.20191 \\
0.37095 & 0.62905  & 0       \\
0       & 0        & 1.00000 \\
\end{bmatrix}
\begin{bmatrix} L \\ M \\ S \end{bmatrix}_\text{HPE}$$ 

RGB Colourspaces

• Can be seen as a linear transform of the CIE 1931 XYZ colourspace
• Depending on the reference white and calibration different transforms arise
• Some popular ones are:
  • CIE 1931 RGB
  • sRGB (1996)
  • Adobe RGB (1998)

 $$\begin{bmatrix} R \\ G \\ B \end{bmatrix}_\text{CIE} =
\begin{bmatrix}
  2.3706743 & -0.9000405 & -0.4706338 \\
 -0.5138850 &  1.4253036 &  0.0885814 \\
  0.0052982 & -0.0146949 &  1.0093968 \\
\end{bmatrix}
\begin{bmatrix} X \\ Y \\ Z \end{bmatrix}$$ 

 $$\begin{bmatrix} R \\ G \\ B \end{bmatrix}_\text{sRGB} =
\begin{bmatrix}
  3.2404542 & -1.5371385 & -0.4985314 \\
 -0.9692660 &  1.8760108 &  0.0415560 \\
  0.0556434 & -0.2040259 &  1.0572252 \\
\end{bmatrix}
\begin{bmatrix} X \\ Y \\ Z \end{bmatrix}$$ 

CIE 1976 LAB

• Also known as CIE Lab* colourspace
• Based on standard CIE observer model
• Lightness (L*) defines black at 0 and white at 1
• a* axis is green-red opponent colours and unbounded
• b* axis is blue-yellow opponent colours and unbounded
• Approximates human vision better than XYZ

CIE 1976 LAB Manifold

Hue Saturation Value (HSV)

• Developed by computer graphics researchers in the 1970s
• Hue is circular with red at 0°, green at 120°, and blue at 240°
• Saturation determines colour pureness with 0 being white and 1 being pure colour
• Value determines brightness

Color Quantization Gamma

• Human visual system is sensitive to ratios
  • Is this grey twice as bright as that one?
• Linear encoding has lots of information between 128 and 255, but none between 1 and 2
• Ideal encoding? logarithm
  • Problem: value at 0
  • Solution: gamma mapping $x^\gamma$
• Often $\gamma = 2.2$ is used (anything between 1.5 and 3.0 should be good enough for human eyes)
• Maximize signal-to-noise ratio

Working in Linear Colourspace

• For lighting, it is sometimes beneficial to work in a linear colourspace
  • We want an RGB value of $(1, 0, 0)$ to be twice as red as $(0.5, 0, 0)$
  • With the screen gamma mapping ($\gamma=2.2$) it becomes $(1, 0, 0)$ and $(0.5^{2.2}, 0, 0) = (0.217, 0, 0)$ resulting in a 4.5 times difference!
• This can be done by undone by applying $x^{(\frac{1}{\gamma})}$ before passing the colour to the screen
• In GLSL (we will see this next week):

  vec4 colour = process_scene();
  // below is a bad but fast approximation of sRGB gamma correction
  colour.rgb = pow( colour.rgb, vec3(1.0/2.2) ); // also possible with GL_FRAMEBUFFER_SRGB
  gl_FragColor = colour;

• Note: make sure textures aren’t getting converted twice such as setting the GL_SRGB or GL_SRGB_ALPHA internal format in OpenGL, which converts to linear, or do it manually with $x^\gamma$

More Accurate sRGB Gamma Correction

• In reality the transformations are a bit more complicated: sRGB
• sRGB (linear) to sRGB (gamma corrected)

   $$C_\gamma = \begin{cases}
  12.92 C_\text{linear}, & C_\text{linear} \leq 0.0031308 \\
  (1 + 0.055) C_\text{linear}^{\left(\frac{1}{2.4}\right)}-0.055, & \text{else} \\
  \end{cases}$$ 

  col.rgb = mix( 1.055*pow(col.rgb,vec3(1.0/2.4))-0.055,
                col.rgb*12.92, step(col.rgb, vec3(0.0031308)));

• sRGB (gamma corrected) to sRGB (linear)

   $$C_\text{linear} = \begin{cases}
  \frac{C_\gamma}{12.92}, & C_\gamma \leq 0.04045 \\
  \left(\frac{C_\gamma+0.055}{1.055}\right)^{2.4}, & \text{else} \\
  \end{cases}$$ 

  col.rgb = mix( pow((col.rgb+0.055)/1.055,vec3(2.4)),
                col.rgb/12.92, step(col.rgb, vec3(0.04045)));

Better Fast Approximation

• Reference: sRGB approximations
• sRGB (linear) to sRGB (gamma corrected)

  col.rgb = max(1.055 * pow(max(col.rgb, vec3(0.0)), vec3(0.41666666)) - 0.055, vec3(0.0));

• sRGB (gamma corrected) to sRGB (linear)

  col.rgb = col.rgb * (col.rgb * (col.rgb * 0.305306011 + 0.682171111) + 0.012522878);

Transformations

There are three basic classes of transformations (irregardless of dimensions):

1. Rigid Body: preserve distances and angles
   • Examples: translation, and rotations.
2. Conformal: preserves angles
   • Examples: translation, rotations, and uniform scaling
3. Affine: preserves parallelism (lines remain lines)
   • Examples: translation, rotations, uniform scaling, shear, and reflection

Translation

Translation by vector $\bm{t} = (t_x, t_y)$:

 $$\begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} p_x \\ p_y \end{bmatrix} +
\begin{bmatrix} t_x \\ t_y \end{bmatrix}$$ 

vec2 transform( vec2 p ) {
   return p + 0.5*cos(u_time);
}

Rotation

Rotation by $\theta$ around origin $(0,0)$:

 $$\begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix}$$ 

vec2 transform( vec2 p ) {
   float c, s;
   c = cos(u_time);
   s = sin(u_time);
   /* Have to invert because we are not moving object but camera! */
   return inverse(mat2(c,s,-s,c)) * p;
}

Scaling

Scaling by $(s_x, s_y)$:

 $$\begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} s_x & 0 \\ 0 & s_y \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix}$$ 

vec2 transform( vec2 p ) {
   vec2 s = vec2( cos(u_time), sin(u_time) )*0.5 + 1.0;
   /* Have to invert because we are not moving object but camera! */
   return inverse(mat2(s.x,0,0,s.y)) * p;
}

Shearing

Shearing on the $x$-axis by scalar $h$:

 $$\begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} 1 & h \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix}$$ 

vec2 transform( vec2 p ) {
   float h = cos(u_time);
   /* Have to invert because we are not moving object but camera! */
   return inverse(mat2(1,0,h,1)) * p;
}

Reflection

Reflection across the $y$-axis.

 $$\begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix} =
\begin{bmatrix} -p_x \\ p_y \end{bmatrix}$$ 

Affine Transformation

In more general, an affine transformation can be written as:

 $$\bm{q} = \begin{bmatrix} q_x \\ q_y \end{bmatrix} =
\begin{bmatrix} a & b \\ c & d \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix} +
\begin{bmatrix} t_x \\ t_y \end{bmatrix} = A \bm{p} + \bm{t}$$ 

and have the following properties:

1. The inverse of an affine transform is affine, assuming it exists.
2. Lines and parallelism are preserved under affine transformations.
3. Given a closed region under an affine transformation, the area is scaled by $\det(A)$.
4. A composition of affine transformations is still affine.

Homogeneous Coordinates 1

Affine transformations defined as $\bm{q} = \bm{A} \bm{p} + \bm{t}$ are tedious to compose and need lots of book keeping. Can we represent them directly with matrices?

Answer: homogeneous coordinates

We do the following rewrite:

 $$\bm{q} = \begin{bmatrix} a & b \\ c & d \end{bmatrix}
\begin{bmatrix} p_x \\ p_y \end{bmatrix} +
\begin{bmatrix} t_x \\ t_y \end{bmatrix} =
\begin{bmatrix} a & b & t_x\\ c & d & t_y\end{bmatrix}
\begin{bmatrix} p_x \\ p_y \\ 1\end{bmatrix} =
\begin{bmatrix} \bm{A} & \bm{t} \end{bmatrix}
\begin{bmatrix} \bm{p} \\ 1\end{bmatrix} = \bm{\hat{p}}$$ 

Note that all points $(\alpha \bm{p}, \alpha)$ represent the same point for $\alpha \neq 0$

Homogeneous Coordinates 2

What if we want $\bm{\hat{q}} = (\bm{q}, 1)$?

 $$\bm{\hat{q}} =
\begin{bmatrix} a & b & t_x\\ c & d & t_y \\ 0 & 0 & 1\end{bmatrix}
\begin{bmatrix} p_x \\ p_y \\ 1\end{bmatrix} =
\begin{bmatrix} \bm{A} & \bm{t} \\ \bm{0}^\intercal & 1\end{bmatrix}
\bm{\hat{p}} =
\bm{H} \bm{\hat{p}}$$ 

To get $\bm{q}$ from $\bm{\hat{q}}$ we simply divide by the last component of $\bm{\hat{q}}$ and discard it.

Transformations can now be composited simply by multiplication. Given two homogeneous matrices $H_1$ and $H_2$ we can compose them as:

 $$\bm{\hat{q}} = \bm{H_2} \bm{H_1} \bm{\hat{p}}$$ 

Note that matrix multiplication is not commutative: the order matters!

Composing Transforms

 $$\bm{H_1} =
\begin{bmatrix} \cos(t) & -\sin(t) & 0 \\ \sin(t) & \cos(t) & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad
\bm{H_2} =
\begin{bmatrix} 1 & 0 & 0.5 \cos(t) \\ 0 & 1 & 0.5 \cos(t) \\ 0 & 0 & 1 \end{bmatrix}$$ 

vec2 transform( vec2 p ) {
   float c, s;
   c = cos(u_time);
   s = sin(u_time);
   mat3 R = mat3( c,  s, 0,
                  -s, c, 0,
                  0,  0, 1 );
   c *= 0.5;
   mat3 T = mat3( 1, 0, 0,
                  0, 1, 0,
                  c, c, 1 );
   /* Have to invert because we are not moving object but camera! */
   vec3 q = inverse(T * R) * vec3(p,1);
   return q.xy / q.z;
}

vec2 transform( vec2 p ) {
   float c, s;
   c = cos(u_time);
   s = sin(u_time);
   mat3 R = mat3( c,  s, 0,
                  -s, c, 0,
                  0,  0, 1 );
   c *= 0.5;
   mat3 T = mat3( 1, 0, 0,
                  0, 1, 0,
                  c, c, 1 );
   /* Have to invert because we are not moving object but camera! */
   vec3 q = inverse(R * T) * vec3(p,1);
   return q.xy / q.z;
}

Minimal HTML Canvas Example

<html>
<head>
 <title>Canvas Demo</title>
 <script>
   let canvas;    // DOM object corresponding to the canvas
   let graphics;  // 2D graphics context for drawing to the canvas
   // Drawing function
   function draw() {
      graphics.fillText("Hello World", 50, 100 );
   }
   // Initialization function
   function init() {
      canvas   = document.getElementById("myCanvas");
      graphics = canvas.getContext("2d");
      draw(); // Draw onto the canvas
   }
   window.onload = init; // Runs when window is loaded
 <／script>
</head>
<body>
 <canvas width="800" height="600" id="myCanvas"></canvas>
</body>
</html>

Shapes

Paths

Path Examples 1

• Note that you can use subpixels
• Drawing a single line segment from $(100,200)$ to $(300,200)$:

  graphics.beginpath();
  graphics.moveTo(100.5,200.5);
  graphics.lineTo(300.5,200.5);
  graphics.stroke()

• Drawing a filled circle

  graphics.beginPath();
  graphics.arc(200,300,100,0,2*Math.PI);
  graphics.fill();

Path Examples 2

• Drawing an octagon

  graphics.beginPath();
  graphics.moveTo(200,300);
  for (let i=1; i<8; i++) {
   let angle = (2*MathPI)/8*i;
   let x = 200 + 100*Math.cos(angle);
   let y = 400 + 100*Math.sin(angle);
   graphics.lineTo(x,y);
  }
  graphics.closePath();
  graphics.fill();

Stroke and Fill

• It is possible to change the line properties directly

  graphics.lineWidth = 2.5; // Set the line width (in pixels)
  graphics.lineCap = "round"; // Set appearance of endpoints
  // Possible lineCap are "round", "square", or "butt"
  graphics.lineJoin = "round"; // Sets how lines are joined
  // Possible lineJoin are "round", "bevel", or "miter"
  graphics.fillStyle = "rgb(200,200,255)"; // Set fill colour
  graphics.strokeStyle = "#0070A0"; // Set stroke colour
  // Also possible to set font as shown below
  graphics.font = "2cm monospace";
  graphics.font = "bold 18px sans-serif";
  graphics.font = "italic 150% serif";

Transformations

Transformation Example

graphics.save();
graphics.translate(x,y);
graphics.scale(sx,sy);
graphics.beginPath();
graphics.arc(0, 0, 1, 0, 2*Math.PI );
graphics.restore();
graphics.stroke();

Images

• Images can be defined in HTML as:

  <img src="image.png" width="400" height="300" id="myImage">

• The image can be accessed from javascript with

  image = document.getElementById("myImage");

• The following API can then be used to draw it on a canvas:
  • graphics.drawImage(image,x,y,w,h): draws an image with upper-left corner at $(x,y)$. w and h are optional parameters to specify the width and height

Summary

• HTML Canvas allow for arbitrary 2D drawing
• Lots of function and fairly rich API (MDN Canvas API)
• Canvases will be also used for WebGL