Transform scene to camera orientation

Assumptions: the camera is at R0 = (1,1,1). The three orthogonal vectors are the camera axes: u = (1,2,−3) (camera x), v = (−5,4,1) (camera y) and n = (1,1,1) (camera z, the viewing direction). Check of orthogonality and handedness:

• u·v = −5 + 8 − 3 = 0, u·n = 1 + 2 − 3 = 0, v·n = −5 + 4 + 1 = 0
• u × v = (14, 14, 14) = 14·n, so (u, v, n) is a right-handed set.

Step 1: unit vectors

• |u| = √14 = 3.742 → u = (0.2673, 0.5345, −0.8018)
• |v| = √42 = 6.481 → v = (−0.7715, 0.6172, 0.1543)
• |n| = √3 = 1.732 → n = (0.5774, 0.5774, 0.5774)

Step 2: translate the camera to the origin

T = | 1 0 0 -1 |
    | 0 1 0 -1 |
    | 0 0 1 -1 |
    | 0 0 0  1 |

Step 3: rotate the camera axes onto the world axes (the rows are u, v, n)

R = |  0.2673  0.5345 -0.8018 0 |
    | -0.7715  0.6172  0.1543 0 |
    |  0.5774  0.5774  0.5774 0 |
    |  0       0       0      1 |

Step 4: M = R · T. The last column is −R·(1,1,1) = −(u·R0, v·R0, n·R0). Here u·R0 = 0, v·R0 = 0 and n·R0 = 1.732.

M = |  0.2673  0.5345 -0.8018   0      |
    | -0.7715  0.6172  0.1543   0      |
    |  0.5774  0.5774  0.5774  -1.7321 |
    |  0       0       0        1      |

Every world point P is described in camera coordinates as P' = M·P. Check: the camera position (1,1,1) maps to (0,0,0), and the point (2,2,2), one step along n, maps to (0, 0, 1.732).

If the vectors are instead taken in the given order as the camera x, y, z axes ((1,1,1), (1,2,−3), (−5,4,1), which is also right-handed), the same method simply reorders the rows: the rows of M become (0.5774, 0.5774, 0.5774, −1.7321), (0.2673, 0.5345, −0.8018, 0) and (−0.7715, 0.6172, 0.1543, 0).

Answer: M above.