Rotation to change 9x²−2√3xy+7y²=10

We want the angle θ for which 9x² − 2√3 xy + 7y² = 10 becomes 3X² + 5Y² = 5 after rotation.

Step 1: principal values

The coefficients of X² and Y² after rotation are the roots of λ² − (a + b)λ + (ab − h²) = 0 with a = 9, b = 7, h = −√3:

λ² − 16λ + (63 − 3) = 0 → λ² − 16λ + 60 = 0 → λ = 6, 10

So the reduced form is 6X² + 10Y² = 10 or 3X² + 5Y² = 5, as required. For this to happen, the X-axis must be along the eigen-direction of λ = 6.

Step 2: angle

For a rotation through θ, the coefficient of X² is

a cos²θ + 2h sinθ cosθ + b sin²θ = 9cos²θ − 2√3 sinθ cosθ + 7 sin²θ.

We need this to equal 6. Also the XY term must vanish: tan 2θ = 2h/(a − b) = −2√3/2 = −√3, so 2θ = 120° (or −60°), giving θ = 60° or θ = −30°.

Test θ = 60° (cos = 1/2, sin = √3/2):

9(1/4) − 2√3(√3/4) + 7(3/4) = 9/4 − 6/4 + 21/4 = 24/4 = 6 ✓

(Test θ = −30° gives 10, which would produce 5X² + 3Y² = 5, not the required form.)

Answer: the axes must be rotated through 60° (anticlockwise).

Verification

With x = (X − √3 Y)/2, y = (√3 X + Y)/2, substitution gives 6X² + 10Y² = 10, i.e. 3X² + 5Y² = 5.