Balancing four masses in one plane

For masses in one plane, balance requires $\sum mr = 0$ (vector sum). Take mass A along the x-axis ($0^\circ$) and find the resultant of the $mr$ vectors by resolving.

$$\sum H = \sum mr\cos\theta = 1.539\ \text{kg·m}, \qquad \sum V = \sum mr\sin\theta = 0.646\ \text{kg·m}$$

Resultant

$$R = \sqrt{(\sum H)^2 + (\sum V)^2} = \sqrt{(1.539)^2 + (0.646)^2} = 1.670\ \text{kg·m}$$ $$\tan\alpha = \frac{\sum V}{\sum H} = \frac{0.646}{1.539} \Rightarrow \alpha = 22.8^\circ \text{ (measured from A)}$$

Balancing mass

The balancing mass $m_b$ must cancel this resultant, so it acts opposite to it ($180^\circ$ away):

$$m_b r_b = R \Rightarrow m_b = \frac{1.670}{0.2} = 8.35\ \text{kg}$$ $$\theta_b = 22.8^\circ + 180^\circ = 202.8^\circ \text{ from mass A, in the same sense as B, C, D}$$

Answer: Balancing mass $= 8.35$ kg at 200 mm radius, at $202.8^\circ$ from mass A.