Transfer function and stability from [0 −1; −2 −3]

For $\dot{x} = Ax + Bu$, $y = Cx + Du$:

$$\frac{Y(s)}{U(s)} = C(sI - A)^{-1}B + D = \frac{C\,\operatorname{adj}(sI-A)\,B}{|sI-A|} + D$$ $$A = \begin{bmatrix} 0 & -1 \\ -2 & -3 \end{bmatrix},\quad B = \begin{bmatrix} 1 \\ 2 \end{bmatrix},\quad C = \begin{bmatrix} 1 & 0 \end{bmatrix}$$

Step 1: $(sI-A)$ and determinant

$$sI - A = \begin{bmatrix} s & 1 \\ 2 & s+3 \end{bmatrix},\qquad |sI-A| = s(s+3) - 2 = s^2 + 3s - 2$$

Step 2: Inverse

$$(sI-A)^{-1} = \frac{1}{s^2+3s-2}\begin{bmatrix} s+3 & -1 \\ -2 & s \end{bmatrix}$$

Step 3: Transfer function

$$\begin{aligned} (sI-A)^{-1}B &= \frac{1}{s^2+3s-2}\begin{bmatrix} (s+3) - 2 \\ -2 + 2s \end{bmatrix} = \frac{1}{s^2+3s-2}\begin{bmatrix} s+1 \\ 2s-2 \end{bmatrix} \\ \frac{Y(s)}{U(s)} &= \begin{bmatrix} 1 & 0 \end{bmatrix}\frac{1}{s^2+3s-2}\begin{bmatrix} s+1 \\ 2s-2 \end{bmatrix} = \frac{s+1}{s^2+3s-2} \end{aligned}$$

Stability

Characteristic equation $s^2 + 3s - 2 = 0$:

$$s = \frac{-3 \pm \sqrt{9 + 8}}{2} = \frac{-3 \pm 4.123}{2} = 0.562,\ -3.562$$

One pole lies in the right half of the s-plane ($s = +0.562$). Also the coefficients of the characteristic polynomial do not all have the same sign, which already fails the necessary condition for stability. So the system is unstable; its response contains a growing term $e^{0.562t}$.

Answer: $\dfrac{Y(s)}{U(s)} = \dfrac{s+1}{s^2+3s-2}$; unstable.