Calculus: Early Transcendentals, Chapter 7, 7.3, Section 7.3, Problem 5

You need to re-write the expression t^2 - 1 , such that:
t^2 - 1 = t^2(1 - (1/t)^2)
You need to use the following substitution, such that:
1/t = sin u => -1/(t^2) dt = cos u du => (dt)/(t^2) = -cos u du
u = arcsin (1/t)
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) = - int_(u_1)^(u_2) (cos u du)/(1/(sin u)*sqrt(1 - sin^2 u))
You need to use the basic trigonometric formula 1 - sin^2 u = cos^2 u, such that:
- int_(u_1)^(u_2) (cos u*sin u du)/(sqrt(1 - sin^2 u)) = - int_(u_1)^(u_2)
- int_(u_1)^(u_2) (cos u*sin u du)/(sqrt(cos^2 u)) = - int_(u_1)^(u_2) (cos u*sin u du)/(cos u)
Reducing like terms yields:
-int_(u_1)^(u_2) (sin udu) = -(-cos u)|_(u_1)^(u_2)
Replacing back the variable, yields:
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) =(cos (arcsin (1/t)))_(sqrt 2)^2
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) =cos (arcsin (1/2)) - cos (arcsin (1/sqrt 2))
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) = cos(pi/6) - cos(pi/4)
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) = sqrt3/2 - sqrt2/2
int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) = (sqrt3 - sqrt2)/2
Hence, evaluating the given integral yields int_(sqrt 2)^2 (dt)/(t^3*sqrt(t^2-1)) = (sqrt3 - sqrt2)/2.