Calculus: Early Transcendentals, Chapter 7, 7.1, Section 7.1, Problem 39

You need to use the substitution theta^2= t , such that:
theta^2 = t => 2theta d theta= dt => theta d theta= (dt)/2
Replacing the variable, yields:
int_(sqrt(pi/2))^(sqrt pi) theta^3(cos(theta^2)d theta = int_(t_1)^(t_2) t*cos t*(dt)/2
You need to use the integration by parts such that:
int udv = uv - int vdu
u = t => du = dt
dv = cos t => v = sin t
int t*cos t = t*sin t - int sin t dt
int t*cos t = t*sin t + cos t + C
Replacing back the variable, yields:
int_(sqrt(pi/2))^(sqrt pi) theta^3(cos(theta^2)d theta = (theta^2*sin (theta^2) + cos (theta^2))|_(sqrt(pi/2))^(sqrt pi)
Using the fundamental theorem of integration, yields:
int_(sqrt(pi/2))^(sqrt pi) theta^3(cos(theta^2)d theta = (pi*sin (pi) + cos (pi) - (pi/2)*sin(pi/2) - cos(pi/2))
int_(sqrt(pi/2))^(sqrt pi) theta^3(cos(theta^2)d theta = -1 - pi/2
Hence, evaluating the integral, using substitution, then integration by parts, yields int_(sqrt(pi/2))^(sqrt pi) theta^3(cos(theta^2)d theta = (1/2)(-1 - pi/2).