Calculus: Early Transcendentals, Chapter 7, 7.1, Section 7.1, Problem 38

intt^3e^(-t^2)dt
Let x=t^2
dx=2tdt
intt^3e^(-t^2)dt=intxe^(-x)dx/2
=1/2intxe^(-x)dx
Now apply integration by parts,
If f(x) and g(x) are differentiable functions then,
intf(x)g'(x)dx=f(x)g(x)-intf'(x)g(x)dx
If we write f(x)=u and g'(x)=v, then
intuvdx=uintvdx-int((du)/dxintvdx)dx
So, let's take u=x , then u'=1
and v=e^-x
then v'=-e^-x
intxe^-xdx=x*int(e^-xdx)-int(1inte^-xdx)dx
=x(-e^-x)-int(-e^-x)dx
=-xe^-x+int(e^-x)dx
=-xe^-x+(-e^-x)
=-xe^-x-e^-x
:.intt^3e^(-t^2)dt=1/2(-xe^-x-e^-x)
substitute back x=t^2 and add a constant to the solution,
intt^3e^(-t^2)dt=1/2(-t^2e^(-t^2)-e^(-t^2))+C