What did uneven wealth distribution have to do with Social Darwinism?

Social Darwinism, popularized by Herbert Spencer, took ideas Darwin developed about the animal kingdom in his book Origin of the Species and applied them to human society. Specifically, he glommed onto Darwin's theory of natural selection, which argued that the most viable species will survive and thrive, and the least viable die out. Spencer changed the theory somewhat, then used it as a "scientific" explanation for societal wealth.
Spencer came up with the term "survival of the fittest." He argued that the people in society who ended up with the most wealth did so because they were the fittest, or the best of human beings. This ideology said they inherently deserved their great wealth.
We can quickly see the flaws to this argument. It is a circular argument to say you are deserving of your wealth because you happen to have it. This argument ignores such obvious means of getting wealthy as luck or inheritance, which have nothing to do with "fitness." The person who happens to buy the $400 million lottery ticket may be a total loser who has never done anything in life but sit on the sofa eating potato chips: he just got lucky. People who inherit great wealth also are not necessarily people who flourish because of being fit. Some may be extraordinary, but more are likely ordinary people who just happened to pop out of the right womb. Nevertheless, Spenser's theory became popular because it justified the wealthy in having and holding onto their wealth.