What do you think Silas meant when he told Bod, "Wherever you go, you take yourself with you"? Do you agree with this observation?

What Silas means here is that we have more control over who we are than our surroundings. Let me explain.
This particular conversation between Silas and Bod is an interesting one. During the exchange, Bod asks Silas what the far corner of the graveyard holds. Silas answers that it is unconsecrated land. This land is the burial place of criminals, those who died by way of suicide, and others who do not belong to the "faith" (presumably the Christian faith).
Upon hearing this, Bod inquires whether people who die on unconsecrated ground are "bad people." Silas tells him no; it all depends upon one's perspective. In the olden days, a thief could be hanged for stealing a shilling. Also, there are others who decide that they would better off dead.
Bod then inquires whether people are happier if they decide to end their lives themselves. Here is Silas's answer:

Sometimes. Mostly, no. It’s like the people who believe they’ll be happy if they go and live somewhere else, but who learn it doesn’t work that way. Wherever you go, you take yourself with you. If you see what I mean.

What Silas means is that we have more control over our happiness than we think. Moving from one place to another does not necessarily guarantee greater happiness. Instead, it is how we approach or think about life that makes a difference. For example, are we willing to take specific steps to address the challenges in our lives? In the meantime, will we accept that some things are beyond our power to change? Essentially, Silas is saying that our perspective about life and its challenges is more consequential than we think.