Why does Lyddie's family leave the farm?

The first of Lyddie's family to leave the farm is her father, Mr. Worthen. From the text, we can speculate that Mr. Worthen left to find better work opportunities. The text tells us that Mr. Worthen had been unlucky: his crops had been poor, and the bottom had fallen out of the wool market before he could make his money. So, Mr. Worthen left for financial reasons.
After the bear incident in chapter 1, Lyddie's mother, Mrs. Worthen, leaves for Uncle Judah's farm with two of the youngest children, Agnes and Rachel. Mrs. Worthen leaves because she believes that the farm would no longer be able to sustain her little family indefinitely.
Additionally, Mrs. Worthen also believes that the end of the world is upon humanity. She imagines that it would be best to congregate with others who share her beliefs. Although Lyddie thinks that her mother's beliefs are odd, she does not try to keep her mother and the younger children from leaving.
As for Lyddie and Charlie, the two eventually have to leave the farm. After their mother leaves, they receive a letter from her, informing them that she has leased the farm, pasture, and animals to Mr. Westcott to repay some family debts. To their dismay, their mother had also hired them out as workers at the tavern and the Bakers Mill.
So, it can be said that Lyddie's family mainly left the farm for financial reasons.