How does Holmes deduce that the man was bringing the goose to his wife as a peace offering?

In "The Blue Carbuncle," Sherlock Holmes is trying to establish how a famous stolen diamond ended up inside the belly of a goose. A man by the name of Mr. Peterson had helped the goose's original owner fight off a gang of street ruffians. But in the ensuing fracas, the owner of the goose—one Mr. Baker—left behind the goose as well as his hat. Peterson has no clues as to the man's identity, so enlists the help of Holmes, knowing how much he enjoys dealing with unusual cases.
As we might expect, Holmes quickly deduces a number of salient facts regarding the mystery man. Among other things, he concludes that the man must be married. Why? Well, for one thing, the goose had a little message attached to its leg which read "For Mrs. Henry Baker." Furthermore, Holmes deduces from the tatty state of the man's hat that relations are somewhat strained between him and his wife. If they weren't then she would take better care of his clothing. Instead, the hat is covered in dust, and clearly hasn't been brushed for a while. In Holmes's day, that was considered woman's work. The fact that the hat is of the highest quality, despite its age and slovenly condition, also indicates to Holmes that this Mr. Baker chap has come down in the world, socially and financially. Buying a Christmas goose for his wife is clearly his way of atoning for this.