How did the townspeople finally solve the problem with the smell?

Miss Emily Grierson is effectively the aristocracy of the small town of Jefferson, so her life is both highly scrutinized by the townspeople and veiled by the regard in which the upper class is held. When a sickening smell arises from her house shortly after the disappearance of her fiancé, Homer Barron, the townspeople are conflicted about how to deal with the problem. On the one hand, the smell must be dealt with, because it is a public nuisance. On the other hand, Miss Emily has just been jilted by her lover, is presumably going through a difficult time, and is a woman. More than that, she is a "lady" from a family of high degree. Social mores in the town of Jefferson (and the South more generally) prohibit bluntly confronting Miss Emily with the problem.
The townspeople believe the source of the smell is a dead animal of some kind, and after making a few complaints to the mayor, Judge Stevens, the Board of Aldermen approach him and ask him to take action:

"It's simple enough," [a young alderman] said. "Send her word to have her place cleaned up. Give her a certain time to do it in, and if she don't..."

Judge Stevens is appalled by this breach of protocol:

"Dammit, sir," Judge Stevens said, "will you accuse a lady to her face of smelling bad?"

Unable to order Miss Emily to solve the problem herself, the townspeople take matters into their own hands, and a group of four men sneak onto her property under cover of darkness and spread powdered lime all around the house:

[After] midnight, four men crossed Miss Emily's lawn and slunk about the house like burglars, sniffing along the base of the brickwork and at the cellar openings while one of them performed a regular sowing motion with his hand out of a sack slung from his shoulder. They broke open the cellar door and sprinkled lime there, and in all the outbuildings.

This seems to resolve the issue—although not immediately—for "after a week or two the smell went away." Nobody knows until many years later that the corpse of Homer Barron was the source of the smell.