Was everything Brown saw in "Young Goodman Brown" real or imaginary?

Even though Hawthorne leaves the interpretation of Goodman Brown's experience up to the reader, there are various elements that suggest Brown's experience was indeed imaginary. The magical nature of the enigmatic fellow traveler's staff, the ominous black cloud that follows Brown through the forest, the diverse congregation participating in the Black Mass, and the sudden disappearance of the deacon, minister, and Goody Cloyse suggest that Brown's experience was imaginary. The fact that Brown also uses the traveler's staff to fly through the forest to participate in the Black Mass, where Faith is suddenly present, also suggests that his experience was imaginary.
Regardless of whether Brown's experience was real or imaginary, the outcome of his loss of faith negatively impacts the remainder of Brown's life. After waking up in the forest, Brown becomes suspicious of his community's religious leaders and recognizes them has debased hypocrites who hide their sins and attempt to conceal their wicked actions. Brown lives the remainder of his life as a distant, cynical man who has abandoned all hope and faith in humanity. Through Brown's experience, Hawthorne comments on the nature of faith, religious hypocrisy, loss of innocence, and humanity's inherent nature.

The short answer is that we don't really know whether what Goodman Brown saw was real or if he simply imagined it. At the witches' meeting, he tells his wife, Faith, to resist the Devil, but 

Whether Faith obeyed, he knew not. Hardly had he spoken, when he found himself amid calm night and solitude, listening to a roar of the wind, which died heavily away through the forest. . . .  Had Goodman Brown fallen asleep in the forest, and only dreamed a wild dream of a witch-meeting? Be it so, if you will.

As soon as he tells Faith to resist, the fiery and fiendish scene around him disappears, and Brown stands in the cool, calm forest alone. The narrator says that we are free to believe that Brown had only fallen asleep and dreamed everything he saw. But why? Doesn't it matter whether or not it was real?
Brown lives the remainder of his life as though it were real, as though all of it did really happen. When he sees Faith, happy to see him, he "looked sternly and sadly into her face, and passed on without a greeting." He became a "stern, a sad, a darkly meditative, a distrustful, if not a desperate man . . . from the night of that fearful dream." He cannot stand to hear the congregation sing, to listen to the minister preach. He even "shrank from the bosom of Faith." In other words, he has lost his religious faith. Brown can no longer find solace in the faith he had assumed would simply be there waiting for him to pick up again after his one last night of sin. He had turned his back on God even before he met the Devil in the forest. He had thought to himself: "after this one night, I'll cling to [Faith's] skirts and follow her to Heaven." However, this is not how faith works; one must work, actively and constantly, to abide by God's laws—one cannot simply decide to ignore them one moment and then resume them the next. What Brown saw or did not see in the forest is not important. What is important is that he decided to go into the forest, for some dark intent, in the first place.