When does Rip Van Winkle begin to understand how long he has been away?

Rip Van Winkle has a long series of clues that he has been gone for quite some time, but he doesn't completely understand what has happened until he finds his daughter.
When Rip first wakes up, he discovers that his gun is rusted and his dog is gone, but he dismisses both of these. He assumes that someone has tricked him by swapping out his gun and that his dog has run off.
As he begins to come down the mountain, he begins to notice that the village is different, and that his own house and several other houses that he used to know are in varying states of decay.
Rip goes to the old pub that used to be named after King George III, but instead finds that it is called the General Washington. He doesn't know anyone in the crowd at the General Washington, and the people ask him what side he will vote on (which completely confuses him, since when he went to sleep New York was still governed by the king).
Rip questions the people about several of his old acquaintances and finds that they have died some time ago.
Rip asks if anyone knows Rip Van Winkle, and in response, the crowd points to a young man that looks almost like a young copy of himself (this is his son). Rip is still confused here, despite all the signs that he's been given that a lot of time has passed.
Finally, he hears someone in the crowd addressing a young boy, and "the name of the child, the air of the mother, the tone of her voice, all awakened a train of recollections in his mind" (Irving). Rip asks this woman's name, and discovers that it is his daughter. His daughter tells him what happened to him and to his wife, at which point Rip finally realizes what has occurred. After his daughter's story about him and what happened to his wife, Rip cries out "'I am your father!' cried he—'Young Rip Van Winkle once—old Rip Van Winkle now!—Does nobody know poor Rip Van Winkle!'"
Rip finally puts all of the pieces together, and tells his story to the villagers.
https://www.bartleby.com/195/4.html