Of what does Doodle seem to be most fearful in "The Scarlet Ibis"?

Doodle seems afraid of being left behind by his brother.
Perhaps, it is the fact that the narrator is the one who urges Doodle to make physical accomplishments that effects Doodle's dependency upon his brother. Then, too, the brother is made to take Doodle with him after his father fashions a cart in which to pull the small boy. Thus, having to transport Doodle everywhere with him, the narrator decides,

Doodle was my brother, and he was going to cling to me forever, no matter what I did, so I dragged him across the burning cotton field to share with him the only beauty I knew, Old Woman Swamp.

The ironic thing about Doodle's "clinging" is that it becomes his tragic undoing because the brother is what he himself characterizes as a slave to his pride, and he coerces the frail Doodle into learning how to swim and row a boat so that he will not seem so pitiful. Then, when a storm comes up one day that they are out on the water, 

The knowledge that Doodle's and my plans had come to naught was bitter, and that streak of cruelty within me awakened. 

The narrator races ahead of Doodle, who pleads for his brother to wait for him. When the brother's "childish spite evanesced," he finally stops. However, he does not hear Doodle, so he turns back and discovers that his frail "scarlet ibis" of a brother has died.