Thomas Cole's 1828 sketch below (now in the collection of the Detroit Institute of Arts) clearly shows that he understood the Willey House size and shape and it's relationship to the tall mountains that surrounded it. For more info on this sketch see this link.
Here are some details from the sketches.
The sketches match up quite well with one of the earliest known American photographs by Samuel Bemis.
The Bierstadt brothers included the view in a book of stereoviews that they printed.
This fascinating book had the stereo lenses built into the cover of the book.
Ironically, the sketches by Cole and Durand were done in the summer, so in his painting Cole is resetting the story into the autumn.