While I appreciate the fine fall color and the inclination to pair this feature with Nyssa sylvatica, I think everything about your images and descriptions say Vaccinium sp.
Nyssa sylvatica will have more prominent buds - which your images do not portray. The small leaf size you describe also fits Vaccinium sp. much better than Nyssa sylvatica.
Your last picture of the bark iced it for me. I don't know any time that Nyssa sylvatica bark will ever look like that, but it is normal for Vaccinium sp.
Question then arises: which Vaccinium sp.? I'm not as familiar with all the available southern species in the Blueberry clan, but I'll venture Vaccinium stamineum for discussion purposes.
After using Google Images to get a look at quite a few Vaccinium stamineum leaves, It appears to me that in general the leaves are not the right ratio of length to width. In most of the images I looked at they appear longer compared to width than do the leaves I've posted.
Do any of that bunch attain heights like 9 feet... and now I've examined a few more of that same kind of tree nearby I see they can get 12-15 feet and I believe they are all youngish.
I've gotten better trunk pictures too, from a tree that appears a bit older than the previously posted trunk image. As you can see, its more shaggy looking than the previous post... (Sorry about the slight blurriness of 1 of them)