I just realized this year that the blossoms of my Kentucky Coffee Trees have a delicious lemony fragrance. I guess they are usually too high up to notice because until I stuck my nose in them I had no idea.
They are not really that much to look at, but they do smell fine.
Don't give up if you have the room. I started my trees by just planting the 5 seeds I purloined from the Minnesota Arboretum and they are about 20 feet tall now. Trees grow faster than you would think when they are young.