I have always wondered what this bush is. It blooms in early May, zone 5. The blooms are 1 1/4 inches and lightly scented. The bush has been trimmed back in the past, but right now it is about 8 feet tall. It is loaded with clusters of these flowers and it almost looks like snow after they fall.
I'd say a painting by Monet or perhaps Degas but not likely to be a Philadelphus that I know which is pure white flowers, at least 2 inches across and deeply scented. There are however hybrids and even a dwarf variety. One species, P. coulteri (the Rose Syringa) has petals stained purplish red. But the Common Mock Orange, P. coronarius, is a very fragrant white-flowered, fountain-shaped shrub.
No, there aren't any oranges. They are called Mock Oranges because the scent of the bloom is supposed to resemble orange blossoms, but that's where the similarity ends. They are not related to oranges and no fruit is produced.
I have a feeling that the plant you heard about being described as having inedible oranges was Poncirus trifoliata, a.k.a. Trifoliate Orange or Bitter Orange, although I suppose Mock Orange would be an acceptable name as well. This indeed is a citrus relative and does produce small "oranges" that are inedible although some claim they can be made into a marmalade. The fruit itself turns yellow when ripe and is covered with a sticky fuzz and have the distinctive odor of citrus. The interesting thing about this citrus relative is that it is hardy through zone 6 and possible protected areas of zone 5. Here is a link for some good pictures of the fruit. http://www.saalfelds.freeserve.co.uk/poncirus.htm