I think that UUlace is on the right track. There is azalea scale in which cottony 1/4 -1/2 masses can be seen on the bottom of the leaves at one stage and in spring you may actually see the insects crawling along the branch ends and hiding on seed pods and along end branches. later on near Summer you can see small yellow spots on the leaves where the insects have sucked the leaf juices out. Malathion can be effective when you can see the larvae moving along the stems. but for long term control you can use a drench of Orthene in Spring( read the directions and apply carefully) at the base of the plant where the roots extend to. This is a systemic insecticide which is taken up by the entire plant and seems to give good control but takes a while to be effective ( month or more)
There is also a Maple leaf scale which invades azaleas and has similiar effects and can be controlled in the same above ways
Still a 3rd possibility is Lace bug which gives a black sooty deposit under the leaves but no yellow spots on the leaves. Again the controls are the same
You win the prize - it was scale. Luckily it looks like a drench of some soapy water, a good spray and Neem oil have been successful so far. I'm an organic nut and darwin-based - if it cant survive it will be replaced!