We have the regular ol' mimosa tree in our yard. (It came with the house and it is our only real shade tree.) It's got to be one of the worst things to have with cacti. The leaf litter is minor, absolutely nothing compared to the flower litter. It sticks in the spines and stays there for months unless you go around with tweezers and pick it out.
So far my Chocolate mimosa has not been too prolific at the flowering thing, but as it ages I have been concerned about that, too. I am still not sure that this form isn't just a brown form of the normal form, or if perhaps it has some differences (like being smaller, slower growing and less prolific in flowering).