I am concerned about the possibility of poisoning to animals or children from Colchicum autumnale. Does this plant pose a legitimate threat to their safety?
I have already decided to remove oleander. What about nandina, english ivy, lilly, boxwood, ligustrum. Am I being overly cautious? Should these plants be removed?
You're probably being a bit overly cautious--but only you know how cautious you need to be and what will make you comfortable. If you have kids/pets that tend to get into things and you tend to leave them unsupervised in the yard then it's probably best to play it safe and get rid of anything poisonous. Many poisonous plants though need to be consumed in a decent quantity before they'd cause serious harm, and most of them taste bad enough that probably nobody would eat enough to get to that point. The oleander I think was good to get rid of though because it is quite poisonous and also has caustic sap which could burn someone if they broke off a branch.
From the link you posted: "Colchicum plants have been mistaken by foragers for ramsons, which they vaguely resemble, but are deadly poisonous due to their colchicine content. The symptoms of colchicine poisoning resemble those of arsenic, and no antidote is known."
The OP is right to be concerned. However, I think most of the colchicine is in the corms, not in the flowers.
Animals tend to leave the plant alone. Unless the children like to nibble on pretty flowers, then there should not be any problem.
I would not worry about animals so much, their keen nose tends to keep them safe. Children are another story though, toddlers tend to put everything in their mouths. If they have access to the plants it might be a concern. Many plants are posionous to some degree and we can't dig them all out, best precaution is to supervise the little guys when they are in the garden and remove the worst offenders or those plants they might get access to.