Peach trees will often drop some fruit if there are too many for the tree to really support. The fact that the peaches you have are really small and some are falling suggests that the tree produced too many fruit and is getting rid of some. If you want larger fruits, I'd thin the tree yourself earlier in the year next year, that way the peaches that are left can get larger.
For the nectarine--did it bloom this year? No blooms = no fruit. If it bloomed but didn't fruit, I wonder if it needs a second variety around for cross-pollination? That is the case for some fruit trees but I don't know about nectarines specifically.
I don't know enough about persimmons to have any thoughts there, but one other question that comes to mind is whether all of your trees look healthy otherwise. There are cultural & disease issues that could also cause all the issues you mention. My thoughts on the nectarine & the peach hold true if the trees are looking healthy.