Although I haven't seen them growing for many years, this reminds me of mayhaw. It grows in swamps, sloughs and backwaters in parts of La. and Ms. I wouldn't have thought it would thrive in your area, but maybe so. One of you Cajun cousins, am I just too senile or could this be? If so, it makes jelly that is food for the gods.
I looked up mayhaw and the berries have more of a shape that a bell pepper would have, you know the indentations. Anyway, the berries on this tree are round, no indentation on the outside at all. I don't know if I'm making sense here. I hope I am