Last year I made a rock garden, but I didn't have many plants, so when the weeds came both annual and rhizomatous perennials, I was able to control them, to some extent, with glyphosate, but this autumn has been very wet and the whole area is awash with tiny - mainly chickweed and groundsel - seedlings and I don't know how to extricate them from the surrounding plants... Also it is a bit late for glyphosate treatment.
I think you just need to pull them. There is an herbicide the kills just grass but wouldn't work with your weeds from the sounds of it. You might have to dig out the plants you want and remove/spray the weeds if its really bad. You could also sprinkle corn gluten meal in the spring & again in the fall to stop the seeds from germinating.
I'm fighting crown vetch - I'd cleared the area before planting. It took a few years of glyphosate & hand pulling but I thought I was successful. Now 5 years after planting I'm seeing some larger clumps coming in. I just spent a few hours pulling those yesterday but the roots are so deep and the area so rocky I know I didn't get them all.
Tammy, CV is a wonderful mulch - I purposely grew it, back in Missouri, to conserve water and keep weeds under control. Many states use it as a means of controlling weeds and preventing soil erosion along roads - wish it would grow here!