I'm definitely no expert with outdoor gardening, but with my indoor soil, I sterilize it first by pouring boiling water over it. Maybe some time before you're planting (if there's nothing else in there), you could do that to kill off a portion of the ant population?
More knowledgeable people can correct me if I'm wrong.
Ants are unlikely to do any harm in your vegetable garden and are an essential part of the ecosystem. Trying to eradicate ants in any yard is a pretty pointless exercise, and one you won't win. Best to live and let live, and avoid digging, standing on or planting directly in their nests.
Everyone is giving good advice re the ants, be very aware when they are working around where they are so your not getting bit, the ants will help to keep a FEW other bugs at bay as they want to defend their own territory.
The only time they become a problem is IF you have any diseases like mold etc on your plants, the ants will spread the spores as they crawl from plant to plant but look on the other hand, the breeze can do this also,There are several ant powders and sprays avaliable from garden stores that helps kill off the ants, these work by attracting the ants to the powder and the ants carry this back to the nest to feed the queen and in turn the worker ants and nursery ants feed the young which eventually kills all the ants BUT, be aware that the next ants will be waiting in line to move into your garden, there is no garden that is ant free, AND it all depends on what type of ants you have as some are more persistant than others.
Good luck. WeeNel.
IF you must use say an ant powder the best way to do this is, get a small saucer, place a very small sprinkle of powder onto the saucer, place it beside the ant nest, use a larger upturned plant pot over the saucer of powder and get a small stone to prop up one edge of the upturned flower pot to allow the ants to crawl into the sauser with the powder. the upturned pot needs to have a heavy stone placed on top to prevent this exposing the sauser and in turn allowing other animals to lick, sniff or ingest any ant poisen. and always wash hands after using any chemicals as soon as you put the cap back on the container and place it away safely.
Hope this helps you out. Good luck. WeeNel.