If you have access to aged manure (old, been laying around for awhile) use it liberally.
The best manure has been pushed into a pile, covered with a waterproof tarp and left alone for 4-6 months. That usually gets rid of the pathogens (weed seed, etc). If it is fresh it stinks, draws pests and is full of weed seeds. We get composted manure for $1,20 a bag, about a cubic foot. I put it down 6" deep when I can. Keeps the weeds to a minimum, retains moisture, feeds the plants and makes the earthworm population explode.
If you have to use concentrated (chemical) fertilizer, then you should apply it and cover it with some kind of mulch. That will decimate the earthworms, but, improve plant growth.
Worms are natures best tillers of the soil. Up and down thru the soil, making tunnels that air, water and nutrients flow. That is how gardening was designed to work on the planet.