Why would you want to counteract it? Plants need nitrogen. Unless you've dumped excessive amounts of fertilizer in the area you shouldn't have any problems. If you did dump way too much fertilizer there, if it was solid stuff I'd scrape up as much excess as you can and then water really thoroughly to try and wash out some more. If it was liquid fertilizer then you'll have to do the watering to flush it out.