I'm attempting all my germinations under grow lights. I have one of those 4 shelf units that seem to do pretty well. I get almost 100% germination rates out of my veggies, and about 75-80% on my more 'exotic' herbs, including the Costmary.
One thing I do to that's a little different than the text book wisdom, is that I mix some worm castings in with the peat pellets in my growing trays. Provides some early nutrients to the seedlings. Seems to have improved my survival rate on seedlings.
Haven't transplanted the Costmary yet. Will keep group informed as I go along.
I have grown them from seed. You can surface sow and and start at room temp, but as I remember, I put used Norm Deno's Outdoor Treatment to get them started--sprinkled them on a paper towel that had been wet and wrung out (I always add some kelp to the water), fold the towel up, pressing gently, and put in a cheap plastic baggie, left unsealed, to put outside in a sheltered area, like an unheated garage. I put mine in a metal cabinet I have out on my covered patio. Then check for germination in the spring. When they germinated, I just tore pieces off the paper towel and gently pressed them into my planting soil. The roots grew right through the paper towels.
With perennials, I generally use Outdoor Treatment even if the directions don't necessarily call for cold stratification. They just seem to germinate more readily that way.