Move them to San Diego? Really, mine come back every Spring after some 15 deg. nights here in NC. (they reside in a pile of dirt and worm casting with some leaves on top - nothing fancy) If you don't have a garage you can buy a plastic storage bin and put some of the worms and media in there and keep it inside. Tear up & moisten some newspapers as described in the vermicomposting threads if you need more media to fill the tub. Without us knowing how cold you expect or what your worms are in now, it's a little hard to guess what is even needed.
Worms are pretty resilient in my experience if you don't drown, bake or totally dry them out.
Rhaposdy, as our friend Paul says red worms are very resiliant, but are not gennerally thought of as garden worms. Since winters are probably not harsh in your area red worms, or red wigglers will probably remain well in leaf piles. As long as you leave a bottom layer of several inches where you are storing these leaves there will be an ample supply of worms to begin decomposing your next seasons leaves. As Paul mentions as well, indoor cultuing of red wigglers is quite easy, and a very good way of disposing of certain kitchen scraps. You might visit the vermiculture section here in DG for more information.