There are a ton of good record shops in the City, particularly in the Haight-Ashbury area. Amoeba is there as Mick mentioned (definitely the largest) and a half dozen or so other smaller shops. Unfortunately scoring any original stuff will be tough, but I've had a bit of luck at some of the "boutique" shops. You could easily make an afternoon of checkin 'em all out.
If you're in the East Bay, Berkeley's Telegraph Ave. area is like a smaller version of the Haight-Ashbury ("vintage" clothes, head shops, and record shops).
One of my favorite record stores in SF is called Explorist, in the Mission. It's small but just a wonderful selection of jazz, folk, world, some reggae. It's one of those shops where everything is handpicked and in great taste- I usually walk out thinking I could have bought practically the entire store.
Enjoy your time in SF, and remember to always pay your parking meter! (In fact, utilize BART when practicable, driving can be a nightmare).