I actually completely agree with you and take much of the same approach when it comes to both cities. I used to live in the TL and really loved my time in San Francisco. Each city has its own style and I think the attitude is very different, although I will say that most San Franciscans I've met spend a lot of time hating LA and most LA people I've met think SF is a great place to visit. I've been in situations where my LA friends were blatantly shunned by SF residents who think their shit doesn't shit because of their environmentally safe, vegan, bike riding, peace loving attitudes, which I don't know...just feels like a bunch of horse shit to me sometimes.
SF tends to be a bit too politically charged...and well, it's supposed to be, it's San Francisco. Also, there's places like *cough cough* the Marina that is just as bad as *cough cough* Hollywood/Downtown hipsters.
I have great friends in SF and I go there often. The guy who introduced me to Bukowski lives by Dolores Park. Nothing like getting super baked and hanging out in Dolores Park on a Saturday afternoon.