Bukowski insulted Ginsberg at a party after a reading in San Francisco, if I'm remembering correctly. Ridiculed him as having written only one great poem, "Howl," and nothing else of value after that. I suspect he was jealous of Ginsberg's fame.
(As always, it's early in the morning, memory sucks, and I could have garbled the story...)