From Kampot to Kep, a half hour ride by car, tuk tuk, or motorcycle. About $4.00 by motorcycle taxi, $12 for 4 people in a tuk tuk, or you can rent a motorcycle in Kampot for about $3-$5/day. ($3 in gas gets you a complete tour of Kampot and Kep.)
From Phnom Penh to Kep, taxis and minibuses everyday to Kampot and the Kep for about $4. A two to three hour ride. Also, 7am or so, and 1pm, busses from Phnom Penh for $5. Check for exact times and bus stations while in Phnom Penh. A private taxi costs about $35 for the trip from Phnom Penh.
From SihanoukVille, there is no direct service. You must go to Kampot first and arrange travel from there. However, a taxi from SihanoukVille is about $30 to Kampot, and $40 to Kep for the whole car.
To Vietnam, you can go through the Prek Chak (Cambodia) / Xaxia (Vietnam) border, about 45KM from Kep, but you need to get a visa beforehand. This is now the quickest route to get to Phu Quoc island in Vietnam. Motorcycle taxis are about $6 per person, and a car will cost about $35.
From Vietnam, you can usually get a Cambodian tourist visa good for 30 days at the border for $25. A $6 motorcycle trip, (probably one person with luggage per motorcycle), or a $35 car trip, which takes about a half hour. Enter Cambodia through th Prek Chak border.
Andy, at the Oasis Bar in Ha Tien, Vietnam, has a better description of how to get to and from Vietnam from Kep