Oliver didn’t mean to crash the party, but he ended up in a steamy threesome