The neighbor came to check on his wife on Valentine's Day and got intimate