A ski mitten with a waterproof zip across the fingers.
Unzip and your fingers are free for a phone, a buckle, a binding. Then closed again before your hands notice.