Challenge mode is distressing, but it really depends on the controls of the 3 minigames that you have to play. I unlocked Switch Mode after playing the two games that require the arrows keys (up-down, left-right) and the Shift button one. Just keep trying.