Junctions created by coupling two superconductors via a semiconductor nanowire in the presence of high magnetic fields are the basis for detection, fusion, and braiding of Majorana bound states. We study NbTiN/InSb nanowire/NbTiN Josephson junctions and find that their critical currents in the few mode regime are strongly suppressed by magnetic field. Furthermore, the dependence of the critical current on magnetic field exhibits gate-tunable nodes. Based on a realistic numerical model we conclude that the Zeeman effect induced by the magnetic field and the spin-orbit interaction in the nanowire are insufficient to explain the observed evolution of the Josephson effect. We find the interference between the few occupied one-dimensional modes in the nanowire to be the dominant mechanism responsible for the critical current behavior. The suppression and non-monotonic evolution of critical currents at finite magnetic field should be taken into account when designing circuits based on Majorana bound states.