We study memories in a soft material that stays far from equilibrium: a non-Brownian suspension. The system has ‘amplitude memory’, where it can learn the strain amplitude of oscillatory shear and be readout later. We looked at how displacements, both in magnitude and type, orthogonal to oscillatory shear can affect memory retention besides leaving their own signatures in the microstructure. The study showed non-reciprocal motion degrades memory stored qualitatively and quantitatively different from reciprocal motion.