Many natural and industrial processes rely on constrained transport, such as proteins moving through cells, particles confined in nanocomposite materials or gels, individuals in highly dense collectives and vehicular traffic conditions. These are examples of motion through crowded environments, in which the host matrix may retain some glass-like dynamics. Here we investigate constrained transport in a colloidal model system, in which dilute small spheres move in a slowly rearranging, glassy matrix of large spheres. Using confocal differential dynamic microscopy and simulations, here we discover a critical size asymmetry, at which anomalous collective transport of the small particles appears, manifested as a logarithmic decay of the density autocorrelation functions. We demonstrate that the matrix mobility is central for the observed anomalous behaviour. These results, crucially depending on size-induced dynamic asymmetry, are of relevance for a wide range of phenomena ranging from glassy systems to cell biology.