Probing the center-of-mass of an ultracold atomic cloud can be used to measure Chern numbers, the topological invariants underlying the quantum Hall effects. In this work, we show how such center-of-mass observables can have a much richer dependence on topological invariants than previously discussed. In fact, the response of the center of mass depends not only on the current density, typically measured in a solid-state system, but also on the particle density, which itself can be sensitive to the topology of the band structure. We apply a semiclassical approach, supported by numerical simulations, to highlight the key differences between center-of-mass responses and more standard conductivity measurements. We illustrate this by analyzing both the two- and four-dimensional quantum Hall effects. These results have important implications for experiments in engineered topological systems, such as ultracold gases and photonics. 