We study the rheological behavior of suspensions of noncolloidal spheres in yield stress fluids (concentrated emulsions). These are good model systems for understanding, e.g., the rheology of fresh concrete or debris flows, and more generally, the behavior of particles dispersed in any nonlinear material. We use magnetic resonance imaging techniques to investigate the flows of these yield stress suspensions in a concentric-cylinder Couette geometry. We extend the theoretical approach of Chateau et al. [J. Rheol. 52, 489–506 (2008)], valid for isotropic suspensions, to describe suspensions in simple shear flows, in which an anisotropic spatial distribution of particles is induced by flow. Theory and experiments show that the suspensions can be modeled by a Herschel–Bulkley behavior of same index as their interstitial fluid. We characterize the increase of their consistency and their yield stress with the particle volume fraction / in the 0%–50% range. We observe a good agreement between the experimental variations of the consistency with / and the theoretical prediction. This shows that the average apparent viscosity of the sheared interstitial material is correctly estimated and taken into account. We also observe shear-induced migration with similar properties as in a Newtonian fluid, which we predict theoretically, suggesting that particle normal stresses are proportional to the shear stress. However, the yield stress at flow stoppage increases much less than predicted. We also show that new features emerge in the rheology of the yield stress fluid when adding particles. We predict and observe the emergence of a nonzero normal stress difference at the yielding transition. We observe that the yield stress at flow start can differ from the yield stress at flow stoppage, and depends on flow history. It is likely a signature of a shear-dependent microstructure, due to the nonlinear behavior of the interstitial fluid, which makes these materials different from suspensions in Newtonian media. This is confirmed by direct characterization of shear-rate-dependent pair distribution functions using X-ray microtomography. This last observation explains why the theory predictions for the consistency can be correct while failing to model the yield stress at flow stoppage: a unique microstructure was indeed assumed as a first approximation. More sophisticated theories accounting for a shear-dependent microstructure are thus needed.