Rare signals carve out distinct subspaces in the latent space without dilution