Either way works as long both are not big. My personal rule is if I’m wearing an oversized hoodie/jacket, I wear slim/skinny pants, and vice versa (baggy pants, fit shirt).