Yes, it’s true, you do. Left doesn’t really imply an up so much as it comes as a package with one, though. I’m not OP, but historically I had the same issue. I just didn’t automatically jump to “in is down, and I’m on the rim”, and instead was thinking about my actual physical left and right at that moment.