The rebalancing ignores what population growth does to the ratio - each representative is responsible for more and more constituents over time. It’s just like teacher to student ratio, the higher the number the less effective they are at the job. The physical logistics of having a building that can’t seat a realistic number of representatives is something that could be figured out, especially with today’s tech.