Yeah I feel like this runs into the typical problem: you need so much rubber that the weight of the band itself is much much much higher than that of the rocket. So you need to add more rubber band for the rubber band, for which you need to add more rubber band again…

The same as why you can’t build infinitely high buildings. The materials are just to heavy compared to their strength