EGGROLL is a novel optimization algorithm that utilizes low-rank perturbations to significantly enhance the scalability and efficiency of Evolution Strategies for training large neural networks, achieving up to a hundredfold improvement in training throughput while maintaining competitive performance.