Thanks for explanation.
I've tried to use 1000000 baud rate and it seems good on both sides - I can send it from Linux and receive on Arduino.
With such speed of USB/Serial transfer it's possible to pass information for ~500 LEDs with 60 FPS.
I don't know yet if there will be problems with setting such amount of LEDs on Arduino side (seems like all works good with 60 LEDs I have here).
As I see common use of ESP32/ESP8266 is done via WLAN protocol, but I'm trying to stick to USB/Serial for now. Maybe I will be forced to switch to network communication if this path with USB/Serial is not possible for my final configuration.