Streaming does not make the model faster. It moves the moment the user sees something from the end of generation to the first token, which for a 600-token answer is the difference between eight seconds of nothing and 400 milliseconds of something. The work is on your side: read the body as it a...
Source: [Dev.to](https://dev.to/multigrid/streaming-tokens-into-a-react-ui-2g59)