Text/Event Stream response doesnt work through Nginx

[spyhunter1280]

Select Topic Area

Question

Body

Hello, guys!
I am building the chatbot and built the backend as fastAPI. with /chat route, it returns streaming response. I deployed it on Ubuntu instance and configured nginx and connected to subdomain. But /chat route doenst work as streaming response. after finish all responses, it returned all responses at one time. This problem causes from nginx, when I was using it as IP address, it worked well as streaming response. I tried with several referenced related with nginx configuration, but I couldn't . Please help me with nginx configuration for Text/Event streaming

Thanks

[yusofy]

Hi @spyhunter1280 ,

Have you been able to resolve this? Running into the same issue.

[spyhunter1280]

I couldn't solve that problem, so I didn't use nginx.

[yusofy]

Figured it out with the help of colleagues . Running it in Kubernetes, I translated it back to nginx config:

location / {
    if ($request_method = 'OPTIONS') {
        add_header 'Content-Type' 'text/plain charset=UTF-8';
        add_header 'Content-Length' 0;
        return 204;
    }

    proxy_buffering off;
    proxy_http_version 1.1;
    proxy_max_temp_file_size 0;

    # Cors Preflight methods needs additional options and different Return Code
    chunked_transfer_encoding on;
    add_header 'Access-Control-Allow-Origin' $http_origin;
    add_header 'Access-Control-Allow-Credentials' 'true';
    add_header 'Access-Control-Allow-Methods' 'GET, PUT, POST, DELETE, PATCH, OPTIONS';
    add_header 'Access-Control-Allow-Headers' 'Authorization, Cache-Control, Content-Type, Keep-Alive, x-feature-events, responseType, stream';
    add_header 'Access-Control-Max-Age' 1728000;
}

Here are the annotations used:

  annotations:
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/proxy-max-temp-file-size: "0"
    nginx.ingress.kubernetes.io/configuration-snippet: |
       # Cors Preflight methods needs additional options and different Return Code
       chunked_transfer_encoding "on";
       more_set_headers 'Access-Control-Allow-Origin: $http_origin';
       more_set_headers 'Access-Control-Allow-Credentials: true';
       more_set_headers 'Access-Control-Allow-Methods: GET, PUT, POST, DELETE, PATCH, OPTIONS';
       more_set_headers 'Access-Control-Allow-Headers: Authorization, Cache-Control, Content-Type, Keep-Alive, x-feature-events, responseType, stream';
       more_set_headers 'Access-Control-Max-Age: 1728000';
       if ($request_method = 'OPTIONS') {
           more_set_headers 'Content-Type: text/plain charset=UTF-8';
           more_set_headers 'Content-Length: 0';
           return 204;
       }

The issue I had was that there was an extra nginx server in front I wasn't aware of that was blocking it.

The following had to be put in:

location / {
       chunked_transfer_encoding on;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-port 443;
        proxy_buffering off;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_pass XXXXX
    }