Socket.io falls back to polling (websocket gives 400) behind nginx proxy when remote but not local
I am using nginx 1.4.6 as a proxy for a Django app and its associated NodeJS app that uses socket.io 1.2.1 along with socketio-jwt 2.3.5
Locally, I am coding using a Vagrant instance (ubuntu 14.04.1) and remotely I am using a dedicated AWS EC2 instance - both configured using the same Vagrantfile and installation shell script to be very close to identical.
When running locally, my client connects to the nginx server which refreshes the connection and feeds it to the socket.io server implementation in Node. Everything works really well.
Remotely, I log the first connection from transport=polling
, which also instructs the client to upgrade to the websocket transport, then the second request with transport=websockets
, and then the third with transport=polling
.
While the app is still running, polling <websockets for many reasons and I hope I can figure out a way to sort this.
The second request is answered 400 Bad Request
. Running my Node application with DEBUG=* node index.js
gives the following output:
engine intercepting request for path "/socket.io/" +24s
engine handling "GET" http request "/socket.io/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1hcmNlbCIsIm9yaWdfaWF0IjoxNDE4MDU5ODg1LCJ1c2VyX2lkIjozLCJlbWFpbCI6IiIsImV4cCI6MTQxODE0NjI4NX0.2PE0TNRol9G4hby4OzQ-af2e0yjfgFAb-gQJF5tKWRwxWnFLv1NGp3Yo87UaqNaQceMW6KqzMIx2gLcRFnk09A&EIO=3&transport=polling&t=1418062322853-0" +0ms
engine handshaking client "6YDFOjBfvZ9UyutVAAAB" +1ms
engine:socket sending packet "open" ({"sid":"6YDFOjBfvZ9UyutVAAAB","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":60000}) +0ms
engine:polling setting request +0ms
engine:socket flushing buffer to transport +0ms
engine:polling writing " 0{"sid":"6YDFOjBfvZ9UyutVAAAB","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":60000}" +1ms
engine:socket executing batch send callback +1ms
socket.io:server incoming connection with id 6YDFOjBfvZ9UyutVAAAB +1.7m
socket.io:client connecting to namespace / +1.7m
socket.io:namespace adding socket to nsp / +1.7m
socket.io:socket socket connected - writing packet +1.7m
socket.io:socket joining room 6YDFOjBfvZ9UyutVAAAB +0ms
socket.io:client writing packet {"type":0,"nsp":"/"} +79ms
socket.io-parser encoding packet {"type":0,"nsp":"/"} +1.7m
socket.io-parser encoded {"type":0,"nsp":"/"} as 0 +0ms
engine:socket sending packet "message" (0) +79ms
socket id: "6YDFOjBfvZ9UyutVAAAB" connected to user: "marcel"
socket.io:socket joined room 6YDFOjBfvZ9UyutVAAAB +1ms
engine intercepting request for path "/socket.io/" +121ms
engine handling "GET" http request "/socket.io/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1hcmNlbCIsIm9yaWdfaWF0IjoxNDE4MDU5ODg1LCJ1c2VyX2lkIjozLCJlbWFpbCI6IiIsImV4cCI6MTQxODE0NjI4NX0.2PE0TNRol9G4hby4OzQ-af2e0yjfgFAb-gQJF5tKWRwxWnFLv1NGp3Yo87UaqNaQceMW6KqzMIx2gLcRFnk09A&EIO=3&transport=polling&t=1418062323048-1&sid=6YDFOjBfvZ9UyutVAAAB" +0ms
engine setting new request for existing client +1ms
engine:polling setting request +0ms
engine:socket flushing buffer to transport +1ms
engine:polling writing " 40" +38ms
engine:socket executing batch send callback +1ms
engine intercepting request for path "/socket.io/" +0ms
engine handling "GET" http request "/socket.io/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1hcmNlbCIsIm9yaWdfaWF0IjoxNDE4MDU5ODg1LCJ1c2VyX2lkIjozLCJlbWFpbCI6IiIsImV4cCI6MTQxODE0NjI4NX0.2PE0TNRol9G4hby4OzQ-af2e0yjfgFAb-gQJF5tKWRwxWnFLv1NGp3Yo87UaqNaQceMW6KqzMIx2gLcRFnk09A&EIO=3&transport=websocket&sid=6YDFOjBfvZ9UyutVAAAB" +1ms
engine bad request: unexpected transport without upgrade +0ms
engine intercepting request for path "/socket.io/" +120ms
engine handling "GET" http request "/socket.io/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1hcmNlbCIsIm9yaWdfaWF0IjoxNDE4MDU5ODg1LCJ1c2VyX2lkIjozLCJlbWFpbCI6IiIsImV4cCI6MTQxODE0NjI4NX0.2PE0TNRol9G4hby4OzQ-af2e0yjfgFAb-gQJF5tKWRwxWnFLv1NGp3Yo87UaqNaQceMW6KqzMIx2gLcRFnk09A&EIO=3&transport=polling&t=1418062323180-2&sid=6YDFOjBfvZ9UyutVAAAB" +1ms
engine setting new request for existing client +0ms
engine:polling setting request +0ms
The problem is with the line engine bad request: unexpected transport without upgrade +0ms
, but I don't get it. The nginx config definitely mentions the upgrade and it works on my vagrant machine.
The relevant part of the nginx configuration looks like this:
server {
...
location / {
...
}
location /socket.io/ {
proxy_pass http://127.0.0.1:4000/socket.io/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
}
}
The only difference I can run into between local and remote settings is that locally the client connects to localhost on two different ports (I proxy the Node app behind nginx, but the web app goes through Gulp) and remotely the Node app is in a different domain, behind port 80.
Any hints are greatly appreciated :)
source to share
I had exactly the same problem. Looking at some tcp entries, it looked like nginx didn't even get the incoming Upgrade header, so it didn't pipe it to node.js. (I suspect there is a cloud application made for this, but I'm not sure).
The way I fixed this is indeed a hack, but it works pretty well. Basically I modified the nginx config to look for a header Sec-Websocket-Key
and when it detects that it sets the Upgrade header to websocket
and the connection header to upgrade
. It works.
Configuration example:
map $http_sec_websocket_key $upgr {
"" ""; # If the Sec-Websocket-Key header is empty, send no upgrade header
default "websocket"; # If the header is present, set Upgrade to "websocket"
}
map $http_sec_websocket_key $conn {
"" $http_connection; # If no Sec-Websocket-Key header exists, set $conn to the incoming Connection header
default "upgrade"; # Otherwise, set $conn to upgrade
}
You must define these 2 blocks of the card in front of the block server {}
.
Now in your socket.io location add the following 2 lines:
proxy_set_header Upgrade $upgr;
proxy_set_header Connection $conn;
This sets the update headers and connections to the displayed values.
I hope this is the solution to your problem.
source to share