GKE + WebSocket + NodePort 30s lost connections

I have a golang service that implements a WebSocket client using gorillas, which in this case is exposed to a Google Container Engine (GKE) / k8s cluster via NodePort (30002).

I have a manually created load balancer (i.e. NOT on ingress / k8s load balancing) with HTTP / HTTPS interfaces (i.e. 80/443) that route traffic to my GKE / k8s cluster nodes on port 30002.

I can get JavaScript implementation in WebSocket in browser (Chrome 58.0.3029.110 on OSX) to connect, update and send / receive messages.

I am logging ping / pongs in golang WebSocket client and everything looks good up to 30s within 30 seconds after connecting. My golang WebSocket client gets EOF / close 1006 (abnormal close) and my JavaScript code gets a closed event. As far as I can tell, neither my Golang code nor JavaScript initiates a WebSocket close.

In this case AFAIK doesn't really care about session affinity, but I've tried IP and cookie merge on a load balancer with long lasting cookies.

Also, this exact same set of k8s deployment / ordering / maintenance specs and golang maintenance code works fine on my k8s KOPS cluster on AWS via AWS ELB.

Any ideas as to which 30s forced closings might occur? Could this be the default k8s cluster setting specific to GKE or something in the GCE load balancer?

Thanks for reading!

- UPDATE -

The load balancer has a backend configuration timeout setting which is for "How long to wait for the backend service to respond before considering it as a failed request."

WebSocket is not responsive. It sends ping / pong messages and other messages until killed, which I can check with console.log in the browser and log into the golang service.

However, if I remove the load balancer timeout setting of 30,000 seconds, everything "works".

Doesn't look like a real fix, although because the load balancer will keep feeding actual useless service traffic incorrectly, it doesn't matter if the WebSocket becomes unresponsive.

I have isolated the high timeout setup to a specific backend setup using a route map, but hoping to come up with a real solution to the problem.

+3


source to share


2 answers


I think it might be "Work as directed". Google recently updated the documentation (about an hour ago).

LB Proxy Support docs

Backend Service Component Documentation



Greetings,

Matt

0


source


Check out the following example: https://github.com/kubernetes/ingress-gce/tree/master/examples/websocket



0


source







All Articles