Magento 2 with varnish and Nginx as SSL termination - Endless Redirect
Any Nginx / Varnish / SSL gurus out there? I had a CentOS7 server configured successfully and Magento 2.17 running. Then I decided that I wanted to run everything over SSL and Varnish still has pages cached.
Since Varnish does not handle SSL, I reconfigured Varnish and Nginx to terminate SSL. Nginx is currently configured to listen on port 443 and relay requests for varnish on port 8081. Then, varnish is expected to execute the request and return it to listen to Nginx on port 8080. When I make an HTTPS request, nginx successfully forwards it to varnish and then, as far as I can tell from the logs, it redirects 20 times before the browser turns it off and displays the message: "The page is not redirecting as expected."
Perhaps something small and silly that I was unable to detect and a second set of eyes would really be appreciated.
My nginx test.conf file for domain test.com:
## Proxy server to terminate ssl, loop to Varnish-HTTP -> nginx-HTTPS
server {
listen 443 ssl http2;
server_name test.com;
sub_filter 'http://' 'https://';
sub_filter_once off;
## SSL CONFIGURATION
ssl_certificate /etc/letsencrypt/live/test.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/test.com/privkey.pem;
# proxy-pass to Varnish
location / {
proxy_pass http://127.0.0.1:8081;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header Host $host;
proxy_set_header X-Scheme https;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header X-Secure on;
}
}
server {
listen 8080;
server_name test.com;
access_log /var/log/nginx/access.log main if=$writelog;
error_log /var/log/nginx/error.log error;
if ($bad_client) { return 444; }
set $MAGE_ROOT /home/test/public_html;
root $MAGE_ROOT/pub;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
## Process php files (strict rule, define files to be executed)
location ~ (index|get|static|report|btNenBhQtLNu_opcache_gui|404|503)\.php$ {
try_files $uri =404;
## specific security and compatibility headers
add_header X-Config-By 'Test' always;
add_header X-Processing-Time $request_time always;
add_header X-Request-ID $request_id always;
add_header Strict-Transport-Security $hsts_header always;
add_header X-UA-Compatible 'IE=Edge,chrome=1';
add_header Link "<$scheme://$http_host$request_uri>; rel=\"canonical\"" always;
## php backend settings
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_keep_conn on;
# microcache
fastcgi_no_cache $no_cache;
fastcgi_cache_bypass $no_cache;
fastcgi_cache microcache;
fastcgi_cache_key $scheme|$host|search|$arg_q;
fastcgi_cache_valid 200 301 302 2h;
fastcgi_cache_use_stale updating error timeout invalid_header http_500;
fastcgi_pass_header Set-Cookie;
fastcgi_pass_header Cookie;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
# microcache
}
## Block other undefined php files, possible injections and random malware hooks.
location ~* \.php$ { return 404; }
}
Varnish.params file:
# Varnish environment configuration description. This was derived from # the old style sysconfig/defaults settings # Set this to 1 to make systemd reload try to switch VCL without restart. RELOAD_VCL=1 # Main configuration file. You probably want to change it. VARNISH_VCL_CONF=/etc/varnish/default.vcl # Default address and port to bind to. Blank address means all IPv4 # and IPv6 interfaces, otherwise specify a host name, an IPv4 dotted # quad, or an IPv6 address in brackets. VARNISH_LISTEN_ADDRESS=127.0.0.1 VARNISH_LISTEN_PORT=8081 # Admin interface listen address and port VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1 VARNISH_ADMIN_LISTEN_PORT=6082 # Shared secret file for admin interface VARNISH_SECRET_FILE=/etc/varnish/secret # Backend storage specification, see Storage Types in the varnishd(5) # man page for details. VARNISH_STORAGE="malloc,1G" # User and group for the varnishd worker processes VARNISH_USER=varnish VARNISH_GROUP=varnish # Other options, see the man page varnishd(1) DAEMON_OPTS="-p thread_pool_min=200 \ -p thread_pool_max=4000 \ -p thread_pool_add_delay=2 \ -p cli_timeout=25 \ -p cli_buffer=26384 \ -p syslog_cli_traffic=off \ -p feature=+esi_disable_xml_check,+esi_ignore_other_elements \ -p vcc_allow_inline_c=on \ -p timeout_linger=100"
Default.vcl varnish:
vcl 4.0;
import std;
# The minimal Varnish version is 4.0
# For SSL offloading, pass the following header in your proxy server or load balancer: 'SSL-OFFLOADED: https'
backend default {
.host = "127.0.0.1";
.port = "8080";
}
acl purge {
"localhost";
"127.0.0.1";
}
sub vcl_recv {
if (req.method == "PURGE") {
if (client.ip !~ purge) {
return (synth(405, "Method not allowed"));
}
if (!req.http.X-Magento-Tags-Pattern) {
return (synth(400, "X-Magento-Tags-Pattern header required"));
}
ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
return (synth(200, "Purged"));
}
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
# We only deal with GET and HEAD by default
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
# Bypass shopping cart, checkout and search requests
if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") {
return (pass);
}
# normalize url in case of leading HTTP scheme and domain
set req.url = regsub(req.url, "^http[s]?://", "");
# collect all cookies
std.collect(req.http.Cookie);
# Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") {
# No point in compressing these
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
unset req.http.Accept-Encoding;
}
}
# Remove Google gclid parameters to minimize the cache objects
set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA"
set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar"
set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz"
# static files are always cacheable. remove SSL flag and cookie
if (req.url ~ "^/(pub/)?(media|static)/.*\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)$") {
unset req.http.Https;
unset req.http.SSL-OFFLOADED;
unset req.http.Cookie;
}
return (hash);
}
sub vcl_hash {
if (req.http.cookie ~ "X-Magento-Vary=") {
hash_data(regsub(req.http.cookie, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1"));
}
# For multi site configurations to not cache each other content
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# To make sure http users don't see ssl warning
if (req.http.SSL-OFFLOADED) {
hash_data(req.http.SSL-OFFLOADED);
}
}
sub vcl_backend_response {
if (beresp.http.content-type ~ "text") {
set beresp.do_esi = true;
}
if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") {
set beresp.do_gzip = true;
}
# cache only successfully responses and 404s
if (beresp.status != 200 && beresp.status != 404) {
set beresp.ttl = 0s;
set beresp.uncacheable = true;
return (deliver);
} elsif (beresp.http.Cache-Control ~ "private") {
set beresp.uncacheable = true;
set beresp.ttl = 86400s;
return (deliver);
}
if (beresp.http.X-Magento-Debug) {
set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control;
}
# validate if we need to cache it and prevent from setting cookie
# images, css and js are cacheable by default so we have to remove cookie also
if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
unset beresp.http.set-cookie;
if (bereq.url !~ "\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|gz|tgz|bz2|tbz|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)(\?|$)") {
set beresp.http.Pragma = "no-cache";
set beresp.http.Expires = "-1";
set beresp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
set beresp.grace = 1m;
}
}
# If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass
if (beresp.ttl <= 0s ||
beresp.http.Surrogate-control ~ "no-store" ||
(!beresp.http.Surrogate-Control && beresp.http.Vary == "*")) {
# Mark as Hit-For-Pass for the next 2 minutes
set beresp.ttl = 120s;
set beresp.uncacheable = true;
}
return (deliver);
}
sub vcl_deliver {
if (resp.http.X-Magento-Debug) {
if (resp.http.x-varnish ~ " ") {
set resp.http.X-Magento-Cache-Debug = "HIT";
} else {
set resp.http.X-Magento-Cache-Debug = "MISS";
}
} else {
unset resp.http.Age;
}
unset resp.http.X-Magento-Debug;
unset resp.http.X-Magento-Tags;
unset resp.http.X-Powered-By;
unset resp.http.Server;
unset resp.http.X-Varnish;
unset resp.http.Via;
unset resp.http.Link;
}
Netstat ports report:
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 1027/php-fpm: maste tcp 0 0 127.0.0.1:9001 0.0.0.0:* LISTEN 733/hhvm tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 2721/mysqld tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 13657/redis-server tcp 0 0 0.0.0.0:5131 0.0.0.0:* LISTEN 1412/proftpd: (acce tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 997/memcached tcp 0 0 127.0.0.1:6380 0.0.0.0:* LISTEN 1028/redis-server 1 tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 9142/nginx: master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 9142/nginx: master tcp 0 0 127.0.0.1:8081 0.0.0.0:* LISTEN 9760/varnishd tcp 0 0 127.0.0.1:11000 0.0.0.0:* LISTEN 2308/lookup-domain- tcp 0 0 0.0.0.0:7224 0.0.0.0:* LISTEN 984/sshd tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 9142/nginx: master tcp 0 0 127.0.0.1:6082 0.0.0.0:* LISTEN 9760/varnishd tcp 0 0 0.0.0.0:17668 0.0.0.0:* LISTEN 2861/perl tcp6 0 0 :::7224 :::* LISTEN 984/sshd
Larger log of forwarding requests (there are 20 of them, this is 2):
* << Request >> 294986
- Begin req 294985 rxreq
- Timestamp Start: 1496891464.408694 0.000000 0.000000
- Timestamp Req: 1496891464.408694 0.000000 0.000000
- ReqStart 127.0.0.1 4702
- ReqMethod GET
- ReqURL /
- ReqProtocol HTTP/1.0
- ReqHeader X-Forwarded-For: 69.227.13.164
- ReqHeader X-Forwarded-Host: test.com
- ReqHeader Host: test.com
- ReqHeader X-Scheme: https
- ReqHeader X-Real-IP: 69.227.13.164
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-Port: 443
- ReqHeader X-Secure: on
- ReqHeader Connection: close
- ReqHeader User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
- ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
- ReqHeader Accept-Language: en-US,en;q=0.5
- ReqHeader Accept-Encoding: gzip, deflate, br
- ReqHeader Cookie: PHPSESSID=344hlkmot8oi8ooop0ugnprc47
- ReqHeader DNT: 1
- ReqHeader Upgrade-Insecure-Requests: 1
- ReqUnset X-Forwarded-For: 69.227.13.164
- ReqHeader X-Forwarded-For: 69.227.13.164, 127.0.0.1
- VCL_call RECV
- ReqURL /
- ReqUnset Accept-Encoding: gzip, deflate, br
- ReqHeader Accept-Encoding: gzip
- ReqURL /
- ReqURL /
- ReqURL /
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 294987 fetch
- Timestamp Fetch: 1496891464.447563 0.038869 0.038869
- RespProtocol HTTP/1.1
- RespStatus 302
- RespReason Found
- RespHeader Server: nginx
- RespHeader Date: Thu, 08 Jun 2017 03:11:04 GMT
- RespHeader Content-Type: text/html; charset=UTF-8
- RespHeader Location: https://test.com/
- RespHeader Pragma: no-cache
- RespHeader Cache-Control: max-age=0, must-revalidate, no-cache, no-store
- RespHeader Expires: Wed, 08 Jun 2016 03:11:04 GMT
- RespHeader X-Magento-Debug: 1
- RespHeader X-Content-Type-Options: nosniff
- RespHeader X-XSS-Protection: 1; mode=block
- RespHeader X-Frame-Options: SAMEORIGIN
- RespHeader X-Config-By: Honey
- RespHeader X-Processing-Time: 0.039
- RespHeader X-Request-ID: ce5053ee7177817d0d8f612aff673a32
- RespHeader X-UA-Compatible: IE=Edge,chrome=1
- RespHeader Link: <http://test.com/>; rel="canonical"
- RespHeader Content-Encoding: gzip
- RespHeader Vary: Accept-Encoding
- RespHeader X-Varnish: 294986
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish-v4
- VCL_call DELIVER
- RespHeader X-Magento-Cache-Debug: MISS
- RespUnset X-Magento-Debug: 1
- RespUnset Server: nginx
- RespUnset X-Varnish: 294986
- RespUnset Via: 1.1 varnish-v4
- RespUnset Link: <http://test.com/>; rel="canonical"
- VCL_return deliver
- Timestamp Process: 1496891464.447622 0.038928 0.000059
- RespHeader Content-Length: 20
- Debug "RES_MODE 2"
- RespHeader Connection: close
- Timestamp Resp: 1496891464.447669 0.038975 0.000047
- ReqAcct 540 0 540 600 20 620
- End
* << Session >> 294985
- Begin sess 0 HTTP/1
- SessOpen 127.0.0.1 4702 127.0.0.1:8081 127.0.0.1 8081 1496891464.408655 19
- Link req 294986 rxreq
- SessClose RESP_CLOSE 0.039
- End
+3
source to share
1 answer
It looks like you are redirecting to https://
because your back-end does not know that it is being served by HTTPS (and presumably you configured it to be, so it generates https links, etc.). You need to convince Magento to HTTPS even if it doesn't seem like it. In a similar setup with Apache, I use:
<If "-R '192.0.0.1'">
SetEnvIfNoCase X-Forwarded-Proto https HTTPS=on
</If>
Not sure what the syntax is for NGINX, but hopefully this should put you on the right track.
0
source to share