Clever Geek Handbook

Gzip compression doesn't work in Solr 5.1

I am trying to apply gzip compression in Solr 5.1. I understand that Running Solr on Tomcat is no longer supported by Solr 5.0, so I tried to implement it in Solr.

I downloaded jetty-servlets-9.3.0.RC0.jar and placed it in my webapp \ WEB-INF folder and added the following to webapp \ WEB-INF \ web.xml:

<filter>
   <filter-name>GzipFilter</filter-name>
   <filter-class>org.eclipse.jetty.servlets.GzipFilter</filter-class>
      <init-param>
         <param-name>methods</param-name>
         <param-value>GET,POST</param-value>
         <param-name>mimeTypes</param-name>
         <param-value>text/html,text/plain,text/xml,text/json,text/javascript,text/css,application/xhtml+xml,application/javascript,image/svg+xml,application/json,application/xml; charset=UTF-8</param-value>
      </init-param>
</filter>
<filter-mapping>
<filter-name>GzipFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

However, when I run Solr and check the browser, there is no gzip compression and I only get the following in the Response Headers output: Content-Type: text / plain, encoding = UTF-8 Transfer-Encoding: fragmented

Is there something I have configured incorrectly or may have missed? I am also running zookeeper-3.4.6.

+3


source to share


1 answer


Download the kosher version of Jetty, which exactly matches what is currently in Solr: http://www.eclipse.org/jetty/download.html

Extract this .zip to a location you discard. Copy these files to your copy of the dock in Solr (located in the path to solr / server /):

  • modules/gzip.mod

  • etc/gzip.xml

Edit modules/gzip.mod

:

#
# GZIP module
# Applies GzipHandler to entire server
#

[depend]
server

[xml]
etc/jetty-gzip.xml

[ini-template]
## Minimum content length after which gzip is enabled
jetty.gzip.minGzipSize=2048

## Check whether a file with *.gz extension exists
jetty.gzip.checkGzExists=false

## Gzip compression level (-1 for default)
jetty.gzip.compressionLevel=-1

## User agents for which gzip is disabled
jetty.gzip.excludedUserAgent=.*MSIE.6\.0.*

Edit etc/gzip.xml

:



<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<!-- =============================================================== -->
<!-- Mixin the GZIP Handler                                          -->
<!-- This applies the GZIP Handler to the entire server              -->
<!-- If a GZIP handler is required for an individual context, then   -->
<!-- use a context XML (see test.xml example in distribution)        -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
    <Call name="insertHandler">
        <Arg>
            <New id="GzipHandler" class="org.eclipse.jetty.server.handler.gzip.GzipHandler">
                <Set name="minGzipSize">
                    <Property name="jetty.gzip.minGzipSize" deprecated="gzip.minGzipSize" default="2048"/>
                </Set>
                <Set name="checkGzExists">
                    <Property name="jetty.gzip.checkGzExists" deprecated="gzip.checkGzExists" default="false"/>
                </Set>
                <Set name="compressionLevel">
                    <Property name="jetty.gzip.compressionLevel" deprecated="gzip.compressionLevel" default="-1"/>
                </Set>
                <Set name="excludedAgentPatterns">
                    <Array type="String">
                        <Item>
                            <Property name="jetty.gzip.excludedUserAgent" deprecated="gzip.excludedUserAgent" default=".*MSIE.6\.0.*"/>
                        </Item>
                    </Array>
                </Set>
                <Set name="includedMethods">
                    <Array type="String">
                        <Item>GET</Item>
                        <Item>POST</Item>
                    </Array>
                </Set>

    <Set name="includedPaths"><Array type="String"><Item>/*</Item></Array></Set>

    <Set name="excludedPaths"><Array type="String"><Item>*.gz</Item></Array></Set>

    <Call name="addIncludedMimeTypes"><Arg><Array type="String">
        <Item>text/html</Item>
        <Item>text/plain</Item>
        <Item>text/xml</Item>
        <Item>application/xml</Item><!-- IMPORTANT - DO NOT FORGET THIS LINE -->
        <Item>application/xhtml+xml</Item>
        <Item>text/css</Item>
        <Item>application/javascript</Item>
        <Item>image/svg+xml</Item>
    </Array></Arg></Call>
                <!--
    <Call name="addExcludedMimeTypes"><Arg><Array type="String"><Item>some/type</Item></Array></Arg></Call>
    -->
            </New>
        </Arg>
    </Call>
</Configure>

Here's the part that should make you cringe a little. Editbin\solr.cmd 

...
set "SOLR_JETTY_CONFIG=--module=http,gzip"
...
set "SOLR_JETTY_CONFIG=--module=https,gzip"
...

Please note what --module=http

already exists. Just add ",gzip"

it to match the lines above. I'd rather find a better way to tell the gzip module to download, but I don't know how. If you know how to answer that, please answer this question and tell me how much I hate modifying the script that comes with the product - it's a maintenance nightmare and, well, I guess you get the picture.

After that restart the solr server and now gzip will be enabled - at least for &wt=xml

which is sent as Content-Type: application/xml

. You can add what you need to etc/gzip.xml

and restart the solr server so that it recognizes your changes.

I tested 1000 documents with and without compression. For me, it was the difference between 3.8 MB and 637 KB.

+1


source






More articles:


All Articles