Continental libraries with WAR-separated versions

In a Java servlet container (preferably Tomcat, but if it can be done in another container then say so), I wish for what is theoretically possible. My question here is if there are tools to support it, and if so, which tools (or which names should I research next).

Here's my problem: in one servlet container, I want to run a large number of different WAR files. They have several large shared libraries (like Spring). Blush first, I have two unacceptable alternatives:

  • Include a large library (Spring for example) in each WAR file. This is not acceptable because it will load a large number of Spring copies, running out of memory on the server.

  • Place the large library on the container classpath. Now all WAR files have one instance of the library (good). But this is unacceptable because I cannot update the Spring version without updating ALL WAR files at the same time, and such a big change is hard to take away from.

In theory, however, there is an alternative that might work:

  1. Place each version of the large library in the container classpath. Do some container level magic so that each WAR file declares which version it wants to use and it will find that in its classpath.

The "magic" has to be done at the container level (I think), because this can only be achieved by loading each version of the library with a different classloader, and then configuring which classloaders are visible for each WAR file.

So, have you ever heard of this? If so, how? Or tell me what it's called, so I can continue my research.

+3


source to share


3 answers


As for Tomcat, for 7th version you can use VirtualWebappLocader like so

<Context>
    <Loader className="org.apache.catalina.loader.VirtualWebappLoader"
            virtualClasspath="/usr/shared/lib/spring-3/*.jar,/usr/shared/classes" />
</Context>

      

For the 8th version instead of

pre and post resources should be used.
<Context>
    <Resources>
        <PostResources className="org.apache.catalina.webresources.DirResourceSet"
                       base="/usr/shared/lib/spring-3" webAppMount="/WEB-INF/lib" />
        <PostResources className="org.apache.catalina.webresources.DirResourceSet"
                       base="/usr/shared/classes" webAppMount="/WEB-INF/classes" />
    </Resources>
</Context>

      

Remember to put the appropriate context.xml in the META-INF of your web application.

The same method can be used for the quay as well as other containers. The only difference is how to specify additional classpath elements for the webapp.


UPDATE The above examples do not share loaded classes, but the idea is the same - use a custom classloader. Here's just a rather ugly sample that also tries to prevent the classloader from leaking during undeployment.


SharedWebappLoader

package com.foo.bar;

import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.WebappLoader;

public class SharedWebappLoader extends WebappLoader {

    private String pathID;
    private String pathConfig;

    static final ThreadLocal<ClassLoaderFactory> classLoaderFactory = new ThreadLocal<>();

    public SharedWebappLoader() {
        this(null);
    }

    public SharedWebappLoader(ClassLoader parent) {
        super(parent);
        setLoaderClass(SharedWebappClassLoader.class.getName());
    }

    public String getPathID() {
        return pathID;
    }

    public void setPathID(String pathID) {
        this.pathID = pathID;
    }

    public String getPathConfig() {
        return pathConfig;
    }

    public void setPathConfig(String pathConfig) {
        this.pathConfig = pathConfig;
    }

    @Override
    protected void startInternal() throws LifecycleException {
        classLoaderFactory.set(new ClassLoaderFactory(pathConfig, pathID));
        try {
            super.startInternal();
        } finally {
            classLoaderFactory.remove();
        }
    }

}

      


SharedWebappClassLoader



package com.foo.bar;

import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.ResourceEntry;
import org.apache.catalina.loader.WebappClassLoader;

import java.net.URL;

public class SharedWebappClassLoader extends WebappClassLoader {

    public SharedWebappClassLoader(ClassLoader parent) {
        super(SharedWebappLoader.classLoaderFactory.get().create(parent));
    }

    @Override
    protected ResourceEntry findResourceInternal(String name, String path) {
        ResourceEntry entry = super.findResourceInternal(name, path);
        if(entry == null) {
            URL url = parent.getResource(name);
            if (url == null) {
                return null;
            }

            entry = new ResourceEntry();
            entry.source = url;
            entry.codeBase = entry.source;
        }
        return entry;
    }

    @Override
    public void stop() throws LifecycleException {
        ClassLoaderFactory.removeLoader(parent);
    }
}

      


ClassLoaderFactory

package com.foo.bar;

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

public class ClassLoaderFactory {

    private static final class ConfigKey {
        private final String pathConfig;
        private final String pathID;
        private ConfigKey(String pathConfig, String pathID) {
            this.pathConfig = pathConfig;
            this.pathID = pathID;
        }
        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (o == null || getClass() != o.getClass()) return false;

            ConfigKey configKey = (ConfigKey) o;

            if (pathConfig != null ? !pathConfig.equals(configKey.pathConfig) : configKey.pathConfig != null)
                return false;
            if (pathID != null ? !pathID.equals(configKey.pathID) : configKey.pathID != null) return false;

            return true;
        }

        @Override
        public int hashCode() {
            int result = pathConfig != null ? pathConfig.hashCode() : 0;
            result = 31 * result + (pathID != null ? pathID.hashCode() : 0);
            return result;
        }
    }

    private static final Map<ConfigKey, ClassLoader> loaders = new HashMap<>();
    private static final Map<ClassLoader, ConfigKey> revLoaders = new HashMap<>();
    private static final Map<ClassLoader, Integer> usages = new HashMap<>();

    private final ConfigKey key;

    public ClassLoaderFactory(String pathConfig, String pathID) {
        this.key = new ConfigKey(pathConfig, pathID);
    }

    public ClassLoader create(ClassLoader parent) {
        synchronized (loaders) {
            ClassLoader loader = loaders.get(key);
            if(loader != null) {
                Integer usageCount = usages.get(loader);
                usages.put(loader, ++usageCount);
                return loader;
            }

            Properties props = new Properties();
            try (InputStream is = new BufferedInputStream(new FileInputStream(key.pathConfig))) {
                props.load(is);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }

            String libsStr = props.getProperty(key.pathID);
            String[] libs = libsStr.split(File.pathSeparator);
            URL[] urls = new URL[libs.length];
            try {
                for(int i = 0, len = libs.length; i < len; i++) {
                    urls[i] = new URL(libs[i]);
                }
            } catch (MalformedURLException e) {
                throw new RuntimeException(e);
            }

            loader = new URLClassLoader(urls, parent);
            loaders.put(key, loader);
            revLoaders.put(loader, key);
            usages.put(loader, 1);

            return loader;
        }
    }

    public static void removeLoader(ClassLoader parent) {
        synchronized (loaders) {
            Integer val = usages.get(parent);
            if(val > 1) {
                usages.put(parent, --val);
            } else {
                usages.remove(parent);
                ConfigKey key = revLoaders.remove(parent);
                loaders.remove(key);
            }
        }
    }

}

      


context.xml of the first application

<Context>
    <Loader className="com.foo.bar.SharedWebappLoader"
            pathConfig="${catalina.base}/conf/shared.properties"
            pathID="commons_2_1"/>
</Context>

      


context.xml of the second application

<Context>
    <Loader className="com.foo.bar.SharedWebappLoader"
            pathConfig="${catalina.base}/conf/shared.properties"
            pathID="commons_2_6"/>
</Context>

      


$ TOMCAT_HOME / CONF / shared.properties

commons_2_1=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.1/commons-lang-2.1.jar
commons_2_6=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar

      

+5


source


I was able to implement this for Tomcat (Tested on Tomcat 7.0.52). My solution includes implementing a custom version of WebAppLoader that extends the standard Tomcat WebAppLoader. With this solution, you can pass a custom classloader to load classes for each of your web applications.

To use this new loader, you need to declare it for each application (either in the Context.xml file hosted in each war, or in the Tomcat server.xml file). This loader accepts an additional custom webappName parameter, which is then passed to the LibrariesStorage class to determine which libraries should be used by any application.

    <Context  path="/pl-app" >
        <Loader className="web.DynamicWebappLoader" webappName="pl-app"/>
    </Context>      
    <Context  path="/my-webapp" >
        <Loader className="web.DynamicWebappLoader" webappName="myApplication2"/>
    </Context>

      

Once this is determined, you need to install this DynamicWebappLoader in Tomcat. To do this, copy all the compiled classes into the lib directory from Tomcat (so you must have the following files [tomcat dir] /lib/web/DynamicWebappLoader.class, [tomcat dir] /lib/web/LibrariesStorage.class, [tomcat dir] / lib / web / LibraryAndVersion.class, [tomcat dir] /lib/web/WebAppAwareClassLoader.class).

You need to also download xbean-classloader-4.0.jar and put it in the Tomcat lib dir (so you must have [tomcat dir] /lib/xbean-classloader-4.0.jar. NOTE: xbean-classloader provides a custom implementation of classloader (org .apache.xbean.classloader.JarFileClassLoader) which allows you to load the required jars at runtime.



The main trick is in the LibraryStorgeClass (full implementation at the end). It stores the mapping between each application (defined by webappName) and the libraries that application is allowed to download. This is hardcoded in the current implementation, but can be rewritten to dynamically create a list of libraries required for each application. Each library has its own JarFileClassLoader instance, which ensures that each library is loaded only once (the mapping between the library and its classloader is stored in the static field "libraryToClassLoader", so this mapping is the same for every web application due to the static nature of the field)

class LibrariesStorage {
    private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";

  private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();

  private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();

  static {
    try {
      addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
      addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
      addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");

      mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars

      mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");

     (...)    
 } 

      

In the above example, assume that in the directory with all the jars (defined here by JARS_DIR), we only have the commons-lang3-3.3.2.jar file. This would mean that the application identified by the name "pl-app" (the name comes from the webappName attribute in the tag in the Context.xml as above) would be able to load classes from the commons-lang jar. The application identified by "myApplication2" will receive a ClassNotFoundException at this point because it only has access to commons-lang3-3.3.0.jar, but this file is not in the JARS_DIR directory.

Full implementation here:

package web;

import org.apache.catalina.loader.WebappLoader;
import org.apache.xbean.classloader.JarFileClassLoader;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;


public class DynamicWebappLoader extends WebappLoader {
  private String webappName;
  private WebAppAwareClassLoader webAppAwareClassLoader;

  public static final ThreadLocal lastCreatedClassLoader = new ThreadLocal();

  public DynamicWebappLoader() {
    super(new WebAppAwareClassLoader(Thread.currentThread().getContextClassLoader()));

    webAppAwareClassLoader = (WebAppAwareClassLoader) lastCreatedClassLoader.get(); // unfortunately I did not find better solution to access new instance of WebAppAwareClassLoader created in previous line so I passed it via thread local
    lastCreatedClassLoader.remove();

  }

  // (this method is called by Tomcat because of Loader attribute in Context.xml - <Context> <Loader className="..." webappName="myApplication2"/> )
  public void setWebappName(String name) {
    System.out.println("Setting webapp name: " + name);
    this.webappName = name;
    webAppAwareClassLoader.setWebAppName(name); // pass web app name to ClassLoader 
  }


}


class WebAppAwareClassLoader extends ClassLoader {
  private String webAppName;

  public WebAppAwareClassLoader(ClassLoader parent) {
    super(parent);
    DynamicWebappLoader.lastCreatedClassLoader.set(this); // store newly created instance in ThreadLocal .. did not find better way to access the reference later in code
  }

  @Override
  public Class<?> loadClass(String className) throws ClassNotFoundException {
    System.out.println("Load class: " + className + " for webapp: " + webAppName);
    try {
      return LibrariesStorage.loadClassForWebapp(webAppName, className);
    } catch (ClassNotFoundException e) {
      System.out.println("JarFileClassLoader did not find class: " + className + " " + e.getMessage());
      return super.loadClass(className);
    }

  }

  public void setWebAppName(String webAppName) {
    this.webAppName = webAppName;
  }
}

class LibrariesStorage {
  private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";

  private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();

  private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();

  static {
    try {
      addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
      addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
      addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");

      mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
      mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");

    } catch (MalformedURLException e) {
      throw new RuntimeException(e.getMessage(), e);
    }

  }

  private static void mapApplicationToLibrary(String applicationName, String libraryName, String libraryVersion) {
    LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
    if (!webappLibraries.containsKey(applicationName)) {
      webappLibraries.put(applicationName, new ArrayList<LibraryAndVersion>());
    }
    webappLibraries.get(applicationName).add(libraryAndVersion);
  }

  private static void addLibrary(String libraryName, String libraryVersion, String filename)
                          throws MalformedURLException {
    LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
    URL libraryLocation = new File(JARS_DIR + File.separator + filename).toURI().toURL();

    libraryToClassLoader.put(libraryAndVersion,
      new JarFileClassLoader("JarFileClassLoader for lib: " + libraryAndVersion,
        new URL[] { libraryLocation }));
  }

  private LibrariesStorage() {
  }


  public static Class<?> loadClassForWebapp(String webappName, String className) throws ClassNotFoundException {
    System.out.println("Loading class: " + className + " for web application: " + webappName);

    List<LibraryAndVersion> webappLibraries = LibrariesStorage.webappLibraries.get(webappName);
    for (LibraryAndVersion libraryAndVersion : webappLibraries) {
      JarFileClassLoader libraryClassLoader = libraryToClassLoader.get(libraryAndVersion);

      try {
        return libraryClassLoader.loadClass(className); // ok current lib contained class to load
      } catch (ClassNotFoundException e) {
        // ok.. continue in loop... try to load the class from classloader connected to next library
      }
    }

    throw new ClassNotFoundException("Class " + className + " was not found in any jar connected to webapp: " +
      webappLibraries);

  }

}


class LibraryAndVersion {
  private final String name;
  private final String version;

  LibraryAndVersion(String name, String version) {
    this.name = name;
    this.version = version;
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) {
      return true;
    }
    if ((o == null) || (getClass() != o.getClass())) {
      return false;
    }

    LibraryAndVersion that = (LibraryAndVersion) o;

    if ((name != null) ? (!name.equals(that.name)) : (that.name != null)) {
      return false;
    }
    if ((version != null) ? (!version.equals(that.version)) : (that.version != null)) {
      return false;
    }

    return true;
  }

  @Override
  public int hashCode() {
    int result = (name != null) ? name.hashCode() : 0;
    result = (31 * result) + ((version != null) ? version.hashCode() : 0);
    return result;
  }

  @Override
  public String toString() {
    return "LibraryAndVersion{" +
      "name='" + name + '\'' +
      ", version='" + version + '\'' +
      '}';
  }
}

      

+1


source


JBoss has a structure called Modules that addresses this problem. You can save a shared library with its version and reference it from your war file.

I have no idea if it works on Tomcat, but it works like a Wildfly spell.

+1


source







All Articles