Solve an incompatible version conflict problem in Java by relocation and custom ClassLoader
javaWhen your Java application occurs an incompatible version conflict problem, even if you build uber-jar, classes in the jar are not preferentially loaded but loaded by the classpath order. At that time, if there are incompatible classes, it will be in a state called JAR hell, and NoSuchMethodError or unintended behavior can occur at runtime.
Say you have legacylib.jar containing guava 17.0 and modernlib.jar containing guava 32.1, and the former calls ByteStreams.asByteSource() removed in guava 18.0 and the latter calls ByteStreams.exhaust() added in guava 20.0. In this case, NoSuchMethodError will occur regardless of which jar is placed before the classpath. I will solve this problem. The whole code is on GitHub.
$ cat Legacy.java
...
public class Legacy {
public static void foo() throws IOException {
InputSupplier<InputStream> inputSupplier = () -> new ByteArrayInputStream("legacy ok".getBytes()); // interface removed in guava 20
byte[] bytes = ByteStreams.asByteSource(inputSupplier).read(); // method removed in guava 18
System.out.println(new String(bytes, "UTF-8"));
}
}
$ cat Modern.java
...
public class Modern {
public static void bar() throws IOException {
InputStream inputStream = new ByteArrayInputStream("aaaaa".getBytes());
ByteStreams.exhaust(inputStream); // method added in guava 20.0
System.out.println("modern ok");
}
}
$ java -cp myapp/target/myapp-1.0.0.jar:legacylib/target/legacylib-1.0.0.jar:modernlib/target/modernlib-1.0.0.jar net.sambaiz.Main
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.io.ByteStreams.exhaust(Ljava/io/InputStream;)J
at net.sambaiz.modernlib.Modern.bar(Modern.java:13)
at net.sambaiz.Main.main(Main.java:10)
$ java -cp myapp/target/myapp-1.0.0.jar:modernlib/target/modernlib-1.0.0.jar:legacylib/target/legacylib-1.0.0.jar net.sambaiz.Main
modern ok
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.io.ByteStreams.asByteSource(Lcom/google/common/io/InputSupplier;)Lcom/google/common/io/ByteSource;
at net.sambaiz.legacylib.Legacy.foo(Legacy.java:16)
at net.sambaiz.Main.main(Main.java:11)
Besides, InputSupplier interface removed in guava 20.0 is also used, but this is not a conflict because there is no class in 32.1.
Solution with relocation
maven-shade-plugin has relocation settings to rewrite package names.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>com.google</pattern>
<shadedPattern>com.google.legacy</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
This can avoid conflict of incompatible classes.
$ jar tf legacylib/target/legacylib-1.0.0.jar
...
com/google/legacy/common/io/ByteStreams.class
com/google/legacy/common/io/CharSequenceReader.class
com/google/legacy/common/io/CharSink.class
$ java -cp myapp/target/myapp-1.0.0.jar:modernlib/target/modernlib-1.0.0.jar:legacylib/target/legacylib-1.0.0.jar net.sambaiz.Main
modern ok
legacy ok
By the way, the version of libraries that is contained in the jar is determined by maven’s nearest definition policy, so the version can change if you change the order of dependencies.
$ mvn clean package dependency:tree
...
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ myapp ---
[INFO] net.sambaiz:myapp:jar:1.0.0
[INFO] +- net.sambaiz:legacylib:jar:1.0.0:compile
[INFO] | \- com.google.guava:guava:jar:17.0-rc2:compile
[INFO] \- net.sambaiz:modernlib:jar:1.0.0:compile
You can fix the version by dependencyManagement.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactI>guava</artifactId>
<version>32.1.2-jre</version>
</dependency>
</dependencies>
</dependencyManagement>
Solution with custom ClassLoader
This problem occurs because the loaded classes are selected at the application level, so it can also be solved by changing the class to be loaded for each library with a custom ClassLoader. Relocation is required to modify the library’s settings, but this solution does not require it. However, codes such as creating instances and passing it to an argument become complicated, so that itself can cause a runtime error.
package net.sambaiz;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
public class MyClassLoader extends ClassLoader {
private String jarFilePath;
public MyClassLoader(ClassLoader parent, String jarFilePath) {
super(parent);
this.jarFilePath = jarFilePath;
}
@Override
protected Class<?> findClass(String className) throws ClassNotFoundException {
System.out.printf("findClass %s\n", className);
try {
JarFile jarFile = new JarFile(this.jarFilePath);
JarEntry entry = jarFile.getJarEntry(className.replace('.', '/') + ".class");
if (entry == null) {
throw new ClassNotFoundException(className);
}
InputStream inputStream = jarFile.getInputStream(entry);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
byte[] bytecode = outputStream.toByteArray();
return defineClass(className, bytecode, 0, bytecode.length);
} catch (IOException e) {
throw new ClassNotFoundException(className);
}
}
public static void main(String[] args) throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException {
System.out.println(ClassLoader.getSystemClassLoader());
MyClassLoader legacyClassLoader = new MyClassLoader(ClassLoader.getSystemClassLoader(), "legacylib/target/legacylib-1.0.0.jar");
Class<?> legacyClass = legacyClassLoader.loadClass("net.sambaiz.legacylib.Legacy");
Object legacyInstance = legacyClass.getDeclaredConstructor().newInstance();
legacyClass.getMethod("foo").invoke(legacyInstance);;
MyClassLoader modernClassLoader = new MyClassLoader(ClassLoader.getSystemClassLoader(), "modernlib/target/modernlib-1.0.0.jar");
Class<?> modernClass = modernClassLoader.loadClass("net.sambaiz.modernlib.Modern");
Object modernInstance = modernClass.getDeclaredConstructor().newInstance();
modernClass.getMethod("bar").invoke(modernInstance);;
}
}
The library paths is not included even in the classpath, but they are loaded by the custom class loaders.
$ java -cp myapp/target/myapp-1.0.0.jar net.sambaiz.MyClassLoader
findClass net.sambaiz.legacylib.Legacy
findClass com.google.common.io.InputSupplier
findClass com.google.common.io.ByteStreams
...
legacy ok
findClass net.sambaiz.modernlib.Modern
findClass com.google.common.io.ByteStreams
findClass com.google.common.io.ByteStreams$1
findClass com.google.common.io.ByteStreams$LimitedInputStream
findClass com.google.common.io.ByteArrayDataInput
findClass com.google.common.io.ByteArrayDataOutput
modern ok