Converting a Project from Ant to Maven
Contents
Introduction
I recently wanted to convert a number of Java projects from the Ant build system to the Maven build system. Unless you are very familiar with Ant and Maven this is not a trivial task. On this page I describe how I approached the problem, and how I dealt with the inevitable bumps in the road.
Note that the project conversions in this document are pretty much one-on-one. If you have a larger Ant project, you should consider a more modular approach for the resulting Maven projects.
Be sure to keep the Ant documentation and Maven documentation handy!
Analysing the Ant Project
As an example Ant project we look at a loader for the UniProt protein database [license].
We analyze the first two levels of the directory structure with the tree -L 2 command:
~/ant_projects$ tree -L 2 . ├── uniprot-loader │ ├── build.xml │ ├── docs │ ├── etc │ ├── lib │ ├── src │ └── test └── utils ├── bin ├── src └── xsd 10 directories, 1 file
We have two directories, and the project we are after is called uniprot-loader. In the /uniprot-loader directory we see the Ant configuration file build.xml. The project has some dependencies in the /util directory, which we will see later.
The contents of the /uniprot-loader/src directory are:
~/ant_projects$ tree uniprot-loader/src/ uniprot-loader/src/ └── com └── sri └── biospice └── warehouse └── uniprot ├── parser │ └── UniprotParser.java ├── UniprotEntry.java └── UniprotMain.java 6 directories, 3 files
You may think “Only three source files, this will be a walk in the park!”, but don’t be fooled. We have a swamp of dependencies to wade through, and that’s just the tip of the iceberg.
Our starting point will be the build.xml file of the uniprot-loader project. We ‘just’ have to convert the Ant targets defined there to their Maven equivalents.
Creating a Fresh Maven Project
Ok, let’s create a new, empty Maven project.
~/mvn_projects/$ mvn archetype:generate -DgroupId=com.sri.biospice.warehouse.uniprot -DartifactId=uniprot-loader -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
This will create a new Maven project in the directory /mvn_projects/uniprot-loader with a default structure. It is good practice to let the groupId reflect the package structure of the Java files, and you can see we followed that convention.
~/mvn_projects$ tree . └── uniprot-loader ├── pom.xml └── src ├── main │ └── java │ └── com │ └── sri │ └── biospice │ └── warehouse │ └── uniprot │ └── App.java └── test └── java └── com └── sri └── biospice └── warehouse └── uniprot └── AppTest.java 16 directories, 3 files ~/mvn_projects$ cat uniprot-loader/pom.xml <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0modelVersion> <groupId>com.sri.biospice.warehouse.uniprotgroupId> <artifactId>uniprot-loaderartifactId> <packaging>jarpackaging> <version>1.0-SNAPSHOTversion> <name>uniprot-loadername> <url>http://maven.apache.orgurl> <dependencies> <dependency> <groupId>junitgroupId> <artifactId>junitartifactId> <version>3.8.1version> <scope>testscope> dependency> ...
After moving the original source files into the new Maven project, we get something like this:
~/mvn_projects$ tree . ├── pom.xml └── src ├── main │ └── java │ └── src │ └── com │ └── sri │ └── biospice │ └── warehouse │ └── uniprot │ ├── parser │ │ └── UniprotParser.java │ ├── UniprotEntry.java │ └── UniprotMain.java └── test └── java └── src └── com └── sri └── biospice └── warehouse └── uniprot └── UniprotEntry_Test.java 18 directories, 5 files
Migrating the Dependencies
If we try to compile the new project using the mvn compile command, we run into errors because the Ant project has many dependencies that we haven't transferred to the Maven project yet.
~/mvn_projects$ mvn compile ... /src/com/sri/biospice/warehouse/uniprot/UniprotEntry.java:[21,42] package com.sri.biospice.warehouse.database does not exist /src/com/sri/biospice/warehouse/uniprot/UniprotEntry.java:[27,23] package org.apache.log4j does not exist /src/com/sri/biospice/warehouse/uniprot/UniprotEntry.java:[28,26] package org.apache.xmlbeans does not exist ...
To fix these errors we will use on one of the key strengths of Maven: its dependency system. Instead of collecting libraries locally, they are placed in (public) repositories, and the pom.xml refers to libraries in these repositories. When you build your project Maven will automatically download the libraries it needs from these repositories and add them to your classpath.
In a typical Ant build, you probably have a lib folder in your build tree with a pile of jars with no version number. The key is figuring out the Maven coordinates for each jar so you can add it as a dependency in your pom.xml. So let's take a look at our build.xml file to see what dependencies our Ant project declares for the compilation process. A simplified example of the dependency definition is displayed below:
<property name="lib.dir" location="${loader.dir}/lib"/> <property name="loader.dir" location="."/> <property name="common.dir" location="${loader.dir}/../utils/src/java"/> <property name="common.lib.dir" location="${common.dir}/lib"/> -- Third party libraries --> <property name="xmlbeans.jar" location="${lib.dir}/xbean.jar"/> <property name="saxdomix.jar" location="${lib.dir}/saxdomix.jar"/> <property name="cli.jar" location="${common.lib.dir}/commons-cli-1.0.jar"/> -- CLASSPATHS --> <path id="classpath.compile" description="The classpath used for compiling the program"> <pathelement path="${xmlbeans.jar}"/> <pathelement path="${saxdomix.jar}"/> <pathelement path="${cli.jar}"/> path> -- Compile --> <javac srcdir="${build.src.dir}" destdir="${build.classes.dir}" debug="true"> <classpath refid="classpath.compile"/> <exclude name="**/*${unit.test.suffix}.java"/> javac>
We see that the classpath is configured to contain dependencies in two different directories: ${lib.dir} and ${common.lib.dir}. The contents of ${lib.dir} are:
~/ant_projects$ tree uniprot-loader/lib uniprot-loader/lib/ ├── saxdomix.jar ├── uniprot-schema.jar └── xbean.jar 0 directories, 3 files
So we have several jar files without a version number. The easiest way to find the Maven coordinates for these files is using the excellent Sonatype Nexus open source repository search feature. Other search engines can be found at mvnrepository.com and search.maven.org. Calculate the SHA1 checksums of the jar files, and run a search for them on Sonatype.
~/ant_projects$ sha1sum uniprot-loader/lib/* 2050c7bd5efb90ccc43ff537ceed077c591b88f7 uniprot-loader/lib/saxdomix.jar e074d54ce7d702307b6d4eeb14c991c19b8c8f02 uniprot-loader/lib/uniprot-schema.jar 8704dcf5c9f10265a08f5020b0fab70eb64ac3c4 uniprot-loader/lib/xbean.jar
Dependency in External Repository
In the case of the xbeans.jar file we get a hit on Sonatype on the SHA1 checksum. It turns out to be a library--or artifact, as it is called in Maven-land-- named xmlbeans, version 2.3.0.
The checksum of the xbean.jar file is used to search, and it turns out that we are dealing with version 2.3.0 of org.apache.xmlbeans.
After tracking down the dependency on Sonatype, we are even provided with the dependency directive we need to add to our pom.xml file. After adding it Maven takes care of the rest, and we don't need the xbeans.jar file anymore.
By default Maven uses the central repository at repo.maven.apache.org/maven2 to download dependencies. Sometimes you need to include other Maven repositories. You can do this by adding them to your pom.xml.
Dependency in Internal Repository
In the case of saxdomix.jar and uniprot-schema.jar we cannot find the Maven coordinates of the libraries. We still need to get these dependecies into our Maven project somehow, so what do we do?
The trick is to include the jar files into your "local repository", then include them in your pom.xml file. First, browse the contents of the .jar to get an idea of what the unique identifier of the file should be. Then install the library in your local repository using mvn install.
To add saxdomix.jar to our local repository with group identifier com.devsphere.xml.saxdomix:
~/ant_projects/uniprot-loader/lib$ mvn install:install-file -Dfile=saxdomix.jar -DgroupId=com.devsphere.xml.saxdomix -DartifactId=saxdomix -Dversion=1.0 -Dpackaging=jar
After installing, you can add the custom library details into your pom.xml file like usual.
> > com.devsphere.xml.saxdomix>> saxdomix>> 1.0> >
We repeat this process for all dependencies in the build.xml file.
Migrating a Project Dependency
Eventually we run into an issue: one of the .jar files is actually generated from a different Ant project in the ~/ant_projects/utils/src/java directory. In the build.xml file we see a statement that is executed for every goal:
dir="${common.dir}" inheritall="false"/> This directive will run Ant on the build.xml file in the ${common.dir} directory.
We can either generate the jar once and put that in our local repository, or completely migrate this second project to Maven as well. If you are planning to make any changes to a depending project, then it makes sense to migrate it to Maven. That way you don't have to update the jar in your repository when you make a change, because Maven will keep the dependency to date for you.
If you migrate a depending project it will take some extra effort, but it will save you time in the long run.
Creating a Second Maven Project
We will migrate the second Ant project to Maven, and learn how to set up Maven project dependencies in the process.
Let's take a look at the first two levels of the directory structure:
~/ant_projects/utils/src/java$ tree -L 2 . ├── build.properties ├── build.xml ├── docs │ └── Common_Java_Components.html ├── lib │ ├── activation.jar │ ├── commons-cli-1.0.jar │ ├── db-schema.jar │ ├── element-map-beans.jar │ ├── jaxb-api.jar │ ├── jaxb-impl.jar │ ├── jaxb-xjc.jar │ ├── jsr173_api.jar │ ├── junit-3.8.1.jar │ ├── log4j-1.2.8.jar │ ├── mysql-connector-java-3.0.9-stable-bin.jar │ ├── ojdbc14.jar │ ├── oracle-license.txt │ ├── saxdomix.jar │ ├── saxon8.jar │ ├── xbean.jar │ ├── xercesImpl.jar │ └── xml-apis.jar ├── src │ ├── com │ └── version.prop └── test ├── data ├── lib ├── master-test.xml ├── src └── unittest.properties.template
This project has a lot more dependencies than the first Ant project. If you now think this post is far from finished, you are correct. It's going to be a long day.
First we have to make a second Maven project:
~/mvn_projects$ mvn archetype:generate -DgroupId=com.sri.biospice.warehouse.utils -DartifactId=utils -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
This will create a new directory at ~/mvn_projects/utils/. Now we copy all the Java source files from the Ant project into this new directory. Then we turn our attention to the dependencies.
Migrating Dependencies (Again)
This should be familiar by now. We examine the build.xml to find the dependencies, and migrate them one by one, exactly like we did for the first project. The easiest way to do this is to find the relevant
When trying to migrate the library mysql-connector-java-3.0.9-stable-bin.jar we run into a problem. We can find the jar on Sonatype, but only versions 3.0.8 and 3.0.10 are available in the repository. Both versions will probably work, so we decide to go with the more recent version 3.0.10. Note that in old projects many of the jar files will be outdated (the current version of the mysql connector is 5.1.19).
It is a good idea to migrate all dependencies in their original versions - unless you are certain the new version won't break things. When you have everything up and running you can try updating outdated dependencies to a more recent version. While this is an error-prone process, if you manage to replace local artifacts with external artifacts it is rewarding. Decreasing the amount of jars you have to supply with your software is a good thing!
You may encounter cases where a jar can be found on the repository search, but Maven can't download it. A common reason for this problem is that the jar cannot be distributed via a repository due to licensing issues. This manifests in errors of the form: "Could not find artifact javax.sql:jdbc-stdext:jar:2.0 in central."
FIX #1: Add an exclusion for the problem artifact and replace it with an alternative.
FIX #2: Download the jar directly from the vendor and install it in your local repository.
After adding all of the dependencies to the pom.xml, we check if the code compiles correctly:
~mvn_projects/utils$ mvn compile [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building utils 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ utils --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ utils --- [INFO] Nothing to compile - all classes are up to date [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.020s [INFO] Finished at: Wed Apr 11 13:58:48 CEST 2012 [INFO] Final Memory: 3M/53M [INFO] ------------------------------------------------------------------------
Hurray, we managed to compile the project with Maven! If you reach this point you can be rightly proud of yourself, because you've made build management of this project a lot easier. The original build.xml file was 22 kB and 453 lines, while the pom.xml is only 3 kB and 92 lines! Even though we haven't transferred all functionality to the maven project yet (only the build target), this is quite impressive!
Installing the Utils Project in the Local Repository
Remember that the uniprot-loader project depends on the utils project. We need to package up the utils project and install it in our local repository, so uniprot-loader can refer to it. To do this we execute mvn install on the utils project:
~mvn_projects/utils$ mvn install ... [INFO] Installing /home/arto/mvn_projects/utils/target/utils-1.0-SNAPSHOT.jar to /home/arto/.m2/repository/com/sri/biospice/warehouse/utils/utils/1.0-SNAPSHOT/utils-1.0-SNAPSHOT.jar [INFO] Installing /home/arto/mvn_projects/utils/pom.xml to /home/arto/.m2/repository/com/sri/biospice/warehouse/utils/utils/1.0-SNAPSHOT/utils-1.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.718s [INFO] Finished at: Wed Apr 11 16:22:55 CEST 2012 [INFO] Final Memory: 9M/85M [INFO] ------------------------------------------------------------------------
Now we can reference the utils artifact by adding a dependency directive to the uniprot-loader pom.xml:
> > com.sri.biospice.warehouse.utils>> utils>> 1.0-SNAPSHOT> >
Now we find that the uniprot-loader project also compiles correctly!
We managed to convert two Ant projects to Maven, and compile them correctly. Phew, finally! There may be other
I was curious if you ever thought of changing the page layout of your website?
Its very well written; I love what youve got to
say. But maybe you could a little more in the way of content so people could
connect with it better. Youve got an awful lot of text for only having 1 or 2 pictures.
Maybe you could space it out better?
The article was very helpful. If you could go further to talk about how to converts the s in ant to maven that would be nice. Thank you
Good article.
Way cool! Some very valid points! I appreciate you penning this write-up and the rest of the website is extremely good.