I’ve been playing with Hadoop recently and ran into an issue with the Maven dependency plugin copying all the JAR files from all the scopes into my lib folder. No problem I thought, you can exclude scoped dependencies with the
excludeScope configuration parameter, so I set that to provided but this still left the test dependencies being copied. As you can’t set two
excludeScope elements and the one element you can set only takes a single scope, this is a bit of an issue.
It turns out that if you want to exclude dependencies from both the test and provided scopes, you need to exclude the provided scope and include the runtime scope. So your plugin snippet becomes something like:
This means that your lib folder isn’t polluted with your test JARs like JUnit, Hamcrest and Mockito but more importantly without all of the Hadoop dependencies. Which all means that your Hadoop standalone mode classpath for testing out those MapReduce jobs isn’t full of unnecessary clutter.