Final subproject sbt setup

Question

Final subproject sbt setup

I am working on some code with several sub-projects, many of which depend on each other. For example, there is a project utils

with various utilities, a project queueing

with some code for managing queues of things, and then a project mainApp

. The project queueing

depends on utils

, but mainApp

depends on as well as utils

on queueing

. There are many more projects out there, but hopefully you get the general idea.

We use the standard sbt submodule setup for this with one build root, many subprojects and standard tools aggregate

and dependsOn

. This worked, but was problematic:

It would be very slow to create and run tests for the project. Cleaning a building takes about 15 minutes and unit testing takes longer.
Because of the way the aggregate works, unit testing mainApp

has caused all tests to run in utils

and queuing

even if they haven't changed. True, you can do "test-fast", but that stops working when you change git branches, etc., so you run the whole test suite often anyway.
While sbt only allows you to create a subproject, you must be in the root directory and remember to qualify the assembly. For example, you have to remember to run sbt utils/compile

instead of sbt compile

.

So we went to a completely separate project and split up the git repos. Each project builds and then deploys the artifact to the Nexus. So to build and test mainApp

, you just build it and it pulls the already compiled .jar files for other projects from the nexus server. This makes it easier to work in just one project, but it becomes more difficult to make changes, for example, in utils, and then use it in mainApp

. In particular, you often want to do something like add a method to utils

, and then immediately test that method in mainApp

to see if it works before pushing the new version utils

. But now for this you need:

Add your code to utils
Decrease version number in utils
Run "sbt publish-loca"
Change build.sbt to mainApp

and add -SNAPSHOT

to utils

dependency
Run sbt update
Build and Test mainApp

If this all works, you need to push utils

and mainApp

in the correct order to the continuous build server and remember to wait for the build to complete utils

before pushing mainApp

.

Worse, maintaining versions becomes very difficult. Let's assume we start with all projects in version 1.0 and depending on version 1.0 other projects. Now suppose we find a bug in utils

that affects mainApp

. So we'll fix it utils

, change its version number to 1.1, update mainApp

to depend on 1.1, and rebuild. Suppose the code that corrects the error in mainApp

introduces the error in queue

. The problem is that if you go to queue

and run your tests, they will work against version 1.0 utils

(even if you use ranges, since sbt only re-checks them). However, when building mainApp

onutils

there will be a dependency conflict that will be resolved by sbt. Regardless of how he resolves it, it will be "wrong" for at least one project. We can set the converter to "strict", but then version control becomes very time consuming in 10+ projects.

What I'm really looking for is an assembly that works best for both worlds. In particular:

I have a root directory with sub-dirs for each project
If pwd

is one of these subdirectories, all sbt commands are specific to that project only. So, if pwd

- queue

, then sbt test

compiles and tests only queue

.
If there is a version of the artifact on the Nexus, it is fetched from there, but if there is more recent code on disk, the dependency is created locally and used instead.