Our applications follow a sidecar-based deployment, which looks more or less like:
jar -xf $APP.jar
java -cp '.:BOOT-INF/lib/*:BOOT-INF/classes:sidecar/X.Y.Z/*' [some -D params in here] org.myco.MyBootApplication
the interesting bit is the sidecar/X.Y.Z folder. We have a volume with each sidecar version, the sidecar containing all spring, spring-boot, spring-cloud and other related frameworks, in our case around 100MB.
This way the applications usually don't take more than hundreds of KB, or if they need a framework not present on the sidecar, a few MB.
The main advantage this model presents for us is that we are able to upgrade spring boot versions (provided there's binary compatibility) for all applications just by switching the volume. As this is done via docker service update.. there is no service loss, and there is no need to rebuild and redeploy the applications. If there's a new CVE, switching to newer versions is really fast, especially when having lot of services.
Another advantage would be decreased size on applications' docker images, this would go inline with recent docker build improvements on Spring Boot.
In order to be able to do this we had to introduce a new Layout on the spring-boot maven plugin, so the libraries present on the sidecar don't get included on the packaged jar.
As we feel that this may be useful for other teams out there, we'd like to provide a PR for this new Layout if it is deemed interesting (which branch should be targetted?)
Comment From: bclozel
We still need to discuss this as a team, but in the meantime I wanted to point out several things.
I'm not sure "sidecar" is the right name here, as it's not really about externalizing cross-cutting infrastructures, but rather sharing a common set of dependencies between applications. This actually reminds of the Java EE application server approach with lightweight WAR/EAR. I'm also wondering if you're running into issues with this approach, since you're adding a lot of libraries on the classpath (and Spring Boot reacts to that!). Aren't features related to Spring Security and Spring Data automatically enabled? Isn't it hurting the startup performance for applications that don't need all those dependencies?
About the Spring Boot upgrade process - I don't think we want to promote a way to upgrade Spring Boot without running the CI and repackaging the application. This is not a use case we can safely support if we decide to apply subtle packaging/classpath changes, or really make any behavior change whatsoever.
Now when it comes to rebuilding containers and saving storage costs, the current layered approach enables just that as you could create a "spring-boot" layer which could be shared and cached by the builder.
In general, I think this use case would be better supported with a custom buildpack. I'm leaving this issue opened until it's discussed during a team meeting.
Thanks!
Comment From: juanpablo-santos
Hi @bclozel ,
thanks for your feedback, much appreciated :-) some comments inline:
I'm not sure "sidecar" is the right name here, as it's not really about externalizing cross-cutting infrastructures, but rather sharing a common set of dependencies between applications. This actually reminds of the Java EE application server approach with lightweight WAR/EAR.
Any name you find reasonable will be ok. The reason of calling it sidecar is that we started calling it "the shared libraries zip" and we kept receiving questions about the use we were giving to it (go figure). Came across the link on the original description, began to called it "the sidecar" and suddenly its intention was understood. As for the lightweight WAR/EAR, yes, in fact we are coming from there, where we had a similar kind of deployment.
I'm also wondering if you're running into issues with this approach, since you're adding a lot of libraries on the classpath (and Spring Boot reacts to that!). Aren't features related to Spring Security and Spring Data automatically enabled? Isn't it hurting the startup performance for applications that don't need all those dependencies?
Not really; if any, we get the same base behaviour for all applications, which is not that bad. A typical application -for us- takes a bit less than 10 seconds to start up. As this startup time is done in the background, and newer versions don't start to serve requests until they are ready, it isn't a big deal (again, for us). Smaller apps take a couple of seconds, and our larger application may take around a minute and a half to start up, but again, it was taking a similar amount of time when it was a lightweight war.
Using the spring-context-indexer helped with the startup time of the applications, and it was more noticeable on larger, legacy applications (they're way bigger than a typical newer service). Hopefully the /startup actuator enpoint on Spring Boot 2.4 will help us pinpoint where we spend the startup time and see how we can improve it, but right now is not a pressing issue for us, around 10 seconds is ok.
As for features not needed but enabled, usually Spring Boot offers a property to switch it off. If this property doesn't exist, you can always exclude the autoconfiguration class from loading from the application.properties/yaml file. We have had to do this a couple of times on the last 4 years though, as the "base" behaviour is usually ok for us (again this may vary for others).
About the Spring Boot upgrade process - I don't think we want to promote a way to upgrade Spring Boot without running the CI and repackaging the application. This is not a use case we can safely support if we decide to apply subtle packaging/classpath changes, or really make any behavior change whatsoever.
Of course, being able to do so doesn't mean you have to do it blindly. Proper/thorough testing throughout all environments is still needed, as it still needed if you re-build the application. The fact that Spring Framework / Boot / Cloud & related projects provide detailed changelogs helps us a lot to measure the impact of one of these changes.
The important thing for us is that we have ~100 applications running in production, so being able to perform these kind of changes has been a game changer for us: certain teams have tight release dates and pushing this kind of changes means delays, other are in the middle of a sprint and cutting a release from latest tag, means distractions, other applications aren't actively developed, so they don't have a dedicated team, and only if a support ticket is raised, then a team (one person usually) is allocated to look into it... Also, f.ex, in addition to Spring * frameworks we also ship some "corporate" starters which enable certain features regarding observability, traceability, etc.
Being able to deploy this "sidecar" (or whatever ends up being called) to all applications in a transparent way is key for us, and that's why we thought it may be useful to others. Again it might not be suitable / useful for others, but we felt that there could be others which might find this approach useful. And of course, it's not a silver bullet, and there are trade-offs to consider. We don't want to try to replace any current approach, but only to add another option. If it is not deemed useful, well at least we asked :-)
Now when it comes to rebuilding containers and saving storage costs, the current layered approach enables just that as you could create a "spring-boot" layer which could be shared and cached by the builder.
In general, I think this use case would be better supported with a custom buildpack. I'm leaving this issue opened until it's discussed during a team meeting.
We'll take a look at the custom buildpack
best regards + thanks for your time and comments!
Comment From: bclozel
Thanks for your comments, they're super useful and giving more context definitely helps!
Comment From: bclozel
We've discussed that during a team call today and we don't think we should include that approach in Spring Boot, as it's at odds with many assumptions and design principles. We think that a custom buildpack could contribute this feature consistently to many applications and even add more to it, like rebuilding images as updates happen or tracking changes with layers metadata.
Thanks for creating this issue!