Giter Club home page Giter Club logo

modern-java-practices's Introduction

Creative Commons 0

Read the book! (this jumps to the wiki)
Jump to the project card wall to see upcoming book and code changes (the card wall tracks Issues for the project).

Some highlighted documentation pages:

Modern Java/JVM Build Practices

Modern Agile

Gradle build Maven build CodeQL vulnerabilities coverage pull requests issues license

Warning

For those using the DependencyCheck plugins for Gradle or Maven, over the July 1st weekend the upstream API for fetching security CVEs changed a major version, and stopped supporting older versions of the data. To get up to date, update to at least version 10.0.2 of either the Gradle or Maven plugin.

After the update, the first build will take a very long time, but should perform normally afterwards. And during the first week or so after this change, you may see multiple connection failures as OWASP NVD is overloaded with projects all catching up at the same time. The Maven plugin shows progress as CVE records are pulled: to see progress with the Gradle plugin, use the --info command-line flag.

Modern Java/JVM Build Practices is an article-as-repo on building modern Java/JVM projects using Gradle and Maven, and a starter project in Java (the advice works for non-Java languages on the JVM though details may change).

Important

See the wiki for all pages and sections. This README is only introduction, motivation, and project status. You can use the table of contents below to quickly jump to bits that interest you.

Regardless of what language(s) or build tool(s) you choose, and you should treat your build and your pipeline as worthy of your attention just as you would your project source code: If it doesn't build right for customers as it does for developers, you have something to think about.

I'm showing you practices and tools that help you make your build and pipeline to production as first-class the same as your own source code. An example of this philosophy for a non-Java language is Clojure.

Your focus, and the focus of this article, is best build practices and project hygiene, and helping you have local work that is identical in production. This project is agnostic between Gradle and Maven: discussion in each section covers both tools (alphabetical order, Gradle before Maven). See My Final Take on Gradle (vs. Maven) for an opinionated view (not my own).

This is not a JVM starter for only Java: I use it for starting my Kotlin projects, and substitute complilation and code quality plugins. Any language on the JVM can find practices and tips.

Note

Scala and Clojure have their own prefered build tools not covered here; however, the advice and examples for your build pipeline are intended to be just as helpful for those JVM languages. Groovy and Kotlin can use the examples directly (they both tend towards the Gradle option on build tools).

As a guide, this project focuses on:

  • A quick starter for JVM projects using Gradle or Maven. Fork me, clone me, copy/paste freely! I am Creative Commons Public Domain Dedication (CC0).
  • Discuss—and illustrate (through code)—sensible default practices; highlight good build tools and plugins
  • Document pitfalls that turned up. Some were easy to address after Internet search; some were challenging (see "Tips" sections)
  • Do not be an "all-in-one" solution. You know your circumstances best. I hope this project helps you discover build improvements you love. Please share with others through issues or PRs

Two recurring themes

  • Shift problems left — Find issues earlier in your build—before you see them in production
  • Make developer life easier — Automate build tasks often done by hand: get your build to complain (fail) locally before sharing with your team, or fail in CI before deployment

These can be summed up as a Software supply chain: ensuring reliable, trusted software from local development through ready-to-deploy: Build with confidence.

But ... you must judge and measure the advice here against your own systems and processes. Some things (many or most things) may work for you: keep an eye for things that do not work for you.

What is a Starter?

A project starter has several goals:

  • Help a new project get up and running with minimal fuss.
  • Show examples of best practices.
  • Explain the why for choices, and help you pick what makes most sense for your project.

This starter project is focused on build:

  • Easy on-ramp for new folks to try out your project for themselves
  • Support new contributors to your project that they become productive quickly
  • Support current contributors in the build, get out of their way, and make everyday things easy

This starter project has minimal dependencies. The focus is on Gradle and Maven plugins and configuration so that you and contributors can focus on the code, not on setting up the build.

Summing up

  • I'm not a great programmer; I'm just a good programmer with great habits.Kent Beck
  • Make it work, make it right, make it fastC2 Wiki

Note

NB — This is a living document. The project is frequently updated to pick up new dependency or plugin versions, and improved practices; the README.md and wiki update recommendations. This is part of what great habits looks like: you do not just show love for your developers and users, but enable them to feed back into projects and help others. See Reusing this project for tips on pulling in updates.

(Credit to Yegor Bugayenko for Elegant READMEs.)


Run from a local script

Try it

You should "kick the tires" and get a feel for what parts of this project you'd like to pull into your own projects and builds. You run across lots of projects: Let's make this one helpful for you.

After cloning or forking this project to your machine, try out the local build that makes sense for you:

$ earthly +build-with-gradle  # CI build with Earthly
$ earthly +build-with-maven  # CI build with Earthly
$ ./gradlew build  # Local-only build
$ ./mvnw verify  # Local-only build

Notice that you can run the build purely locally, or in a container?

I want to convince you that running your builds in a container fixes the "it worked on my machine" problem, and show you how to pick up improvements for your build that helps you and others be awesome.

Note

This project uses NVD to check for CVEs with your dependencies which can take a long time to download. You can speed up your build time by requesting an NVD API key (it can take quite a while to fetch the CVEs list or update it, and may fail with 403 or 404 without a key).

When you request a key, NVD sends you an email to confirm your identity, and then share an API key web page. See Shift security left for more details.

See what the starter "run" program does:

Both Gradle and Maven (after building if needed) should print:

TheFoo(label=I AM FOOCUTUS OF BORG)

A "starter" program is the simplest of all possible "smoke tests", meaning, the minimal things just work, and when you check other things, maybe smoke drifts from your computer as circuits burn out1.


Changes

Recent significant changes

(For detailed changes in the example code, browse the commit log.)

  • Move to a CC0 license from Public Domain.
  • Gradle: Bump to Gradle 8.9.
  • Migrate most of the README.md to the GitHub project wiki. This is breaks up an overlong (14k+ words and growing) README into digestible sections.
  • Earthly and Batect: Remove support for Batect as the author has archived that project. Please use Earthly for local containerized builds. So your local command line is:
    $ earthly +build-with-gradle
    # OR
    $ earthly +build-with-maven
    I'll be researching other options, and updating to show those and examples. Advice remains the same: Run your local build in a container for reproducibility, and have CI do the same to exactly repeat your local builds.
  • JVM: Move to JDK 21. This project has no sample code relying on recent/modern versions of Java or the JVM; however, moving between versions does need changes to build scripts and supporting files. Here is the last commit using JDK 17
  • Gradle: Build with Gradle 8.x.
  • Gradle: Bemove use of testsets plugin for integration testing in favor of native Gradle support. This supports Gradle 8.

Table of Contents

Table of Contents

The writing for this project is fully moved to the wiki pages. Use the sidebar navigation in the wiki to browse or jump to topics, or to follow in a reading order. You can also use the droplist control next to "Pages" for an alphabetical listing (including subheaders within pages), and for a search box.

Lastly, the wiki pages are themselves a repo, and you can clone it using [email protected]:binkley/modern-java-practices.wiki.git as you can for any GitHub wiki.


Contributing

See CONTRIBUTING.md. Please file issues, or contribute pull requests! I'd love a conversation with you.


Credits

Special thanks to my co-author, John Libby.

And many thanks to all the contributions from:

All suggestions and ideas welcome! Please file an issue. ☺

Footnotes

  1. No, I'm just kidding. Amazon or Google or Microsoft cloud would have quite different problems than "white smoke" from computers2.

  2. Actually, this really happened me in a data center before the cloud when a power supply burned out. We rushed to use a fire extinguisher before the Halon system triggered.

modern-java-practices's People

Contributors

adilsonarc avatar binkley avatar boxleytw avatar bukharovsi avatar dependabot[bot] avatar github-actions[bot] avatar jwlibby avatar lemmingavalanche avatar sergeibukharov avatar tw-kendrickbarrett avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

modern-java-practices's Issues

SpotBugs complains about test throwables

After upgrading SpotBugs:

[INFO] --- spotbugs-maven-plugin:4.7.0.0:check (default) @ modern-java-practices ---
[INFO] BugInstance size is 4
[INFO] Error size is 0
[INFO] Total bugs: 4
[ERROR] Medium: Method lists Exception in its throws clause. [demo.ApplicationIT] At ApplicationIT.java:[line 19] THROWS_METHOD_THROWS_CLAUSE_BASIC_EXCEPTION
[ERROR] Medium: Method lists Exception in its throws clause. [demo.ApplicationIT] At ApplicationIT.java:[lines 19-22] THROWS_METHOD_THROWS_CLAUSE_BASIC_EXCEPTION
[ERROR] Medium: Method lists Throwable in its throws clause. [demo.TheFooTest] At TheFooTest.java:[line 33] THROWS_METHOD_THROWS_CLAUSE_THROWABLE
[ERROR] Medium: Method lists Throwable in its throws clause. [demo.TheFooTest] At TheFooTest.java:[line 55] THROWS_METHOD_THROWS_CLAUSE_THROWABLE

Move from Batect

Batect has been archived by it's author after long effort. Credit to @charleskorn and his incredible work.

This project needs to pivot from Batect to another "build as container" system. The sensible option seems to be Earthly.

Information on Java 15

Hi @binkley, just came across this repository, looks great!

Having read the README, I'm finding myself querying a bullet point that says, "Java 15 is the current LTS ("long-term support") version". I was under the impression that Java 11 was the latest LTS version, and that the next one will be Java 17. Have I misunderstood something here?

Track down Gradle issue with Java 17

It turns out Gradle, because of it's split between command-line client and background daemon, uses separate settings for the JVM (ie, the --add-opens java.base/java.lang=ALL-UNNAMED flag for plugins based on Ant -- which default Gradle has several of) between those two, and using --no-daemon falls into an not-well-documented mix of client/daemon settings.

Solution is to use JVM_FLAGS in the Batect/Docker build.

(Interestingly, local Gradle build does not show this behavior.)

Discuss Docker

Goals:

  • Provide example generation of Docker images in Gradle & Maven
  • Discuss diffs between JDK 11 and JDK 17 for choice of parent image

Add testcontainers example

https://www.testcontainers.org/ is recommended for system/integration testing needing to bring up the full application, connecting to a disposable test database, etc.

These kinds of tests should be rare for the main code base which relies on unit tests or "slice" tests ala Spring Boot's approach, and so forth, but are important for higher levels of the test pyramid like user journey tests and such.

Provide a "whole book" view of the wiki pages

Previous README.md was everything together in sections of a single file. That remains helpful.

Either:

  1. Provide a link to a "everything" page for long reading & printing.
  2. Clarify at top, and put the pages into the README.md — how to keep updated?

Key -- how to automate?

Publish to gh-pages branch for the project.

A challenge is duplicated documentation between the wiki and the README.md.

REMINDER: Remove compat6 PMD dependency when Maven PMD plugin updates

The current Maven PMD plugin relies on classes from 6.x PMD. Gradle's PMD plugin does not have this concern.

PMD provides a "pmd-compat6" jar usable by the plugin to patch the differences.
This needs removing once the PMD plugin catches up.

See pom.xml and:

<dependency>
    <groupId>net.sourceforge.pmd</groupId>
    <artifactId>pmd-compat6</artifactId>
    <version>${pmd.version}</version>
</dependency>

Without this, the plugin falls over. Tempting, but ./mvnw dependency:resolve-plugins does not reveal the problem.

Dummy issue to test GitHub markdown

Picture HTML tag and GitHub footnote markdown

This is an image that depends on time of day1:

Shows an illustrated sun in light mode and a moon with stars in dark mode.

This is for testing GitHub markdown [^2].

The kbd tag

What does Enter (aka ⎆) or Return (aka ⏎) look like?

Footnotes

  1. Either Sun or Moon; reload in day and in night in your time zone to see the contrast

BCEL vulnerability in Spotbugs

BCEL from Apache Commons has a security vulnerability, CVE-2022-42920.
Spotbugs relies on this library, but presently uses a pre-fixed version of BCEL.

Cleaning this up in the build configuration is messy: Best addressed when Spotbugs releases an update that includes a newer BCEL version.

Wrong report link

When run batect gradle build, if it has errors the report link is not right.

Checkstyle rule violations were found. See the report at: file:///code/build/reports/checkstyle/test.html

A badge for build stability

This is part of the epic #586.

I would like to show a badge on the project for build stability: what has been the recent pass/fail rate, and bucket those into good descriptive words.

This will help teams visually spot when a project is going through rough times (could be the code, could be the environment).
Ideal would be to have knobs to tweak to configure. Example:

  • Green if last 5 builds were green
  • Amber if last 5 builds were a mix of red & green
  • Red if last 5 builds were red

An example with the right approach is https://github.com/dwyl/repo-badges but it is unclear how to use in this project.
See dwyl/repo-badges#53.

Fix PIT test report for Maven

From ./mvnw site:

Warning:  An issue has occurred with pitest-maven:1.7.4:report report, skipping LinkageError Receiver class org.pitest.maven.report.PitReportMojo does not define or inherit an implementation of the resolved method 'abstract void generate(org.apache.maven.doxia.sink.Sink, java.util.Locale)' of interface org.apache.maven.reporting.MavenReport., please report an issue to Maven dev team.
java.lang.AbstractMethodError: Receiver class org.pitest.maven.report.PitReportMojo does not define or inherit an implementation of the resolved method 'abstract void generate(org.apache.maven.doxia.sink.Sink, java.util.Locale)' of interface org.apache.maven.reporting.MavenReport.

GitHub action redownloads Maven plugin ignoring the cache

Expected behavior: this is all cached, and not redownloaded on each CI run.

NB -- ignore the adoptopenjdk image download: To be addressed separately from this issue.

Running build-maven...
Pulling adoptopenjdk:11-jdk-hotspot...
Pulled adoptopenjdk:11-jdk-hotspot.
Running ./mvnw --strict-checksums clean verify in build-env...
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------< hm.binkley.md:modern-java-practice-maven >--------------
[INFO] Building Modern JAva/JVM build practices 0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
Progress (1): 2.7/3.9 kB
Progress (1): 3.9 kB    
                    
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom (3.9 kB at 33 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.jar
Progress (1): 2.7/25 kB
Progress (1): 5.5/25 kB
Progress (1): 8.2/25 kB
Progress (1): 11/25 kB 
Progress (1): 14/25 kB
Progress (1): 16/25 kB
Progress (1): 19/25 kB
Progress (1): 22/25 kB
Progress (1): 25 kB   
                   
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.jar (25 kB at 616 kB/s)

Address log4shell

Many resources to explain this. Document in README.md

In the docs, note that DependencyCheck catches these
Check that transitive dependencies fail the build as well

NOTE -- this demo project does not use Log4j

Investigate Dagger and other container builds

https://dagger.io/
This was mentioned to me by @charleskorn.
It is one of other alternatives for a containerized build.

Background
Originally, the builds were containerized in Batect. The maintainer (Charles Korn) has move on from this project, and the best option is Earthly. Research other options, and provide alternative config/scripts for keeping your build identical in CI and local.

Dagger is in the style of Gradle: Programming your build.
Consider wiki discussion on declarative vs programmed builds.

The front page of Dagger offers:

Transform your Messy CI Scripts into Clean Code
Powerful, programmable open source CI/CD engine that runs your pipelines in containers — pre-push on your local machine and/or post-push in CI

Update runscripts

Show variants in the runscripts depending on platform/libraries.

For example, Spring Boot will rewrite the basenamed jar to just <name>-<version>.jar, but vanilla Maven uses <name>-<version>-jar-with-dependencies.jar.

Drop building the Maven project web site

Does anyone use Maven "site"? Perhaps drop Maven site.

Running ./mvnw site is very noisy. It isn't clear what is working/broken.

Some Maven plugins may only write their results for the "site" target. Address this.
For example, update the target location of the Maven profiling plugin:

Gradle version checking is broken

The Groovy code in build.gradle for dependencyUpdates task is bad. It's a copy from a Kotlin project, and has incomplete translation to Groovy.

Fix the Build badge

The README build badge has not loaded for some time.
Investigate and fix.

Provide example CI builds without a container

The advice here is strongly towards containers (like Earthly), but it is good to show working examples of relying on direct GitHub containers in the CI pipeline when a project does not have a local build that matches CI.

This is both reading & code samples.

Result of this card should be:

  • GitHub workflow for Gradle directly without using Earthly
  • GitHub workflow for Maven directly without using Earthly

Breakup the README

Current README.md is 1873 lines and 10,784 words in length. This is excessive.

Consider strategies to break this up into meaningful, digestible chapters in separate files, possibly using the GH wiki feature.

Using Batect on GitHub Actions

Thanks for pulling this together @binkley, it's a great resource.

I'm interested in understanding more about this comment in the readme:

Do not run Batect in your CI pipeline directly: use GitHub Actions (or GitLab equivalent). Batect is for your local build. (This is the classic "Docker in Docker" issue, and there are ways address this , if it is important for your team.)

I haven't seen any issues with running Batect inside GitHub Actions - have you encountered any issues? From what I understand, GitHub Actions spins up an ephemeral virtual machine, not a container, so there shouldn't be any issues with Docker in Docker.

I strongly encourage teams to use Batect both locally and on CI - this ensures that the environment remains consistent in both places, making it easier to reproduce issues (or catch problems before they even become issues).

Information on Java 8

Following on from my last issue #1, I was also under the impression that AdoptOpenJDK 8 was still being supported, which the following link on the website seems to confirm: https://adoptopenjdk.net/support.html. However I'm not sure if this applies to all distributions of OpenJDK 8. Any thoughts on whether this info should be included on the README or not?

Missing CPD with Gradle

CPD (part of PMD) runs for Maven (including report), but not for Gradle.
This seems a request on the Gradle plugin project, but unclear how to progress.

Build caching

See broader card, #466.

Proposal

  1. Move this card to done with "wontfix" label. The "Buildless" tool suggested in this card is commercial.
  2. Keep #466 (GitHub caching) as an active card -- it is orthogonal to this card.
  3. Add a card for Earthly caching (build container), which would interact with point 2.

Hey there @binkley,

We make a build cache tool called Buildless, with support especially for Gradle and Maven.

I was wondering if you'd be open to me filing a PR to add it to this sample.

Buildless is usable both as a cloud and entirely for free, through a local agent which accelerates Gradle's cache traffic. It does so by upgrading from HTTP/1.1 to HTTP/3.

Point is, it's a great tool, and the agent can be used for free and without an account. Is this the kind of thing that would be good to add?

Also, thanks for making this 😄 it's a pain to get started on a new Gradle project and this looks great

Update to JDK 21

It's been 4 months since JDK 21 release, it would be good to update project (and README) to use it.

Also I'm not sure that the wordings in README:

Java 17 is the most current LTS ("long-term support") version

Should be more like:

Java 21 is the most current version that has LTS builds available

Because Java per se doesn't have LTS, only the distribuitions/builds of Java have LTS.

In this project, you'll see the choice of Java 17 as this is the version to recommend in production.

Why is 18 or 19 a bad choice? There is nothing inherently bad in those versions, they are not less tested or "beta". The only difference is that none of the vendors publishes security patches to them after the new version is released -> forcing to do the upgrades every 6 months, which is a good practice.

I would drop the "recommend in production" unless a good explanation is given.

Track CVE in PMD for Gradle

OWASP complains about commons-io 2.6. 2.8.0 is current version

Work out how to update a plugin dependency without making it a runtime dependency.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.