Working with Gradle and GitHub Packages

Europace and other parts of our IT use a toolchain with GitHub, Jetbrains TeamCity, and Sonatype Nexus to implement CI/CD Pipelines. Developers have to be in the private network or connected to the VPN, if they want to access Maven and Gradle artifacts from our Nexus repository. Especially in times of COVID-19 more developers work from home, where connecting to the VPN becomes a necessity… does it?

With our efforts to leverage the GitHub platform with many of its features, GitHub Actions and GitHub Packages are an obvious choice to become more independent of services behind our firewalls.

Simply migrating to another artifact repository isn’t something we do on a Friday afternoon: There are shared dependencies across several teams, and we wouldn’t like any of them to miss a new release, because we didn’t publish it to the Nexus anymore.

In this article we want to show you how easy it is to use both the “old” Nexus, and the “new” GitHub Packages in your Gradle projects.

Note: We didn’t find a way to apply similar patterns to Maven projects, because Maven allows deployments only to a single repository. Migrating a Maven project to Gradle might be the first step.

Gradle Configuration: Publishing

The GitHub documentation gives you some examples for your Gradle config, along with details about authenticating to GitHub Packages. So, we’ll keep it short and simple. This is an example Gradle config to publish artifacts to multiple repositories:

gradle.properties:

nexus.publish.url=http://example.local/content/repositories/example-releases
nexus.publish.username=
nexus.publish.password=

github.packages.owner=example-owner
github.packages.repository=example-repo
github.packages.username=
github.packages.password=

build.gradle.kts:

plugins {
  id("maven-publish")
}

//...

publishing {
  fun findProperty(s: String) = project.findProperty(s) as String?

  repositories {
    maven {
      name = "Nexus"
      url = uri("${property("nexus.publish.url")}")
      credentials.apply {
        username = findProperty("nexus.publish.username")
        password = findProperty("nexus.publish.password")
      }
    }
    maven {
      name = "GitHubPackages"
      url = uri("https://maven.pkg.github.com/${property("github.packages.owner")}/${property("github.packages.repository")}")
      credentials {
        username = findProperty("github.packages.username")
        password = findProperty("github.packages.password")
      }
    }
  }
  publications {
    register("jar", MavenPublication::class) {
      from(components["java"])
      group = "de.example"
      artifactId = "example-lib"
      version = "0.1"
    }
  }
}

//...

The shown config ensures that a ./gradlew publish creates the JarMavenPublication only once, but uploads it to both Nexus and GitHub Packages.

We won’t be able to perform the publish outside of our private network, though. That’s not an issue, because our existing CI config in TeamCity will be enhanced with the GitHub Packages credentials, and it can work just like before.

Gradle Configuration: Dependency Resolution

We’re now ready to consume the published artifacts. For existing projects nothing has to be changed, because our Nexus still provides even the most recent releases, but our goal to work independently of the VPN isn’t reached, yet.

Our consuming Gradle projects need to know about the new GitHub Packages repository. Simply adding a new repository “GitHub Packages” sadly won’t work, yet, because we have to add a new repository entry for each dependency on a GitHub Packages hosted artifact. This is different from a full fledged repository manager like Nexus (also acting as cache and proxy) and different from artifact collections like Maven Central.

In contrast to the official GitHub documentation we’d like to show you a more advanced example consuming multiple repositories:

//...

repositories {
  listOf("/example-owner/example-repo",
         "/example-owner/foo-repo",
         "/another-organization/another-bar").forEach { path ->
    maven {
      setUrl("https://maven.pkg.github.com${path}")
      content {
        includeGroup("de.example")
        includeGroup("net.another.stuff")
      }
      credentials {
        username = findProperty("github.packages.username")
        password = findProperty("github.packages.password")
      }
    }
  }
  mavenCentral() {
    content {
      excludeGroup("de.example")
      excludeGroup("net.another.stuff")
    }
  }
}

dependencies {
  implementation("de.example:example-lib:0.1")
  // ...
}

//...

The example is a bit simplified, because we use a single token to access all GitHub Packages. YMMV.

With our Nexus not acting as cache anymore, we applied the Repository Content Filtering Gradle feature to reduce failing lookups, thus increasing performance.

Final Thoughts

We’re aware of GitHub working on improving the overall user experience, so the examples above will certainly change over time.

Using modern tooling like Gradle and GitHub helps to slowly improve the CI/CD setup and developer experience when working from home or when traveling. We’re missing some features when comparing GitHub Packages with Nexus, but that’s something where we can give GitHub our feedback and tell them about our use cases.

The combination of GitHub Actions and GitHub Packages works great for us, and we’re working on transferring many “pull request builds” to run on GitHub. That shift removes load from our internal TeamCity agents, which improves our deployment pipeline performance.