2BAB's Blog

I/O 23 and AGP's 10th Anniversary: Why Can't I Still Learn Gradle?

2023/06/06

It's been ten years since Google first released Android Gradle Plugin (AGP) 0.1 at I/O 2013. However, even experienced Android developers may feel confused about using and deeply mastering Gradle and AGP. I've written many Gradle/AGP related articles, done online sharing, and even written a book and a booklet. But through daily communication, I find it's still hard for most people to find a learning path, and even after investing time, there's no obvious progress. So on AGP's tenth anniversary, I decided to record some observations about "why I still can't learn Gradle/AGP" in brief words.

This article will review Gradle's evolution, especially past difficulties and recent improvements over the past two to three years. Finally, I'll provide some learning strategies to help beginners and experienced engineers better master this build system.

Disclaimer: Everything covered in this article is publicly available material, mainly from official public sharing or community voices.

Gradle itself has many difficulties, including but not limited to changing DSL, lifecycle details, many compatibility issues, and difficulties in debugging, locating, and optimizing. Overall, implementing an extension plugin requires mastering many concepts at once.

(Why not ask the magic conch)

But having many concepts and difficulties isn't the problem. Frameworks like React.js and Jetpack Compose are also quite complex conceptually, so why is getting started not that hard for people?

In fact, many common frameworks and tools can help developers quickly find applicable scenarios in the early stages of learning, while learning Gradle and AGP requires investing a lot of time before seeing results. For most beginners, even if your English is good and you can understand Gradle's official User Guide, you can't quickly find the right usage scenarios, let alone combine related components for best practices. And from another angle, we may have forgotten that as Android developers, this tool fundamentally needs to work in the Android App build process to maximize its value, and learning AGP is neglected by many people.

The above observations lead to two thoughts in this article:

  1. Gradle's official documentation consists of scattered knowledge points, lacking practical tutorials oriented toward beginners, i.e., Tutorial-type documentation.
  2. AGP's public APIs and extensibility have certain limitations. AGP's goals are long-term, with its primary purpose being to promote the overall development of the Android build ecosystem, but to balance maintenance costs, it's impossible to open all APIs.

Gradle's Documentation Problem

First point, the documentation problem. "Scattered" refers to two aspects:

  1. Many necessary knowledge points lack guidance documentation: For example, everyone has used the android.buildTypes{...} definition, which is essentially basic use of the NamedDomainObjectContainer class and its series of parent classes. What you can find is API annotation documentation, but you can't find guidance on the User Guide, such as comparison of multiple similar APIs, CRUD in Groovy DSL, CRUD in Kotlin, how to migrate and use between the two DSLs, etc. The earliest reference material from Google searches on this is Mr.HAKi's Gradle Goodness series articles, which also has many other API use cases in the blog series.
  2. The documentation has some 0-to-1 content, such as lifecycle and task basics. But a lot of other common components, API techniques, and how to combine and apply multiple tools are not detailed enough, or may be placed in obscure corners. The comprehensive reading experience is somewhat lacking: you clearly seem to have seen it somewhere, but can't remember, and even keyword searching is somewhat difficult. Users lack a learning roadmap of their own.

From this perspective, some well-known Gradle video series follow the design approach of official documentation and are not conducive to beginners getting started quickly. Of course, in terms of knowledge coverage, they remain a powerful supplement to official documentation.

But here we must insert a turning point: most framework/tool documentation, even with sufficient knowledge points, still has problems of being scattered and without priority, with specific reasons behind it. For many excellent framework developers, writing good API comments in each file, plus a lot of testing, constitutes a rigorous documentation that other developers can learn from on their own. And the exported API documentation is the core "documentation" — it doesn't require a lot of cases and guidance materials to be called "documentation"; that's the icing on the cake, which is Tutorials.

This is also why many tech companies and tech products hire special developer advocates, technical writers, etc. Guidance documentation and open-source Sample repositories are mostly released through their hands. A typical example is Now-In-Android.

Without certain complete guidance documentation, what remains most in Gradle documentation are Script Plugin examples. Due to DSL API reasons, although the documentation looks neat, once you need to write your own binary plugin (i.e., a plugin published after compilation, containing one or more Plugin implementations internally — libraries we reference are usually published as binary plugins), you often need native APIs. And in the previous seven or eight years due to the Groovy-dominated script ecosystem, finding corresponding native APIs from some DSLs was somewhat difficult — NamedDomainObjectContainer again becomes a typical case of this problem. The find-and-add integrated API implemented through Groovy's dynamic features means users actually don't know whether they're using create() or getByName() or named() etc. behind the scenes. This is an advantage when writing build.gradle configuration, but a disadvantage when really writing binary plugins. In my observation, Groovy amplified the difficulty for ordinary people wanting to master this tool; but for many industry pioneers, proficient Groovy skills made them feel Gradle was flexible and easy to use, and until K2 was released, Groovy kept configuration phase script compilation faster.

Finally, because of Gradle's product stance, it won't give too many Android ecosystem examples. For AGP, refer to the next issue's discussion. Unfortunately, due to some special reasons, many monitoring APIs haven't been released either. We've even seen some third-party plugins already calling private APIs, yet there's still no explanation for this part of the issue. This includes the deprecation of most public lifecycle hooks (due to Configuration Cache), and there's no complete explanation for alternatives to these public APIs. Some official best practices can only be seen from Gradle forums or Slack discussion groups.

AGP's Openness Problem

Gradle has a steep learning curve, but "learning a lot of basics" yet not being able to produce things is more because of lack of understanding of the ecosystem. Gradle as an underlying foundation supports not just Android, but also Java, Kotlin, Scala and other language ecosystems, as well as the nearly largest JVM backend framework ecosystem including Spring, etc. It's not hard to imagine that for Android developers, understanding the various details of Android Gradle Plugin (AGP) and related compilation tools is a core part of the packaging process.

In the first five or six years, AGP rarely publicized its development plans and community development support, and it was hard to find related technical documentation. In 2018 and 2019, it gradually spoke up at some important official conferences, understood third-party plugin developers' pain points and synchronized its progress on public API construction, and fully released major changes related to AGP 7.0 at the I/O conference in 2021, including documentation and Sample support.

From the issue tracker situation, although P1/S1 issues have been resolved, there are still many "secondary" issues to be improved. I personally found some problems while trying to help promote new APIs, such as:

  1. Currently only a dozen or so public Artifact APIs, while private APIs have over 200 types.
  2. Public bytecode modification Artifact API differs from private implementation, causing cache invalidation when there are multiple transformation tasks.

Based on the above objective content, developers have no intuitive path to learning AGP. Many articles directly list its task list and analyze one by one when introducing AGP, which is also a helpless choice. But the learning effect of this approach may... not be too good. Since there's no documentation and architecture design, systematic learning requires reading source code and understanding overall design paradigms. The reading process needs to be accompanied by thinking about multiple single-point problems encountered in practice, such as what paradigm AGP's task registration process has, how the chain structure behind the new Artifact API helps data transmission and final API exposure, etc. And the prerequisite for having these questions is that you, as an efficiency-focused developer, or putting yourself in that position, write one plugin after another that may not conform to Gradle and AGP "specifications," see problems from them, read source code, solve problems, and eventually grow. I call the plugins written this way "Android ecosystem cooperative plugins," which are worlds apart from some platform-independent plugins (like plugins that analyze Gradle dependencies).

Most people can't or don't get the opportunity to go through the above process, so the market only has two types of people: beginners and experts. And the middle levels have obvious gaps. This is also reflected in the distribution of topic exposure — over the past ten years, AGP topic sharing has often been mixed in with AS or Android development tool updates. It wasn't until recently that AGP finally has its own sub-tab in the documentation area (independent navigation).

Although the various problems mentioned above once made learning and applying Gradle difficult, there have been some major improvements in the past two to three years:

  1. Kotlin became the default scripting language: Kotlin is now Gradle's default scripting language. For Android developers, this is good news because they can use the same language for both app development and build script development. Compared to Groovy, Kotlin is easier to get started with.

  2. New features in AGP: AGP has followed Gradle's Lazy Property, Configuration Cache, Work Executor and other features; provided truly public APIs, including a complete developer-facing Variant lifecycle, Artifact exposure mechanism, and related Sample documentation. Developers can more easily extend AGP's functionality.

  3. Compilation speed improvements: Collaboration between Google, JetBrains, and Gradle, for example:

    1. Various caching mechanisms, parallel execution mechanisms, and lazy registration mechanisms have been introduced to AGP.
    2. Kotlin's compilation speed continues to improve.
    3. KSP symbol processing tool developed by Google has been introduced to various Android and Kotlin development tools, greatly improving symbol processor efficiency.
    4. Android Studio's various Gradle peripheral function support, including AGP API migration, performance analysis tools, etc.
    5. Future new Kotlin compiler K2 and complete compiler plugin ecosystem toolchain will bring more extreme performance improvements.

Based on the above improvements, if I were to give some learning advice to beginners and intermediate-advanced engineers, I believe it would be:

  1. Strategies for beginners:

    • 1.1 First learn basic concepts of Kotlin and Gradle, such as DSL, project organization structure, task mechanism, etc.
    • 1.2 Then learn common plugins and their Extension usage, such as Android, Java, Kotlin, Maven publishing, etc.
    • 1.3 Finally gradually delve into some advanced topics, such as simple script modifications, simple task writing, basic compilation problem locating, etc.
    • 1.4 KOGE, an open-source booklet I organized, covers most of the above basic concepts. Welcome to read & provide suggestions & participate in writing.

  2. Strategies for intermediate-advanced engineers:

    • 2.1 Centered around "enhancing Android Gradle Plugin functionality," learn to write plugins that cooperate with the Android ecosystem. This is an important threshold for improving all-around compilation and build quality that must be firmly crossed. Driven by goals, discover problems, read source code, solve problems, repeat this cycle, and continuously improve understanding of ecosystem cooperative plugins.
    • 2.2 Apply plugin technology to all aspects of production, such as resource modification, source code modification, bytecode modification, etc. Not only learn related compilation toolchains, but also think about problems from multiple angles, combining compile-time and runtime solutions to complement each other's advantages.
    • 2.3 Learn more about AGP's core ideas and architecture design, optimize the above various plugin solutions, and bring excellent architectural ideas into the App's efficiency architecture, bringing the team a development experience of "low cost, low compilation time, high performance, high maintainability."
    • For friends who want to accelerate learning, you can also get my new book "Extending Android Builds" to systematically learn Gradle plugin development and AGP core design.

In summary, although the early learning process of Gradle and AGP may seem daunting, with recent improvements and the right learning strategies, developers can better master this powerful build tool to improve productivity and code quality.

For comments and further discussion, mail to [email protected]