About
==

This is the Debian packages for Kotlin, an open-source, statically typed
programming language. This document describes how to build a Debian package for
Kotlin from source and discusses the challenges and solutions.

Kotlin Bootstrapping Problem
==

Significant portion of Kotlin compiler and standard libraries are written in
Kotlin language. However, in Debian, packages are not allowed to use external
binaries for build (for a good reason). This creates a chicken-and-egg problem
known as bootstrapping.

To exacerbate the problem, Gradle is also involved in the dependency loop.
Gradle is a build tool similar to GNU Make, Ant and Maven. Gradle build scripts
can be written either in Groovy or in Kotlin. For this reason, later versions of
Gradle depend on Kotlin. The version of Gradle currently in Debian does not
support writing build scripts in Kotlin. Kotlin's build system uses Gradle with
Kotlin as the chosen language for writing build scripts.

The solution to the bootstrapping problem is to first upload a binary only
package and then do a source upload that depends on the uploaded binary. See the
bootstrapping [discussion][1] for more information. Since uploading both Kotlin
and Gradle binaries at the same time is harder, the mutual dependency had to be
broken. For this, Kotlin's build system was ported from Gradle/Kotlin to
Gradle/Groovy, the code to which resides under debian/patches/.

After Kotlin is fully bootstrapped and available in Debian, latest Gradle will
be uploaded using Kotlin. Once latest Gradle is available in Debian, the build
system patches in Kotlin can be dropped lightening the maintenance burden. Then
Kotlin itself can be advanced to later versions with reduced difficulty.

Building Kotlin
==

Kotlin Debian package is built in two stages. In the first stage, the compiler
for compiling the sources is downloaded from upstream repositories. Then using
this compiler the Debian package for Kotlin compiler is built. In the second
stage, this build process is repeated but instead of using the downloaded binary
compiler, we use the compiler we built in the first stage. The output of second
stage (or third/fourth stages) is uploaded to Debian repositories.

```sh
+-----------------+         +----------------+         +----------------+
| Upstream Binary |   =>    | Debian Package |   =>    | Debian Package |
|     Compiler    | stage 1 |  Intermediate  | stage 2 |   for Upload   |
+-----------------+         +----------------+         +----------------+
```

Stage 1:

  In this step, generate a Debian package out of the compiler binaries
  downloaded from upstream Kotlin repositories.

  ```
  $ debian/bootstrap
  $ git clean -fd
  ```

  This will build kotlin_<version>.deb into the parent directory, as per
  dpkg-buildpackage. However, it is useful to have a dedicated directory for
  local build inputs, so move this and use it to build kotlin.

  ```
  $ mkdir -p ../build-deps
  $ mv ../kotlin_*_all.deb ../build-deps
  $ gbp buildpackage --git-builder=sbuild --extra-package=../build-deps/
  ```

  If openjdk-8 is not available in your distribution at the time you are
  building, fetch it as follows.

  ```
  $ cd ../build-deps/
  $ curl -O https://snapshot.debian.org/archive/debian/20201202T151734Z/pool/main/o/openjdk-8/openjdk-8-jre-headless_8u275-b01-1_amd64.deb
  $ curl -O https://snapshot.debian.org/archive/debian/20201202T151734Z/pool/main/o/openjdk-8/openjdk-8-jdk-headless_8u275-b01-1_amd64.deb
  ```

Stage 2:

  In this step, use the Kotlin compiler generated in the previous step to build
  it again.

  ```
  $ rm ../build-deps/kotlin_*_all.deb
  $ cp ../kotlin_*_all.deb ../build-deps/
  $ gbp buildpackage --git-builder=sbuild --extra-package=../build-deps/
  ```

For more details on how to build Kotlin see the [discussion][2].

Multiple Upstream Tarballs
==

See man pages for uscan(1) and other [MUT][4] [documentation][5] for details.
This is used to break a Build-Depends cycle in Stage 2 that kotlin requires
coroutines-core and coroutines requires kotlin. We currently only build the bare
minimum required for kotlin, so eventually the rest of the buildsystem will need
to be enabled and shipped in separate binary packages.

  - kotlin is imported into / (component = <none>)
  - kotlinx.coroutines is imported into /coroutines (component = coroutines)
  - kotlinx.atomicfu is imported into /atomicfu (component = atomicfu)

You need at least version 2.20.3 of uscan to download the tarballs. If another
external dependency is found in the future, here's how to add a component:

  - debian/watch: add stanza for new component
  - debian/changelog: append required version to compound version
  - debian/gbp.conf: add component name to list

  ```
  $ uscan --download-current-version # repeat until all downloads succeed
  $ gbp import-orig ../kotlin_1.3.31+~1.0.1+~0.11.12.orig.tar.xz
  ```

Debugging Tips
==

- If the build fails in non-obvious ways due to issues in the build scripts, see
  Gradle build system [debugging tips][3].

- If the compiler crashes, try to create a minimal program that crashes the
  compiler. Eliminate the need for Gradle and compile the test program using
  `kotlinc test.kt`. Use it to understand if a fix from upstream needs to be
  backported. Then proceed to create a Java Debugger (jdb) remote debugging
  session for the kotlin compiler as follows:

  ```
  $ kotlinc -J-agentlib:jdwp=transport=dt_socket,address=0.0.0.0:5005,server=y,suspend=y test.kt
  ```

  The program starts and will wait for the jdb to connect. Connect to it using:

  ```
  $ jdb -attach 5005
  ```

  Then use breakpoints, backtraces, monitors, kotlin source and your favourite Java
  GUI debugger to debug the problem.

Links
==

[1]: https://salsa.debian.org/java-team/kotlin/-/issues/11
[2]: https://salsa.debian.org/java-team/kotlin/-/issues/14
[3]: https://wiki.debian.org/Java/Packaging/Gradle#Debugging_Tips
[4]: https://wiki.debian.org/Projects/DebSrc3.0#How_to_use_multiple_upstream_tarballs_in_3.0_.28quilt.29_format.3F
[5]: https://raphaelhertzog.com/2010/09/07/how-to-use-multiple-upstream-tarballs-in-debian-source-packages/

- https://salsa.debian.org/java-team/kotlin/-/issues/11
- https://salsa.debian.org/java-team/kotlin/-/issues/14
- https://wiki.debian.org/Java/Packaging/Gradle#Debugging_Tips
- https://wiki.debian.org/Projects/DebSrc3.0#How_to_use_multiple_upstream_tarballs_in_3.0_.28quilt.29_format.3F
- https://raphaelhertzog.com/2010/09/07/how-to-use-multiple-upstream-tarballs-in-debian-source-packages/
