diff --git a/docs/publishing/README.md b/docs/publishing/README.md index 147227b6b..ba5d9d48d 100644 --- a/docs/publishing/README.md +++ b/docs/publishing/README.md @@ -295,41 +295,98 @@ You may want to publish the shadowed JAR instead of the original JAR. This can b ``` Because the default `archiveClassifier` of [`Jar`][Jar] is `""` (empty), setting the `archiveClassifier` of -[`ShadowJar`][ShadowJar] to `""` (empty) will make collisions between the outputs of these two tasks in some cases. If -you don't need the standard JAR, you can disable the `jar` task like: +[`ShadowJar`][ShadowJar] to `""` (empty) will make collisions between the outputs of standard tasks and `shadowJar`: + +- The binary shadowed JAR is output to `-.jar`, conflicting with the `jar` task. +- When `generateSourcesJar` is enabled (such as when `java.withSourcesJar()` is used), the companion shadowed sources + JAR is output to `--sources.jar`, conflicting with the standard `sourcesJar` task. + +If you want to replace standard JARs with the shadowed ones, disable the standard `jar` and `sourcesJar` tasks: === ":material-language-kotlin: build.gradle.kts" ```kotlin + plugins { + java + id("com.gradleup.shadow") + } + + java { + withSourcesJar() + } + tasks.jar { enabled = false } + + tasks.named("sourcesJar") { + enabled = false + } ``` === ":simple-apachegroovy: build.gradle" ```groovy + plugins { + id('java') + id('com.gradleup.shadow') + } + + java { + withSourcesJar() + } + tasks.named('jar', Jar) { enabled = false } + + tasks.named('sourcesJar', Jar) { + enabled = false + } ``` -Or set a different `archiveClassifier` for the standard [`Jar`][Jar] like: +Or set different `archiveClassifier` values for the standard tasks: === ":material-language-kotlin: build.gradle.kts" ```kotlin + plugins { + java + id("com.gradleup.shadow") + } + + java { + withSourcesJar() + } + tasks.jar { archiveClassifier = "ignored" } + + tasks.named("sourcesJar") { + (this as org.gradle.jvm.tasks.Jar).archiveClassifier = "ignored-sources" + } ``` === ":simple-apachegroovy: build.gradle" ```groovy + plugins { + id('java') + id('com.gradleup.shadow') + } + + java { + withSourcesJar() + } + tasks.named('jar', Jar) { archiveClassifier = 'ignored' } + + tasks.named('sourcesJar', Jar) { + archiveClassifier = 'ignored-sources' + } ``` ## Publishing the Shadowed Gradle Plugins @@ -588,16 +645,156 @@ When Gradle's standard `java.withSourcesJar()` is enabled, the Shadow plugin aut The published Maven publication will include both `--all.jar` and `--all-sources.jar`. +### Local File Names vs. Published Classifiers + +The Shadow plugin distinguishes between the **local output file** on disk and the **published artifact classifier** in +Maven repositories and Gradle Module Metadata: + +| Configuration | Local Output File (`archiveSourcesFile` in `build/libs`) | Published Classifier | Published File (Maven Repository) | Use Case | +|:----------------------------------------|:---------------------------------------------------------|:---------------------|:--------------------------------------------|:---------------------------------------------------| +| `archiveClassifier = "all"` *(default)* | `--all-sources.jar` | `all-sources` | `--all-sources.jar` | **Coexistence** (coexists with standard `sources`) | +| `archiveClassifier = "shaded"` | `--shaded-sources.jar` | `shaded-sources` | `--shaded-sources.jar` | **Coexistence** (custom classifier) | +| `archiveClassifier = ""` | `--sources.jar` | `sources` | `--sources.jar` | **Replacement** (replaces standard `sources`) | + +#### Coexistence Scenario + +When publishing alongside standard Java artifacts (e.g. publishing `from(components["java"])` with +`shadow.addShadowVariantIntoJavaComponent = true`), the standard sources variant uses classifier `sources`. To prevent +coordinate collisions within the same publication, the shadowed sources variant dynamically derives its classifier as +`-sources` (such as `all-sources` or `shaded-sources`). + +#### Replacement Scenario + +When configuring `shadowJar` to replace the standard JAR (`archiveClassifier = ""`), the companion shadowed sources JAR +automatically uses the standard `sources` classifier. + +To publish shadowed artifacts as the primary publication: + +1. **Publish from `components["shadow"]` (Recommended)**: Publish the `shadow` component directly in your Maven + publication, and disable standard archive tasks to prevent destination file collisions in `build/libs`: + +=== ":material-language-kotlin: build.gradle.kts" + + ```kotlin + plugins { + java + `maven-publish` + id("com.gradleup.shadow") + } + + java { + withSourcesJar() + } + + tasks.jar { + enabled = false + } + + tasks.named("sourcesJar") { + enabled = false + } + + tasks.shadowJar { + archiveClassifier = "" + } + + publishing { + publications { + create("shadow") { + from(components["shadow"]) + } + } + } + ``` + +=== ":simple-apachegroovy: build.gradle" + + ```groovy + plugins { + id 'java' + id 'maven-publish' + id 'com.gradleup.shadow' + } + + java { + withSourcesJar() + } + + tasks.named('jar', Jar) { + enabled = false + } + + tasks.named('sourcesJar', Jar) { + enabled = false + } + + tasks.named('shadowJar', com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar) { + archiveClassifier = '' + } + + publishing { + publications { + shadow(MavenPublication) { + from components.shadow + } + } + } + ``` + +2. **Publish from `components["java"]`**: If publishing `from(components["java"])`, disabling the `jar` or `sourcesJar` + tasks does not remove standard variants from the `java` software component. You must also explicitly skip the + standard publication variants: + +=== ":material-language-kotlin: build.gradle.kts" + + ```kotlin + plugins { + java + `maven-publish` + id("com.gradleup.shadow") + } + + java { + withSourcesJar() + } + + components.named("java") { + withVariantsFromConfiguration(configurations["runtimeElements"]) { skip() } + withVariantsFromConfiguration(configurations["sourcesElements"]) { skip() } + } + ``` + +=== ":simple-apachegroovy: build.gradle" + + ```groovy + plugins { + id 'java' + id 'maven-publish' + id 'com.gradleup.shadow' + } + + java { + withSourcesJar() + } + + components.named('java', org.gradle.api.component.AdhocComponentWithVariants) { + withVariantsFromConfiguration(configurations.runtimeElements) { skip() } + withVariantsFromConfiguration(configurations.sourcesElements) { skip() } + } + ``` + > [!NOTE] > Generating the companion shadowed sources JAR is controlled by [`generateSourcesJar`][ShadowJar.generateSourcesJar]. > In Java projects, it defaults to `true` when `java.withSourcesJar()` is enabled, and `false` otherwise to avoid > unnecessary build overhead for application builds. If `withSourcesJar()` is omitted, publishing from -> `components["shadow"]` will only publish the shadowed binary JAR, preserving backward compatibility for existing builds. +> `components["shadow"]` will only publish the shadowed binary JAR, preserving backward compatibility for existing +builds. > You can also explicitly toggle generation via `generateSourcesJar = true` (or `--generate-sources-jar`). ### Customizing the Sources Archive File -The companion shadowed sources JAR output location is configured via [`ShadowJar.archiveSourcesFile`][ShadowJar.archiveSourcesFile], +The companion shadowed sources JAR output location is configured via +[`ShadowJar.archiveSourcesFile`][ShadowJar.archiveSourcesFile], which defaults to the same destination and base name as `archiveFile` with `-sources.jar` suffix: === ":material-language-kotlin: build.gradle.kts" diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/BasePluginTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/BasePluginTest.kt index 0c3b31f68..77b64140f 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/BasePluginTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/BasePluginTest.kt @@ -408,10 +408,16 @@ abstract class BasePluginTest { } } - fun createEmptyClassBytes(internalName: String): ByteArray { + fun createEmptyClassBytes( + internalName: String, + sourceFile: String? = "${internalName.substringAfterLast('/')}.java", + ): ByteArray { return ClassWriter(0) .apply { visit(Opcodes.V1_8, Opcodes.ACC_PUBLIC, internalName, null, "java/lang/Object", null) + if (sourceFile != null) { + visitSource(sourceFile, null) + } visitEnd() } .toByteArray() diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/FilteringTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/FilteringTest.kt index d349a07d8..74211d1c5 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/FilteringTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/FilteringTest.kt @@ -2,8 +2,6 @@ package com.github.jengelman.gradle.plugins.shadow import assertk.assertThat import com.github.jengelman.gradle.plugins.shadow.testkit.classLoader -import com.github.jengelman.gradle.plugins.shadow.testkit.containsAtLeast -import com.github.jengelman.gradle.plugins.shadow.testkit.containsNone import com.github.jengelman.gradle.plugins.shadow.testkit.containsOnly import com.github.jengelman.gradle.plugins.shadow.testkit.loadClass import kotlin.io.path.appendText @@ -260,12 +258,10 @@ class FilteringTest : BasePluginTest() { runWithSuccess(shadowJarPath) assertThat(outputShadowedJar).useAll { - containsAtLeast("g/G.class") - containsNone("h/H.class", "h/UnusedH.class") + containsOnly(*entriesInAB, "g/", "g/G.class", *manifestEntries) } assertThat(outputShadowedSourcesJar).useAll { - containsAtLeast("g/G.java") - containsNone("h/H.java", "h/UnusedH.java") + containsOnly("g/", "g/G.java", *manifestEntries) } } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/JavaPluginsTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/JavaPluginsTest.kt index 979623bd8..fe75032fe 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/JavaPluginsTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/JavaPluginsTest.kt @@ -295,7 +295,7 @@ class JavaPluginsTest : BasePluginTest() { // The fact that server compiled successfully against `client.junit.framework.Test` // means it consumed the shadowed artifact during compilation. assertThat(jarPath("server/build/libs/server-1.0.jar")).useAll { - containsAtLeast("server/Server.class") + containsOnly("server/", "server/Server.class", *manifestEntries) } } @@ -1176,7 +1176,13 @@ class JavaPluginsTest : BasePluginTest() { runWithSuccess(":app:$SHADOW_JAR_TASK_NAME") assertThat(jarPath("app/build/libs/app-all.jar")).useAll { - containsAtLeast("com/company/Main.class", "com/company/Utils.class", manifestEntry) + containsOnly( + "com/", + "com/company/", + "com/company/Main.class", + "com/company/Utils.class", + *manifestEntries, + ) } } @@ -1316,6 +1322,69 @@ class JavaPluginsTest : BasePluginTest() { ) } + @Test + fun sourcesJarPreservesResourceRelativePath() { + writeClass() + path("src/main/resources/config/sub/app.properties").writeText("key=value") + + projectScript.appendText( + """ + |$shadowJarTask { + | generateSourcesJar = true + |} + """ + .trimMargin() + ) + + runWithSuccess(shadowJarPath) + + assertThat(outputShadowedSourcesJar).useAll { + containsOnly( + "my/", + "config/", + "config/sub/", + "my/Main.java", + "config/sub/app.properties", + *manifestEntries, + ) + } + } + + @Test + fun sourcesJarHandlesOverlappingSourceDirectoryPrefixes() { + writeClass() + path("src/main/res/a.properties").writeText("a=1") + path("src/main/resources/b.properties").writeText("b=2") + + projectScript.appendText( + """ + |sourceSets { + | main { + | resources { + | srcDir 'src/main/res' + | } + | } + |} + |$shadowJarTask { + | generateSourcesJar = true + |} + """ + .trimMargin() + ) + + runWithSuccess(shadowJarPath) + + assertThat(outputShadowedSourcesJar).useAll { + containsOnly( + "my/", + "my/Main.java", + "a.properties", + "b.properties", + *manifestEntries, + ) + } + } + private fun dependencies(configuration: String, vararg flags: String): String { return runWithSuccess("dependencies", "--configuration", configuration, *flags).output } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/KotlinPluginsTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/KotlinPluginsTest.kt index 76c98bdb3..f0b8d13c9 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/KotlinPluginsTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/KotlinPluginsTest.kt @@ -8,7 +8,6 @@ import com.github.jengelman.gradle.plugins.shadow.internal.mainClassAttributeKey import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar.Companion.SHADOW_JAR_TASK_NAME import com.github.jengelman.gradle.plugins.shadow.testkit.classLoader import com.github.jengelman.gradle.plugins.shadow.testkit.containsAtLeast -import com.github.jengelman.gradle.plugins.shadow.testkit.containsNone import com.github.jengelman.gradle.plugins.shadow.testkit.containsOnly import com.github.jengelman.gradle.plugins.shadow.testkit.getMainAttr import com.github.jengelman.gradle.plugins.shadow.testkit.loadClass @@ -337,6 +336,7 @@ class KotlinPluginsTest : BasePluginTest() { @Test fun generateShadowedSourcesJarNormalizesPackageDirectory() { + val stdlib = compileOnlyStdlib(true) path("src/main/kotlin/FlatFile.kt") .writeText( """ @@ -349,6 +349,9 @@ class KotlinPluginsTest : BasePluginTest() { projectScript.writeText( """ |${getDefaultProjectBuildScript(plugin = "org.jetbrains.kotlin.jvm")} + |dependencies { + | $stdlib + |} |$shadowJarTask { | generateSourcesJar = true | relocate 'my.custom', 'shadow.custom' @@ -360,8 +363,13 @@ class KotlinPluginsTest : BasePluginTest() { runWithSuccess(shadowJarPath) assertThat(outputShadowedSourcesJar).useAll { - containsAtLeast("shadow/custom/nested/FlatFile.kt") - containsNone("FlatFile.kt") + containsOnly( + "shadow/", + "shadow/custom/", + "shadow/custom/nested/", + "shadow/custom/nested/FlatFile.kt", + *manifestEntries, + ) } } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/MinimizeTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/MinimizeTest.kt index 7239ad0e4..7868cf770 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/MinimizeTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/MinimizeTest.kt @@ -156,12 +156,26 @@ class MinimizeTest : BasePluginTest() { runWithSuccess(shadowJarPath) assertThat(outputShadowedJar).useAll { - containsAtLeast("my/Main.class", "h/H.class", "k/CustomUtils.class") - containsNone("h/UnusedH.class", "k/CustomUnusedUtils.class") + containsOnly( + "my/", + "h/", + "k/", + "my/Main.class", + "h/H.class", + "k/CustomUtils.class", + *manifestEntries, + ) } assertThat(outputShadowedSourcesJar).useAll { - containsAtLeast("my/Main.java", "h/H.java", "k/Utils.kt") - containsNone("h/UnusedH.java", "k/UnusedUtils.kt") + containsOnly( + "my/", + "h/", + "k/", + "my/Main.java", + "h/H.java", + "k/Utils.kt", + *manifestEntries, + ) } } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/PublishingTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/PublishingTest.kt index 4c3fab5e3..783a68314 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/PublishingTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/PublishingTest.kt @@ -348,6 +348,158 @@ class PublishingTest : BasePluginTest() { assertShadowSourcesVariantCommon(gmm) } + @Test + fun publishWithSourcesJarAndCustomClassifier() { + projectScript.appendText( + publishConfiguration( + projectBlock = + """ + |java { + | withSourcesJar() + |} + """ + .trimMargin(), + shadowBlock = + """ + |archiveClassifier = 'shaded' + |archiveSourcesFile = layout.buildDirectory.file('custom.jar') + """ + .trimMargin(), + publicationsBlock = + """ + |shadow(MavenPublication) { + | from components.shadow + |} + """ + .trimMargin(), + ) + ) + + publish() + + val artifactRoot = "my/maven/1.0" + assertThat(repoPath(artifactRoot).entries.filter { it.endsWith(".jar") }) + .containsOnly( + "maven-1.0-shaded.jar", + "maven-1.0-shaded-sources.jar", + ) + val gmm = gmmAdapter.fromJson(repoPath("$artifactRoot/maven-1.0.module")) + assertThat(gmm.shadowSourcesElementsVariant.fileNames.single()) + .isEqualTo("maven-1.0-shaded-sources.jar") + } + + @Test + fun publishWithSourcesJarAndCustomClassifierAfterPublishingBlock() { + projectScript.appendText( + """ + |apply plugin: 'maven-publish' + |java { + | withSourcesJar() + |} + |publishing { + | repositories { + | maven { url = '${remoteRepoPath.toUri()}' } + | } + | publications { + | shadow(MavenPublication) { + | from components.shadow + | } + | } + |} + |tasks.named('shadowJar', com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar) { + | archiveClassifier = 'shaded' + | archiveSourcesFile = layout.buildDirectory.file('custom.jar') + |} + """ + .trimMargin() + ) + + publish() + + val artifactRoot = "my/maven/1.0" + assertThat(repoPath(artifactRoot).entries.filter { it.endsWith(".jar") }) + .containsOnly( + "maven-1.0-shaded.jar", + "maven-1.0-shaded-sources.jar", + ) + val gmm = gmmAdapter.fromJson(repoPath("$artifactRoot/maven-1.0.module")) + assertThat(gmm.shadowSourcesElementsVariant.fileNames.single()) + .isEqualTo("maven-1.0-shaded-sources.jar") + } + + @Test + fun publishJavaComponentWithShadowAndSourcesVariants() { + projectScript.appendText( + publishConfiguration( + projectBlock = + """ + |java { + | withSourcesJar() + |} + """ + .trimMargin(), + publicationsBlock = + """ + |shadow(MavenPublication) { + | from components.java + |} + """ + .trimMargin(), + ) + ) + + publish() + + val artifactRoot = "my/maven/1.0" + assertThat(repoPath(artifactRoot).entries.filter { it.endsWith(".jar") }) + .containsOnly( + "maven-1.0.jar", + "maven-1.0-sources.jar", + "maven-1.0-all.jar", + "maven-1.0-all-sources.jar", + ) + } + + @Test + fun dontPublishSourcesWhenGenerateSourcesJarDisabled() { + projectScript.appendText( + publishConfiguration( + projectBlock = + """ + |java { + | withSourcesJar() + |} + """ + .trimMargin(), + shadowBlock = + """ + |archiveClassifier = '' + |generateSourcesJar = false + """ + .trimMargin(), + publicationsBlock = + """ + |shadow(MavenPublication) { + | from components.shadow + |} + """ + .trimMargin(), + ) + ) + + val result = publish(infoArgument) + + assertThat(result.output) + .contains("Skipping adding shadowSourcesElements variant to shadow component.") + val artifactRoot = "my/maven/1.0" + assertThat(repoPath(artifactRoot).entries.filter { it.contains("sources") }).isEmpty() + assertShadowJarCommon(repoJarPath("$artifactRoot/maven-1.0.jar")) + assertPomCommon(repoPath("$artifactRoot/maven-1.0.pom")) + val gmm = gmmAdapter.fromJson(repoPath("$artifactRoot/maven-1.0.module")) + assertShadowVariantCommon(gmm) + assertThat(gmm.variantNames).containsOnly(SHADOW_RUNTIME_ELEMENTS_CONFIGURATION_NAME) + } + @Test fun publishCustomShadowJar() { projectScript.appendText( @@ -767,21 +919,31 @@ class PublishingTest : BasePluginTest() { ) assertThat(repoJarPath("$artifactRoot/my-all-1.0.jar")).useAll { - containsAtLeast( + containsOnly( + "my/", + "g/", + "h/", "my/CommonMain.class", "my/JvmMain.class", "g/G.class", "h/H.class", + "h/UnusedH.class", + "META-INF/my_maven.kotlin_module", *manifestEntries, ) } assertThat(repoJarPath("$artifactRoot/my-all-1.0-sources.jar")).useAll { - containsAtLeast( + containsOnly( + "my/", + "g/", + "h/", "my/CommonMain.kt", "my/JvmMain.kt", "g/G.java", "h/H.java", + "h/UnusedH.java", + *manifestEntries, ) } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/RelocationTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/RelocationTest.kt index 8f509f158..7879115f7 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/RelocationTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/RelocationTest.kt @@ -882,6 +882,51 @@ class RelocationTest : BasePluginTest() { } } + @Test + fun relocateShadowedSourcesJarRespectsSourceDirectorySetFilters() { + path("src/main/java/my/Main.java") + .writeText( + """ + |package my; + |public class Main {} + """ + .trimMargin() + ) + path("src/main/java/my/Excluded.java") + .writeText( + """ + |package my; + |public class Excluded {} + """ + .trimMargin() + ) + projectScript.appendText( + """ + |sourceSets { + | main { + | java { + | exclude '**/Excluded.java' + | } + | } + |} + |$shadowJarTask { + | generateSourcesJar = true + |} + """ + .trimMargin() + ) + + runWithSuccess(shadowJarPath) + + assertThat(outputShadowedSourcesJar).useAll { + containsOnly( + "my/", + "my/Main.java", + *manifestEntries, + ) + } + } + @Test fun generateShadowedSourcesJarWithCustomIncludedSourcesJars() { writeClass() diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/TransformersTest.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/TransformersTest.kt index a707a2fc6..8f901d4bc 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/TransformersTest.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/TransformersTest.kt @@ -219,7 +219,7 @@ class TransformersTest : BaseTransformerTest() { runWithSuccess(shadowJarPath) assertThat(outputShadowedJar).useAll { - containsOnly("META-INF/", "META-INF/LICENSE", *manifestEntries) + containsOnly("META-INF/LICENSE", *manifestEntries) getContent("META-INF/LICENSE") .isEqualTo( """ @@ -379,7 +379,7 @@ class TransformersTest : BaseTransformerTest() { runWithSuccess(shadowJarPath) assertThat(outputShadowedJar).useAll { - containsOnly("META-INF/", "META-INF/kotlin-stdlib.shadow.kotlin_module", *manifestEntries) + containsOnly("META-INF/kotlin-stdlib.shadow.kotlin_module", *manifestEntries) getBytes("META-INF/kotlin-stdlib.shadow.kotlin_module").isNotEqualTo(moduleBytes) } } diff --git a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/util/LocalMavenRepository.kt b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/util/LocalMavenRepository.kt index 26dfb6d0e..5eead420c 100644 --- a/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/util/LocalMavenRepository.kt +++ b/src/functionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/util/LocalMavenRepository.kt @@ -80,8 +80,11 @@ fun createDefaultLocalMavenRepository(junitJar: Path): AppendableMavenRepository val k = jarModule("my", "k", "1.0") { buildJar { - insert("k/CustomUtils.class", createEmptyClassBytes("k/CustomUtils")) - insert("k/CustomUnusedUtils.class", createEmptyClassBytes("k/CustomUnusedUtils")) + insert("k/CustomUtils.class", createEmptyClassBytes("k/CustomUtils", "Utils.kt")) + insert( + "k/CustomUnusedUtils.class", + createEmptyClassBytes("k/CustomUnusedUtils", "UnusedUtils.kt"), + ) } buildSourcesJar { insert( diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.kt index 9ed6780d7..0a842bb83 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.kt @@ -44,10 +44,18 @@ constructor(private val softwareComponentFactory: SoftwareComponentFactory) : Pl val taskProvider = registerShadowJarCommon(tasks.named("jar", Jar::class.java)) { task -> task.from(mainSourceSet.map { it.output }) - task.sourceSetsSourceDirs.convention(mainSourceSet.map { it.allSource.srcDirs }) task.generateSourcesJar.convention( provider { configurations.findByName(SOURCES_ELEMENTS_CONFIGURATION_NAME) != null } ) + task.sourceSetsSourceDirs.convention( + task.generateSourcesJar.flatMap { generate -> + if (generate) { + mainSourceSet.map { it.allSource.sourceDirectories + it.allSource } + } else { + provider { emptySet() } + } + } + ) task.configurations.convention(provider { listOf(runtimeConfiguration) }) } artifacts.add(configurations.shadow.name, taskProvider) @@ -85,10 +93,21 @@ constructor(private val softwareComponentFactory: SoftwareComponentFactory) : Pl objects.named(DocsType::class.java, DocsType.SOURCES), ) } - outgoing.artifact(tasks.shadowJar.flatMap { it.archiveSourcesFile }) { artifact -> - artifact.builtBy(tasks.shadowJar) - artifact.classifier = "sources" - artifact.type = "jar" + val shadowJarTask = tasks.shadowJar + outgoing.artifact(shadowJarTask.flatMap { it.archiveSourcesFile }) { artifact -> + with(artifact) { + builtBy(shadowJarTask) + name = shadowJarTask.flatMap { it.archiveBaseName }.orNull.orEmpty() + extension = shadowJarTask.flatMap { it.archiveExtension }.orNull ?: "jar" + type = "jar" + classifier = + shadowJarTask + .flatMap { it.archiveClassifier } + .orNull + .let { shadowClassifier -> + if (shadowClassifier.isNullOrEmpty()) "sources" else "$shadowClassifier-sources" + } + } } } @@ -130,8 +149,11 @@ constructor(private val softwareComponentFactory: SoftwareComponentFactory) : Pl val addIntoJavaComponent = shadow.addShadowVariantIntoJavaComponent val shadowRuntimeElements = configurations.shadowRuntimeElements val shadowSourcesElements = configurations.shadowSourcesElements - // If `withSourcesJar` is present. - val sourcesElements = { configurations.findByName(SOURCES_ELEMENTS_CONFIGURATION_NAME) } + // If `withSourcesJar` is present and `generateSourcesJar` is enabled. + val shouldAddSources = { + configurations.findByName(SOURCES_ELEMENTS_CONFIGURATION_NAME) != null && + tasks.shadowJar.flatMap { it.generateSourcesJar }.get() + } val shadowComponent = softwareComponentFactory.adhoc(COMPONENT_NAME) components.add(shadowComponent) @@ -144,7 +166,7 @@ constructor(private val softwareComponentFactory: SoftwareComponentFactory) : Pl shadowComponent.addVariants( outgoingConfiguration = shadowSourcesElements, logger = logger, - shouldAdd = { sourcesElements() != null }, + shouldAdd = shouldAddSources, ) components.named("java", AdhocComponentWithVariants::class.java) { component -> @@ -158,7 +180,7 @@ constructor(private val softwareComponentFactory: SoftwareComponentFactory) : Pl component.addVariants( outgoingConfiguration = shadowSourcesElements, logger = logger, - shouldAdd = { addIntoJavaComponent.get() && sourcesElements() != null }, + shouldAdd = { addIntoJavaComponent.get() && shouldAddSources() }, ) { mapToOptional() } diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowKmpPlugin.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowKmpPlugin.kt index 68c1d389f..ddeba32c5 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowKmpPlugin.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowKmpPlugin.kt @@ -37,7 +37,17 @@ public abstract class ShadowKmpPlugin : Plugin { registerShadowJarCommon(tasks.named(target.artifactsTaskName, Jar::class.java)) { task -> task.from(kotlinJvmMain.map { it.output.allOutputs }) task.sourceSetsSourceDirs.convention( - kotlinJvmMain.map { it.allKotlinSourceSets.flatMap { ss -> ss.kotlin.srcDirs } } + task.generateSourcesJar.flatMap { generate -> + if (generate) { + kotlinJvmMain.map { + it.allKotlinSourceSets.flatMap { ss -> + listOf(ss.kotlin.sourceDirectories, ss.kotlin) + } + } + } else { + provider { emptySet() } + } + } ) task.configurations.convention( kotlinJvmMain diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/DefaultDependencyFilter.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/DefaultDependencyFilter.kt index 12edefe99..03ac02909 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/DefaultDependencyFilter.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/DefaultDependencyFilter.kt @@ -44,22 +44,22 @@ internal class DefaultDependencyFilter(@Transient private val project: Project) includedDependencies = includes, excludedDependencies = excludes, ) - val componentIds = + + val includedDependenciesResults = configuration.incoming.resolutionResult.allDependencies .filterIsInstance() - .map { it.selected.id } - .toSet() + .filter { dep -> + includes.any { inc -> + inc.moduleGroup == dep.selected.moduleVersion?.group && + inc.moduleName == dep.selected.moduleVersion?.name && + inc.moduleVersion == dep.selected.moduleVersion?.version + } + } val externalComponentIds = - componentIds + includedDependenciesResults + .map { it.selected.id } .filterIsInstance() - .filter { id -> - includes.any { - it.moduleGroup == id.group && - it.moduleName == id.module && - it.moduleVersion == id.version - } - } .toSet() val externalSourcesFiles = @@ -73,7 +73,12 @@ internal class DefaultDependencyFilter(@Transient private val project: Project) .filterIsInstance() .map { it.file } - val includedProjectNames = includes.map { it.moduleName }.toSet() + val projectComponentIds = + includedDependenciesResults + .map { it.selected.id } + .filterIsInstance() + .toSet() + val projectSourcesFiles = try { configuration.incoming @@ -89,9 +94,7 @@ internal class DefaultDependencyFilter(@Transient private val project: Project) project.objects.named(DocsType::class.java, DocsType.SOURCES), ) } - view.componentFilter { id -> - id is ProjectComponentIdentifier && id.projectName in includedProjectNames - } + view.componentFilter { id -> id in projectComponentIds } view.lenient(true) } .files diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJar.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJar.kt index 3a6e40c27..b94dc63e5 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJar.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJar.kt @@ -5,11 +5,16 @@ import com.github.jengelman.gradle.plugins.shadow.relocation.relocatePath import java.io.File import java.nio.charset.Charset import org.gradle.api.tasks.bundling.ZipEntryCompression +import org.vafer.jdeb.shaded.objectweb.asm.ClassReader +import org.vafer.jdeb.shaded.objectweb.asm.ClassVisitor +import org.vafer.jdeb.shaded.objectweb.asm.Opcodes internal fun generateShadowedSourcesJar( sourcesJarFile: File, sourceSetsSourceDirs: Iterable, includedSourcesJars: Iterable, + classesDirs: Iterable = emptyList(), + dependencies: Iterable = emptyList(), relocators: Iterable, unusedClasses: Set = emptySet(), entryCompression: ZipEntryCompression, @@ -18,10 +23,15 @@ internal fun generateShadowedSourcesJar( preserveFileTimestamps: Boolean, ) { val sourcesJars = includedSourcesJars.filter { it.exists() && it.isFile }.sortedBy { it.path } - if (sourceSetsSourceDirs.none() && sourcesJars.isEmpty()) return val visitedFiles = mutableSetOf() val charset = metadataCharset?.let(Charset::forName) ?: Charsets.UTF_8 + val sourceToClasses = + if (unusedClasses.isNotEmpty()) { + buildSourceToClassesMap(classesDirs = classesDirs, dependencies = dependencies) + } else { + emptyMap() + } try { sourcesJarFile @@ -41,54 +51,79 @@ internal fun generateShadowedSourcesJar( write("Manifest-Version: 1.0\n\n".toByteArray(charset)) } - val sortedSourceDirs = sourceSetsSourceDirs.filter { it.exists() }.sortedBy { it.path } - for (srcDir in sortedSourceDirs) { - srcDir - .walkTopDown() - .filter { it.isFile } - .toList() - .sortedBy { it.relativeTo(srcDir).invariantSeparatorsPath } - .forEach { file -> - val relPath = file.relativeTo(srcDir).invariantSeparatorsPath - val isSource = isSourceFile(relPath) - if (isSource) { - val text = file.readText(charset) - val pkg = extractPackage(text) - val simpleName = file.name - if (isUnused(simpleName, pkg, text, unusedClasses)) return@forEach - val canonicalPath = - if (pkg.isEmpty()) simpleName else "${pkg.replace('.', '/')}/$simpleName" - val relocatedPath = relocators.relocatePath(canonicalPath) - if (visitedFiles.add(relocatedPath)) { - var transformedText = text - for (relocator in relocators) { - transformedText = relocator.applyToSourceContent(transformedText) - } - val bytes = transformedText.toByteArray(charset) - zos.writeEntry( - name = relocatedPath, - preserveLastModified = preserveFileTimestamps, - lastModified = file.lastModified(), - unixMode = UnixMode.file(), - ) { - write(bytes) - } - } - } else { - val relocatedPath = relocators.relocatePath(relPath) - if (visitedFiles.add(relocatedPath)) { - val bytes = file.readBytes() - zos.writeEntry( - name = relocatedPath, - preserveLastModified = preserveFileTimestamps, - lastModified = file.lastModified(), - unixMode = UnixMode.file(), - ) { - write(bytes) - } - } + val sourceItems = sourceSetsSourceDirs.filter { it.exists() } + val (dirs, files) = sourceItems.partition { it.isDirectory } + val normalizedDirs = + dirs.map { it to it.normalize().toPath() }.sortedByDescending { it.second.nameCount } + + val filesWithRelPaths = mutableListOf>() + val coveredDirs = mutableSetOf() + + for (file in files.sortedBy { it.path }) { + val filePath = file.normalize().toPath() + val matchingDir = + normalizedDirs.firstOrNull { (_, dirPath) -> filePath.startsWith(dirPath) }?.first + if (matchingDir != null) { + coveredDirs.add(matchingDir) + filesWithRelPaths.add(file to file.relativeTo(matchingDir).invariantSeparatorsPath) + } else { + filesWithRelPaths.add(file to file.name) + } + } + + for ((dir, dirPath) in normalizedDirs.sortedBy { it.second.nameCount }) { + if (coveredDirs.none { dirPath.startsWith(it.normalize().toPath()) }) { + coveredDirs.add(dir) + dir + .walkTopDown() + .filter { it.isFile } + .toList() + .sortedBy { it.relativeTo(dir).invariantSeparatorsPath } + .forEach { f -> + filesWithRelPaths.add(f to f.relativeTo(dir).invariantSeparatorsPath) + } + } + } + + for ((file, relPath) in filesWithRelPaths) { + val isSource = isSourceFile(relPath) + if (isSource) { + val text = file.readText(charset) + val pkg = extractPackage(text) + val simpleName = file.name + val canonicalPath = + if (pkg.isEmpty()) simpleName else "${pkg.replace('.', '/')}/$simpleName" + if (isUnused(canonicalPath, unusedClasses, sourceToClasses)) continue + val relocatedPath = relocators.relocatePath(canonicalPath) + if (visitedFiles.add(relocatedPath)) { + var transformedText = text + for (relocator in relocators) { + transformedText = relocator.applyToSourceContent(transformedText) + } + val bytes = transformedText.toByteArray(charset) + zos.writeEntry( + name = relocatedPath, + preserveLastModified = preserveFileTimestamps, + lastModified = file.lastModified(), + unixMode = UnixMode.file(), + ) { + write(bytes) + } + } + } else { + val relocatedPath = relocators.relocatePath(relPath) + if (visitedFiles.add(relocatedPath)) { + val bytes = file.readBytes() + zos.writeEntry( + name = relocatedPath, + preserveLastModified = preserveFileTimestamps, + lastModified = file.lastModified(), + unixMode = UnixMode.file(), + ) { + write(bytes) } } + } } sourcesJars.forEach { jarFile -> @@ -113,9 +148,9 @@ internal fun generateShadowedSourcesJar( val text = getInputStream(entry).bufferedReader(charset).readText() val pkg = extractPackage(text) val simpleName = name.substringAfterLast('/') - if (isUnused(simpleName, pkg, text, unusedClasses)) return@forEach val canonicalPath = if (pkg.isEmpty()) simpleName else "${pkg.replace('.', '/')}/$simpleName" + if (isUnused(canonicalPath, unusedClasses, sourceToClasses)) return@forEach val relocatedPath = relocators.relocatePath(canonicalPath) if (visitedFiles.add(relocatedPath)) { var transformedText = text @@ -171,34 +206,87 @@ internal fun generateShadowedSourcesJar( private val packageRegex = """(?:^|\n)\s*package\s+([a-zA-Z0-9_.]+)""".toRegex() -private val jvmNameRegex = - """@file\s*:\s*(?:\[[^]]*?)?(?:kotlin\s*\.\s*jvm\s*\.\s*)?JvmName\s*\(\s*(?:name\s*=\s*)?"([^"]+)"""" - .toRegex() - internal fun extractPackage(text: String): String { val matches = packageRegex.findAll(text).map { it.groupValues[1] }.toList() return if (matches.isEmpty()) "" else matches.joinToString(".") } +internal fun buildSourceToClassesMap( + classesDirs: Iterable, + dependencies: Iterable, +): Map> { + val sourceToClasses = mutableMapOf>() + + fun processClassBytes(bytes: ByteArray) { + try { + var internalName: String? = null + var sourceFile: String? = null + ClassReader(bytes) + .accept( + object : ClassVisitor(Opcodes.ASM9) { + override fun visit( + version: Int, + access: Int, + name: String, + signature: String?, + superName: String?, + interfaces: Array?, + ) { + internalName = name + super.visit(version, access, name, signature, superName, interfaces) + } + + override fun visitSource(source: String?, debug: String?) { + sourceFile = source + super.visitSource(source, debug) + } + }, + ClassReader.SKIP_CODE or ClassReader.SKIP_FRAMES, + ) + + val name = internalName ?: return + val source = sourceFile ?: return + val pkg = name.substringBeforeLast('/', "") + val canonicalSourcePath = if (pkg.isEmpty()) source else "$pkg/$source" + val className = name.replace('/', '.') + sourceToClasses.getOrPut(canonicalSourcePath) { mutableSetOf() }.add(className) + } catch (_: Exception) { + // Ignore invalid class files + } + } + + for (dir in classesDirs.filter(File::isDirectory)) { + dir + .walkTopDown() + .filter { it.isFile && it.name.endsWith(".class") } + .forEach { file -> processClassBytes(file.readBytes()) } + } + + for (file in + dependencies.filter { it.isFile && (it.extension == "jar" || it.extension == "zip") }) { + try { + file.useZip { + entries() + .toList() + .filter { !it.isDirectory && it.name.endsWith(".class") } + .forEach { entry -> processClassBytes(getInputStream(entry).readBytes()) } + } + } catch (_: Exception) { + // Ignore invalid archives + } + } + + return sourceToClasses +} + internal fun isUnused( - fileName: String, - pkg: String, - text: String, + canonicalPath: String, unusedClasses: Set, + sourceToClasses: Map>, ): Boolean { if (unusedClasses.isEmpty()) return false - val simpleName = fileName.substringBeforeLast('.') - val className = if (pkg.isEmpty()) simpleName else "$pkg.$simpleName" - if (unusedClasses.contains(className)) return true - - if (fileName.endsWith(".kt")) { - val customJvmName = jvmNameRegex.find(text)?.groupValues?.get(1) - val facadeName = customJvmName ?: "${simpleName}Kt" - val facadeClassName = if (pkg.isEmpty()) facadeName else "$pkg.$facadeName" - if (unusedClasses.contains(facadeClassName)) return true - } - - return false + val classes = sourceToClasses[canonicalPath] ?: return false + return classes.isNotEmpty() && classes.all { it in unusedClasses } } private fun isSourceFile(path: String): Boolean { diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocator.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocator.kt index c9677d0d6..1e33e9d26 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocator.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocator.kt @@ -66,32 +66,16 @@ constructor( if (!excludes.isNullOrEmpty()) { this.excludes.addAll(excludes) } - - if (!rawString) { - // Create exclude pattern sets for sources. - for (exclude in this.excludes) { - // Excludes should be subpackages of the global pattern. - if (exclude.startsWith(this.pattern)) { - sourcePackageExcludes.add( - exclude.substring(this.pattern.length).replaceFirst("[.][*]$".toRegex(), "") - ) - } - // Excludes should be subpackages of the global pattern. - if (exclude.startsWith(pathPattern)) { - sourcePathExcludes.add( - exclude.substring(pathPattern.length).replaceFirst("/[*]$".toRegex(), "") - ) - } - } - } } public open fun include(pattern: String) { includes.addAll(normalizePatterns(listOf(pattern))) + includes.add(pattern) } public open fun exclude(pattern: String) { excludes.addAll(normalizePatterns(listOf(pattern))) + excludes.add(pattern) } override fun canRelocatePath(path: String): Boolean { @@ -128,10 +112,26 @@ constructor( } override fun applyToSourceContent(sourceContent: String): String { - if (rawString) return sourceContent + if (rawString || pattern.isEmpty()) return sourceContent + val sourceIncludes = getSourceSubpatterns(includes, pattern) + val sourceExcludes = getSourceSubpatterns(excludes, pattern) val content = - shadeSourceWithExcludes(sourceContent, pattern, shadedPattern, sourcePackageExcludes) - return shadeSourceWithExcludes(content, pathPattern, shadedPathPattern, sourcePathExcludes) + shadeSourceWithFilters( + sourceContent = sourceContent, + patternFrom = pattern, + patternTo = shadedPattern, + includedPatterns = sourceIncludes, + hasIncludes = includes.isNotEmpty(), + excludedPatterns = sourceExcludes, + ) + return shadeSourceWithFilters( + sourceContent = content, + patternFrom = pathPattern, + patternTo = shadedPathPattern, + includedPatterns = sourceIncludes, + hasIncludes = includes.isNotEmpty(), + excludedPatterns = sourceExcludes, + ) } override fun equals(other: Any?): Boolean { @@ -143,8 +143,6 @@ constructor( pathPattern == other.pathPattern && shadedPattern == other.shadedPattern && shadedPathPattern == other.shadedPathPattern && - sourcePackageExcludes == other.sourcePackageExcludes && - sourcePathExcludes == other.sourcePathExcludes && includes == other.includes && excludes == other.excludes } @@ -157,8 +155,6 @@ constructor( pathPattern, shadedPattern, shadedPathPattern, - sourcePackageExcludes, - sourcePathExcludes, includes, excludes, ) @@ -171,8 +167,6 @@ constructor( append("pathPattern='$pathPattern'").append(", ") append("shadedPattern='$shadedPattern'").append(", ") append("shadedPathPattern='$shadedPathPattern'").append(", ") - append("sourcePackageExcludes=$sourcePackageExcludes").append(", ") - append("sourcePathExcludes=$sourcePathExcludes").append(", ") append("includes=$includes").append(", ") append("excludes=$excludes") append(")") @@ -239,31 +233,71 @@ constructor( } } - fun shadeSourceWithExcludes( + private fun getSourceSubpatterns(patterns: Set, patternPrefix: String): Set { + if (patternPrefix.isEmpty()) return emptySet() + val result = mutableSetOf() + val dotPrefix = patternPrefix.replace('/', '.') + val slashPrefix = patternPrefix.replace('.', '/') + val trailingWildcardRegex = "[./][*]+$".toRegex() + + for (pat in patterns) { + val dotPat = pat.replace('/', '.') + if (dotPat.startsWith(dotPrefix)) { + val sub = dotPat.substring(dotPrefix.length).replaceFirst(trailingWildcardRegex, "") + if (sub.isEmpty()) { + result.add("") + } else { + result.add(sub) + result.add(sub.replace('.', '/')) + } + } + val slashPat = pat.replace('.', '/') + if (slashPat.startsWith(slashPrefix)) { + val sub = slashPat.substring(slashPrefix.length).replaceFirst(trailingWildcardRegex, "") + if (sub.isEmpty()) { + result.add("") + } else { + result.add(sub) + result.add(sub.replace('/', '.')) + } + } + } + return result + } + + private fun matchesSubpattern(snippet: String, subpattern: String): Boolean { + if (!snippet.startsWith(subpattern)) return false + if (subpattern.isEmpty() || snippet.length == subpattern.length) return true + if (subpattern.endsWith('.') || subpattern.endsWith('/')) return true + val nextChar = snippet[subpattern.length] + return !nextChar.isLetterOrDigit() && nextChar != '_' + } + + private fun shadeSourceWithFilters( sourceContent: String, patternFrom: String, patternTo: String, + includedPatterns: Set, + hasIncludes: Boolean, excludedPatterns: Set, ): String { - // Usually shading makes package names a bit longer, so make buffer 10% bigger than original - // source. + if (hasIncludes && includedPatterns.isEmpty()) { + return sourceContent + } + val shadedSourceContent = StringBuilder(sourceContent.length * 11 / 10) - // Make sure that search pattern starts at word boundary and that we look for literal ".", not - // regex jokers. val snippets = sourceContent .split(("\\b" + patternFrom.replace(".", "[.]") + "\\b").toRegex()) .filter(CharSequence::isNotEmpty) + snippets.forEachIndexed { i, snippet -> val isFirstSnippet = i == 0 val previousSnippet = if (isFirstSnippet) "" else snippets[i - 1] - var doExclude = false - for (excludedPattern in excludedPatterns) { - if (snippet.startsWith(excludedPattern)) { - doExclude = true - break - } - } + + val isIncluded = !hasIncludes || includedPatterns.any { matchesSubpattern(snippet, it) } + val isExcluded = excludedPatterns.any { matchesSubpattern(snippet, it) } + if (isFirstSnippet) { shadedSourceContent.append(snippet) } else { @@ -271,8 +305,9 @@ constructor( val afterDotSlashSpace = RX_ENDS_WITH_DOT_SLASH_SPACE.matcher(previousSnippetOneLine).find() val afterJavaKeyWord = RX_ENDS_WITH_JAVA_KEYWORD.matcher(previousSnippetOneLine).find() - val shouldExclude = doExclude || afterDotSlashSpace && !afterJavaKeyWord - shadedSourceContent.append(if (shouldExclude) patternFrom else patternTo).append(snippet) + val shouldRelocate = + isIncluded && !isExcluded && (!afterDotSlashSpace || afterJavaKeyWord) + shadedSourceContent.append(if (shouldRelocate) patternTo else patternFrom).append(snippet) } } return shadedSourceContent.toString() diff --git a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.kt b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.kt index e1ed6c22e..98fab688c 100644 --- a/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.kt +++ b/src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowJar.kt @@ -201,6 +201,21 @@ public abstract class ShadowJar : Jar() { dependencyFilter.zip(configurations) { df, cs -> df.resolve(cs) } } + /** + * If `true`, generates a companion shadowed sources JAR containing project and dependency + * sources. + * + * In projects applying the `shadow` plugin for Java, this convention defaults to `true` when + * `java.withSourcesJar()` is enabled, and `false` otherwise. + */ + @get:Input + @get:Option( + option = "generate-sources-jar", + description = + "Generates a companion shadowed sources JAR containing project and dependency sources.", + ) + public open val generateSourcesJar: Property = objectFactory.property(false) + /** * Source JARs resolved from bundled dependencies to be merged into the companion shadowed sources * JAR. @@ -208,8 +223,16 @@ public abstract class ShadowJar : Jar() { @get:InputFiles @get:PathSensitive(PathSensitivity.NONE) public open val includedSourcesJars: ConfigurableFileCollection = objectFactory.fileCollection { - dependencyFilter.zip(configurations) { df, cs -> - (df as? DefaultDependencyFilter)?.resolveSourcesJars(cs) ?: project.files() + // Avoid resolving sources JARs during task input snapshotting when sources JAR generation is + // disabled. + generateSourcesJar.flatMap { + if (it) { + dependencyFilter.zip(configurations) { df, cs -> + (df as? DefaultDependencyFilter)?.resolveSourcesJars(cs) ?: project.files() + } + } else { + project.provider { emptySet() } + } } } @@ -224,21 +247,6 @@ public abstract class ShadowJar : Jar() { @get:PathSensitive(PathSensitivity.RELATIVE) public open val sourceSetsSourceDirs: ConfigurableFileCollection = objectFactory.fileCollection() - /** - * If `true`, generates a companion shadowed sources JAR containing project and dependency - * sources. - * - * In projects applying the `shadow` plugin for Java, this convention defaults to `true` when - * `java.withSourcesJar()` is enabled, and `false` otherwise. - */ - @get:Input - @get:Option( - option = "generate-sources-jar", - description = - "Generates a companion shadowed sources JAR containing project and dependency sources.", - ) - public open val generateSourcesJar: Property = objectFactory.property(false) - /** * The destination location of the companion shadowed sources JAR. * @@ -817,6 +825,8 @@ public abstract class ShadowJar : Jar() { sourcesJarFile = archiveSourcesFile.get().asFile, sourceSetsSourceDirs = sourceSetsSourceDirs.files, includedSourcesJars = includedSourcesJars.files, + classesDirs = sourceSetsClassesDirs.files, + dependencies = includedDependencies.files, relocators = relocators.get() + packageRelocators, unusedClasses = unusedClasses, entryCompression = entryCompression, diff --git a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowPropertiesTest.kt b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowPropertiesTest.kt index 04994e251..aae0aaf97 100644 --- a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowPropertiesTest.kt +++ b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/ShadowPropertiesTest.kt @@ -168,10 +168,7 @@ class ShadowPropertiesTest { isEqualTo(destinationDirectory.file("my-project-1.0.0-all-sources.jar").get().asFile) isEqualTo(projectDir.resolve("build/libs/my-project-1.0.0-all-sources.jar")) } - assertThat(sourceSetsSourceDirs.files) - .containsOnly( - *javaPluginExtension.sourceSets.getByName("main").allSource.srcDirs.toTypedArray() - ) + assertThat(sourceSetsSourceDirs.files).isEmpty() assertThat(includedSourcesJars.files).isEmpty() } } @@ -183,6 +180,13 @@ class ShadowPropertiesTest { javaPluginExtension.withSourcesJar() val shadowJarTask = tasks.shadowJar.get() assertThat(shadowJarTask.generateSourcesJar.get()).isTrue() + val mainSourceSet = javaPluginExtension.sourceSets.getByName("main") + assertThat(shadowJarTask.sourceSetsSourceDirs.files) + .containsOnly( + *(mainSourceSet.allSource.sourceDirectories + mainSourceSet.allSource) + .files + .toTypedArray() + ) } @Test diff --git a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJarTest.kt b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJarTest.kt index f02787e7d..8bfc411ab 100644 --- a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJarTest.kt +++ b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/SourcesJarTest.kt @@ -2,7 +2,7 @@ package com.github.jengelman.gradle.plugins.shadow.internal import assertk.assertFailure import assertk.assertThat -import assertk.assertions.containsAtLeast +import assertk.assertions.containsOnly import assertk.assertions.hasMessage import assertk.assertions.isEqualTo import assertk.assertions.isFalse @@ -67,115 +67,41 @@ class SourcesJarTest { val unusedSet = setOf( "com.example.UnusedJava", + "com.example.UnusedJava\$Inner", "com.example.UnusedKtClass", "com.example.DefaultFacadeKt", "com.example.CustomFacade", ) + val sourceToClasses = + mapOf( + "com/example/UnusedJava.java" to + setOf("com.example.UnusedJava", "com.example.UnusedJava\$Inner"), + "com/example/PartiallyUsedJava.java" to + setOf("com.example.UnusedJava", "com.example.UsedHelper"), + "com/example/UsedJava.java" to setOf("com.example.UsedJava"), + "com/example/UnusedKtClass.kt" to setOf("com.example.UnusedKtClass"), + "com/example/DefaultFacade.kt" to setOf("com.example.DefaultFacadeKt"), + "com/example/Utils.kt" to setOf("com.example.CustomFacade"), + "com/example/MixedUtils.kt" to setOf("com.example.CustomFacade", "com.example.UsedClass"), + "Main.java" to setOf("Main"), + ) - assertThat(isUnused("UnusedJava.java", "com.example", "class UnusedJava {}", unusedSet)) - .isTrue() - assertThat(isUnused("UsedJava.java", "com.example", "class UsedJava {}", unusedSet)).isFalse() + // All classes unused in file -> unused + assertThat(isUnused("com/example/UnusedJava.java", unusedSet, sourceToClasses)).isTrue() + assertThat(isUnused("com/example/UnusedKtClass.kt", unusedSet, sourceToClasses)).isTrue() + assertThat(isUnused("com/example/DefaultFacade.kt", unusedSet, sourceToClasses)).isTrue() + assertThat(isUnused("com/example/Utils.kt", unusedSet, sourceToClasses)).isTrue() + assertThat(isUnused("Main.java", setOf("Main"), sourceToClasses)).isTrue() - assertThat(isUnused("UnusedKtClass.kt", "com.example", "class UnusedKtClass", unusedSet)) - .isTrue() - assertThat( - isUnused( - "DefaultFacade.kt", - "com.example", - "fun topLevel() {}", - unusedSet, - ) - ) - .isTrue() - assertThat( - isUnused( - "Utils.kt", - "com.example", - """ - @file:JvmName("CustomFacade") - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isTrue() - assertThat( - isUnused( - "Utils.kt", - "com.example", - """ - @file:kotlin.jvm.JvmName(name = "CustomFacade") - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isTrue() - assertThat( - isUnused( - "UsedUtils.kt", - "com.example", - """ - @file:JvmName("UsedFacade") - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isFalse() - assertThat( - isUnused( - "BracketedUtils.kt", - "com.example", - """ - @file:[JvmName("CustomFacade")] - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isTrue() - assertThat( - isUnused( - "BracketedMultiUtils.kt", - "com.example", - """ - @file:[Suppress("unused") JvmName("CustomFacade")] - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isTrue() - assertThat( - isUnused( - "BracketedMultiUtilsReversed.kt", - "com.example", - """ - @file:[JvmName("CustomFacade") Suppress("unused")] - package com.example - fun util() {} - """ - .trimIndent(), - unusedSet, - ) - ) - .isTrue() + // At least one class is used in file -> NOT unused (kept!) + assertThat(isUnused("com/example/PartiallyUsedJava.java", unusedSet, sourceToClasses)).isFalse() + assertThat(isUnused("com/example/MixedUtils.kt", unusedSet, sourceToClasses)).isFalse() + assertThat(isUnused("com/example/UsedJava.java", unusedSet, sourceToClasses)).isFalse() - assertThat(isUnused("UnusedJava.java", "com.example", "class UnusedJava {}", emptySet())) - .isFalse() - assertThat(isUnused("Main.java", "", "class Main {}", setOf("Main"))).isTrue() - assertThat(isUnused("Main.java", "", "class Main {}", setOf("Other"))).isFalse() + // Unknown source file or empty unused set -> kept + assertThat(isUnused("com/example/Unknown.java", unusedSet, sourceToClasses)).isFalse() + assertThat(isUnused("com/example/UnusedJava.java", emptySet(), sourceToClasses)).isFalse() + assertThat(isUnused("Main.java", setOf("Other"), sourceToClasses)).isFalse() } @Test @@ -205,7 +131,15 @@ class SourcesJarTest { assertThat(outputJar.exists()).isTrue() val entries = ZipFile(outputJar).use { zip -> zip.entries().toList().map { it.name } } - assertThat(entries).containsAtLeast("shadow/example/nested/Mismatched.kt") + assertThat(entries) + .containsOnly( + "META-INF/", + "META-INF/MANIFEST.MF", + "shadow/", + "shadow/example/", + "shadow/example/nested/", + "shadow/example/nested/Mismatched.kt", + ) } @Test diff --git a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocatorTest.kt b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocatorTest.kt index 4a88226aa..030cfdb26 100644 --- a/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocatorTest.kt +++ b/src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/SimpleRelocatorTest.kt @@ -357,15 +357,54 @@ class SimpleRelocatorTest { assertThat(relocator.applyToSourceContent(sourceFile)).isEqualTo(sourceFile) } + @Test + fun relocateSourceFileWithPrefixCollision() { + val relocator = + SimpleRelocator( + "org.example", + "relocated.org.example", + includes = listOf("org.example.In"), + ) + val source = + """ + |import org.example.In; + |import org.example.Input; + |import org.example.In.Nested; + | + |public class Test { + | org.example.In a; + | org.example.Input b; + |} + """ + .trimMargin() + val expected = + """ + |import relocated.org.example.In; + |import org.example.Input; + |import relocated.org.example.In.Nested; + | + |public class Test { + | relocated.org.example.In a; + | org.example.Input b; + |} + """ + .trimMargin() + assertThat(relocator.applyToSourceContent(source)).isEqualTo(expected) + } + @Test fun relocateSourceWithExcludes() { - // Main relocator with in-/excludes + // Main relocator with excludes val relocator = SimpleRelocator( "org.apache.maven", "com.acme.maven", - listOf("foo.bar", "zot.baz"), - listOf("irrelevant.exclude", "org.apache.maven.exclude1", "org.apache.maven.sub.exclude2"), + excludes = + listOf( + "irrelevant.exclude", + "org.apache.maven.exclude1", + "org.apache.maven.sub.exclude2", + ), ) // Make sure not to replace variables 'io' and 'ioInput', package 'java.io' val ioRelocator = SimpleRelocator("io", "shaded.io") @@ -383,6 +422,70 @@ class SimpleRelocatorTest { .isEqualTo(relocatedFile) } + @Test + fun relocateSourceWithIncludes() { + val relocator = + SimpleRelocator( + "org.apache.maven", + "com.acme.maven", + includes = listOf("org.apache.maven.hello.*", "org.apache.maven.In"), + ) + val input = + """ + |package org.apache.maven.hello; + |import org.apache.maven.hello.World; + |import org.apache.maven.other.Other; + |import org.apache.maven.In; + |import org.apache.maven.NotIn; + """ + .trimMargin() + val expected = + """ + |package com.acme.maven.hello; + |import com.acme.maven.hello.World; + |import org.apache.maven.other.Other; + |import com.acme.maven.In; + |import org.apache.maven.NotIn; + """ + .trimMargin() + assertThat(relocator.applyToSourceContent(input)).isEqualTo(expected) + } + + @Test + fun relocateSourceWithDslExcludeAndInclude() { + val relocatorExclude = SimpleRelocator("org.apache.maven", "com.acme.maven") + relocatorExclude.exclude("org.apache.maven.exclude1.*") + val inputExclude = + """ + |import org.apache.maven.hello.World; + |import org.apache.maven.exclude1.Ex1; + """ + .trimMargin() + val expectedExclude = + """ + |import com.acme.maven.hello.World; + |import org.apache.maven.exclude1.Ex1; + """ + .trimMargin() + assertThat(relocatorExclude.applyToSourceContent(inputExclude)).isEqualTo(expectedExclude) + + val relocatorInclude = SimpleRelocator("org.apache.maven", "com.acme.maven") + relocatorInclude.include("org.apache.maven.hello.*") + val inputInclude = + """ + |import org.apache.maven.hello.World; + |import org.apache.maven.other.Other; + """ + .trimMargin() + val expectedInclude = + """ + |import com.acme.maven.hello.World; + |import org.apache.maven.other.Other; + """ + .trimMargin() + assertThat(relocatorInclude.applyToSourceContent(inputInclude)).isEqualTo(expectedInclude) + } + private companion object { val sourceFile = """