Conversation
tuhaihe
force-pushed
the
binary-license-notice
branch
from
September 15, 2026 01:34
3be6cd4 to
efdf511
Compare
The RPM and DEB convenience packages currently ship the source release's
LICENSE, NOTICE and licenses/ verbatim. Those describe the source
release, not the packages, and the two differ in both directions.
Missing from the source release but present in the packages:
- Apache Xerces-C++ 3.3.0 (Apache-2.0), built from the Apache source
release in the Rocky build images and copied into lib/; bin/postgres
links against the bundled copy rather than a system library. RPM
packages only -- DEB packages use the distribution's libxerces-c.
- psutil 5.7.0 (BSD-3-Clause) in lib/python/psutil/
- PyYAML 5.4.1 (MIT) in lib/python/yaml/
- PyGreSQL 5.2 (PostgreSQL License) in lib/python/pg.py, pgdb.py and
_pg*.so
The three Python modules are downloaded and built during packaging (see
gpMgmt/bin/Makefile), so they are deliberately absent from the source
release and can only be declared by a binary-specific LICENSE.
Present in the source release but absent from the packages. Every entry
was checked against the build tree and the installed artifacts rather
than assumed:
- src/port/inet_aton.c not built; pg_config.h has HAVE_INET_ATON,
and bin/postgres resolves inet_aton to
glibc (U inet_aton@GLIBC_2.2.5)
- src/port/glob.c Windows only, not built
- src/port/win32gettimeofday.c Windows only, not built
- src/common/sha2.c the fallback SHA-2 implementation is not
built in an OpenSSL configuration;
libpgcommon.a carries cryptohash_openssl.o
instead. Its header is a different matter,
see below.
- src/test/locale/test-ctype.c test only, not installed
- googletest, googlebench and cpp-stub, which contribute no symbols to
pax.so, libpaxformat.so or gpcloud.so and are reached only from
pax_gtest.cc, pax_gbench.cc and the *_test.cc files
- dependency/yyjson built only when USE_MANIFEST_API is ON and
USE_PAX_CATALOG is OFF, which is not the
default and not what the release build
uses; pax.so and libpaxformat.so contain
no yyjson symbols
- amalgamate.py the Python tool tabulate uses to generate
its single header; nothing compiles it
What remains was confirmed positively: the object files exist in the
build tree, the loadable modules and executables ship, and the symbols
are present in them -- http_parser and ini in gpcloud.so, regex, pg_cron,
inet_net_pton and Ryu in bin/postgres, tabulate in both pax.so and
libpaxformat.so.
Two entries are worth calling out because the obvious answer is wrong.
Ryu is compiled in: src/common/Makefile lists d2s.o and f2s.o and
bin/postgres carries its symbols. And while src/common/sha2.c is not
built, src/common/sha2_int.h is still needed: it supplies Aaron D.
Gifford's context structures, sm3.c uses them throughout, sm3.o is built,
and lib/libpgcommon.a ships in the packages.
LICENSE-binary, NOTICE-binary and licenses-binary/ describe the packages
instead, following the convention Apache Spark and Apache Kafka use.
NOTICE-binary carries Xerces-C's NOTICE contents as required by section
4(d) of the Apache License. Components whose presence depends on the
package format are labelled as such rather than split into per-format
files.
build-rpm.sh and both debian/rules install the -binary variants under
their plain names, so an installed GPHOME still has LICENSE, NOTICE and
licenses/ -- now describing what is actually in it. The compliance
files are treated as required inputs: packaging fails if they are
missing, instead of emitting a warning and producing a package without
them.
Verified by building both package formats on Rocky 9 and Ubuntu 22.04:
GPHOME/LICENSE, GPHOME/NOTICE and GPHOME/licenses/ match the -binary
sources byte for byte, licenses/ holds exactly the 23 expected files with
none of the dropped ones, debian/copyright is generated from the -binary
files, and every path LICENSE-binary names is present in the built RPM.
Assisted-by: Claude Code
tuhaihe
pushed a commit
to tuhaihe/gpbackup
that referenced
this pull request
Sep 16, 2026
The source release bundles no third-party code: there is no vendor/ directory, so LICENSE and NOTICE describe the source tree correctly. The convenience binary packages are a different artifact. They are statically linked Go binaries and therefore physically contain the code of every module in the build graph, plus the Go runtime and standard library and, because CGO is enabled for SQLite support, the SQLite amalgamation. Until now `make package` copied the source LICENSE and NOTICE verbatim into those packages, where the only third-party component listed is the Greenplum-derived code. That describes neither the 53 Go modules linked into the binaries nor the NOTICE files that eight of them ship, which the Apache License 2.0 requires us to propagate in section 4(d). Add the -binary variants, following the convention used by Apache Spark, Apache Kafka and apache/cloudberry#1994: LICENSE-binary -> installed as LICENSE NOTICE-binary -> installed as NOTICE licenses-binary/ -> installed as licenses/ They are generated by scripts/generate-binary-license.sh rather than maintained by hand. The script resolves the modules actually linked into each shipped binary on each released platform, so test-only dependencies such as Ginkgo and Gomega are left out, while platform-gated modules that appear only on Linux are included. A dependency bump that changes the inventory is caught by the new binary-license-check CI job, which runs the script with --check. The reproduced license texts carry their own upstream copyright notices and must not be given ASF headers, so they are excluded from the RAT scan. Verified against the actual artifacts: building the packages on Linux and reading the module list back out of the six shipped binaries with `go version -m` matches LICENSE-binary exactly, module for module and version for version. Backpatch-through: REL_2_STABLE
tuhaihe
added a commit
to tuhaihe/gpbackup
that referenced
this pull request
Sep 16, 2026
The source release bundles no third-party code: there is no vendor/ directory, so LICENSE and NOTICE describe the source tree correctly. The convenience binary packages are a different artifact. They are statically linked Go binaries and therefore physically contain the code of every module in the build graph, plus the Go runtime and standard library and, because CGO is enabled for SQLite support, the SQLite amalgamation. Until now `make package` copied the source LICENSE and NOTICE verbatim into those packages, where the only third-party component listed is the Greenplum-derived code. That describes neither the 53 Go modules linked into the binaries nor the NOTICE files that eight of them ship, which the Apache License 2.0 requires us to propagate in section 4(d). Add the -binary variants, following the convention used by Apache Spark, Apache Kafka and apache/cloudberry#1994: LICENSE-binary -> installed as LICENSE NOTICE-binary -> installed as NOTICE licenses-binary/ -> installed as licenses/ They are generated by scripts/generate-binary-license.sh rather than maintained by hand. The script resolves the modules actually linked into each shipped binary on each released platform, so test-only dependencies such as Ginkgo and Gomega are left out, while platform-gated modules that appear only on Linux are included. A dependency bump that changes the inventory is caught by the new binary-license-check CI job, which runs the script with --check. The reproduced license texts carry their own upstream copyright notices and must not be given ASF headers, so they are excluded from the RAT scan. Verified against the actual artifacts: building the packages on Linux and reading the module list back out of the six shipped binaries with `go version -m` matches LICENSE-binary exactly, module for module and version for version. Backpatch-through: REL_2_STABLE
tuhaihe
added a commit
to tuhaihe/gpbackup
that referenced
this pull request
Sep 16, 2026
The source release bundles no third-party code: there is no vendor/ directory, so LICENSE and NOTICE describe the source tree correctly. The convenience binary packages are a different artifact. They are statically linked Go binaries and therefore physically contain the code of every module in the build graph, plus the Go runtime and standard library and, because CGO is enabled for SQLite support, the SQLite amalgamation. Until now `make package` copied the source LICENSE and NOTICE verbatim into those packages, where the only third-party component listed is the Greenplum-derived code. That describes neither the 53 Go modules linked into the binaries nor the NOTICE files that eight of them ship, which the Apache License 2.0 requires us to propagate in section 4(d). Add the -binary variants, following the convention used by Apache Spark, Apache Kafka and apache/cloudberry#1994: LICENSE-binary -> installed as LICENSE NOTICE-binary -> installed as NOTICE licenses-binary/ -> installed as licenses/ They are generated by scripts/generate-binary-license.sh rather than maintained by hand. The script resolves the modules actually linked into each shipped binary on each released platform, so test-only dependencies such as Ginkgo and Gomega are left out, while platform-gated modules that appear only on Linux are included. A dependency bump that changes the inventory is caught by the new binary-license-check CI job, which runs the script with --check. The reproduced license texts carry their own upstream copyright notices and must not be given ASF headers, so they are excluded from the RAT scan. Verified against the actual artifacts: building the packages on Linux and reading the module list back out of the six shipped binaries with `go version -m` matches LICENSE-binary exactly, module for module and version for version. Backpatch-through: REL_2_STABLE
tuhaihe
added a commit
to tuhaihe/cloudberry-pxf
that referenced
this pull request
Sep 16, 2026
The convenience RPM, DEB and tarball are not the source release. `make stage` builds the PXF server into a Spring Boot application JAR that bundles its entire runtime classpath under BOOT-INF/lib: 144 third-party components on top of PXF's own 8 modules, including Hadoop, Hive, HBase, Parquet, ORC, Avro, the AWS and Azure SDKs, Spring, embedded Tomcat and ZooKeeper. None of that is in the source tree, yet the repository's LICENSE and NOTICE were what the packages installed, so the compliance files in GPHOME described something other than the package contents: * LICENSE listed only PXF's own source directories plus one Dremio-derived script. * The RPM `License:` tag said `ASL 2.0`, while the bundle also carries BSD-2, BSD-3, MIT, EDL-1.0, CC0 and public-domain components, and jakarta.annotation-api under EPL-2.0 / GPL-2.0-with-classpath-exception. That last one is ASF Category B, which may ship in binary form only, and only when labelled. * The DEB built its payload tree straight from the module stage directories, so it shipped no LICENSE, NOTICE or DISCLAIMER at all. Following Apache Spark, Apache Kafka and apache/cloudberry#1994, add a separate set for the binary distribution: LICENSE-binary -> installed in the package as LICENSE NOTICE-binary -> installed in the package as NOTICE licenses-binary/ -> installed in the package as licenses/ DISCLAIMER -> installed in the package as DISCLAIMER `make stage` (used by `make tar` and `make rpm`) and `make deb` both install them and fail if any is missing. The repository root LICENSE and NOTICE keep describing the source release; a scan of the tree found no third-party copyright they fail to cover, so they are left unchanged. At 152 bundled JARs a hand-maintained list would rot within a release, so package/licensing/ holds a curated inventory plus a generator: * bundled-components.tsv names every JAR, its coordinate, its license and, when it is not plain Apache-2.0, its text under licenses-binary/. Third-party rows pin the exact file name, so a version bump forces the license to be re-reviewed. * generate-binary-license.py --generate rewrites LICENSE-binary and NOTICE-binary from the built JAR, folding in the NOTICE files of the 65 components that ship one. --check fails on any drift: a JAR added or removed, a missing or orphaned license text, a Makefile License: tag that no longer covers the inventory, or an ASF Category X license. A new lightweight workflow runs --check on every relevant pull request. It builds only the server, so it needs no Apache Cloudberry installation. License texts come from the shipped JARs where they carry one, and otherwise from the component's upstream repository at the tag matching the bundled version. Two are reconstructed and say so in their notes. Verified on Rocky 9 and Ubuntu 22.04 against real packages: the RPM `License:` tag now carries the full SPDX expression, both the RPM and the DEB install LICENSE, NOTICE, DISCLAIMER and 21 license texts under licenses/, staging fails when a compliance file is missing, and `mvn clean verify` reports 0 unapproved licenses. See: https://www.apache.org/legal/resolved.html Backpatch-through: REL_2_STABLE
tuhaihe
added a commit
to tuhaihe/cloudberry-pxf
that referenced
this pull request
Sep 16, 2026
The convenience RPM, DEB and tarball are not the source release. `make stage` builds the PXF server into a Spring Boot application JAR that bundles its entire runtime classpath under BOOT-INF/lib: 144 third-party components on top of PXF's own 8 modules, including Hadoop, Hive, HBase, Parquet, ORC, Avro, the AWS and Azure SDKs, Spring, embedded Tomcat and ZooKeeper. None of that is in the source tree, yet the repository's LICENSE and NOTICE were what the packages installed, so the compliance files in GPHOME described something other than the package contents: * LICENSE listed only PXF's own source directories plus one Dremio-derived script. * The RPM `License:` tag said `ASL 2.0`, while the bundle also carries BSD-2, BSD-3, MIT, EDL-1.0, CC0 and public-domain components, and jakarta.annotation-api under EPL-2.0 / GPL-2.0-with-classpath-exception. That last one is ASF Category B, which may ship in binary form only, and only when labelled. * The DEB built its payload tree straight from the module stage directories, so it shipped no LICENSE, NOTICE or DISCLAIMER at all. Following Apache Spark, Apache Kafka and apache/cloudberry#1994, add a separate set for the binary distribution: LICENSE-binary -> installed in the package as LICENSE NOTICE-binary -> installed in the package as NOTICE licenses-binary/ -> installed in the package as licenses/ DISCLAIMER -> installed in the package as DISCLAIMER `make stage` (used by `make tar` and `make rpm`) and `make deb` both install them and fail if any is missing. The repository root LICENSE and NOTICE keep describing the source release; a scan of the tree found no third-party copyright they fail to cover, so they are left unchanged. At 152 bundled JARs a hand-maintained list would rot within a release, so package/licensing/ holds a curated inventory plus a generator: * bundled-components.tsv names every JAR, its coordinate, its license and, when it is not plain Apache-2.0, its text under licenses-binary/. Third-party rows pin the exact file name, so a version bump forces the license to be re-reviewed. * generate-binary-license.py --generate rewrites LICENSE-binary and NOTICE-binary from the built JAR, folding in the NOTICE files of the 65 components that ship one. --check fails on any drift: a JAR added or removed, a missing or orphaned license text, a Makefile License: tag that no longer covers the inventory, or an ASF Category X license. A new lightweight workflow runs --check on every relevant pull request. It builds only the server, so it needs no Apache Cloudberry installation. License texts come from the shipped JARs where they carry one, and otherwise from the component's upstream repository at the tag matching the bundled version. Two are reconstructed and say so in their notes. Verified on Rocky 9 and Ubuntu 22.04 against real packages: the RPM `License:` tag now carries the full SPDX expression, both the RPM and the DEB install LICENSE, NOTICE, DISCLAIMER and 21 license texts under licenses/, staging fails when a compliance file is missing, and `mvn clean verify` reports 0 unapproved licenses. See: https://www.apache.org/legal/resolved.html Backpatch-through: REL_2_STABLE
tuhaihe
force-pushed
the
binary-license-notice
branch
from
September 17, 2026 03:38
efdf511 to
22a7748
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions