Update sysroots

README.md

@@ -15,6 +15,7 @@ It contains an amalgamation of tools, libraries and binaries for all the needs:
 We don't really need all of this amalgamation for the following reason:
 - we don't need cross-compiler and other tools, because we use LLVM infrastructure (clang, lld, llvm-ar, ...) and it supports cross-compilation by default;
 - we don't need C++ headers and libraries because we include libc++, libc++abi, LLVM's libunwind as a source code and compile it from sources during build process;
+- we don't need `libgcc_eh`, `libgcc_s`, sanitizer runtimes or any other GCC runtime: ClickHouse builds `compiler-rt` from sources and links with `-nodefaultlibs`;
 - we definitely don't need Fortran headers;
 
 The idea is to strip down the "toolchain" as much as possible and provide it as a submodule instead of tarball.
@@ -26,41 +27,148 @@ This gives us the following advantages:
 - avoid risks of supply-chain attacks;
 - allow to use custom sysroot even for default (non-cross) build to get reproducible, hermetic builds;
 - opens up for experiment of building the libc from sources;
-- simplify using musl-libc instead of glibc.
+- simplify using musl-libc instead of glibc (the musl build uses only the kernel headers from `linux-x86_64` / `linux-aarch64`; musl itself is built from sources in the main repository).
+
+## What every sysroot must contain
+
+Per target, exactly the files the clang driver and `lld` open when ClickHouse is linked with
+`-nodefaultlibs -lc -lm -lrt -lpthread -ldl` (see `cmake/linux/default_libs.cmake` and
+`cmake/freebsd/default_libs.cmake` in the main repository):
+
+| Piece | Why |
+|---|---|
+| `usr/include` (libc + kernel/uapi headers) | compilation |
+| `crt1.o`, `crti.o`, `crtn.o` (glibc / FreeBSD) and `Scrt1.o` where PIE is possible | program entry / `.init` / `.fini` |
+| `crtbegin.o`, `crtend.o` (+ `S` variants) from GCC, or from FreeBSD base | static constructors / EH frame registration |
+| `libgcc.a` | only linked by the Rust host build on x86_64 / aarch64 (`-lgcc`); kept on every target for uniformity, it is small |
+| `libc.so*`, `libm.so*`, `librt.so*`, `libpthread.so*`, `libdl.so*`, `libc_nonshared.a`, `ld-*.so*` | link-time symbol resolution |
+
+Anything else (static `libc.a`, `gconv`, GCC's own headers, `libgcc_eh`, `libgcc_s`, sanitizer
+runtimes, OpenSSL, C++ headers of the target OS, DWARF sections inside the shared objects) is dead
+weight and is intentionally not checked in. The shared objects are used by the linker only, so
+`llvm-strip --strip-debug` is safe and is applied where the upstream ships unstripped files.
+
+## The sysroots
+
+| Directory | Source | glibc / libc | Kernel headers | GCC files | Notes |
+|---|---|---|---|---|---|
+| `linux-x86_64` | Ubuntu 20.04 blobs; headers from Ubuntu 18.04 (`usr/include`) | glibc 2.27 | 4.15 | gcc 9 | **Defines the minimum glibc of the official x86_64 binaries.** Do not upgrade casually. |
+| `linux-aarch64` | Arm GNU Toolchain `gcc-arm-8.3-2019.03` (`aarch64-linux-gnu`) | glibc 2.28 | 4.19 | gcc 8.3 | **Defines the minimum glibc of the official aarch64 binaries.** Shared objects are stripped of debug info. |
+| `linux-s390x` | Docker `s390x/ubuntu:18.04` + `apt install gcc` | glibc 2.27 | 4.15 | gcc 11 | |
+| `linux-powerpc64le` | Docker `ppc64le/ubuntu:14.04` + `apt install gcc` | glibc 2.19 | 3.13 | gcc 4.8 | Deliberately old for compatibility with old ppc64le distributions. |
+| `linux-riscv64` | Debian 13 (trixie) packages via `refresh-debian-sysroot.sh riscv64` | glibc 2.41 | 6.12 | gcc 14 | Debian merged-`/usr` layout, see `SOURCE.txt` inside. |
+| `linux-loongarch64` | Debian unstable (sid) packages via `refresh-debian-sysroot.sh loongarch64` | glibc 2.43 | 7.1 | gcc 16 | loong64 is not in Debian 13; official only from Debian 14. See `SOURCE.txt` inside. |
+| `freebsd-x86_64` | `FreeBSD 14.5-RELEASE` `base.txz` via `refresh-freebsd-sysroot.sh` | FreeBSD libc 7 | - | crt from base | Target triple `x86_64-pc-freebsd14` |
+| `freebsd-aarch64` | same | | | | Target triple `aarch64-unknown-freebsd14` |
+| `freebsd-powerpc64le` | same | | | | Target triple `powerpc64le-unknown-freebsd14` |
+
+The linker records, for every glibc symbol, the newest symbol version present in the sysroot's
+`libc.so`. The official x86_64 and aarch64 binaries must not reference symbol versions newer than
+`GLIBC_2.4` / `GLIBC_2.18` (enforced by `ci/jobs/compatibility_check.py`); this is achieved by
+`base/glibc-compatibility` in the main repository, which provides its own definitions of the
+functions whose symbol versions are newer than that. A newer glibc in `linux-x86_64` /
+`linux-aarch64` means more symbols to shim and a real risk of silently raising the glibc floor of
+the release binaries, so these two are frozen. The other Linux sysroots have no such check.
+
+## How to update
+
+All updates happen in this repository on a branch, then the submodule pointer is bumped in
+`ClickHouse/ClickHouse`. In every case, after updating, build ClickHouse for the affected
+target (`cmake -DCMAKE_TOOLCHAIN_FILE=cmake/<os>/toolchain-<arch>.cmake ...`) and link
+`programs/clickhouse` before opening the pull request; the CI jobs `amd_freebsd`, `ppc64le`,
+`riscv64`, `s390x`, `loongarch64`, `amd_musl` cover the cross targets.
+
+### FreeBSD (`freebsd-x86_64`, `freebsd-aarch64`, `freebsd-powerpc64le`)
+
+```bash
+# pick the release at https://download.freebsd.org/releases/ (old ones move to https://archive.freebsd.org/old-releases/)
+./refresh-freebsd-sysroot.sh https://download.freebsd.org/releases/amd64/14.5-RELEASE/base.txz
+./refresh-freebsd-sysroot.sh https://download.freebsd.org/releases/arm64/aarch64/14.5-RELEASE/base.txz
+./refresh-freebsd-sysroot.sh https://download.freebsd.org/releases/powerpc/powerpc64le/14.5-RELEASE/base.txz
+```
 
-This repository contains some blobs like `libc.so`.
-The source:
-- for `x86_64` they are from Ubuntu 20.04 image. Include files `linux-x86_64/x86_64-linux-gnu/libc/usr/include` come from `usr/include/` from Ubuntu 18.04 image.
-- for `aarch64` they are from [developer.arm.com](https://developer.arm.com/-/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz?revision=2e88a73f-d233-4f96-b1f4-d8b36e9bb0b9&la=en)
+The script downloads `base.txz`, keeps `usr/include` (minus `c++` and `private`), the startup
+objects and the shared `libc`, `libm`, `librt`, `libthr` (installed as `libpthread.so`), and
+prints the resulting `__FreeBSD_version`.
 
-- for `s390x` it is extracted from Docker image:
-```
-docker run -it s390x/ubuntu:18.04
-apt update
-apt install gcc
+If the *major* version changes, also change the target triple (`x86_64-pc-freebsdNN`,
+`aarch64-unknown-freebsdNN`, `powerpc64le-unknown-freebsdNN`) in the main repository:
+`cmake/freebsd/toolchain-*.cmake` and `ci/jobs/build_toolchain.py` (`CROSS_BUILTIN_TARGETS`).
+The triple selects the libc symbol versions the binary is allowed to use, so a binary built with
+sysroot N and triple N-1 is wrong. Binaries built against release N run on N and newer.
 
-docker export b38a367a8a05 > s390x.tar
-```
+Stay on the oldest *supported* FreeBSD major (see https://www.freebsd.org/security/#sup): moving
+to a newer major raises the minimum FreeBSD version users need.
 
-- for `powerpc64le` it is extracted from Docker image:
+### Debian-based Linux (`linux-riscv64`, `linux-loongarch64`)
 
-```
-docker run -it ppc64le/ubuntu:14.04
-apt update
-apt install gcc
+Requires docker.
 
-docker export b38a367a8a05 > ppc64.tar
+```bash
+./refresh-debian-sysroot.sh riscv64        # Debian stable (trixie); override with DEBIAN_SUITE=...
+./refresh-debian-sysroot.sh loongarch64    # Debian unstable (sid): loong64 is not in stable yet
 ```
 
-The ubuntu version 14.04 is selected for better compatibility.
+The script runs a throwaway `debian:<suite>-slim` container, downloads `libc6`, `libc6-dev`,
+`linux-libc-dev` and the `libgcc-<N>-dev` of the suite's default GCC for the target architecture,
+unpacks them, keeps the pieces listed above and writes the exact package versions to
+`<sysroot>/SOURCE.txt`. The resulting layout is Debian's merged-`/usr`:
+`usr/include`, `usr/lib/<triple>`, `usr/lib/gcc/<triple>/<N>`, `lib -> usr/lib`.
 
-- for `x86_64-musl` some headers and libraries come from the ubuntu image, others are built from our musl fork: github.com/ClickHouse/musl (see https://github.com/ClickHouse/sysroot/pull/28)
-- for `riscv` they are from Debian Unstable [libc6-dev package](`https://packages.debian.org/sid/riscv64/libc6-dev`).
-- for `loongarch64` they are from Debian Unstable image
+The toolchain files pass `--sysroot=<sysroot>` and `--gcc-toolchain=<sysroot>/usr`; clang
+discovers the GCC version directory itself, so a new GCC major needs no cmake change.
 
-FreeBSD:
+To move a target from sid to stable once the architecture is released there, change the default
+`SUITE` in the script (or run with `DEBIAN_SUITE=`).
 
-- Run ./refresh-freebsd-sysroot.sh
+### Ubuntu-based Linux (`linux-x86_64`, `linux-s390x`, `linux-powerpc64le`)
+
+There is no script because these are frozen on purpose (glibc floor). If a refresh is ever
+needed, the procedure that produced them is:
+
+```bash
+docker run -it --platform linux/<amd64|s390x|ppc64le> ubuntu:<version>
+apt update && apt install gcc
+docker export <container-id> > rootfs.tar
+```
 
-TODO:
-- simplify directory structure even more.
+Then copy from the tarball, keeping the existing directory layout of the sysroot
+(`<triple>/libc/{lib,lib64,usr/include,usr/lib64}` and `lib/gcc/<triple>/<N>/`):
+
+- `usr/include/**` -> `<triple>/libc/usr/include/`
+- `lib/<triple>/{libc.so.6,libm.so.6,libpthread.so.0,librt.so.1,libdl.so.2,libresolv.so.2}` -> `<triple>/libc/lib64/`
+- `lib/<triple>/ld-*.so*` -> `<triple>/libc/lib/`
+- `usr/lib/<triple>/{crt1.o,Scrt1.o,crti.o,crtn.o,libc.so,libc_nonshared.a}` -> `<triple>/libc/usr/lib64/`
+- `usr/lib/gcc/<triple>/<N>/{crtbegin.o,crtbeginS.o,crtbeginT.o,crtend.o,crtendS.o,libgcc.a}` -> `lib/gcc/<triple>/<N>/`
+
+To only add kernel headers (the usual reason to touch `linux-x86_64`, e.g. a new `linux/*.h`),
+take them from the `linux-libc-dev` package of the *same* Ubuntu release and put them under
+`x86_64-linux-gnu/libc/usr/include/linux/`; do not mix a newer glibc's `bits/` headers with the old
+`libc.so`.
+
+### `linux-aarch64`
+
+Taken from the Arm GNU Toolchain release
+https://developer.arm.com/downloads/-/gnu-a/8-3-2019-03 (`gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz`).
+Newer Arm GNU Toolchain releases ship newer glibc (15.2.Rel1 = glibc 2.42) and would raise the
+glibc floor of the official aarch64 binaries, so the same caution as for `linux-x86_64` applies.
+The layout is the toolchain's own: `aarch64-linux-gnu/libc/{lib,lib64,usr/include,usr/lib64}` and
+`lib/gcc/aarch64-linux-gnu/<N>/`. After copying, run
+`llvm-strip --strip-debug` on the shared objects (the toolchain ships them with DWARF).
+
+### Adding a new Linux target
+
+1. Get a rootfs with glibc + gcc for the target (Debian packages via `refresh-debian-sysroot.sh`
+   with a new `case` entry are the easiest).
+2. Add `cmake/linux/toolchain-<arch>.cmake` in the main repository, modelled on
+   `toolchain-loongarch64.cmake` (`CMAKE_SYSROOT` = sysroot root, `--gcc-toolchain` = the
+   directory containing `lib/gcc/<triple>/<N>`).
+3. Add compiler-rt builtins sources for the architecture in `contrib/compiler-rt-cmake` and the
+   triple to `CROSS_BUILTIN_TARGETS` in `ci/jobs/build_toolchain.py`.
+
+## musl
+
+The musl build (`cmake/linux/toolchain-x86_64-musl.cmake`, `toolchain-aarch64-musl.cmake`) uses
+no musl sysroot: musl is built from sources (`contrib/musl`, `contrib/musl-cmake`), and the only
+thing taken from this repository is the kernel uapi headers of `linux-x86_64` / `linux-aarch64`
+(`cmake/musl.cmake` symlinks `linux/`, `asm/`, `asm-generic/`, ... into the build directory).

freebsd-aarch64/usr/include/__libunwind_config.h

@@ -53,6 +53,9 @@
 #    else
 #      define _LIBUNWIND_CURSOR_SIZE 66
 #    endif
+#  elif defined(__ILP32__)
+#    define _LIBUNWIND_CONTEXT_SIZE 21
+#    define _LIBUNWIND_CURSOR_SIZE 28
 #  else
 #    define _LIBUNWIND_CONTEXT_SIZE 21
 #    define _LIBUNWIND_CURSOR_SIZE 33
@@ -180,6 +183,10 @@
 #endif
 #define _LIBUNWIND_HIGHEST_DWARF_REGISTER                                      \
   _LIBUNWIND_HIGHEST_DWARF_REGISTER_LOONGARCH
+#elif defined(__wasm__)
+// Unused
+#define _LIBUNWIND_CONTEXT_SIZE 0
+#define _LIBUNWIND_CURSOR_SIZE 0
 # else
 #  error "Unsupported architecture."
 # endif

freebsd-aarch64/usr/include/alias.h

@@ -121,9 +121,6 @@ int		LibAliasSaveFragment(struct libalias *, void *_ptr);
 unsigned short	LibAliasInternetChecksum(struct libalias *, unsigned short *_ptr, int _nbytes);
 void		LibAliasSetTarget(struct libalias *, struct in_addr _target_addr);
 
-/* never used and never worked, to be removed in FreeBSD 14 */
-int		LibAliasCheckNewLink(struct libalias *);
-
 /* Transparent proxying routines. */
 int		LibAliasProxyRule(struct libalias *, const char *_cmd);
 

freebsd-aarch64/usr/include/archive.h

@@ -34,12 +34,15 @@
  * assert that ARCHIVE_VERSION_NUMBER >= 2012108.
  */
 /* Note: Compiler will complain if this does not match archive_entry.h! */
-#define	ARCHIVE_VERSION_NUMBER 3007004
+#define	ARCHIVE_VERSION_NUMBER 3008007
 
 #include <sys/stat.h>
 #include <stddef.h>  /* for wchar_t */
 #include <stdio.h> /* For FILE * */
+#if ARCHIVE_VERSION_NUMBER < 4000000
+/* time_t is slated to be removed from public includes in 4.0 */
 #include <time.h> /* For time_t */
+#endif
 
 /*
  * Note: archive.h is for use outside of libarchive; the configuration
@@ -63,12 +66,15 @@
 #define __LA_INT64_T_DEFINED
 # if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__WATCOMC__)
 typedef __int64 la_int64_t;
+typedef unsigned __int64 la_uint64_t;
 # else
 # include <unistd.h>  /* ssize_t */
 #  if defined(_SCO_DS) || defined(__osf__)
 typedef long long la_int64_t;
+typedef unsigned long long la_uint64_t;
 #  else
 typedef int64_t la_int64_t;
+typedef uint64_t la_uint64_t;
 #  endif
 # endif
 #endif
@@ -94,6 +100,22 @@ typedef ssize_t la_ssize_t;
 # endif
 #endif
 
+#if ARCHIVE_VERSION_NUMBER < 4000000
+/* Use the platform types for time_t */
+#define __LA_TIME_T time_t
+#else
+/* Use 64-bits integer types for time_t */
+#define __LA_TIME_T la_int64_t
+#endif
+
+#if ARCHIVE_VERSION_NUMBER < 4000000
+/* Use the platform types for dev_t */
+#define __LA_DEV_T dev_t
+#else
+/* Use 64-bits integer types for dev_t */
+#define __LA_DEV_T la_int64_t
+#endif
+
 /* Large file support for Android */
 #if defined(__LIBARCHIVE_BUILD) && defined(__ANDROID__)
 #include "android_lf.h"
@@ -132,7 +154,7 @@ typedef ssize_t la_ssize_t;
 #define	__LA_PRINTF(fmtarg, firstvararg)	/* nothing */
 #endif
 
-#if defined(__GNUC__) && __GNUC__ >= 3 && __GNUC_MINOR__ >= 1
+#if defined(__GNUC__) && (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1))
 # define __LA_DEPRECATED __attribute__((deprecated))
 #else
 # define __LA_DEPRECATED
@@ -155,7 +177,7 @@ __LA_DECL int		archive_version_number(void);
 /*
  * Textual name/version of the library, useful for version displays.
  */
-#define	ARCHIVE_VERSION_ONLY_STRING "3.7.4"
+#define	ARCHIVE_VERSION_ONLY_STRING "3.8.7"
 #define	ARCHIVE_VERSION_STRING "libarchive " ARCHIVE_VERSION_ONLY_STRING
 __LA_DECL const char *	archive_version_string(void);
 
@@ -178,6 +200,23 @@ __LA_DECL const char *  archive_liblzma_version(void);
 __LA_DECL const char *  archive_bzlib_version(void);
 __LA_DECL const char *  archive_liblz4_version(void);
 __LA_DECL const char *  archive_libzstd_version(void);
+__LA_DECL const char *  archive_liblzo2_version(void);
+__LA_DECL const char *  archive_libexpat_version(void);
+__LA_DECL const char *  archive_libbsdxml_version(void);
+__LA_DECL const char *  archive_libxml2_version(void);
+__LA_DECL const char *  archive_mbedtls_version(void);
+__LA_DECL const char *  archive_nettle_version(void);
+__LA_DECL const char *  archive_openssl_version(void);
+__LA_DECL const char *  archive_libmd_version(void);
+__LA_DECL const char *  archive_commoncrypto_version(void);
+__LA_DECL const char *  archive_cng_version(void);
+__LA_DECL const char *  archive_wincrypt_version(void);
+__LA_DECL const char *  archive_librichacl_version(void);
+__LA_DECL const char *  archive_libacl_version(void);
+__LA_DECL const char *  archive_libattr_version(void);
+__LA_DECL const char *  archive_libiconv_version(void);
+__LA_DECL const char *  archive_libpcre_version(void);
+__LA_DECL const char *  archive_libpcre2_version(void);
 
 /* Declare our basic types. */
 struct archive;
@@ -449,6 +488,8 @@ __LA_DECL int archive_read_support_format_by_code(struct archive *, int);
 __LA_DECL int archive_read_support_format_cab(struct archive *);
 __LA_DECL int archive_read_support_format_cpio(struct archive *);
 __LA_DECL int archive_read_support_format_empty(struct archive *);
+/* archive_read_support_format_gnutar() is an alias for historical reasons
+ * of archive_read_support_format_tar(). */
 __LA_DECL int archive_read_support_format_gnutar(struct archive *);
 __LA_DECL int archive_read_support_format_iso9660(struct archive *);
 __LA_DECL int archive_read_support_format_lha(struct archive *);
@@ -828,6 +869,10 @@ __LA_DECL int archive_write_set_format_filter_by_ext(struct archive *a, const ch
 __LA_DECL int archive_write_set_format_filter_by_ext_def(struct archive *a, const char *filename, const char * def_ext);
 __LA_DECL int archive_write_zip_set_compression_deflate(struct archive *);
 __LA_DECL int archive_write_zip_set_compression_store(struct archive *);
+__LA_DECL int archive_write_zip_set_compression_lzma(struct archive *);
+__LA_DECL int archive_write_zip_set_compression_xz(struct archive *);
+__LA_DECL int archive_write_zip_set_compression_bzip2(struct archive *);
+__LA_DECL int archive_write_zip_set_compression_zstd(struct archive *);
 /* Deprecated; use archive_write_open2 instead */
 __LA_DECL int archive_write_open(struct archive *, void *,
 		     archive_open_callback *, archive_write_callback *,
@@ -1083,6 +1128,10 @@ __LA_DECL int		 archive_compression(struct archive *)
 				__LA_DEPRECATED;
 #endif
 
+/* Parses a date string relative to the current time.
+ * NOTE: This is not intended for general date parsing, and the resulting timestamp should only be used for libarchive. */
+__LA_DECL time_t	archive_parse_date(time_t now, const char *datestr);
+
 __LA_DECL int		 archive_errno(struct archive *);
 __LA_DECL const char	*archive_error_string(struct archive *);
 __LA_DECL const char	*archive_format_name(struct archive *);
@@ -1201,8 +1250,10 @@ __LA_DECL int	archive_match_include_gname_w(struct archive *,
 		    const wchar_t *);
 
 /* Utility functions */
+#if ARCHIVE_VERSION_NUMBER < 4000000
 /* Convenience function to sort a NULL terminated list of strings */
 __LA_DECL int archive_utility_string_sort(char **);
+#endif
 
 #ifdef __cplusplus
 }