Release prep 4.0.0: SMP install package, webroot disk, and a shipped configuration that is actually gated - #255
Merged
Merged
Conversation
Makes `make package` produce an installable product instead of a bare load library, and clears two things that would have broken the release run itself. [distribution] follows ufsd and ftpd: the load modules go through SMP (FMID THTP400, target HTTPD.@vrm@.LINKLIB), the sample library rides beside it as its own XMIT restored by TSO RECEIVE. Both DD names fall out of the dataset names, so HTTPLOAD/AHTTPLOD is a one-shot choice -- it goes into the FMID's JCLIN and changing it later is a new functional level. THTP400 is verified free in the SMPPTS on mvsdev only; the CDS and ACDS store hashed member names, so LIST CDS/ACDS SYSMOD(THTP400) still has to answer RC 04 on both stands before the tag. All five modules ship, ABEND0C1 included: mbtdist packages every [[module]] and offers no exclude, and since 4.0.0 a module is inert unless a Parmlib route names it. samplib/httpd had DSN=HTTPD.LINKLIB hardcoded in its STEPLIB, so the shipped procedure would have pointed at an unversioned library while SMP installed into HTTPD.V4R0M0.LINKLIB. It now carries @linklib@, the placeholder ufsd's and ftpd's procedures use, which mbtdist replaces while staging. docs/installation.md is new and wired in as the archive's README -- the generated install job tells the operator to read it, and without the file the package shipped a dangling instruction. It documents what differs from the other two products: APF failure is fatal here rather than a warning, UFSD is optional but takes static file serving with it, the shipped HTTPPRM0 routes to mvsMF, and removal needs UCLIN because the install accepts the FMID. VERSION said 4.0.0 while project.toml said 4.0.0-dev. mvsrelease.py searches each version file for the *current* string, so `make release VERSION=4.0.0` would have bumped project.toml, failed to find 4.0.0-dev in VERSION, and aborted before any commit -- leaving a modified tree and no tag. Both files now carry the full pre-release string, as ftpd's and ufsd's do. The toolchain pin moves to libc370 1.0.3 and the ufsd range to >=1.2.1, both released 2026-08-23. Verified: `make modules` builds all five clean under -Werror against the new ufsd, and `make package` produces the SYSMOD, both jobs and a samplib XMIT whose extracted procedure reads DSN=HTTPD.V4R0M0D.LINKLIB.
The document planned by hand what mbt now generates, and it read as current while being wrong in several places. Its first two links -- ../SMP-COOKBOOK.md and ../SMP-INSTALLATION.md -- point at files that no longer exist. It listed six load modules including HTTPJES2 and HTTPDSL, which 4.0.0 stopped building. And its question round proposed two answers the [distribution] block settles the other way. Dropped: the spike (done 2026-08-08 in ../smptest, which proved SMP copies the LKLIB module rather than re-binding it), the MCS/JCLIN/allocation generators, and the plan to ship the sample library as ++MAC elements inside the SYSMOD -- an MCS card ends at column 72 and HTTPPRM0 has comment lines reaching 80. Recorded as reversed rather than quietly changed: - The target is HTTPD.@vrm@.LINKLIB, not SYS2.LINKLIB. Versioned product datasets mean no IPL for an APF entry and no collision with a system library, and the procedure keeps its STEPLIB. That answers the old trap T6: the SVC 244 self-authorization path stays the normal case. - ACCEPT rides along, where the old F4 said "never". Accepting the FMID fills the distribution library, which is the base a later RESTORE of a PTF returns to; without it a RESTORE would delete the module instead of reverting it. The cost is that the FMID becomes permanent and removal needs UCLIN. Service is still never accepted. What is left is what is actually open: the LIST CDS/ACDS check for THTP400 on both stands, a test install on the throwaway TTST400, the UFS webroot SMP cannot deliver, and whether to ship a setup job at all. The traps that still bite are kept; T8 and T5 are struck with the reason they no longer apply.
…g away Three corrections from a read-through against the live system. The shipped HTTPPRM0 left the /zosmf/* catch-all without an AUTH=, which is PUBLIC -- the whole mvsMF API, every dataset and job endpoint, to anyone who can reach the port. The running configuration on mvsdev has carried AUTH=TOKEN on that line all along; the pattern we ship did not. TOKEN is the right mode here rather than BASIC: it answers a bare 401 with no WWW-Authenticate, which is what an API client handles itself, and it keeps a browser from caching Basic credentials that outlive the token session. The commented-out display module lines had the same shape: uncommenting `# MOD=HTTPDM /.dm` produced a public arbitrary-storage reader. They now carry AUTH=BASIC, so uncommenting a line does what it looks like it does. docs/installation.md now says what the four non-server modules are for and where they are going. HTTPDSRV, HTTPDM, HTTPDMTT and ABEND0C1 are diagnostic, debugging and development tools -- they ship in 4.0.x so a problem on a live system can be looked at without a private build, and the next release will not deliver them. Nothing depends on them: HTTPD is the only module a started task names, and a server that never routed to the others cannot notice their absence. Step 3b told the reader to pre-allocate and then showed only the upload. It now shows the allocation too, with `zowe files create ps` and sizes derived from the artifacts (the load XMIT is ~650 KB, the sample library under 20 KB). The flag spellings were checked against the installed zowe rather than assumed.
static/index.html was in the repository and in no release artifact, so the
package installed a server whose DOCROOT had nothing behind it.
A UFS disk cannot travel the way the rest of the package does, and not for
lack of tooling. SMP has no element type for a DSORG=PS/RECFM=U image and
must never touch site content; TSO RECEIVE allocates its own target and
refuses to merge into an existing dataset, which makes it a
first-install-only transport for something an operator replaces. So the
image ships as a plain file in the archive and goes up over IND$FILE, which
needs nothing on the target beyond a 3270 session.
make webroot builds dist/httpd-webroot.img from static/ with a pinned
ufsd-utils (fetched into .mbt/tools/ when absent, because
release CI runs the shared workflow and cannot install
anything of its own). package and dist depend on it, and
[distribution] extra puts it in the zip/tar.gz.
samplib/httpwebr allocates HTTPD.@vrm@.WEBROOT.UFS -- SPACE=(4096,256),
RECFM=U BLKSIZE=4096, primary extent only.
installation.md 9 the operator's side: allocate, upload, mount, verify,
and unmount before replacing.
Measured rather than assumed:
Blocking RECFM=U buffers to BLKSIZE through __fputc (libc370
@@fputc.c:24), so BLKSIZE(4096) writes whole 4096-byte
blocks; the 1 MB image is 256 of them with no remainder.
Options binary is IND$FILE's default and RECFM then defaults to U
(ind_file370 indparse.c:249).
Pre-allocate IND$FILE allocates an existing dataset DISP=SHR and takes
its DCB (indmain.c:190, :273), so the documented upload
needs no options and does not depend on which IND$FILE
build a system carries.
The trap that same DISP=SHR rewrites a dataset UFSD has mounted, so
UNMOUNT is a numbered step rather than a footnote.
Detection a BLKSIZE mismatch surfaces at MOUNT as UFSD062E
(ufsd ufsd#sbl.c:69), not as a corrupt page.
Encoding nothing to configure: http_send_file() translates UFS files
with the hard-coded IBM-1047 table independent of CODEPAGE=
(httpfile.c:70), which is what ufsd-utils writes. The
round trip was measured byte for byte over the whole file
and over all 256 values; only 0x85 and 0xF7 do not survive.
The version in the dataset name settles what smp-todo called U6: a shipped
image cannot land on a webroot the site built for itself.
Fixes #252
The ranking said nothing open blocks 4.0.0, which was true and unhelpful: the release work is in smp-todo.md (the FMID check and the dry run), not in the tracker at all. Say so, and record that O3 closed with #252.
…ne changes The prerequisite list was files only. Adding one is covered -- make expands the wildcard when it parses the Makefile, so the new file is in the list and is newer than the image -- but deleting one is not: the image stays newer than everything still there, and the archive would go on shipping the file that was removed. A directory's mtime moves when an entry is added or removed, so the directories join the file list. Measured both ways with the image backdated so the comparison is not decided by whole-second timestamps: adding rebuilds, deleting rebuilds and drops the file from the image, an unchanged tree does nothing. Also two notes rather than behaviour: why the webroot dataset is the one name with a fourth qualifier, and that the emulator types IND$FILE into whatever field the cursor is on -- it does not find a command line for you.
dist/ is not a scratch directory: the release workflow publishes every file in it as a GitHub Release asset (`gh release create ... dist/*`). An image built there therefore shipped twice -- once inside the archive, where it belongs beside the README and the jobs, and once on its own as `httpd-webroot.img`, a name with no version in it sitting between httpd-<version>-load.xmit and httpd-<version>-dist.zip. Nothing else moves: `extra` takes the basename, so the file inside the archive is called what it was called before, and the installation guide is unchanged.
Ship the document root as a UFS disk image, uploaded with IND$FILE
LIST CDS SYSMOD(THTP400) and LIST ACDS SYSMOD(THTP400), run 2026-08-24 on mvsdev (MVS/CE, JOB01948) and drnmig3a (TK5, JOB00028), both HMASMP LVL 04.48. Each of the four answers RC 04 with "THE FOLLOWING SELECTED ENTRIES WERE NOT FOUND". The FMID is now verified the way TUFS120 and TFTP100 were before their tags -- the earlier SMPPTS scan proved only that nothing had been *received* under the id, while the CDS and ACDS store hashed member names and can only be read with LIST. Two things measured on the way, both recorded in smp-todo.md because they cost time to rediscover: The SMPAPP procstep is HMASMP on both stands and the procedure brings no SMPCNTL of its own, so both //HMASMP.SMPCNTL DD * and the unqualified //SMPCNTL DD * work -- the procedure has a single step. The documented job was right as written. The criterion is the report, not the return code. RC 04 ends every LIST, including one that read no control statements at all; what says the id is free is the SELECT SUMMARY REPORT in SMPOUT. A caller that checks only the job RC cannot tell an empty list from a hit. What is left before the tag is O2, the dry run under TTST400.
…ack off
Run on mvsdev from the unpacked httpd-4.0.0-dev-dist.zip -- the artifact an
operator downloads -- with fmid temporarily on TTST400 (not committed;
project.toml says THTP400 again).
alloc JOB01972 CC 0000, LINKLIB + AHTTPLOD created
inst JOB01973 all eight steps CC 0000: DELOLD, RECV1, RECV2, RECV,
APPLYCHK, APPLY, ACCEPT, CLEANUP
list JOB01974 RC 00 in both zones: JCLIN=YES, REC/APP/ACC stamped,
MOD = HTTPD HTTPDM HTTPDMTT HTTPDSRV ABEND0C1
uclin JOB01975 CC 0000, inventory cut and datasets scratched
list JOB01976 TTST400 gone from the CDS and the ACDS
list JOB01977 THTP400 still free -- the rehearsal did not touch it
Beyond the return codes it settles four things that were derivation until
now: the @vrm@/@linklib@ substitution arrives correctly on the target (the
installed HTTPWEBR names HTTPD.V4R0M0D.WEBROOT.UFS, the procedure's STEPLIB
names the versioned LINKLIB), SMP copies rather than re-binds, the UCLIN
removal path in installation.md 12 works as written, and the -dev VRM keeps
the rehearsal clear of both the tagged release and the site's own datasets.
Nothing was left behind: mvsdev carries HTTPD.LINKLIB and HTTPD.WWWROOT
again, both site content, both untouched.
One trap recorded with it: scratch datasets from a job with IDCAMS, never
through the REST API -- a DELETE there leaves the ENQ standing and the next
DISP=SHR job hangs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything that has to be true before the 4.0.0 tag, except the two SMP checks
that need a live system (see the end). Ten commits, four themes.
1.
make packageproduces an installable product[distribution]inproject.tomlfollows ufsd and ftpd: the load modules gothrough SMP (FMID
THTP400, targetHTTPD.@VRM@.LINKLIB), the sample libraryrides beside it as its own XMIT restored by TSO RECEIVE. Both ddnames fall out
of the dataset names, so
HTTPLOAD/AHTTPLODis a one-shot choice — it goesinto the FMID's JCLIN, and changing it later is a new functional level.
samplib/httpdhadDSN=HTTPD.LINKLIBhardcoded in itsSTEPLIB, so theshipped procedure would have pointed at an unversioned library while SMP
installed into the versioned one. It carries
@LINKLIB@now.docs/installation.mdis new and wired in as the archive's README — thegenerated install job tells the operator to read it, and without the file the
package shipped a dangling instruction.
[toolchain] libc370 = "1.0.3"pins what the release CI links against.HTTPD005Inames the runtime, so an unpinned release build makes that lineunreproducible. Dependency bumped to ufsd 1.2.1,
mbt.lockcommitted.VERSIONsaid4.0.0whileproject.tomlsaid4.0.0-dev.mvsrelease.pysearches each version file for the current string, so
make release VERSION=4.0.0would have failed on the mismatch. Both say4.0.0-devnow, and the tag is what moves them.2. The shipped configuration is gated (bf60f1b)
HTTPPRM0left the/zosmf/*catch-all with noAUTH=, which is public —the whole mvsMF API, every dataset and job endpoint, to anyone who can reach the
port. The live configuration on the reference system has carried
AUTH=TOKENall along; the pattern we ship did not. The commented-out display module lines
had the same shape: uncommenting
# MOD=HTTPDM /.dmproduced a publicarbitrary-storage reader. They carry
AUTH=BASICnow, so uncommenting a linedoes what it looks like it does.
The guide also says what the four non-server modules are for and that the next
release will not deliver them.
3. The document root ships (#252, PR #253)
static/index.htmlwas in the repository and in no release artifact: thepackage installed a server whose
DOCROOThad nothing behind it.It travels as a UFS370 disk image in the archive and goes up over
IND$FILE — SMP has no element type for a
DSORG=PS/RECFM=Uimage and mustnever touch site content, and TSO RECEIVE allocates its own target and refuses
to merge into an existing dataset, which makes it a first-install-only transport
for something an operator replaces.
make webrootbuilds it (pinnedufsd-utils, fetched when absent),samplib/httpwebrallocatesHTTPD.@VRM@.WEBROOT.UFS, anddocs/installation.md§9 is the operator's path. The measurements behind everyclaim — blocking, the DISP=SHR trap, the codepage question that turned out not
to be one — are in #252 and PR #253.
The welcome page itself was reworked over two commits before that.
4.
smp-todo.mdsays what is left, and only thatRewritten against the generated distribution: what mbt now does is a table of
closed items, not a plan.
TODO.mdgained the line it was missing — that the4.0.0 blockers are not in the tracker at all.
What still has to happen before the tag
Both need MVS and neither is a code change:
LIST CDS/ACDS SYSMOD(THTP400)on both stands (mvsdev = MVS/CE,drnmig3a = TK5). Only the SMPPTS has been checked, and it shows only what was
received; the CDS and ACDS store hashed member names, so the
LISTjob isthe only way to know the FMID is free.
TTST400. A run underTHTP400that half-applies occupies exactly the id the tagged release needs.Also unexercised until a real release: the Linux
ufsd-utilsfetch in therelease workflow (PR CI runs
make deps/test/lib, nevermake package).make prereleasewalks the same path.