Add azd ai dataset extension for Foundry datasets

[Mohamed Hessien (m7md7sien)]

Adds azd ai dataset ΓÇö a standalone extension for registering and versioning Foundry datasets.

Why its own extension

A dataset is a general Foundry asset. Fine-tuning and other scenarios need one as much as evaluation does, so the CRUD group does not belong inside the evaluation extension. This extension has its own Go module, lint workflow and release pipeline, and no dependency on azure.ai.evaluations.

The line drawn with the evaluation extension: CRUD moved, generation stayed. generate writes the datasets: entry into the evaluation configuration, so moving it would strand a config write across an extension boundary ΓÇö a generated dataset registered with the service and absent from configuration, which azd up would not reconcile. create/update/list/show/delete/versions need no configuration at all, which is why they could move.

Surface

azd ai dataset create <name>      register a dataset, publishing version 1
azd ai dataset update <name>      publish a new version
azd ai dataset list               list the project's datasets
azd ai dataset show <name>        show a version
azd ai dataset delete <name>      delete a version
azd ai dataset versions list <name>

Notable

• Every user-facing string lives in one file (internal/messages/messages.go) so the whole voice of the CLI can be reviewed in one sitting. 69 messages.
• TYPE, not FORMAT. The API accepts format on upload and never returns it, so the previous column was blank for every dataset ever listed.
• Identity fixed. Carved from the evaluation extension, it had inherited its User-Agent (azd-ext-azure-ai-evaluations) and debug log name (azd-ai-eval-*.log), so a dataset request could not be told from an eval one in a service log or a local one.
• Auth errors say what to do. A credential that cannot mint a token fails before any 401 and surfaced as a bare SDK error. It now suggests retrying first and logging in second ΓÇö observed failing transiently against a login that was valid.
• A name the service will reject is refused locally. A dataset name with a space came back as a 400 wrapped in four levels of nested JSON; it is now checked against the accepted character set before the request.
• A UTF-8 BOM no longer corrupts an upload. Windows editors and PowerShell write one by default, and it was being uploaded into the first row's first key rather than skipped.
• Nil-guarded the endpoint resolution cascade and closed the azd client on the error paths that abandoned it.

Verified

go build, go vet -tags live,hero, go test ./... and gofmt clean. Exercised live against a Foundry project: create ΓåÆ 1.0, update ΓåÆ 2.0, versions list, show, list, and published to a local registry for a bug bash through real azd.

Not included, deliberately

No registry.json entry and no microsoft.foundry bundle dependency. Both need published artifacts; listing an unpublished extension in the bundle breaks azd extension install microsoft.foundry for everyone. Those land after the first release.

Open question for reviewers

azd ai dataset versions list <unknown-name> errors and exits 1 here, while the same command in the evaluation extension (#9500) lists nothing and exits 0. I tried making the eval side match this one and it broke two tests whose names are the argument against it ΓÇö "an unknown name lists nothing rather than failing" and "deleting an unregistered dataset is idempotent, not an error", the second because delete idempotence is verified by listing what is left.

A list is a filter rather than a lookup, so empty-and-successful is defensible and -o json callers range over the array instead of branching on an error. By that reasoning this extension's versions list is the one that should move. dataset show on an unknown name is not in question: that one is a lookup and should fail, as it does.

Draft ΓÇö not requesting reviewers yet.

Fixes #9550

[azure-pipelines]

Azure Pipelines:
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[azure-pipelines]

Azure Pipelines:
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds a standalone Foundry dataset extension with CRUD/versioning commands, API clients, tests, documentation, and release automation.

Changes:

• Adds dataset creation, updates, listing, inspection, deletion, and version listing.
• Adds endpoint resolution, validation, structured errors, and JSON/table output.
• Adds unit/live tests plus build, lint, and release infrastructure.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
eng/pipelines/release-ext-azure-ai-dataset.yml	Adds extension release pipeline.
cli/azd/extensions/azure.ai.dataset/version.txt	Sets initial beta version.
cli/azd/extensions/azure.ai.dataset/tests/cli/harness_test.go	Adds live CLI harness.
cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go	Tests dataset CLI workflows.
cli/azd/extensions/azure.ai.dataset/README.md	Documents installation and commands.
cli/azd/extensions/azure.ai.dataset/main.go	Adds extension entry point.
cli/azd/extensions/azure.ai.dataset/internal/version/version.go	Defines build metadata.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/version_test.go	Tests version handling.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/uri_test.go	Tests URI resolution.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/upload_version_test.go	Tests conflict-aware uploads.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go	Implements dataset and storage operations.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations_wire_test.go	Tests API request contracts.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go	Defines API models and file handling.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go	Implements listing and version ordering.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_wire_test.go	Tests download HTTP behavior.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_test.go	Tests blob selection logic.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/bom_test.go	Tests BOM and file handling.
cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go	Centralizes user-facing messages.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/validator.go	Validates Foundry endpoints.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/types.go	Defines endpoint-resolution types.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/store.go	Reads shared project context.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go	Implements endpoint precedence.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver_test.go	Tests endpoint resolution.
cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go	Adds structured error helpers.
cli/azd/extensions/azure.ai.dataset/internal/exterrors/codes.go	Defines structured error codes.
cli/azd/extensions/azure.ai.dataset/internal/cmd/table_test.go	Tests table rendering.
cli/azd/extensions/azure.ai.dataset/internal/cmd/surface_test.go	Tests command surface and flags.
cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go	Builds the command tree.
cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go	Implements JSON and text output.
cli/azd/extensions/azure.ai.dataset/internal/cmd/output_test.go	Tests output formatting.
cli/azd/extensions/azure.ai.dataset/internal/cmd/names.go	Validates dataset names.
cli/azd/extensions/azure.ai.dataset/internal/cmd/names_test.go	Tests naming constraints.
cli/azd/extensions/azure.ai.dataset/internal/cmd/helpers_test.go	Tests command helpers.
cli/azd/extensions/azure.ai.dataset/internal/cmd/debug.go	Configures debug logging.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go	Implements dataset commands.
cli/azd/extensions/azure.ai.dataset/internal/cmd/context.go	Creates command service context.
cli/azd/extensions/azure.ai.dataset/internal/cmd/artifacts.go	Defines persistence helpers.
cli/azd/extensions/azure.ai.dataset/internal/cmd/apiversions.go	Defines service API versions.
cli/azd/extensions/azure.ai.dataset/go.mod	Defines the Go module.
cli/azd/extensions/azure.ai.dataset/extension.yaml	Declares extension metadata.
cli/azd/extensions/azure.ai.dataset/cspell.yaml	Configures spell checking.
cli/azd/extensions/azure.ai.dataset/ci-test.ps1	Adds CI test execution.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1	Adds CI binary builds.
cli/azd/extensions/azure.ai.dataset/CHANGELOG.md	Records the initial release.
cli/azd/extensions/azure.ai.dataset/build.sh	Adds Unix cross-platform builds.
cli/azd/extensions/azure.ai.dataset/build.ps1	Adds Windows cross-platform builds.
cli/azd/extensions/azure.ai.dataset/.golangci.yaml	Configures Go linting.
cli/azd/extensions/azure.ai.dataset/.gitignore	Ignores generated artifacts.
.github/workflows/lint-ext-azure-ai-dataset.yml	Adds GitHub lint workflow.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 58 out of 59 changed files in this pull request and generated no new comments.

Suppressed comments (13)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:89

• [azd-code-reviewer] The point-read probe also treats every error as “not found.” If authentication expires or the service fails after an empty listing, update reports that the asset is absent. Ignore only 404 responses and surface other probe failures.

				for _, v := range firstDatasetVersions {
					if _, err := ec.datasetClient.GetDataset(
						ctx, name, v, ProjectEndpointAPIVersion,
					); err == nil {
						exists = true

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

• [azd-code-reviewer] These linker targets still use the evaluation module path. The release pipeline invokes this script, so the assignments miss azureaidataset/internal/version.Version and every published binary sends azd-ext-azure-ai-dataset/dev instead of its release version. Stamp the dataset module path here.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

• [azd-code-reviewer] The local cross-platform build also stamps the copied evaluation module path, so these binaries retain the dev user-agent version. Point VERSION_PATH at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

• [azd-code-reviewer] This PowerShell build stamps azureaieval, not the dataset module, leaving version.Version at dev. Use the module path declared by this extension.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:80

• [azd-code-reviewer] A listing failure is currently converted into exists == false. In particular, update turns a 403, timeout, or service failure into the misleading “dataset does not exist” error instead of preserving the real failure. Only a 404/empty result should enter the consistency probe.

This issue also appears on line 85 of the same file.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

• [azd-code-reviewer] This contradicts the new idempotent-delete contract and its live test: whenever the service returns 404 rather than 204 for an absent version, cleanup fails. Treat not-found as successful deletion and only wrap other failures.

			if err := ec.datasetClient.DeleteDatasetVersion(
				ctx, name, version, ProjectEndpointAPIVersion,
			); err != nil {
				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)

cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go:52

• [azd-code-reviewer] None of these runnable commands registers SDK flag options for inherited --output. As a result, arbitrary values such as -o yaml succeed and silently render a table, while generated metadata cannot advertise the supported values. Register json/table with a table default on each runnable command, following extensions/azure.ai.skills/internal/cmd/skill_list.go:51.

	rootCmd.AddCommand(
		newDatasetCreateCommand(),
		newDatasetUpdateCommand(),
		newDatasetListCommand(),
		newDatasetShowCommand(),
		newDatasetDeleteCommand(),
		newDatasetVersionsCommand(),
	)

cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go:358

• [azd-code-reviewer] This helper is also called when --no-prompt was not supplied, so dataset create <name> and dataset delete <name> falsely tell users they are running non-interactively. Since these commands do not prompt in either mode, report only that the flag is required.
  cli/azd/extensions/azure.ai.dataset/README.md:31
• [azd-code-reviewer] The companion extension exposes azd ai eval generate, not azd ai eval dataset generate, so this onboarding command fails with “unknown command.” The companion PR also names azure.eval.yaml; avoid pinning the stale evals/eval.yaml path here.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that
extension's file. Splitting the two would leave a generated dataset registered
with the service but absent from the configuration, so `azd up` would not
reconcile it and no eval could name it.

cli/azd/extensions/azure.ai.dataset/README.md:41

• [azd-code-reviewer] This documented cascade omits two implemented sources and therefore gives the wrong precedence. Global project context is checked before host variables, and AZURE_AI_PROJECT_ENDPOINT is a fallback at both environment levels.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

• [azd-code-reviewer] Debug mode creates azd-ai-dataset-<date>.log, but this copied pattern ignores eval logs instead. Dataset debug logs will remain untracked files in the extension directory.

# Debug log written when --debug or AZD_EXT_DEBUG is set.
azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

• [azd-code-reviewer] Cleanup silently returns when listing or JSON decoding fails, and it also ignores each delete result. A transient cleanup failure therefore leaves live Foundry datasets behind while the suite still passes. Assert every cleanup operation so leaked state is reported.
  cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:17
• [azd-code-reviewer] No CLI command calls DownloadDatasetContent; only internal tests do. This release note therefore advertises a content-read capability users cannot access. Remove the bullet or add the corresponding public command before release.

- Reads dataset content back, whether the service hands out a blob URI or the
  container holding it.

[Copilot]

Pull request overview

Copilot reviewed 58 out of 59 changed files in this pull request and generated no new comments.

Suppressed comments (15)

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

• These linker targets still use the evaluation module path, so the release pipeline's -X flags do not stamp this extension's Version, Commit, or BuildDate. In particular, the dataset User-Agent continues to report dev, defeating the identity/versioning change. Target azureaidataset/internal/version instead.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

• The local PowerShell build also points its linker stamps at the evaluation module, so locally packaged dataset binaries retain the default dev/none/unknown values. Use this module's import path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

• The shell build stamps azureaieval, which is not this module, so its generated binaries do not receive the dataset version metadata. Use the dataset module path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:28

• Both references are stale relative to the companion evaluation extension: its surface is azd ai eval generate, and its canonical configuration is azure.eval.yaml, not an eval dataset generate command writing evals/eval.yaml. As written, the documented command does not exist.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

• This advertised resolution order omits two implemented sources: the shared project context is checked before host variables, and AZURE_AI_PROJECT_ENDPOINT is a fallback at both environment levels. Documenting only three levels makes endpoint selection surprising when the omitted higher-precedence value wins.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

• Debug logging now writes azd-ai-dataset-*.log, but this copied evaluation pattern no longer matches it. Running with --debug from the extension directory leaves generated logs unignored.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:80

• A non-404 listing failure is discarded here. For example, a 403 or timeout makes update report that the dataset does not exist, while create proceeds toward an upload without establishing that the name is free. Only not-found/empty results should enter the eventual-consistency probe; propagate other listing failures.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:299

• This treats the eventually consistent version listing as authoritative even though the upload path explicitly handles it lagging after a publish. Immediately after create, show can incorrectly return “no dataset”; immediately after update, it can show the previous version as latest. Resolve latest with point reads past the listed version (bounded, as the upload conflict walk is) before rendering or declaring the dataset absent.

				list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
				if err != nil {
					return messages.ResolvingLatestDatasetVersion(name, err)
				}
				// The service answers an unknown name with an empty list rather

cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go:29

• No leaf command registers azdext.RegisterFlagOptions, so unsupported values such as -o yaml are accepted and silently fall through to human output. Register the supported json/table values (and the intended default) on every runnable command so help, metadata, completion, and parse-time validation agree with the renderer.

	v, err := cmd.Flags().GetString("output")
	if err != nil {
		return ""
	}
	return strings.ToLower(v)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go:44

• The service compatibility fields leak into -o json, because commands encode Dataset directly. The same logical URI is therefore emitted as dataUri or data_uri depending on which service spelling arrived, giving scripts an unstable schema. Keep both fields for decoding, but render a normalized output DTO with one canonical field name.
  cli/azd/extensions/azure.ai.dataset/README.md:6
• This first-run command cannot work in this PR: the same README states that the registry entry is deliberately absent, so azd extension install azure.ai.dataset cannot resolve. Replace this with complete local pack/publish instructions until the registry entry lands, or defer the install snippet with that release change.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

• The documented delete invocation omits the required --version flag, so copying it always fails with “--version is required.” Include the required argument in the command table.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

• Cleanup silently returns when listing/decoding fails and ignores every delete result, so failed live runs can leave datasets behind while still reporting successful cleanup. Make cleanup failures visible and require each delete to succeed.
  cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go:139
• These environment-variable reads are not fully reflected in the repository's environment-variable reference: FOUNDRY_PROJECT_ENDPOINT currently names only azure.ai.agents, and AZURE_AI_PROJECT_ENDPOINT has no entry. Update cli/azd/docs/environment-variables.md so both new extension inputs and their precedence are documented.
  cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go:21
• The PR description says every user-facing string lives in internal/messages/messages.go, but this root help text and the command/flag help strings in dataset.go are defined inline. Either centralize those strings as claimed or narrow the PR description; the current “69 messages in one file” review guarantee is not true.

		Short: fmt.Sprintf(
			"Register and version Foundry datasets from your terminal. %s",
			color.YellowString("(Beta)"),

[Copilot]

Pull request overview

Copilot reviewed 59 out of 60 changed files in this pull request and generated no new comments.

Suppressed comments (10)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

• The API client explicitly treats a 404 from this endpoint as an unknown/versionless dataset, but this command returns that 404 before reaching the advertised empty-list behavior. Convert IsNotFound(err) to an empty DatasetList; otherwise versions list <unknown> still exits nonzero.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

• These linker targets still use the evaluation module path. This module is azureaidataset, so the release build does not stamp its Version, Commit, or BuildDate variables and the dataset User-Agent remains .../dev.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

• This copied evaluation module path means local PowerShell builds do not stamp azureaidataset/internal/version; the binary keeps the development version in its User-Agent. Point VERSION_PATH at this module.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

• This copied evaluation module path means shell builds do not stamp azureaidataset/internal/version; the binary keeps the development version in its User-Agent. Point VERSION_PATH at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

• Debug logging creates azd-ai-dataset-YYYY-MM-DD.log, but this pattern only ignores the old evaluation filename. Dataset debug logs will remain unignored and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

• The live harness checks AZURE_AI_DATASET_E2E_LIVE, so this setup instruction names an environment variable that cannot enable the suite.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/README.md:28

• Both references are stale: the evaluation extension exposes azd ai eval generate, not azd ai eval dataset generate, and its configuration is azure.eval.yaml. As written, the documented command fails and points readers at the obsolete filename.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

• This omits two implemented resolution levels and the AZURE_AI_PROJECT_ENDPOINT fallback. Users relying on shared project context or the legacy key cannot infer why a value wins; document the same four-level order implemented by projectctx.Resolve.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

• Listing failures are currently indistinguishable from an absent dataset. A 403 or transport error makes exists false, and both point probes discard their errors too, so update reports “does not exist” while create may proceed despite being unable to verify existence. Propagate non-404 errors and only treat empty/404 responses as absence.

This issue also appears on line 229 of the same file.

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:17

• No command in this extension calls DownloadDatasetContent, so the first-release changelog promises a read-back capability that users cannot invoke. Remove this feature bullet or add the corresponding public command before release.

- Reads dataset content back, whether the service hands out a blob URI or the
  container holding it.

[Mohamed Hessien (m7md7sien)]

Copilot review threads — disposition

Working through the open Copilot threads. All of them predate the current head, so several were already addressed; the rest are fixed in the commits below. Grouping them rather than replying 31 times.

Fixed earlier in this PR

• SAS credential written to logs and error text (~10 threads). url.URL.Redacted() masks a userinfo password and leaves the query, so sig survived. Every log site now goes through logSafeURL, which drops the query and fragment, and every SAS-backed Do wraps failures with redactURLError, because http.Client.Do embeds the full URL in a *url.Error. Tests pin the premise as well as the fix — they assert that Redacted() alone still leaks, so the tests fail if someone reverts to it.
• latestRegisteredVersion treating every listing failure as "no versions". A 403 or a timeout restarted an existing dataset at 1.0. It now returns the error and only a 404 means versionless.
• Legacy errors.As. Both IsVersionConflict and IsNotFound use errors.AsType[*azcore.ResponseError].
• Point-read probe looking for 1. It derives from NextVersion("") so it looks for the 1.0 a create actually writes, and keeps 1 for datasets registered by a generation job, the SDK or the portal.

Fixed now

• LatestVersion fallback unreachable (2 threads). bestOrder started below -1, so the first unorderable version became best and the documented fallback to the last entry never ran. Correct here — and the sibling copy in the evaluations extension still had it, which I have also fixed.
• Build scripts stamping the wrong module. build.ps1, build.sh and ci-build.ps1 named azureaieval/internal/version inside a module called azureaidataset. The linker discards -X for a symbol that does not exist, so this failed silently and every release binary reported dev in its User-Agent — nothing could tell which build a caller was running. A test now compares the linker path in all three scripts against the module in go.mod, since a silently discarded flag is exactly what nothing else would catch.
• .gitignore ignoring azd-ai-eval-*.log while debug.go writes azd-ai-dataset-*.log, so debug logs were never ignored.
• ci-test.ps1 documenting AZURE_AI_EVAL_E2E_LIVE while the harness reads AZURE_AI_DATASET_E2E_LIVE, so anyone following it would set the wrong variable and watch the live suite skip while reporting success.
• --version offered on create. The shared builder registered it on both verbs, but the upload derives the next version from it, so create --version 4.0 published 5.0. Now registered on update only; verified against --help.
• Version-not-found hint pointed at azd ai dataset list, which lists datasets rather than versions. It now names versions list <name>.
• README naming azd ai eval dataset generate and evals/eval.yaml. The surface is azd ai eval generate and the file is evals/azure.eval.yaml.
• Case-sensitive .jsonl scan in ReadFirstJSONLFile while the command layer used EqualFold. Not reachable from the CLI, which resolves a directory to a single file first, but the two layers disagreed.

Also addressed, by a different route than suggested

• Listing failures becoming "does not exist" on update. Rather than reporting the listing error, the gate now refuses only when the service confirms the name is unknown with a 404. An empty listing cannot distinguish an unknown dataset from one the listing has not caught up with, so gating on it meant create then update was refused for a dataset that plainly existed — and sent to create, which fails in turn once the listing catches up. The two errors pointed at each other.

Not taking

• Deterministic blob name <name>.jsonl. Raised as an overwrite risk under concurrent publishes. The alternative — a unique name per publish — leaves several .jsonl beside each other, and the container-listing fallback in DownloadDatasetContent picks the first by name, so a download could return rows no version points at. That is the wider harm. The tradeoff is documented at the call site and holds until the service allocates the version rather than the client guessing it from a lagging listing.

Resolving these threads now. Anything I have mischaracterised, please reopen.

[Copilot]

Pull request overview

Copilot reviewed 60 out of 61 changed files in this pull request and generated no new comments.

Suppressed comments (9)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:370

• [azd-code-reviewer] This 404 branch contradicts TestCLIDeleteIsIdempotent: a missing version still exits nonzero. Treat not-found as a successful no-op and continue to the normal JSON/text result; only other delete failures should be returned.

				return err
			}
			defer ec.Close()

			if err := ec.datasetClient.DeleteDatasetVersion(

cli/azd/extensions/azure.ai.dataset/README.md:28

• [azd-code-reviewer] The referenced generation command does not exist in the companion extension; its surface is azd ai eval generate, and its configuration is azure.eval.yaml. As written, users are sent to an unknown command and the wrong configuration name.

Generation is `azd ai eval generate`, in `azure.ai.evaluations`, and stays
there: it writes the `datasets:` entry in `evals/azure.eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

• [azd-code-reviewer] This list omits two sources the implementation checks before the host environment: the azure.ai.agents global project context and the AZURE_AI_PROJECT_ENDPOINT fallback. Document the actual precedence so users can diagnose why a particular endpoint was selected.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:237

• [azd-code-reviewer] An unknown dataset can return 404 here, so this exits before renderDatasets can produce the documented empty-success result. latestRegisteredVersion already treats this response as versionless. Translate 404 to an empty list so the new CLI tests and implementation agree.

			ec, err := newDatasetContext(ctx, endpointFlg)
			if err != nil {
				return err

cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/validator.go:72

• [azd-code-reviewer] Every caller discards pathWarning, so an endpoint missing /api/projects/<project> is silently accepted and requests are sent to the wrong base path. Either make this path check a validation error or propagate the warning through Resolve and render it before the request.
  cli/azd/extensions/azure.ai.dataset/README.md:6
• [azd-code-reviewer] This onboarding command cannot currently resolve the extension because the registry entry is deliberately absent, as lines 54–57 acknowledge. Replace it with complete local azd x pack/azd x publish instructions until the registry entry lands, otherwise the first documented step always fails.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

• [azd-code-reviewer] The documented delete invocation omits the mandatory --version flag, so copying it fails with “--version is required.” Include the required flag in the command table.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

• [azd-code-reviewer] Cleanup silently returns when listing or JSON decoding fails and ignores every delete result, so failed cleanup leaves datasets in the shared live project while the suite still reports success. Assert each cleanup operation and decode through the existing helpers so leaked service state is visible.
  cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go:143
• [azd-code-reviewer] This adds reads of AZURE_AI_PROJECT_ENDPOINT, but that variable is absent from the repository’s environment-variable reference, and the existing FOUNDRY_PROJECT_ENDPOINT entry mentions only the agents extension. Update cli/azd/docs/environment-variables.md to document both variables and this extension’s precedence.

[Copilot]

Pull request overview

Copilot reviewed 60 out of 61 changed files in this pull request and generated no new comments.

Suppressed comments (9)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:374

• This reverses the documented and tested idempotence guarantee: when the service returns 404 for an already-absent version, delete exits nonzero. Treat not-found as an already-completed delete and continue to the normal human/JSON success response. [azd-code-reviewer]

				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)

cli/azd/extensions/azure.ai.dataset/README.md:41

• The documented cascade omits both the persisted azd ai project set value and the AZURE_AI_PROJECT_ENDPOINT fallback, so users may troubleshoot or override the wrong source. Document all four implemented levels in their actual precedence order. [azd-code-reviewer]

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:243

• The advertised empty-and-success behavior does not handle the API's 404 form for an unknown dataset. TestUploadNextVersionTreatsAnUnknownDatasetAsVersionless explicitly models /versions returning 404, but this command returns that error instead of []. Normalize only not-found here and keep propagating other failures. [azd-code-reviewer]

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/README.md:22

• This usage is incomplete: the implementation requires --version, so copying the documented command always fails before deletion. Include the required flag in the command table. [azd-code-reviewer]

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go:139

• This extension newly reads AZURE_AI_PROJECT_ENDPOINT, but that variable is absent from cli/azd/docs/environment-variables.md, and the existing FOUNDRY_PROJECT_ENDPOINT entry still says it is agents-only. Update the environment-variable reference so this fallback is discoverable and its precedence is accurate. [azd-code-reviewer]
  cli/azd/extensions/azure.ai.dataset/tests/cli/harness_test.go:130
• This test harness cannot actually detect the hang asserted by TestCLIRequiredValuesFailInsteadOfHanging: exec.Command has no deadline, so a regression blocks until the entire go test timeout. Run subprocesses with exec.CommandContext and a short per-command timeout, and fail with the captured output when it expires. [azd-code-reviewer]
  cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:17
• The release note claims a user-visible read-back feature, but no command calls DownloadDatasetContent; only tests do, and the package is internal to this module. Either expose the claimed capability through the CLI or remove it from the release notes. [azd-code-reviewer]

- Reads dataset content back, whether the service hands out a blob URI or the
  container holding it.

cli/azd/extensions/azure.ai.dataset/.golangci.yaml:12

• The repository's Go guidance sets lll to 125 characters (cli/azd/AGENTS.md:97-108), but this module allows 220, so its dedicated CI accepts lines the repository standard rejects. Restore the 125-character limit and wrap the existing overlong source line. [azd-code-reviewer]

      line-length: 220

cli/azd/extensions/azure.ai.dataset/internal/cmd/surface_test.go:245

• Remove the accidental // embedded in this sentence and wrap it to the repository's 125-character limit. [azd-code-reviewer]

// A message pointing at `azd ai eval dataset ...` is almost always the copy// these commands came from rather than a deliberate cross-extension pointer.

[Copilot]

Pull request overview

Copilot reviewed 61 out of 62 changed files in this pull request and generated no new comments.

Suppressed comments (8)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:375

• [azd-code-reviewer] This explicitly turns a not-found delete into failure, while TestCLIDeleteIsIdempotent defines deleting an unregistered version as successful. Treat 404 as an idempotent success so cleanup scripts do not fail when the version is already absent.

				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)
				}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:99

• [azd-code-reviewer] A non-404 listing failure leaves exists=false, the probes ignore their errors, and create is then allowed to continue. If the transient failure clears when UploadNextVersion lists again, create can publish a new version of an existing dataset instead of failing as already-exists. Propagate an inconclusive list error for create.

			if err := checkAssetExistence(
				verb, "dataset", name, exists, dataset_api.IsNotFound(listErr),
			); err != nil {

cli/azd/extensions/azure.ai.dataset/README.md:41

• [azd-code-reviewer] This omits two implemented resolution levels: AZURE_AI_PROJECT_ENDPOINT is a fallback at both environment levels, and persisted azd ai project set context is checked before host variables. Users following this order can diagnose or update the wrong source. Document the actual four-level cascade.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go:16

• [azd-code-reviewer] The inherited --environment/AZD_ENVIRONMENT value is discarded here. Both endpoint resolution and version persistence call Environment().GetCurrent(), which resolves the project's default environment, so -e staging can target the default Foundry project and write EVAL_DATASET_VERSION into the wrong environment. Thread extCtx.Environment through the context/resolver and use the named environment when it is set.

	rootCmd, _ := azdext.NewExtensionRootCommand(azdext.ExtensionCommandOptions{

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:243

• [azd-code-reviewer] A 404 still returns an error here, contradicting the new command contract and the live tests that require an unknown name to produce an empty successful list. Normalize IsNotFound(err) to an empty DatasetList; otherwise versions list <unknown> still exits 1 on services that report absence with 404.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go:34

• [azd-code-reviewer] Unsupported output values are silently treated as human/table output because only json is recognized. For example, -o yaml exits successfully with non-YAML text, which can break automation. Register per-command output options with the extension SDK (for example json and table) so unsupported values fail validation.

	return strings.ToLower(v)
}

// isJSON reports whether the command should emit machine-readable output.
func isJSON(cmd *cobra.Command) bool {
	return outputFormat(cmd) == outputJSON

cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go:357

• [azd-code-reviewer] This message always claims --no-prompt is active, but requireFlag is also called during ordinary interactive invocations such as dataset delete <name>. The resulting error is factually wrong; either prompt when allowed or use a mode-neutral required-flag message.
  cli/azd/extensions/azure.ai.dataset/README.md:22
• [azd-code-reviewer] The documented delete invocation always fails because the implementation requires --version. Include the required flag so the command table is directly runnable.

| `azd ai dataset delete <name>` | Delete a dataset version |

[Copilot]

Pull request overview

Copilot reviewed 61 out of 62 changed files in this pull request and generated no new comments.

Suppressed comments (7)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:376

• A 404 currently becomes DatasetVersionNotFound, so deleting an already-absent version exits non-zero. This directly contradicts the added idempotence test. Treat not-found as successful deletion and only return other service failures. (azd-code-reviewer)

				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)
				}
				return messages.DeletingDatasetVersion(name, version, err)

cli/azd/extensions/azure.ai.dataset/README.md:42

• The documented cascade omits two implemented sources: persisted azd ai project set context and the AZURE_AI_PROJECT_ENDPOINT fallback at both environment levels. Users can therefore get an endpoint they did not expect without the README explaining its precedence. Document all four levels in resolver order. (azd-code-reviewer)

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:249

• An unknown dataset is returned as HTTP 404 by this API, so this branch currently exits with an error even though the new CLI tests require versions list to succeed with an empty array. Normalize a not-found response to an empty DatasetList; preserve other failures. (azd-code-reviewer)

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:309

• The API can return 404 for this version listing, but this path wraps it as a raw “resolving latest version” failure. That contradicts TestCLIUnknownDatasetIsBrief, which expects the same concise DatasetNotFound result used for an empty listing. Translate not-found before wrapping other errors. (azd-code-reviewer)

				list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
				if err != nil {
					return messages.ResolvingLatestDatasetVersion(name, err)

cli/azd/extensions/azure.ai.dataset/README.md:22

• This documented invocation always fails because delete requires --version. Include the required flag so the command table is executable as written. (azd-code-reviewer)

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go:44

• These transport aliases are serialized directly by every -o json path. A response using data_uri therefore emits data_uri, while a response using dataUri emits dataUri, so scripts receive different schemas for the same field. Map Dataset to a stable CLI output type that always exposes one resolved dataUri field. (azd-code-reviewer)
  cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51
• Cleanup silently returns when listing or JSON decoding fails and ignores every delete exit code, so the live suite can pass while leaving datasets behind. Make cleanup failures fail the test instead of swallowing them. (azd-code-reviewer)