diff --git a/docs/.pages b/docs/.pages index 612e668d5..ef7c60e3f 100644 --- a/docs/.pages +++ b/docs/.pages @@ -2,6 +2,7 @@ nav: - Build: build - Explore and Author: explore-and-author - Consume: consume + - Distribution: distribution - Deploy and Configure: deploy-and-configure - Automate: automate - Develop: develop diff --git a/docs/automate/cmemc-command-line-interface/command-reference/admin/view/index.md b/docs/automate/cmemc-command-line-interface/command-reference/admin/view/index.md new file mode 100644 index 000000000..c94b31c5e --- /dev/null +++ b/docs/automate/cmemc-command-line-interface/command-reference/admin/view/index.md @@ -0,0 +1,267 @@ +--- +title: "cmemc: Command Group - admin view" +description: "List and update explore application view configurations." +icon: octicons/cross-reference-24 +tags: + - cmemc +--- + +# admin view Command Group + + + +List and update explore application view configurations. + +This command group manages Explore (DataPlatform) application view configurations. Application view configurations control the behavior of specific Explore view profiles including companion services and other settings. + + +## admin view list + +List explore application view configurations. + +```shell-session title="Usage" +cmemc admin view list [OPTIONS] +``` + + + + +Outputs a list of application view configurations from the Explore component. The default application view (id: 'default') is always listed first, followed by any custom application view configurations. + +Profile IDs can be used as a reference for the other commands of the `admin view` command group. + + + +??? info "Options" + ```text + + --raw Outputs raw JSON. + --id-only Lists only profile IDs. This is useful for piping + the IDs into other commands. + --filter ... Filter application view configurations by one of + the following filter names and a corresponding + value: id, label. + ``` + +## admin view export + +Export application view configurations to a JSON file. + +```shell-session title="Usage" +$ cmemc admin view export [OPTIONS] [PROFILE_IDS]... +``` + + + + +Application view configurations can be exported based on profile IDs, filters, or all at once. The exported JSON can be imported back using the `admin view import` command. + +```shell-session title="Example" +$ cmemc admin view export --all +``` + + +```shell-session title="Example" +$ cmemc admin view export --all --output-file configs.json +``` + + +```shell-session title="Example" +$ cmemc admin view export --filter id my-view +``` + + +```shell-session title="Example" +$ cmemc admin view export my-view +``` + + + + +??? info "Options" + ```text + + -a, --all Export all application view configurations. + --filter ... Filter application view configurations by one + of the following filter names and a + corresponding value: id, label. + --output-file FILE Export to this file. Use '-' for stdout. If + specified, overrides --output-dir and + --filename-template. + --output-dir DIRECTORY The base directory where the export file will + be created. Ignored if --output-file is + specified. [default: .] + -t, --filename-template TEXT Template for the export file name. Possible + placeholders are (Jinja2): {{connection}} + (from the --connection option) and {{date}} + (the current date as YYYY-MM-DD). Ignored if + --output-file is specified. [default: + {{date}}-{{connection}}.view-configs.json] + --replace Replace an existing export file. This is a + dangerous option, so use it with care. + ``` + +## admin view import + +Import application view configurations from a JSON file. + +```shell-session title="Usage" +cmemc admin view import [OPTIONS] INPUT_FILE +``` + + + + +This command imports application view configurations from a JSON file that was created using the `admin view export` command. + +If `--replace` is specified, existing configurations with the same profile ID will be updated. Otherwise, existing configurations will be skipped. + +!!! note + Importing the default application view configuration updates the project-level overrides stored in /api/conf/workspaces/projectDefault. + + +```shell-session title="Example" +cmemc admin view import configs.json +``` + + +```shell-session title="Example" +cmemc admin view import --replace configs.json +``` + + + + +??? info "Options" + ```text + + --replace Replace existing application view configurations. By default, + import will skip configurations that already exist. + --id TEXT Import the configuration under this profile ID instead of the + one stored in the file. + ``` + +## admin view delete + +Delete custom application view configurations. + +```shell-session title="Usage" +cmemc admin view delete [OPTIONS] [PROFILE_IDS]... +``` + + + + +!!! warning + Application view configurations will be deleted without prompting. + + +!!! note + The default application view configuration cannot be deleted. Use the `admin view list` command to list available application view configurations. + + + + +??? info "Options" + ```text + + -a, --all Delete all custom application view configurations. + This is a dangerous option, so use it with care. + --filter ... Filter application view configurations by one of + the following filter names and a corresponding + value: id, label. + ``` + +## admin view create + +Create a new explore application view configuration. + +```shell-session title="Usage" +cmemc admin view create [OPTIONS] PROFILE_ID +``` + + + + +The new profile is created with its ID and label only. Use the `admin view update` command to set configuration values such as enableCompanion or module toggles. + +!!! note + Application view configurations can be listed with the `admin view list` command. + + + + +??? info "Options" + ```text + + --label TEXT Label for the application view configuration. Defaults to the + profile ID. + ``` + +## admin view update + +Update a key in an existing explore application view configuration. + +```shell-session title="Usage" +cmemc admin view update [OPTIONS] PROFILE_ID +``` + + + + +Any configuration key can be updated, including nested module keys. All other fields are preserved. + +```shell-session title="Example" +cmemc admin view update my-profile --key enableCompanion --value true +``` + + +```shell-session title="Example" +cmemc admin view update my-profile --key modules.marketplaceModuleConfiguration.enabled --value false +``` + + + + +??? info "Options" + ```text + + --key TEXT The configuration key to update. Supports nested keys using + 'a.b[i].c' notation, e.g. + modules.marketplaceModuleConfiguration.enabled. [required] + --value TEXT The new value. Parsed as JSON when possible (e.g. true, false, + 1), otherwise used as a plain string. [required] + ``` + +## admin view inspect + +Inspect the configuration of an application view profile. + +```shell-session title="Usage" +cmemc admin view inspect [OPTIONS] PROFILE_ID +``` + + + + +For accessing nested configuration values, use the following notation: exploreGraphLists[4].comments[0] + +!!! note + Some shell environments require quotes around expressions with square brackets. + + +Examples: cmemc admin view inspect my-profile + +cmemc admin view inspect my-profile `--key` enable + +cmemc admin view inspect my-profile `--key` "exploreGraphLists[4].comments[0]" + + + + +??? info "Options" + ```text + + --key TEXT Get a specific key only from the configuration. + --raw Outputs raw JSON. + ``` diff --git a/docs/automate/cmemc-command-line-interface/command-reference/admin/workspace/index.md b/docs/automate/cmemc-command-line-interface/command-reference/admin/workspace/index.md index 0c2bab3f6..4814883cc 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/admin/workspace/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/admin/workspace/index.md @@ -37,6 +37,9 @@ The file name is optional and will be generated with by the template if absent. option, so use it with care. --type TEXT Type of the exported workspace file. [default: xmlZip] + --without-userdata Do not export user-identifying metadata + (creation/modification timestamps and account + names) in the exported archive. -t, --filename-template TEXT Template for the export file name. Possible placeholders are (Jinja2): {{connection}} (from the --connection option) and {{date}} diff --git a/docs/automate/cmemc-command-line-interface/command-reference/graph/insights/index.md b/docs/automate/cmemc-command-line-interface/command-reference/graph/insights/index.md index f228757af..dfaccc9f2 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/graph/insights/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/graph/insights/index.md @@ -121,7 +121,7 @@ After the update, the snapshot is hot-swapped. following filter names and a corresponding value: id, main-graph, status, affected- graph, valid. - -a, --all Delete all snapshots. + -a, --all Update all snapshots. --wait Wait until snapshot creation is done. --polling-interval INTEGER RANGE How many seconds to wait between status diff --git a/docs/automate/cmemc-command-line-interface/command-reference/index.md b/docs/automate/cmemc-command-line-interface/command-reference/index.md index 456fdfdef..939627609 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/index.md @@ -48,6 +48,13 @@ tags: | [admin user](admin/user/index.md) | [delete](admin/user/index.md#admin-user-delete) | Delete user accounts. | | [admin user](admin/user/index.md) | [password](admin/user/index.md#admin-user-password) | Change the password of a user account. | | [admin user](admin/user/index.md) | [open](admin/user/index.md#admin-user-open) | Open user in the browser. | +| [admin view](admin/view/index.md) | [list](admin/view/index.md#admin-view-list) | List explore application view configurations. | +| [admin view](admin/view/index.md) | [export](admin/view/index.md#admin-view-export) | Export application view configurations to a JSON file. | +| [admin view](admin/view/index.md) | [import](admin/view/index.md#admin-view-import) | Import application view configurations from a JSON file. | +| [admin view](admin/view/index.md) | [delete](admin/view/index.md#admin-view-delete) | Delete custom application view configurations. | +| [admin view](admin/view/index.md) | [create](admin/view/index.md#admin-view-create) | Create a new explore application view configuration. | +| [admin view](admin/view/index.md) | [update](admin/view/index.md#admin-view-update) | Update a key in an existing explore application view configuration. | +| [admin view](admin/view/index.md) | [inspect](admin/view/index.md#admin-view-inspect) | Inspect the configuration of an application view profile. | | [admin workspace](admin/workspace/index.md) | [export](admin/workspace/index.md#admin-workspace-export) | Export the complete workspace (all projects) to a ZIP file. | | [admin workspace](admin/workspace/index.md) | [import](admin/workspace/index.md#admin-workspace-import) | Import the workspace from a file. | | [admin workspace](admin/workspace/index.md) | [reload](admin/workspace/index.md#admin-workspace-reload) | Reload the workspace from the backend. | @@ -90,6 +97,14 @@ tags: | [graph validation](graph/validation/index.md) | [inspect](graph/validation/index.md#graph-validation-inspect) | List and inspect errors found with a validation process. | | [graph validation](graph/validation/index.md) | [cancel](graph/validation/index.md#graph-validation-cancel) | Cancel a running validation process. | | [graph validation](graph/validation/index.md) | [export](graph/validation/index.md#graph-validation-export) | Export a report of finished validations. | +| [package](package/index.md) | [inspect](package/index.md#package-inspect) | Inspect the manifest of a package. | +| [package](package/index.md) | [list](package/index.md#package-list) | List installed packages. | +| [package](package/index.md) | [install](package/index.md#package-install) | Install packages. | +| [package](package/index.md) | [uninstall](package/index.md#package-uninstall) | Uninstall installed packages. | +| [package](package/index.md) | [export](package/index.md#package-export) | Export installed packages to package directories. | +| [package](package/index.md) | [build](package/index.md#package-build) | Build a package archive from a package directory. | +| [package](package/index.md) | [publish](package/index.md#package-publish) | Publish a package archive to the marketplace server. | +| [package](package/index.md) | [search](package/index.md#package-search) | Search for available packages with a given search text. | | [project](project/index.md) | [open](project/index.md#project-open) | Open projects in the browser. | | [project](project/index.md) | [list](project/index.md#project-list) | List available projects. | | [project](project/index.md) | [export](project/index.md#project-export) | Export projects to files. | @@ -97,6 +112,7 @@ tags: | [project](project/index.md) | [delete](project/index.md#project-delete) | Delete projects. | | [project](project/index.md) | [create](project/index.md#project-create) | Create projects. | | [project](project/index.md) | [reload](project/index.md#project-reload) | Reload projects from the workspace provider. | +| [project](project/index.md) | [status](project/index.md#project-status) | Show task loading errors of projects. | | [project file](project/file/index.md) | [list](project/file/index.md#project-file-list) | List available file resources. | | [project file](project/file/index.md) | [delete](project/file/index.md#project-file-delete) | Delete file resources. | | [project file](project/file/index.md) | [download](project/file/index.md#project-file-download) | Download file resources to the local file system. | @@ -120,7 +136,7 @@ tags: | [query](query/index.md) | [delete](query/index.md#query-delete) | Delete queries from a query catalog. | | [vocabulary](vocabulary/index.md) | [open](vocabulary/index.md#vocabulary-open) | Open / explore a vocabulary graph in the browser. | | [vocabulary](vocabulary/index.md) | [list](vocabulary/index.md#vocabulary-list) | Output a list of vocabularies. | -| [vocabulary](vocabulary/index.md) | [install](vocabulary/index.md#vocabulary-install) | Install one or more vocabularies from the catalog. | +| [vocabulary](vocabulary/index.md) | [install](vocabulary/index.md#vocabulary-install) | Install one or more vocabularies from the catalog (deprecated). | | [vocabulary](vocabulary/index.md) | [uninstall](vocabulary/index.md#vocabulary-uninstall) | Uninstall one or more vocabularies. | | [vocabulary](vocabulary/index.md) | [import](vocabulary/index.md#vocabulary-import) | Import a turtle file as a vocabulary. | | [vocabulary cache](vocabulary/cache/index.md) | [update](vocabulary/cache/index.md#vocabulary-cache-update) | Reload / updates the data integration cache for a vocabulary. | diff --git a/docs/automate/cmemc-command-line-interface/command-reference/package/index.md b/docs/automate/cmemc-command-line-interface/command-reference/package/index.md new file mode 100644 index 000000000..09b091d76 --- /dev/null +++ b/docs/automate/cmemc-command-line-interface/command-reference/package/index.md @@ -0,0 +1,229 @@ +--- +title: "cmemc: Command Group - package" +description: "List, (un)install, export, create, or inspect packages." +icon: material/shopping +tags: + - cmemc + - Package +--- + +# package Command Group + + + +List, (un)install, export, create, or inspect packages. + + +## package inspect + +Inspect the manifest of a package. + +```shell-session title="Usage" +cmemc package inspect [OPTIONS] PACKAGE_PATH +``` + + + + + +??? info "Options" + ```text + + --key TEXT Get a specific key only from the manifest. + --raw Outputs raw JSON. + ``` + +## package list + +List installed packages. + +```shell-session title="Usage" +cmemc package list [OPTIONS] +``` + + + + + +??? info "Options" + ```text + + --filter ... Filter installed packages by one of the following + filter names and a corresponding value: type, name, + id. + --id-only Lists only package IDs. This is useful for piping + the IDs into other commands. + --raw Outputs raw JSON. + ``` + +## package install + +Install packages. + +```shell-session title="Usage" +cmemc package install [OPTIONS] [PACKAGE_ID] +``` + + + + +This command installs a package either from the marketplace or from local package archives (.cpa) or package directories. + +If a local package is chosen which has unzipped project directories, the installation will handle the zipping silently. See the `package export` command for more information. + + + +??? info "Options" + ```text + + -i, --input PATH Install a package from a package archive (.cpa) or + directory. + --replace Replace (overwrite) an existing package version or + package content, if present. + --no-cache Disable using cached package versions. + --ignore-lock Ignore and release the package lock file for this + operation. Use this to recover from a stale lock + left by an interrupted run. Dangerous under + concurrent access (it removes the lock other + processes rely on); use with care. + --version TEXT Specific version to install from the marketplace. + Defaults to the latest version. + --marketplace-url TEXT Base URL of the Marketplace - uses environment + variable ECCENCA_MARKETPLACE_URL if available. + [default: https://eccenca.market] + ``` + +## package uninstall + +Uninstall installed packages. + +```shell-session title="Usage" +cmemc package uninstall [OPTIONS] [PACKAGE_ID] +``` + + + + + +??? info "Options" + ```text + + --ignore-lock Ignore and release the package lock file for this + operation. Use this to recover from a stale lock + left by an interrupted run. Dangerous under + concurrent access (it removes the lock other + processes rely on); use with care. + --filter ... Filter installed packages by one of the following + filter names and a corresponding value: type, name, + id. + -a, --all Uninstall all packages. This is a dangerous option, + so use it with care. + ``` + +## package export + +Export installed packages to package directories. + +```shell-session title="Usage" +$ cmemc package export [OPTIONS] [PACKAGE_ID] +``` + + + + + +??? info "Options" + ```text + + --mime_type [text/turtle|text/turtle+pretty] + Choose the MIME type for graphs when + exporting packages. [default: + text/turtle+pretty] + --filter ... Filter installed packages by one of the + following filter names and a corresponding + value: type, name, id. + -a, --all Export all installed packages. + --output-dir DIRECTORY Create package directories in this base + directory. [default: .] + --replace Replace (overwrite) existing files, if + present. + --extract Extract the project files specified in the + manifest and replace the archive with + itsextracted directory. This is useful for + version controlled package directories. + ``` + +## package build + +Build a package archive from a package directory. + +```shell-session title="Usage" +cmemc package build [OPTIONS] PACKAGE_DIRECTORY +``` + + + + +This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory Package Archive (.cpa) with the following naming convention: {package_id}-v{version}.cpa + +If the package contains an extracted project (directory) instead of a ZIP, it is zipped automatically in a temporary copy — the original package directory is never modified. The manifest still need to reference the project ZIP. See the `package export` command for more information. + +Package archives can be published to the marketplace using the `package publish` command. + + + +??? info "Options" + ```text + + --version TEXT Set the package version. + --replace Replace package archive, if present. + --output-dir DIRECTORY Create the package archive in a specific directory. + [default: .] + ``` + +## package publish + +Publish a package archive to the marketplace server. + +```shell-session title="Usage" +cmemc package publish [OPTIONS] PACKAGE_ARCHIVE +``` + + + + + +??? info "Options" + ```text + + --timeout INTEGER Timeout for marketplace requests. + --marketplace-url TEXT Base URL of the Marketplace - uses environment + variable ECCENCA_MARKETPLACE_URL if available. + [default: https://eccenca.market] + --marketplace-account TEXT Marketplace account - uses environment variable + ECCENCA_MARKETPLACE_ACCOUNT if available. + --marketplace-password TEXT Marketplace password - uses environment + variable ECCENCA_MARKETPLACE_PASSWORD if + available. + ``` + +## package search + +Search for available packages with a given search text. + +```shell-session title="Usage" +cmemc package search [OPTIONS] [SEARCH_TERMS]... +``` + + + + + +??? info "Options" + ```text + + --raw Outputs raw JSON. + --marketplace-url TEXT Base URL of the Marketplace - uses environment + variable ECCENCA_MARKETPLACE_URL if available. + [default: https://eccenca.market] + ``` diff --git a/docs/automate/cmemc-command-line-interface/command-reference/project/index.md b/docs/automate/cmemc-command-line-interface/command-reference/project/index.md index 22b732e37..a9fb682fa 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/project/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/project/index.md @@ -117,6 +117,10 @@ $ cmemc config list | parallel -I% cmemc -c % project export --all -t "dump/{{co note that not all export types are extractable. --help-types Lists all possible export types. + --without-userdata Do not export user-identifying metadata + (creation/modification timestamps and account + names) in the exported archives or + directories. ``` ## project import @@ -233,3 +237,29 @@ This command reloads all tasks of a project from the workspace provider. This is -a, --all Reload all projects ``` + +## project status + +Show task loading errors of projects. + +```shell-session title="Usage" +cmemc project status [OPTIONS] [PROJECT_IDS]... +``` + + + + +This command checks the given projects (or all projects with the `--all` option) for task loading errors and outputs them as warnings. + +Use this to find out if your projects have tasks which could not be loaded, e.g. because a needed plugin is not installed. + + + +??? info "Options" + ```text + + -a, --all Check all projects + --exit-1 Exit with code 1 if at least one project has task loading + errors. + --raw Outputs raw JSON of the task loading status. + ``` diff --git a/docs/automate/cmemc-command-line-interface/command-reference/project/variable/index.md b/docs/automate/cmemc-command-line-interface/command-reference/project/variable/index.md index 45c9ed5bf..706adf634 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/project/variable/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/project/variable/index.md @@ -59,7 +59,7 @@ cmemc project variable get [OPTIONS] VARIABLE_ID Use the ``--key`` option to specify which information you want to get. !!! note - Only the `value` key is always available on a project variable. Static value variables have no `template` key, and the `description` key is optional for both types of variables. + Only the `value` key is always available on a project variable. Static value variables have no `template` key, and the `description` key is optional for both types of variables. Use `--raw` to access all fields. diff --git a/docs/automate/cmemc-command-line-interface/command-reference/vocabulary/index.md b/docs/automate/cmemc-command-line-interface/command-reference/vocabulary/index.md index 3dc930188..9bf3966eb 100644 --- a/docs/automate/cmemc-command-line-interface/command-reference/vocabulary/index.md +++ b/docs/automate/cmemc-command-line-interface/command-reference/vocabulary/index.md @@ -59,7 +59,7 @@ Vocabularies are graphs (see `graph` command group) which consists of class and ## vocabulary install -Install one or more vocabularies from the catalog. +Install one or more vocabularies from the catalog (deprecated). ```shell-session title="Usage" cmemc vocabulary install [OPTIONS] [IRIS]... @@ -70,6 +70,10 @@ cmemc vocabulary install [OPTIONS] [IRIS]... Vocabularies are identified by their graph IRI. Installable vocabularies can be listed with the vocabulary list command. +!!! note + This command is deprecated. Vocabularies are now managed via packages; use the `package` command group instead. + + ??? info "Options" @@ -92,6 +96,10 @@ cmemc vocabulary uninstall [OPTIONS] [IRIS]... Vocabularies are identified by their graph IRI. Already installed vocabularies can be listed with the vocabulary list command. +!!! note + This command is deprecated. Vocabularies are now managed via packages; use the `package` command group instead. Vocabularies which are managed by a package can not be uninstalled with this command. + + ??? info "Options" diff --git a/docs/develop/.pages b/docs/develop/.pages index a11a2dfb9..1d1dbc14b 100644 --- a/docs/develop/.pages +++ b/docs/develop/.pages @@ -2,6 +2,7 @@ nav: - Develop: index.md - Accessing Graphs with Java Applications: accessing-graphs-with-java-applications - Python Plugins: python-plugins + - Marketplace Packages: packages - cmempy - Python API: cmempy-python-api - cmemc - Python Scripts: cmemc-scripts - Build (DataIntegration) APIs: dataintegration-apis diff --git a/docs/develop/index.md b/docs/develop/index.md index c31d88f00..20505e37f 100644 --- a/docs/develop/index.md +++ b/docs/develop/index.md @@ -21,6 +21,12 @@ API documentation and programming recipes. For Python developers, we offer a [Plugin SDK](python-plugins/index.md) as well as an API for accessing and manipulating Corporate Memory Instances ([cmem-cmempy](cmempy-python-api/index.md)). +- :material-shopping: Marketplace Packages + + --- + + [Marketplace Packages](packages/index.md) bundle existing content (graphs, Build projects, dependencies, ...) into a single shareable artifact. [Create your own packages](packages/development/index.md) for easy distribution and reuse. + - :material-api: OpenAPI specification --- diff --git a/docs/develop/packages/.pages b/docs/develop/packages/.pages new file mode 100644 index 000000000..1f4c61800 --- /dev/null +++ b/docs/develop/packages/.pages @@ -0,0 +1,4 @@ +nav: + - Marketplace Packages: index.md + - Installation and Management: installation + - Development and Publication: development diff --git a/docs/develop/packages/development/.pages b/docs/develop/packages/development/.pages new file mode 100644 index 000000000..48c0722db --- /dev/null +++ b/docs/develop/packages/development/.pages @@ -0,0 +1,3 @@ +nav: + - Development and Publication: index.md + - Tutorial: tutorial diff --git a/docs/develop/packages/development/index.md b/docs/develop/packages/development/index.md new file mode 100644 index 000000000..ebfd40356 --- /dev/null +++ b/docs/develop/packages/development/index.md @@ -0,0 +1,271 @@ +--- +title: "Marketplace Packages: Development and Publication" +icon: material/code-json +tags: + - Marketplace + - Package +--- +# Development and Publication of Marketplace Packages + +## Introduction + +Marketplace Packages are archives that bundle content, functionality, and configuration from Corporate Memory for sharing and reuse. + +Each package has its own release cycle. +Packages can be installed and uninstalled during runtime. + +In order to support the development and publication of Marketplace Packages, we published a [package-template](https://github.com/eccenca/cmem-package-template). +Please have a look at this template to get started. + +This page gives an overview of the concepts you need to understand in order to develop packages. +If you prefer to learn by doing, follow the [step-by-step tutorial](tutorial/index.md), which builds a package with a graph and a Build project from scratch. + +## Package Structure + +Use the [package-template](https://github.com/eccenca/cmem-package-template) to create the boilerplate for a package repository: + +```shell title="Create a package repository from the template" +copier copy gh:eccenca/cmem-package-template my-package +``` + +The template asks for the following variables: + +`package_type` +: `vocabulary` (default) or `project`, see [Metadata](#metadata). + +`package_id` +: Unique package identifier in lowercase letters, numbers, and hyphens (e.g. `eccenca-supply-chain-vocab`). + +`package_name` +: Human-readable package name (3 - 50 characters). + +`package_description` +: Short description of the package (10 - 150 characters). + +`python_dependencies` +: Comma-separated [Python plugin](../../python-plugins/index.md) dependencies (only asked for `project` packages). + +`vocab_dependencies` +: Comma-separated dependencies on other Marketplace Packages (only asked for `project` packages). + +`github_page` +: Optional URL of the package repository, used as the base for icons and the homepage link. + +The generated repository has two levels: +the top level holds the generic package repository files (changelog, README, license, CI configuration, and a `Taskfile.yaml`), while the nested `{package_id}/` folder is the **package directory** - the actual package content plus its manifest. + +### License + +!!! info "No publication without license" + + Packages without a license declaration cannot be published to a Corporate Memory Marketplace Server. + +Our template will bootstrap your package with an _Apache License 2.0 ([`Apache-2.0`](https://spdx.org/licenses/Apache-2.0.html))_. +See if you need a different license. +You can remove a license entirely; however, a package that does not declare a license cannot be published. + +### Manifest + +The `cpa-manifest.json` in the package directory is the central package definition. +It contains all relevant package metadata and describes the package contents. +It is used to present package details and contents to the `inspect` command, to install, configure and uninstall all parts of a package. + +#### Metadata + +`package_type` +: `project` + : A package that may ship any content, mainly intended to contain Build projects, (instance/data) graphs, SHACL shapes, workspace configuration, query catalogs, etc. + + `vocabulary` + : A package that is supposed to contribute vocabulary / ontology contents, such as `rdf:`, `org:`, `sso:`, etc. Such a package may contain multiple vocabularies / ontologies. Packaging related SHACL shapes is reasonable, too. + +`package_id` +: Unique package identifier + +`package_version` +: Semantic version identifier string of the package, but limited to proper releases. + +`metadata.name` +: The package name in English. + +`metadata.description` +: The package description in English. + +`metadata.license` +: The [SPDX license identifier](https://spdx.org/licenses/) of the package, e.g. `Apache-2.0`. + +`metadata.comment` +: A maintainer or publisher comment. + +`metadata.agents` +: Publishers, authors, and contributors of the package. + +`metadata.urls` +: Related links, e.g. the homepage or the issue tracker of the package. + +`metadata.tags` +: Free-text tags used to categorize the package on a Marketplace Server. + +#### Files + +A package can contain graphs, Build projects, text files, and images. +These contents are referenced in the `files` section of the `cpa-manifest.json`. + +##### Graphs + +Use the following structure to include a graph. +`register_as_vocabulary` and `import_into` are optional instructions. +We suggest to organize graphs in a respective sub-folder (here `graphs/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "graph", + "file_path": "graphs/file.ttl", + "graph_iri": "http://www.example.org/file/", + "register_as_vocabulary": true, + "import_into": [ + "http://www.example.org/integration_graph/" + ] + }, + … +] +``` + +##### Projects + +Use the following structure to include a project. +We suggest to organize projects in a respective sub-folder (here `projects/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "project", + "file_path": "projects/my-build-project.zip", + "project_id": "my-build-project" + }, + … +] +``` + +##### Texts and Images + +Text files and images describe the package itself rather than shipping content. +The template declares `README.md`, `CHANGELOG.md`, and `LICENSE` this way; images are used to represent the package on a Marketplace Server: + +```json +"files": [ + … + { + "file_type": "text", + "file_path": "README.md", + "file_role": "readme" + }, + { + "file_path": "icon.png", + "file_type": "image", + "file_role": "icon" + }, + … +] +``` + +#### Dependencies + +Dependencies to other packages or to Python plugins can be declared in the `copier copy` answers. +The dependencies are added to the `cpa-manifest.json` as described in the next sections. + +##### Python Plugin Packages + +Use the following to declare a dependency to a Python plugin: + +```json +"dependencies": [ + … + { + "dependency_type": "python-package", + "pypi_id": "cmem-plugin-pyshacl" + }, + … +] +``` + +##### Marketplace Packages + +Use the following to declare a dependency to another Marketplace Package: + +```json +"dependencies": [ + … + { + "dependency_type": "marketplace-package", + "package_id": "w3c-rdfs-vocab" + } + … +] +``` + +## Package Development Cycle + +!!! info "`cmemc package` reference" + + The [cmemc package command group](../../../automate/cmemc-command-line-interface/command-reference/package/index.md) + contains all needed commands to support the complete package lifecycle. + +Some packages are simply wrapping existing artifacts into a managed structure (e.g. existing vocabulary/ontology). + +Most (solution) package development and evolution will be a back and forth between a package repository (making changes to `cpa-manifest.json` in terms of adding/removing dependencies, graph files, or Build project files) and a Corporate Memory (package development) instance. + +![Corporate Memory Marketplace Package Lifecycle](../mpp-lifecycle.svg){ width="50%" } + +!!! tip "Task wrappers" + + The generated package repository ships a `Taskfile.yaml` which wraps the commands below into `task import`, `task export`, `task build`, `task check`, `task delete`, and `task publish`. + The [tutorial](tutorial/index.md) uses these wrappers. + +### Install (local) Packages + +Use the following command to install a local package folder content (or built `.cpa` file) to a Corporate Memory (package development) instance. + +```shell +cmemc package install --input PATH +``` + +Make changes to graphs, configuration, or Build projects as needed. +Newly created or imported graphs or Build projects need to be registered in `cpa-manifest.json` so they will be fetched by `export`. + +### Export Contents into a Package + +Use the following command to export the file artifacts declared in `cpa-manifest.json` from a Corporate Memory (package development) instance to a local package folder. + +```shell +cmemc package export PACKAGE_ID +``` + +Run this to initially populate package contents from a solution configuration. You can also use it to update contents after making changes on your Corporate Memory (package development) instance, capturing them for building and releasing as a Marketplace Package. + +For version controlled package directories, add `--extract` to store Build projects as extracted directories instead of ZIP archives (the manifest still references the ZIP; `build` and `install` zip it silently). + +### Inspect Packages + +Review and verify the contents of a package with the following command: + +```shell +cmemc package inspect PACKAGE_PATH +``` + +### Build Packages + +During development you can install a package from a local path (plain folder or a `.cpa` package) using the `cmemc package install --input PATH` command. + +Use the `cmemc package build` command. +This will build a package archive from a package directory. + +This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory package archive (`.cpa`) with the following naming convention: `{package_id}-v{version}.cpa`. + +### Publish Packages + +Package archives can be published to the Marketplace Server using the `cmemc package publish` command. +After being published packages can be found and installed directly from the Marketplace Server (potential users do not need to have the local package folder or `.cpa` file available). diff --git a/docs/develop/packages/development/tutorial/index.md b/docs/develop/packages/development/tutorial/index.md new file mode 100644 index 000000000..fdf4720b4 --- /dev/null +++ b/docs/develop/packages/development/tutorial/index.md @@ -0,0 +1,157 @@ +--- +title: "Marketplace Packages: Development Tutorial" +icon: material/school +tags: + - Marketplace + - Package +--- +# Tutorial: Develop your first Marketplace Package + +This tutorial walks you through a basic example of creating a new Marketplace Package, adding different types of content to it, and finally building it into a package archive ready for distribution. + +It is a "how to" and does not replace the full documentation of the [package-template](https://github.com/eccenca/cmem-package-template) repository. +See [Development and Publication](../index.md) for the underlying concepts, and note that advanced scenarios such as publishing are only outlined at the end. + +## Initialize the Package Repository + +Follow the [template usage instructions](https://github.com/eccenca/cmem-package-template/tree/main#usage) to create a local package repository. +For our example, we answer the template questions as follows: + +```shell title="copier copy gh:eccenca/cmem-package-template my-package-id" +🎤 Type of package + Project Package +🎤 Package ID (e.g., 'eccenca-supply-chain-vocab', 'w3c-org-vocab') + my-package-id +🎤 Human-readable package name (e.g., 'My Awesome Vocabulary', 'My Great Project') + My own package +🎤 Short description of the package (e.g., 'A vocabulary for ...', 'A project that ...') + My project and graphs +🎤 Comma-separated Python package dependencies (e.g., 'cmem-plugin-pyshacl, cmem-plugin-llm') + +🎤 Comma-separated vocabulary or project dependencies (e.g., 'aksw-rut-vocab, my-other-project') + +🎤 github_page: This URL (e.g. https://github.com/user/repo) will be used as the base for icons and the homepage link. Leave blank if your package is not on github. + +``` + +You should now have a folder with two levels of files: + +- Top level - generic package repository information such as the changelog, README, CI instructions, licensing, and the `Taskfile.yaml`. +- Nested folder (`my-package-id`) - the package directory holding the actual package content, along with the `cpa-manifest.json` manifest. + +## Add Package Content + +The nested folder `my-package-id` represents your working directory for developing the package. + +To add content to the package, simply copy the files you want to add into this folder, or extract existing content from a live Corporate Memory instance into the working directory. + +!!! example "Extracting Corporate Memory content to add to the package" + + ```shell + cmemc graph export https://my-company.org/queries/ --output-file my-package-id/queries.ttl + + cmemc project export MyProject_78e981443900a761 --output-dir my-package-id + Export project 1/1: MyProject_78e981443900a761 to my-package-id/2026-07-08-unnamed-MyProject_78e981443900a761.project.zip ... done + + mv my-package-id/2026-07-08-unnamed-MyProject_78e981443900a761.project.zip my-package-id/project.zip + ``` + +## Declare the Files in the Manifest + +In order for the package to know about these added files, the `cpa-manifest.json` needs to be edited. + +The `"files": []` section of the manifest references the files the package needs to bundle. +Complete information about the [package manifest can be found here](https://github.com/eccenca/cmem-package-template/tree/main#package-manifest), and more specifically [how to declare new files](https://github.com/eccenca/cmem-package-template/tree/main#adding-files). + +For our example, we add a query graph and a project file. +Make sure each `file_path` is valid and relative to your package directory (the nested folder): + +```json title="my-package-id/cpa-manifest.json" +"files": [ + {…}, + { + "file_path": "queries.ttl", + "file_type": "graph", + "graph_iri": "https://my-company.org/queries/", + "import_into": [], + "register_as_vocabulary": false + }, + { + "file_path": "project.zip", + "file_type": "project", + "project_id": "MyProject_78e981443900a761" + } +] +``` + +## Test your Package + +To ensure the package correctly detects your added files, you can try to import it into a Corporate Memory instance. + +The package template comes with a predefined `Taskfile.yaml` allowing you to wrap your development steps in single commands: + +```shell +task: Available tasks for this project: +* build: Build package archive +* check: Run whole test suite +* clean: Removes dist, *.cpa, ... +* delete: Delete (uninstall) package from Corporate Memory +* export: Export package content from Corporate Memory +* import: Import (install) package to Corporate Memory +* publish: Publish package archive to the marketplace +``` + +To tell the package system to take files from the local working directory and to import them into Corporate Memory, we use **task import**. +An import always tries to uninstall a previously installed version of the same package first, to ensure it is correctly replaced. + +```shell +task import +task: [delete] cmemc package uninstall $package_id +Package 'my-package-id' is not installed. +task: [import] cmemc package install --input $package_dir +Installing package 'my-package-id' from 'my-package-id' ... done +``` + +!!! warning "Importing duplicated content" + + If you extracted already existing content from Corporate Memory, added it to your package with the same identifiers (graph IRIs, project IDs, ...), and try to import it back in the form of a new package, the instance might raise a `MarketplacePackagesImportError` due to conflicting elements, e.g. `Repository item 'https://my-company.org/queries/' already exists.` + + In this case, you can simply delete the duplicated content inside Corporate Memory (make sure you have backups) before importing it back as package content. + The difference is that Corporate Memory now knows this content is part of a managed package, and will handle import/export of that file from now on. + +## Update the Package File Content + +If you make modifications to your package content in Corporate Memory, the files will not automatically sync back to your local working directory. + +To extract all updated content from Corporate Memory into your package working directory in a managed way, simply run **task export**: + +```shell +task export +``` + +!!! warning "Exporting without installing first" + + The platform can only export updated versions of package files that were imported at least once before. If you create new information directly in Corporate Memory that the package manifest does not yet declare, such as new graphs, you need to manually add them to your working directory and to your manifest, and then import them. + + The rule of thumb is: if you need to make a structural change to your package that requires you to edit your manifest, then make sure to run `import` right after, to let Corporate Memory keep track of new files. + + Adding a workflow inside a project is not impacted by this limitation, since it is part of the "project" that is managed and tracked by the package. + +## Build your Package + +To generate a `.cpa` file ready to be distributed and installed in different Corporate Memory instances, you can run **task build**. + +Make sure your local package folder is a git repository with a clean state - the task derives the package version from `git describe`, so the commit hash ends up in the archive name (e.g. `my-package-id-v0.0.0-4b7516f.cpa`). + +```shell +task build +``` + +To check how this output `.cpa` file can be installed in different places, refer to the [Installation and Management](../../installation/index.md) section. + +!!! success "Next steps" + + There are many improvements you can add to your package, such as declaring dependencies to other plugins or packages, to ensure your `.cpa` file can be installed with all its requirements everywhere. For that, you can refer to existing package examples, the [Development and Publication](../index.md) page, or the template documentation. + + The final step is usually publishing a version of the package to a remote Marketplace Server, to avoid having to transfer the `.cpa` archive manually. This requires you to have publishing permissions on an eccenca Marketplace Server (either a public or private instance). + This can be done with **task publish**, either manually or from a CI runner. diff --git a/docs/develop/packages/index.md b/docs/develop/packages/index.md new file mode 100644 index 000000000..48a18149a --- /dev/null +++ b/docs/develop/packages/index.md @@ -0,0 +1,59 @@ +--- +status: new +title: "Marketplace Packages: Overview" +icon: material/shopping +tags: + - Marketplace + - Package +hide: + - toc +--- + +# Marketplace Packages + +Starting with version 26.1, we support the creation and use of Marketplace Packages. + +Marketplace Packages bundle everything for a specific Corporate Memory–based solution or project into a single shareable, managed artifact: + +- Vocabularies / Ontologies +- (SKOS) Taxonomies +- (Instance / Data) Graphs +- Build Projects +- Dependencies on + - [python-plugins](../python-plugins/index.md) + - (other) Marketplace Packages + +This lets you share and reuse them across projects, teams, and different Corporate Memory instances. + +A Marketplace Package is distributed as a **C**orporate Memory **P**ackage **A**rchive (`.cpa` file), a zip-based archive which you can either hand over directly or publish to a Marketplace Server - a central repository which supports pushing and pulling packages. + +The lifecycle of a Corporate Memory Marketplace Package is shown in the following flowchart. + +![Corporate Memory Marketplace Package Lifecycle](mpp-lifecycle.svg){ width="50%" } + +!!! info "Looking for the user interface?" + + The [Marketplace](../../distribution/marketplace/index.md) chapter describes how to discover, install and uninstall packages in the Corporate Memory user interface. + The pages below focus on the command line and on package development. + +The following pages give an overview of this feature: + +
+ +- :material-download-circle-outline: [Installation and Management](installation/index.md) + + --- + + Intended for Linked Data Experts, Deployment Engineers, and Corporate Memory Admins, this page outlines how to (un)install and manage Marketplace Packages, and where installed contents appear in Corporate Memory. + + This section discusses the lifecycle commands and stages `search`, `install`, `list` and `uninstall`. + +- :material-code-json: [Development and Publication](development/index.md) + + --- + + Intended for Developers, Linked Data Experts, Consultants, and Partners, this page gives an overview of how to start developing and publish Marketplace Packages, followed by a [step-by-step tutorial](development/tutorial/index.md). + + This section discusses the lifecycle commands and stages `copier copy`, _Package Definition and Release_, `inspect`, `install --input PATH` (from local), _Solution Development and Configuration_, `export`, `build`, and `publish`. + +
diff --git a/docs/develop/packages/installation/example-project.png b/docs/develop/packages/installation/example-project.png new file mode 100644 index 000000000..753ed3baf Binary files /dev/null and b/docs/develop/packages/installation/example-project.png differ diff --git a/docs/develop/packages/installation/example-vocabulary.png b/docs/develop/packages/installation/example-vocabulary.png new file mode 100644 index 000000000..013961a18 Binary files /dev/null and b/docs/develop/packages/installation/example-vocabulary.png differ diff --git a/docs/develop/packages/installation/index.md b/docs/develop/packages/installation/index.md new file mode 100644 index 000000000..49788a23a --- /dev/null +++ b/docs/develop/packages/installation/index.md @@ -0,0 +1,105 @@ +--- +title: "Marketplace Packages: Installation and Management" +icon: material/download-circle-outline +tags: + - Marketplace + - Package +--- +# Installation and Management of Marketplace Packages + +## Introduction + +Marketplace Packages can be installed directly from a Corporate Memory Marketplace Server (e.g. [https://eccenca.market](https://eccenca.market)), or from local **C**orporate Memory **P**ackage **A**rchives (`.cpa` files) and package directories. + +This page describes how to search, install, list, and uninstall Marketplace Packages using `cmemc`. + +!!! info "Managing packages in the user interface" + + Packages published on a Marketplace Server can also be installed and uninstalled in the [Marketplace](../../../distribution/marketplace/index.md) module of Corporate Memory. + +!!! info "`cmemc package` reference" + + The [cmemc package command group](../../../automate/cmemc-command-line-interface/command-reference/package/index.md) + contains all needed commands to support the complete package lifecycle. + +## Search Packages + +Use the following command to search a Marketplace Server for available packages: + +```shell title="Search the Marketplace Server" +cmemc package search vocab +``` + +## Install Packages + +Use the following command to install a package from a Marketplace Server: + +```shell-session title="Install a package from the Marketplace Server" +$ cmemc package install w3c-xsd-vocab +Installing package 'w3c-xsd-vocab' from marketplace ... done +``` + +For installing local package archives (`.cpa` files) or package directories, use the `--input` option: + +```shell-session title="Install a package from a .cpa file" +$ cmemc package install --replace --input my-package-v0.0.0-4b7516f.cpa +Installing package 'my-package' from 'my-package-v0.0.0-4b7516f.cpa' +done +``` + +!!! info "Replacing installed packages" + + Use `--replace` to overwrite an already installed package version or package content. + Without this option, installing over existing content fails. + +## List Packages + +Use the following command to list all installed packages: + +```shell title="List installed packages" +cmemc package list +``` + +To review the manifest of a package (installed, local directory, or `.cpa` file), use the `inspect` command: + +```shell title="Inspect a package manifest" +cmemc package inspect my-package-v0.0.0-4b7516f.cpa +``` + +## Where Package Contents Appear + +Depending on the content types inside it, an installed package appears in different places in Corporate Memory, with each item (graph, project, workflow, ...) surfacing in its respective component. + +
+ +!!! info inline "" + + ![Example: Graphs](example-vocabulary.png "Example: Graphs") + +**Graphs** such as data graphs but also **Vocabularies** or **Shapes Catalogs** are listed in [**Explore > Graphs**](../../../explore-and-author/graph-exploration/index.md#graphs). + +
+ +
+ +!!! info inline "" + + ![Example: Projects](example-project.png "Example: Projects") + +**Projects** are imported into [**Build**](../../../build/introduction-to-the-user-interface/index.md#projects). +When you install your first project package, Corporate Memory also creates a special project to store all installed files. +This project is automatically managed by the package system, and removed once the last package is uninstalled. + +
+ +
+ +## Uninstall Packages + +Use the following command to uninstall a package: + +```shell title="Uninstall a package" +cmemc package uninstall PACKAGE_ID +``` + +This removes all package contents from the Corporate Memory instance, including graphs and Build projects that were installed as part of the package. diff --git a/docs/develop/packages/mmd.txt b/docs/develop/packages/mmd.txt new file mode 100644 index 000000000..fc2ee5b30 --- /dev/null +++ b/docs/develop/packages/mmd.txt @@ -0,0 +1,13 @@ +# mermaid + +## live editor + +https://mermaid.live/edit#pako:eNqFU12L2zAQ_CuLIHVb7DSxL3YiykFJHnpwgaMfL636oNgbR0SWjCIdlwv575WcuA6FUvxg7WhmZ3clnUipKySUJEnClBVWIoWlNq023CKssdHmCGtu9mhbyUuEJ17ueY3wKLZYHkuJTHXa0egklLAUThBJXT_iM8qIQlThxtVRDJHdYYMBUeis4TKCM5xHI6YO1lutBK8Nb5LnlKnwdSAw0tutcBvSC60-bsw9VxV8QYn8gCF8K3XJ5TtGgB-g3dc3creR4rDDqqPd9NGT3WYgj8uW9_0NaeENLB8-LFdXhecMiq9aulCTL-8ZQlFLrbai7sR_T7HXN9gMCR7CSsprff-QCHUI84Ve9F2JXnYhOCXCzH6-_wVJch8mABRK3Qo04XdkKkDDlpe3WFqmQsMBDkVd8JAWQjEeF6p1Fp4-ffs8JPg_s2MMXvjiW7I3CbwlhY0Tshr8_TF4sL0cVmglAGHD-wxuLPR9A3uyHbDQPQ2j6MkkJrURFaHWOIxJg6bhISQnpsCPsbuQjFC_vF5JRpg6e1nL1Q-tm15ptKt3hG65PPjItdVwX_-gBlWFZqmdsoSmWZp1WQg9kRdCs2k-nswWxSy7mxaLfF7MYnIkdDpPx5NJUeR5kWZ5ls7PMXntfCfjYpIWd9lsvkizrJjli5hgJaw268tz7V7t-TfNwDqo + +## cli + +https://github.com/mermaid-js/mermaid-cli + +```sh +mmdc -i mpp-lifecycle.mmd -o mpp-lifecycle.svg -b transparent +``` diff --git a/docs/develop/packages/mpp-lifecycle.mmd b/docs/develop/packages/mpp-lifecycle.mmd new file mode 100644 index 000000000..22960e9d9 --- /dev/null +++ b/docs/develop/packages/mpp-lifecycle.mmd @@ -0,0 +1,25 @@ +--- +#title: Corporate Memory Marketplace Package Lifecycle +--- +%%{init: { 'logLevel': 'debug', 'theme': 'neutral' } }%% +stateDiagram-v2 + + +state "Package Definition
and Release
(local)" as pkg +state "Published
(Marketplace)" as pub +state ".cpa Package
(local & CI/CD)" as cpa +state "Solution Dev and Config
(Corporate Memory)" as cmem +state "Installed
(Corporate Memory)" as ins +%% state "Uninstalled" as uni + +[*] --> pkg : copier copy +pkg --> pkg : inspect +cpa --> cmem : install
--input PATH +pkg --> cmem : install
--input PATH +cmem --> pkg : export +pkg --> cpa : build +cpa --> pub : publish + +pub --> ins : install +ins --> ins : list +ins --> [*] : uninstall diff --git a/docs/develop/packages/mpp-lifecycle.svg b/docs/develop/packages/mpp-lifecycle.svg new file mode 100644 index 000000000..a6835805a --- /dev/null +++ b/docs/develop/packages/mpp-lifecycle.svg @@ -0,0 +1 @@ +

copier copy

inspect

install
--input PATH

install
--input PATH

export

build

publish

install

list

uninstall

Package Definition
and Release
(local)

Published
(Marketplace)

.cpa Package
(local & CI/CD)

Solution Dev and Config
(Corporate Memory)

Installed
(Corporate Memory)

 \ No newline at end of file diff --git a/docs/develop/python-plugins/installation/index.md b/docs/develop/python-plugins/installation/index.md index 7b4a73896..9cfd0f98c 100644 --- a/docs/develop/python-plugins/installation/index.md +++ b/docs/develop/python-plugins/installation/index.md @@ -7,7 +7,7 @@ tags: --- # Installation and Usage of Python Plugins -Plugins are a released as parts of Python packages. +Plugins are released as parts of Python packages. They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain multiple plugins. ## Installation diff --git a/docs/distribution/.pages b/docs/distribution/.pages new file mode 100644 index 000000000..716b2d80b --- /dev/null +++ b/docs/distribution/.pages @@ -0,0 +1,3 @@ +nav: + - Distribution: index.md + - Marketplace: marketplace diff --git a/docs/distribution/index.md b/docs/distribution/index.md new file mode 100644 index 000000000..aa8815f1a --- /dev/null +++ b/docs/distribution/index.md @@ -0,0 +1,33 @@ +--- +status: new +icon: material/star +tags: + - Marketplace + - Package +hide: + - toc +--- + +# :material-star: Distribution + +This section describes how ready-made Corporate Memory content is distributed, shared and reused - across projects, teams and Corporate Memory instances. + +Vocabularies / ontologies, taxonomies, data graphs, Build projects and query catalogs do not need to be moved around one by one. +They are bundled into **Marketplace Packages**: single, versioned artifacts which are offered on a Marketplace Server and can be installed into your Corporate Memory instance with a few clicks. + +**:octicons-people-24: Intended audience**: All Corporate Memory users + +
+ +- :eccenca-module-marketplace: [Marketplace](marketplace/index.md) + + --- + + Discover ready-made ontologies, vocabularies, demo projects and complete solutions in the Marketplace module, and install, update or uninstall them in your Corporate Memory instance. + +
+ +!!! info "Related sections" + + - [Marketplace Packages: Installation and Management](../develop/packages/installation/index.md) describes the same lifecycle on the command line with [cmemc](../automate/cmemc-command-line-interface/index.md), including the installation of local package archives. + - [Marketplace Packages: Development and Publication](../develop/packages/development/index.md) describes how to create and publish your own packages. diff --git a/docs/distribution/marketplace/index.md b/docs/distribution/marketplace/index.md new file mode 100644 index 000000000..ce14f8c5c --- /dev/null +++ b/docs/distribution/marketplace/index.md @@ -0,0 +1,232 @@ +--- +status: new +title: "Marketplace" +icon: eccenca/module-marketplace +tags: + - Marketplace + - Package +--- + +# Marketplace + +## Introduction + +The Marketplace is the place in Corporate Memory where you find ready-made content and add it to your instance: ontologies and vocabularies, taxonomies, data graphs, Build projects, query catalogs, as well as complete demo and solution setups. + +All of this content is delivered as **Marketplace Packages**. +A package is a single, versioned artifact which bundles everything belonging to a solution, together with the packages and plugins it depends on. +Instead of collecting and importing graphs and projects one by one, you install a package, and Corporate Memory places all its contents where they belong. + +Packages are offered by a **Marketplace Server**, a central repository your Corporate Memory instance is connected to. +The public Marketplace Server operated by eccenca is available at [https://eccenca.market](https://eccenca.market). + +!!! info "Availability" + + The Marketplace is available starting with Corporate Memory version 26.1. + + In order to open the Marketplace and to install or uninstall packages, your user account needs the `:Marketplace-Frontend` action, plus access to the graphs a package writes to. + If the **Packages** entry is missing from the navigation menu, or if the **Install** and **Uninstall** buttons do not react, contact your Corporate Memory administrator. + See [Access Conditions](../../deploy-and-configure/configuration/access-conditions/index.md) for details. + +## Open the Marketplace + +To open the Marketplace: + +- Click **:material-menu: Open main navigation** in the header. +- Click **:eccenca-module-marketplace: Packages** in the **MARKETPLACE** group. + +![The Packages entry in the navigation menu](marketplace-navigation.png "The Packages entry in the navigation menu"){ class="bordered" } + +## Discover Packages + +The overview page lists all packages offered by the [selected Marketplace](#marketplace-top-right), and marks the ones already installed in your instance with an **Uninstall** action. + +![The Marketplace overview page](marketplace-overview.png "The Marketplace overview page"){ class="bordered" } + +Each package is shown as a card with + +- the package icon, name and description, +- a badge stating the [package type](#package-types) (`vocabulary` or `project`), +- the publisher or author, the license and the version, and +- the actions **Install** or **Uninstall**, and **Details**. + +### Filter and Search + +Use the controls above the package list to narrow down what is shown. +They can be combined. + +![The controls above the package list](marketplace-controls.png "The controls above the package list"){ class="bordered" } + +
+ +!!! info inline "" + + ![The open Package type drop-down](marketplace-filter-package-type.png "The open Package type drop-down") + +#### Package type + +Show only **Vocabulary** or only **Project** packages, see [Package Types](#package-types). + +
+ +
+ +!!! info inline "" + + ![The open License drop-down](marketplace-filter-license.png "The open License drop-down") + +#### License + +Show only packages published under one of the offered [SPDX licenses](https://spdx.org/licenses/). + +
+ +
+ +#### :eccenca-module-search: Search + +Show only packages whose name or description contains the entered keyword. + +
+ +
+ +!!! info inline "" + + ![The Installed toggle, switched on](marketplace-filter-installed.png "The Installed toggle, switched on") + +#### Installed + +Switch on to show only the packages currently installed in your instance. + +
+ +
+ +!!! info inline "" + + ![The open Marketplace drop-down](marketplace-filter-server.png "The open Marketplace drop-down") + +#### Marketplace (top right) + +Select which Marketplace you work with. +The drop-down lists all Marketplaces configured for your Corporate Memory instance with their name and URL, and the package list shows the packages of the selected one. +It is inactive if only one Marketplace is configured. + +
+ +
+ +If nothing matches, the page states _No packages match the current filters._ +Reset the controls to see the full list again. + +### Package Types + +`vocabulary` +: Packages which contribute vocabulary / ontology content, such as `rdf:`, `org:` or `sso:`. + Such a package can contain several vocabularies as well as the matching SHACL shapes. + +`project` +: Packages which can ship any kind of content, mainly Build projects, (instance / data) graphs, SHACL shapes, workspace configuration and query catalogs. + Demo and solution packages are of this type. + +## Inspect a Package + +Click **Details** on a package card to open the package details page. +Use it to check what a package contains and what it pulls in before you install it. + +![The details page of the Product Data Demo package](marketplace-package-details.png "The details page of the Product Data Demo package"){ class="bordered" } + +The details page shows: + +- The package type badge, icon, name and short description, together with the **Install** or **Uninstall** button. +- **Publisher** - the organization or person publishing the package, linked to its homepage if provided. +- **Identifier** - the unique package ID, for example `ecc-product-data-project`. You need this ID when you work with [cmemc](#manage-packages-on-the-command-line). +- **License** - the license the package is published under, linked to the license text. +- **Version** - the released version the page shows. Open the list to look at the description, tags and dependencies of another version of this package. +- The long description of the package, describing in detail which graphs, projects, queries and configurations are installed. +- **Tags** - free-text keywords used by the publisher to categorize the package. +- **Changelog** - click **View History** to read the release notes of all versions of this package. +- **Dependencies** - the other Marketplace Packages and [Python plugins](../../develop/python-plugins/index.md) this package needs. They are installed together with the package. +- Screenshots and further images shipped by the package, if any. + +Click **:material-arrow-left: Back** to return to the overview. + +## Install a Package + +To install a package, click **Install**, either on the package card in the overview or on the package details page. +The package is fetched from the Marketplace Server and installed into your Corporate Memory instance, together with the dependencies listed on its details page. + +Clicking **Install** starts the installation right away - there is no additional confirmation dialog, so make sure you picked the right package and version beforehand. +Depending on the size of the package and the number of dependencies, the installation can take a while. +Once it has finished, the card and the details page offer **Uninstall** instead of **Install**. + +!!! tip "Installing a specific version" + + The latest version is preselected. + To install an older one, open the package details page, select the version you want in the **Version** list, and click **Install**. + +!!! info "One installation at a time" + + Only one installation can run at a time. + Starting another one while an installation is still in progress results in an error. + Wait until the running installation has finished, then start the next one. + +### Change the Version of an Installed Package + +There is no update action in the Marketplace, and the **Version** list does not offer to install another version of a package that is already installed: +as long as a package is installed, its card and its details page only offer **Uninstall**, no matter which version you select. + +To move an installed package to another version - a newer as well as an older one: + +- [Uninstall](#uninstall-a-package) the package. +- Select the version you want in the **Version** list on the package details page. +- Click **Install**. + +!!! warning "The detour via uninstall removes the package contents" + + Since the package has to be uninstalled first, its contents - including your changes to them - are removed before the other version is installed, see [Uninstall a Package](#uninstall-a-package). + Export what you want to keep beforehand. + + On the command line, `cmemc package install --replace` overwrites an installed package version in place, without uninstalling it first, see [Manage Packages on the Command Line](#manage-packages-on-the-command-line). + +## Where the Package Contents Appear + +An installed package does not show up as one single item in Corporate Memory. +Its contents surface in the module they belong to: + +- **Graphs**, including vocabularies and shapes catalogs, are listed in [**Knowledge Graphs**](../../explore-and-author/graph-exploration/index.md#graphs). +- **Build projects** are imported into [**Projects**](../../build/introduction-to-the-user-interface/index.md#projects). + When your first project package is installed, Corporate Memory also creates a special project holding all installed files. + It is managed automatically and removed again once the last package has been uninstalled. +- **Queries** are shipped as a query catalog graph and are therefore available in the [Query module](../../explore-and-author/query-module/index.md). + +For a package such as _Product Data Demo_, this means that right after the installation you can start exploring the installed integration graph, run the shipped queries, and execute the shipped workflows. + +## Uninstall a Package + +To remove a package, click **Uninstall** on the package card or on the package details page, and confirm the **Confirm action** dialog with **Confirm**. + +This removes all contents of the package from your Corporate Memory instance, including the graphs and the Build projects that came with it. + +!!! warning "Uninstalling removes data" + + Changes you made to the graphs or Build projects of a package are removed together with the package. + Export what you want to keep before you uninstall. + +## Manage Packages on the Command Line + +Everything described on this page can also be done with the [cmemc command line interface](../../automate/cmemc-command-line-interface/index.md), which additionally allows you to install packages that are not published on a Marketplace Server, for example a **C**orporate Memory **P**ackage **A**rchive (`.cpa` file) you received directly: + +```shell-session title="Install a package from the Marketplace Server" +$ cmemc package install ecc-product-data-project +Installing package 'ecc-product-data-project' from marketplace ... done +``` + +See [Marketplace Packages: Installation and Management](../../develop/packages/installation/index.md) for the complete workflow and the [package command group](../../automate/cmemc-command-line-interface/command-reference/package/index.md) for the command reference. + +## Build Your Own Packages + +Vocabularies, graphs, Build projects and configurations of your own solution can be bundled into a Marketplace Package as well, and then be shared with other teams or published to a Marketplace Server. + +See [Marketplace Packages: Development and Publication](../../develop/packages/development/index.md) for an overview and the [step-by-step tutorial](../../develop/packages/development/tutorial/index.md) to build your first package. diff --git a/docs/distribution/marketplace/marketplace-controls.png b/docs/distribution/marketplace/marketplace-controls.png new file mode 100644 index 000000000..f8cabf708 Binary files /dev/null and b/docs/distribution/marketplace/marketplace-controls.png differ diff --git a/docs/distribution/marketplace/marketplace-filter-installed.png b/docs/distribution/marketplace/marketplace-filter-installed.png new file mode 100644 index 000000000..c022de87b Binary files /dev/null and b/docs/distribution/marketplace/marketplace-filter-installed.png differ diff --git a/docs/distribution/marketplace/marketplace-filter-license.png b/docs/distribution/marketplace/marketplace-filter-license.png new file mode 100644 index 000000000..82bcaed77 Binary files /dev/null and b/docs/distribution/marketplace/marketplace-filter-license.png differ diff --git a/docs/distribution/marketplace/marketplace-filter-package-type.png b/docs/distribution/marketplace/marketplace-filter-package-type.png new file mode 100644 index 000000000..886ffda4c Binary files /dev/null and b/docs/distribution/marketplace/marketplace-filter-package-type.png differ diff --git a/docs/distribution/marketplace/marketplace-filter-server.png b/docs/distribution/marketplace/marketplace-filter-server.png new file mode 100644 index 000000000..89c493b38 Binary files /dev/null and b/docs/distribution/marketplace/marketplace-filter-server.png differ diff --git a/docs/distribution/marketplace/marketplace-navigation.png b/docs/distribution/marketplace/marketplace-navigation.png new file mode 100644 index 000000000..bac34a7fc Binary files /dev/null and b/docs/distribution/marketplace/marketplace-navigation.png differ diff --git a/docs/distribution/marketplace/marketplace-overview.png b/docs/distribution/marketplace/marketplace-overview.png new file mode 100644 index 000000000..2edf47f13 Binary files /dev/null and b/docs/distribution/marketplace/marketplace-overview.png differ diff --git a/docs/distribution/marketplace/marketplace-package-details.png b/docs/distribution/marketplace/marketplace-package-details.png new file mode 100644 index 000000000..0f71f837d Binary files /dev/null and b/docs/distribution/marketplace/marketplace-package-details.png differ diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md index d681df92f..b3ac12db8 100644 --- a/docs/getting-started/index.md +++ b/docs/getting-started/index.md @@ -69,8 +69,6 @@ The user interface of Corporate Memory usually consists of two sections: Each module provides a set of functionalities and views for specific use cases. To access a module, click the module name. The active module is highlighted. - - By default, Corporate Memory provides the following modules: - EXPLORE - for Knowledge Graph browsing and exploration, specifically @@ -82,7 +80,7 @@ By default, Corporate Memory provides the following modules: - Datasets - the Datasets across all BUILD Projects - Workflows - the Workflows across all BUILD Projects - Activities - activities overview and monitoring -- [MARKETPLACE](#) - for discovering, installing and managing solution packages such as ontologies / vocabularies or project and demo packages +- [MARKETPLACE](../distribution/marketplace/index.md) - for discovering, installing and managing solution packages such as ontologies / vocabularies or project and demo packages !!! note diff --git a/docs/index.md b/docs/index.md index 64a1fe407..5302e156e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -45,6 +45,12 @@ hide: This section outlines how to consume data from the Knowledge Graph. +- :material-star: [Distribution](distribution/index.md) + + --- + + Discover, install and manage ready-made Marketplace Packages. + - :material-star-outline: [Deploy and Configure](deploy-and-configure/index.md) ---