diff --git a/docs/code-navigation/auto-indexing-configuration.mdx b/docs/code-navigation/auto-indexing-configuration.mdx index c89272953..b27e57bf9 100644 --- a/docs/code-navigation/auto-indexing-configuration.mdx +++ b/docs/code-navigation/auto-indexing-configuration.mdx @@ -33,33 +33,33 @@ The pre-indexing steps, indexer container, local steps, and indexer arguments ar Each indexing job object can be configured with the following keys. -#### [`steps`](#index-job-steps) +#### [`steps`](#steps) The steps field defines an ordered sequence of pre-indexing actions (formatted as a [Docker step object](#docker-step-object)). Each step is executed before the indexer itself is invoked. -#### [`indexer`](#index-job-indexer) +#### [`indexer`](#indexer) The name of the Docker image distribution of the target indexer. -#### [`local_steps`](#index-job-local-steps) +#### [`local_steps`](#local_steps) An ordered sequence of commands to execute within a container running the configured indexer image. These commands are passed directly into the target container via `docker exec`, one at a time. Local steps should be used over Docker steps when the intended side effects alter state outside of the workspace on disk (e.g., setting up environment variables or installing OS packages require by the indexer tool). -#### [`indexer_args`](#index-job-indexer-args) +#### [`indexer_args`](#indexer_args) An ordered sequence of arguments that make up the indexing command. The indexing command is passed directly into the target container via `docker exec`. This step is expected to produce a code graph index artifact (as described by the `root` and `outfile` fields, described below). -#### [`root`](#index-job-root) +#### [`root`](#root) The working directory within the Docker container where the provided local steps and indexer commands are executed. This working directory is relative to the root of the target repository. An empty value (the default) indicates the root of the repository. This is also the directory relative to the path where the code graph index artifact is produced. -#### [`outfile`](#index-job-outfile) +#### [`outfile`](#outfile) The path to the code graph index artifact produced by the indexer, which is uploaded to the target Sourcegraph instance via [`src` CLI](/cli/) after the index step has completed successfully. This path is relative to the index job `root` (defined above). If not supplied, the value is assumed to be `dump.lsif` (which is the default artifact name of many indexers). Supply this argument when the target indexer produces a differently named artifact. Alternatively, some indexers provide flags to change the artifact name; in which case `dump.lsif` can be supplied there and a value for this key can be omitted. -#### [`requested_envvars`](#index-job-requested-envvars) +#### [`requested_envvars`](#requested_envvars) List of environment variables needed. These are made accessible to steps, local_steps, and the indexer_args command. @@ -102,15 +102,15 @@ Each configured Docker step is executed sequentially using the same volume-mount Each Docker step object can be configured with the following keys. -#### [`image`](#docker-step-image) +#### [`image`](#image) The name of the Docker image in which the configured commands are executed. -#### [`commands`](#docker-step-commands) +#### [`commands`](#commands) An ordered sequence of commands to execute within a container running the configured image. These commands are passed directly into the target container via `docker exec`, one at a time. -#### [`root`](#docker-step-root) +#### [`root`](#root-1) The working directory within the Docker container where the provided commands are executed. This working directory is relative to the root of the target repository. An empty value (the default) indicates the root of the repository. diff --git a/docs/code-navigation/auto-indexing.mdx b/docs/code-navigation/auto-indexing.mdx index 25647f9c8..bdc2a7e9f 100644 --- a/docs/code-navigation/auto-indexing.mdx +++ b/docs/code-navigation/auto-indexing.mdx @@ -35,7 +35,7 @@ This step will control the scheduling of indexing jobs which are made available ### Tune the index scheduler -The frequency of index job scheduling can be tuned via the following environment variables read by `worker` service containers running the [`codeintel-auto-indexing`](/self-hosted/workers#codeintel-auto-indexing) task. +The frequency of index job scheduling can be tuned via the following environment variables read by `worker` service containers running the [`codeintel-auto-indexing`](/self-hosted/workers#codeintel-autoindexing-scheduler) task. - **`PRECISE_CODE_INTEL_AUTO_INDEXING_TASK_INTERVAL`**: The time to run periodic codeintel auto-indexing tasks. The default is every 2 minutes - **`PRECISE_CODE_INTEL_AUTO_INDEXING_REPOSITORY_PROCESS_DELAY`**: The minimum time that the same repository can be considered for auto-index scheduling. The default is every 24 hours diff --git a/docs/code-navigation/envvars.mdx b/docs/code-navigation/envvars.mdx index b74815c5f..cbd526055 100644 --- a/docs/code-navigation/envvars.mdx +++ b/docs/code-navigation/envvars.mdx @@ -30,7 +30,7 @@ The following are variables are read from the `worker` service to control code g ### `codeintel-commitgraph` -The following variables influence the behavior of the [`codeintel-commitgraph` worker task](/self-hosted/workers#codeintel-commitgraph). +The following variables influence the behavior of the [`codeintel-commitgraph` worker task](/self-hosted/workers#codeintel-commitgraph-updater). | **Name** | **Default** | **Description** | | ------------------------------------------------------ | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------- | @@ -40,7 +40,7 @@ The following variables influence the behavior of the [`codeintel-commitgraph` w ### `codeintel-auto-indexing` -The following variables influence the behavior of the [`codeintel-auto-indexing` worker task](/self-hosted/workers#codeintel-auto-indexing). +The following variables influence the behavior of the [`codeintel-auto-indexing` worker task](/self-hosted/workers#codeintel-autoindexing-scheduler). | **Name** | **Default** | **Description** | | --------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------- | @@ -59,7 +59,7 @@ The following settings should be the same for the [`frontend`](#frontend) servic ### `codeintel-janitor` -The following variables influence the behavior of the [`codeintel-janitor` worker task](/self-hosted/workers#codeintel-janitor). +The following variables influence the behavior of the [`codeintel-janitor` worker task](/self-hosted/workers#codeintel-upload-janitor). | **Name** | **Default** | **Description** | | | ------------------------------------------------------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --- | diff --git a/docs/code-navigation/explanations/uploads.mdx b/docs/code-navigation/explanations/uploads.mdx index 8436af333..b31e0ab8b 100644 --- a/docs/code-navigation/explanations/uploads.mdx +++ b/docs/code-navigation/explanations/uploads.mdx @@ -40,7 +40,7 @@ Sourcegraph maintains a mapping from a commit of a repository to the set of uplo Upon a state change in an upload, we flag the repository as needing an update. Subsequently, the worker service updates the commit graph and asynchronously clears the flag for that repository. -When an upload changes state, the repository is flagged as requiring an update status. Then the [`worker` service](/self-hosted/workers#codeintel-commitgraph) +When an upload changes state, the repository is flagged as requiring an update status. Then the [`worker` service](/self-hosted/workers#codeintel-commitgraph-updater) will update the commit graph and unset the flag for that repository asynchronously. While this flag is set, the repository's commit graph is considered `stale`. This means there may be some upload records in a `COMPLETED` state that aren't yet used to resolve code navigation queries. diff --git a/docs/code-navigation/inference-configuration.mdx b/docs/code-navigation/inference-configuration.mdx index 92de99eaf..04fae0356 100644 --- a/docs/code-navigation/inference-configuration.mdx +++ b/docs/code-navigation/inference-configuration.mdx @@ -193,7 +193,7 @@ This auto-indexing-specific library defines the following four path pattern cons This library also defines the following two pattern collection constructors. -- `new_path_combine(patterns)` creates a pattern collection object (to be used with [recognizers](#sg-autoindex-recognizers)) from the given set of path `pattern`s. +- `new_path_combine(patterns)` creates a pattern collection object (to be used with [recognizers](#sgautoindexrecognizer)) from the given set of path `pattern`s. - Type: `((pattern | array[pattern])...) -> pattern` - `new_path_exclude(patterns)` creates a new _inverted_ pattern collection object. Paths matching these `pattern`s are filtered out from the set of matching filepaths given to a recognizer's `generate` function. - Type: `((pattern | array[pattern])...) -> pattern` diff --git a/docs/code-navigation/precise-code-navigation.mdx b/docs/code-navigation/precise-code-navigation.mdx index 5400a4308..2d3a67c75 100644 --- a/docs/code-navigation/precise-code-navigation.mdx +++ b/docs/code-navigation/precise-code-navigation.mdx @@ -31,8 +31,8 @@ Precise code navigation relies on the open source [SCIP Code Intelligence Protoc 1. **Manual indexing**. Index a repository and upload it to your Sourcegraph instance: - - [Index a Go repository](/code-navigation/how-to/index-a-go-repository#manual-indexing) - - [Index a TypeScript or JavaScript repository](/code-navigation/how-to/index-a-typescript-and-javascript-repository#manual-indexing) + - [Index a Go repository](/code-navigation/how-to/index-a-go-repository#indexing) + - [Index a TypeScript or JavaScript repository](/code-navigation/how-to/index-a-typescript-and-javascript-repository#one-off-indexing-using-scip-typescript-locally) - [Index a Java, Scala, or Kotlin repository](https://sourcegraph.github.io/scip-java/docs/getting-started.html) - [Index a Python repository](https://sourcegraph.com/github.com/sourcegraph/scip-python) - [Index a Ruby repository](https://sourcegraph.com/github.com/sourcegraph/scip-ruby) diff --git a/docs/code-search/features.mdx b/docs/code-search/features.mdx index b86a4d8f3..fc119ae8d 100644 --- a/docs/code-search/features.mdx +++ b/docs/code-search/features.mdx @@ -43,7 +43,7 @@ See the [query syntax](/code-search/queries#filters-diff-and-commit-searches-onl Searching over commit messages is supported in Sourcegraph by adding `type:commit` to your search query. Separately, you can also use the `message:"any string"` parameter to filter `type:diff` searches for a given commit message. Commit message searches can narrowed down further with filters such as author and time. -See our [query syntax](/code-search/queries#diff-and-commit-searches-only) documentation for a comprehensive list of supported parameters. +See our [query syntax](/code-search/queries#filters-diff-and-commit-searches-only) documentation for a comprehensive list of supported parameters. ## Symbol search diff --git a/docs/code-search/how-to/opengrok.mdx b/docs/code-search/how-to/opengrok.mdx index 5ec10f588..89f727b4d 100644 --- a/docs/code-search/how-to/opengrok.mdx +++ b/docs/code-search/how-to/opengrok.mdx @@ -15,9 +15,9 @@ Oracle OpenGrok provides wildcard support for searches. For example, to find all strings beginning with `foo`, you can use the wildcard search `foo*`. Similarly, OpenGrok provides the `?` operator for single character wildcards. -Sourcegraph, [provides full regular expression search](/code-search/queries#regexp-search), with support for the [RE2 syntax](https://golang.org/s/re2syntax). The same search above would take the form `foo.*` (or in this case, just `foo`, since Sourcegraph automatically supports partial matches). Much more powerful regexp expressions are available. +Sourcegraph, [provides full regular expression search](/code-search/queries#regular-expression-search), with support for the [RE2 syntax](https://golang.org/s/re2syntax). The same search above would take the form `foo.*` (or in this case, just `foo`, since Sourcegraph automatically supports partial matches). Much more powerful regexp expressions are available. -(Note that Sourcegraph also provides a [literal search mode](/code-search/queries#Literal-search-default) by default, in which there's no need to escape special characters. This simplifies searches such as `foo(`, which would result in an error in regexp mode.) +(Note that Sourcegraph also provides a [literal search mode](/code-search/queries#keyword-search-default) by default, in which there's no need to escape special characters. This simplifies searches such as `foo(`, which would result in an error in regexp mode.) ### Selecting repositories and branches @@ -29,7 +29,7 @@ Sourcegraph provides a search filter (`repo:`) that supports regexp and partial | - To search in a distinct list of repositories, you can use a ` |` character as a regexp OR operator: `pattern repo:github.com/org/repository1 | github.com/org/repository2`. | | - Note this query could be simplified further using more advanced regexp matching if the two repos share part of their names, such as: `pattern repo:github.com/org/repository(1 | 2)`. | | -Sourcegraph also allows site admins to create pre-defined repository groupings, using [version contexts](/code-search/features#version-contexts-experimental). +Sourcegraph also allows site admins to create pre-defined repository groupings, using [search contexts](/code-search/features#search-contexts). ### Searching in non-master (unindexed) branches, tags, and commits @@ -43,9 +43,9 @@ Sourcegraph also provides the ability to search on multiple Git revisions in a s Oracle OpenGrok doesn't index most single-character strings (such as for special characters like `{`, `}`, `[`, `]`, `+`, `-`, and more), and non-alpha-numeric characters generally. -Sourcegraph indexes all characters, and can search for strings of any length. Using the default [literal search mode](/code-search/queries#Literal-search-default), any search (including those with special characters like `foo.bar`, `try {`, `i++`, `i-=1`, `foo->bar`, and more), will all be searchable without special handling. Using [regexp mode](/code-search/queries#regexp-search) would require escaping special characters. +Sourcegraph indexes all characters, and can search for strings of any length. Using the default [literal search mode](/code-search/queries#keyword-search-default), any search (including those with special characters like `foo.bar`, `try {`, `i++`, `i-=1`, `foo->bar`, and more), will all be searchable without special handling. Using [regexp mode](/code-search/queries#regular-expression-search) would require escaping special characters. -The only exceptions are colon characters, which are by default used for specifying a [search keyword](#search-keywords) on Sourcegraph. Any search containing colons can be done using the `content:` keyword (for example, `content:"foo::bar"`) to explicitly mark it as the search string. +The only exceptions are colon characters, which are by default used for specifying a [search keyword](#search-filters) on Sourcegraph. Any search containing colons can be done using the `content:` keyword (for example, `content:"foo::bar"`) to explicitly mark it as the search string. ### Boolean operators diff --git a/docs/code-search/queries/index.mdx b/docs/code-search/queries/index.mdx index b8dfa9f3b..a2612276a 100644 --- a/docs/code-search/queries/index.mdx +++ b/docs/code-search/queries/index.mdx @@ -13,7 +13,7 @@ For a graphical view of Sourcegraph's query syntax, see the [search language ref ## Search patterns -This section documents the search pattern syntax in Sourcegraph. To match file content, you need to specify a search pattern. Search patterns are optional when searching [commits](#filters-diff-and-commit-searches-only), [filenames](#filename-search), or [repository names](#repository-name-search). +This section documents the search pattern syntax in Sourcegraph. To match file content, you need to specify a search pattern. Search patterns are optional when searching [commits](#filters-diff-and-commit-searches-only), [filenames](#filename-search), or [repository names](#repository-names). ### Keyword search (default) @@ -70,7 +70,7 @@ The following filters can be used on all searches (using [RE2 syntax](https://go | **rev:revision-pattern**
_alias: revision_ | Search a revision instead of the default branch. `rev:` can only be used in conjunction with `repo:` and may not be used more than once. See our [revision syntax](#repository-revisions) documentation to learn more | [`repo:sourcegraph/sourcegraph rev:v3.14.0 mux`](https://sourcegraph.com/search?q=repo:sourcegraph/sourcegraph+rev:v3.14.0+mux&patternType=keyword) | | **file:regexp-pattern**
_alias: f_ | Only include file results whose full path matches the regexp. The regexp is unanchored by default: to match against the entire path, use regexp anchors like `^README.md$`. | [`file:\.js$ httptest`](https://sourcegraph.com/search?q=file:%5C.js%24+httptest)
[`file:internal/ httptest`](https://sourcegraph.com/search?q=file:internal/+httptest) | | **-file:regexp-pattern**
_alias: -f_ | Exclude file results whose full path matches the regexp. The regexp is unanchored by default: to match against the entire path, use regexp anchors like `^README.md$`. | [`file:\.js$ -file:test http`](https://sourcegraph.com/search?q=file:%5C.js%24+-file:test+http) | -| **content:"pattern"** | Set the search pattern with a dedicated parameter. Useful when searching literally for a string that may conflict with the [search pattern syntax](#search-pattern-syntax). In between the quotes, the `\` character will need to be escaped (`\\` to evaluate for `\`) | [`repo:sourcegraph content:"repo:sourcegraph"`](https://sourcegraph.com/search?q=repo:sourcegraph+content:"repo:sourcegraph"&patternType=keyword) | +| **content:"pattern"** | Set the search pattern with a dedicated parameter. Useful when searching literally for a string that may conflict with the [search pattern syntax](#search-patterns). In between the quotes, the `\` character will need to be escaped (`\\` to evaluate for `\`) | [`repo:sourcegraph content:"repo:sourcegraph"`](https://sourcegraph.com/search?q=repo:sourcegraph+content:"repo:sourcegraph"&patternType=keyword) | | **-content:"pattern"** | Exclude results from files whose content matches the pattern. | [`file:Dockerfile alpine -content:alpine:latest`](https://sourcegraph.com/search?q=file:Dockerfile+alpine+-content:alpine:latest&patternType=keyword) | | **select:_result-type_**
**select:repo**
**select:commit.diff.added**
**select:commit.diff.removed**
**select:file**
**select:content**
**select:symbol._symbol-type_**
**select:file.owners** _(Experimental)_ | Shows only query results for a given type. For example, `select:repo` displays only distinct repository paths from search results, and `select:commit.diff.added` shows only added code matching the search. See [language definition](/code-search/queries/language#select) for full list of possible values | [`fmt.Errorf select:repo`](https://sourcegraph.com/search?q=fmt.Errorf+select:repo&patternType=keyword) | | **language:language-name**
_alias: lang, l_ | Only include results from files in the specified programming language | [`language:typescript encoding`](https://sourcegraph.com/search?q=language:typescript+encoding) | diff --git a/docs/code-search/types/symbol.mdx b/docs/code-search/types/symbol.mdx index 18d9cf71f..2e857e23f 100644 --- a/docs/code-search/types/symbol.mdx +++ b/docs/code-search/types/symbol.mdx @@ -22,7 +22,7 @@ The extracted `ctags` symbols are also used for the symbol sidebar, which catego Here is the query path for symbol searches: -- **Zoekt**: if [indexed search](/admin/search#indexed-search) is enabled and the search is for the tip commit of an indexed branch, then Zoekt will service the query and it should respond quickly. Zoekt indexes the default branch (usually `master` or `main`) and can be configured for [multi-branch indexing](/code-search/features#multi-branch-indexing-experimental). The high commit frequency of monorepos reduces the likelihood that Zoekt will be able to respond to symbol searches. Zoekt **eagerly** indexes by listening to repository updates, whereas the searcher service **lazily** indexes the commit being searched. +- **Zoekt**: if [indexed search](/admin/search#indexed-search) is enabled and the search is for the tip commit of an indexed branch, then Zoekt will service the query and it should respond quickly. Zoekt indexes the default branch (usually `master` or `main`) and can be configured for [multi-branch indexing](/code-search/features#multi-branch-indexing). The high commit frequency of monorepos reduces the likelihood that Zoekt will be able to respond to symbol searches. Zoekt **eagerly** indexes by listening to repository updates, whereas the searcher service **lazily** indexes the commit being searched. - **Searcher service with Rockskip enabled**: if [Rockskip](/code-navigation/rockskip) is enabled, it'll search for symbols stored in Postgres. After initial indexing, queries should be resolved quickly. - **Searcher service with an index for the commit**: if the searcher service has already indexed this commit (i.e. someone has visited the commit before) then the query should be resolved quickly. Indexes are deleted in LRU fashion to remain under the configured maximum disk usage which [defaults to 100GB](/code-navigation/search-based-code-navigation#what-configuration-settings-can-i-apply). - **Searcher service with an index for a different commit**: if the searcher service has already indexed a **different** commit in the same repository, then it will make a copy of the previous index on disk then run [ctags](https://github.com/universal-ctags/ctags#readme) on the files that changed between the two commits and update the symbols in the new index. This process takes roughly 20 seconds on a monorepo with 40M LOC and 400K files.