Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion docs/guides/executables.md
Original file line number Diff line number Diff line change
Expand Up @@ -602,13 +602,14 @@ All imported executables are automatically tagged with `generated` and their fil

#### **Script Files**

Script files (`.sh`, `.bat`, `.cmd`, `.ps1`) are imported as single executables with the script's filename as the name and `exec` as the default verb. Each script type is executed with its native interpreter:
Script files (`.sh`, `.bat`, `.cmd`, `.ps1`, `.py`) are imported as single executables with the script's filename as the name and `exec` as the default verb. Each script type is executed with its native interpreter:

| Extension | Interpreter | Platforms |
|-----------|------------|-----------|
| `.sh` | Built-in POSIX shell | All (cross-platform) |
| `.bat`, `.cmd` | `cmd.exe /C` | Windows |
| `.ps1` | `pwsh` or `powershell` | All (requires PowerShell) |
| `.py` | Resolved Python (see [Running Python](#running-python)) | All (requires Python) |

You can use special comments to override executable metadata. The comment syntax depends on the script type:

Expand Down Expand Up @@ -644,6 +645,19 @@ kubectl apply -f k8s\
Write-Host "Deploying to production..."
kubectl apply -f k8s/
```

```python [Python (.py)]
#!/usr/bin/env python3
# f:name=production f:verb=deploy
# f:description="Deploy to production environment"
# f:tag=production f:tag=critical
# f:timeout=10m

import subprocess

print("Deploying to production...")
subprocess.run(["kubectl", "apply", "-f", "k8s/"], check=True)
```
:::

See the [generated configuration reference](generated-config.md) for more details.
Expand Down
28 changes: 26 additions & 2 deletions docs/guides/generated-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ title: Imported Executables Config Reference
# Imported Executables Config Reference

flow can automatically generate executables from scripts and Makefiles using special comments.
Supported script types include shell scripts (`.sh`), batch files (`.bat`, `.cmd`), and PowerShell scripts (`.ps1`).
Supported script types include shell scripts (`.sh`), batch files (`.bat`, `.cmd`), PowerShell scripts (`.ps1`), and Python scripts (`.py`).
flow parses these comments during workspace synchronization and creates executable definitions that can be run
like any other flow executable. See [Importing Executables](executables.md#importing-executables) for more details.

> [!NOTE] The configuration comments must be at the top of the script file or right above the Makefile target definition.
> - **Shell / PowerShell:** Use `# ` as the comment prefix (e.g., `# f:name=deploy`)
> - **Shell / PowerShell / Python:** Use `# ` as the comment prefix (e.g., `# f:name=deploy`)
> - **Batch files:** Use `REM ` or `:: ` as the comment prefix (e.g., `REM f:name=deploy`)

## Supported Fields
Expand Down Expand Up @@ -54,6 +54,16 @@ echo Deploying to %ENV_NAME% with token: %API_TOKEN%...

Write-Host "Deploying to $env:ENV_NAME with token: $($env:API_TOKEN.Substring(0,8))..."
```

```python [Python (.py)]
#!/usr/bin/env python3
# f:name=deploy-with-secrets f:verb=deploy
# f:params=secretRef:api-key:API_TOKEN|prompt:Environment?:ENV_NAME|text:production:DEFAULT_ENV

import os

print(f"Deploying to {os.environ['ENV_NAME']} with token: {os.environ['API_TOKEN'][:8]}...")
```
:::

**Parameter Types:**
Expand Down Expand Up @@ -100,6 +110,20 @@ if ($env:DRY_RUN -eq "true") {
Write-Host "Building version $env:VERSION"
}
```

```python [Python (.py)]
#!/usr/bin/env python3
# f:name=build-app f:verb=build
# f:args=flag:dry-run:DRY_RUN|pos:1:VERSION|flag:verbose:VERBOSE

import os

version = os.environ["VERSION"]
if os.environ.get("DRY_RUN") == "true":
print(f"DRY RUN: Would build version {version}")
else:
print(f"Building version {version}")
```
:::

**Argument Types:**
Expand Down
2 changes: 2 additions & 0 deletions internal/fileparser/fileparser.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ func parseScriptFile(wsPath, fn, expandedFile string) (executable.ExecutableList
exec, err = ExecutablesFromBatFile(wsPath, expandedFile)
case ".ps1":
exec, err = ExecutablesFromPs1File(wsPath, expandedFile)
case ".py":
exec, err = ExecutablesFromPyFile(wsPath, expandedFile)
default:
logger.Log().Warn("unable to import executables - unsupported file type", "file", fn)
return nil, nil
Expand Down
8 changes: 8 additions & 0 deletions internal/fileparser/fileparser_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ var _ = Describe("ExecutablesFromImports", func() {
Expect(result[0].Exec.File).To(Equal("simple.ps1"))
})

It("should return executables from py file imports", func() {
flowFile.Imports = append(flowFile.Imports, "simple.py")
result, err := fileparser.ExecutablesFromImports("ws", flowFile)
Expect(err).NotTo(HaveOccurred())
Expect(result).To(HaveLen(1))
Expect(result[0].Exec.File).To(Equal("simple.py"))
})

It("should log a warning for invalid file type", func() {
mockLogger.EXPECT().Warn(gomock.Any(), "file", "invalidfile").AnyTimes()
flowFile.Imports = append(flowFile.Imports, "invalidfile")
Expand Down
42 changes: 42 additions & 0 deletions internal/fileparser/py_file_parser.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package fileparser

import (
"os"
"path/filepath"

"github.com/flowexec/flow/v2/types/executable"
)

func ExecutablesFromPyFile(wsPath, filePath string) (*executable.Executable, error) {
fn := filepath.Base(filePath)
verb := InferVerb(fn)
execName := NormalizeName(fn, verb.String())
dir := executable.Directory(shortenWsPath(wsPath, filepath.Dir(filePath)))
exec := &executable.Executable{
Verb: verb,
Name: execName,
Exec: &executable.ExecExecutableType{
Dir: dir,
File: filepath.Base(filePath),
},
}

fileBytes, err := os.ReadFile(filepath.Clean(filePath))
if err != nil {
return nil, err
}

// Python uses # for line comments, same as shell scripts, so the metadata
// comment syntax is identical. The interpreter is left unset: the .py
// extension already implies it, and setting it would only add noise.
cfg, err := ExtractExecConfig(string(fileBytes), "# ")
if err != nil {
return nil, err
}
if err := ApplyExecConfig(exec, cfg); err != nil {
return nil, err
}

exec.Tags = append(exec.Tags, generatedTag)
return exec, nil
}
75 changes: 75 additions & 0 deletions internal/fileparser/py_file_parser_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package fileparser_test

import (
"time"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"

"github.com/flowexec/flow/v2/internal/fileparser"
"github.com/flowexec/flow/v2/types/executable"
)

var _ = Describe("ExecutablesFromPyFile", func() {
const wsPath = "testdata"

It("should parse a simple py file", func() {
exec, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/simple.py")
Expect(err).NotTo(HaveOccurred())
Expect(exec).NotTo(BeNil())
Expect(exec.Name).To(Equal("hello"))
Expect(exec.Verb).To(Equal(executable.VerbShow))
Expect(exec.Exec).NotTo(BeNil())
Expect(exec.Exec.File).To(Equal("simple.py"))
Expect(exec.Exec.Dir).To(Equal(executable.Directory("//")))
Expect(exec.Tags).To(ContainElement("generated"))
})

It("should leave the interpreter unset, since .py already implies it", func() {
exec, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/simple.py")
Expect(err).NotTo(HaveOccurred())
Expect(exec.Exec.InterpreterIsSet()).To(BeFalse())
// The extension is what routes it, so the executable still runs as python.
Expect(exec.Exec.InterpreterForFile()).To(Equal(executable.InterpreterPython))
})

It("should parse a complex py file with all metadata", func() {
exec, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/complex.py")
Expect(err).NotTo(HaveOccurred())
Expect(exec).NotTo(BeNil())
Expect(exec.Name).To(Equal("deploy"))
Expect(exec.Verb).To(Equal(executable.VerbDeploy))
Expect(exec.Description).To(Equal("Deploy to production"))
Expect(exec.Tags).To(ContainElements("production", "critical", "generated"))
expectedTimeout := 10 * time.Minute
Expect(exec.Timeout).To(Equal(&expectedTimeout))
})

It("should parse params from a py file", func() {
exec, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/params.py")
Expect(err).NotTo(HaveOccurred())
Expect(exec).NotTo(BeNil())
Expect(exec.Name).To(Equal("test-params"))
Expect(exec.Exec.Params).To(HaveLen(3))
Expect(exec.Exec.Params[0].SecretRef).To(Equal("my-secret"))
Expect(exec.Exec.Params[0].EnvKey).To(Equal("SECRET_VAR"))
Expect(exec.Exec.Params[1].Prompt).To(Equal("Enter name"))
Expect(exec.Exec.Params[1].EnvKey).To(Equal("NAME_VAR"))
Expect(exec.Exec.Params[2].Text).To(Equal("default-value"))
Expect(exec.Exec.Params[2].EnvKey).To(Equal("DEFAULT_VAR"))
})

It("should read metadata that follows a shebang line", func() {
// A shebang is idiomatic in python scripts, so it must not shadow the
// metadata comments beneath it.
exec, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/shebang.py")
Expect(err).NotTo(HaveOccurred())
Expect(exec.Name).To(Equal("with-shebang"))
Expect(exec.Verb).To(Equal(executable.VerbRun))
})

It("should error on a missing file", func() {
_, err := fileparser.ExecutablesFromPyFile(wsPath, "testdata/does-not-exist.py")
Expect(err).To(HaveOccurred())
})
})
6 changes: 6 additions & 0 deletions internal/fileparser/testdata/complex.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# f:name=deploy f:verb=deploy
# f:description="Deploy to production"
# f:tag=production f:tag=critical
# f:timeout=10m

print("Deploying...")
8 changes: 8 additions & 0 deletions internal/fileparser/testdata/params.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# f:name=test-params f:verb=test
# f:params=secretRef:my-secret:SECRET_VAR|prompt:"Enter name":NAME_VAR|text:default-value:DEFAULT_VAR

import os

print("Secret:", os.environ["SECRET_VAR"])
print("Name:", os.environ["NAME_VAR"])
print("Default:", os.environ["DEFAULT_VAR"])
4 changes: 4 additions & 0 deletions internal/fileparser/testdata/shebang.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env python3
# f:name=with-shebang f:verb=run

print("shebang tolerated")
3 changes: 3 additions & 0 deletions internal/fileparser/testdata/simple.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# f:name=hello f:verb=show

print("Hello, world!")