Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
*
* <p>A single greeting step configured with TWO plugins registered together, in order: first a faulty plugin whose
* every exercised hook (invocation-start, operation-start, attempt-start, attempt-end, operation-end, invocation-end)
* logs a record then throws, then a healthy plugin that logs the corresponding six records normally. The SDK's {@code
* PluginRunner} isolates each plugin at every hook boundary (swallows the faulty plugin's exceptions), so the healthy
* plugin still receives every hook and the execution result/history are identical to running without the faulty
* logs a record then throws, then a healthy plugin that logs the corresponding six records normally. The SDK's
* {@code PluginRunner} isolates each plugin at every hook boundary (swallows the faulty plugin's exceptions), so the
* healthy plugin still receives every hook and the execution result/history are identical to running without the faulty
* plugin. Attempt boundaries are the real user-function hooks ({@code onUserFunctionStart}/{@code onUserFunctionEnd},
* filtered to step attempts); the healthy attempt-end reports the SDK's real success/failure outcome.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
*
* <p>A parallel operation named "parallel" with two branches (max-concurrency 1, so they run sequentially in index
* order); each branch returns a constant directly. The plugin, filtering to parallel-branch operations, logs fn-start
* and fn-end (with outcome) from the real user-function hooks, carrying the branch operation id and the parallel
* parent id. These hooks run on the branch's own thread, so start-before-end order per branch is deterministic.
* and fn-end (with outcome) from the real user-function hooks, carrying the branch operation id and the parallel parent
* id. These hooks run on the branch's own thread, so start-before-end order per branch is deterministic.
*/
@SuppressWarnings("deprecation")
public class PluginParallelBranchHooks extends DurableHandler<Object, List<String>> {
Expand Down
8 changes: 4 additions & 4 deletions conformance-tests/src/main/java/plugin/PluginReplayFlags.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
/**
* 10-13: Non-terminal operations replay with replay=true; terminal operations are not re-emitted.
*
* <p>Two sequential steps. Step A succeeds on its first attempt (terminal before the retry invocation). Step B fails
* on its first attempt and succeeds on the second, using the SDK's built-in exponential-backoff retry strategy
* (~1s delay). The plugin, filtering to step-type operations, logs operation-start with the SDK's is-replayed
* indicator ({@code OperationInfo#isReplay()}) and operation-end with the terminal status.
* <p>Two sequential steps. Step A succeeds on its first attempt (terminal before the retry invocation). Step B fails on
* its first attempt and succeeds on the second, using the SDK's built-in exponential-backoff retry strategy (~1s
* delay). The plugin, filtering to step-type operations, logs operation-start with the SDK's is-replayed indicator
* ({@code OperationInfo#isReplay()}) and operation-end with the terminal status.
*/
@SuppressWarnings("deprecation")
public class PluginReplayFlags extends DurableHandler<Object, String> {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
/**
* 10-15: Attempt hooks fire for every attempt until exhaustion, then operation-end reports FAILED.
*
* <p>A single step that always throws, configured with the SDK's built-in exponential-backoff retry strategy allowing
* 2 total attempts (1 initial + 1 retry, ~1s delay). The plugin, filtering to step-type operations, logs attempt-start
* <p>A single step that always throws, configured with the SDK's built-in exponential-backoff retry strategy allowing 2
* total attempts (1 initial + 1 retry, ~1s delay). The plugin, filtering to step-type operations, logs attempt-start
* and attempt-end (with outcome) from the real user-function hooks (which carry the 1-based attempt number) and
* operation-end when the step reaches its terminal FAILED status.
*/
Expand Down
13 changes: 8 additions & 5 deletions conformance-tests/src/main/java/plugin/PluginSupport.java
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,19 @@
/**
* Shared helpers for the plugin conformance handlers (requirements 10-8..10-18).
*
* <p>Every plugin captures the durable execution ARN from the invocation-start hook's info parameter and stamps it as
* a top-level {@code durableExecutionArn} field on every stdout JSON record, so the runner's execution-scoped
* CloudWatch filter ({@code $.durableExecutionArn = "<arn>"}) locates the records. These helpers only format that
* field and classify operation types reported by the real SDK; no behavior is fabricated here.
* <p>Every plugin captures the durable execution ARN from the invocation-start hook's info parameter and stamps it as a
* top-level {@code durableExecutionArn} field on every stdout JSON record, so the runner's execution-scoped CloudWatch
* filter ({@code $.durableExecutionArn = "<arn>"}) locates the records. These helpers only format that field and
* classify operation types reported by the real SDK; no behavior is fabricated here.
*/
final class PluginSupport {

private PluginSupport() {}

/** Operation type token for step operations as reported by {@code OperationInfo#type()} (AWS SDK {@code OperationType}). */
/**
* Operation type token for step operations as reported by {@code OperationInfo#type()} (AWS SDK
* {@code OperationType}).
*/
static boolean isStep(String type) {
return "STEP".equals(type);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
* 10-10: Plugin operation-start and operation-end hooks fire for wait-type operations.
*
* <p>A single 2-second wait. The plugin, filtering to wait-type operations, logs operation-start when the wait's
* STARTED checkpoint is observed and operation-end with the terminal status. The type token is normalized to
* upper-case (WAIT).
* STARTED checkpoint is observed and operation-end with the terminal status. The type token is normalized to upper-case
* (WAIT).
*/
@SuppressWarnings("deprecation")
public class PluginWaitOperationHooks extends DurableHandler<Object, String> {
Expand Down
2 changes: 2 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ mvn test -Dtest=CloudBasedIntegrationTest \
| [WaitForConditionExample](src/main/java/software/amazon/lambda/durable/examples/wait/WaitForConditionExample.java) | Poll a condition until met with `waitForCondition()` |
| [OtelExample](src/main/java/software/amazon/lambda/durable/examples/general/OtelExample.java) | OpenTelemetry instrumentation with logging span export |
| [OtelXRayStepExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayStepExample.java) | Export step spans to X-Ray through the ADOT Lambda Layer |
| [OtelXRayExecutionStepExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayExecutionStepExample.java) | Export spans to X-Ray with `new ExecutionOtelPlugin()` using workflow-rooted trace structure |
| [OtelXRayExecutionWaitExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayExecutionWaitExample.java) | Trace a step-wait-step workflow with ExecutionOtelPlugin across Lambda invocations |
| [OtelXRayWaitExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayWaitExample.java) | Trace a step-wait-step workflow across Lambda invocations |
| [OtelXRayMapExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayMapExample.java) | Trace concurrent map operations and item steps in X-Ray |
| [OtelXRayParallelExample](src/main/java/software/amazon/lambda/durable/examples/otel/OtelXRayParallelExample.java) | Trace parallel branches and branch steps in X-Ray |
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.examples.otel;

import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.DurableContext;
import software.amazon.lambda.durable.DurableHandler;
import software.amazon.lambda.durable.examples.ExampleTemplate;
import software.amazon.lambda.durable.examples.types.GreetingRequest;
import software.amazon.lambda.durable.otel.ExecutionOtelPlugin;

/**
* OTel + X-Ray example using the ExecutionOtelPlugin with the no-arg constructor.
*
* <p>{@link ExecutionOtelPlugin#ExecutionOtelPlugin()} uses the global provider initialized by the ADOT Java agent. The
* ExecutionOtelPlugin renders the Workflow span as the trace root with operations as siblings of the invocation span.
*/
@ExampleTemplate(tracing = true, javaAgent = true)
public class OtelXRayExecutionStepExample extends DurableHandler<GreetingRequest, String> {

@Override
protected DurableConfig createConfiguration() {
return DurableConfig.builder().withPlugins(new ExecutionOtelPlugin()).build();
}

@Override
public String handleRequest(GreetingRequest input, DurableContext context) {
context.getLogger().info("Starting OTel X-Ray execution view example for {}", input.getName());

var greeting = context.step("exec-create-greeting", String.class, stepCtx -> "Hello, " + input.getName());

var result = context.step("exec-transform", String.class, stepCtx -> greeting.toUpperCase() + "!");

context.getLogger().info("OTel X-Ray execution view example complete: {}", result);
return result;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
package software.amazon.lambda.durable.examples.otel;

import java.time.Duration;
import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.DurableContext;
import software.amazon.lambda.durable.DurableHandler;
import software.amazon.lambda.durable.examples.ExampleTemplate;
import software.amazon.lambda.durable.examples.types.GreetingRequest;
import software.amazon.lambda.durable.otel.ExecutionOtelPlugin;

/**
* OTel + X-Ray example using ExecutionOtelPlugin with a step → wait → step pattern.
*
* <p>Exercises the multi-invocation tracing scenario with the workflow-rooted trace structure. The Workflow span is
* only exported on the terminal invocation, producing a clean single-execution trace.
*/
@ExampleTemplate(tracing = true, javaAgent = true)
public class OtelXRayExecutionWaitExample extends DurableHandler<GreetingRequest, String> {

@Override
protected DurableConfig createConfiguration() {
return DurableConfig.builder().withPlugins(new ExecutionOtelPlugin()).build();
}

@Override
public String handleRequest(GreetingRequest input, DurableContext context) {
context.getLogger().info("Starting OTel X-Ray execution view wait example for {}", input.getName());

var before = context.step("exec-before-wait", String.class, stepCtx -> "Prepared: " + input.getName());

context.wait("exec-pause", Duration.ofSeconds(5));

var after = context.step("exec-after-wait", String.class, stepCtx -> before + " | Resumed and completed");

context.getLogger().info("OTel X-Ray execution view wait example complete: {}", after);
return after;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -127,35 +127,8 @@ void simpleSteps_producesUnifiedTraceInXRay() throws Exception {
// 2. Wait for X-Ray ingestion
Thread.sleep(XRAY_INGESTION_DELAY.toMillis());

// 3. Query X-Ray for the trace
var traces = queryTracesWithRetry(startTime, Instant.now(), "otel-xray-step-example");

assertFalse(traces.isEmpty(), "Expected at least one trace in X-Ray after execution");

// Get full trace details (batch in groups of 5 — X-Ray API limit)
var traceIds = traces.stream().map(TraceSummary::id).toList();
var allTraces = new java.util.ArrayList<software.amazon.awssdk.services.xray.model.Trace>();
for (int i = 0; i < traceIds.size(); i += 5) {
var batch = traceIds.subList(i, Math.min(i + 5, traceIds.size()));
var batchResult = xrayClient.batchGetTraces(
BatchGetTracesRequest.builder().traceIds(batch).build());
allTraces.addAll(batchResult.traces());
}

// Find the trace that contains our durable spans
var durableTrace = allTraces.stream()
.filter(trace -> trace.segments().stream().anyMatch(seg -> segmentContains(seg, "create-greeting")))
.findFirst()
.orElse(null);

assertNotNull(
durableTrace,
"Expected to find a trace with create-greeting segment. " + "Found " + traces.size()
+ " traces in the time window. Segment names: "
+ allTraces.stream()
.flatMap(t -> t.segments().stream())
.map(seg -> getSegmentName(seg))
.collect(Collectors.joining(", ")));
// 3. Query X-Ray for the trace, retrying until durable spans appear
var durableTrace = queryTraceWithDurableSpans(startTime, "otel-xray-step-example", "create-greeting");

// 5. Verify span structure
var segmentDocuments =
Expand Down Expand Up @@ -198,31 +171,8 @@ void waitAndResume_producesUnifiedTraceAcrossInvocations() throws Exception {
// 2. Wait for X-Ray ingestion (extra time since multi-invocation takes longer)
Thread.sleep(XRAY_INGESTION_DELAY.plus(Duration.ofSeconds(5)).toMillis());

// 3. Query X-Ray for the trace
var traces = queryTracesWithRetry(startTime, Instant.now(), "otel-xray-wait-example");

assertFalse(traces.isEmpty(), "Expected at least one trace in X-Ray after multi-invocation execution");

// Get full trace details (batch in groups of 5 — X-Ray API limit)
var traceIds = traces.stream().map(TraceSummary::id).toList();
var allTraces = new java.util.ArrayList<software.amazon.awssdk.services.xray.model.Trace>();
for (int i = 0; i < traceIds.size(); i += 5) {
var batch = traceIds.subList(i, Math.min(i + 5, traceIds.size()));
var batchResult = xrayClient.batchGetTraces(
BatchGetTracesRequest.builder().traceIds(batch).build());
allTraces.addAll(batchResult.traces());
}

// Find the trace containing our durable spans
var durableTrace = allTraces.stream()
.filter(trace -> trace.segments().stream().anyMatch(seg -> segmentContains(seg, "before-wait")))
.findFirst()
.orElse(null);

assertNotNull(
durableTrace,
"Expected to find a trace with before-wait segment. " + "Found " + traces.size()
+ " traces in the time window.");
// 3. Query X-Ray for the trace, retrying until durable spans appear
var durableTrace = queryTraceWithDurableSpans(startTime, "otel-xray-wait-example", "before-wait");

// 4. Verify multi-invocation trace structure
var segmentDocuments =
Expand Down Expand Up @@ -330,4 +280,50 @@ private static int countOccurrences(String text, String substring) {
private static String summarizeSegments(List<String> segmentDocuments) {
return extractSegmentNames(segmentDocuments).stream().collect(Collectors.joining(", ", "[", "]"));
}

/**
* Queries X-Ray for a trace containing durable spans, retrying until the expected span appears or timeout is
* reached. Handles eventual consistency where the trace exists but OTLP-exported spans haven't been ingested yet.
*/
private software.amazon.awssdk.services.xray.model.Trace queryTraceWithDurableSpans(
Instant startTime, String functionName, String expectedSpanName) throws InterruptedException {
var maxAttempts = 5;
for (int attempt = 1; attempt <= maxAttempts; attempt++) {
var traces = queryTracesWithRetry(startTime, Instant.now(), functionName);
if (traces.isEmpty()) {
fail("Expected at least one trace in X-Ray after execution of " + functionName);
}

var traceIds = traces.stream().map(TraceSummary::id).toList();
var allTraces = new java.util.ArrayList<software.amazon.awssdk.services.xray.model.Trace>();
for (int i = 0; i < traceIds.size(); i += 5) {
var batch = traceIds.subList(i, Math.min(i + 5, traceIds.size()));
var batchResult = xrayClient.batchGetTraces(
BatchGetTracesRequest.builder().traceIds(batch).build());
allTraces.addAll(batchResult.traces());
}

var durableTrace = allTraces.stream()
.filter(trace -> trace.segments().stream().anyMatch(seg -> segmentContains(seg, expectedSpanName)))
.findFirst()
.orElse(null);

if (durableTrace != null) {
return durableTrace;
}

if (attempt < maxAttempts) {
var segmentNames = allTraces.stream()
.flatMap(t -> t.segments().stream())
.map(CloudBasedOtelIntegrationTest::getSegmentName)
.collect(Collectors.joining(", "));
System.out.println("⏳ Trace found but missing '" + expectedSpanName + "' span (attempt " + attempt + "/"
+ maxAttempts + "). Current segments: " + segmentNames + ". Retrying in 10s...");
Thread.sleep(XRAY_RETRY_DELAY.toMillis());
}
}

fail("Expected to find a trace with '" + expectedSpanName + "' span after " + maxAttempts + " attempts");
return null; // unreachable
}
}
Loading
Loading