Efficiently Automating Tests with AWS Cloud Processing

Picture the scene: You need to test a process that starts with a large file upload, gets picked up for processing, and ends with a set of results stored in a database.

It’s not easy to stub something like that. Add in data validations, mapping rules, enums, and it quickly becomes an end-to-end testing problem – whether you like it or not.

However, these processes can be slow. So, how do you ensure that your automated tests run efficiently and don’t become the bottleneck?

We recently tackled this exact scenario using AWS, and here's how we did it.

In our case, the process worked like this:

• A file was uploaded to an S3 bucket.

• That event triggered a Step Function.

• The Step Function validated, mapped, and processed the data.

• The results were loaded into a database.

We used Playwright for our test automation, and since we were working in TypeScript, we used the AWS SDK for JavaScript. (You can find the SDK docs here.)

Here’s a simplified example of how we set this up in code:

First, a separate AWS function:

// awsUtils.ts
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import {
  SFNClient,
  DescribeExecutionCommand,
  StartExecutionCommand,
} from "@aws-sdk/client-sfn";


const REGION = "eu-west-1";
const s3 = new S3Client({ region: REGION });
const sfn = new SFNClient({ region: REGION });

export async function uploadFileToS3(bucket: string, key: string, body: Buffer) {

  await s3.send(new PutObjectCommand({ Bucket: bucket, Key: key, Body: body }));

  console.log(`Uploaded file ${key} to bucket ${bucket}`);

}

export async function startStepFunctionAndWait(
  stateMachineArn: string,
  input: object,
  timeoutMs = 30000
): Promise<string> {
  const startCommand = new StartExecutionCommand({
    stateMachineArn,
    input: JSON.stringify(input),
  });

  const startResponse = await sfn.send(startCommand);
  const executionArn = startResponse.executionArn!;
  const pollIntervalMs = 3000;
  const startTime = Date.now();

  while (true) {
    const result = await sfn.send(
      new DescribeExecutionCommand({ executionArn })
    );

    if (result.status === "SUCCEEDED") return result.output || "";

    if (
      result.status === "FAILED" ||
      result.status === "TIMED_OUT" ||
      result.status === "ABORTED"
    ) {
      throw new Error(`Step function failed with status: ${result.status}`);
    }

    if (Date.now() - startTime > timeoutMs) {
      throw new Error("Step function polling timed out");
    }

    await new Promise((res) => setTimeout(res, pollIntervalMs));
  }
}

And then the actual Playwright Test:

// aws-step-function.spec.ts

import { test, expect } from "@playwright/test";
import fs from "fs";
import path from "path";
import { uploadFileToS3, startStepFunctionAndWait } from "./awsUtils";

test("AWS-based file processing flow works end-to-end", async () => {
	const bucketName = "your-bucket-name";
	const testFileKey = "test/test-file.csv";
	const stateMachineArn = "arn:aws:states:eu-west-1:123456789012:stateMachine:YourStateMachine";

// Load the test file into a buffer
	const filePath = path.resolve(__dirname, "test-data/test-file.csv");
	const fileBuffer = fs.readFileSync(filePath);
// Step 1: Upload file to S3
	await uploadFileToS3(bucketName, testFileKey, fileBuffer);

// Step 2: Start Step Function and wait
	const stepFunctionOutput = await startStepFunctionAndWait(stateMachineArn, {
		s3Key: testFileKey,
	});
// Step 3: Validate the results (stubbed)
	console.log("Step Function Output:", stepFunctionOutput);

// Example placeholder assertion
	expect(stepFunctionOutput).toContain("expected-value");
});

Obviously, this is very generic code, and we had more specific assertions and checks, but it should give you the idea.

From here, we waited for the Step Function to complete, validated the output in the database, and asserted that everything processed as expected.

We focused on controlling the flow precisely to keep our tests lean and fast. Rather than relying on arbitrary wait times, we polled the Step Function status to respond as soon as processing completed. Test data was cleared between runs using the AWS SDK, ensuring a clean environment each time.

We also tagged each file upload with test-specific metadata, making tracing and verifying results easy. And by using Playwright fixtures to manage setup and teardown, we kept our test logic tidy and repeatable.

This approach mirrored real user behaviour — a genuine file upload kicked off the entire process. It gave us confidence in our AWS configuration and the data handling logic, without introducing flaky dependencies on the UI. Best of all, it was fast enough to run in our CI pipeline without slowing everything down.

End-to-end tests like this are often avoided because of their complexity, but they can be efficient and reliable with the right tools.

By combining AWS SDKs with Playwright, we could simulate real-world workflows in a controlled, automated way.

In addition, to verify that the results were properly persisted in the database, we used Playwright to query the API, but GraphQL and Playwright will be the subject of another post!

The result? A test suite that gives us confidence, without dragging down our release speed and is agnostic of any UI changes.