---
status: accepted
---

# Streaming file inputs resolve to a lazy ApStreamingFile

## Decision

`Property.File({ streaming: true })` resolves to `ApStreamingFile = { filename, extension?, size?, body: Readable }` instead of `ApFile`. The engine's `fileProcessor` branches on the flag: a URL input is `fetch`ed and its **undrained** response body exposed as a Node `Readable` (`Readable.fromWeb`) with `size` from `Content-Length`; a base64 data URL decodes to a one-shot `Readable` with an exact `size`. It reuses the same `PropertyType.FILE`, so the frontend file picker is unchanged. Plain `Property.File()` still returns a buffered `ApFile`.

## Context

A piece that uploads a large file to an external service (Amazon S3, Dropbox, Google Drive, …) declared its input as `Property.File()` and received an `ApFile` — which the engine produced by buffering the **entire** file into a `Buffer` (unbounded `arrayBuffer()` on the URL path) before `run()` was even called. The upload then streamed from that in-memory buffer, so the peak-RAM win of streaming was already lost upstream. This resolves the "Property.file streaming (read side)" item that [000008](000008-streaming-file-writes-go-through-the-app-one-path.md) deferred as YAGNI, now that concrete large-file upload pieces need it.

## Why

`ApStreamingFile` is a plain type, not a class like `ApFile` (which carries a `base64` getter over its `Buffer`): a one-shot body can be read exactly once and has no replayable representation, so there is nothing for methods to wrap. The S3 consumer originally used `putObject({ Body, ContentLength })` rather than `@aws-sdk/lib-storage`, on the reasoning that an upload source (URL `Content-Length` / base64 length) almost always reports a size, so a single streamed PUT needed **no new dependency**. [#14347](https://github.com/activepieces/activepieces/pull/14347) **superseded that**: Upload File now uses `lib-storage`'s `Upload` like the write side, because a size-dependent PUT silently fell back to buffering on every sizeless source (notably any `Content-Encoding`-compressed URL, see below) — which is the case streaming exists for. `Upload` needs no length up front and buffers each \~5 MB part, so parts are individually replayable. A flag on `Property.File` beat a separate `Property.StreamingFile` builder: the wire contract and renderer are identical either way, and the flag keeps both file shapes discoverable under one name (typed via overloads with the house `R extends true ? …` conditional-return pattern so `required` still narrows through `createAction`).

## Consequences

- The unconditional win is deleting the **unbounded** `arrayBuffer()` buffer on the URL input path. The additional peak-RAM win over the buffered path is marginal at the `AP_MAX_FILE_SIZE_MB` default (25 MB) and only material once that cap is raised or the source is an uncapped external URL.
- `size` is best-effort: absent/invalid `Content-Length` → `undefined`. It is **also** dropped when the response carries a `Content-Encoding` (gzip/br/deflate) — undici transparently decompresses the body but leaves the *compressed* `Content-Length` in place, so trusting it would understate the streamed byte count and silently truncate the destination object. Since #14347 a sizeless body no longer forces the consumer to buffer; `size` is informational, and only a consumer that genuinely needs an explicit content length has to fall back. Three consumers still do: Dropbox, SharePoint and OneDrive upload in a single request and must send a `Content-Length` (OneDrive/SharePoint additionally need the total up front for Graph's `Content-Range`), so they keep a `readableToBuffer` fallback for the sizeless case. S3, Azure, Google Drive and SFTP never read `size`.
- There is **no** `AP_MAX_FILE_SIZE_MB` **ceiling** on the streamed URL input. Intentional — the feature exists to move large files, and the prior buffered path was likewise unbounded. A cap is deferred; it would need a counting pass-through stream that aborts past the limit.
- The `body` is **one-shot** — no whole-stream retry. Since #14347 `lib-storage` buffers each \~5 MB part before sending, so transient part-level errors (connection reset, throttling, HTTP 500) *are* retried within the upload. What remains non-retryable is the transfer as a whole: the source `Readable` cannot be re-read, so a failure that outlives the part retries cannot be replayed. Whole-stream retry would require buffering, which defeats streaming.
- **A stream body bypasses `httpClient`'s retry loop entirely** (`isStream ? 0 : retries` in `packages/pieces/common/src/lib/http/core/fetch-http-client.ts`). The loop reuses the body serialized before the first attempt, and both shapes it treats as a stream — a raw `Readable` and the `PassThrough` a `form-data` payload is piped into — are one-shot, so a retried request replays a drained stream and sends an empty or truncated body. Silently corrupting an upload is worse than not retrying it. The blast radius is wider than the file pieces: this is shared `pieces-common` behaviour, so *any* piece passing a stream or `form-data` body to `httpClient` now loses its `retries` setting. The rejected alternative was a body-*factory* API (`() => Readable`, as Azure's `BufferScheduler` does per block): neither source is re-readable, so every caller would have to learn to rebuild its body — a large change for a case the chunking uploaders (S3 `Upload`, Azure `uploadStream`) already handle better.
- Engines older than 0.87.0 ignore the `streaming` flag and deliver a buffered `ApFile` (no `body`, no `size`), and the registry serves latest piece versions to them regardless. Consumers must therefore never read `.body`/`.size` off the prop directly: `streamUtils.toStreamingBody` in `pieces-common` normalizes both shapes, and `FileProperty` types the streaming value as `ApStreamingFile | ApFile` so a direct read fails to compile (GIT-1808).
- The URL `fetch` opens the source connection at **input-resolution time** (before `run()`), like the buffered path.
- SSRF posture is unchanged from the existing buffered `handleUrlFile`: the same raw `fetch` also legitimately retrieves AP's own internal http `readUrl`s, so the https-only + `redirect:'error'` guard used by external-only piece code (e.g. SimplyPrint) is deliberately **not** applied here.
