Allowing users to upload high-resolution media directly through your application server is a recipe for infrastructure failure. On high-traffic marketplace platforms like ABCsPuppyzs, pet breeders and adopters upload multi-gigabyte 4K puppy videos, adoption contracts, and medical certificates daily.
Routing these files through a Node.js or Next.js API route ties up memory buffers, saturates server network bandwidth, and results in frequent 413 Payload Too Large or 504 Gateway Timeout errors. Here is how we re-architected our media pipeline: routing uploads directly from the user's browser to Amazon S3 via presigned multipart URLs and serving assets globally via CloudFront with Origin Access Control (OAC).
The Anti-Pattern: Uploading Through the Application Server
When an upload passes through your web server, your server pays for the bandwidth twice: once to receive the file from the user, and once to push it to object storage. If 50 users upload a 100MB video simultaneously, your server must buffer 5GB of in-flight data, starving CPU resources needed for database queries and user checkout flows.
Engineering the Multipart Upload Lifecycle
For files larger than 20MB, a single network interruption can corrupt the upload, forcing the user to restart from 0%. By implementing S3 Multipart Uploads directly in the browser, the client slices the file into 5MB chunks and uploads them in parallel across multiple threads.
If chunk #4 drops due to a mobile network switch, only that specific 5MB chunk is retried. Once all chunks are confirmed by S3, the client calls our backend to finalize the upload and trigger asynchronous media transcoding.
Securing S3 with CloudFront Origin Access Control (OAC)
Allowing public read access directly to an S3 bucket is a major security risk and incurs higher AWS data egress costs. In our architecture, the S3 bucket is strictly private: all public read access is locked down using an AWS IAM bucket policy that permits reads exclusively through CloudFront Origin Access Control (OAC).
- Zero Server Load: Media ingestion consumes zero application server memory or network bandwidth, freeing web nodes to handle customer interactions at peak efficiency.
- Instant Global Delivery: Images and video streams are cached at CloudFront edge points of presence across the globe, cutting media load times from 1,200ms to under 45ms.
- Automatic Resilient Resumption: Client-side chunking ensures large media uploads complete reliably even over unstable 4G and 5G cellular connections.
