Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: CI

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
test:
name: Node.js Unit & Integration Tests
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Node.js 20
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
cache-dependency-path: './src/package-lock.json'

- name: Install dependencies
working-directory: ./src
run: npm install

- name: Run test suite
working-directory: ./src
run: npm test

terraform:
name: Terraform Validation
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: "1.13.0"

- name: Check Terraform formatting
run: terraform fmt -check -recursive

- name: Validate Cloud CDN Terraform
working-directory: ./infra
run: |
terraform init -backend=false
terraform validate

- name: Validate Media CDN Terraform
working-directory: ./infra_mcdn
run: |
terraform init -backend=false
terraform validate
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,6 @@ npm-debug.log
accessories
# ignore everything under service extensions
media_cdn_wasm

infra/.terraform*
infra_mcdn/.terraform*
206 changes: 164 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,196 @@
# Google CDN Image Optimizer

## Use case
This Node.js module is for converting large images in common formats to smaller, web-friendly **JPEG**, **JPEG 2000**, **JPEG XL**, **PNG**, **WebP**, **GIF** images of varying dimensions.
## Overview
A high-performance, containerized image transformation service built on **Sharp**. It provides dynamic on-the-fly image resizing, format conversion, and compression optimized for **Google Cloud CDN** and **Google Media CDN**.

Cloud CDN support - with the integration of `Cloud CDN Content Targeting`(under Private Preview), you get the benefits of image optimization based on `device_type` and `user_agent_famliy` out of the box, with optional customizations of image **width**, **height**, and **quality**.
### Key Features
- **Device-Aware Optimization:** Automatically tunes image quality based on client device characterization (`desktop`, `tablet`, `mobile`, `smart_tv`, etc.).
- **Next-Gen Image Formats:** Converts legacy formats to smaller, web-friendly **WebP**, **JPEG XL**, **JPEG 2000**, and optimized **JPEG/PNG/GIF**.
- **Two-Tier Caching Architecture:**
- **Tier 1:** Google Cloud CDN / Media CDN edge cache with custom cache keys based on format, dimensions, device type, and UA family.
- **Tier 2:** High-speed in-memory LRU cache on Cloud Run to eliminate redundant image processing.
- **Auto-Rotation:** Automatically reads EXIF orientation tags and rotates images correctly.
- **Legacy Browser Support:** Automatically falls back to standard JPEG for legacy user agents (such as MSIE).

**[NEW]** Media CDN support - with the recent released `Device-Type and User-Agent Characterization capability` (Generally Available), you get the benefits of image optimization based on `device_type` and `user_agent_famliy` out of the box, with optional customizations of image **width**, **height**, and **quality**.
---

## Formats
## Supported Formats

This module supports reading **JPEG, PNG, WebP, GIF, TIFF and SVG** images.
| Category | Formats |
| :--- | :--- |
| **Input / Source** | JPEG, PNG, WebP, GIF, TIFF, SVG |
| **Output / Optimized** | WebP, JPEG, JPEG 2000 (`jp2`), JPEG XL (`jxl`), PNG, GIF, TIFF, Raw pixel data |

Output images can be in **JPEG**, **JPEG 2000**, **JPEG XL**, **PNG**, **WebP**, **GIF** and **TIFF** formats as well as uncompressed **raw pixel data**.
> [!NOTE]
> **AVIF support** is currently disabled pending upstream libvips heif plugin updates.

**[AVIF support]** is currently disabled.
---

## Diagrams

![Architecture_Diagram](./architecture_diagrams/CDN_ImageOptimization_Architecture_Diagram.png)

![Sequence_Flow_Diagram](./architecture_diagrams/Sequence_Flow_Diagram.png)

---

## Deploy to Cloud CDN via Terraform
The following instructions will lead the path to a complete deployment of Cloud CDN and Image Optimizer.
## Transformation API & Query Parameters

1. Clone this repo.
```
Requests to `/images/*` accept the following query parameters for dynamic transformations:

| Parameter | Type | Default | Allowed Values / Description | Example |
| :--- | :--- | :--- | :--- | :--- |
| `w` | Integer | Source width | Target width in pixels (`1` to `4096`). | `?w=800` |
| `h` | Integer | Source height | Target height in pixels (`1` to `4096`). | `?h=600` |
| `f` | String | `webp` | Target format: `webp`, `jpg`, `jpeg`, `png`, `gif`, `jp2`, `jxl`, `tiff`, `raw`. | `?f=webp` |
| `q` | Integer | Device-based | Compression quality (`1` to `100`). If omitted, automatically determined by device type. | `?q=75` |
| `fit` | String | `cover` | Resizing strategy: `cover`, `contain`, `fill`, `inside`, `outside`. | `?fit=contain` |
| `p` | String | `center` | Crop anchor position / gravity: `top`, `bottom`, `left`, `right`, `left_top`, `right_top`, `left_bottom`, `right_bottom`, `center`. | `?p=top` |

### Default Quality by Device Type
When `q` is not specified in the query string, quality is auto-assigned based on upstream CDN characterization headers:
- **Desktop:** `q=60`
- **Tablet / Smart TV / Game Console / Set-Top Box:** `q=40`
- **Mobile / Wearable / Smart Speaker:** `q=20`
- **Default / Unspecified:** `q=40`

### Fit Modes Explained
- `cover` *(default)*: Preserves aspect ratio, crops image to cover both provided dimensions.
- `contain`: Preserves aspect ratio, letterboxes image inside provided dimensions.
- `fill`: Stretches image to exact width and height without maintaining aspect ratio.
- `inside`: Preserves aspect ratio, resizes image to fit completely within width/height boundaries.
- `outside`: Preserves aspect ratio, resizes image to be as small as possible while ensuring dimensions are greater than or equal to targets.

---

## CDN Header Integration

The service integrates seamlessly with Google Cloud CDN and Media CDN using custom request and response headers:

### Upstream Request Headers
- `x-client-device-type`: Injected by CDN (`desktop`, `tablet`, `mobile`, `smart_tv`, `game_console`, etc.).
- `x-client-ua-family`: Injected by CDN user-agent characterization (e.g. `Chrome`, `Safari`, `MSIE`).
- `x-client-host`: Populated by CDN / TLS SNI for multi-tenant or origin domain resolution.
- `x-client-accept`: Client `Accept` header forwarded for content negotiation.

### Response Headers
- `X-IO-Cache`: Reports Cloud Run origin cache status (`HIT` or `MISS`).
- `X-IO-Cache-Key`: Cache key generated from format, quality, dimensions, position, fit, and URL path.
- `x-cache-status` / `x-mcdn-cache-status`: Upstream CDN edge cache status (`HIT`, `MISS`, `REVALIDATED`).

---

## Deployment

### Option 1: Deploy to Google Cloud CDN via Terraform

1. **Clone the repository:**
```bash
git clone https://github.com/CDN-guy/google-cdn-image-optimization.git
cd google-cdn-image-optimization/infra
```

1. Change to the Infrastructure deployment directory.
2. **Create your `infra.tfvars` file:**
```hcl
project_id = "your-gcp-project-id"
project_number = "123456789012"
cloudrun_region = "us-central1"
origin_fqdn = "images.example.com"
origin_base_path = "/original/"
```
cd infra

| Variable | Description | Default |
| :--- | :--- | :--- |
| `project_id` | GCP Project ID | *Required* |
| `project_number` | GCP Project Number | *Required* |
| `cloudrun_region` | GCP Region where Cloud Run is deployed | `us-east1` |
| `origin_fqdn` | FQDN of the backend origin holding original images | *Required* |
| `origin_base_path` | Base path on origin server | `/original/` |
| `imageopt_svc_image`| Artifact Registry container image URL | Default prebuilt image |

3. **Deploy with Terraform:**
```bash
terraform init
terraform plan -var-file="infra.tfvars"
terraform apply -var-file="infra.tfvars" -auto-approve
```

---

1. Create a file named **infra.tfvars** under the **infra** directory
- Set the variable values for `project_id`, `project_number`, `cloudrun_region`, `origin_fqdn` and `origin_base_path` with your preference.
- (leave `imageopt_svc_image` as default value - unless you prefer to custom build your own container image [steps in bootstrap.sh])
example:
### Option 2: Deploy to Google Media CDN via Terraform

```
project_id = "abc123xyz"
project_number = "1111111111111"
cloudrun_region = "us-central1"
origin_fqdn = "www.google.com"
origin_base_path = "/"
```
> [!IMPORTANT]
> Media CDN requires your Google Cloud project to be allowlisted. Contact your Google Cloud sales or account team if access is not yet enabled.

| Variable | Description |
| ----------- | ----------- |
| project_id | GCP project ID |
| project_number | GCP project Number |
| cloudrun_region | Region where CloudRun should be deployed |
| origin_fqdn | FQDN of origin server |
| origin_base_path | Base Path of origin, default `/original/` |
1. **Change to the Media CDN infrastructure directory:**
```bash
cd google-cdn-image-optimization/infra_mcdn
```

1. Run the following commands to initiate Terraform for deployment.
```
terraform init
terraform plan -var-file="infra.tfvars"
terraform apply -var-file="infra.tfvars" -auto-approve
```
2. **Create your `infra_mcdn.tfvars` file:**
```hcl
project_id = "your-gcp-project-id"
project_number = "123456789012"
cloudrun_region = "us-central1"
origin_fqdn = "images.example.com"
origin_base_path = "/"
```

3. **Deploy with Terraform:**
```bash
terraform init
terraform plan -var-file="infra_mcdn.tfvars"
terraform apply -var-file="infra_mcdn.tfvars" -auto-approve
```

---

## Deploy to Media CDN via Terraform & gCloud SDK
## Custom Container Build (Optional)

[Check out the Instruction for MediaCDN](src/media_cdn.md)
If you wish to build and push your own container image to Artifact Registry rather than using the default prebuilt image:

1. Edit the configuration variables in [`infra/bootstrap.sh`](infra/bootstrap.sh) (or [`infra_mcdn/bootstrap.sh`](infra_mcdn/bootstrap.sh)):
```bash
PROJECT_ID="your-gcp-project-id"
PROJECT_NUM="123456789012"
LOCATION="us-central1"
REPO_NAME="img-opt"
```

2. Run the bootstrap script:
```bash
chmod +x bootstrap.sh
./bootstrap.sh
```

---

## Local Development & Docker

### Running Locally with Node.js
```bash
cd src
npm install
npm start
```
The server will start listening on `http://localhost:8080`.

### Running Locally with Docker
```bash
cd src
docker build -t image-optimizer .
docker run -p 8080:8080 image-optimizer
```

---

## Demo Pages

[Cloud CDN Demo Page](https://images.thegoogle.cloud/cdn-IO.html)
- [Cloud CDN Demo Page](https://images.thegoogle.cloud/cdn-IO.html)
- [Media CDN Demo Page](https://media-cdn.thegoogle.cloud/image-opt-demo.html)

---

## License

[Media CDN Demo Page](https://media-cdn.thegoogle.cloud/image-opt-demo.html)
This project is licensed under the [MIT License](LICENSE).
6 changes: 5 additions & 1 deletion changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

### v1.0.6

* Disable AVIF support

### v1.0.5

* Remove Service Extensions and Switch to MediaCDN's native support of device-type, ua-family
Expand All @@ -10,7 +14,7 @@

### v1.0.3

* Add auto-rorate function
* Add auto-rotate function

### v1.0.2

Expand Down
40 changes: 0 additions & 40 deletions infra/.terraform.lock.hcl

This file was deleted.

2 changes: 1 addition & 1 deletion infra/bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ gcloud services enable artifactregistry.googleapis.com cloudapis.googleapis.com
printf "=========================================\n"
printf "Creating Artifact Registry docker repo...\n"
printf "=========================================\n"
gcloud artifacts repositories create "${REPO_NAME}-${PROJECT_NUM}" --location "${LOCATION}" --repository-format=docker --project "${PROJECT_ID}" ||
gcloud artifacts repositories create "${REPO_NAME}-${PROJECT_NUM}" --location "${LOCATION}" --repository-format=docker --project "${PROJECT_ID}"

# Run Cloud Build for backend with name of repo to build and push proxy images and imaginary image
printf "=========================================\n"
Expand Down
Loading
Loading