Cloudflare
Cloudflare Logpush pushes HTTP request logs to an S3 bucket managed by EINBLIQ.IO. CMCD headers can be added as custom log fields.
Updated
Values provided by EINBLIQ.IO
| Value | Used for |
|---|---|
<BUCKET_NAME> |
S3 bucket |
<S3_ENDPOINT> |
Host name of the S3 storage endpoint |
<BUCKET_PREFIX> |
Path for your files inside the bucket |
<BUCKET_REGION> |
Bucket region |
<ACCESS_KEY_ID> |
Access key ID for the bucket |
<SECRET_ACCESS_KEY> |
Secret access key |
Prerequisites
- Logpush with the HTTP requests dataset for the zones that deliver your streams. Logpush is available on all Cloudflare plans with usage-based pricing; which datasets you can use depends on the Cloudflare products on your account.
- Permission to create Logpush jobs and, for CMCD headers, to edit the custom log fields of the zone.
Create the Logpush job
- Open the Logpush page of your zone and select Create a Logpush job.
- Select S3-Compatible as the destination and enter the values below.
- Select the HTTP requests dataset and name the job, for example
einbliq-io. - Under Send the following fields, select the fields listed below.
- Under Advanced Options, set the timestamp format to
rfc3339msand leave sampling off, so every request is logged. - Create the job.
| Setting | Value |
|---|---|
| Bucket | <BUCKET_NAME> |
| Path | <BUCKET_PREFIX> |
| Organize logs into daily subfolders | Enabled |
| Endpoint URL | <S3_ENDPOINT> |
| Bucket region | <BUCKET_REGION> |
| Access Key ID | <ACCESS_KEY_ID> |
| Secret Access Key | <SECRET_ACCESS_KEY> |
The S3-compatible destination needs no ownership challenge. Cloudflare writes gzip-compressed, newline-delimited JSON and sets the push interval itself.
Log fields
| Field | Content |
|---|---|
EdgeStartTimestamp |
Request start time |
ClientRequestHost |
Requested host |
ClientIP |
IP address of the requesting client |
EdgeColoCode |
Cloudflare data centre that served the request |
EdgeServerIP |
Edge server that contacted the origin, empty without an origin request |
ClientRequestUserAgent |
HTTP User-Agent header |
ClientRequestReferer |
HTTP Referer header |
ClientRequestURI |
Requested path including the query string, with CMCD= when present |
EdgeResponseStatus |
HTTP response status |
EdgeResponseBytes |
Response size in bytes |
EdgeTimeToFirstByteMs |
Time to first byte at the edge, in milliseconds |
CacheCacheStatus |
Cache status |
RequestHeaders |
Request headers configured as custom log fields, including CMCD headers |
Log CMCD headers
For players that send CMCD headers, add the headers as custom log fields of the zone. On the Logpush page, open Custom log fields, select Edit Custom Fields and add each header as a field of type Request Header. Enter the header names in lower case:
cmcd-requestcmcd-objectcmcd-statuscmcd-session
The values then appear in the RequestHeaders field of the job.
Keep CMCD out of the cache key
Create a cache rule for the hostnames that deliver your streams. Under Cache key, set the query string option to All query string parameters except and enter CMCD. Cloudflare documents this option for Enterprise plans; on other plans, Ignore query string removes the whole query string from the cache key.
Check the delivery
When the job is active, inform EINBLIQ.IO. The delivery check then confirms that the files arrive with all fields and contain CMCD for your players.
Cloudflare documentation
- Enable S3-compatible endpoints describes the destination settings.
- HTTP requests dataset lists every available field.
- Custom fields explains how request headers are logged.
- Cache Rules settings lists the cache key options and their plan availability.