--- title: API configuration description: Configure Logpush jobs via the API. image: https://developers.cloudflare.com/core-services-preview.png --- > Documentation Index > Fetch the complete documentation index at: https://developers.cloudflare.com/logs/llms.txt > Use this file to discover all available pages before exploring further. [Skip to content](#%5Ftop) # API configuration ## Endpoints The table below summarizes the job operations available for both Logpush and Edge Log Delivery jobs. Make sure that Account-scoped datasets use `/accounts/{account_id}` and Zone-scoped datasets use `/zone/{zone_id}`. For more information, refer to the [Datasets](https://developers.cloudflare.com/logs/logpush/logpush-job/datasets/) page. You can locate `{zone_id}` and `{account_id}` arguments based on the [Find zone and account IDs](https://developers.cloudflare.com/fundamentals/account/find-account-and-zone-ids/) page. The `{job_id}` argument is numeric, like 123456\. The `{dataset_id}` argument indicates the log category (such as `http_requests` or `audit_logs`). | Operation | Description | API | | --------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | POST | Create job | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/jobs/methods/create/) | | GET | Retrieve job details | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/datasets/subresources/jobs/methods/get/) | | GET | Retrieve all jobs for all datasets | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/jobs/methods/list/) | | GET | Retrieve all jobs for a dataset | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/datasets/subresources/jobs/methods/get/) | | GET | Retrieve all available fields for a dataset | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/datasets/subresources/fields/methods/get/) | | PUT | Update job | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/jobs/methods/update/) | | DELETE | Delete job | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/jobs/methods/delete/) | | POST | Check whether destination exists | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/validate/methods/destination/) | | POST | Get ownership challenge | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/ownership/methods/validate/) | | POST | Validate ownership challenge | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/ownership/methods/validate/) | | POST | Validate log options | [Documentation](https://developers.cloudflare.com/api/resources/logpush/subresources/validate/methods/origin/) | For concrete examples, refer to the tutorials in [Logpush examples](https://developers.cloudflare.com/logs/logpush/examples/). ## Connecting The Logpush API requires credentials like any other Cloudflare API. Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/)is required: * `Logs Write` List Logpush jobs ``` curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs" \ --request GET \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" ``` ## Ownership Before creating a new job, ownership of the destination must be proven. To issue an ownership challenge token to your destination: Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/)is required: * `Logs Write` Get ownership challenge ``` curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/ownership" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "destination_conf": "s3://?region=us-west-2" }' ``` A challenge file will be written to the destination, and the filename will be in the response (the filename may be expressed as a path, if appropriate for your destination): ``` { "errors": [], "messages": [], "result": { "valid": true, "message": "", "filename": ".txt" }, "success": true } ``` You will need to provide the token contained in the file when creating a job. Note When using Sumo Logic, you may find it helpful to have [Live Tail ↗](https://help.sumologic.com/05Search/Live-Tail/About-Live-Tail) open to see the challenge file as soon as it is uploaded. ## Destination You can specify your cloud service provider destination via the required **destination\_conf** parameter. Note As of May 2022, defining a unique destination for a Logpush job will no longer be required. As this constraint has been removed, you can now have more than one job writing to the same destination. The `destination_conf` parameter must follow this format: ``` :// ``` Supported schemes are listed below, each tailored to specific providers such as R2, S3, etc. Additionally, generic use cases like `https` are also covered: * `r2`, * `gs`, * `s3`, * `sumo`, * `https`, * `azure`, * `splunk`, * `sentinelone`, * `datadog`. The `destination-address` should generally be provided by the destination provider. However, for certain providers, we require the `destination-address`to follow a specific format: * **Cloudflare R2** (scheme `r2`): bucket path + account ID + R2 access key ID + R2 secret access key; for example: `r2://?account-id=&access-key-id=&secret-access-key=` * **AWS S3** (scheme `s3`): bucket + optional directory + region + optional encryption parameter (if required by your policy); for example: `s3://bucket/[dir]?region=[&sse=AES256]` * **Datadog** (scheme `datadog`): Datadog endpoint URL + Datadog API key + optional parameters; for example: `datadog://?header_DD-API-KEY=&ddsource=cloudflare&service=&host=&ddtags=` * **Google Cloud Storage** (scheme `gs`): bucket + optional directory; for example: `gs://bucket/[dir]` * **Microsoft Azure** (scheme `azure`): service-level SAS URL with `https` replaced by `azure` \+ optional directory added before query string; for example: `azure:///[dir]?` * **New Relic** (use scheme `https`): New Relic endpoint URL which is `https://log-api.newrelic.com/log/v1` for US or `https://log-api.eu.newrelic.com/log/v1` for EU + a license key + a format; for example: for US `"https://log-api.newrelic.com/log/v1?Api-Key=&format=cloudflare"` and for EU `"https://log-api.eu.newrelic.com/log/v1?Api-Key=&format=cloudflare"` * **Splunk** (scheme `splunk`): Splunk endpoint URL + Splunk channel ID + insecure-skip-verify flag + Splunk sourcetype + Splunk authorization token; for example: `splunk://?channel=&insecure-skip-verify=&sourcetype=&header_Authorization=` * **Sumo Logic** (scheme `sumo`): HTTP source address URL with `https` replaced by `sumo`; for example: `sumo:///receiver/v1/http/` * **SentinelOne** (scheme `sentinelone`): SentinelOne endpoint URL + SentinelOne sourcetype + SentinelOne authorization token; for example: `sentinelone://?sourcetype=&header_Authorization=` For **R2**, **S3**, **Google Cloud Storage**, and **Azure**, you can organize logs into daily subdirectories by including the special placeholder `{DATE}` in the URL path. This placeholder will automatically be replaced with the date in the `YYYYMMDD` format (for example, `20180523`). For example: * `s3://mybucket/logs/{DATE}?region=us-east-1&sse=AES256` * `azure://myblobcontainer/logs/{DATE}?[QueryString]` This approach is useful when you want your logs grouped by day. For more information on the value for your cloud storage provider, consult the following conventions: * [AWS S3 CLI ↗](https://docs.aws.amazon.com/cli/latest/reference/s3/index.html) (S3Uri path argument type) * [Google Cloud Storage CLI ↗](https://cloud.google.com/storage/docs/gsutil) (Syntax for accessing resources) * [Microsoft Azure Shared Access Signature ↗](https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview) * [Sumo Logic HTTP Source ↗](https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source) To check if a destination is already in use: Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/)is required: * `Logs Write` Check destination exists ``` curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/validate/destination/exists" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "destination_conf": "s3://foo" }' ``` Response ``` { "errors": [], "messages": [], "result": { "exists": false }, "success": true } ``` ## Name A human-readable, optional job name that does not need to be unique. We recommend choosing a meaningful name, such as the domain name, to help you easily identify and manage your job. You can update the name later if needed. ## Kind The kind parameter (optional) is used to differentiate between Logpush and Edge Log Delivery jobs. For Logpush jobs, this parameter can be left empty or omitted. For Edge Log Delivery jobs, set `"kind": "edge"`. Currently, Edge Log Delivery is only supported for the `http_requests` dataset. Note The kind parameter cannot be used to update existing Logpush jobs. You can only specify the kind parameter when creating a new job. Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/)is required: * `Logs Write` Create Logpush job ``` curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "name": "", "destination_conf": "s3://?region=us-west-2", "dataset": "http_requests", "output_options": { "field_names": [ "ClientIP", "ClientRequestHost", "ClientRequestMethod", " ClientRequestURI", "EdgeEndTimestamp", "EdgeResponseBytes", "EdgeResponseStatus", "EdgeStartTimestamp", "RayID" ], "timestamp_format": "rfc3339" }, "kind": "edge" }' ``` ## Options Logpull\_options has been replaced with Custom Log Formatting output\_options. Please refer to the [Log Output Options](https://developers.cloudflare.com/logs/logpush/logpush-job/log-output-options/) documentation for instructions on configuring these options and updating your existing jobs to use these options. If you are still using logpull\_options, here are the options that you can customize: 1. **Fields** (optional): Refer to [Datasets](https://developers.cloudflare.com/logs/logpush/logpush-job/datasets/) for the currently available fields. The list of fields is also accessible directly from the API: `https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/datasets/{dataset_id}/fields`. Default fields: `https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/datasets/{dataset_id}/fields/default`. 2. **Timestamp format** (optional): The format in which timestamp fields will be returned. Value options: `unixnano` (nanoseconds unit - default), `unix` (seconds unit), `rfc3339` (seconds unit). 3. **Redaction for CVE-2021-44228** (optional): This option will replace every occurrence of `${` with `x{`. To enable it, set `"CVE-2021-44228": true`. Note The **CVE-2021-44228** parameter can only be set through the API at this time. Updating your Logpush job through the dashboard will set this option to false. To check if the selected **logpull\_options** are valid: Required API token permissions At least one of the following [token permissions](https://developers.cloudflare.com/fundamentals/api/reference/permissions/)is required: * `Logs Write` Validate origin ``` curl "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/validate/origin" \ --request POST \ --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ --json '{ "logpull_options": "fields=RayID,ClientIP,EdgeStartTimestamp×tamps=rfc3339&CVE-2021-44228=true", "dataset": "http_requests" }' ``` Response ``` { "errors": [], "messages": [], "result": { "valid": true, "message": "" }, "success": true } ``` ## Configuration change timing When you modify a Logpush job configuration, changes do not take effect immediately. ### Destination changes If you reconfigure a job to use a new destination, logs may continue to be sent to the old destination for approximately 10-15 minutes during the transition period. This delay allows the system to complete in-flight uploads and propagate the new configuration across Cloudflare's network. ### Field changes When you add new fields to an existing Logpush job, the new fields will appear in your logs within approximately 10-15 minutes. This timing is an estimate and may vary based on system load. Note These timeframes are estimates. If you need to verify that changes have taken effect, monitor your destination for the updated log format or check the [Health Dashboard](https://developers.cloudflare.com/logs/logpush/logpush-health/) for recent uploads. ## Filter Use filters to select the events to include and/or remove from your logs. For more information, refer to [Filters](https://developers.cloudflare.com/logs/logpush/logpush-job/filters/). ## Sampling rate Value can range from `0.0` (exclusive) to `1.0` (inclusive). `sample=0.1` means return 10% (1 in 10) of all records. The default value is `1`, meaning logs will be unsampled. ### Understanding sample\_rate and SampleInterval The `sample_rate` parameter and `SampleInterval` field are independent mechanisms that operate at different stages of the logging pipeline: * **`sample_rate`**: A configuration parameter you set on your Logpush job to control what percentage of logs are delivered to your destination (0.0-1.0). For example, setting `sample_rate: 0.1` delivers approximately 10% of logs. * **`SampleInterval`**: A data field that appears in certain datasets (particularly [Network Analytics Logs](https://developers.cloudflare.com/logs/logpush/logpush-job/datasets/account/network%5Fanalytics%5Flogs/)) indicating upstream sampling applied during data collection. A `SampleInterval` of 1000 means the log entry represents 1 in 1000 packets. The `sample_rate` you configure applies on top of any pre-existing sampling. If your data already has `SampleInterval: 1000` and you set `sample_rate: 0.1`, you receive approximately 1 in 10,000 of the original events (1000 × 10). Note When customer-configured sampling is applied, the `SampleInterval` field value in the logs is not modified. When there is no internal sampling, `SampleInterval` will always be 1 regardless of your configured `sample_rate`. ## Max upload parameters These parameters control the size of each upload batch — not how quickly data is delivered. Use them to prevent overloading your destination with uploads that are too large or too small. | Parameter | Description | Default | | ------------------------------ | ------------------------------------------------------------------------- | --------------------- | | max\_upload\_bytes | Maximum uncompressed file size of a batch of logs. | Varies by destination | | max\_upload\_records | Maximum number of log lines per batch. | 100,000 | | max\_upload\_interval\_seconds | Maximum time-span of log data per batch (used during catch-up scenarios). | Varies by destination | Note These settings influence upload size, not delivery latency. Logpush processes data approximately once per minute, regardless of these parameter values. Adjusting these settings results in smaller or larger uploads per batch, which can help you avoid overloading destinations that have memory or request-size constraints. ### When to adjust these parameters * Reduce `max_upload_records` if your destination struggles with large payloads or runs out of memory processing big batches. * Increase `max_upload_records` if you want fewer, larger files (for example, when pushing to object storage like R2 or S3). * For destinations like Datadog that have strict payload limits, Logpush automatically uses smaller batch sizes (for example, 1,000 rows). Tip If you need to estimate the number of files generated for cost planning (for example, R2 write operations), run Logpush for a representative period and measure the actual output. The number of uploads depends on your data volume and cannot be precisely calculated in advance. ## Custom fields You can add custom fields to your HTTP request log entries in the form of HTTP request headers, HTTP response headers, and cookies. Custom fields configuration applies to all the Logpush jobs in a zone that use the HTTP requests dataset. To learn more, refer to [Custom fields](https://developers.cloudflare.com/logs/logpush/logpush-job/custom-fields/). ## Audit The following Logpush actions are recorded in **Cloudflare Audit Logs**: create, update, and delete job. ```json {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"item":{"@id":"/directory/","name":"Directory"}},{"@type":"ListItem","position":2,"item":{"@id":"/logs/","name":"Logs"}},{"@type":"ListItem","position":3,"item":{"@id":"/logs/logpush/","name":"Logpush"}},{"@type":"ListItem","position":4,"item":{"@id":"/logs/logpush/logpush-job/","name":"Logpush job setup"}},{"@type":"ListItem","position":5,"item":{"@id":"/logs/logpush/logpush-job/api-configuration/","name":"API configuration"}}]} ```