> For the complete documentation index, see [llms.txt](https://docs.nimbus.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.nimbus.dev/overview/optimization.log/optimization.log.reduce.md).

# Reduce Optimizations

Reduce optimizations reduce the volume of your logs, either along number of events or raw ingested bytes.

Nimbus analyzes all your logs and finds high volume log patterns based on incoming data. From these patterns, Nimbus generates transformations that aggregate related logs into a single event.

We refer to this style of transformation as **lossless aggregation**. You can see an example of how this works below.

![Lossless Aggregation](https://ik.imagekit.io/fpjzhqpv1/Cursor_and_Excalidraw_l96xP4vXM.png?updatedAt=1701279438503)

## Optimization Triggers

Nimbus can automatically optimize logs when it detects the following situations:

1. Logs with common message patterns
2. Logs with common identifiers
3. Multi-line Logs

For before and after examples of these triggers, see [examples](https://github.com/nimbushq/public-docs/blob/main/examples.md).

### Logs with common message patterns

These are high volume log events that repeat most of their content. For most applications most of the time, this will be the primary driver of log volume. Examples include health checks and heart beat notifications.

### Logs with common identifiers

These are logs that describe a sequence of related events. These sequences usually have some sort of common identifier like a `transactionId` or a `jobId`. Examples include a background job and business specific user flows.

### Multi-line Logs

These are logs where the message body can be spread across multiple new lines. Unless you add special logic on the agent side, the default behavior is to emit each newline delimited message as a separate log event.

## Optimization Dimensions

Nimbus optimizes logs across the following dimensions:

1. Volume: Optimize to reduce the number of events logged
2. Size: Optimize to reduce the size of events logged

For before and after examples of optimizations along these dimensions, see [examples](https://github.com/nimbushq/public-docs/blob/main/examples.md).

### Volume

When optimizing for volume, Nimbus aggregates as many logs as it can given the constraints of the destination.

For example, [datadog](https://docs.datadoghq.com/api/latest/logs/#send-logs) has specific limits around total array size as well as log size. Nimbus makes sure to aggregate underneath this limit to maximize volume reduction.

### Size

When optimizing for size, Nimbus deduplicates and removes redundant metadata as it aggregates logs.

For example, when aggregating [logs with common message patterns](https://github.com/nimbushq/public-docs/blob/main/optimizations.log.md#logs-with-common-message-patterns), its often the case that 40% or more of the metadata (tags and attributes) are the same.

## Optimization Fidelity

Nimbus generated optimizations can be tuned via fidelity levels to indicate how much of the original log message to preserve.

### High

Nimbus optimizes for preserving original log data with perfect fidelity. This means there is no reduction in ingest size and aggregated logs contain all fields of the original log entires with only identical fields deduplicated.

### Medium

Nimbus preserves most of the data. Individual timestamps in aggregated logs are discarded.

### Low

Nimbus optimizes for ingest size. Low value fields are nominated for removal. All [nimbus attributes](/resources/ref.attributes.md) except `nimsize` are removed from the resulting log.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.nimbus.dev/overview/optimization.log/optimization.log.reduce.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.