Zabbix 8.0 Beta2: The ClickHouse Revolution for Historical Data
A few weeks ago, the new Beta of Zabbix 8.0 was released. I actually reported the news on our Telegram channel, ZabbixItalia, initially stating that there wasn't much new to see. Well, I have to admit—I was wrong. I hadn't looked closely enough at the small details in the release notes.
Hidden among the updates was a massive piece of news: ClickHouse History Back-End. They finally did it! This is a small revolution for the Zabbix ecosystem, shifting how we handle massive amounts of historical data. In this post, I am going to walk you through exactly what ClickHouse is, why it's a game-changer, and how you can configure it in a standard Zabbix installation.
What is ClickHouse and Why is it a Game-Changer?
If you are monitoring 100 machines, you might not strictly need this. But if you are monitoring 10,000 servers and ingesting enormous amounts of observability data, ClickHouse is exactly what you've been waiting for.
Unlike traditional relational databases like MySQL or PostgreSQL, ClickHouse is a column-oriented database built specifically to manage huge volumes of data. Here is why it stands out:
- Ultrafast Queries: Because data is managed at the column level rather than the row level, aggregation queries and summations are incredibly fast and efficient.
- Linear Scale and Low Cost: ClickHouse scales horizontally. If you need more computing power or space, you simply add more nodes, and all nodes remain active to deliver data.
- High Data Efficiency: It features a highly effective data compression system that can reduce the data footprint inside its tables by up to 80%.
Traditional databases maximize data resilience and relational coherence (row-based). ClickHouse, on the other hand, prioritizes raw performance and ingestion speed (column-based), which is perfect for the key-value workload typical of historical monitoring data.
How ClickHouse Integrates with Zabbix 8.0
It's important to understand that ClickHouse does not replace your primary database. You cannot simply turn off MySQL or PostgreSQL. Your traditional relational database is still strictly required to save all configuration data, maintenance operations, events, and audit logs.
ClickHouse acts as a commodity—an extra back-end layer dedicated solely to containing historical data. This solves the age-old problem of managing massive historical data maps that are difficult to partition or delete in relational databases. It also paves the way for future Zabbix releases to better manage logs and JSON arrays.
Crucial Limitations and Caveats to Know
Before you dive in, there are a few fundamental differences in how Zabbix interacts with ClickHouse compared to traditional databases:
- The Housekeeper Doesn't Work Here: The standard Zabbix Housekeeper process does not delete data on ClickHouse. ClickHouse wasn't designed to delete data row-by-row in that manner. Instead, data retention is managed internally by ClickHouse using a TTL (Time To Live) setting. By default, data is kept for 31 days. If you want to change this, you must alter the TTL directly within the ClickHouse database schema.
- No Trends Calculation: Zabbix does not calculate or save Trend data on ClickHouse. Because ClickHouse is so performant at querying raw data over long periods, aggregating data into separate Trend tables is deemed unnecessary.
- No Binary Data: ClickHouse supports numeric, character, log, text, and JSON data, but it does not support binary data (like the images saved by the item browser).
- No Proxy Support: ClickHouse is not supported for Zabbix Proxies.
- Data Migration: If you want to migrate existing historical data from MySQL to ClickHouse, you will need to manually analyze and adapt the scripts provided by Zabbix.
Step-by-Step: Configuring ClickHouse in Zabbix
Ready to test it out? You can run ClickHouse using Docker (great for testing) or by installing the packages directly on your distribution. Here is a high-level overview of the setup process:
1. Set Up ClickHouse
If you are testing, spinning up a ClickHouse container via Docker is the easiest route. You'll need to define the database name, username, password, and expose the default HTTP/TCP port (8123). Once the container is up, verify that it responds correctly.
2. Initialize the Database Schema
Zabbix 8.0 now includes database scripts specifically for ClickHouse. You will need to run the provided schema creation script against your ClickHouse endpoint. Remember, this is where the default 31-day TTL is defined. If you need a longer retention period, modify the TTL in this schema before or after applying it.
3. Configure the Zabbix Server
Next, we need to instruct the Zabbix Server component to send historical data to ClickHouse instead of the primary database. Open your zabbix_server.conf file and locate the new ClickHouse section. Here, you will define:
- The URL/Endpoint (IP and port 8123)
- The Database name, User, and Password
- Which data types (e.g., UINT, LOG, TEXT) should be routed to ClickHouse. (Remember, keep BIN out of this list!)
4. Configure the Zabbix Web Interface
Finally, the frontend needs to know where to fetch the historical data for your graphs. Open your zabbix.conf.php file. Below your MySQL/PostgreSQL configuration, you will find a new section for the history back-end. Repeat the same endpoint, credentials, and data type routing configurations here.
Once everything is saved, restart your Zabbix Server service. (No need to restart the web interface since it's PHP). When you navigate to your Zabbix frontend and check your graphs, everything will load seamlessly and transparently—just much faster at scale!
The Future: ClickHouse Clusters and Zabbix
One question that naturally arises—and was brought up on the Zabbix support forums—is whether the default schema provided by Zabbix is optimized for a full ClickHouse cluster. The short answer is: not quite yet. You will likely need to optimize the schema for a distributed environment. However, Zabbix developers are actively working on this issue, and we can expect better native cluster support or official guidelines as we move toward the final release candidates.
I will definitely be making more videos on this topic, exploring how the ClickHouse world works, how to manage it, and how to back it up properly.
Let's Discuss!
What do you think of this new ClickHouse feature? While it might not be indispensable for everyone, it is a massive upgrade for enterprise-scale environments. If you have any extra information, tips, or questions, please let me know in the comments below!
Don't forget to subscribe to my YouTube channel, Quadrata, for more in-depth tutorials and updates. Also, if you want to stay in touch and discuss all things Zabbix with the community, join our ZabbixItalia Telegram Channel.
See you next week!
— Dimitri Bellini