One-click multi-node TimescaleDB: announcing the support for multi-node deployments in Timescale Cloud

One-click multi-node TimescaleDB: announcing the support for multi-node deployments in Timescale Cloud

Four weeks ago, we kicked off #AlwaysBeLaunching - an ambitious effort to execute 10+ launches throughout the month of October - by sharing our vision for Timescale Cloud: a database cloud for relational and time-series workloads, built on PostgreSQL, and architected as a modern data platform combining the ease of use expected from a cloud service with the flexibility that developers need.

And we kept a relentless pace: in the past weeks, we’ve announced storage autoscaling and larger storage capacities on Timescale Cloud; support for OpenTelemetry traces in Promscale; the introduction of functional programming in PostgreSQL through function pipelines; a comprehensive benchmark comparing ClickHouse to Postgres/TimescaleDB; a new way to downsample your observability data in Promscale; continuous aggregates for distributed hypertables; the launch of Timescale’s first additional regions in North America and Europe; and the Timescale NFT Starter Kit.

We could have left it there, but we thought - what better way to wrap up #AlwaysBeLaunching than by bringing it full-circle and getting back to where we started, aka Timescale Cloud?

So today, we're bringing you one extra launch 🔥.

We’re releasing easy, transparent, and production-ready support for multi-node deployments on Timescale Cloud, available immediately. Users can now create and configure multi-node services exactly the same way as with single node services - in one click and without leaving the UI.

Now, you can create a multi-node service in seconds in Timescale Cloud.

With today’s release, Timescale Cloud introduces a number of features to help developers balance the productivity they expect with the flexibility they need - also in a multi-node environment:

  • Create a multi-node Timescale Cloud service in one click. Want to use multi-node TimescaleDB? Just pick the “multi-node” option when creating your service. You can also pick the region of your preference, plus modify the configuration of your multi-node cluster (number of nodes plus CPU/RAM/storage per node)... Or just use the default configuration, and change it anytime later.
  • Scale compute and storage per node independently. In Timescale Cloud, the compute and storage plans are decoupled. If you’re not sure about your optimal compute per node, you don’t have to worry about it - just pick an initial configuration, and scale it up and down according to the needs of your workload. Similarly for your storage: you can start small, and scale up your plan progressively and without downtime as your data volume grows.
  • Monitor the internals of your multi-node database transparently and from the UI. You can use the Explorer, your easy-to-use operations center, to inspect the internals of your multi-node database - getting a quick overview of the state of your distributed hypertables, data retention policies, and general information about the state of your cluster. Plus, you inspect metrics and logs straight from the console in the same way as with single node services.

If you are a Timescale Cloud user, you can create and manage multi-node services starting today 🎉 . Just click on "Create Service" and select "Advanced Options".

If you’re new to Timescale, create a free account to get started with a fully-managed Timescale Cloud service (100% free for 30 days, no credit card required).

Once you are using TimescaleDB, please join the Timescale community and ask any questions you may have about time-series data, databases, and more. And, for those who share our mission and want to join our fully remote, global team: we are hiring broadly across many roles.

Want more information about multi-node? Keep reading!

PS: We recently published a blog post announcing the support for continuous aggregation on multi-node TimescaleDB. Check it out to learn how multi-node and continuous aggregates work together!

How does multi-node TimescaleDB work?

As a user, you interact with multi-node TimescaleDB similarly to a single-node TimescaleDB instance. For example: in single-node TimescaleDB, data is automatically partitioned across the time dimension into what we call chunks. Together, chunks form a hypertable, which looks like a regular Postgres table to users. TimescaleDB takes care of this time-based partitioning automatically and in the background, so (for the most part) you don’t have to worry about chunking when interacting with your data.

The same concept expands naturally to multiple nodes. In single-node TimescaleDB, hypertables are hosted on the same instance, whether on the same or different disks. In multi-node TimescaleDB, chunks can now be distributed across different instances, forming what we call a distributed hypertable.

Distributed hypertables maintain the illusion (and user experience) of a single continuous table across all time; in other words, you also interact with distributed hypertables as if they were regular Postgres tables. But now, TimescaleDB is able to process your requests in parallel - giving you access to greater scalability and a boost in query performance.

In regular hypertables, TimescaleDB automatically partitions the data across the time dimension into chunks. Chunks are hosted on the same instance, whether on the same or different disks.
In distributed hypertables, chunks are partitioned by time and space and placed in different instances - allowing the parallel processing of your requests, with the subsequent scalability and performance improvements.

When configuring multi-node TimescaleDB in Timescale Cloud, you create an access node (abbreviated as AN) and three or more data nodes (abbreviated as DNs). The access node stores metadata for the distributed hypertable and performs the query planning across the cluster, while the data nodes store individual (or replicated) chunks of the distributed hypertable. As data continues to be ingested, the access node automatically creates new chunks on the corresponding data nodes, according to the corresponding time and space partitions.

Diagram illustrating the architecture of multi-node TimescaleDB, divided into access nodes (AN) and data nodes (DNs). When a distributed hypertable experiences time-ordered inserts, the "space" dimension determines which data node a chunk will be placed on.

As we’ve shared in previous announcements, TimescaleDB multi-node helps developers meet the needs of the most demanding time-series workloads - both in terms of scalability and performance. With Multi-node TimescaleDB, you can optimize your ingests and query responses as your time-series workloads grow in size and scope.

In our initial testing of Timescale Cloud multi-node, we saw a +260% increase in ingest performance when moving from a single-node deployment to a multi-node cluster with 3 data nodes. When we tested a cluster with 5 data nodes, we saw a +400% increase, ingesting over a million rows per second.  

Timescale Cloud’s ingest performance as a function of cluster size for 1, 3, and 5 data nodes. (For this test, we used the Time Series Benchmark Suite with a time interval of 4 days, 10 seconds between events, cardinality of 4k hosts, and 4h chunks - making a total of 100 million rows and 1 billion metrics).

Our tests also revealed significant improvements in terms of query performance. We saw a -45% reduction in query response time when comparing a single-node service to a multi-node cluster with 3 data nodes. In the case of a 5-data-node deployment, we saw a -67% reduction - in other words, queries got x3 times faster.

Timescale Cloud’s query response time as a function of cluster size for 1, 3, and 5 data nodes. (We used the Time Series Benchmark Suite with a time interval of 4 days, 10 seconds between events, cardinality of 4k hosts, and 4h chunks - making a total of 100 million rows and 1 billion metrics. These results correspond to an aggregate query across all CPU metrics over-5 minute intervals - see our GitHub for more details on the queries we use in the Time Series Benchmark Suite.)

Setting up multi-node services in Timescale Cloud

With today’s release, you can set up a multi-node service in Timescale Cloud in a few seconds, involving almost exactly the same steps as if you were creating a single-node service.

To set up your multi-node cluster, start by clicking on “Create Service” in Timescale Cloud. Then, select “Advanced Options”. In the resulting configuration screen, you can click on “Create Database” to immediately deploy a multi-node cluster with the default configuration options... Or you can define them yourself:

  • Select the region you’d like your service to run on, us-east-1, us-west-2, or eu-west-1 (there’ll be more regions coming up in the future!)
  • Select your cluster size (3, 4, or 5 data nodes)
  • Select the initial compute and storage for each of your nodes. Compute and storage are fully decoupled in Timescale Cloud: you can always adjust your compute resources up and down according to the specific needs of your workload, and scale your storage as your data volume increases.

With just a few clicks, you’re ready to meet the needs of the most demanding time-series workloads!

Demo video: multi-node services in action

In this video, Timescale team member Carlota Soto gives you an overview of how multi-node works in Timescale Cloud, walking you through all the steps required from creating a multi-node service until running your first query.

Get started with multi-node in Timescale Cloud

For existing Timescale Cloud users, you can create and manage multi-node services starting today. If you’re new to Timescale, create a free account to get started with a fully-managed Timescale Cloud service (100% free for 30 days, no credit card required).

Once you are using Timescale Cloud, please join the Timescale community to ask us any questions you may have about multi-node, TimescaleDB, time-series data, databases, and more!

Do you share our mission of helping developers worldwide? 🌎 Consider joining our fully remote, global team: we are hiring broadly across many roles!

Special shoutout to Anthony Dodd, Cam Hutchison, Danil Zhigalin, Feike Steenbergen, Igor Chuev, Ivan Tolstosheev, James Hong, Kim Schumacher, Lee Hampton, Monae Payne, Nicholas Calibey, Nicholas Nosenzo, Oleksii Kliukin, Rob Kiefer, Samuel Gichohi, Solar Olugebefola, Ted Sczelecki, Camila Hirthe, Niksa Jakovljevic, Erik Nordström, Nikhil Sontakke, Dmitry Simonenko, and everyone else working on this release.

This post was written by
7 min read
Cloud
Contributors

Related posts

TimescaleDB - Timeseries database for PostgreSQL

Explore TimescaleDB

Learn more about how TimescaleDB works, compare versions, and get technical guidance and tutorials.

Go to docs Go to products