Compatibility requirements for ZDM Proxy

ZDM Proxy supports protocol versions v3, v4, DSE_V1, and DSE_V2.

ZDM Proxy technically doesn’t support v5. If v5 is requested, the proxy handles protocol negotiation so that the client application properly downgrades the protocol version to v4. This means that you can use ZDM Proxy with any client application that uses a driver version supporting protocol version v5, as long as the application doesn’t use v5-specific functionality.

You can use ZDM Proxy to support migrations between Astra, DataStax Enterprise (DSE), Hyper-Converged Database (HCD), open-source Apache Cassandra®, and and other Cassandra-based databases. Supported migration paths include cross-platform migrations and same-platform upgrades.

DataStax tests ZDM Proxy compatibility with Astra, DSE, HCD, and open-source Cassandra. Support for other Cassandra-based databases is possible if the origin and target clusters share a common protocol version. However, DataStax doesn’t test all data store providers, and DataStax doesn’t guarantee full support for any specific data store aside from the platforms and versions listed here.

ZDM Proxy doesn’t modify or transform CQL statements with the exception of optional replacement for now().

A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both clusters. This means that any keyspace that your client application uses must exist on both the origin and target clusters with the same keyspace name, table names, column names, and compatible data types. However, the keyspaces can have different replication strategies and durable write settings.

At the column level, the schema doesn’t need to be a one-to-one match as long as the CQL statements can be executed successfully on both clusters. For example, if a table has 10 columns, and your client application uses only five of those columns, then you can recreate the table on the target cluster with only the five required columns. However, the data from the other columns won’t be migrated to the target cluster if those columns don’t exist on the target cluster. Before you decide to omit a column from the target cluster, make sure that it is acceptable to permanently lose that data after the migration.

In some cases, you can change the primary key on the target cluster. For example, if your compound primary key is PRIMARY KEY (A, B), and you always provide parameters for the A and B columns in your CQL statements, then you could change the key to PRIMARY KEY (B, A) when creating the schema on the target because your CQL statements will still run successfully.

See How ZDM Proxy handles reads and writes.

Non-idempotent CQL operations don’t produce the exact same result each time the request is executed. Whether the request is executed and the actual value written to the database depend on the column’s current value or other conditions at the time of the request. Examples of non-idempotent operations include the following:

By design, a ZDM migration involves two separate clusters, and it is possible for the clusters to have differences due to non-idempotent operations.

If you use non-idempotent operations, DataStax recommends adding a reconciliation phase to your migration before and after Phase 4. This allows you an additional opportunity to resolve any data inconsistencies that are produced by non-idempotent operations. The Cassandra Data Migrator is ideal for detecting and reconciling these types of inconsistencies.

However, if your application workloads can tolerate inconsistencies produced by LWTs and non-idempotent operations, you might not need to perform any additional validation or reconciliation steps. This depends entirely on your application business logic and requirements. It is your responsibility to determine whether your workloads can tolerate these inconsistencies and to what extent.

Throughout the migration process, you must restart ZDM Proxy instances to apply configuration changes. Client applications handle this in the same way as typical rolling restarts of DSE or Cassandra clusters.

If your applications already tolerate rolling restarts of your origin cluster, then you shouldn’t expect any issues during rolling restarts of ZDM Proxy instances.

To ensure that your client application retries requests that fail due to connection errors caused by the rolling restart process, check your driver’s retry policies and whether your requests are marked as idempotent. Most drivers treat all statements as non-idempotent by default, and they don’t automatically retry them. This means that you must explicitly mark statements as idempotent to trigger retries after connection errors. For more information, see Query idempotence in Cassandra drivers and Cassandra driver retry policies.

The binary protocol used by Astra, DSE, HCD, and open-source Cassandra supports optional compression of transport-level requests and responses that reduces network traffic at the cost of CPU overhead.

When establishing connections from client applications, ZDM Proxy responds with a list of compression algorithms supported by both clusters. The compression algorithm configured in your DataStax-compatible driver must match any item from the common list, or CQL request compression must be disabled completely. ZDM Proxy cannot decompress and recompress CQL requests using different compression algorithms.

This isn’t related to storage compression, which you can configure on specific tables with the compression table property. Storage/table compression doesn’t affect the client application or ZDM Proxy in any way.

A cluster’s authorizer doesn’t affect client applications or ZDM Proxy, which means that you can use any kind of authorizer configuration on your clusters, and they can use different authorizers.

In contrast, a cluster’s authenticator must be compatible with ZDM Proxy. ZDM Proxy supports the following cluster authenticator configurations:

ZDM Proxy doesn’t support DseAuthenticator with kerberos scheme.

The origin and target clusters can have different authentication configurations because ZDM Proxy treats them independently.

This section describes how ZDM Proxy handles certain DSE advanced workloads. For more DSE-specific migration considerations, see Migrate to DataStax Enterprise.

This section describes specific considerations for Astra migrations. If you need to make any changes to your data model or application logic for compatibility with Astra, do so before starting the migration. As mentioned previously, this is because ZDM Proxy requires that the same CQL statements can be executed successfully on both clusters, and the data migration tools require matching schemas.

Complex migration scenarios, such as multi-datacenter migrations or many-to-one migrations, require additional planning to configure ZDM Proxy and migrate the data efficiently.

Multi-region migrations require additional planning and action depending on factors like the amount of data, number of regions, data consistency requirements, and orchestration of multi-region traffic.

Multi-region migrations can include one or more of the following scenarios:

For relatively small migrations to Astra DB with consistent data across all regions, you can migrate the primary region first, and then add additional regions after the initial migration is complete. This strategy allows Astra DB’s eventual consistency model to replicate the data from the primary region to the additional regions. However, this approach isn’t suitable for all migrations.

It is difficult to provide a one-size-fits-all solution for multi-region migrations due to the potential complexity and variability of these scenarios. For assistance planning a multi-region migration, contact your DataStax account representative or DataStax Support.

To configure ZDM Proxy for a multi-datacenter migration, see the ZDM Proxy infrastructure guidelines for multi-datacenter clusters.

For more guidance on migrations to Astra, see the Astra DB Sideloader preparations for specific migration scenarios.

Next, prepare the ZDM Proxy infrastructure.