NDB 8.4 中的 NDB 复制支持使用通用 MySQL 服务器多线程应用程序机制（MTA），它允许在副本上并行应用独立的二进制日志事务，从而提高复制的峰值吞吐量。NDB replication in NDB 8.4 supports the use of the generic MySQL Server Multithreaded Applier mechanism (MTA), which allows independent binary log transactions to be applied in parallel on a replica, increasing peak replication throughput.

要求Requirements

MySQL 服务器的 MTA 实现将单独的二进制日志事务处理委派给一个工作线程池（其大小可配置），并协调工作线程以确保遵守二进制日志中编码的事务依赖关系，如果需要，还要保持提交顺序（参见 第 19.2.3 节，“复制线程”）。要在 NDB Cluster 中使用此功能，必须将副本配置为使用多个工作线程。为此，请设置 replica_parallel_workers 以控制副本上的工作线程数量。默认值为 4。The MySQL Server MTA implementation delegates the processing of separate binary log transactions to a pool of worker threads (whose size is configurable), and coordinates the worker threads to ensure that transaction dependencies encoded in the binary log are respected, and that commit ordering is maintained if required (see Section 19.2.3, “Replication Threads”). To use this functionality with NDB Cluster, it is necessary that the replica be configured to use multiple worker threads. To do this, set replica_parallel_workers to control the number of worker threads on the replica. The default is 4.

MTA 配置：源MTA Configuration: Source

如果在源 mysqld 上设置，replica_parallel_type 必须为 LOGICAL_CLOCK（默认值）。If set on the source mysqld, replica_parallel_type must be LOGICAL_CLOCK (the default value).

此外，建议您在源端将用于跟踪二进制日志事务写集合的内存量（binlog_transaction_dependency_history_size）设置为 E * P，其中 E 是平均纪元大小（按每个纪元的操作数计算），P 是预期的最大并行度。有关更多信息，请参阅 写集合跟踪内存使用情况。In addition, it is recommended that you set the amount of memory used to track binary log transaction writesets on the source (binlog_transaction_dependency_history_size) to E * P, where E is the average epoch size (as the number of operations per epoch) and P is the maximum expected parallelism. See Writeset Tracking Memory Usage, for more information.

MTA 配置：副本MTA Configuration: Replica

对于 NDB MTA，副本 mysqld 配置要求 replica_parallel_workers 必须大于 1。在首次启用 MTA 时推荐的起始值是 4，这是默认值。Replica mysqld configuration for the NDB MTA requires that replica_parallel_workers is greater than 1. The recommended starting value when first enabling MTA is 4, which is the default.

此外，replica_preserve_commit_order 必须设置为 ON。这也是默认值。In addition, replica_preserve_commit_order must be ON. This is also the default value.

事务依赖和写集处理Transaction Dependency and Writeset Handling

事务依赖通过分析每个事务的写集来检测，即事务写入的行（表、键值）集合。当两个事务修改同一行时，它们被认为是相互依赖的，必须按顺序（换句话说，串行地）执行，以避免死锁或错误的结果。当一个表有辅助唯一键时，这些值也会被添加到事务的写集中，以检测不同事务影响同一唯一键值而暗示事务依赖的情况，从而需要排序。当依赖关系无法高效确定时，mysqld会出于安全考虑退回到将事务视为依赖状态。Transaction dependencies are detected using analysis of each transaction's writeset, that is, the set of rows (table, key values) written by the transaction. Where two transactions modify the same row they are considered to be dependent, and must be applied in order (in other words, serially) to avoid deadlocks or incorrect results. Where a table has secondary unique keys, these values are also added to the transaction's writeset to detect the case where there are transaction dependencies implied by different transactions affecting the same unique key value, and so requiring ordering. Where dependencies cannot be efficiently determined, mysqld falls back to considering transactions dependent for reasons of safety.

事务依赖关系由源 mysqld 编码在二进制日志中。依赖关系使用一种称为“逻辑时钟”的方案在 ANONYMOUS_GTID 事件中进行编码。（参见 第 19.1.4.1 节，“复制模式概念”。）Transaction dependencies are encoded in the binary log by the source mysqld. Dependencies are encoded in an ANONYMOUS_GTID event using a scheme called 'Logical clock'. (See Section 19.1.4.1, “Replication Mode Concepts”.)

MySQL（以及 NDB 集群）采用的写集实现使用基于匹配相关表和索引值的 64 位行哈希的哈希冲突检测。这可以可靠地检测到何时相同的键被两次访问，但如果不同的表和索引值哈希到相同的 64 位值时，也可能产生误报；这可能导致人为依赖，从而降低可用的并行性。The writeset implementation employed by MySQL (and NDB Cluster) uses hash-based conflict detection based on matching 64-bit row hashes of relevant table and index values. This detects reliably when the same key is seen twice, but can also produce false positives if different table and index values hash to the same 64-bit value; this may result in artificial dependencies which can reduce the available parallelism.

事务依赖由以下任一情况强制产生：Transaction dependencies are forced by any of the following:

MySQL MTA 实现尝试并行应用独立的二进制日志事务。NDB 会记录在一个时期（TimeBetweenEpochs，默认 100 毫秒）中提交的所有用户事务中发生的所有更改，并将其作为一个二进制日志事务，该事务称为时期事务。因此，为了使两个连续的时期事务彼此独立，并且可以并行应用，需要保证没有任何行在两个时期中被修改。如果任何单行在两个时期中被修改，那么它们就是互相依赖的，将按顺序应用，这可能限制可利用的并行性。The MySQL MTA implementation attempts to apply independent binary log transactions in parallel. NDB records all changes occurring in all user transactions committing in an epoch (TimeBetweenEpochs, default 100 milliseconds), in one binary log transaction, referred to as an epoch transaction. Therefore, for two consecutive epoch transactions to be independent, and possible to apply in parallel, it is required that no row is modified in both epochs. If any single row is modified in both epochs, then they are dependent, and are applied serially, which can limit the expolitable parallelism available.

基于在 epoch 中源集群上修改的行集合，epoch 事务被视为独立的，但不包括传递 epoch 元数据的生成的mysql.ndb_apply_statusWRITE_ROW事件。这避免了每个 epoch 事务都显式依赖于前一个 epoch，但确实要求在副本上应用 binlog 时必须保持提交顺序。这也意味着，带有 writeset 依赖关系的 NDB 二进制日志不适用于使用不同 MySQL 存储引擎的副本数据库。Epoch transactions are considered independent based on the set of rows modified on the source cluster in the epoch, but not including the generated mysql.ndb_apply_status WRITE_ROW events that convey epoch metadata. This avoids every epoch transaction being trivially dependent on the preceding epoch, but does require that the binlog is applied at the replica with the commit order preserved. This also implies that an NDB binary log with writeset dependencies is not suitable for use by a replica database using a different MySQL storage engine.

可能有必要或可取修改应用程序事务行为，以避免在短时间内在不同事务中重复修改相同行的模式，从而提高可用的应用并行性。It may be possible or desirable to modify application transaction behavior to avoid patterns of repeated modifications to the same rows, in separate transactions over a short time period, to increase exploitable apply parallelism.

Writeset 跟踪内存使用Writeset Tracking Memory Usage

用于跟踪二进制日志事务写集的内存量可以通过服务器系统变量 binlog_transaction_dependency_history_size 设置，默认值为 25000 行哈希。The amount of memory used to track binary log transaction writesets can be set using the binlog_transaction_dependency_history_size server system variable, which defaults to 25000 row hashes.

如果平均二进制日志事务修改 N 行，那么为了能够识别高达 P 并行级别的独立（可并行化）事务，我们需要 binlog_transaction_dependency_history_size 至少为 N * P。（最大值为 1000000。）If an average binary log transaction modifies N rows, then to be able to identify independent (parallelizable) transactions up to a parallelism level of P, we need binlog_transaction_dependency_history_size to be at least N * P. (The maximum is 1000000.)

历史的有限大小导致可以可靠确定的最大依赖长度是有限的，从而可表达的并行度也是有限的。历史中未找到的任何行可能依赖于最后一笔从历史中清除的事务。The finite size of the history results in a finite maximum dependency length that can be reliably determined, giving a finite parallelism that can be expressed. Any row not found in the history may be dependent on the last transaction purged from the history.

写集历史不会像一个滑动窗口那样覆盖最近的N笔事务；相反，它是一个有限的缓冲区，可以完全填满，当缓冲区满时，其内容将被完全丢弃。这意味着历史大小随时间呈锯齿形变化，因此可检测的最大依赖长度也随时间呈锯齿形变化，这样即使在它们处理过程中写集历史缓冲区已经被重置，独立事务仍可能被标记为依赖。Writeset history does not act like a sliding window over the last N transactions; rather, it is a finite buffer which is allowed to fill up completely, then its contents entirely discarded when it becomes full. This means that the history size follows a sawtooth pattern over time, and therefore the maximum detectable dependency length also follows a sawtooth pattern over time, such that independent transactions may still be marked as dependent if the writeset history buffer has been reset between their being processed.

在此方案中，二进制日志文件中的每个事务都会带有一个sequence_number（1、2、3……），以及它所依赖的最近的二进制日志事务的顺序号，我们称之为last_committed。In this scheme, each transaction in a binary log file is annotated with a sequence_number (1, 2, 3, ...), and as well as the sequence number of the most recent binary log transaction that it depends on, to which we refer as last_committed.

在给定的二进制日志文件中，第一个事务的sequence_number为 1，last_committed为 0。Within a given binary log file, the first transaction has sequence_number 1 and last_committed 0.

如果一个二进制日志事务依赖于其直接前序事务，则其应用是串行的。如果依赖于更早的事务，则可能能够将该事务与前面的独立事务并行应用。Where a binary log transaction depends on its immediate predecessor, its application is serialized. If the dependency is on an earlier transaction then it may be possible to apply the transaction in parallel with the preceding independent transactions.

ANONYMOUS_GTID 事件的内容，包括 sequence_number 和 last_committed（从而事务依赖关系），可以使用 mysqlbinlog 查看。The content of ANONYMOUS_GTID events, including sequence_number and last_committed (and thus the transaction dependencies), can be seen using mysqlbinlog.

在源端生成的 ANONYMOUS_GTID 事件与包含批量 BEGIN、TABLE_MAP*、WRITE_ROW*、UPDATE_ROW*、DELETE_ROW* 和 COMMIT 事件的压缩事务负载分开处理，从而可以在解压之前确定依赖关系。这意味着副本协调线程可以将事务负载的解压任务委托给工作线程，从而为副本上的独立事务提供自动的并行解压。The ANONYMOUS_GTID events generated on the source are handled separately from the compressed transaction payload with bulk BEGIN, TABLE_MAP*, WRITE_ROW*, UPDATE_ROW*, DELETE_ROW*, and COMMIT events, allowing dependencies to be determined prior to decompression. This means that the replica coordinator thread can delegate transaction payload decompression to a worker thread, providing automatic parallel decompression of independent transactions on the replica.

已知限制Known Limitations

二级唯一列。具有二级唯一列的表（即除主键之外的唯一键）会将所有列发送到源，以便检测与唯一键相关的冲突。Secondary unique columns.  Tables with secondary unique columns (that is, unique keys other than the primary key) have all columns sent to the source so that unique-key related conflicts can be detected.

当当前的二进制日志模式不包括所有列，而只包括已更改的列（--ndb-log-updated-only=OFF，--ndb-log-update-minimal=ON，--ndb-log-update-as-write=OFF）时，这可能会增加从数据节点发送到 SQL 节点的数据量。Where the current binary logging mode does not include all columns, but only changed columns (--ndb-log-updated-only=OFF, --ndb-log-update-minimal=ON, --ndb-log-update-as-write=OFF), this can increase the volume of data sent from data nodes to SQL nodes.

影响取决于这些表中行的修改（更新或删除）速率以及实际未被修改的列的数据量。The impact depends on both the rate of modification (update or delete) of rows in such tables and the volume of data in columns which are not actually modified.

将 NDB 复制到 InnoDB。 NDB 二进制日志注入器的事务依赖跟踪会故意忽略由生成的 mysql.ndb_apply_status 元数据事件所创建的事务间依赖关系，这些依赖关系会在副本执行器上作为 epoch 事务提交的一部分单独处理。对于复制到 InnoDB 的情况，没有特殊处理；当使用 InnoDB 多线程执行器来消费 NDB MTA 二进制日志时，这可能导致性能下降或其他问题。Replicating NDB to InnoDB.  NDB binary log injector transaction dependency tracking intentionally ignores the inter-transaction dependencies created by generated mysql.ndb_apply_status metadata events, which are handled separately as part of the commit of the epoch transaction on the replica applier. For replication to InnoDB, there is no special handling; this may result in reduced performance or other issues when using an InnoDB multithreaded applier to consume an NDB MTA binary log.