NDBCLUSTER(也称为 NDB)是一种内存存储引擎,提供高可用性和数据持久性功能。NDBCLUSTER
(also known as NDB) is an in-memory
storage engine offering high-availability and data-persistence
features.
NDBCLUSTER 存储引擎可以配置多种故障转移和负载均衡选项,但最简单的方法是在集群级别开始配置存储引擎。NDB Cluster 的 NDB 存储引擎包含完整的数据集,仅依赖于集群内部的其他数据。The NDBCLUSTER storage engine can be
configured with a range of failover and load-balancing options,
but it is easiest to start with the storage engine at the cluster
level. NDB Cluster's NDB storage
engine contains a complete set of data, dependent only on other
data within the cluster itself.
NDB Cluster 的 “集群” 部分独立于 MySQL 服务器进行配置。在 NDB Cluster 中,集群的每个部分都被视为一个 节点。The “Cluster” portion of NDB Cluster is configured independently of the MySQL servers. In an NDB Cluster, each part of the cluster is considered to be a node.
在许多情况下,术语“节点”用于表示计算机,但在讨论 NDB Cluster 时,它指的是一个进程。在一台计算机上可以运行多个节点;对于运行一个或多个集群节点的计算机,我们使用术语集群主机。In many contexts, the term “node” is used to indicate a computer, but when discussing NDB Cluster it means a process. It is possible to run multiple nodes on a single computer; for a computer on which one or more cluster nodes are being run we use the term cluster host.
集群节点有三种类型,在一个最小的 NDB Cluster 配置中,至少有三个节点,每种类型各一个:There are three types of cluster nodes, and in a minimal NDB Cluster configuration, there are at least three nodes, one of each of these types:
管理节点:这种类型节点的作用是管理 NDB 集群中的其他节点,执行诸如提供配置数据、启动和停止节点以及运行备份等功能。由于该类型节点管理其他节点的配置,因此应该先启动此类型的节点,然后再启动任何其他节点。可以使用命令 ndb_mgmd 来启动管理节点。Management node: The role of this type of node is to manage the other nodes within the NDB Cluster, performing such functions as providing configuration data, starting and stopping nodes, and running backups. Because this node type manages the configuration of the other nodes, a node of this type should be started first, before any other node. A management node is started with the command ndb_mgmd.
数据节点:这种类型的节点用于存储集群数据。数据节点的数量等于片段副本的数量乘以片段的数量(参见 第 25.2.2 节,“NDB 集群节点、节点组、片段副本和分区”)。例如,对于两个片段副本,每个副本有两个片段,则需要四个数据节点。一个片段副本足以存储数据,但无法提供冗余;因此,推荐使用两个(或更多)片段副本以提供冗余,从而实现高可用性。数据节点可以使用命令 ndbd 启动(参见 第 25.5.1 节,“ndbd — NDB 集群数据节点守护进程”)或 ndbmtd 启动(参见 第 25.5.3 节,“ndbmtd — NDB 集群数据节点守护进程(多线程)”)。Data node: This type of node stores cluster data. There are as many data nodes as there are fragment replicas, times the number of fragments (see Section 25.2.2, “NDB Cluster Nodes, Node Groups, Fragment Replicas, and Partitions”). For example, with two fragment replicas, each having two fragments, you need four data nodes. One fragment replica is sufficient for data storage, but provides no redundancy; therefore, it is recommended to have two (or more) fragment replicas to provide redundancy, and thus high availability. A data node is started with the command ndbd (see Section 25.5.1, “ndbd — The NDB Cluster Data Node Daemon”) or ndbmtd (see Section 25.5.3, “ndbmtd — The NDB Cluster Data Node Daemon (Multi-Threaded)”).
NDB 集群表通常完全存储在内存中,而不是在磁盘上(这就是为什么我们称 NDB 集群为内存数据库)。然而,一些 NDB 集群数据可以存储在磁盘上;有关更多信息,请参见第 25.6.11 节,“NDB 集群磁盘表”。NDB Cluster tables are normally stored completely in memory rather than on disk (this is why we refer to NDB Cluster as an in-memory database). However, some NDB Cluster data can be stored on disk; see Section 25.6.11, “NDB Cluster Disk Data Tables”, for more information.
SQL 节点:这是一个访问集群数据的节点。在 NDB 集群的情况下,SQL 节点是使用
NDBCLUSTER存储引擎的传统 MySQL 服务器。SQL 节点是一个mysqld进程,通过--ndbcluster和--ndb-connectstring选项启动,这些选项在本章其他地方有说明,也可能带有其他 MySQL 服务器选项。SQL node: This is a node that accesses the cluster data. In the case of NDB Cluster, an SQL node is a traditional MySQL server that uses theNDBCLUSTERstorage engine. An SQL node is a mysqld process started with the--ndbclusterand--ndb-connectstringoptions, which are explained elsewhere in this chapter, possibly with additional MySQL server options as well.SQL 节点实际上只是 API 节点 的一种特殊类型,它指定任何访问 NDB 集群数据的应用程序。API 节点的另一个例子是 ndb_restore 工具,用于恢复集群备份。可以使用 NDB API 编写此类应用程序。有关 NDB API 的基本信息,请参见 NDB API 入门。An SQL node is actually just a specialized type of API node, which designates any application which accesses NDB Cluster data. Another example of an API node is the ndb_restore utility that is used to restore a cluster backup. It is possible to write such applications using the NDB API. For basic information about the NDB API, see Getting Started with the NDB API.
在生产环境中期望使用三节点设置是不现实的。这种配置没有冗余;要利用 NDB Cluster 的高可用性功能,您必须使用多个数据节点和 SQL 节点。同时也强烈建议使用多个管理节点。It is not realistic to expect to employ a three-node setup in a production environment. Such a configuration provides no redundancy; to benefit from NDB Cluster's high-availability features, you must use multiple data and SQL nodes. The use of multiple management nodes is also highly recommended.
有关 NDB Cluster 中节点、节点组、片段副本和分区之间关系的简要介绍,请参见 第 25.2.2 节,“NDB Cluster 节点、节点组、片段副本和分区”。For a brief introduction to the relationships between nodes, node groups, fragment replicas, and partitions in NDB Cluster, see Section 25.2.2, “NDB Cluster Nodes, Node Groups, Fragment Replicas, and Partitions”.
集群的配置包括配置集群中的每个节点以及设置节点之间的单独通信链路。NDB 集群当前的设计目标是使数据节点在处理器性能、内存空间和带宽方面保持同质性。此外,为了提供单点配置,整个集群的所有配置数据都位于一个配置文件中。Configuration of a cluster involves configuring each individual node in the cluster and setting up individual communication links between nodes. NDB Cluster is currently designed with the intention that data nodes are homogeneous in terms of processor power, memory space, and bandwidth. In addition, to provide a single point of configuration, all configuration data for the cluster as a whole is located in one configuration file.
管理服务器管理集群配置文件和集群日志。集群中的每个节点都从管理服务器获取配置数据,因此需要一种方法来确定管理服务器的位置。当数据节点发生重要事件时,节点将这些事件的信息传输到管理服务器,然后管理服务器将信息写入集群日志。The management server manages the cluster configuration file and the cluster log. Each node in the cluster retrieves the configuration data from the management server, and so requires a way to determine where the management server resides. When interesting events occur in the data nodes, the nodes transfer information about these events to the management server, which then writes the information to the cluster log.
此外,可以有任意数量的集群客户端进程或应用程序。这些包括标准 MySQL 客户端、NDB 特定的 API 程序,以及管理客户端。这些将在接下来的几段中描述。In addition, there can be any number of cluster client processes
or applications. These include standard MySQL clients,
NDB-specific API programs, and management
clients. These are described in the next few paragraphs.
标准 MySQL 客户端。 NDB Cluster 可以与用 PHP、Perl、C、C++、Java、Python 等编写的现有 MySQL 应用程序一起使用。这些客户端应用程序向作为 NDB Cluster SQL 节点的 MySQL 服务器发送 SQL 语句并接收响应,其方式与与独立 MySQL 服务器交互的方式大致相同。Standard MySQL clients. NDB Cluster can be used with existing MySQL applications written in PHP, Perl, C, C++, Java, Python, and so on. Such client applications send SQL statements to and receive responses from MySQL servers acting as NDB Cluster SQL nodes in much the same way that they interact with standalone MySQL servers.
使用 NDB 集群作为数据源的 MySQL 客户端可以修改以利用连接到多个 MySQL 服务器的能力,从而实现负载均衡和故障切换。例如,使用 Connector/J 5.0.6 及更高版本的 Java 客户端可以使用 jdbc:mysql:loadbalance:// URL(在 Connector/J 5.1.7 中有所改进)来透明地实现负载均衡;有关使用 Connector/J 与 NDB 集群的更多信息,请参见 使用 Connector/J 与 NDB 集群。MySQL clients using an NDB Cluster as a data source can be
modified to take advantage of the ability to connect with multiple
MySQL servers to achieve load balancing and failover. For example,
Java clients using Connector/J 5.0.6 and later can use
jdbc:mysql:loadbalance:// URLs (improved in
Connector/J 5.1.7) to achieve load balancing transparently; for
more information about using Connector/J with NDB Cluster, see
Using Connector/J with NDB Cluster.
NDB 客户端程序。 可以编写客户端程序,直接从 NDBCLUSTER 存储引擎访问 NDB Cluster 数据,绕过可能连接到集群的任何 MySQL 服务器,使用 NDB API,一个高级 C API。这类应用程序在不需要 SQL 接口访问数据的特殊情况下可能很有用。欲了解更多信息,请参见 NDB API。NDB client programs.
Client programs can be written that access NDB Cluster data
directly from the NDBCLUSTER storage engine,
bypassing any MySQL Servers that may be connected to the
cluster, using the NDB
API, a high-level C++ API. Such applications may be
useful for specialized purposes where an SQL interface to the
data is not needed. For more information, see
The NDB API.
NDB 特定的 Java 应用程序也可以使用 NDB Cluster Connector for Java 为 NDB Cluster 编写。这个 NDB Cluster Connector 包含 ClusterJ,一个高级数据库 API,类似于对象关系映射持久化框架,如 Hibernate 和 JPA,能够直接连接到 NDBCLUSTER,因此不需要访问 MySQL 服务器。有关更多信息,请参阅 Java 与 NDB Cluster 以及 ClusterJ API 与数据对象模型。NDB-specific Java applications can also be
written for NDB Cluster using the NDB
Cluster Connector for Java. This NDB Cluster Connector
includes ClusterJ, a
high-level database API similar to object-relational mapping
persistence frameworks such as Hibernate and JPA that connect
directly to NDBCLUSTER, and so does not require
access to a MySQL Server. See
Java and NDB Cluster, and
The ClusterJ API and Data Object Model, for more
information.
NDB 集群还支持使用 Node.js 编写的 JavaScript 应用程序。JavaScript 的 MySQL 连接器包括用于直接访问 NDB 存储引擎以及 MySQL 服务器的适配器。使用此连接器的应用程序通常是事件驱动的,并使用在许多方面类似于 ClusterJ 的域对象模型。欲了解更多信息,请参见 MySQL JavaScript NoSQL 连接器。NDB Cluster also supports applications written in JavaScript using
Node.js. The MySQL Connector for JavaScript includes adapters for
direct access to the NDB storage engine and as
well as for the MySQL Server. Applications using this Connector
are typically event-driven and use a domain object model similar
in many ways to that employed by ClusterJ. For more information,
see MySQL NoSQL Connector for JavaScript.
管理客户端。 这些客户端连接到管理服务器,并提供用于优雅地启动和停止节点、启动和停止消息跟踪(仅限调试版本)、显示节点版本和状态、启动和停止备份等命令。此类程序的示例是 NDB Cluster 提供的 ndb_mgm 管理客户端(参见 第 25.5.5 节,“ndb_mgm — NDB Cluster 管理客户端”)。此类应用程序可以使用 MGM API 编写,这是一种 C 语言 API,可直接与一个或多个 NDB Cluster 管理服务器通信。有关更多信息,请参见 MGM API。Management clients. These clients connect to the management server and provide commands for starting and stopping nodes gracefully, starting and stopping message tracing (debug versions only), showing node versions and status, starting and stopping backups, and so on. An example of this type of program is the ndb_mgm management client supplied with NDB Cluster (see Section 25.5.5, “ndb_mgm — The NDB Cluster Management Client”). Such applications can be written using the MGM API, a C-language API that communicates directly with one or more NDB Cluster management servers. For more information, see The MGM API.
Oracle 还提供了 MySQL Cluster Manager,它提供了一个高级命令行界面,简化了许多复杂的 NDB Cluster 管理任务,例如重启具有大量节点的 NDB Cluster。MySQL Cluster Manager 客户端还支持获取和设置大多数节点配置参数的值,以及与 NDB Cluster 相关的 mysqld 服务器选项和变量。更多信息请参见 MySQL Cluster Manager 8.4.9 用户手册。Oracle also makes available MySQL Cluster Manager, which provides an advanced command-line interface simplifying many complex NDB Cluster management tasks, such restarting an NDB Cluster with a large number of nodes. The MySQL Cluster Manager client also supports commands for getting and setting the values of most node configuration parameters as well as mysqld server options and variables relating to NDB Cluster. See MySQL Cluster Manager 8.4.9 User Manual, for more information.
事件日志。 NDB 集群按类别(启动、关闭、错误、检查点等)、优先级和严重性记录事件。所有可报告事件的完整列表可在 第 25.6.3 节,“NDB 集群生成的事件报告” 中找到。事件日志有以下两种类型:Event logs. NDB Cluster logs events by category (startup, shutdown, errors, checkpoints, and so on), priority, and severity. A complete listing of all reportable events may be found in Section 25.6.3, “Event Reports Generated in NDB Cluster”. Event logs are of the two types listed here:
集群日志:记录整个集群中所有需要报告的事件。Cluster log: Keeps a record of all desired reportable events for the cluster as a whole.
节点日志:为每个单独节点单独保存的日志。Node log: A separate log which is also kept for each individual node.
在正常情况下,仅保留和检查集群日志是必要且足够的。节点日志仅在应用开发和调试时需要查看。Under normal circumstances, it is necessary and sufficient to keep and examine only the cluster log. The node logs need be consulted only for application development and debugging purposes.
检查点。 一般来说,当数据被保存到磁盘时,就说达到了一个 检查点。更具体地说,对于 NDB Cluster,检查点是一个时间点,在这个时间点上所有已提交的事务都被存储到磁盘中。关于 NDB 存储引擎,有两种类型的检查点,它们协同工作以确保集群数据的一致视图得以保持。如下所示:Checkpoint.
Generally speaking, when data is saved to disk, it is said that
a checkpoint has been
reached. More specific to NDB Cluster, a checkpoint is a point
in time where all committed transactions are stored on disk.
With regard to the NDB storage
engine, there are two types of checkpoints which work together
to ensure that a consistent view of the cluster's data is
maintained. These are shown in the following list:
本地检查点 (LCP):这是特定于单个节点的检查点;然而,LCPs 几乎同时在集群的所有节点上发生。LCP 通常每隔几分钟发生一次;具体间隔会有所不同,取决于节点存储的数据量、集群的活动水平及其他因素。Local Checkpoint (LCP): This is a checkpoint that is specific to a single node; however, LCPs take place for all nodes in the cluster more or less concurrently. An LCP usually occurs every few minutes; the precise interval varies, and depends upon the amount of data stored by the node, the level of cluster activity, and other factors.
NDB 8.4 支持部分 LCP,这在某些情况下可以显著提高性能。请参阅
EnablePartialLcp和RecoveryWork配置参数的说明,这些参数用于启用部分 LCP 并控制它们使用的存储量。NDB 8.4 supports partial LCPs, which can significantly improve performance under some conditions. See the descriptions of theEnablePartialLcpandRecoveryWorkconfiguration parameters which enable partial LCPs and control the amount of storage they use.全局检查点 (GCP):每隔几秒就会发生一次 GCP,此时所有节点的事务都会同步,重做日志会刷新到磁盘。Global Checkpoint (GCP): A GCP occurs every few seconds, when transactions for all nodes are synchronized and the redo-log is flushed to disk.
有关本地检查点和全局检查点创建的文件和目录的更多信息,请参见 NDB 集群数据节点文件系统目录。For more information about the files and directories created by local checkpoints and global checkpoints, see NDB Cluster Data Node File System Directory.
传输器。 我们使用术语 传输器 来表示数据节点之间使用的数据传输机制。MySQL NDB 集群 8.4 支持其中三种,列在此处:Transporter. We use the term transporter for the data transport mechanism employed between data nodes. MySQL NDB Cluster 8.4 supports three of these, which are listed here:
基于以太网的 TCP/IP。请参见 第 25.4.3.10 节,“NDB 集群 TCP/IP 连接”。TCP/IP over Ethernet. See Section 25.4.3.10, “NDB Cluster TCP/IP Connections”.
直接 TCP/IP。使用机器到机器的连接。参见 第 25.4.3.11 节,“使用直接连接的 NDB Cluster TCP/IP 连接”。Direct TCP/IP. Uses machine-to-machine connections. See Section 25.4.3.11, “NDB Cluster TCP/IP Connections Using Direct Connections”.
虽然这个传输器使用与前一项中提到的相同的 TCP/IP 协议,但它需要不同的硬件设置,并且配置也不同。因此,它被认为是 NDB Cluster 的一种独立传输机制。Although this transporter uses the same TCP/IP protocol as mentioned in the previous item, it requires setting up the hardware differently and is configured differently as well. For this reason, it is considered a separate transport mechanism for NDB Cluster.
共享内存 (SHM)。参见 第 25.4.3.12 节,“NDB Cluster 共享内存连接”。Shared memory (SHM). See Section 25.4.3.12, “NDB Cluster Shared-Memory Connections”.
由于其普遍性,大多数用户在 NDB Cluster 中使用以太网上的 TCP/IP。Because it is ubiquitous, most users employ TCP/IP over Ethernet for NDB Cluster.
无论使用哪种传输器,NDB 都会尝试确保数据节点进程之间的通信尽可能使用较大的块,因为这对所有类型的数据传输都有好处。Regardless of the transporter used, NDB
attempts to make sure that communication between data node
processes is performed using chunks that are as large as possible
since this benefits all types of data transmission.