MySQL 服务器能够维护语句摘要信息。摘要处理过程将每个 SQL 语句转换为标准化形式(语句摘要),并从标准化结果计算 SHA-256 哈希值(摘要哈希值)。标准化允许将相似的语句分组并汇总,以展示服务器正在执行的语句类型及其发生频率。对于每个摘要,会存储产生该摘要的代表性语句作为样本。本节描述了语句摘要和采样的发生方式及其用途。The MySQL server is capable of maintaining statement digest information. The digesting process converts each SQL statement to normalized form (the statement digest) and computes a SHA-256 hash value (the digest hash value) from the normalized result. Normalization permits statements that are similar to be grouped and summarized to expose information about the types of statements the server is executing and how often they occur. For each digest, a representative statement that produces the digest is stored as a sample. This section describes how statement digesting and sampling occur and how they can be useful.
无论性能模式是否可用,摘要处理都会在解析器中进行,因此其他功能,如 MySQL 企业防火墙和查询重写插件,可以访问语句摘要。Digesting occurs in the parser regardless of whether the Performance Schema is available, so that other features such as MySQL Enterprise Firewall and query rewrite plugins have access to statement digests.
当解析器接收到 SQL 语句时,如果需要该摘要,则会计算语句摘要。如果满足以下任一条件,则需要计算摘要:When the parser receives an SQL statement, it computes a statement digest if that digest is needed, which is true if any of the following conditions are true:
性能模式摘要检测已启用Performance Schema digest instrumentation is enabled
MySQL 企业防火墙已启用MySQL Enterprise Firewall is enabled
查询重写插件已启用A query rewrite plugin is enabled
解析器还被 STATEMENT_DIGEST_TEXT() 和 STATEMENT_DIGEST() 函数使用,应用程序可以调用这些函数分别从 SQL 语句计算标准化语句摘要和摘要哈希值。The parser is also used by the
STATEMENT_DIGEST_TEXT() and
STATEMENT_DIGEST() functions,
which applications can call to compute a normalized statement
digest and a digest hash value, respectively, from an SQL
statement.
max_digest_length 系统变量的值决定了每个会话可用于计算规范化语句摘要的最大字节数。一旦在摘要计算过程中使用了该空间,就会发生截断:不会再收集解析语句的其他标记,也不会将其纳入摘要值。仅在解析标记的字节数超过该限制后才不同的语句会生成相同的规范化语句摘要,如果进行比较或聚合摘要统计,它们会被视为相同。The max_digest_length system
variable value determines the maximum number of bytes available
per session for computation of normalized statement digests.
Once that amount of space is used during digest computation,
truncation occurs: no further tokens from a parsed statement are
collected or figure into its digest value. Statements that
differ only after that many bytes of parsed tokens produce the
same normalized statement digest and are considered identical if
compared or if aggregated for digest statistics.
将 max_digest_length 系统变量设置为零会禁用摘要生成,这也会禁用需要摘要的服务器功能。Setting the max_digest_length
system variable to zero disables digest production, which also
disables server functionality that requires digests.
在计算出标准化语句之后,会从中计算出 SHA-256 哈希值。此外:After the normalized statement has been computed, a SHA-256 hash value is computed from it. In addition:
如果启用了 MySQL 企业防火墙,它将被调用,并且计算出的摘要对其可用。If MySQL Enterprise Firewall is enabled, it is called and the digest as computed is available to it.
如果启用了任何查询重写插件,它将被调用,并且语句摘要和摘要值对其可用。If any query rewrite plugin is enabled, it is called and the statement digest and digest value are available to it.
如果性能模式(Performance Schema)启用了摘要(digest)检测,它会创建归一化语句摘要的副本,为其分配最多
performance_schema_max_digest_length字节。因此,如果performance_schema_max_digest_length小于max_digest_length,该副本相对于原始摘要会被截断。归一化语句摘要的副本存储在相应的性能模式表中,同时还存储了从原始归一化语句计算的 SHA-256 哈希值。(如果性能模式相对于原始摘要截断了归一化语句摘要的副本,它不会重新计算 SHA-256 哈希值。)If the Performance Schema has digest instrumentation enabled, it makes a copy of the normalized statement digest, allocating a maximum ofperformance_schema_max_digest_lengthbytes for it. Consequently, ifperformance_schema_max_digest_lengthis less thanmax_digest_length, the copy is truncated relative to the original. The copy of the normalized statement digest is stored in the appropriate Performance Schema tables, along with the SHA-256 hash value computed from the original normalized statement. (If the Performance Schema truncates its copy of the normalized statement digest relative to the original, it does not recompute the SHA-256 hash value.)
语句规范化将语句文本转换为更标准的摘要字符串表示形式,同时保留一般语句结构,而去除对结构不必要的信息:Statement normalization transforms the statement text to a more standardized digest string representation that preserves the general statement structure while removing information not essential to the structure:
对象标识符,如数据库和表名,会被保留。Object identifiers such as database and table names are preserved.
字面值会被转换为参数标记。规范化后的语句不会保留诸如名称、密码、日期等信息。Literal values are converted to parameter markers. A normalized statement does not retain information such as names, passwords, dates, and so forth.
注释会被移除,空白会被调整。Comments are removed and whitespace is adjusted.
考虑以下语句:Consider these statements:
SELECT * FROM orders WHERE customer_id=10 AND quantity>20 SELECT * FROM orders WHERE customer_id = 20 AND quantity > 100
为了规范化这些语句,解析器将数据值替换为?并调整空白。两个语句生成相同的规范化形式,因此被认为是“相同”:To normalize these statements, the parser replaces data values
by ? and adjusts whitespace. Both statements
yield the same normalized form and thus are considered
“the same”:
SELECT * FROM orders WHERE customer_id = ? AND quantity > ?
归一化语句包含的信息较少,但仍能代表原始语句。其他具有不同数据值的类似语句具有相同的归一化形式。The normalized statement contains less information but is still representative of the original statement. Other similar statements that have different data values have the same normalized form.
现在考虑这些语句:Now consider these statements:
SELECT * FROM customers WHERE customer_id = 1000 SELECT * FROM orders WHERE customer_id = 1000
在这种情况下,归一化语句不同,因为对象标识符不同:In this case, the normalized statements differ because the object identifiers differ:
SELECT * FROM customers WHERE customer_id = ? SELECT * FROM orders WHERE customer_id = ?
如果归一化产生的语句超过了摘要缓冲区可用的空间(由 max_digest_length 决定),则会发生截断,并且文本以 “...” 结尾。仅在 “...” 后部分不同的长归一化语句被视为相同。考虑这些语句:If normalization produces a statement that exceeds the space
available in the digest buffer (as determined by
max_digest_length), truncation
occurs and the text ends with “...”. Long
normalized statements that differ only in the part that occurs
following the “...” are considered the same.
Consider these statements:
SELECT * FROM mytable WHERE cola = 10 AND colb = 20 SELECT * FROM mytable WHERE cola = 10 AND colc = 20
如果截断恰好发生在AND之后,两条语句都有这种规范化形式:If the cutoff happens to be right after the
AND, both statements have this normalized
form:
SELECT * FROM mytable WHERE cola = ? AND ...
在这种情况下,第二列名称的差异会丢失,两条语句被认为是相同的。In this case, the difference in the second column name is lost and both statements are considered the same.
在性能模式中,语句摘要涉及以下元素:In the Performance Schema, statement digesting involves these elements:
A
statements_digestconsumer in thesetup_consumerstable controls whether the Performance Schema maintains digest information. See Statement Digest Consumer.语句事件表(
events_statements_current、events_statements_history和events_statements_history_long)有用于存储规范化语句摘要及相应摘要 SHA-256 哈希值的列:The statement event tables (events_statements_current,events_statements_history, andevents_statements_history_long) have columns for storing normalized statement digests and the corresponding digest SHA-256 hash values:DIGEST_TEXT是规范化语句摘要的文本。这是原始规范化语句的副本,计算到最大max_digest_length字节,如果需要进一步截断到performance_schema_max_digest_length字节。DIGEST_TEXTis the text of the normalized statement digest. This is a copy of the original normalized statement that was computed to a maximum ofmax_digest_lengthbytes, further truncated as necessary toperformance_schema_max_digest_lengthbytes.DIGEST是从原始规范化语句计算的摘要 SHA-256 哈希值。DIGESTis the digest SHA-256 hash value computed from the original normalized statement.
参见 第 29.12.6 节,“性能模式语句事件表”。See Section 29.12.6, “Performance Schema Statement Event Tables”.
events_statements_summary_by_digest汇总表提供聚合的语句摘要信息。此表按SCHEMA_NAME和DIGEST组合聚合语句信息。性能模式使用 SHA-256 哈希值进行聚合,因为它们计算速度快,并且具有有利的统计分布,可以最小化冲突。请参见 第 29.12.20.3 节,“语句汇总表”。Theevents_statements_summary_by_digestsummary table provides aggregated statement digest information. This table aggregates information for statements perSCHEMA_NAMEandDIGESTcombination. The Performance Schema uses SHA-256 hash values for aggregation because they are fast to compute and have a favorable statistical distribution that minimizes collisions. See Section 29.12.20.3, “Statement Summary Tables”.
一些性能表有一列存储用于计算摘要的原始 SQL 语句:Some Performance Tables have a column that stores original SQL statements from which digests are computed:
events_statements_current、events_statements_history和events_statements_history_long语句事件表的SQL_TEXT列。TheSQL_TEXTcolumn of theevents_statements_current,events_statements_history, andevents_statements_history_longstatement event tables.events_statements_summary_by_digest摘要表的QUERY_SAMPLE_TEXT列。TheQUERY_SAMPLE_TEXTcolumn of theevents_statements_summary_by_digestsummary table.
默认情况下,语句显示的最大可用空间为 1024 字节。要更改此值,请在服务器启动时设置performance_schema_max_sql_text_length系统变量。更改会影响刚才列出的所有列所需的存储空间。The maximum space available for statement display is 1024 bytes
by default. To change this value, set the
performance_schema_max_sql_text_length
system variable at server startup. Changes affect the storage
required for all the columns just named.
performance_schema_max_digest_length 系统变量决定 Performance Schema 中每个语句用于存储摘要值的最大字节数。然而,由于语句元素(如关键字和字面值)的内部编码,语句摘要的显示长度可能会超过可用的缓冲区大小。因此,从语句事件表的 DIGEST_TEXT 列中选择的值可能看起来超过 performance_schema_max_digest_length 值。The
performance_schema_max_digest_length
system variable determines the maximum number of bytes available
per statement for digest value storage in the Performance
Schema. However, the display length of statement digests may be
longer than the available buffer size due to internal encoding
of statement elements such as keywords and literal values.
Consequently, values selected from the
DIGEST_TEXT column of statement event tables
may appear to exceed the
performance_schema_max_digest_length
value.
events_statements_summary_by_digest 汇总表提供了服务器执行语句的概况。它显示了应用程序正在执行的语句类型以及执行的频率。应用程序开发者可以将这些信息与表中的其他信息结合起来评估应用程序的性能特征。例如,显示等待时间、锁定时间或索引使用情况的表列可能会突出显示效率低下的查询类型。这使开发者能够了解应用程序哪些部分需要关注。The
events_statements_summary_by_digest
summary table provides a profile of the statements executed by
the server. It shows what kinds of statements an application is
executing and how often. An application developer can use this
information together with other information in the table to
assess the application's performance characteristics. For
example, table columns that show wait times, lock times, or
index use may highlight types of queries that are inefficient.
This gives the developer insight into which parts of the
application need attention.
events_statements_summary_by_digest 汇总表的大小是固定的。默认情况下,Performance Schema 会在启动时估算要使用的大小。要显式指定表的大小,请在服务器启动时设置 performance_schema_digests_size 系统变量。如果表已满,Performance Schema 会将 SCHEMA_NAME 和 DIGEST 值与表中现有值不匹配的语句分组到一个特殊行中,该行的 SCHEMA_NAME 和 DIGEST 设置为 NULL。这允许统计所有语句的数量。然而,如果该特殊行占执行语句的显著百分比,可能需要通过增加 performance_schema_digests_size 来增大汇总表的大小。The
events_statements_summary_by_digest
summary table has a fixed size. By default the Performance
Schema estimates the size to use at startup. To specify the
table size explicitly, set the
performance_schema_digests_size
system variable at server startup. If the table becomes full,
the Performance Schema groups statements that have
SCHEMA_NAME and DIGEST
values not matching existing values in the table in a special
row with SCHEMA_NAME and
DIGEST set to NULL. This
permits all statements to be counted. However, if the special
row accounts for a significant percentage of the statements
executed, it might be desirable to increase the summary table
size by increasing
performance_schema_digests_size.
对于生成非常长且仅在末尾不同的语句的应用程序,增加max_digest_length可以计算出区分那些原本会聚合到相同摘要的语句的摘要。相反,减少max_digest_length会使服务器在摘要存储上分配更少的内存,但会增加较长语句聚合到相同摘要的可能性。管理员应记住,更大的值会相应地增加内存需求,尤其是对于涉及大量并发会话的工作负载(服务器为每个会话分配max_digest_length字节)。For applications that generate very long statements that differ
only at the end, increasing
max_digest_length enables
computation of digests that distinguish statements that would
otherwise aggregate to the same digest. Conversely, decreasing
max_digest_length causes the
server to devote less memory to digest storage but increases the
likelihood of longer statements aggregating to the same digest.
Administrators should keep in mind that larger values result in
correspondingly increased memory requirements, particularly for
workloads that involve large numbers of simultaneous sessions
(the server allocates
max_digest_length bytes per
session).
如前所述,由解析器计算的规范化语句摘要限制为最大 max_digest_length 字节,而存储在性能模式中的规范化语句摘要使用 performance_schema_max_digest_length 字节。关于 max_digest_length 和 performance_schema_max_digest_length 相对值的内存使用考虑如下:As described previously, normalized statement digests as
computed by the parser are constrained to a maximum of
max_digest_length bytes,
whereas normalized statement digests stored in the Performance
Schema use
performance_schema_max_digest_length
bytes. The following memory-use considerations apply regarding
the relative values of
max_digest_length and
performance_schema_max_digest_length:
如果
max_digest_length小于performance_schema_max_digest_length:Ifmax_digest_lengthis less thanperformance_schema_max_digest_length:除 Performance Schema 外的服务器功能使用规范化语句摘要,占用最多
max_digest_length个字节。Server features other than the Performance Schema use normalized statement digests that take up tomax_digest_lengthbytes.Performance Schema 对其存储的规范化语句摘要不再进行截断,但为每个摘要分配的内存超过
max_digest_length个字节,这是不必要的。The Performance Schema does not further truncate normalized statement digests that it stores, but allocates more memory thanmax_digest_lengthbytes per digest, which is unnecessary.
如果
max_digest_length等于performance_schema_max_digest_length:Ifmax_digest_lengthequalsperformance_schema_max_digest_length:除 Performance Schema 外的服务器功能使用规范化语句摘要,占用最多
max_digest_length个字节。Server features other than the Performance Schema use normalized statement digests that take up tomax_digest_lengthbytes.Performance Schema 不会进一步截断它存储的规范化语句摘要,并且为每个摘要分配与
max_digest_length字节相同的内存。The Performance Schema does not further truncate normalized statement digests that it stores, and allocates the same amount of memory asmax_digest_lengthbytes per digest.
如果
max_digest_length大于performance_schema_max_digest_length:Ifmax_digest_lengthis greater thanperformance_schema_max_digest_length:除 Performance Schema 外的服务器功能使用规范化语句摘要,占用最多
max_digest_length个字节。Server features other than the Performance Schema use normalized statement digests that take up tomax_digest_lengthbytes.Performance Schema 进一步截断它存储的规范化语句摘要,并为每个摘要分配的内存少于
max_digest_length字节。The Performance Schema further truncates normalized statement digests that it stores, and allocates less memory thanmax_digest_lengthbytes per digest.
因为 Performance Schema 语句事件表可能存储许多摘要,将 performance_schema_max_digest_length 设置得比 max_digest_length 小,使管理员能够在以下因素之间取得平衡:Because the Performance Schema statement event tables might
store many digests, setting
performance_schema_max_digest_length
smaller than max_digest_length
enables administrators to balance these factors:
服务器在 Performance Schema 外部功能需要长规范化语句摘要The need to have long normalized statement digests available to server features outside the Performance Schema
许多并发会话,每个会话都会分配摘要计算内存Many concurrent sessions, each of which allocates digest-computation memory
在存储大量语句摘要时,需要限制 Performance Schema 语句事件表的内存消耗The need to limit memory consumption by the Performance Schema statement event tables when storing many statement digests
performance_schema_max_digest_length 设置不是按会话的,它是按语句的,并且一个会话可以在 events_statements_history 表中存储多个语句。该表中每个会话的语句数量通常为 10,因此仅该表,每个会话消耗的内存是 performance_schema_max_digest_length 值的 10 倍。The
performance_schema_max_digest_length
setting is not per session, it is per statement, and a session
can store multiple statements in the
events_statements_history table. A
typical number of statements in this table is 10 per session, so
each session consumes 10 times the memory indicated by the
performance_schema_max_digest_length
value, for this table alone.
此外,全球还收集了许多语句(和摘要),最著名的是 events_statements_history_long 表。在这里,存储 N 条语句会消耗 N 倍于 performance_schema_max_digest_length 值所示的内存。Also, there are many statements (and digests) collected
globally, most notably in the
events_statements_history_long
table. Here, too, N statements stored
consumes N times the memory indicated
by the
performance_schema_max_digest_length
value.
要评估用于 SQL 语句存储和摘要计算的内存量,请使用 SHOW ENGINE
PERFORMANCE_SCHEMA STATUS 语句,或监控以下工具:To assess the amount of memory used for SQL statement storage
and digest computation, use the
SHOW ENGINE
PERFORMANCE_SCHEMA STATUS statement, or monitor these
instruments:
mysql>SELECT NAMEFROM performance_schema.setup_instrumentsWHERE NAME LIKE '%.sqltext';+------------------------------------------------------------------+ | NAME | +------------------------------------------------------------------+ | memory/performance_schema/events_statements_history.sqltext | | memory/performance_schema/events_statements_current.sqltext | | memory/performance_schema/events_statements_history_long.sqltext | +------------------------------------------------------------------+ mysql>SELECT NAMEFROM performance_schema.setup_instrumentsWHERE NAME LIKE 'memory/performance_schema/%.tokens';+----------------------------------------------------------------------+ | NAME | +----------------------------------------------------------------------+ | memory/performance_schema/events_statements_history.tokens | | memory/performance_schema/events_statements_current.tokens | | memory/performance_schema/events_statements_summary_by_digest.tokens | | memory/performance_schema/events_statements_history_long.tokens | +----------------------------------------------------------------------+
Performance Schema 使用语句抽样来收集在 events_statements_summary_by_digest 表中生成每个摘要值的代表性语句。这些列存储样本语句信息:QUERY_SAMPLE_TEXT(语句文本)、QUERY_SAMPLE_SEEN(语句被查看的时间)和 QUERY_SAMPLE_TIMER_WAIT(语句等待或执行时间)。每次选择样本语句时,Performance Schema 都会更新这三列。The Performance Schema uses statement sampling to collect
representative statements that produce each digest value in the
events_statements_summary_by_digest
table. These columns store sample statement information:
QUERY_SAMPLE_TEXT (the text of the
statement), QUERY_SAMPLE_SEEN (when the
statement was seen), and
QUERY_SAMPLE_TIMER_WAIT (the statement wait
or execution time). The Performance Schema updates all three
columns each time it chooses a sample statement.
当插入新的表格行时,生成该行摘要值的语句会被存储为与摘要关联的当前示例语句。此后,当服务器遇到具有相同摘要值的其他语句时,它会确定是否使用新语句来替换当前示例语句(即是否重新采样)。重新采样策略基于当前示例语句和新语句的等待时间比较,并且可选地考虑当前示例语句的年龄:When a new table row is inserted, the statement that produced the row digest value is stored as the current sample statement associated with the digest. Thereafter, when the server sees other statements with the same digest value, it determines whether to use the new statement to replace the current sample statement (that is, whether to resample). Resampling policy is based on the comparative wait times of the current sample statement and new statement and, optionally, the age of the current sample statement:
基于等待时间的重新采样:如果新语句的等待时间大于当前示例语句的等待时间,它将成为当前示例语句。Resampling based on wait times: If the new statement wait time has a wait time greater than that of the current sample statement, it becomes the current sample statement.
基于年龄的重采样:如果
performance_schema_max_digest_sample_age系统变量的值大于零,并且当前采样语句的时间超过该秒数,则当前语句被视为“太旧”,并且新语句会替换它。即使新语句的等待时间小于当前采样语句的等待时间,也会发生这种情况。Resampling based on age: If theperformance_schema_max_digest_sample_agesystem variable has a value greater than zero and the current sample statement is more than that many seconds old, the current statement is considered “too old” and the new statement replaces it. This occurs even if the new statement wait time is less than that of the current sample statement.
默认情况下,performance_schema_max_digest_sample_age 为 60 秒(1 分钟)。要更改样本语句因年龄而“过期”的速度,请增加或减少该值。要禁用基于年龄的重新采样策略部分,请将 performance_schema_max_digest_sample_age 设置为 0。By default,
performance_schema_max_digest_sample_age
is 60 seconds (1 minute). To change how quickly sample
statements “expire” due to age, increase or
decrease the value. To disable the age-based part of the
resampling policy, set
performance_schema_max_digest_sample_age
to 0.