内置的 MySQL 全文解析器使用单词之间的空格作为分隔符来确定单词的起始和结束,这在处理不使用单词分隔符的表意文字语言时存在局限性。为了解决日语的这一局限性,MySQL 提供了 MeCab 全文解析器插件。MeCab 全文解析器插件支持与 InnoDB 和 MyISAM 一起使用。The built-in MySQL full-text parser uses the white space between
words as a delimiter to determine where words begin and end,
which is a limitation when working with ideographic languages
that do not use word delimiters. To address this limitation for
Japanese, MySQL provides a MeCab full-text parser plugin. The
MeCab full-text parser plugin is supported for use with
InnoDB and
MyISAM.
MySQL 还提供支持日语的 ngram 全文解析插件。更多信息,请参见第 14.9.8 节,“ngram 全文解析器”。MySQL also provides an ngram full-text parser plugin that supports Japanese. For more information, see Section 14.9.8, “ngram Full-Text Parser”.
MeCab 全文解析插件是一个用于日语的全文解析插件,它将文本序列分词为有意义的词。例如,MeCab 将“データベース管理”(“数据库管理”)分词为“データベース”(“数据库”)和“管理”(“管理”)。相比之下,ngram全文解析插件将文本分词为连续的n个字符序列,其中n表示 1 到 10 之间的数字。The MeCab full-text parser plugin is a full-text parser plugin
for Japanese that tokenizes a sequence of text into meaningful
words. For example, MeCab tokenizes
“データベース管理”
(“Database Management”) into
“データベース”
(“Database”) and
“管理”
(“Management”). By comparison, the
ngram full-text
parser tokenizes text into a contiguous sequence of
n characters, where
n represents a number between 1 and
10.
除了将文本分词为有意义的词之外,MeCab 的索引通常比 ngram 索引更小,MeCab 的全文搜索通常也更快。一个缺点是,与 ngram 全文解析器相比,MeCab 全文解析器对文档进行分词可能需要更长的时间。In addition to tokenizing text into meaningful words, MeCab indexes are typically smaller than ngram indexes, and MeCab full-text searches are generally faster. One drawback is that it may take longer for the MeCab full-text parser to tokenize documents, compared to the ngram full-text parser.
第 14.9 节,“全文搜索函数”中描述的全文搜索语法适用于 MeCab 解析器插件。本节描述了解析行为的差异。全文相关的配置选项也适用。The full-text search syntax described in Section 14.9, “Full-Text Search Functions” applies to the MeCab parser plugin. Differences in parsing behavior are described in this section. Full-text related configuration options are also applicable.
有关 MeCab 解析器的更多信息,请参阅 Github 上的 MeCab:另一种词类和形态分析器 项目。For additional information about the MeCab parser, refer to the MeCab: Yet Another Part-of-Speech and Morphological Analyzer project on Github.
安装 MeCab 解析器插件Installing the MeCab Parser Plugin
MeCab 解析器插件需要 mecab 和 mecab-ipadic。The MeCab parser plugin requires mecab and
mecab-ipadic.
在受支持的 Fedora、Debian 和 Ubuntu 平台上(Ubuntu 12.04 除外,因为系统版本过旧),如果安装在默认位置,MySQL 会动态链接到系统的mecab安装。在其他受支持的类 Unix 平台上,mecab以静态方式链接在libmecab.so中,该目录位于 MySQL 插件目录。libpluginmecab.so包含在 MySQL 二进制文件中,位于mecab-ipadic在 MySQL 配置文件中,将.On supported Fedora, Debian and Ubuntu platforms (except Ubuntu
12.04 where the system MYSQL_HOME\lib\mecabmecab version is too
old), MySQL dynamically links to the system
mecab installation if it is installed to
the default location. On other supported Unix-like platforms,
libmecab.so is statically linked in
libpluginmecab.so, which is located in the
MySQL plugin directory. mecab-ipadic is
included in MySQL binaries and is located in
.MYSQL_HOME\lib\mecab
你可以使用本地包管理工具(在 Fedora、Debian 和 Ubuntu 上)安装 mecab 和 mecab-ipadic,或者也可以从源码构建 mecab 和 mecab-ipadic。有关使用本地包管理工具安装 mecab 和 mecab-ipadic 的信息,请参见 从二进制分发安装 MeCab(可选)。如果你想从源码构建 mecab 和 mecab-ipadic,请参见 从源码构建 MeCab(可选)。You can install mecab and
mecab-ipadic using a native package
management utility (on Fedora, Debian, and Ubuntu), or you can
build mecab and
mecab-ipadic from source. For information
about installing mecab and
mecab-ipadic using a native package
management utility, see
Installing MeCab From a
Binary Distribution (Optional). If you want to build
mecab and mecab-ipadic
from source, see
Building MeCab From
Source (Optional).
在 Windows 上,libmecab.dll 位于 MySQL 的 bin 目录中。mecab-ipadic 位于 。On Windows, MYSQL_HOME/lib/mecablibmecab.dll is found in the
MySQL bin directory.
mecab-ipadic is located in
.MYSQL_HOME/lib/mecab
要安装和配置 MeCab 解析器插件,请执行以下步骤:To install and configure the MeCab parser plugin, perform the following steps:
配置选项设置为
mecab_rc_file的所在位置mecabrc配置文件,即 MeCab 的配置文件。如果您使用的是随 MySQL 分发的 MeCab 包,mecabrc文件位于MYSQL_HOME/lib/mecab/etc/.In the MySQL configuration file, set themecab_rc_fileconfiguration option to the location of themecabrcconfiguration file, which is the configuration file for MeCab. If you are using the MeCab package distributed with MySQL, themecabrcfile is located inMYSQL_HOME/lib/mecab/etc/.[mysqld] loose-mecab-rc-file=MYSQL_HOME/lib/mecab/etc/mecabrc
loose前缀是一个 选项修饰符。在安装 MeCaB 解析器插件之前,MySQL 不会识别mecab_rc_file选项,但必须在尝试安装 MeCaB 解析器插件之前设置它。loose前缀允许你在遇到未识别变量时重新启动 MySQL 而不会出现错误。Thelooseprefix is an option modifier. Themecab_rc_fileoption is not recognized by MySQL until the MeCaB parser plugin is installed but it must be set before attempting to install the MeCaB parser plugin. Thelooseprefix allows you restart MySQL without encountering an error due to an unrecognized variable.如果您使用自己的 MeCab 安装,或者从源码构建 MeCab,
mecabrc配置文件的位置可能会有所不同。If you use your own MeCab installation, or build MeCab from source, the location of themecabrcconfiguration file may differ.有关 MySQL 配置文件及其位置的信息,请参见 第 6.2.2.2 节,“使用选项文件”。For information about the MySQL configuration file and its location, see Section 6.2.2.2, “Using Option Files”.
同样在 MySQL 配置文件中,将最小令牌大小设置为 1 或 2,这是建议与 MeCab 解析器一起使用的值。对于
InnoDB表,最小令牌大小由innodb_ft_min_token_size配置选项定义,默认值为 3。对于MyISAM表,最小令牌大小由ft_min_word_len定义,默认值为 4。Also in the MySQL configuration file, set the minimum token size to 1 or 2, which are the values recommended for use with the MeCab parser. ForInnoDBtables, minimum token size is defined by theinnodb_ft_min_token_sizeconfiguration option, which has a default value of 3. ForMyISAMtables, minimum token size is defined byft_min_word_len, which has a default value of 4.[mysqld] innodb_ft_min_token_size=1
修改
mecabrc配置文件以指定您想要使用的词典。MySQL 二进制文件分发的mecab-ipadic包含三种词典(ipadic_euc-jp、ipadic_sjis和ipadic_utf-8)。MySQL 随附的mecabrc配置文件包含类似以下的条目:Modify themecabrcconfiguration file to specify the dictionary you want to use. Themecab-ipadicpackage distributed with MySQL binaries includes three dictionaries (ipadic_euc-jp,ipadic_sjis, andipadic_utf-8). Themecabrcconfiguration file packaged with MySQL contains and entry similar to the following:dicdir = /path/to/mysql/lib/mecab/lib/mecab/dic/ipadic_euc-jp
例如,要使用
ipadic_utf-8词典,请按如下方式修改条目:To use theipadic_utf-8dictionary, for example, modify the entry as follows:dicdir=
MYSQL_HOME/lib/mecab/dic/ipadic_utf-8如果您使用自己安装的 MeCab 或者从源代码构建了 MeCab,那么
mecabrc文件中的默认dicdir条目可能会有所不同,词典及其位置也会不同。If you are using your own MeCab installation or have built MeCab from source, the defaultdicdirentry in themecabrcfile is likely to differ, as are the dictionaries and their location.Note安装 MeCab 解析器插件后,您可以使用
mecab_charset状态变量来查看 MeCab 使用的字符集。MySQL 二进制文件提供的三种 MeCab 词典支持以下字符集。After the MeCab parser plugin is installed, you can use themecab_charsetstatus variable to view the character set used with MeCab. The three MeCab dictionaries provided with the MySQL binary support the following character sets.ipadic_euc-jp词典支持ujis和eucjpms字符集。Theipadic_euc-jpdictionary supports theujisandeucjpmscharacter sets.ipadic_sjis词典支持sjis和cp932字符集。Theipadic_sjisdictionary supports thesjisandcp932character sets.ipadic_utf-8字典支持utf8mb3和utf8mb4字符集。Theipadic_utf-8dictionary supports theutf8mb3andutf8mb4character sets.
mecab_charset仅报告第一个支持的字符集。例如,ipadic_utf-8字典同时支持utf8mb3和utf8mb4。mecab_charset在使用此字典时始终报告utf8。mecab_charsetonly reports the first supported character set. For example, theipadic_utf-8dictionary supports bothutf8mb3andutf8mb4.mecab_charsetalways reportsutf8when this dictionary is in use.重启 MySQL。Restart MySQL.
安装 MeCab 解析器插件:Install the MeCab parser plugin:
MeCab 解析器插件是使用
INSTALL PLUGIN安装的。插件名称是mecab,共享库名称是libpluginmecab.so。有关安装插件的更多信息,请参见 第 7.6.1 节,“安装和卸载插件”。The MeCab parser plugin is installed usingINSTALL PLUGIN. The plugin name ismecab, and the shared library name islibpluginmecab.so. For additional information about installing plugins, see Section 7.6.1, “Installing and Uninstalling Plugins”.INSTALL PLUGIN mecab SONAME 'libpluginmecab.so';
安装后,MeCab 解析器插件会在每次正常 MySQL 重启时加载。Once installed, the MeCab parser plugin loads at every normal MySQL restart.
使用
SHOW PLUGINS语句验证 MeCab 解析器插件是否已加载。Verify that the MeCab parser plugin is loaded using theSHOW PLUGINSstatement.mysql> SHOW PLUGINS;
mecab插件应出现在插件列表中。Amecabplugin should appear in the list of plugins.
创建使用 MeCab 解析器的 FULLTEXT 索引Creating a FULLTEXT Index that uses the MeCab Parser
要创建使用 mecab 解析器的 FULLTEXT 索引,请在 CREATE TABLE、ALTER TABLE 或 CREATE INDEX 中指定 WITH PARSER ngram。To create a FULLTEXT index that uses the
mecab parser, specify WITH PARSER ngram with
CREATE TABLE,
ALTER TABLE, or
CREATE INDEX.
此示例演示了如何创建一个带有 mecab FULLTEXT 索引的表,插入示例数据,并在信息模式 INNODB_FT_INDEX_CACHE 表中查看分词数据:This example demonstrates creating a table with a
mecab FULLTEXT index,
inserting sample data, and viewing tokenized data in the
Information Schema
INNODB_FT_INDEX_CACHE table:
mysql> USE test;
mysql> CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT,
FULLTEXT (title,body) WITH PARSER mecab
) ENGINE=InnoDB CHARACTER SET utf8mb4;
mysql> SET NAMES utf8mb4;
mysql> INSERT INTO articles (title,body) VALUES
('データベース管理','このチュートリアルでは、私はどのようにデータベースを管理する方法を紹介します'),
('データベースアプリケーション開発','データベースアプリケーションを開発することを学ぶ');
mysql> SET GLOBAL innodb_ft_aux_table="test/articles";
mysql> SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE ORDER BY doc_id, position;
要向现有表添加FULLTEXT索引,可以使用 ALTER TABLE 或 CREATE INDEX。例如:To add a FULLTEXT index to an existing table,
you can use ALTER TABLE or
CREATE INDEX. For example:
CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT
) ENGINE=InnoDB CHARACTER SET utf8mb4;
ALTER TABLE articles ADD FULLTEXT INDEX ft_index (title,body) WITH PARSER mecab;
# Or:
CREATE FULLTEXT INDEX ft_index ON articles (title,body) WITH PARSER mecab;
MeCab 分析器空格处理MeCab Parser Space Handling
MeCab 解析器在查询字符串中使用空格作为分隔符。例如,MeCab 解析器将 データベース管理 分词为 データベース 和 管理。The MeCab parser uses spaces as separators in query strings. For example, the MeCab parser tokenizes データベース管理 as データベース and 管理.
MeCab 解析器停用词处理MeCab Parser Stopword Handling
默认情况下,MeCab 解析器使用默认的停用词列表,其中包含少量英文停用词。对于适用于日语的停用词列表,您必须自行创建。有关创建停用词列表的信息,请参见第 14.9.4 节,“全文停用词”。By default, the MeCab parser uses the default stopword list, which contains a short list of English stopwords. For a stopword list applicable to Japanese, you must create your own. For information about creating stopword lists, see Section 14.9.4, “Full-Text Stopwords”.
MeCab 解析器术语搜索MeCab Parser Term Search
对于自然语言模式搜索,搜索词会被转换为令牌的联合。例如,データベース管理 会被转换为 データベース 管理。For natural language mode search, the search term is converted to a union of tokens. For example, データベース管理 is converted to データベース 管理.
SELECT COUNT(*) FROM articles
WHERE MATCH(title,body) AGAINST('データベース管理' IN NATURAL LANGUAGE MODE);
对于布尔模式搜索,搜索词会被转换为搜索短语。例如,データベース管理 会被转换为 データベース 管理。For boolean mode search, the search term is converted to a search phrase. For example, データベース管理 is converted to データベース 管理.
SELECT COUNT(*) FROM articles
WHERE MATCH(title,body) AGAINST('データベース管理' IN BOOLEAN MODE);
MeCab 分析器通配符搜索MeCab Parser Wildcard Search
通配符搜索词不会被分词。对 データベース管理* 的搜索是在前缀 データベース管理 上执行的。Wildcard search terms are not tokenized. A search on データベース管理* is performed on the prefix, データベース管理.
SELECT COUNT(*) FROM articles
WHERE MATCH(title,body) AGAINST('データベース*' IN BOOLEAN MODE);
MeCab 解析器短语搜索MeCab Parser Phrase Search
短语已被标记化。例如,データベース管理 被标记化为 データベース 管理。Phrases are tokenized. For example, データベース管理 is tokenized as データベース 管理.
SELECT COUNT(*) FROM articles
WHERE MATCH(title,body) AGAINST('"データベース管理"' IN BOOLEAN MODE);
从二进制分发安装 MeCab(可选)Installing MeCab From a Binary Distribution (Optional)
本节介绍如何使用本地包管理工具从二进制发行版安装mecab和mecab-ipadic。例如,在 Fedora 上,可以使用 Yum 进行安装:This section describes how to install mecab
and mecab-ipadic from a binary distribution
using a native package management utility. For example, on
Fedora, you can use Yum to perform the installation:
$> yum mecab-devel
在 Debian 或 Ubuntu 上,你可以执行 APT 安装:On Debian or Ubuntu, you can perform an APT installation:
$>apt-get install mecab$>apt-get install mecab-ipadic
从源码安装 MeCab(可选)Installing MeCab From Source (Optional)
如果您想从源代码构建 mecab 和 mecab-ipadic,下面提供了基本的安装步骤。有关更多信息,请参阅 MeCab 文档。If you want to build mecab and
mecab-ipadic from source, basic
installation steps are provided below. For additional
information, refer to the MeCab documentation.
从 http://taku910.github.io/mecab/#download 下载
mecab和mecab-ipadic的 tar.gz 包。截至 2016 年 2 月,最新可用的包是mecab-0.996.tar.gz和mecab-ipadic-2.7.0-20070801.tar.gz。Download the tar.gz packages formecabandmecab-ipadicfrom http://taku910.github.io/mecab/#download. As of February, 2016, the latest available packages aremecab-0.996.tar.gzandmecab-ipadic-2.7.0-20070801.tar.gz.安装
mecab:Installmecab:$>
tar zxfv mecab-0.996.tar$>cd mecab-0.996$>./configure$>make$>make check$>su$>make install安装
mecab-ipadic:Installmecab-ipadic:$>
tar zxfv mecab-ipadic-2.7.0-20070801.tar$>cd mecab-ipadic-2.7.0-20070801$>./configure$>make$>su$>make install使用
WITH_MECABCMake 选项编译 MySQL。如果你已经将mecab和mecab-ipadic安装到默认位置,请将WITH_MECAB选项设置为system。Compile MySQL using theWITH_MECABCMake option. Set theWITH_MECABoption tosystemif you have installedmecabandmecab-ipadicto the default location.-DWITH_MECAB=system
如果你定义了自定义安装目录,请将
WITH_MECAB设置为自定义目录。例如:If you defined a custom installation directory, setWITH_MECABto the custom directory. For example:-DWITH_MECAB=/path/to/mecab