Home

MySQL Utilities - Oracle Documentation

1. C newcerts server cert pem However at this moment the root account is not using an encrypted ssl connection as you can see with the MySQL Command Line Tool running the status command shell gt mysql uroot ppass port 3307 e status mysql Ver 14 14 Distrib 5 6 15 for Win64 x86_64 Connection id iil Current database Current user root localhost SSL Not in use You need to add the SSL options necessarily to establish an encrypted connection with SSL this can be done in the following form shell gt mysql uroot ppass port 3307 ssl ca C newcerts cacert pem ssl cert C newcerts server cert pem ssl key C newcerts server key pem e status mysql Ver 14 14 Distrib 5 6 15 for Win64 x86_64 Connection id 13 Current database Current user root localhost SSE Cipher in use is DHE RSA AES256 SHA Note R To configure an account to only permit SSL encrypted connections the grants for that account must include the REQUIRE SSL clause in your GRANT Syntax statement In the same form that you use the SSL options with the MySQL Command Line Tool you can use the SSL options on each of the MySQL Utilities The following is an example of mysqiserverinfo using SSL options shell gt mysqlserverinfo server root pass localhost 3307 ssl ca C newcerts cacert pem ssl cert C newcerts client cert pem ssl key C newcerts client key pem format vertical Sou
2. Parameters stream file type a valid file type for example the result of the built in Python function open version Returns Version of the MySQL server that produced the log Return type tuple program Returns Full path of the MySQL server executable Return type str port Returns TCP IP port on which the MySQL server was listening Return type int socket Returns Full path of the MySQL server Unix socket Return type str start_datetime Returns Date and time of the first read log entry Return type datetime datetime lastseen_datetime Returns Date and time of the last read log entry Return type datetime datetime 208 Chapter 7 MySQL Utilities Testing MUT Table of Contents FAL mut MySQL Utilities TOStinG wnccsssscorstssieeddvss erect enneiecvss eeesesesnboedaas aaa a aE A E 209 7 1 mut MySQL Utilities Testing This utility executes predefined tests to test the MySQL Utilities The tests are located under the mysql test directory and divided into suites stored as folders By default all tests located in the t folder are considered the main suite You can select any number of tests to run select one or more suites to restrict the tests exclude suites and tests and specify the location of the utilities and tests The utility requires the existence of at least one
3. 4 You can use the discover slaves feature if and only if all slaves report their host and port to the master A sample command to generate a replication health report with discovery is shown below Note that the option discover slaves login cannot be used in conjunction with the s laves option shell gt mysqlrpladmin master root localhost 3332 discover slaves login root health Discovering slaves for master at localhost 3332 Discovering slave at localhost 3331 Found slave localhost 3331 Discovering slave at localhost 3333 Found slave localhost 3333 Discovering slave at localhost 3334 Found slave localhost 3334 Checking privileges Replication Topology Health 4 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 4 4 localhost 3332 MASTER UP ON ORK localhost 3331 SLAVE UP ON OK localhost 3333 SLAVE UP ON OK localhost 3334 SLAVE UP ON OKs 4 4 4 4 4 4 done PERMISSIONS REQUIRED The users on the master need the following privileges SELECT and INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION The slave users need the SUPER privilege The repl user used as the argument for
4. 4 Variable_name Value 4 4 audit_log_buffer_size 1048576 audit _log_connection_policy ALL audit_log_current_session ON audit_log_exclude_accounts audit_log_file audit log audit_log_flush OFF audit_log_format OLD audit_log_include_accounts awebi ihe jae llirey ALL audit_log_rotate_on_size 0 audit_log_statement_policy ALL audit_log_strategy ASYNCHRONOUS 4 4 Executing POLICY command Showing options after command Audit Log Variables and Options 4 4 Variable_name Value 4 4 audit_log_buffer_size 1048576 audit_log_connection_policy NONE audit_log_current_session ON audit_log_exclude_accounts audit_log_file audit log audit_log_flush OFF audit_log_format OLD audit_log_include_accounts aweh ihe jae llaveyy ALL audit_log_rotate_on_size 28672 audit_log_statement_policy ALL audit_log_strategy ASYNCHRONOUS 4 4 To perform a copy of a audit log file to another location use the following command shell gt mysqlauditadmin audit log name SERVER data audit log 13486539046497235 70 PERMISSIONS REQUIRED copy copy to BACKUP Audit_Logs To copy a audit log file from a remote server location to the current
5. MASTER SLAVE and SLAVE MASTER indicate that a server is a master only slave only and both slave and master respectively A circular replication topology is shown like this where lt gt indicates circularity Replication Topology Graph localhost 3311 MASTER aes localhost 3312 SLAVE MASTER a localhost 3313 SLAVE MASTER ae lecalhost 3311 lt gt SLAVE To produce a column list in addition to the graph specify the show 1ist option In this case to specify how to display the list use one of the following values with the format option e grid default Display output in grid or table format like that of the mysql client command line tool e CSV Display output in comma separated values format tab Display output in tab separated format e vertical Display output in single column format like that of the G command for the mysq1 client command line tool The utility uses of the SHOW SLAVE HOSTS statement to determine which slaves the master has If you want to use the recurse option slaves should have been started with the report host and report port options set to their actual host name and port number or the utility may not be able to connect to the slaves to determine their own slaves OPTIONS mysqlrplshow accepts the following command line options e help Display a help message and exit e license Display license information and exit e discover sla
6. 4 4 4 john localhost 3306 TABLE teal test john localhost 3306 TABLE 2 cest john localhost 3306 TABLE tm test To find all object that contain t2 in the name or the body for routines triggers and events shell gt mysqlmetagrep b pattern t2 server john localhost 3306 4 4 4 4 Connection Object Type Object Name Database 4 4 4 4 PxohnizseClocalhost 3306 TRIGGER ire EOG test john localhost 3306 TABLE 1 amp 2 test 4 4 4 4 In the preceding output the trigger name does not match the pattern but is displayed because its body does This is the same as the previous example but using the REGEXP operator Note that in the pattern it is not necessary to add wildcards before or after t2 shell gt mysqlmetagrep Gb pattern t2 server john localhost 4 4 Connection Object Type Object Name Database 4 4 Poot localhost 3306 TRIGGER te tee test Poot 2 0localhos t 0e TABLE eee test
7. done SUMMARY No data consistency issue found To check the data consistency on an active replication system using slave discovery shell gt mysqlrplsync master user pass localhost 3310 discover slaves login rpl pass Discovering slaves for master at localhost 3310 Discovering slave at localhost 3311 Found slave localhost 3311 Discovering slave at localhost 3312 Found slave localhost 3312 GTID differences between Master and Slaves Slave localhost 3311 is 15 transactions behind Master Slave localhost 3312 is 15 transactions behind Master Checking data consistency Using Master localhost 3310 as base server for comparison Checking test_rplsync_db database Checking t0 table data OK test rplsync db tO checksum for server localhost 3311 OK test rplsync db t0 checksum for server localhost 3312 Checking ti table data OK test rplsync db m tl checksum for server localhost 32311 OK test_replsyncidb tl checksum for server localhost 3212 Checking test_db database Checking 0 table data OK testdb t0 checksum for server localhost 3311 OK test dab 0 checksum for server localhost 3312 Checking tl table data OK test db tl checksum for server localhost OK test db tli checksum for server localhost 3312 done SUMMARY No data consistency issue found
8. cceeeeeeeeeeeeeeeeeeeeeeeeeeeeeaeaaes 279 8 9 7 Do have to use MySQL Replication for implementing HA ccceeeeeeeeeeeeeeeeeeeeeeeeeeeeeaeaaes 280 8 9 8 Is the MySQL Fabric node itself fault tolerant What happens when the MySQL Fabric node is 8 9 1 8 9 2 8 9 3 8 9 4 8 9 5 8 9 6 NOtAVAIADIS sn ove E E E shan eae ie eta ee ell etek Shatin tao ue 280 How is High Availability achieved with MySQL Fabric MySQL Fabric manages one or more HA Groups where each HA Group contains one or more MySQL Servers For High Availability a HA Group contains a Primary and one or more Secondary MySQL Servers The Primary is currently a MySQL Replication master which replicates to each of the secondaries MySQL Replication slaves within the group By default the Fabric aware connectors route writes to the Primary and load balance reads across the available secondaries Should the Primary fail MySQL Fabric will promote one of the Secondaries to be the new Primary automatically promoting the MySQL Server to be the replication Master and updating the routing performed by the Fabric aware connectors How are MySQL Server failures detected The MySQL Fabric node has a built in monitoring function that checks on the status of the master In addition the Fabric aware connectors report to MySQL Fabric when the Primary becomes unavailable to them The administrator can configure how many problems need to be reported
9. 4 4 o 4 4 4 None SELECT FROM INFORMATION_SCHEMA PLUGINS WHERE COMMIT None None To display the audit log entries matching a specific SQL L KE pattern use the following command shell gt mysqlauditgrep pattern SERVER data audit log EXAMPLES FOZ O9 2 7 1 FOLZ O9 2Et 2OLZ O9 2Et 13 33 39 Query SET session autocommit OFF 8 11 26 50 Query SET session autocommit OFF 9 11 26 50 Query SET session autocommit OFF 10 4 4 5 5 5 5 5 5 To display the audit log entries matching a specific REGEXP pattern use the following command shell gt mysqlauditgrep pattern regexp SERVER data audit log 4 4 4 4 4 STATUS TIMESTAMP NAME SOLTEXT CONNECTION_ID 4 4 4 4 oO 2012 09 27T13 33 39 Query SET session autocommit OFF 7 0 e009 2 aS 33359 Miery SET MG sesismonerauit Conmuit l Onr 6 I 2012 09 28T11 26 50 Query SET session autocommit OFF 9 0 20IZ WS Acnnilils agg Overy SET GCecssion Cwr ocomim
10. conn mysql connector connect aloe Hnos a Wikeeallaosie Wixom a SAATA username admin password adminpass yy user webuser password webpass autocommit True conn set_property tables employees employees scope fabric SCOPE_GLOBAL mode fabric MODE_READWRITE cur conn cursor cur execute CREATE DATABASE IF NOT EXISTS employees cur execute USE employees cur execute DROP TABLE IF EXISTS employees cur execute CREATE TABLE employees n emp no INT n first_name CHAR 40 last_name CHAR 40 myn gtid_executed prepare_synchronization cur conn set_property scope fabric SCOPE_LOCAL first_names John Buffalo Michael Kate Deep Genesis Iase mamssi Vilee RRN Cekson Vsehurslol Wieyuireoylle list_emp_no for count in range 10 emp_no pick_shard_key list_emp_no append emp_no add_employee conn emp_no first_names emp_no len first_names last_names emp_no len last_names gtid_executed for emp_no in list_emp_no find_employee conn emp_no gtid_executed Shard Move If the current set of servers for a shard is not powerful enough we can move the shard to a more powerful server set The shard move functionality can be used to move a shard from one group to another These are the steps necessary to move a shard 1 Setup the required group or groups shell gt mysqlfabric group create group_id MOVE shell gt mysq
11. Discussion In order to change the type of events recorded to the audit log file the policy settings must be changed This is done with the mysqlauditadmin utility using the command policy and specifying the desired policy value with the value option As expected the specification of the target server is also required using the server option In the above example the policy value was set to LOGINS to write only login events to the log file Nevertheless other values are also permitted to control the information written to the log file ALL write all events QUERIES write only query event NONE disable logging DEFAULT use the default policy Permissions Required User must have the SELECT privilege for the mysql database To view the log file the user must have read access to the audit log file on the server 56 How Do Copy Move The Audit Log Tips and Tricks The policy value was specified using uppercase in this example however upper and lower cases can be mixed to specify the policy value such as LoGiNs The values for this command will still be read correctly independently of the used cases case insensitive but if an unsupported value is specified an error will be issued In the above example the show options option was additionally used but it is not required This option simply displays the audit log settings variables However when this option is combined with a command that changes one of
12. Fortunately the mysqluc utility does this and more It is named the MySQL Users Console and provides type completion for options utility names and even user defined variables for working with common option values Not only that it also provides the ability to get help for any utility supported Objectives Discover what utilities exist and find the options for certain utilities Run several utilities with the same server using the type completion feature to make using the suite of utilities easier Example Execution Note In the example below keystrokes are represented using square brackets For example TAB indicates the tab key was pressed Similarly portions in the commands specific with angle brackets are values you would replace with actual values For example lt user gt indicates you would place the user s login name here shell gt mysqluc Launching console Welcome to the MySQL Utilities Client mysqluc version 1 4 2 Copyright c 2010 2014 Oracle and or its affiliates All rights reserved This is a release of dual licensed MySQL Utilities For the avoidance of doubt this particular copy of the software is released under the version 2 of the GNU General Public License MySQL Utilities is brought to you by Oracle help Type for a list of commands or press TAB twice for list of utilities mysqluc gt help Command help utilities help lt uLility show errors clear errors show last error hel
13. 4 5 5 5 5 5 UUID Status 4 4 4 4 5 host moore role uuid 4 4 4 4 5 5 5 5 5 5 localhost 3307 MASTER 00a4e027 a83a 1le3 8bd6 28d244017 26 localhost 3306 SLAVE faf0874f a839 1le3 8bd6 28d244017 26 4 4 4 4 5 5 5 5 5 5 Start multi source replication running as a daemon POSIX only shell gt mysqlrplms slave root root localhost 3306 masters root root localhost 3307 root root localhost 3308 log rplms_daemon log pidfile rplms_daemon pid daemon start Restart multi source replication running as a daemon shell gt mysqlrplms slave root root localhost 3306 masters root root localhost 3307 root root localhost 3308 153 RECOMMENDATIONS log rplms_daemon log pidfile rplms_daemon pid daemon restart Stop multi source replication running as a daemon shell gt mysqlrplms slave root root localhost 3306 masters root root localhost 3307 root root localhost 3308 log rplms_daemon log pidfile rplms_daemon pid daemon stop RECOMMENDATIONS You should set read_only 1 in the my cnf file for the slave to ensure that no accidental data changes such as INSERT DELETE UPDATE and so forth are permitted on the slave other than those produced by events read from the master PERMISSIONS REQUIRED The users on
14. 5 12 mysqlmetagrep Search Database Object Definitions This utility searches for objects matching a given pattern on all the servers specified using instances of the server option It produces output that displays the matching objects By default the first non option argument is taken to be the pattern unless the pattern option is given If the pattern option is given then all non option arguments are treated as connection specifications Internally the utility generates an SQL statement for searching the necessary tables in the INFORMATION_SCHEMA database on the designated servers and then executes it before collecting the result and printing it as a table Use the sq1 option to have mysqimetagrep display the statement rather than execute it This can be useful if you want to feed the output of the statement to another application such as the mysql client command line tool The MySQL server supports two forms of patterns when matching strings SQL Simple Patterns used with the LIKE operator and POSIX Regular Expressions used with the REGEXP operator By default the utility uses the LIKE operator to match the name and optionally the body of objects To use the REGEXP operator instead use the regexp option the expression to the beginning of the string if you want to match the beginning of the string Note R Because the REGEXP operator does substring searching it is necessary to anchor To specify how to di
15. IC OMCs Discussion The above example illustrates how to execute the mysqlislavet rx utility to skip the transactions for the specified GTID set on all given slaves To achieve this task we only need to specify the GTID set for the transactions to be skipped with the gt id set option and the list of connection parameters for the target slaves with the slaves option 47 Server Operations In the above example all of the specific GTIDs were skipped on all target slaves injecting an empty transaction for each one of them However it might happen that some of the GTIDs cannot be skipped on some slaves This can happen if a transaction with the same GTID was previously applied on the target slave The reason is due to the purpose of GTIDs which is to uniquely identify a transaction therefore two distinct transaction cannot be applied with the same GTID otherwise an error is issued The mysqlslavet rx utility checks the transactions that can be effectively skipped on each slave at the beginning excluding already executed GTIDs Permissions Required The user for the slaves must have the required permissions to inject an empty transaction for a specific GTID i e to set the gt id_next variable More specifically the login user to connect to slaves requires the SUPER privilege Tips and Tricks The mysqislavetrx provides a dry run mode that allows users to verify the GTID that would be skipped in each slave withou
16. Tips and Tricks In the above example the mysqlirplsync utility was used to check all the data on the servers However it is possible to check only specific databases and tables For that purpose the user only need to specify the target database and tables as arguments when invoking the utility It is also possible to exclude specific database and tables from the check using the exclude 175 option For example exclude test_rplsync_db test_db t0 excludes the database test_rplsync_db and table test_db t0 from the check performed by the utility The utility provides important options to control the execution time of the checksum queries performed on each table and the waiting time for slaves to reach an established synchronization point namely the checksum timeout 175 and rpl timeout 176 options A polling process is applied on each slave to periodically check if replication caught up with the defined sync point The periodic interval to perform this check can be adjusted with the interval 175 option These option are fundamental to control the impact of the execution of the utility on the replication system limiting the execution time of the checksum queries for large tables and the time slaves wait for replication to catch up In case the timeouts defined by those options are reached the check is skipped Nevertheless the user can always execute the utility later only for the skipped tables using higher timeo
17. idx_id_hdate and idx_id_bday might not be needed This happens because in InnoDB secondary indexes contain the primary key columns for the row that are not in the secondary index It must be noted however that these are just indications they must not be followed blindly because redundant indexes can be useful depending on how you use query your tables Permissions Required Regarding the privileges needed to run this utility the test_user needs SELECT privilege on the mysql database as well as for the databases which tables are being checked Tips and Tricks You can use the d option to generate the SQL drop statements needed to remove the indexes The stats option can be used alone or together with either best or worst to show statistics about the indexes Use the show indexes option to show each table together with its indexes 3 3 6 Need to Find an Object on My Server But All I Have is a Partial Name How Do Find All Objects with That Name Prefix One of the challenges for database administrators who manage servers with thousands of objects is the task of finding an object by name Sometimes all you have to go on is the name of a table or perhaps an obscure reference to a partial name This can come about through a diagnosis of a problem connection application or via an incomplete description from a defect report 30 Need to Find an Object on My Server But All Have is a Partial Name How
18. server root pass localhost new data Users cbell source test123 new port 3310 root password pass mysqld log bin mysql bin Cloning the MySQL server running on localhost Creating new data directory Configuring new instance MOC cies Ceemiy S d MECCSRE Setting up empty database and mysql tables Sh Se Sh SE H 183 PERMISSIONS REQUIRED Starting new instance of the server Testing connection to new instance Success Setting the root password co clome PERMISSIONS REQUIRED Sh Sh SE SE OE The user must have permission to read all databases Since we are using the root account for these examples and you typically would permissions are not generally a problem You also need permissions to create the new data directory and write data to it 5 21 mysqlserverinfo Display Common Diagnostic Information from a Server This utility displays critical information about a server for use in diagnosing problems The information displayed includes the following e Server connection information e Server version number Data directory path name e Base directory path name e Plugin directory path name Configuration file location and name e Current binary log coordinates file name and position Current relay log coordinates file name and position This utility can be used to see the diagnostic information for servers that are running or offline If you want to see information
19. CountryCode REFERENCES Country Code Compare failed One or more differences found Discussion Notice to accomplish this task we simply specified each server with serverl and server2 then specified the database objects to compare with the lt db gt lt object gt lt db gt lt object gt syntax Permissions Required The user must have SELECT privileges for both objects on both servers as well as SELECT on the mysql database Tips and Tricks You can set the direction of the compare by using the changes for option For example to see the changes for server1 as the target use changes for server1 3 2 5 How Can I Synchronize a Table on Two Servers Where Neither is Up to date When working with servers that are used in different networks or are compartmentalized or simply intentionally manually redundant they do not use replication or perhaps through some crisis you may encounter a situation where a table or an entire database becomes out of synch We don t simply want to know which rows differ rather we need to know the SQL statements needed to bring the tables into synch Furthermore we aren t sure which table is most out of date so we d like to see the transformation statements for both directions In this case it would be very helpful to know exactly how the tables differ For this we use the mysqldbcompare utility Objectives The goal is to generate the SQL transformation statements to bring two
20. Display the master s list of executed GTIDs contents of the GTID variables GLOBAL GTID_EXECUTED GLOBAL GTID_PURGED and GLOBAL GTID_OWNED e uuid Display universally unique identifiers UUIDs for all servers Default health 122 OPTIONS rpl user lt replication_user gt The user and password for the replication user requirement in the form lt user gt lt password gt or lt login path gt E g rpl passwd Default None script threshold lt return_code gt Value for external scripts to trigger aborting the operation if result is greater than or equal to the threshold Default None no threshold checking seconds behind lt seconds gt Used to detect slave delay only for health reporting purposes The maximum number of seconds behind the master permitted before slave is considered behind the master in the health report state Default 0 slaves lt slave connections gt Connection information for slave servers List multiple slaves in comma separated list The list will be evaluated literally whereby each server is considered a slave to the master listed regardless if they are a slave of the master To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a comman
21. If this is software or related documentation that is delivered to the U S Government or anyone licensing it on behalf of the U S Government then the following notice is applicable U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptation of the programs including any operating system integrated software any programs installed on the hardware vii Legal Notices and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of information management applications It is not developed or intended for use in any inherently dangerous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropriate fail safe backup redundancy and other measures to ensure its safe use Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous appl
22. Should the copy fail in the middle the destination databases may be incomplete or inconsistent Should this occur drop the destination database in question repair the cause of the failure and restart the copy 3 2 3 How Can Make a Copy of a Database and Change the Storage Engine Sometimes you may have need to create a copy of a database but change the storage engine of all tables to another engine 17 How Can Make a Copy of a Database and Change the Storage Engine For example if you are migrating your database to InnoDB a wise choice you can copy the database to a new database on a new server and change the storage engine to InnoDB For this we can use the mysqldbcopy utility Objectives In this example we want to make a copy of the world_innodb database but change the storage engine to MyISAM and rename the database accordingly You can cause all tables in the destination databases to use a different storage engine with the new storage engine option Example Execution shell gt mysqldbcopy source root root localhost 3306 destination root root localhost 3307 new storage engine myisam world _innodb world_myisam Source on localhost connected Destination on localhost connected Copying database world_innodb renamed as world_myisam Replacing ENGINE InnoDB with ENGINE myisam for table world_myisam City Copying TABLE world_innodb City WARNING FOREIGN KEY constraints
23. format option grid default Display output in grid or table format like that of the mysql client command line tool CSV Display output in comma separated values format tab Display output in tab separated format vertical Display output in single column format like that of the G command for the mysq1 client command line tool To turn off the headers for grid csv or tab display format specify the no headers option OPTIONS mysqldiskusage accepts the following command line options e help Display a help message and exit 114 OPTIONS license Display license information and exit all a Display all disk usage This includes usage for databases logs and InnoDB tablespaces binlog b Display binary log usage empty m Include empty databases format lt format gt f lt format gt Specify the output display format Permitted format values are grid csv tab and vertical The default is grid innodb i Display InnoDB tablespace usage This includes information about the shared InnoDB tablespace as well as idb files for InnoDB tables with their own tablespace logs l Display general query log error log and slow query log usage no headers h Do not display column headers This option applies only for grid csv and tab output quiet q Suppress informational messages relaylog r Display relay log usage server lt server gt Connection in
24. rotate This command is used to perform an on demand audit log rotation and only requires the server option to be passed Note this command has no effect if the audit log file size is smaller than 4096 which is the minimum value allowed that is greater than 0 for the audit_log_rotate_on_size variable OPTIONS mysqlauditadmin accepts the following command line options audit log name lt AUDIT_LOG_FILE gt Full path and file name for the audit log file Used by the file stats option and the copy command copy to lt COPY_DESTINATION gt The location to copy the specified audit log file The path must be locally accessible for the current user file stats Display the audit log file statistics help Display a help message and exit 66 OPTIONS license Display license information and exit remote login lt REMOTE_LOGIN gt User name and host to be used for the remote login for copying log files It is defined using the following format lt user gt lt host or IP gt Usage will prompt for the password server lt SERVER gt Connection information for the server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The
25. Switching slaves to new master Starting slaves Performing START on all slaves Checking slaves for errors Failover complete done To see the replication health of a topology with GTID MODE ON server version 5 6 5 or higher and discover all slaves attached to the master run the following command We use the result of the failover command above shell gt mysqlrpladmin master root localhost 3333 slaves root localhost 3332 root localhost 3334 health Getting health for master localhost 3333 Replication Topology Health f 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 localhost 3333 MASTER UP ON OK localhost 3332 SLAVE WE ON OK 161 EXAMPLES localhost 3334 SLAVE we To view a detailed replication health report but with all of the replication health checks revealed use the verbose option as shown below In this example we use vertical format to make viewing easier shell gt mysqlrpladmin master root localhost 3331 slaves root localhost 3332 root localhost 3333 root localhost 3334 verbose health Getting health for master localhost 3331 Attempting to contact localhost success Attempting to contact localhost success Attempting to contact localhost success Attempting to conta
26. differ serverl on localhost connected Comparing employees salaries to empl salaries FAIL Object definitions are not the same CREATE TABLE salaries ea_ine abe liL N CaE INU seulerey above NENON from_date date NOT NULL to_date date NOT NULL PRIMARY KEY emp_no from_date KEY emp_no emp_no ENGINE InnoDB DEFAULT CHARSET latinl se AARAA EE AAA Compare failed One or more differences found 2 ENGINE MyISAM DEFAULT CHARSET latinl 2 The following examples show how to generate a transformation report Assume the following object definitions Host1 CREATE TABLE dbl tablel num int misc char 30 Host2 112 EXAMPLES CREATE TABLE dbx table3 num int notes char 30 misc char 55 To generate a set of SQL statements that transform the definition of db1 tablel to dbx table3 use this command shell gt mysqldiff serverl root host1 server2 root host2 changes for serverl difftype sql db1 tablel1 dbx table3 serverl on Nosti connected server2 on host2 connected Comparing dbl tablel to dbx table3 FAIL Transformation statements Se Se Se SE ALTER TABLE dbl tablel ADD COLUMN notes char 30 AFTER a CHANGE COLUMN misc misc char 55 Compare failed One or more differences found To generate a set of SQL statements that transform the definition of dbx table3 to db1 tablel use this command shell gt mysq
27. ds setFabricPassword fabricPassword Load the driver if running under Java 5 ie leo inySsell jclee ics tewicloe4 4 Class forName com mysql fabric jdbc FabricMySQLDriver 1 Create database and table for our demo ds setDatabaseName mysql connect to the mysql database before creating our employees databa ds setFabricServerGroup fabric_testl_global connect to the global group Connection rawConnection ds getConnection user password Statement statement rawConnection createStatement statement executeUpdate create database if not exists employees statement close rawConnection close We should connect to the global group to run DDL statements they will be replicated to the server The 1 st way is to set its name explicitly via the fabricServerGroup datasource property ds setFabricServerGroup fabric_testl_global rawConnection ds getConnection user password statement rawConnection createStatement statement executeUpdate create database if not exists employees statement close rawConnection close The 2 nd way is to get implicitly connected to global group when the shard key isn t provided ie don t set fabricShardKey ds setFabricServerGroup null clear the setting in the datasource for previous connections ds setFabricShardTable employee employees rawConnection ds getConnection user password At this poin
28. exec after new master host new master port xec fail check master host master port exec post failover no old master host old master port new master host new master port errors during failover exec post failover old master host old master port errors during failover EXAMPLES To launch the utility you must specify at a minimum the master option and either the discover slaves login option or the slaves option The discover slaves login option can be used in conjunction with the s laves option to specify a list of known slaves or slaves that do not report their host and IP and to discover any other slaves connected to the master An example of the user interface and some of the report views are shown in the following examples STATUS view If there are multiple GTIDs listed the utility shall display to indicate there are additional GTIDs to view You can view the complete list of GTIDs Note R The GTID Executed Set will display the first GTID listed in the SHOW MASTER on the GTID display screens The default interface will display the replication health report like the following In this example the log file is enabled A sample startup command is shown below shell gt mysqlfailover master root localhost 3331 discover slaves login root log log txt 125 EXAMPLES MySQL Replication Monitor and Failover Utility Failover Mode auto Next Interval Mon Ma
29. pedantic p Fail if both servers do not have the same set of storage engines the same default storage engine and the same InnoDB storage engine rpl user lt replication_user gt The user and password for the replication user in the format lt user gt lt passworads or lt login path gt slave lt slave gt Connection information for the slave server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt Start from beginning b Start replication at the beginning of events logged in the master binary log This option is not valid unless both master log file and master log pos are given ssl ca The path to a file that contains a list of trusted
30. permission_id int 10 unsigned Permission s ID subsystem varchar 60 Sub system identification component varchar 60 Sub system component function varchar 60 Sub system function Currently this is not used description varchar 1000 Description The roles table contains information on possible roles a user may have and by consequence his her permissions Table 8 8 roles Column Type Comment role_id int 10 unsigned Roles ID name varchar 80 Role s name description varchar 1000 Role s description The role_permissions table associates roles and permissions Table 8 9 role_permissions Column Type Comment role_id int 10 unsigned Roles ID permission_id int 10 unsigned Permission s ID The servers table contains a list of all servers managed by Fabric Table 8 10 servers Column Type Comment server_uuid varchar 40 UUID of the server server_address varchar 128 Address of the server mode int 11 Mode of the server OFFLINE READ_ONLY WRITE_ONLY READ_WRITE status int 11 Status of the server FAULTY SPARE SECONDARY PRIMARY weight float Likelihood of receiving a request group_id varchar 64 Group the server belongs to The shard_maps table contains the names and properties of the shard maps Table 8 11 shard_maps Column shard_mapping_id Type int 11 Comment Shard map identifier type_nam
31. throws SQLException releaseAnyConnection connection We don t track connections returns false tf public boolean supportsAggressiveRelease return false And finally to implement the Wrapped role public boolean isUnwrappableAs Class unwrapType return false public lt T gt T unwrap Class lt T gt unwrapType return null Using a custom MultiTenantConnectionProvider The SessionFactory can be created like this 274 Connector J Fabric Support Reference create a new instance of our custom connection provider supporting MySQL Fabric FabricMultiTenantConnectionProvider connProvider new FabricMultiTenantConnectionProvider fabricUrl employees employees username password create a service registry with the connection provider to construct the session factory ServiceRegistryBuilder srb new ServiceRegistryBuilder srb addService org hibernate service jdbc connections spi MultiTenantConnectionProvider class connProvider srb applySetting hibernate dialect org hibernate dialect MySQLInnoDBDialect create the configuration and build the session factory Configuration config new Configuration config setProperty hibernate multiTenancy DATABASE config addResource com mysql fabric demo employee hbm xml return config buildSessionFactory srb buildServiceRegistry Using Hibernate multi tenancy Once you have created a SessionFactory with your
32. 4 4 4 shell gt mysql uroot proot e SHOW CREATE FUNCTION griots adjust_cost G KKKKKKKKKKKKKKKKKKKKKKKKKKK i row KKKKKKKKKKKKKKKKKKKKKKKKKKK TPINCGEICING tadus icosE sql_mode Create Function CREATE DEFINER root localhost FUNCTION adjust_cost cost double RETURNS double DETERMINISTIC feb Coe IKIO character_set_client latinl collation_connection latinl_swedish_ci Database Collation latinl_swedish_ci Discussion In this example we see the use of the database pattern cost to find objects that have cost anywhere in their name We also see the use of the body option to instruct the utility to look inside procedures and functions This can be very handy to locate routines that manipulate data as you can see Notice once we found a routine that had cost mentioned we can examine its body via the SHOW CREATE FUNCTION command to see just how it is using the column cost In this case we see someone has written a function to adjust the cost by 10 Therefore not only can you find objects that have anything named cost you can also discover any hidden logic that may operate on something named cost Permissions Required The user must have the SELECT privilege on the mysql database Tips and Tricks If you are familiar with using regular expressions you can use the regexp option to use regular expressions instead of database patterns For example the r
33. 5 19 mysqlrplsync Replication synchronization checker cccecececeeeeeeeceeeeeeeeeeeeeeeeeeeeeeeeeaees 174 5 20 mysqlserverclone Clone Existing Server to Create New Server 00ccccceeeeeeeeeaeeeeeeees 181 5 21 mysqlserverinfo Display Common Diagnostic Information from a Server sesse 184 5 22 mysqluc Command line client for running MySQL Utilities 2 00 0 cece cece ee eteeeeeeeeeeeeeaes 187 5 23 mysqluserclone Clone Existing User to Create New User ccccccseeeeceeeeeeeeeeeeeeeeeseaees 191 This chapter includes the manual pages for each of the utilities Each manual page is formatted similar to a typical Unix man page 5 1 mysqlauditadmin Allows users to perform maintenance action on the audit log This utility allow you to maintain the audit log allowing you to monitor the audit log file growth and control its rotation Rotation refers to the action of replacing the current audit log file by a new one for continuous use renaming with a timestamp extension and copying the previously used audit log file to a defined location This utility allows you to view and modify a subset of audit log control variables display the audit log file status perform on demand rotation of the log file and copy files to other locations These features enable you to easily monitor the audit log file growth and control its rotation automatically based on the defined file size threshold or manually by a
34. How Do Display Connections by the User root and Show the Result in CSV Format 1146 201S O8 OlWiss2ilso0s Gmesay TSERE INO mecie moet exis Aplo c VAS 1 2 5 37 1054 2OLS OF OlWiSs23 510 Oes WNSHIE INIT cesiccicll Karor nor Ew WANES 1 2 5 37 1146 e20ts Os SOT ie 26 4 Ce SUPDATH l oe eaae SE a all Sy 1054 2O0LS OF OIWIGs26553 Ghesy WET icesiccicil Saw mor coll il 37 4 4 a 4 Discussion As expected the use of the mysqlauditgrep utility requires the specification of the target audit log file to search and a few options corresponding to the needed search criteria In this case the query type option was used to restrict the displayed results to specific types of queries i e only INSERT and UPDATE and the status option was used to specify the considered MySQL error codes i e all ranging from 1 to 9999 The query type option allows the specification of a comma separated list of different SQL statements Apart from INSERT and UPDATE the list of supported values for this option also includes CREATE ALTER DROP TRUNCATE RENAME GRANT REVOKE SELECT DELETE COMMIT SHOW SET CALL PREPARE EXECUTE DEALLOCATE The status option accepts a comma separated list of non negative integers corresponding to MySQL error codes or intervals marked with a dash For example 1051 1100 1199 1146 In this particular
35. INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS IN NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE Cookie management The Cookie module contains the following notice Copyright 2000 by Timothy O Malley lt timo alum mit edu gt All Rights Reserved Permission to use copy modify and distribute this software and its documentation for any purpose and without fee is hereby granted provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of Timothy O Malley not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission Timothy O Malley DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS IN NO EVENT SHALL Timothy O Malley BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ProT The prof
36. KREKKKKKKKKK KK KKK KKK KKKKKEK Ls row KRKKKKKKKKKKKKKKKKKKKKKEKEK localhost 3306 Sala 50 1eg usr local mysql data fusr local mysql 5 1 50 osx10 6 x86_64 fusr local mysql 5 1 50 osx10 6 x86_64 lib plugin server version datadir basedir plugin dir config_file binary_log binary_log_pos relay_log etc my cn my_log 000068 212383 None relay_log_pos None 1 rows Defaults for server localhost 3306 port 3306 basedir usr local mysql datadir usr local mysql data server_id 5 log bin my_log general_log slow_query_log innodb_data_file_path ibdatal 778M ibdata2 50M autoextend done PERMISSIONS REQUIRED The permissions required include the ability to read the mysql database and to have read access to the data directory The user must have permissions to read the data directory or use an administrator or super user Sudo account to obtain access to the data directory 5 22 mysqluc Command line client for running MySQL Utilities This utility provides a command line environment for running MySQL Utilities The mysqluc utility hence console allows users to execute any of the currently installed MySQL Utilities command The option ut ildir is used to provide a path to the MySQL Utilities if the location is different from when the utility is executed 187 mysqluc Command line client for running MySQL Utilities The console has a
37. OK test_rplsync_dol tl checksum for server localhost s3i2 SKIP Table t2 check for localhost 3311 filtered by replication rule SKIP Table t2 check for localhost 3312 filtered by replication rule SKIP Table t3 check for localhost 3311 filtered by replication rule SKIP Table t3 check for localhost 3312 filtered by replication rule SKIP Database test_rplsync_db0O check filtered by replication rule SKIP Database test_rplsync_db2 check filtered by replication rule SKIP Database test_rplsync_db3 check filtered by replication rule done SUMMARY No data consistency issue found PERMISSIONS REQUIRED The user for the master must have permissions to lock tables perform the checksum and get information about the master status Specifically the user used to connect to the master requires the following privileges SUPER or REPLICATION CLIENT LOCK TABLES and SELECT The user for the slaves must have permissions to start stop the slave perform the checksum and get information about the slave status More specifically the login user to connect to slaves requires the following privileges SUPER and SELECT 5 20 mysqlserverclone Clone Existing Server to Create New Server This utility enables you to clone an existing MySQL server instance to create a new server instance on the same host The utility creates a new datadir new data and on Unix systems star
38. Permitted format values are grid csv tab sql and vertical The default is grid report indexes r Reports if a table has neither UNIQUE indexes nor a PRIMARY key server lt source gt Connection information for the server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt show drops d Display DROP statements for dropping indexes show indexes i Display indexes for each table Skip S Skip tables that do not exist ssl ca The path to a file that contains a list of trusted SSL CAs 134 NOTES e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key
39. The user account information is stored in the configuration file The user account must have full privileges for the database named fabric To create the user and grant the privileges needed use the following statements CREATE USER fabric localhost IDENTIFIED BY secret GRANT ALL ON fabric TO fabric localhost In the preceding example substitute a password of your choice replace secret Also if you are going to run MySQL Fabric on a host other than where the backing store resides substitute the localhost for the host name MySQL Fabric uses the same user account who must have all privileges on all databases to access all MySQL servers that it will manage The user and password are defined in the configuration file as we shall see To create this user and grant all the necessary privileges execute the following command on all MySQL servers CREATE USER fabric localhost IDENTIFIED BY secret GRANT ALL ON TO fabric localhost In the preceding example substitute a password of your choice replace secret Also if you are going to run MySQL Fabric on a host other than where the managed MySQL servers reside substitute the localhost for the Fabric s host name Configuration File The next step is to modify the configuration file with the user and password we defined in the previous step Open the configuration file MySQL Fabric configuration file location Table 8 2 MySQL Fabric
40. and over what time period before the failover is initiated What happens when the primary master MySQL Server fails The MySQL Fabric node will orchestrate the promotion of one of the Secondaries to be the new Primary This involves two main actions e Promoting the Secondary to be the replication master and any other surviving Secondaries will become slaves to the new master Updating the routing information such that Fabric aware connectors will no longer send any queries or transactions to the failed Primary and instead send all writes to the new Primary Does my application need to do anything as part of the failover No The failover is transparent to the application as the Fabric aware connectors will automatically start routing transactions and queries based on the new server topology The application does need to handle the failure of a number of transactions when the Primary has failed but before the new Primary is in place but this should be considered part of normal MySQL error handling Is a recovered MySQL Server automatically put back into service No the user must explicitly invoke MySQL Fabric to return a recovered MySQL Server to a HA Group This is intentional so that the user can ensure that the server really is ready to take on an active role again Does MySQL Fabric work with semisynchronous replication In this version MySQL Fabric sets up the HA Group to use asynchronous replication If the user prefers
41. file var log fabric log mysqldump_program usr bin mysqldump mysqlclient_program usr bin mysql storage section store the user and password of the user created in the previous step 228 Quick Start Stetistics prune_time 3600 failure_tracking motifications 300 notification_clients 50 notification_interval 60 failover_interval 0 detections 3 detection_interval 6 detection_timeout 1 prune_time 3600 connector etl i Each section has one or more variables defined that provide key information to the MySQL Fabric system libraries You should not have to change any of these variables other than the user and password for the backing store in the storage section 8 3 Quick Start This section demonstrates how to get started using MySQL Fabric Two examples are included in this section one for using MySQL Fabric with replication to demonstrate how Fabric reduces the overhead of directing reads and writes from applications and another showing how Fabric makes using sharding much easier If you have not installed and configured Fabric please refer to the previous sections before proceeding with the examples 8 3 1 Example Fabric and Replication This section presents a quick start for using MySQL replication features in Fabric To run this example you should have four server instances running MySQL version 5 6 10 or later The commands in this example are executed on the same server
42. portions copyright 200 Autenomous Zones Industries Inex ald rights err reserved and offered to the public under the terms of the Python 2 2 license Author Zooko O Whielacronx http zooko com mailto zooko zooko com Copyright 2000 Mojam Media Inc all rights reserved Author Skip Montanaro Copyright 1999 Bioreason Inc all rights reserved Author Andrew Dalke Cojowicnegine IS s LOS 7 Muicomeiecais iine aulil aeee eserves Author Skip Montanaro Copyright 1991 1995 Stichting Mathematisch Centrum all rights reserved Permission to use copy modify and distribute this Python software and its associated documentation for any purpose without fee is hereby granted provided that the above copyright notice appears in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of neither Automatrix Bioreason or Mojam Media be used in advertising or publicity pertaining to distribution of the software without specific written prior permission UUencode and UUdecode functions The uu module contains the following notice Copyright 1994 by Lance Ellinghouse Cathedral City California Republic United States of America All Rights Reserved Permission to use copy modify and distribute this software and its documentation for any purpose and without fee is hereby granted provided that the above copyright notice appear in all
43. user mysql log error logs me local err pid file logs me local pid port 3306 Notice we now have all of the options from the configuration file as well as the startup options We can now construct the proper options for creating a clone of this server using the mysqlserverclone utility Specifically we can set the following options using the mysqid option e log bin my_log e general_log e slow_query_log user mysq e log error lt path gt Using these options and choosing a new data directory we can create a new instance of the host server using the following command shell gt mysqlserverclone server root root localhost new data source temp_clone new port 3307 root root delete new id 123 mysqld log bin my_log general log slow query log user mysql log error source temp_clone Cloning the MySQL server running on localhost Creating new data directory Configuring new instance roca cingimy sgi ESOS Setting up empty database and mysql tables Starting new instance of the server Testing connection to new instance Success Setting the root password Connection Information 49 How Can Find What MySQL Servers Are Running uroot proot socket source temp_clone mysql sock done Now that we have a running instance we can export all of the data from the host to the clone shell gt mysqldbexport server root root lo
44. 283 9 2 Third Party LICGNS S ssscuictssaniooccessneoscnasaveeennd EEES vabdecead Ueedeaaehuencend genset vaedecend ees 284 9 2 1 Doctrine DBAL 234 sen caeanadescace coseavadaeseecencretuvatanssasvedesnieadaadeaeecesieeaseds ieee teedes 284 9 2 2 Python LICENSE ssceesenieecideestekeyncdebseauenes EAEE ESRAR EAE SANEERATAAN TRAA 285 9 2 3 Python License Supplement Windows Only ssssssssssssssesrssssssssnrrrrrsrssssrrrrrrerssns 295 vi Preface This is the User Manual for the MySQL Utilities MySQL Utilities is both a set of command line utilities as well as a Python library for making the common tasks easy to accomplish The library is written entirely in Python meaning that it is not necessary to have any other tools or libraries installed to make it work It is currently designed to work with Python v2 6 or later and there is no support yet for Python v3 1 Layout This manual is arranged in an order designed to provide a quick reference for how to use MySQL Utilities It begins with a brief introduction of MySQL Utilities then presents a list of common administration tasks with examples of how utilities can be used to perform the tasks From there the manual begins a deeper dive into the utilities starting with overviews of each utility leading to a detailed description of each via a manual page format Thus the manual provides a documentation solution for several needs How to Use This Manual You can use this man
45. 4 4 PERMISSIONS REQUIRED The user must have the SELECT privilege on the mysql database 5 13 mysqlprocgrep Search Server Process Lists This utility scans the process lists for the servers specified using instances of the server option and selects those that match the conditions specified using the age and match xxx options For a process to match all conditions given must match The utility then either prints the selected processes the default or executes certain actions on them 140 Options If no age or match xxx options are given the utility selects all processes The match xxx options correspond to the columns in the INFORMATION_SCHEMA PROCESSLIST table For example mat ch command specifies a matching condition for PROCESSLIST COMMAND column values There is no mat ch t ime option To specify a condition based on process time use age Processes that can be seen and killed are subject to whether the account used to connect to the server has the PROCESS and SUPER privileges Without PROCESS the account cannot see processes belonging to other accounts Without SUPER the account cannot kill processes belonging to other accounts To specify how to display output use one of the following values with the format option e grid default Display output in grid or table format like that of the mysql client command line tool e c
46. Differences in the data are shown using a similar diff style format Changed or missing rows are shown ina standard format of GRID CSV TAB or VERTICAL Use the notation db1 db2 to name two databases to compare or alternatively just do1 to compare two databases with the same name The latter case is a convenience notation for comparing same named databases on different servers 78 mysqldbcompare Compare Two Databases and Identify Differences The comparison may be run against two databases of different names on a single server by specifying only the server1 option The user can also connect to another server by specifying the server2 option In this case db1 is taken from server1 and db2 from server2 All databases between two servers can also be compared using the a11 option In this case only the databases in common with the same name between the servers are successively compared Therefore no databases need to be specified but the server1 and server2 options are required Users can skip the comparison of some of the databases using the exclude option Note R The data must not be changed during the comparison Unexpected errors may occur if data is changed during the comparison The objects considered in the database include tables views triggers procedures functions and events A count for each object type can be shown with the vv option The check is performed using a series of steps called te
47. During this waiting step the slaves status is periodically polled according to a predefined time interval This polling interval to verify if the slaves are synced can be adjusted with the interval 175 option A checksum query is used to compare the data of each table between servers The checksum calculation step is skipped if its execution exceeds a predefined time avoiding undesirable performance impacts on the target system if it takes too long to execute The user can change the checksum timeout using the checksum timeout 175 option Users can also use the verbose 176 option to see additional information when the utility executes This utility is designed to work exclusively for servers that support global transaction identifiers GTIDs and have gt id_mode ON Servers with GTID disabled will be skipped by the utility See Replication with Global Transaction Identifiers for more information about GTID The utility takes into consideration the use of replication filtering rules on the servers skipping the check for filtered databases and tables according to the defined options Nevertheless the use of replication filters can still lead to data consistency issues depending on how statements are evaluated See How Servers Evaluate Replication Filtering Rules for more information OPTIONS mysqlrplsync accepts the following command line options e checksum timeout lt checksum_timeout_in_seconds gt Maximum timeout in second
48. EATE EESE AE TEE A I TA EET E AA E E ST 19 3 2 5 How Can Synchronize a Table on Two Servers Where Neither is Up to date 20 3 S General Operations a eee ie kite te he a a a 21 3 3 1 How Can Find Out How Much Space My Data Uses ccccceeeeeeeeeeeeeeeeeeeeeeeeeeeaeaaeeees 21 3 3 2 My Server Crashed Need to Know the Structure of a Table How Can Do That 23 3 3 3 Creating a New User With The Same Privileges as Another User secese 24 3 3 4 What Options Are Used With Each Utility 20 00 00 ecceecceceeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaaaneeeeees 25 3 3 5 I ve Got Too Many Indexes How Do Know Which Ones to Drop ecceseeeeeeeeeeeeeeeee 28 3 3 6 Need to Find an Object on My Server But All Have is a Partial Name How Do Find All Objects with That Name Prefix cccccccceeeececeeeeeeeeeeeeeeeeeeeeseeseeeeseesseeeeeeseeeeseeeeeeeseeaeegeeeeaaenes 30 3 3 7 How Can Run a Process Every Night To Kill Certain Connections seese 32 3 4 High Availability Operations 2 0 2 2 cece ee eeee cece eee ee ee eee e ee ee eaten eres ee tsrs eaten sees ee aeaaaaieeeeeeeeaeaaaaaeeees 33 3 4 1 How Can Use Replication ii serako anaa Ea A AAN TAANE aAa 33 3 4 2 How Do Add New Servers to My Topology and Change Master Role o s 35 3 4 3 Setup Automatic Failover sec sehnen aae ane ka vice tees aaia aaa a eaa Taaa Ea et vies 37 3 4 4 Restore the Previous Master After Failover sssse
49. For local operations as in the preceding example the key argument must be specified to indicate which row to use For global operations do not specify the key attribute because the operation is performed on all rows in the table fonx set_property tables employees employees scope fabric SCOPE_GLOBAL cur fcnx cursor cur execute UPDATE employees SET last_name UPPER last_name cnx commit fcnx close The default mode is read write so the driver connects to the master The mode property can be given to specify read write or read only mode explicitly fonx set_property group myGroup mode fabric MODE_READWRITE cur fcnx cursor cur execute UPDATE employees SET last_name UPPER last_name ene Commits fcnx close Applications for which read only mode is sufficient can specify a mode attribute of fabric MODE_READONLY In this case a connection is established to a slave if one is available or to the master otherwise Connector Python 2 0 1 or later supports RANGE_STRING and RANGE_DATETIME as sharding types These are similar to the regular RANGE sharding type but instead of an integer key require a value of a different type e For RANGE_STRING a UTF 8 encoded string key is required For example cnx set_property tables employees employees key u employee_name mode fabric MODE_READONLY Only Unicode strings are supported Any other type given when using a shard define
50. N gt The maximum recursion depth This option is valid only if recurse is given num retries lt num_retries gt n lt num_retries gt The number of retries permitted for failed slave login attempts This option is valid only if prompt is given prompt p Prompt for the slave user and password if different from the master user and password If you give this option the utility sets num ret ries to 1 if that option is not set explicitly This ensures at least one attempt to retry and prompt for the user name and password should a connection fail quiet q Turn off all messages for quiet execution This option does not suppress errors or warnings recurse f Traverse the list of slaves to find additional master slave connections User this option to map a replication topology show list l Display a column list of the topology ssl ca 171 NOTES The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose v Specify how much information to display If this option is used the IO thread status of each slave is also displa
51. OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT INFRINGE ANY THIRD PARTY RIGHTS 4 BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE SOFTWARE FOR ANY INCIDENTAL SPECIAL OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF USING MODIFYING OR DISTRIBUTING THE SOFTWARE OR ANY DERIVATIVE THEREOF EVEN IF ADVISED OF THE POSSIBILITY THEREOF 5 This License Agreement will automatically terminate upon a material breach of its terms and conditions 6 This License Agreement shall be governed by and interpreted in all respects by the law of the State of California excluding conflict of law provisions Nothing in this License Agreement shall be deemed to create any relationship of agency partnership or joint venture between BeOpen and Licensee This License Agreement does not grant permission to use BeOpen trademarks or trade names in a trademark sense to endorse or promote products or services of Licensee or any 287 Python License third party As an exception the BeOpen Python logos available at http www pythonlabs com logos html may be used according to the permissions granted on that web page 7 By copying installing or otherwise using the software Licensee agrees to be bound by the terms and conditions of this License Agreement CNRI LICENSE AGREEMENT FOR PYTHON 1 6 1 1 This LICENSE AGREEMENT is between the Corporation for National Research Initiati
52. SQL transformation statement output 108 OPTIONS The changes for option controls the direction of the difference by specifying the object to be transformed in either the difference report default or the transformation report designated with the difftype sql option Consider the following command shell gt mysqldiff serverl root host1 server2 root host2 difftype sql db1 tablel dbx table3 The leftmost database db1 exists on the server designated by the server1 option host1 The rightmost database dbx exists on the server designated by the server2 option host 2 e changes for server1 Produces output that shows how to make the definitions of objects on server1 like the definitions of the corresponding objects on server2 changes for server2 Produces output that shows how to make the definitions of objects on server2 like the definitions of the corresponding objects on server1 The default direction is server1 For the sql difference format you can also see the reverse transformation by specifying the show reverse option The utility stops at the first occurrence of missing objects or when an object does not match To override this behavior specify the force option to cause the utility to attempt to compare all objects listed as arguments OPTIONS mysqldiff accepts the following command line options e help Display a help message and exit e license Display
53. SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection 146 NOTES e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required test db lt test_database gt The database name to use for testing the replication setup If this option is not given no testing is done only error checking e verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES The login user for the master server must have the appropriate permissions to grant access to all databases and have the ability to create user accounts For example the user account used to connect to the master must have the WITH GRANT OPTION privilege The server IDs on the master and slave must be nonzero and unique The utility reports an error if the server ID is 0 on either server or the same on the master and slave Set these values before starting this utility Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectiv
54. Specify the export format Permitted format values are definitions export only the definitions metadata for the objects in the database list data export only the table data for the tables in the database list and both export the definitions and the data The default is definitions file per table Write table data to separate files This is Valid only if the export output includes data that is if export data or export both are given This option produces files named db_name tb _name format For example a csv export of two tables named t 1 and t2 in database di results in files named db1 t1 csv and db1 t2 csv If table definitions are included in the export they are written to stdout as usual format lt format gt f lt format gt Specify the output display format Permitted format values are sql grid tab csv and vertical The default is sql locking lt locking gt Choose the lock type for the operation Permitted lock values are no locks do not use any table locks lock all use table locks but no transaction and no consistent read and snapshot consistent read using a single transaction The default is snapshot multiprocess Specify the number of processes to concurrently export the specified databases Special values 0 number of processes equal to the number of detected CPUs and 1 default no concurrency Multiprocessing works at the database level for Windows and at the table level for No
55. The goal is to create one or more users whose permissions are identical to an original user on a single server 24 What Options Are Used With Each Utility Rather than discover what those privileges are with a SHOW GRANTS FOR statement and copy them into a script modify it copy and paste again for each user etc etc we can use a single command to copy one user to a list of new users We can even set different passwords for each user as we go Let s assume we have a user joe localhost who has a long list of permissions We need to create a clone of his user account for two new users sally and john Each of these users will require a new password Example Execution shell gt mysqluserclone source root localhost destination root localhost joe localhost sally secretl localhost john secret2 localhost Source on localhost connected Destination on localhost connected Cloning 2 users Cloning joe localhost to user sally secretl localhost Cloning joe localhost to user john secret2 localhost done Discussion In the above example we see the use of the mysqluserclone utility to clone the joe user to two new user accounts Notice we used the source option to connect to the original server and dest ination for the same server After that we simply list the user we want to clone and the new users we want to create In this case we use the format username password host to specify the user ac
56. The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required timeout lt seconds gt Maximum timeout in seconds to wait for each replication command to complete For example timeout for slave waiting to catch up to master Default 300 seconds verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug version Display version information and exit NOTES The login user must have the appropriate permissions to execute SHOW SLAVE STATUS SHOW MASTER STATUS and SHOW VARIABLES on the appropriate servers as well as grant the REPLICATE SLAVE privilege The utility checks permissions for the master slaves and candidates at startup Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse n
57. This section contains information that the MySQL Fabric node uses for the connection to the backing store For more information on the backing store see Section 8 5 Backing Store 221 Configuring MySQL Fabric address user password database auth_plugin connection_timeout connection_attempts connection_delay Section servers This is the address of the backing store in the form host port The port is optional and if not provided defaults to 3306 User name to use when connecting to the backing store The password to use when connecting to the backing store If no password option is in the configuration file a password is required on the terminal when the MySQL Fabric node is started Although it is possible to set an empty password by not providing a value to the option it is not recommended The database where the tables containing information about the MySQL Fabric farm is stored typically fabric The authentication plugin to use when connecting to the backing store This option is passed to the connector when connecting to the backing store For more information on authentication plugins see Connector Python Connection Arguments Timeout for the connection to the backing store in seconds This option is passed to the connector when connecting to the backing store This is the maximum amount of time that MySQL Fabric waits for access to the backing store to complete before aborting For more
58. Topology and Change Master Role 00 35 3 4 3 Setup Automatic Failover 0 ccccccecceeeeeeeeeee cece ee ae tees eeeeeeaeaaaeeeeeeeeeseeaaaaneeeeeeeeeeeaaa 37 3 4 4 Restore the Previous Master After Failover 00 cccceeeecesneeeeeeaaeeeeeeaaeeeeeeaaeeeeeeaaeeeees 40 3 4 5 How Can Find All of the Slaves Attached to My Master Server sseeeeeeeeeees 43 3 4 6 How To Check If Data Is Correctly Replicated cceceeeeeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeees 44 3 4 7 How To Fix Errant Transactions on the Replication Topology s eseeeeeeeeeeeeees 46 3 5 Server Operations aisanana e ave ave ee daa 48 3 5 1 How Do I Make A Temporary Copy of My Server For Testing cceceecseeeeeeeeeees 48 3 5 2 How Can Find What MySQL Servers Are RUNNING cceeeeeeeeeeeeeeeeeeeaeaaeeteeeeeees 50 3 5 3 How Can Use a secure encrypted connection between Utilities and a MySQL SONVEr 2 hive nel a a a a in late eee a ea eel 51 3 6 Specialized Operations ccccccececcceceeeeeeeeeee ee ae eae eeeeeeeeeeaaeaaaaeeeeeeeeaeaaaaaeeeeeeseeaeaaaneeeeeeees 55 3 6 1 How Do Record Only Login Events ccccccceeeeeeeeeeeeeeeaaeeeeeeeeeeeeeeaaaaeeeeeeeeeeeaaa 55 3 6 2 How Do Copy Move The Audit Log ccceeeeeeeeeee ee eeeeeeeeeeeeeeeeaaeaeeeeeeeeeeaeaaaaaenees 57 3 6 3 How Do Show All INSERT and UPDATE Queries That Failed cssseeeeeeeeees 58 My
59. Total size of InnoDB files 889 192 448 bytes or 848 00 MB nnoDB freespace 635 437 056 bytes or 606 00 MB Discussion To see all of the logs we use the a11 option which shows all logs and the InnoDB disk usage Notice we used elevated privileges to allow for reading of all of the files and databases in the data directory In this case the data directory is owned by the mysql user and a normal user account does not have read access The a11 option instructs the utility to list all databases even if they contain no data 22 My Server Crashed Need to Know the Structure of a Table How Can Do That Permissions Required The user must have permissions to read the data directory or use an administrator or super user Sudo account as shown in the example Tips and Tricks You can run mysqldiskusage without privileges to read the data directory but in this case you will see an estimate of the disk usage rather than actual bytes used You may also not be able to see a list of the logs 3 3 2 My Server Crashed Need to Know the Structure of a Table How Can Do That When things go wrong badly enough that your server is down but you can still access the disks you may find yourself faced with a number of complex recovery tasks One of those is the need to discover the structure of a particular table or set tables Perhaps this is needed for an emergency recovery a redeployment or setup for a forensic in
60. USING BTREE The following indexes for table test_db indexcheck_test contain the clustered index and might be redundant CREATE INDEX idx_uemp_id ON test_db indexcheck_test emp_id USING BTREE CREATE INDEX dx_id_name ON test_db indexcheck_test emp_id name USING BTREE CREATE INDEX idx_id_hdate ON test_db indexcheck_test emp_id hire_date USING BTREE CREATE INDEX idx_id_bday ON test_db indexcheck_test emp_id birthday USING BTREE Discussion As we can see the utility first points out that neither the idx_uemp_id index nor the idx_fnumber are necessary and it points out why The first idx_uemp_id is redundant because the primary key already ensures that emp_id values have to be unique As for idx_fnumber it is also redundant because of idx_ufnumber a UNIQUE type index which also works as a regular index Then it points out that idx_full_name_dup is also not necessary In this case it is a duplicate of the idx_full_name index since it contains the exact same columns on the same order Notice that it also indicates that idx_name idx_surname and even the PRIMARY INDEX on emp_id might be redundant This happens because we are dealing with BTREE type indexes and for this type of indexes an index X is redundant to an index Y if and only if the first n columns in X also appear in Y Given that we are using InnoDB engine it also warns us that idx_uemp_id idx_id_name
61. Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt 120 OPTIONS daemon lt command gt Run as a daemon The command can be start start daemon stop stop daemon restart stop then start the daemon or nodetach start but do not detach the process This option is only available for POSIX systems discover slaves login lt user password gt At startup query master for all registered slaves and use the user name and password specified to connect Supply the user and password in the form lt users lt passwad gt or lt ogin path gt For example discover joe secret will use joe as the user and secret as the password for each discovered slave exec after lt script gt Name of script to execute after failover or switchover Script name may include the path exec before lt script gt Name of script to execute before failover or switchover Script name may include the path exec fail check lt script gt Name of script to execute on each interval to invoke failover exec post failover lt script gt Name of script to execute after failover is complete and the utility has refreshed the health report failover mode lt mode gt f lt mode gt Action to take whe
62. Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt e ssl ca The path to a file that contains a list of trusted SSL CAs e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required e verbose vV Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects in the operation The account used to connect to the source server must have privileges to read the mysql database The account used to connect to the destination server must have privileges to
63. a simple SQL statement that can retrieve all of the users on a server For our purposes we get all of the users except the root and anonymous users then add each to a list for processing later users serverl exec_query SELECT user host PROM mysql user WHERE user root and user for user in users user_list append user 0 user 1 Now we must clone the original server and create a viable running instance When you examine the mysqlserverclone utility code you see that it calls another module located in the mysql utilities command sub folder These modules are where all of the work done by the utilities take place This enables you to create new combinations of the utilities by calling the actual operations directly Let s do that now to clone the server The first thing you notice in examining the serverclone module is that it takes a number of parameters for the new server instance We supply those in a similar way as options parser add_option new data action store dest new_data type string help the full path to the location of the data directory for the new instance parser add_option new port action store dest new_port type string default 3307 help the new port for the new instance default default parser add_option new id action store dest new_id type string default 2 help the server_id for the new instance default default from mysql ut
64. about an offline server the utility starts the server in read only mode In this case you must specify the basedir datadir and start options to prevent the utility from starting an offline server accidentally Note Be sure to consider the ramifications of starting an offline server on the error and similar logs It is best to save this information prior to running this utility To specify how to display output use one of the following values with the format option e grid default Display output in grid or table format like that of the mysql client command line tool e CSV Display output in comma separated values format tab Display output in tab separated format vertical 184 OPTIONS Display output in single column format like that of the G command for the mysq1 client command line tool To turn off the headers for grid csv or tab display format specify the no headers option To see the common default settings for the local server s configuration file use the show defaults option This option reads the configuration file on the machine where the utility is run not the machine for the host that the server option specifies To run the utility against several servers specify the server option multiple times In this case the utility attempts to connect to each server and read the information To see the MySQL servers running on the local machine use the show servers option This shows a
65. above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software THE SOFTWARE IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM DAMAGES OR OTHER LIABILITY WHETHER IN AN ACTION OF CONTRACT TORT OR OTHERWISE ARISING FROM OUT OF OR IN CONNECTION 294 Python License Supplement Windows Only WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE Select kqueue The select and contains the following notice for the kqueue interface Copyright c 2000 Doug White 2006 James Knight 2007 Christian Heimes All rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are Mers 1 Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution HIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS AS IS AND NY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE MPLIED WARRANTI
66. abstract MySQL objects devices and mechanisms For example there is a server class that contains operations to be performed on servers such as connecting logging in and running queries The MySQL Utilities Library Although the library is growing the following lists the current common modules and the major classes and methods as of the 1 0 1 release Module database dbcompare format options rpl See Vics table tools Class Method Database get_create_object diff_objects check_consistency format_tabular_list format_vertical_list print ist setup_common_options add_skip_options check_skip_options check_format_option add_verbosity check_verbosity add_difftype add_engines check_engine_options parse_connection Replication get_replication_tests get_connection_dictionary find_running_servers connect_servers Server Index Table get_tool_path Description Perform database level operations Retrieve object create statement Diff definitions of two objects Check data consistency of two tables Format list in either GRID or delimited format to a file Format list in a vertical format to a file Print list based on format CSV GRID TAB or VERTICAL Set up option parser and options common to all MySQL Utilities Add common skip options Check skip options for validity Check format option for validity Add verbosity and quiet options Check whether both verbosity and qui
67. aed dha ae 2 1 6 OS X ta hen tad sewed eae DA eed ee a ee A a a N 3 2 MrOi che eae ss sitions tend aint E cine ha ees ei ea ee dae 5 2 1 Introduction to MySQL Utilities 2 0 cece teeter ee ee eae te ee ee ee ee ae aaa teeeeeeeeeaeaaaaneeeeeeeeeaeaae 5 2 2 Connecting to MYSQL Servers ecececeeeceeeeeeeeeeeee eee eee ee ener ee ee ae aa eeteeeeeeeeeaeaaeaeeeeeeeeeaeaaaaeenees 6 2 2 1 Connection Parameters cccccceeeeee cece ee ae ea eeeeeeee esse ae aaaateeeeeeeeeaeaaaaaeeeeeeseeaeaaaaaeneeeees 6 2 2 2 Specifying Connections in Python Library 0 ccccceeeeeceeeee eee eeeeeeeeeeeeaeaaeaeeeeeeeeeeeeaaa 9 3 MySQL Utilities Administrative Tasks ccccccceeeeeee cece eee ee eeee sees ee aa aaa eeeeeeeeeeaeaaaeeeeeeeeeeaaaaaeneeeeeeees 11 3 1 Binary Log Operations aeneo enra cece cece cee e cette neces eee aad eee Aaaa ao A te eeeeee seas aaaaeeeeeseseeaaaaaaees 11 3 1 1 How To Change The Location For Binary Log Files 0 cccceeeeeeaeeeeeeeeeeeeeeeaeaaeees 12 3 1 2 How do you purge obsolete binary log files Safely cceceeeseeeeeeeeeeeeeeeaeeeeeeeeeeeeees 13 3 2 Database Operations mor aiiin ika ieee evades dea Db ay eae Goa 15 3 2 1 How Do Provision a Slave ccccceceeeeeee ae eeeeeeeeeeeeee ae aaeeeeeeeeeeeeaeaaeaeeeeeeeeeeeaaaaeeees 15 3 2 2 How Do Make a Copy of a Database on the Same Server c ecceceeseeeeeeaeeeeees 16 3 2 3 How Can Make a Copy of a Database and Change the Sto
68. allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To perform best slave election for a topology with GTID MODE ON server version 5 6 5 or higher where all slaves are specified with the slaves option run the following command shell gt mysqlrpladmin master root localhost 3331 slaves root localhost 3332 root localhost 3333 root localhost 3334 elect Electing candidate slave from known slaves Best slave found is located on localhost 3332 done To perform best slave election supplying a candidate list use the following command shell gt mysqlrpladmin master root localhost 3331 slaves root localhost 3332 root localhost 3333 root localhost 3334 candidates root localhost 3333 root localhost 3334 elect Electing candidate slave from candidate list then slaves list Best slave found is located on localhost 3332 Hne done To perform failover after a master has failed use the following command shell gt mysqlrpladmin slaves root localhost 3332 root localhost 3333 root localhost 3334 candidates root localhost 3333 root localhost 3334 failover Performing failover Candidate slave localhost 3333 will become the new master Preparing candidate for failover Creating replication user if it does not exist Stopping slaves Performing STOP on all slaves
69. are used Set lt Server gt ServerGroup getServers Return the servers in this group String Server getHostname Return the server host name int Server getPort Return the server port Implementing MultiTenantConnectionProvider To begin we declare the class with members to keep necessary information for the connection and the constructor public class FabricMultiTenantConnectionProvider implements MultiTenantConnectionProvider a connection to the MySQL Fabric manager 272 Using Hibernate with MySQL Fabric private FabricConnection fabricConnection the database and table of the sharded data private String database private String table user and password for Fabric manager and MySQL servers private String user private String password shard mapping for database table private ShardMapping shardMapping global group for the shard mapping private ServerGroup globalGroup public FabricMultiTenantConnectionProvider String fabricUrl String database String table String user String password eey this fabricConnection new FabricConnection fabricUrl user password this database database this table table this user user this password password eagerly retrieve the shard mapping and server group from the Fabric manager this shardMapping this fabricConnection getShardMapping this database this table this globalGroup this fabricConnection getServer
70. available to others as provided herein then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python 1 6 1 4 CNRI is making Python 1 6 1 available to Licensee on an AS IS basis CNRI MAKES NO REPRESENTATIONS OR WARRANTIES EXPRESS OR MPLIED BY WAY OF EXAMPLE BUT NOT LIMITATION CNRI MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1 6 1 WILL NOT NFRINGE ANY THIRD PARTY RIGHTS 5 CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 6 1 FOR ANY INCIDENTAL SPECIAL OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING DISTRIBUTING OR OTHERWISE USING PYTHON 1 6 1 OR ANY DERIVATIVE THEREOF EVEN IF ADVISED OF THE POSSIBILITY THEREOF 6 This License Agreement will automatically terminate upon a material breach of its terms and conditions 7 This License Agreement shall be governed by the federal intellectual property law of the United States including without limitation the federal copyright law and to the extent such U S federal law does not apply by the law of the Commonwealth of Virginia excluding Virginia s conflict of law provisions Notwithstanding the foregoing with regard to derivative works based on Python 1 6 1 that incorporate non separable material that was previously distributed under the GNU General Public License GPL the law of the Commonwealth of Virginia sh
71. both option to ensure we include the definitions as well as the data We also add the rpl1 master option which instructs mysqldbexport to generate the replication commands with respect to the source server being the master Lastly we include the replication user and password to be included in the CHANGE MASTER command Next we see an attempt to run the import using mysqldbimport but we see there is an error The reason for the error is the mysqldbimport utility detected a possible problem on the slave whereby there were global transaction identifiers GTIDs recorded from the master You can see this situation if you setup replication prior to running the import The way to resolve the problem is to run the RESET MASTER command on the slave as shown in the next operation 15 How Do Make a Copy of a Database on the Same Server The next operation is a rerun of the import and in this case it succeeds We see a warning that is issued any time there are replication commands detected in the input stream whenever GTIDs are enabled Permissions Required The user used to read data from the master must have the SELECT privilege on all databases exported The user on the slave must have the SUPER privilege to start replication Tips and Tricks The warning issued during the import concerning GTIDs is to ensure you are aware that the process for gathering the proper GTIDs to execute on the slave include transactions from all databases Th
72. case a non supported replication filtering option is detected on a server the utility issues an appropriate error and exits This check is performed at the beginning when the utility starts EXAMPLES To check the data consistency on an active replication system explicitly specifying the master and slaves shell gt mysqlrplsync master user pass localhost 3310 slaves rpl1 pass localhost 3311 rpl pass localhost 3312 GTID differences between Master and Slaves Slave localhost 3311 is 15 transactions behind Master Slave localhost 3312 is 12 transactions behind Master Checking data consistency Using Master localhost 3310 as base server for comparison 177 EXAMPLES Checking test_rplsync_db database Checking 0 table data OF test_eplsync cb t0 checksum for server localhost 3211 OK test_rplsync_ db t0 checksum for server localhost Q32i2 Checking til table data OK testrplsyic cb ti checksum for server localhost 32i11 OK test_rplsync_ cb tl checksum for server localhost Q32i2 Checking test_db database Checking 0 table data OK testib 0 checksum for server lecalhnosLG s31i OK Cestio 0 checksum for server localhost 3312 Checking tl table data OK test db El checksum for server localhost G 231i OK testib 1 echeckeum for server localhost G 3312
73. check any discovered slave in an unordered way Permissions Required The user must have permissions to configure replication Tips and Tricks In the above example the mysqifailover utility was started in the default console mode but it can also be executed as a daemon For that purpose the daemon option needs to be used more specifically simply add daemon start to the command line When mysqlfailover is executed as a daemon no output is displayed and all the information is logged to file specified for the 10g option which is mandatory in this case To stop the execution of the mysql failover daemon simply invoke the utility using the option daemon stop No other options is required to stop the daemon unless a specific pidfile to store the process PID was specified with the pidfile option to start the daemon and in this case the same option value is also required to stop it Another useful feature is the possibility to run external script along the execution of the utility to perform customized actions The following options can be used to execute different scripts at distinct moments of the mysql failover execution exec fail check to specify a script to run periodically at each predefined interval instead of the default check i e master is reachable and alive to detect the need to failover exec before to specify a script to execute before starting failover exec after to execute a script at the end of failove
74. data match state sleep DECLARE kill_done INT DECLARE kill_cursor CURSOR FOR SELECT Id User Host Db Command Time State Info FROM INFORMATION_SCHEMA PROCESSLIST WHERE user LIKE www data AND State LIKE sleep OPENI kami cuiiesio 12 BEGIN DECLARE id BIGINT DECLARE EXIT HANDLER FOR NOT FOUND SET kill_done 1 loop OOP EBETCH kill_cursor INIO id KILL CONNECTION id END LOOP kill_loop END CHOGIm Veni eiiesgieies e ssl ca The path to a file that contains a list of trusted SSL CAs e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key file to use for establishing a secure connection 143 NOTES e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose vV Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value The path to the MySQL clien
75. disclaimer in the documentation and or other materials provided with the 289 Python License Gli Hier owhe aleje 3 The names of its contributors may not be used to endorse or promote products derived from this software without specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Any feedback is very welcome http www math keio ac jp matumoto emt html email matumoto math keio ac jp Sockets The socket module uses the functions getaddrinfo and getnameinfo which are coded in separate source files from the WIDE Project http www wide ad jp Cejoyieseioe CC 1999S 1996 1997 aiael IOS MED Drees All rights reserved Redistribution and use in source and b
76. discussion begins by setting forth the sharding related properties In the examples f cnx represents the Fabric connection object created as shown in Section 8 7 2 Requesting a Fabric Connection is requested from the Fabric connection object or when its cmd_query or cmd_query_iter method is invoked Note R set_property does not connect The connection is opened when a cursor These set_property arguments are shard related group A high availability group name tables The sharding table or tables mode Whether operations are read write or read only 259 Providing Information to Choose a MySQL Server scope Whether operations are local or global key The key that identifies which row to affect group and tables are mutually exclusive so you specify only one of them Applicability of the remaining arguments depends on which of group or tables you use If you specify group mode is optional The default is fabric MODE_READWRITE if this property is omitted scope is inapplicable Do not specify it key is inapplicable Do not specify it If you specify tables mode is optional The default is fabric MODE_READWRITE if this property is omitted scope is optional The default is fabric SCOPE_LOCAL if this property is omitted e key If scope is fabric SCOPE_LOCAL key is required to indicate which row to affect If scope is fabric SCOPE_GLOBAL key is inapplicable do not specify
77. distribution This guide is based on Ubuntu 14 04 but any other distribution should work equally well Unpartitioned space on the local disk to create a DRBD partition e Network connectivity e Both hosts must be accessible through ssh e User mysql and group mysql that have the same ids at the different nodes Linux is used because Pacemaker Corosync and DRBD are commonly available on this platform 245 Target Configuration Note R Pacemaker Corosync and DRBD are not include with MySQL Fabric and need to be installed separately for the target platform If Virtual Machines are used make sure they run in different physical servers to avoid a single point of failure If possible also make the network connectivity redundant 8 6 3 Target Configuration The two physical hosts are host1 localdomain 192 168 1 101 and host2 localdomain 192 168 1 102 It is recommended that you do not rely on an external DNS service as that is an additional point of failure and so these mappings should be configured on each host in the etc hosts file A single Virtual IP VIP is shown in the figure 192 168 1 200 and this is the address that the application will connect to when accessing the MySQL Fabric Pacemaker will be responsible for migrating this between the two hosts One of the final steps in configuring Pacemaker is to add network connectivity monitoring in order to attempt to have an isolated host stop its MySQL serv
78. does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the replication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqlrplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu net you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To check the prerequisites of a master and slave that currently are actively performing replication use the following command shell gt mysqlrplcheck master root host1 3310 slave root host2 3311 master on hostl connected slave on host2 connected Test Description Status hecking for binary logging on master pass re there binlog exceptions pass eplication user exists pass hecking server_id values pass s slave connected to master pass k master information file pass hecking InnoDB compatibility pass hecking s
79. entries for a specific date time range use the following command shell gt mysqlauditgrep start date 2012 09 27T13 33 47 end date 2012 09 28 SERVER data audit log t gt ZOTO OAT OIA S092 TSR AOA S027 TALIS SS AOI S097 7 ALS SS ZO OO 2ielles l l l l l l l l l l l l tester tester tester tester tester tester tester tester root root root root root root root root root root 4 4 4 4 4 4 4 SERVER_ID NAME TIMESTAMP CONNECTION_ID HOST USER o 4 4 Connect 2012 OOTTE 26250 9 localhost root Query AOS OSAT AGES SiO 3 None root Ping 202 095 267 26250 9 None root Query AVI 2 09 2E0 AGS SiO 3 None root Query AVI 209 280 AGE SiO 3 None root Ping 20 12 09 281 26 50 9 None root Query AVI 2 09 28 will s 268 Sw 3 None root Quit AVI 2 O9 Z2E wii s 268 Sw 3 None root Connect 20s I 11 S355 at localhost tester2 Query AMI 2 1G 10 wis 355555 1 None tester2 Query AML2 1LO iL Owls 3 56 S10 1 None tester2 Query AVIL2 1G 10TIS 3s 57826 1 None tester2 Query AVI Z 1O 1O wis 357s 36 1 None tester2 Query AVL 2 1LO 1 Owls s 57s Sil 1 None tester2 Guaita 20 e5 ill None tester2 Connect 2012 10 LOT SA2 2 localhost tester2 Query 20E OTOT AA 2 None tester2 3 Quit AML2 1LO 10 wily says 22 2 None tester2 4
80. execute CREATE USER and DROP USER if the force option is given and privileges to execute GRANT for all privileges to be granted to the new accounts For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value 193 EXAMPLES The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To clone joe aS sam and sally with passwords and logging in as root on the local machine use this command shell gt mysqluserclone source root localhost destination root localhost joe localhost sam secretl localhost sally secret2 localhost Source on localhost connected Destination on localhost connected Cloning 2 users Cloning joe localhost to user sam secretl localhost Cloning joe localhost to user sally secret2 localhost done The following command shows all users on the local server in the most verbose output in CSV format shell gt mysqluserclone source root localhost list format csv vvv Source on localhost connected user
81. file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt demote master Make master a slave after switchover discover slaves login lt slave_login gt At startup query master for all registered slaves and use the user name and password specified to connect Supply the user and password in the form lt user gt lt passwa gt or lt ogin path gt For example discover joe secret will use joe as the user and secret as the password for each discovered slave exec after lt script gt Name of script to execute after failover or switchover Script name may include the path exec before lt script gt 157 OPTIONS Name of script to execute before failover or switchover Script name may include the path force Ignore prerequisite checks or any inconsistencies found such as errant transactions on the slaves or SQL thread errors thus forcing the execution of the specified command This option need to be used carefully as it will not solve any detected issue but only ignores them and displays a warning message format lt format gt f lt format gt Display the replication health output in either grid default tab csv or vertical format log lt log_file gt Specify a log file to use for logging messages log age lt days gt Specify maximum age of log entries in days Entries
82. file sizes may be omitted Some features may be unavailable 190 PERMISSIONS REQUIRED Database totals 4 4 db_name total 4 4 world 4 4 Total database disk usage 1 072 359 052 bytes or 1022 00 MB done The following demonstrates launching the console using the commands shown above but piped into the console on the command line The results are the same as above shell gt echo set SERVER root host123 mysqldiskusage server S SERVER mysqluc The following demonstrates launching the console and setting variables via the command line shell gt mysqluc SERVER root host123 VAR_A 57 e show variables Variable Value SERVER root host123 VAR_A ST PERMISSIONS REQUIRED There are no special permissions required to run mysqluc however you must have the necessary privileges to execute the desired utilities See the PERMISSIONS REQUIRED section for each command you wish to execute 5 23 mysqluserclone Clone Existing User to Create New User This utility uses an existing MySQL user account on one server as a template and clones it to create one or more new user accounts with the same privileges as the original user The new users can be created on the original server or a different server To list users for a server specify the 1 ist option This prints a list of the us
83. following values with the format option sql default Display output using SQL statements For definitions this consists of the appropriate CREATE and GRANT statements For data this is an INSERT statement or bulk insert if the bulk insert option is specified grid Display output in grid or table format like that of the mysql client command line tool e CSV Display output in comma separated values format tab Display output in tab separated format 94 mysqldbexport Export Object Definitions or Data from a Database vertical Display output in single column format like that of the G command for the mysq1 client command line tool To specify how much data to display use one of the following values with the display option e brief Display only the minimal columns for recreating the objects e full Display the complete column list for recreating the objects names Display only the object names Note R The display option is ignored when combined with the SQL format output type To turn off the headers for csv or tab display format specify the no headers option To turn off all feedback information specify the quiet option To write the data for individual tables to separate files use the file per table option The name of each file is composed of the database and table names followed by the file format For example the following command produces files named db1 table_n
84. for table world_myisam City are missing because the new storage engine for the table is not InnoDB Replacing ENGINE InnoDB with ENGINE myisam for table world_myisam Country Copying TABLE world_innodb Country Replacing ENGINE InnoDB with ENGINE myisam for table world_myisam CountryLanguage Copying TABLE world_innodb CountryLanguage WARNING FOREIGN KEY constraints for table world_myisam CountryLanguage are missing because the new storage engine for the table is not InnoDB Copying data for TABLE world_innodb City Copying data for TABLE world_innodb Country Copying data for TABLE world_innodb CountryLanguage done shell gt mysql uroot proot h 127 0 0 1 port 3307 e SHOW DATABASES 4 Database 4 information_schema mysql sakila world world_myisam 4 shell gt mysql uroot proot h 127 0 0 1 port 3307 e SHOW CREATE TABLE world _myisam countrylanguage G KKKKKKKKKKKKKKKKKKKKKKKKKKK le row KKKKKKKKKKKKKKKKKKKKKKKKKKK Table countrylanguage Create Table CREATE TABLE countrylanguage CountryCode char 3 NOT NULL DEFAULT Language char 30 NOT NULL DEFAULT US Owevedeil menun aT a NCAT NUEL IDEyAOMar TE Percentage float 4 1 NOT NULL DEFAULT 0 0 PRIMARY KEY CountryCode Language KEY CountryCode CountryCode ENGINE MyISAM DEFAULT CHARSET latinl Discussion Notic
85. from a group add Add a server to a group lookup_servers Return information on existing servers in a group health Check whether any server within a group has failed and report health information destroy Remove a group demote Demote the current master if there is one promote Promote a server into master lookup_groups Return information on existing groups description Update a group description 8 4 3 Manage Commands The manage commands are used to control Fabric nodes m m anage anage anage anage anage anage teardown Tear down Fabric Storage System stop Stop the Fabric server setup Set up Fabric Storage System ping Check whether Fabric server is running start Start the Fabric server logging_level Set the logging level for module This is only mainly for debugging 8 4 4 Server Commands The server commands are used to get information about servers and set their properties namely status mode and weight server set_weight Seta server s weight which determines its likelihood of being chosen to process requests or replace a failed master The value must be greater than 0 0 and lower or equal to 1 0 server lookup_uuid Return a server s UUID value 239 Sharding Commands server set_mode Seta server s mode which determines whether it can process read_only read_write or both transaction types It can also be set to offline meaning that the server will
86. host as the backing store which happens to be the same host where Fabric was installed You must also have a Fabric node started on that host The replication features in Fabric focus on providing high availability While these features continue to evolve the most unique feature of Fabric replication is the ability to use a Fabric aware connector to seamlessly direct reads and writes to the appropriate servers This redirection is achieved through the use of one of the central concepts in Fabric a high availability group that uses a high availability solution for providing resilience to failures Currently only asynchronous primary backup replication is supported As long as the primary is alive and running it handles all write operations Secondaries replicate everything from the primary to stay up to date and might be used to scale out read operations Creating a High Availability Group The first step consists of creating a group here designated my_group After doing so you can add servers to the group In this case we add three servers Localhost 3307 localhost 3308 and Localhost 3309 Fabric accesses each added server using the user and password provided in the configuration file to guarantee that they are alive and accessible If these requirements are not fulfilled the operation fails and the server is not added to the group 229 Example Fabric and Replication The following demonstrates the commands to execute the
87. host1 etc init d drbd status root host2 etc init d drbd status In order to create the file systems and go on to storing useful data in it one of the hosts must be made the primary for the clusterdb_res resource so execute the following on host1 root host1 drbdadm overwrite data of peer primary all root host1 etc init d drbd status The status output also shows the progress of the block level syncing of the device from the new primary host1 to the secondary host2 This initial sync can take some time but it should not be necessary to wait for it to complete in order to complete the other steps Now that the device is available on host1 it is possible to create a file system on it root host1 mkfs t ext4 dev drbd0 Note R The above does not need to be repeated on the second host as DRBD will handle the syncing of the raw disk data In order for the DRBD file system to be mounted the var 1ib mysql_drbd directory should be created on both hosts root host1 mkdir var 1lib mysql_drbd root host1 chown mysql var 1lib mysql_drbd root host1 chgrp mysql var 1lib mysql_drbd root host2 mkdir var 1lib mysql_drbd root host2 chown mysql var 1lib mysql_drbd root host2 chgrp mysql var 1lib mysql_drbd 249 Setting up and testing your system On just the one DRBD active host the DRBD file system must be temporarily mounted root host1 mount dev drbd0 var lib mysql_drbd 8
88. information for the destination server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt exclude lt exclude gt x lt exclude gt Exclude one or more objects from the operation using either a specific name such as db1 t1 or a search pattern Use this option multiple times to specify multiple exclusions By default patterns use LIKE matching With the regexp option patterns use REGEXP matching This option does not apply to grants drop first 89 OPTIONS Drop each database to be copied if exists before copying anything into it Without this option an error occurs if you attempt to copy objects into an existing database Note R Before MySQL Util
89. information on authentication plugins see Connector Python Connection Arguments The number of attempts to reconnect to the backing store before giving up This is the maximum number of times MySQL Fabric attempts to create a connection to the backing store before aborting The default is 0 retries The delay between attempts to connect to the backing store in seconds The default is 0 seconds This section contains information that MySQL Fabric uses to connect to the servers being managed Note R The backup_user backup_password restore_user and restore_password options were added in MySQL Fabric 1 5 5 1 6 2 user password backup_user backup_password restore_user restore_password unreachable_ timeout User name to use when connecting to the managed server Password to use when connecting to the managed servers User name to use when backing up the MySQL server Password to use when backing up the MySQL servers with the backup_user user User name to use when restoring the MySQL server Password to use when restoring the MySQL servers with the restore_user user Used for the connection timeout when checking faulty servers or servers that are new to the farm Hence for servers that can potentially be unreachable Defaults to 5 can be a value between 1 60 222 Configuring MySQL Fabric Section protocol xmirpc This section contains information about how the client connects to a MySQL Fabric node an
90. is MySQL Fabric 8 9 2 8 9 3 8 9 4 8 9 5 8 9 6 MySQL is a framework for managing groups of MySQL Servers and using those servers to provide services It is designed to be extensible so that over time many different services can be added In the current version the services provided are High Availability built on top of MySQL Replication and scale out by sharding the data MySQL Fabric is implemented as a MySQL Fabric node process which performs management functions and Fabric aware connectors that are able to route queries and transactions directly to the most appropriate MySQL Server The MySQL Fabric node stores state and routing information in its State Store which is a MySQL database Is it necessary to use a MySQL Fabric specific Storage Engine No The MySQL Servers that are being managed by MySQL Fabric continue to use InnoDB and in the future NDB MySQL Cluster may also be supported What versions of MySQL are supported by MySQL Fabric Currently MySQL 5 6 New MySQL releases will be fully supported as they reach General Availability status What connectors support MySQL Fabric Java PHP Python and NET In addition the Hibernate and Doctrine Object Relational Mappings frameworks are also supported Connector C 6 2 also adds fabric support as a labs release Are transactions ACID Yes Because each transaction is local to a single MySQL Server all of the ACID behavior of the InnoDB storage engin
91. makes the slaves replicate only what is stored in the master s binary log from the present moment onward Furthermore using the master log file and master log pos options it is possible to specify respectively the master log file and the master log position from which the slave will start its replication process The p flag can be used to ensure that the replication setup is only executed in case the storage engines match in both the master and the slave Regarding the mysqirplcheck utility we can use the s option to check the output of the show slave status command This can be useful for instance to check what might be causing the Is slave connected test to fail We can also use the master log file option to specify the name of the master information file to read Lastly we can use the verbose option in order to get more information about what is happening under the hood 3 4 2 How Do Add New Servers to My Topology and Change Master Role We will examine a scenario similar to the previous one where we want to make one of the two new servers added the master server perhaps because it has better specs and is faster Objectives Our goal in this example it create replication configuration with 3 servers two new ones and an existing one and we want to replicate all the information but make one of the new servers the master server Like the previous example lets assume that the existing server Server1 is
92. methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt show options Display the audit log system variables ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required value lt VALUE gt Value used to set variables based on the specified commands such as policy and rotate_on_size verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug version 67 NOTES NOTES Display version information and exit This utility can only be applied to servers with the audit log plugin enabled An
93. modify This architecture scales the write operations for a given dataset since resource demands are distributed across the partitions MySQL Servers of the data set Each partition is referred to as a shard 8 3 2 1 Introduction to Sharding The Fabric sharding implementation requires you to provide the sharding key explicitly while executing a query To define sharding over a set of tables using the sharding mechanism built into Fabric it is important to understand two concepts and how they relate Shard Mapping Shards A shard mapping serves to bring a database object a database table into the Fabric sharding system The mapping is a way of informing Fabric that we want a particular sharding scheme range hash list and so forth to be used on a database table using a value in a particular column Create a shard mapping as follows 1 Define a shard mapping to tell Fabric the kind of sharding mechanism to use 2 Adda relation between the mapping and a database object to register the database table and a column in the table with the shard mapping Once these operations have been completed we can describe how the shard mapping should split the tables This is done while creating the shards These are the partitions on the table Since sharding is done on a database table using an attribute column in the table the values in the column influence how the shards are created To explain this further assume that we have two t
94. multiple instances on the same machine Permissions Required The permissions required include the ability to read the mysql database and to have read access to the data directory Tips and Tricks Notice the output shows the data directory for each server You can use this information to examine the files in that folder to discern more information such as what databases exist and find and examine the binary log etc On POSIX systems you can discover the command line arguments such as the port number the server is using with the ps f PID command For example to discover the complete information for PID 2487 you can do the following shell gt ps f 2487 UID PID PPID C STIME ENG TIME CMD 501 2487 I 0 10 58AM ttys001 0 00 41 source mysql 5 6 sql mysqld no defaults datadir source temp_13002 tmpdir source temp_13002 pid file source temp_13002 clone pid port 13002 server id 102 basedir source mysql 5 6 socket source temp_13002 mysql sock log slave update gtid mode on enforce gtid consistency log bin master info repository TABLE report port 13002 report host localhost 3 5 3 How Can I Use a secure encrypted connection between Utilities and a MySQL Server Security is a big concern and MySQL Utilities is prepared to use a secure connection to MySQL server Using SSL for Secure Connections using an encrypted connection with SSL Objectives Use the mysqlserverclone
95. my cnf shell gt mkdir var 1ib mysql_drbd data shell gt mysql_install_db no defaults datadir var lib mysql_drbd data user mysql Configure some permissions on host1 250 Setting up and testing your system shell gt chmod R uogtrw var 1lib mysql_drbd shell gt chown R mysql var lib mysql_drbd shell gt chmod 644 var lib mysql_drbd my cnf Start MySQL Server and configure users and access shell gt etc init d mysql start shell gt mysql u root e GRANT ALL ON to root shell gt mysql u root e CREATE USER fabric localhost IDENTIFIED BY secret shell gt mysql u root e GRANT ALL ON fabric TO fabric localhost 8 6 4 5 Configure MySQL Fabric On just the one DRBD active host do the following shell gt cp etc mysql fabric cfg var 1lib mysql_drbd fabric cfg shell gt chmod 600 var 1lib mysql_drbd fabric cfg shell gt chown root root var 1lib mysql_drbd fabric cfg Edit the var lib mysql_drbd fabric cfg 1 Set address to 192 168 1 200 32274 inthe protocol xmlrpc section 2 Set password to passwordinthe protocol xmlrpc section Set address to 192 168 1 200 32275 in the protocol mysql section GO Set the password to password in the protocol mysql section 5 Set the password to secret inthe storage section Warning gt Do not change the address in the storage section Take the opportunity to set the other options if you need want to do so specially the use
96. n hell gt mysqldbexport server root pass localhost skip GRANTS export DEFINITIONS util_test Source on localhost connected Exporting metadata from util_test ROP DATABASE IF EXISTS util_test CREATE DATABASE util_test USE util test TABLE util_test t CREATE TABLE el a char 30 DEFAULT NULL ENGINE MEMORY DEFAULT CHARSET latinl1 TABLE util_test t2 CREATE TABLE t2 a char 30 DEFAULT NULL ENGINE MyISAM DEFAULT CHARSET latin1l TABLE util_test t3 CREATE TABLE ES 7 a int 11 NOT NULL AUTO_INCREMENT TOi echar 20 DEFAULT NULE PRIMARY KEY a ENGINE InnoDB AUTO_INCREMENT 4 DEFAULT CHARSET latin1l TABLE util_test t4 CREATE TABLE t4 aune KL NOT NUI cl ame H NOT NOE KEY ref ts e V CONSTRAINT rernC FOREIGN KEY c REFERENCES ES a ENGINE InnoDB DEFAULT CHARSET latin1l VIEW util_test vl coool done Oo Similarly to export the data of the database ut i1_test producing bulk insert statements use this command shell gt mysqldbexport server root pass localhost export DATA bulk insert util_test Source on localhost connected VSE util test Exporting data from util_test Data for table util_test tl INSERT INTO util_test t1l VALUES 01 Test Basic database example 02 Test Basic database example 03 Test Basic database example 04 Test Basic database example 05 Test Basic database
97. name using the db obj format For example use the db1 obj1 db2 0b 42 format to compare two named objects or db1 obj1 to compare an object with the same name in databases with the same name It is not legal to mix a database name with an object name For example db1 obj1 db2 and db1 db2 0b 72 are illegal formats The comparison may be run against a single server for comparing two databases of different names on the same server by specifying only the server1 option Alternatively you can also connect to another server by specifying the server2 option In this case the first object to compare is taken from server1 and the second from server2 By default the utility generates object differences as a difference report However you can generate a transformation report containing SQL statements for transforming the objects for conformity instead Use the sql value for the difftype option to produce a listing that contains the appropriate ALTER commands to conform the object definitions for the object pairs specified If a transformation cannot be formed the utility reports the diff of the object along with a warning statement See important limitations in the NOTES section To specify how to display the diff styled output use one of the following values with the difftype option e unified default Display unified format output context Display context format output e differ Display differ style format output sql Display
98. new location and or the entries in the respective index file updated Objectives Change the location for the binary log files on an existing MySQL server Simplify the operation Executing this task manually can be tedious and error prone since it requires existing binary log files to be moved to the new location and the binary log index file to be correctly updated Fortunately the mysqlbinlogmove utility can help us perform this task in an easy way moving all files and automatically updating the respective index file entries appropriately Let s assume that a server is currently running and that it was started with log bin server bin which means that the binary log files are created in the datadi r with the base filename server bin Let s also consider that the dat adir is var lib mysql and that the new target directory for the binary log files is mysql server binlogs Example Execution 1 Stop the running MySQL server 2 Start the mysqlbinlogmove utility and specify the source directory of the binary log files and the target directory shell gt mysqlbinlogmove binlog dir var lib mysql mysql server binlogs Moving bin log files Z server bin 000001 server bin 000002 server bin 000003 server bin 000004 server bin 000005 server Din 000006 server bin 000007 server bin 000008 server bin 000009 server bin 000010 server bin 000011 done 3 Restart th
99. older than this will be purged on startup Default 7 days master lt connection gt Connection information for the master server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt max position lt position gt Used to detect slave delay The maximum difference between the master s log position and the slave s reported read position of the master A value greater than this means the slave is too far behind the master Default 0 new master lt connection gt Connection information for the slave to be used to replace the master for switchover Valid only with switchover command To connect to a server it is necessary to specify connection parameters such a
100. rest of Appendix A It does not include any code or documentation that is identified in the RFC as being copyrighted The original and principal author of md5 h is L Peter Deutsch lt ghost aladdin com gt Other authors are noted in the change history that follows in reverse chronological order 291 Python License 2002 04 13 lpd Removed support for non ANSI compilers removed references to Ghostscript clarified derivation from RFC 1321 now handles byte order either statically or dynamically 999 11 04 lpd Edited comments slightly for automatic TOC extraction 999 10 18 lpd Fixed typo in header comment ansi2knr rather than md5 added conditionalization for C compilation from Martin Purschke lt purschke bnl gov gt SOS GS 03 pd Original Versie Asynchronous socket services The asynchat and asyncore modules contain the following notice Copyright 1996 by Sam Rushing All Rights Reserved Permission to use copy modify and distribute this software and its documentation for any purpose and without fee is hereby granted provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of Sam Rushing not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE
101. search e Kill processes that match query mysqluserclone e Clone a user account to the same or different server e Show user grants mysqluc e Command line client for running MySQL Utilities e Allows a persistent connection to a MySQL Server Tab completion for utility names and options Allows calling the commands with shorter names such as using serverinfo instead of mysqlserverinfo 62 High Availability Operations 4 3 High Availability Operations These utilities are those designed to support replication and high availability operations for MySQL servers mysqlfailover e Provides automatic failover on a replication topology e Uses Global Transaction Identifiers GTID MySQL Server 5 6 5 mysqlreplicate e Setup replication Start from beginning current specific binlog pos e mysqlrplms e Provides round robin multi source replication a slave server continually cycles through multiple masters in order to store a consolidated data set e Uses Global Transaction Identifiers GTID MySQL Server 5 6 9 mysqlrpladmin e Administers the replication topology Allows recovery of the master e Commands include elect failover gtid health start stop and switchover mysqlrplcheck e Check replication configuration e Tests binary logging on master e mysqlrplshow e Show slaves attached to master e Can search recursively e Show the replication topology as a graph or list mysqlrplsync e Check data con
102. sock util _test util_test_copy Source on localhost connected Destination on localhost connected Copying database util_test renamed as util_test_copy Copying TABLE util_test tl Copying table data CopyaAngelABL Aw itlteste te Copying table data Copying TABLE util_test t3 92 EXAMPLES Copying table data Copying TABLE util_test t4 Copying table data Copying VIEW util_test vl Copying TRIGGER util_test trg Copying PROCEDURE util_test pl Copying FUNCTION util_test fl Copying EVENT util_test el Copying GRANTS from util_test done If the database to be copied does not contain only InnoDB tables and you want to ensure data integrity of the copied data by locking the tables during the read step add a locking lock al1 option to the command shell gt mysqldbcopy source root pass localhost 3310 test123 mysql sock destination root pass localhost 3310 test123 mysql sock util_test util_test_copy locking lock all Source on localhost connected Destination on localhost connected Copying database util_test renamed as util_test_copy Copying TABLE util_test tl Copying table data Copying TABLE util_test t2 Copying table data Copying TABLE util_test t3 Copying table data Copying TABLE util_test t4 Copying table data Copying VIEW util_test vl Copying TRIGGER util_test trg Copying PROCEDURE util_test pl Copying FUNCTION util_test fl Copy
103. specific table The user specifies what column from the table s should be used as the shard key as well as indicating whether to use a HASH or RANGE partitioning scheme for that key if using RANGE based sharding then the user should also specify which ranges map to which shards Currently the sharding key must be numeric When accessing the database the application specifies the sharding key which the Fabric aware connector will then map to a shard ID using the mapping data it has retrieved and cached from MySQL Fabric and route the query or transaction to the correct MySQL Server instance Within a HA group the Fabric aware connector is able to direct writes to the Primary and then spread the read queries across all available Secondaries and optionally the Primary 8 9 2 Does scaling apply to both reads and writes Yes Both reads and writes scale linearly as more HA groups are added Reads can also be scaled independently by adding more Secondary servers to a HA Group 8 9 3 What if have table data that needs to be in every shard 280 MySQL Fabric Frequently Asked Questions 8 9 4 8 9 5 8 9 6 8 9 7 8 9 8 8 9 9 A special group can be created called the Global Group which holds the Global Tables Any table whose data should appear in its entirety in all HA Groups should be handled as a Global Table For a Global Table all writes are sent to the Global Group and then replicated to all of the HA Gro
104. tablespace data employees salaries ibd 146800640 file tablespace data employees titles ibd 41943040 file tablespace Total size of InnoDB files 494 125 056 bytes or 471 00 MB done PERMISSIONS REQUIRED The user must have permissions to read the data directory or use an administrator or super user Sudo account to obtain access to the data directory 5 9 mysqlfailover Automatic replication health monitoring and failover This utility permits users to perform replication health monitoring and automatic failover on a replication topology consisting of a master and its slaves The utility is designed to run interactively or continuously 117 mysql failover Automatic replication health monitoring and failover refreshing the health information at periodic intervals Its primary mission is to monitor the master for failure and when a failure occurs execute failover to the best slave available The utility accepts a list of slaves to be considered the candidate slave This utility is designed to work exclusively for servers that support global transaction identifiers GTIDs and have gt id_mode ON MySQL server versions 5 6 5 and higher support GTIDs See Replication with Global Transaction Identifiers for more information The user can specify the interval in seconds to use for detecting the master status and generating the health report using the interval option At each interval the utility will check t
105. the 10g option specified causes the interface to show the entries in the log file such as MySQL Replication Monitor and Failover Utility Failover Mode auto Next Interval Mon Mar 19 16 06 13 2012 127 PERMISSIONS REQUIRED Master Information Binary Log File Position Binlog _De_DB Binlog_ignoere_DB nyse outin OOOO sL iWailil y GTID Executed Set ANG DAO 2 DML Ie TAV IMENO AEZ BEOOR IE i sce AQIZ OS L UDsSSeSs PM AQIZ OS L UDsSSeSs PM INFO Failover console started ao INFO Failover mode auto ao AVILZ O3 1 e TSeSSess PM INFO Getting health for master localhos AMI2 O3 1 6 TSsSSess PM INFO Master status binlog mysql bin 00 4 peee soo Q quit R refresh H health G GTID Lists U UUIDs L log entries Up Down scroll PERMISSIONS REQUIRED The user must have permissions to monitor the servers on the topology and configure replication to successfully perform the failover operation Additional permissions are also required to register and unregister the running mysqlfailover instance on the master and slaves Specifically the login user must have the following privileges SUPER GRANT OPTION REPLICATION SLAVE RELOAD DROP CREATE INSERT and SELECT The referred permissions are required for the login users used for all servers master slaves and candidates 5 10 mysql frm File reader for frm files The mysql frm utility is designed as a recovery tool that r
106. the Fabric node 262 Providing Information to Choose a MySQL Server Note R If you are using Java 5 you must manually load the driver class before attempting to connect Class forName com mysql fabric jdbc Driver Connection now proceeds normally Connection conn DriverManager getConnection Jdbc mysql fabric fabrichost 32274 database user password To use the Connector J APIs that support MySQL Fabric you must cast the Connection object to the public interface that provides the necessary methods The interfaces are com mysql fabric jdbc FabricMySQLConnection JDBC3 interface compatible with Java 5 and later com mysql fabric jdbc JDBC4FabricMySQLConnection JDBC4 interface compatible with Java 6 and later This interface must be used to access JDBC4 APIs on the connection 8 8 3 Providing Information to Choose a MySQL Server If you create a Fabric connection without providing any information about which data to access the connection cannot function To access a database you must provide the driver with one of the following e The name of a high availability group known by the MySQL Fabric instance to which you ve connected only a single shard mapping or the query is rejected A shard mapping can include several tables but they must be sharded on the same index The following discussion describes both ways of providing information In the examples conn represents the Fabric connection ob
107. the audit log variables it display the audit log settings before and after the execution of the command which can be very handy to confirm that the desired change was performed as expected 3 6 2 How Do Copy Move The Audit Log The audit log information can grow quickly and considerably depending on the type of information written to the audit log files and the activity of the MySQL server Therefore it might be a good idea to copy the audit log files to a different location and free some storage on the server The mysqlauditadmin utility also provides this useful functionality Objectives The goal of this task is to copy an existing audit log file to a different location using the mysqlauditadmin utility It is assumed that the utility is executed on the destination host which must be a non Windows system with the scp Secure CoPy command line program and that must have access to the MySQL remote server and its data directory with the provided credentials user and password It is also assumed that the specified audit log file exists and user has write privileges on the target directory Example Execution shell gt mysqlauditadmin audit log name MySQL SERVER data audit log 13753706179878237 copy copy to ARCHIVE Audit_Logs remote login user1 server1 Copying file from serverl MySQL SERVER data audit log 13753706179878237 to ARCHIVE Audit_Logs userl serverl s password areit log SSS HOGI VOSNE 100 4716 4 6KB s
108. the client connects to a MySQL Fabric node using the MySQL Client Server protocol address user password disable_authentication ssl_ca Host and port of a MySQL Fabric node The port number is typically 32275 and the host is typically Localhost User that the client uses to connect to the MySQL Fabric node Password used when the client connects to the MySQL Fabric node If no password is provided the client requests a password on the command line Whether to disable authentication or not Disabling authentication can be useful when experimenting in a closed environment it is not recommended for normal usage Alternatives are yes or no and are case insensitive Path to a file containing a list of trusted SSL certification authorities CAs 223 Configuring MySQL Fabric ssl_cert The name of the SSL certificate file to use for establishing a secure connection ssl_key The name of the SSL key file to use for establishing a secure connection Section executor This section contains parameters to configure the executor The executor executes procedures in a serial order which guarantees that requests do not conflict The requests received are mapped to procedures which can be executed directly or scheduled through the executor Procedures scheduled through the executor are processed within the context of threads spawned by the executor Usually read operations are immediately executed by the XML RPC session t
109. the functionality provided To interact with Fabric a special utility named mysql fabric provides a set of commands you can use to create and manage groups define and manipulate sharding and much more 8 1 1 Fabric Prerequisites Fabric is designed to work with MySQL servers version 5 6 10 and later The mysqli fabric utility requires Python 2 2 6 or later and Connector Python 1 2 1 or later You must have a MySQL instance available to install the backing store This server must not be a member of a Fabric group Note R The backing store must be a MySQL server 5 6 or later To utilize Fabric in your applications you must have a Fabric aware connector installed on the system where the application is running For more information about using a connector with Fabric see the appropriate connector specific section Section 8 7 Using Connector Python with MySQL Fabric Section 8 8 Using Connector J with MySQL Fabric In summary the following items indicate the prerequisites for using MySQL Fabric e MySQL server 5 6 10 or later for Fabric MySQL servers e MySQL server 5 6 x or later for the backing store e Python 2 2 6 or later for the mysql fabric utility 214 Fabric Concepts A Fabric aware connector to use Fabric in applications Permitted connectors and versions are e Connector Python 1 2 1 or later Connector J 5 1 27 or later 8 1 2 Fabric Concepts This section describes some of the concepts us
110. the login configuration file mylogin cnf Due to a known server issue there are some limitations with the use of temporary tables with multi source replication In order to avoid problems we recommend the execution of all statements for a temporary table in a single transaction See Replication and Temporary Tables for more information EXAMPLES To set up multi source replication among two masters and a slave running on different ports of the same host using the default settings use this command shell gt mysqlrplms slave root root localhost 3306 masters root root localhost 3307 root root localhost 3308 Starting mulii source replication Pmesicm DR CREO OUES Switching to master localhost 2307 master on localhost connected slave on localhost connected Current Master Information 4 4 Position eS Saf Sf SE SES SN ee Binary Log File Binlog_Ignore_DB NS fe are ae clone bin 000001 Soom See eee aera eee ee GTID Executed Set 00a4e027 a83a 1le3 8bd6 28d244017f 26 1 2 Health Status 152 EXAMPLES localhost 330 SLAVE UE ON OK localhost 3308 MASTER UP ON OK ee The following command uses report values to report health GTID and UUID status shell gt mysqlrplms slave root root localhost 3306 masters root root localhost 3307 root root localhost 33
111. the masters need the following privileges SELECT and INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION The slave users need the SUPER privilege The rpl user used as the argument for the rpl user 151 option is either created automatically or if it exists it needs the REPLICATION SLAVE privilege 5 16 mysqlrpladmin Administration utility for MySQL replication This utility permits users to perform administrative actions on a replication topology consisting of a master and its slaves The utility is designed to make it easy to recover from planned maintenance of the master or from an event that takes the master offline unexpectedly The act of taking the master offline intentionally and switching control to another slave is called switchover In this case there is no loss of transactions as the master is locked and all slaves are allowed to catch up to the master Once the slaves have read all events from the master the master is shutdown and control switched to a slave in this case called a candidate slave Recovering from the loss of a downed master is more traumatic and since there is no way to know what transactions the master may have failed to send the new master called a candidate slave must be the slave that is most up to date How this is determined depends on the version of the server see below However it can result in the loss of some transactions that were executed on the downe
112. the requirements to become master failover will chose another server from the slaves list to become the new master if the one specified in by the candidates option is not suitable It is important to keep this behavior differences in mind when deciding which command to apply 3 4 5 How Can I Find All of the Slaves Attached to My Master Server When you have a topology that has grown over time slaves have been added from time to time it may not be so easy to remember which servers are connected as slaves Most often you want to know the state of those slaves at a glance Rather than connect to each slave individually it would be nice to know what the state of each slaves threads using a single command Objectives Show a map of the slaves connected to a master including the state of each slaves threads IO and SQL We can do this with a single command using the mysqli rp1show utility Example Execution shell gt mysqlrplshow master root root localhost 13001 disco root root verbosity master on localhost connected Finding slaves for master localhost 13001 Replication Topology Graph localhost 13001 MASTER a Localh st 13002 I0 wes SOL Yes SLAVE E Localhost 13003 I0 wes SOL Yes SLAVE TE Localh st 13004 I0 wes SOL Yes SLAVE T Lo calh st 13005 I0 wes SOL Yes SLAVE Discussion Notice the use of the mysql rpl1show utility Not only did it show us the slaves attached
113. the servers That is the utility requires the master and slaves using the master 175 and slaves 176 options To compare only slaves the user need only provide the slaves 176 option The utility also provides a feature to discover slaves connected to the master using the discover slaves login 175 and master 175 options To use the discover slaves feature all slaves must use the following startup options report host and report port to specify the correct hostname and port of the server If these are missing or report the incorrect information the slave may not be discovered and therefore not included in the synchronization check The discover slaves feature ignores slaves that cannot be reached By default all data is included in the comparison To check specific databases or tables list each element as a separated argument for the utility using fully qualified names The user can also choose to exclude some databases or tables from the check using the exclude 175 option The utility also provides some important features that allow users to adjust the execution of the consistency check to their system For example the user may wish the utility to minimize execution of the synchronization process To do so the user uses the rpl timeout 176 to define the maximum 174 OPTIONS time for each slave to synchronize More specifically allow slaves to catch up with the master in order to compare the data
114. to retrieve all of the databases on the server db_list 198 Example for db in serverl get_all_databases db_list append db 0 None If you wanted to supply your own list of databases you could use an option like the following You could also add an else clause which would enable you to either get all of the databases by omitting the databases option or supply your own list of databases for example databases db1 db2 db3 databases action store dest dbs_to_copy parser add_option d type string help comma separated list of databases to include in the copy omit for all databases default None if opt dbs_to_copy is None for db in serverl get_all_databases db_list append db 0 None else tor Clo ain Gre clos ico _cojory sallaie 3 db_list append db None Notice we are creating a list of tuples This is because the dbcopy module uses a list of tuples in the form old_db new_db to enable you to copy a database to a new name For our purposes we do not want a rename so we leave the new name value set to None Next we want a list of all of the users Once again you could construct the new solution to be flexible by permitting the user to specify the users to copy We leave this as an exercise In this case we do not have a primitive for getting all users created on a server But we do have the ability to run a query and process the results Fortunately there is
115. to use semisynchronous replication then they can activate it manually after MySQL Fabric has created the replication relationships 279 MySQL Fabric Frequently Asked Questions 8 9 7 Do have to use MySQL Replication for implementing HA At present HA Groups are formed using MySQL Replication future releases may offer further options such as MySQL Cluster or DRBD 8 9 8 Is the MySQL Fabric node itself fault tolerant What happens when the MySQL Fabric node is not available There is currently only a single instance of the MySQL Fabric node If that process should fail then it can be restarted on that or another machine and the state and routing information read from the existing state store a MySQL database or a replicated copy of the state store While the MySQL Fabric node is unavailable Fabric aware connectors continue to route queries and transactions to the correct MySQL Servers based on their cached copies of the routing data However should a Primary fail automated failover will not happen until the MySQL Fabric node is returned to service and so it s important to recover the process as quickly as possible Sharding 8 9 1 How is scaling achieved with MySQL Fabric c ccceceeeeeee eee eeeeeeeeeeeeaeaaeaeeeeeeeeeeaeaaaaneeeeeees 280 8 9 2 Does scaling apply to both reads and writes ccccccccseecececeeeeeeeaeeeeeeeaaeeeeeeaaeeeeeeaseeeeeaaenees 280 8 9 3 What if have table data that needs to be i
116. utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the replication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqlrplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu net you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine Similarly in order to avoid issues mixing local IP 127 0 0 1 with localhost all the addresses 127 0 0 1 will be internally converted to localhost by the utility Nevertheless It is best to use the actual hostname of the master when connecting or setting up replication The utility will check to see if the slaves are using the option master info repository TABLE If they are not the utility will stop with an error The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login confi
117. utility to create a new instance of your installed MySQL Server This new instance will be enabled for secure connections using SSL to establish a secure connection by using 51 How Can Use a secure encrypted connection between Utilities and a MySQL Server the SSL options You can also use an Options file to specify the SSL certificates needed for the secure connection Example Execution To meet this objective you need to supply values for the following options of mysqlserverclone e basedir e new port e new data e mysqld e root password If you are unfamiliar with the previous options you can find more info in the Section 5 20 mysqlserverclone Clone Existing Server to Create New Server section In the mysqld option you need to specify the ssl ca ssl cert and ssl key options with his respective SSL certificate for the new instance of the server By doing this the new server instance will use the given certificates to establish a secure connection If you are uncertain of how to create the SSL certificates please following the steps indicated on Creating SSL and RSA Certificates and Keys The ssl ca ssl cert and ssl key options of mysqlserverclone are used to connect to an existing instance of MySQL in case you need to use ssl to connect to it and these options are not used to indicate the certificates to use by the new server instance For that reason it is necessary to use the mysqld option
118. v verbose vv more verbose vvv debug e version Display version information and exit For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value NOTES You must provide connection parameters user host password and so forth for an account that has the appropriate privileges for all objects accessed during the operation The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To show only the disk space usage for the employees and test databases in grid format the default use this command shell gt mysqldiskusage server root localhost employees test Source on localhost connected Database totals employees 205 979 oa test 4 096 o Total database disk usage 205 983 744 bytes or 196 00 MB done To see all disk usage for the server in CSV format use this command 116 PERMISSIONS REQUIRED shell gt mysqldiskusage server root localhost format csv a vv Source on l
119. 0 ZOUZS 09 gt 2 30 ZG 50 Query SHOW VARIABLES LIKE READ_ONLY 9 0 ZOUZ 0952 30 ZG 50 Query SET NAMES latinl COLLATE latinl_swedish_ci 0 0 ZOUZS 09S 28 ZIG 50 Query SET session autocommit OFF 0 0 ZOUZ 0 9 gt 2130 ZG 50 Query SHOW VARIABLES LIKE READ_ONLY 0 0 ZOUZS 09 gt 2130 ZG 50 Query SET GLOBAL audit_log_flush ON 0 0 ZOUZS09 gt 2130 ZG 50 Query SHOW VARIABLES LIKE audit_log_policy 0 0 ZI 09 287 1is2 6 350 Query SHOW VARIABLES LIKE audit_log_rotate_on_size 0 0 AQLZ 10 iL OPiS 3 36 3 iL 0 Query show databases il 1046 ADILZ LO OTL S351326 Query show tables test il 1046 AQIZ 1LO Owls 7 336 Query show tables test i 0 ADILZ 10 iLO s 735i Query show tables in test il ree a a eg a a en a a se ean ww ge ae se a a e ak nee a eS ye ra sh a a an eh ae hh N A i eee er oe Se eh eg ged Eh OR ped ee a To display the audit log entries of specific event types use the following command shell gt mysqlauditgrep event type Ping Connect SERVER data audit log ee es a EEN a a ey Ear EA OAA EA ee EA EA em ey el ey eh ey eh ge ene ke ey Eg ey me ay ee eh me ag a mn Ea er Say ed ey Sed aah APEA EEA Sm EA ar fee eee See fm ayaa eh El pe ce rey rey oe ry Se eh SS Sa ss STATUS NAME TIMESTAMP CONNECTION_ID HOST USER PRIV_USER IP e PEA ene en ees a a a a pe aca A ae A EATA EA le EA pe ay le ye el mag See Se ele Ech ey Sey ee SS AERA Sal eg el me A meyer eg aah ee ley a ih eh te nee ee al ae a eae E
120. 00 01 Discussion The copy operation can be performed with the mysqlauditadmin utility using the copy command and requiring the following options the audit log name option to specify the path and filename of the audit log file to copy the copy to option to indicate the destination folder and in this example the remote login option to specify the user and remote host where the file is located the user password will be prompted The remote login option is not required if the source and destination location are on the same server where the utility is executed Moreover this option is not supported in Windows system where UNC paths should be used instead Permissions Required The user must have permissions to read the audit log on disk and write the file to the remove location 57 How Do Show All INSERT and UPDATE Queries That Failed Tips and Tricks The name of the audit log file audit 10g by default is defined by the audit_log_file variable displayed by mysqlauditadmin when using the show options option Existing audit log files have a timestamp extension except the one that is currently in use That being said it might be useful to know that it is possible to get information about the existing audit log files using mysqirpladmin for instance to determine which files need to be copied To get this information use the file stats option and the audit log name option specifying the full path of the c
121. 08 n report values health gtid uuid Star emage wiellktci sounmee mejolliesieiom Press CMREe ie Cie Switching to master localhost ss207 master Onm localhost ier me connected slavicyonm localhost conncetccde Current Master Information 4 4 4 4 Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB 4 4 4 4 clone bin 000001 594 N A N A 4 4 4 4 GTID Executed Set 00a4e027 ae3a 1le3 8bd6 28d244017 26 1 2 Health Status 4 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 4 localhost 3307 MASTER UP ON OK localhost 3306 SLAVE UP ON OK localhost 3308 MASTER UE ON OK 4 4 4 4 4 GTID Status Transactions executed on the servers 4 4 4 4 5 5 5 5 5 host moore role Giewel 4 4 4 4 5 5 5 5 5 igcalhost 330 MASTER 00a4e027 a83a 1le3 8bd6 28d244017f26 1 2 iscalhost 3306 SLAVE 00a4e027 a83a 1le3 8bd6 28d244017f26 1 2 iocalhost 3306 SLAVE faf0874f a839 11le3 8bd6 28d244017 26 1 4 4 4
122. 10 slaves rpl1 pass localhost 3311 rpl pass localhost 3312 GTID differences between Master and Slaves Slave localhost 23i1i a6 up to date Slave localhost 2312 te up to date Checking data consistency Using Master localhost 3310 as base server for comparison DIFF Database NOT on base server but found on localhost 3311 only_on_slave_db Checking test_rplsync_db database DIFF Table NOT on base server but found on localhost 3311 t3 DIFF Table NOT on base server but found on localhost 3312 t3 DIFF Table test_rplsync_db t0 NOT on server localhost 3311 Checking t0O table data DIFF test rplsync dab t0 checksum for server localhost 3312 Checking ti table data WARNING Slave not active localhost 3311 Sync skipped DIFF test_rplsync_db tl checksum for server localhost 3311 OK test_eplsync_db tl checksum for server localhost 3212 Checking t2 table data WARNING Slave not active localhost 3311 Sync skipped OK test rplsync db m t2 checksum for server localhost 3311 OK test_rplsync_db t2 checksum for server localhost 3312 Checking only_on_master_db database DIFF Database only_on_master_db NOT on server localhost 3311 DIFF Database only_on_master_db NOT on server localhost 3312 done SUMMARY 8 data consistency issues found Check a replication topology with filtering she
123. 1199 1046 4 4 AOA WOO LS 8 57 2 AS AVIA OLS 957836 AQ Oi ar 7 Bat 5S AOA OAL ac 7 BAN 8 7 Note i y a a a SERVER data audit log SQLTEXT show tables test show tables test select from teste employees where salary gt 500 and salary lt 10 select from test_encoding where value lt gt amp You can view all successful commands with st atus 0 and all unsuccessful commands with status 1 9999 To display the audit log entries matching several search criteria use the following command shell gt mysqlauditgrep users root start date 0 end date 2012 10 10 event type Query query type SET status 0 pattern audit_log SERVER data audit log i l l l l l l l tc ic PERMISSIONS REQUIRED The user must have permissions to read the audit log on disk 4 4 4 4 CONNECTION_ID USER PRIV_USER SQLTEXT 4 4 4 eii SAGE 530 alfa Beeie eerie SET GLOBAL 4 4 4 4 5 3 mysqldbcompare Compare Two Databases and Identify Differences This utility compares the objects and data from two databases to find differences It identifies objects having different definitions in the two databases and presents them in a diff style format of choice
124. 13001 instance_13001 user root password Fark host localhost port 13001 Once your mylogin cnf file is configured you need only specify the section of the mylogin cnf file for the server connection For example the section created in the previous example is instance_13001 Thus we use server instance_13001 The following shows the execution of a utility specifying the login path section shell gt mysqlserverinfo server instance_13001 format vertical Source on localhost connected KEK KKK KKK KK KKK KEKKEKKKKKK KK Te row kkkkkkk k xkkkxkxkx x k k kkkkk kxk x xx server localhost 13001 contig file jetc my cnt erc mysql my sont binary_log clone bin 000001 binary_log_pos 341 relay_log relay_log_pos Wersioms Se Galilee datadir Volumes Source source temp_13001 basedir Volumes Source source bzr mysql 5 6 plugin_dir Volumes Source source bzr mysql 5 6 lib plugin general_llog OFF general_log_file general_log_file_size log_error log_error_file_size slow_query_log OFF slow_query_log_file slow_query_log_file_size Connection Parameters 1 row done See the online MySQL Reference Manual for more information about login paths the mylogin cnf file and the mysql_config_editor Client 2 2 1 2 Use a Configuration File MySQL Utilities can also accept a configuration path and section for the server connection data This allows you to store on
125. 2 169 1 104 IAS Connect 2013 08 01T19 le sts SS localhost woo 127 0 0 1 e a E E a errerrr Discussion To perform this operation the mysqlauditgrep utility requires the indication of the target audit log file as expected two criteria search options and one formatting option to convert the output to the desired format In this case the users option was applied to search the records for the specified user i e root and the event type option to retrieve only event of a specific type i e connect The format option is the one used to define the output format of the obtained search results In this example only the Connect value was used for the event t ype option which correspond to the logging in event when a client connects Nevertheless this option accepts a comma separated list of event types with the following supported values beside Connect Audit Binlog Dump Change user Close stmt Out Connect Create DB Daemon Debug Delayed insert Drop DB Execute Fetch Field List Init DB Kill Long Data NoAudit Ping Prepare Processlist Query Quit Refresh Register Slave Reset stmt Set option Shutdown Sleep Statistics Table Dump Time In terms of output formats the following are supported beside CSV GRID used by default TAB VERTICAL and RAW corresponding to the original XML format of the audit log file Permissions Required The user must have permissions to read the audit
126. 2 03 19 14 44 21 PM INFO Failover complete 2 03 19 14 44 21 PM INFO done To perform switchover and demote the current master to a slave use the following command sh Sh Sh Sh SE OSE OSE ell gt mysqlrpladmin master root localhost 3331 slaves root localhost 3332 root localhost 3333 root localhost 3334 new master root localhost 3332 demote master switchover Performing switchover from master at localhost 3331 to slave at localhost 3332 Checking candidate slave prerequisites Waiting for slaves to catch up to old master Stopping slaves Performing STOP on all slaves Demoting old master to be a slave to the new master 163 PERMISSIONS REQUIRED Switching slaves to new master Starting all slaves Performing START on all slaves Checking slaves for errors Switchover complete done If the replication health report is generated on the topology following the above command it will display the old master as a slave as shown below Replication Topology Health 4 4 4 host port role state gtid_mode health 4 4 4 localhost l 3332 MASTER UP ON OK localhost 333A SLAVE we ON OK localhost SSS SilyAwaa UP ON OK localhost 3334 SLAVE UP ON I OK 4
127. 3 3333 SLAVE UP ON Gl server4 334 SLAVE UP ON neg server5 3315 SLAVE UP ON ORK 4 4 G cwilt R rerresh b healtin CACTI Liste U WWLDS loc emcees Discussion In reality the main and most important step is the execution of the switchover command with the mysqlrpladmin utility The previous steps can be seen as a preparation for switchover The first step simply makes sure that the server is running and that there is no mysql failover instance still running that could affect the correct execution of switchover The second step sets the old master as a slave of the new master because the switchover command can only be performed with slaves This step will also allows the old master to catch up with the new master If many transaction have been performed on the new master it is recommended to wait a while here to let the old master catch up before switchover otherwise the switchover command might take too much time As expected the execution of the switchover command requires the specification of the current and new master with the master and new master options as well as the list of slaves in the topology using the slaves option without need to list the new master The replication user is specified with the rpl user option In this specific example the use of the option demote master is important because without it the current master S
128. 6 When the job started finished double 16 6 When the job finished The error_log table contains information on servers errors reported Table 8 4 error_log Column Type Comment server_uuid varchar 40 The server_uuid value from the server that has raised an error reported timestamp 6 When the error was reported reporter varchar 64 Who reported the error IP address host name error text Error message or code reported The group_replication table defines replication among global groups and groups used in shards They are used primarily for shard splitting moving and global updates Table 8 5 group_replication Column Type master_group_id varchar 64 slave_group_id varchar 64 The groups table contains information about the groups being managed Table 8 6 groups Column Comment group_id varchar 64 The identifier for the group description varchar 256 A description of the group master_uuid varchar 40 The server_uuid value from the master server master_defined status timestamp 6 bit 1 When the current master was defined 1 if the group is being monitored 0 otherwise The permissions table contains information on rights to access the different sub systems in Fabric Currently a core sub system is formally defined 242 Backing Store Tables Table 8 7 permissions Column Type Comment
129. 6 4 4 Configure MySQL Server First we have to stop the MySQL Server on both hosts and update configuration files and create new data files First stop the MySQL server on both hosts using the command shell gt etc init d mysql stop Note R If you are using an Ubuntu distribution you need change the etc apparmor d usr sbin mysqldon both hosts according to the following diff 40 8 40 8 usr share mysql r Allow data dir access var lib mysql r jen L io mysc EE eaka var lib mysql_drbd r var lib mysql_drbd rwk Allow log file access var log mysql r 35 de If you do not do that the MySQL server will not be able to access files in the new location and you will get strange errors since AppArmor prevents reading and writing from the new locations To avoid any mismatches the configuration file can be copied from host1 to host2 shell gt scp etc apparmor d usr sbin mysqld host2 etc apparmor d usr sbin mysqld Then restart AppArmor on both hosts using shell gt etc init d apparmor restart Edit the etc mysql my cnf file and set datadir var lib mysql_drbd data inthe mysqld section on both hosts To avoid any mismatches the configuration file can be copied from host1 to host2 shell gt scp etc mysql my cnf host2 etc mysql my cnf Now the configuration file can be copied and the default database files populated on host1 using shell gt cp etc mysql my cnf var lib mysql_drbd
130. A eh aay no LE _ 0 Connect 2012 09 27TL 333333 7 localhost root root 127 0 0 Ping A0VIZ O09 2 113 3 33339 7 None None None None 0 Ping A0VIZ O9 2 7133 33339 7 None None None None 0 Ping Z20IZ2 09 2 711133 33339 7 None None None None 0 Ping A0IZ 09 2 71133 333 39 7 None None None None 0 Connect AVILA D 2 7 0133 33315 BY 8 localhost root root 127 0 0 Ping AVLASW9 2 7 093 335 BY 8 None None None None PERMISSIONS REQUIRED 0 Ping 2012S 9 T 27 Tis a9 8 None None None None 0 Ping 202 VSo27 Til see A 7 None None None None 0 Connect 201 2 09 28T 26250 5 Tocalhost root tester 127 0 0 1 0 Ping 20 2 0S 28T 26250 5 None None None None 0 Ping 201 2 09 28T 26250 9 None None None None 0 Connect 201 2 09 28T 26250 0 localhost root root 127 0 0 1 0 Ping 201 2 09 28T 26250 0 None None None None 0 Ping 201 2 09 28T 26250 0 None None None None 0 Ping 201 2 09 28T 26250 0 None None None None 0 Ping 2012 VS 28T 26250 0 None None None None 0 Ping 2012 VS 28T 26250 0 None None None None 0 Connect 2012 10 i0T IS 255255 ll localhost tester root 127 0 0 1 0 Connect 2012 LOOT A 2 localhost tester root 127 0 0 1 4 4 4 4 To display the audit log entries with a specific status use the following command shell gt mysqlauditgrep status 1100
131. ARE IS PROVIDED BY ERIC YOUNG AS IS AND NY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE MPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE RE DISCLAIMED IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE OR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY UT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF UCH DAMAGE HPH j p no he licence and distribution terms for any publically available version or derivative of this code cannot be changed i e this code cannot simply be copied and put under another distribution licence including the GNU Public Licence 297 298
132. ATH SPYTHONPATH mysqldiskusage server root root localhost all Source on localhost connected Database totals Q or 5 w 3 ct O ct w ja 21 How Can Find Out How Much Space My Data Uses bvm 1S 129 db1 uae db2 Lil O35 employees ANG LY OI griots TIPAS mysql Ose 22 oltpl Le yee room_temp 9 847 sakila PS VR test GAE SI test_arduino 97999 welford_kindle V2 O32 world MY WS world_innodb 829 669 4 4 Total database disk usage 210 175 251 bytes or 200 44 MB Log information log_name size 4 hostla Log S Arn AGD INO LA Ss lowa Lee ISmo AVE host I3 local crr SPARTA A BOs 5 4 Retell elize wie logs Ip 27S 1s Vis ieS wre il iS CE Binary log information Current binary log file my_log 000287 a my_log 000285 252208 my_log 000286 256 my_log 000287 3063 my_log index 48 Totals ize ot Dinary logs 2550 5 pyte or 4 9m sons Server is not an active slave no relay log information InnoDB tablespace information 4 4 innodb_file 4 ib_logfileOdO Sie 2D felleK ale _Il yejigat hen 5 242 880 ibdatal iS 792 123 ibdata2 52 428 800 4
133. C 230s B SOs wereqeicwois Al weceqecSiccmme 2 ESAE Eo ERINO NON rs close ps close 4 Connect to the global group and clean up connection setServerGroupName fabric_testl_global statement executeUpdate drop table if exists employees statement close connection close Here is an alternative using the DataSource API Copyright c 2015 Oracle and or its affiliates All rights reserved 3 z he MySQL Connector J is licensed under the terms of the GPLv2 http www gnu org licenses old licenses gpl 2 0 h tml gt like most MySQL Connectors here are special exceptions to the terms and conditions of the GPLv2 as it is applied to his software see the FOSS License Exception http www mysql com about legal licensing foss exception html gt AMEN i ar This program is free software you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation version 2 of the License This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General Public License for more details You should have received a copy of the GNU General Public License along with this program if not write to the Free Software Foundation Inc 51 Franklin St Fifth Floor Boston MA 02110 1301 USA E pa
134. Column Type Comment user_id int 10 unsigned User s internal ID username varchar 100 User s name protocol varchar 200 Protocol that the user is allowed to use to access Fabric and its sub systems password varchar 128 Hashed user s password 8 5 2 Protecting the Backing Store The backing store is very important to Fabric You should take steps to ensure the database is backed up and the server where it resides is stable and well maintained For the purposes of backup it is sufficient to make periodic backups using either the mysqldump client or mysqidbexport utility in MySQL Utilities 244 Using MySQL Fabric with Pacemaker and Corosync 8 6 Using MySQL Fabric with Pacemaker and Corosync There are a number of ways to make the MySQL Fabric node and its status and configuration data highly available this section describes one such approach but other alternatives are possible This section can be used as a Set of guidelines to build your own framework to add fault tolerance Note E The described Pacemaker setup is not currently a solution that has been through QA 8 6 1 Introduction A MySQL Fabric instance is composed of a MySQL Fabric process together with a MySQL server The MySQL Fabric node contains the protocols and the executor but is in itself stateless in that if it for some reason crashes re starting it will just make it continue where it left off The configuration and state information about the farm is
135. Continuing to press G key cycles through the three GTID lists If the list is longer than the screen permits as shown in the example above the scroll up and down help is also shown In this case press the down arrow key to scroll down Press U to view the list of UUIDs used in the topology for example MySQL Replication Monitor and Failover Utility Failover Mode auto Next Interval Mon Mar 19 16 02 34 2012 Master Information Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB mysql bin 000001 571 126 EXAMPLES GTID Executed Set 2A67DE00 2DA1 11R2 A711 00764F2BE90F 1 7 UUIDs 4 4 4 4 host Epone role uuid 4 4 4 eliccallhost osoll MASTER 55c65a00 71fd 1llel 9f80 ac64ef85c961 elo callhosta oso SLAVE 5dd30888 71fd 1lel 9f80 dce242138b7ec tecalhost 3333 SLAVE 65ccbb38 71fd llel 9f80 bda8146bdb0a_ ptocalliosime oo o4 SLAVE 6dd6abf4 71fd 1le1 9f80 d406a0117519 4 4 4 Q quit R refresh H health G GTID Lists U UUIDs L log entries If once the master is detected as down and failover mode is auto or elect and there are viable candidate slaves the failover feature will engage automatically and the user will see the failover messages appear When failov
136. Data Is Correctly Replicated Once the replication system is setup and running it is not uncommon that one might want to verify if the data is being correctly replicated on the slaves In normal circumstances the same data is expected on the master and its slaves excluding the use of filtering rules Nevertheless faults at the data level can introduce inconsistent changes on servers without raising any kind of error These data inconsistencies can result from bugs hardware malfunction human errors or unauthorized access It is desirable to detect these issues in order to fix them and ultimately prevent them from happening again Determining the cause of such issue might not be an easy task since it might be caused by byzantine failures at distinct levels However the first big step toward a solution to this kind of problem is being able to detect data consistency issues and make sure that the data among the replication servers is synchronized Objectives The goal is to execute the mysqirplsync utility to detect data consistency issues on an active replication system making sure that master and slaves are synchronized Executing this task manually on an active system is difficult and sometimes tedious since changes may be continuously happening on all servers in an asynchronous way and the same data needs to be compared between servers Moreover it can introduce an undesirable and uncontrolled impact on the system performance Fortun
137. Do Find All Objects with That Name Prefix It is also possible you need to simply check to see if certain things exist For example suppose among your databases are parts or inventory data and you want to check to see if there are any functions or procedures that operate on a column named cost Moreover you want to see anything related that has cost as part of its name Whatever the case it would be a big time saver if you could search through all of the database objects and see a list of the objects whose name matches a prefix or pattern Fortunately the mysqimetagrep utility can get this done Objectives Find all objects whose name begins with a known prefix More specifically find any mention of the word cost Example Execution shell gt mysqlmetagrep server root root localhost body pattern cost 4 4 4 4 4 4 Connection Object Type Object Name Database Field Type Matches 4 4 4 4 4 4 mrocot CMO calino siz Si cl0iG um meEUN Gal aeter cose Greioes ROUTINE Badius Ecos nook ACilocalihost si ci0iGs ETARB EE supplies Cpeikoess COLUMN cost root localhost 3306 TABLE aE at thin sakila COLUMN replacement_cost 4 4 4
138. E 214 8 1 1 Fabric PrereQuiSiteS ssssersss issi miiesin pike EAE A R 214 8 1 2 Fabril CONCCPIS iarsa E A A 215 8 2 Installing and Configuring MySQL Fabric ccccceeeeeeeeeeeee ee ae ee eeeeeeeeeeeeaaaaeeeeeeeeeeeeaeaaeaaes 215 8 2 1 Downloading MySQL Fabric cccececeeeee cece cee eeeeeeee sees aa aa eeeeeeeeeeeaaaaeeteeeeeeeeeaeaaea 215 8 2 2 Installing MYSQL Fabri sisemine a Ania 216 8 2 3 Configuring MySQL Fabric ccccceeeeeeeeeeeeeeeeeeeeeaeaaeeneeeeeeeeeaeaaeaeeeeeeeeeeaeaaaaneeeeees 216 8 2 4 Starting and Stopping MySQL Fabric Nodes ccceeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaes 226 8 2 5 Old Configuration System ccccceceeeeeeeeeeeeeeeeeeeeaaeaeeeeeeeeeeeeaaaaeeeeeeeeeeeeaaaaeeeeeseeeees 227 8 3 QUICK Start spronu iyani nian aaia a aE AEAEE E AE E A EA ES a 229 8 3 1 Example Fabric and Replication ccceeeeccceeeeeeeaneeeeeeaaeeeeeeaaeeeeeeaaeeeeeeaaeeeeeeaaieees 229 MySQL Utilities 8 3 2 Example Fabric and Sharding c cccceeceeeceeeeeeeeeeeeeeeeaaeaeeeeeeeeeeeeaaaaeeeeeeeeeeeaaa 233 84 Ihemysa et abort WT seccaes nadon dees encads aniexamanuanaeae caenseenaceuanoaumenecemeta lars tamencuensvaaaeidaveneteans 238 8 4 1 Getting Help irkinin Ea NEEE E ENEA EA EEA A Aaaa 238 8 4 2 Group COMMAS aiina a a a aaa aaa ia aa AREE Ra 239 8 4 3 Manage Commands i seirraisya manida aa aai E EEEE NAEK 239 8 4 4 Server Commands creeis onen a EET E E E a EES 239 8 4 5 S
139. EMENT eien varchar 50 NOT NULL country_id gt smallint 5 unsigned NOT NULL last_update timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP PRIMARY KEY city_id KEY idx fk country_ia cotntry_id ENGINE InnoDB DEFAULT CHARSET utf8 done The following demonstrates reading multiple frm files in the default mode using a running server The frm files are located in different folders Notice the use of the database name option for each of the files The t1 file was given the database name temp1 since that is the folder in which it resides t2 was given db1 since that was specified in the path and t3 was not given a database name since we used the without providing a database name shell gt mysqlfrm server root pass localhost 3306 mysql data temp1 t1 frm mysql data temp2 db1 t2 frm port 3310 Starting the spawned server on port 3333 done Reading frm files Reading the tl frm file CREATE statement for mysql test std_data frm_files tl frm CREATE TABLE templ tl a int 11 DEFAULT NULL ENGINE MyISAM DEFAULT CHARSET latinl Reading the t2 frm file CREATE statement for mysql test std_data frm_files t2 frm CREATE TABLE dabl t2 a int 11 DEFAULT NULL 131 PERMISSIONS REQUIRED ENGINE MyISAM DEFAULT CHARSET latinl Reading the t3 frm file CREATE statement for mysql test std_data frm_files t3 frm CREATE TAB
140. ES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE RE DISCLAIMED IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE OR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL AMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS R SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION OWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT IABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OE Ol Abst WS OND AISLES SORT WARE T EVEN TE ADVTS ID OUT Wisi PODS ILISHILILIL IE 3g QE UCH DAMAGE Hey D GO el iz Oo Io tal strtod and dtoa The file Python dtoa c which supplies C functions dtoa and strtod for conversion of C doubles to and from strings is derived from the file of the same name by David M Gay currently available from http www netlib org fp The original file as retrieved on March 16 2009 contains the following copyright and licensing notice RK KKK HR KK KK RR A RI A IR A A II I I I kk The author of this software is David M Gay Caoc ieime C LOS 2000 OOM Ov amnUGeniemmle ohinelkequveicre Permission to use copy modify and distribute this software for any purpose without fee is hereby granted provided that this entire notice is included in all copies of any software which is or includes a copy or modification of this software and in all copies of the supporting do
141. FAIL Find row differences FAIL Transformation for changes for serverl Data differences found among rows DATE dbl tl SET b Test 123 WHERE a Das DATE dbl tl SET b Test 789 WHERE a 3 LETE FROM dbl t1 WHERE a 4 SERT INTO dbl t a b VALUES 5 New row db2 ig is 2 eal tag ae Transformation for reverse changes changes for server2 Data differences found among rows UPDATE db2 t1l SET b Test 789 WHERE a elo UPDATE db2 t1 SET b Test 123 WHERE a 3 DELETE FROM db2 t1 WHERE a 5 NSERT INTO db2 tl a b VALUES 4 New row dbl Database consistency check failed done PERMISSIONS REQUIRED The user must have the SELECT CREATE TEMPORARY TABLES and INSERT privileges for the databases being compared on both connections The user must also have SELECT privilege on the mysql database If the binary log is enabled and the disable binary logging option is used the user must also have the SUPER privilege 5 4 mysqldbcopy Copy Database Objects Between Servers 87 mysqldbcopy Copy Database Objects Between Servers This utility copies a database on a source server to a database on a destination server If the source and destination servers are different the database names can be the same or different If the source and destination servers are the same the database names must be different The utility ac
142. Group this shardMapping getGlobalGroupName catch FabricCommunicationException ex throw new RuntimeException ex Next create a method to create connections Find a server with mode READ_WRITE in the given server group and create a JDBC connection to it returns a link Connection to an arbitrary MySQL server throws SQLException if connection fails or a READ_WRITE server is not contained in the group oy private Connection getReadWriteConnectionFromServerGroup ServerGroup serverGroup throws SQLException iterate the list of servers in the given group until we find a r w server for Server s serverGroup getServers if ServerMode READ_WRITE equals s getMode create a connection to the server using vanilla JDBC String jdbcUrl String format jdbc mysql s s s s getHostname s getPort this database return DriverManager getConnection jdbcUrl this user this password throw an exception if we are unable to make the connection throw new SQLException Unable to find r w server for chosen shard mapping in group serverGroup getName To implement the interface the following methods must be implemented Connection getAnyConnection throws SQLException This method should obtain a connection to the global group We can implement it like this Get a connection that be used to access data or metadata not specific to any shard tenant The returned conne
143. INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION The slave users need the SUPER privilege The repl user used as the argument for the rp1 user option is either created automatically or if it exists it needs the REPLICATION SLAVE privilege 5 15 mysqlrplms Set Up and Start Replication Among a Slave and Multiple Masters This utility permits a user to start replication from multiple master servers also called sources to a single slave The user provides login information for the slave and each of the masters The utility reports conditions where the storage engines on the masters and the slave differ It also reports a warning if the InnoDB storage engine differs on the master and slave For InnoDB to be the same both servers must be running the same type of InnoDB built in or the InnoDB Plugin and InnoDB on both servers must have the same major and minor version numbers and enabled state By default the utility issues warnings for mismatches between the sets of storage engines the default storage engine and the InnoDB storage engine The vv option displays any discrepancies between the storage engines and InnoDB values A round robin scheduling is used to setup replication among the masters and slave The mysalrplms utility follows these assumptions e All servers have GTIDs enabled e There are no conflicts between transactions from different masters For example there are no upda
144. KKKKKKKKKKKK ls row KKKEKKEKKKKKEKKKKKKKAKKKK KKK server localhost 3307 config_file binary_log pinary e iogipas relay_log relay_log_pos version 5 6 15 datadir C MySQL instance_3307 basedir C MySQL mysql 5 6 15 winx64 plugin_dir C MySQL mysql 5 6 15 winx64 lib plugin general_log OFF general_log_file general_log_file_size log_error C MySQL instance_3307 clone err log_error_file_size 1569 bytes slow_query_log OFF slow_query_log_file slow_query_log_file_size 1 row done Discussion The SSL options ssl ca ssl cert and ssl key are available in the MySQL Utilities that requires a connection to a server or servers as is in the case of the master and slave options 54 Specialized Operations Note E An options file can be used to store the connection values and the MySQL Utilities can read the values stored in them as mentioned in the Section 2 2 Connecting to MySQL Servers documentation Permissions Required Required permissions include the ability to read the SSL certificate files and the path where they are located regardless of the form these SSL certificate paths are given to the MySQL Utilities in addition of the required permissions that each utility requires to accomplish its specific task Tips and Tricks In the configuration file different connection options can be stored and separated in groups The desired group used by the MySQL Utilities can be e
145. KKKKKKKKKKKKKKKKKKKKKK ie row kkxkxkxkxkxkxkkkkkkkkkkkkkkkkxkx k dept_no d002 dept_name dunno dept_name Finance T FONS Rows in empl departments not in emp2 departments KKKKKKKKKKKKKKKKKKKKKKKKK IES row KKKKKKKKKKKKKKKKKKKKKKKKK dept_no d008 dept_name Research rows Rows in emp2 departments not in empl departments KREKKKKKKKKKKKKKKKKKKKKKKK ie row KKKKKKKKKKKKKKKKKKKKKKKKK dept_no d100 dept_name stupid rows TABLE dept_manager pass pass Compare table checksum pass Database consistency check failed done Given two databases with the same table layout Data for each table contains mysql gt select from db1 tl 85 EXAMPLES ll Test 789 2 Test 456 3 Test 123 4 New row dbl i Test 123 2 Test 456 3 Test 739 5 New row db2 4 rows in set 0 00 sec To generate the SQL statements for data transformations to make db1 t1 the same as db2 t1 use the changes for server1 option We must also include the a option to ensure that the data consistency test is run The following command illustrates the options used and an excerpt from the results generated shell gt mysqldbcompare serverl root root localhost server2 root root localhost db1 db2 changes for serverl a difftype sql Defn Row Data Type Object Name DILET Count Check TABLE Ea pass pass S Compare table checksum FAIL Find row differences FAIL Transformation for changes for s
146. LAVE To show the full replication topology of a master running on the local host use the following command shell gt mysqlrplshow master root localhost 3311 recurse discover slaves login root master on localhost connected Finding slaves for master localhost 3311 Replication Topology Graph localhost 3311 MASTER T localhost Ssi SWAN 1 OCGA lho Stoo 2a SAVE MA STUER 1o llocalhost i3313 SLAVE To show the full replication topology of a master running on the local host prompting for the user name and password for slaves that do not have the same user name and password credentials as the master use the following command shell gt mysqlrplshow recurse prompt num retries 1 master root localhost 3331 discover slaves login root Server localhost 3331 is running on localhost master on localhost connected Finding slaves for master localhost 3331 Server localhost 3332 is running on localhost masterkonmiocalhost soo IVAICINDD Connection to localhost 3332 has failed Please enter the following information to connect to this server User name root Password master on localhost connected Finding slaves for master localhost 3332 Server localhost 3333 is running on localhost masterkopi Ieee laosiss oo VAUD 173 PERMISSIONS REQUIRED Connection to localhest 32233 has failed Please enter the following information to connect to this
147. LE t3 la int 11 DEFAULT NULL ENGINE MyISAM DEFAULT CHARSET latinl done The following demonstrates running the utility in diagnostic mode to read all of the frm files in a directory shell gt mysqlfrm diagnostic mysql data sakila WARNING Cannot generate character set or collation names without the server option CAUTION The diagnostic mode is a best effort parse of the frm file As such it may not identify all of th Reading frm file for mysql data sakila city frm The frm file is a TABLE CREATE TABLE Statement CREATE TABLE city city_id smallint 5 unsigned NOT NULL AUTO_INCREMENT city varchar 150 NOT NULL country_id smallint 5 unsigned NOT NULL last_update timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP PRIMARY KEY PRIMARY city_id KEY 20x ek counery_ 10 country ENGINE InnoDB done PERMISSIONS REQUIRED The permissions for using mysql frm will vary and depend entirely on how you use it If you use the utility to read frm files in a protected folder like the example above in either mode you must have the ability to run the spawned server with privileges that allow you to read the protected files For example you could use a user account that has root level privileges If you use the utility with a server connection the user you use to connect must have the ability to read system variables at a minimum including read access t
148. Long Data NoAudit Ping Prepare Processlist Query Quit Refresh Register Slave Reset stmt Set option Shutdown Sleep Statistics Table Dump Time file stats Display the audit log file statistics format FORMAT f FORMAT Output format to display the resulting data Supported format values GRID default TAB CSV VERTICAL and RAW help Display a help message and exit license Display license information and exit pattern lt PATTERNs e lt PATTERN gt Search pattern to retrieve all entries with at least one attribute value matching the specified pattern By default the standard SQL LIKE patterns are used for matching If the regexp option is set then REGEXP patterns must be specified for matching query type lt QUERY_TYPE gt Commaz separated list of SQL statements commands to search for and match Supported values CREATE ALTER DROP TRUNCATE RENAME GRANT REVOKE SELECT INSERT UPDATE DELETE COMMIT SHOW SET CALL PREPARE EXECUTE DEALLOCATE regexp basic regexp G Indicates that pattern matching will be performed using a regular expression REGEXP from the Python re module By default the simple standard SQL L KE patterns are used for matching This affects how the value specified by the pattern option is interpreted 74 NOTES e start date lt START_DATE gt Starting date time to retrieve log entries from the specified date time range If not specified or the val
149. MySQL Utilities Abstract This is the MySQL Utilities Reference Manual It documents both the GPL and commercial editions of the MySQL Utilities 1 5 release series through 1 5 6 If you have not yet installed MySQL Utilities please download your free copy from the download site MySQL Utilities is available for Windows OS X and Linux variants For notes detailing the changes in each release see the MySQL Utilities Release Notes For legal information see the Legal Notices For help with using MySQL please visit either the MySQL Forums or MySQL Mailing Lists where you can discuss your issues with other MySQL users For additional documentation on MySQL products including translations of the documentation into other languages and downloadable versions in variety of formats including HTML and PDF formats see the MySQL Documentation Library Document generated on 2015 12 11 revision 45859 Table of Contents PICTACC este reat ee a ROS A E an 8 A 8 os es eA ea ee hen ah sae Bra T Vil 1 How to Install MySQL Utilities 00 0 eee ee eet reer eee aad ee ee eeee ee aaaaeeeeeeeeeeaeaaea 1 dot Prerequisites ori aves ake ie ee a a eh ees 1 In2 SOurCe COdC nthe nee i ee eee a ee ee 1 1 3 Oracle Linux and Red Hat LINUX 6 00 0 0 eeee ee ee cece ee ee cee e ee te eter esse ae aaa teense eeeeaeaaeaneneeeeeeeeeaaa 2 1 4 Debian LINUX 2s ttn eee i ee eae ee ne ed ee ee 2 15 Mieto solt WIndOWS oie ais opina ieee inl he ah atin
150. NS Dayal 2008 PSF yes 2m6 Deo 2008 PSF yes Dee 2 2008 PSF yes De OA Do 2009 PSF yes Deas Deion 2009 PSF yes 2 6 4 Delco 2010 PSF yes Deal 2 6 2010 PSF yes IGXSVEMIOIEDS 8 1 GPL compatible doesn t mean that we re distributing Python under the GPL All Python licenses unlike the GPL let you distribute a modified version without making your changes open source The GPL compatible licenses make it possible to combine Python with other software that is released under the GPL the others don t 2 According to Richard Stallman 1 6 1 is not GPL compatible because its license has a choice of law clause According to CNRI however Stallman s lawyer has told CNRI s lawyer that 1 6 1 is not incompatible with the GPL Thanks to the many outside volunteers who have worked under Guido s direction to make these releases possible B TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 1 This LICENSE AGREEMENT is between the Python Software Foundation PSF and the Individual or Organization Licensee accessing and otherwise using this software Python in source or binary form and its associated documentation 2 Subject to the terms and conditions of this License Agreement PSF hereby grants Licensee a nonexclusive royalty free world wide license to reproduce analyze test perform and or display publicly prepare derivative works distribute and otherwise use Pyt
151. Num 0 O_ Error 4 rows done To run the same failover command above but specify a log file use the following command shell gt mysqlrpladmin slaves root localhost 3332 root localhost 3333 root localhost 3334 candidates root localhost 3333 root localhost 3334 log test_log txt failover Performing failover Candidate slave localhost 3333 will become the new master Preparing candidate for failover Creating replication user if it does not exist Stopping slaves Performing STOP on all slaves Switching slaves to new master Starting slaves Performing START on all slaves Checking slaves for errors Failover complete done After this command the log file will contain entries like the following 20 20 20 20 20 20 20 20 20 20 20 20 20 2 03 19 14 44 17 PM INFO Executing failover command 2 03 19 14 44 17 PM INFO Performing failover 2 03 19 14 44 17 PM INFO Candidate slave localhost 3333 will become the new master 2 03 19 14 44 17 PM INFO Preparing candidate for failover 2 03 19 14 44 19 PM INFO Creating replication user if it does not exist 2 03 19 14 44 19 PM INFO Stopping slaves 2 03 19 14 44 19 PM INFO Performing STOP on all slaves 2 03 19 14 44 19 PM INFO Switching slaves to new master 2 03 19 14 44 20 PM INFO Starting slaves 2 03 19 14 44 20 PM INFO Performing START on all slaves 2 03 19 14 44 20 PM INFO Checking slaves for errors
152. Oracle Support For information visit http www oracle com pls topic lookup ctx acc amp id info or visit http Avww oracle com pls topic lookup ctx acc amp id trs if you are hearing impaired This documentation is NOT distributed under a GPL license Use of this documentation is subject to the following terms You may create a printed copy of this documentation solely for your own personal use Conversion to other formats is allowed as long as the actual content is not altered or edited in any way You shall not publish or distribute this documentation in any form or on any media except if you distribute the documentation in a manner similar to how Oracle disseminates it that is electronically for download on a Web site with the software or on a CD ROM or similar medium provided however that the documentation is disseminated together with the software on the same medium Any other use such as any dissemination of printed copies or use of this documentation in whole or in part in another publication requires the prior written consent from an authorized representative of Oracle Oracle and or its affiliates reserve any and all rights to this documentation not expressly granted above viii Chapter 1 How to Install MySQL Utilities Table of Contents 1 1 PrerequisiteS 02 taccie nni cc ctezal cect tea li cee eedes oot aid ctegansietdeaneccnvenhs obeys E ahi cce sehen eae ete 1 12 SOURCE COU asec sccacusercscnsactguvannestesaase
153. QL server that it provides it uses the connection arguments other than the fabric argument except that the unix_socket connection argument is not supported To prepare a connection with Fabric do this fabric_config host fabric example com fonx mysql connector connect fabric fabric_config user webuser password webpass database employees If you prefer to pass a dictionary to connect do this config PSDEG E host fabric example com Ih user webuser password webpass database employees fonx mysql connector connect config The fabric dictionary argument permits these values 257 Requesting a Fabric Connection host The host to connect to required port The TCP IP port number to connect to on the specified host optional default 32274 username The user name of the account to use optional password The password of the account to use optional connect_attempts The number of connection attempts to make before giving up optional default 3 connect_delay The delay in seconds between attempts optional default 1 report_errors Whether to report errors to Fabric while accessing a MySQL instance optional default False ssl_ca The file containing the SSL certificate authority optional ssl_cert The file containing the SSL certificate file optional ssl_key The file containing the SSL key optional protocol The conn
154. RCHANTABILITY AND FITNESS IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE Licenses and Acknowledgements for Incorporated Software This section is an incomplete but growing list of licenses and acknowledgements for third party software incorporated in the Python distribution Mersenne Twister The _random module includes code based on a download from http www math keio ac jp matumoto MT2002 emt19937ar html The following are the verbatim comments from the original code A C program for MT19937 with initialization improved 2002 1 26 Coded by Takuji Nishimura and Makoto Matsumoto Before using initialize the state by using init_genrand seed or init_by_array init_key key_length Copyright C 1997 2002 Makoto Matsumoto and Takuji Nishimura All rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions gre Wises 1 Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following
155. SQL Fabric Section 8 8 Using Connector J with MySQL Fabric 8 2 2 Installing MySQL Fabric To install MySQL Fabric install MySQL Utilities 1 5 6 For more information see Chapter 1 How to Install MySQL Utilities 8 2 3 Configuring MySQL Fabric Configuring MySQL Fabric requires creating separate MySQL users to access the backing store and managed MySQL servers and editing the configuration file with the MySQL user details for all of these users Note E The configuration system changed in MySQL Fabric 1 5 5 1 6 2 For information about the previous configuration system see Section 8 2 5 Old Configuration System 8 2 3 1 Create the Associated MySQL Users Fabric uses four different types of users each with a different set of required privileges Note kK The backup and restore users were added in Fabric 1 5 5 1 6 2 Backing store user stores Fabric specific information and is only created on the fabric backing store MySQL server For additional information see Section 8 5 Backing Store Server user accesses the managed MySQL servers and is created on each managed MySQL server Backup user executes backup operations such as mysqldump and is created on each managed MySQL server Restore user executes restore operations that typically use the mysql client and is created on each managed MySQL server Privileges It is possible to use the same MySQL account for the server user backup user an
156. SQL Utilities 3 6 4 How Do Display Connections by the User root and Show the Result in CSV FORMAL eara EE N REE REA E E EEAS 59 4 Overview of MySQL UtiltISS rescission ananiona niaaa aaa Ea AE EA aaa aaa a EEK EEA 61 4 1 Database Operations isricnosrinisieia a a a A EE E E 61 4 2 General OPSratlOnS siroisarseinsi nanan kanian tiia aa aa a aaa a ENNE ANE A AEA AAEE 62 4 3 High Availability Operations 2 0 2 2 cceececeee cece ee ace cent cece tees ae aa eee eeeeeeeeaeaaeaaeeeeeeeeeeeaaaaeneeseeeees 63 4 4 Server Operations ci ciccecsseeveteasbactedasiineds site Ceadsa aaa AKEE Ka AA Ea AEEA EEE AAE 63 4 5 Specialized Operations eiaei aa ai aa Ea E E Ea 64 5 Manual Pages sssesessesrkionta ion E ARARA 65 5 1 mysqlauditadmin Allows users to perform maintenance action on the audit log 65 5 2 mysqlauditgrep Allows users to search the current or an archived audit log 71 5 3 mysqldbcompare Compare Two Databases and Identify Differences ccceseeeeeee es 78 5 4 mysqidbcopy Copy Database Objects Between Servers ccccsseeeeeeeseeeeeeaaeeeeeeaaeeees 87 5 5 mysqldbexport Export Object Definitions or Data from a Database ccceeeeeeeneeeees 94 5 6 mysqldbimport Import Object Definitions or Data into a Database ccccceeeeeeeeeees 102 5 7 mysqidiff Identify Differences Among Database Objects
157. SQL processes running on a host Example Execution shell gt mysqlserverinfo show servers server root root localhost format vertical The following MySQL servers are active on this host Process id 3007 Data path usr local mysql data Process id 8191 Data path Volumes Source source temp_13001 Process id 8196 Data path Volumes Source source temp_13002 Process id 8201 Data path Volumes Source source temp_13003 Process id 8207 Data path Volumes Source source temp_13004 Process id 8212 Data path Volumes Source source temp_13005 SOURCE Cin LOCEIVMOSES coc Conect edk 50 How Can Use a secure encrypted connection between Utilities and a MySQL Server KREKKKKKKKKKK KKK KKK KKKKKKEK alive row KRKKKKKKKKKKKK KKK KKKKKKEKK server localhost 3306 version Ton il S0 lkeg datadir usr local mysql data basedir usr local mysql 5 1 50 osx10 6 x86_64 plugin dir usr local mysql 5 1 50 o5x10 6 x86_64 1ib plugin config_file etc my cnf binary_llog my_log 000286 lonlingay _Ilew_ joss 23 7 relay_log None relay_log_pos None 1 row done Discussion The mysqlserverinfo utility is normally used to find information about a particular server We can see such results in the example above However the utility also has an option show servers that displays a list of all of the MySQL server process ids that are executing on the host This quick glance can help diagnose problems with
158. SV Display output in comma separated values format e tab Display output in tab separated format e vertical Display output in single column format like that of the G command for the mysq1 client command line tool Options mysqlprocgrep accepts the following command line options e help Display a help message and exit e license Display license information and exit e age lt time gt Select only processes that have been in the current state more than a given time The time value can be specified in two formats either using the hh mm ss format with hours and minutes optional or as a sequence of numbers with a suffix giving the period size The permitted suffixes are s second m minute h hour d day and w week For example 4h15m represents 4 hours and 15 minutes For both formats the specification can optionally be preceded by or where means older than the given time and means younger than the given time character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client 141 Options format lt format gt f lt format gt Specify the output display format Permitted format values are grid default csv tab and vertical kill connection Kill the connection for all matching processes like the KILL CONNECTION statement kill query Kill the query for all matching processes like the KILL QUERY state
159. Sies U VUDS E Og Entries Now imagine that the master crashed or is no longer reachable then after a predefined time interval by default 15 seconds we can observe that the failover process will start automatically Failover starting in auto mode Candidate slave server2 3312 will become the new master Checking slaves status before failover Preparing candidate for failover Creating replication user if it does not exist Stopping slaves Performing STOP on all slaves Switching slaves to new master Disconnecting new master as slave 38 Setup Automatic Failover Starting slaves Performing START on all slaves Checking slaves for errors Failover complete Failover console will restart in 5 seconds MySQL Replication Failover Utility Failover Mode auto Next Interval Fri Jul 26 10s25s17 2013 Master Information Binary Log File Position Binilog Do_DB Binlog_ignore_DB master bin 000001 151 GTID Executed Set None Replication Health Status 4 4 4 4 4 l host port role state gtid_mode health 4 4 4 4 server2 Saal MASTER UP ON OK server3 3313 SLAVE UP ON OKs server4 S34 SLAVE UP ON OK server5 3315 SLAVE UP ON OK 4 4 4 Q emie R r
160. TE USER fabric_store localhost IDENTIFIED BY secret GRANT ALTER CREATE CREATE VIEW DELETE DROP EVENT INDEX INSERT REFERENCES SELECT UPDATE ON mysql_fabric TO fabric atore localhost Note R The REFERENCES privilege is only required when working with MySQL 5 7 and above MySQL Fabric does not check for this privilege on earlier versions For additional information about using and setting up the backing store see Section 8 5 Backing Store The Server User MySQL Fabric uses the server user account to access all MySQL servers that it will manage In other words this user must be created on all managed MySQL servers The server account is defined under the servers using user for the user name and password as the password The Fabric server user account on the managed MySQL servers requires the following privileges in global scope 217 Configuring MySQL Fabric DELETE prune_shard PROCESS SESSI Ons E mkiml nll RELOAD RESET SLAVE REPLICATION CLIENT SHOW SLAVE STATUS REPLICATION SLAVE SHOW SLAVE HOSTS DELETE prune shard PROCESS iiet Seceleme co IsHiil RELOAD RESET SLAVE The Fabric server user account on the managed MySQL servers requires the following privileges on mysql_fabric ALTER alter some database objects CREATE create most database objects DELETE delete rows DROP drop most database objects INSERT insert rows SELECT select rows
161. TREE CREATE INDEX idx_full_name_dup ON test_db indexcheck_test name surname USING BTREE may be redundant or duplicate of CREATE INDEX idx_full_name ON test_db indexcheck_test gt name surname USING BTREE CREATE INDEX idx_name ON test_db indexcheck_test name USING BTREE may be redundant or duplicate of CREATE INDEX idx_full_name ON test_db indexcheck_test gt name surname USING BTREE CREATE INDEX idx_surname ON test_db indexcheck_test surname USING BTREE may be redundant or duplicate of CREATE INDEX idx_reverse_name ON test_db indexcheck_test surname name USING BTREE ALTER TABLE test_db indexcheck_test ADD PRIMARY KEY emp_id may be redundant or duplicate of CREATE INDEX idx_id_name ON test_db indexcheck_test emp_id name USING BTREE CREATE INDEX idx_id_hdate ON test_db indexcheck_test emp_id hire_date USING BTREE may be redundant or duplicate of CREATE INDEX idx_id_name ON test_db indexcheck_test emp_id name USING BTREE 29 Need to Find an Object on My Server But All Have is a Partial Name How Do Find All Objects with That Name Prefix CREATE INDEX idx id bday ON test _ db indexcheck_ test emp id birthday USING BTREE may be redundant or duplicate of CREATE INDEX dx_id_name ON test_db indexcheck_test gt emp_id name
162. The SQL transformation feature has these known limitations When tables with partition differences are encountered the utility generates the ALTER TABLE statement for all other changes but prints a warning and omits the partition differences If the transformation detects table options in the source table specified with the changes for option that are not changed or do not exist in the target table the utility generates the ALTER TABLE statement for all other changes but prints a warning and omits the table option differences Rename for events is not supported This is because mysqldiff compares objects by name In this case depending on the direction of the diff the event is identified as needing to be added or a DROP EVENT statement is generated Changes in the definer clause for events are not supported SQL extensions specific to MySQL Cluster are not supported For the difftype option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example di f ft ype d specifies the differ type An error occurs if a prefix matches more than one valid value 111 EXAMPLES The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login config
163. To avoid any mismatches the configuration file can be copied across by using these commands on host1 shell gt scp etc corosync corosync conf host2 etc corosync corosync conf shell gt scp etc corosync service d pcmk host2 etc corosync service d pcmk shell gt scp etc default corosync host2 etc default corosync Start Corosync on both hosts using shell gt etc init d corosync start Run tpcdump to check whether Corosync is working or not shell gt tcpdump i eth0 n port 5405 To start the Pacemaker on host1 execute the following command shell gt etc init d pacemaker start Run Pacemaker s cluster resource monitoring command on host1 to view the status of the cluster shell gt crm mon one shot V As we are configuring a cluster made up of just 2 hosts when one host fails or loses contact with the other there is no node majority quorum left and so by default the surviving node or both if they are still running but isolated from each other would be shut down by Pacemaker This is not the desired behavior as it does not offer High Availability and so that default should be overridden we will later add an extra behavior whereby each node will shut itself down if it cannot ping a 3 node that is external to the cluster thus preventing a split brain situation root host1 crm configure property no quorum policy ignore We turn STONITH Shoot The Other Node In The Head off as this solution will rely on each node shu
164. To check the data consistency on an active replication system but only between specific slaves shell gt mysqlrplsync slaves rp1 pass localhost 3311 rpl pass localhost 3312 Checking data consistency Using Slave localhost 3311 as base server for comparison Checking test_rplsync_db database 178 EXAMPLES Checking tO table data OK test_rplsync_do t0Q checksum for server localhost 3312 Checking tl table data OK test rplisynce db el checksum for server localhost 3312 Checking test_db database Checking C0 table data OK test_db t0 checksum for server localhost 3312 Checking tl vable data OK test_db tl checksum for server localhost 3312 done SUMMARY No data consistency issue found To check the data consistency of a specific database and table on an active replication system shell gt mysqlrplsync master user pass localhost 3310 slaves rp1 pass localhost 3311 rpl1 pass localhost 3312 test_rplsync_db test_db t1l GTID differences between Master and Slaves Slave localhost 3311 is 15 transactions behind Master Slave localhost 3312 is 12 transactions behind Master Checking data consistency Using Master localhost 3310 as base server for comparison Checking test_rplsync_db database Checking t0 table data OK test_rplsync_do tO checksum fo
165. UPDATE update rows Example statements to create the server user to be executed on each managed MySQL server CREATE USER fabric_server localhost IDENTIFIED BY secret GRANT DELETE PROCESS RELOAD REPLICATION CLIENT REPLICATION SLAVE SELECT SUPER TRIGGER ON TO fabric_server localhost GRANT ALTER CREATE DELETE DROP INSERT SELECT UPDATE ON mysql_fabric TO fabric_server localhost The Backup User If you want to use sharding or clone a MySQL server with the intention to add it to a High Availability HA group then you must define restore and backup users Like the server user these users must be created on all managed servers The backup account is defined under the servers using backup_user for the user name and backup_password as the password The backup account on the managed MySQL servers requires the following privileges in global scope if mysqldump s used EVENT show event information EXECUTE show routine information inside view REFERENCES foreign keys SELECT read data SHOW VIEW SHOW CREATE VIEW TRIGGER show trigger information Example statements to create the backup user to be executed on each managed MySQL server CREATE USHER fabrace backup Wocalhosit IDENTIFIED BY secret GRANT EVENT EXECUTE REFERENCES SELECT SHOW VIEW TRIGGER ON TOR FID TCE DACKO OC DNOS CAR 218 Configuring MySQL Fabric Note R The REFERENCES privil
166. What is diagnostic mode and why doesn t it produce the same output as the default mode The diagnostic mode does not use a spawned server to read the frm file Instead it attempts to read the contents of the file byte by byte and forms a best effort approximation of the CREATE statement Due to the many complexities of the server code the diagnostic mode does not currently process all features of a table Future revisions will improve the accuracy of the diagnostic mode 9 1 4 If the diagnostic mode is only a best effort compilation why use it The diagnostic mode is used to attempt to read corrupt or otherwise damaged frm files You would also use it if you had no access to a server installation on the local machine 9 1 5 Why does the default mode require a server The default mode uses a server to create a temporary working copy of the server instance It does not access the donor server in any way other than to execute the mysqld exe process 9 1 6 Can the frm reader read any frm file Although it can read most frm files there are known limits to which storage engines it can process correctly Currently tables with storage engines partition and performance_schema cannot be read However these frm files can be read by the diagnostic mode 9 1 7 My frm files are tucked away in a restricted folder How do get access to them to run the frm reader without copying or modifying file privileges You can use elevated privile
167. Y OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Fs EO eaa C a O P O a O ra O R E O E a O Er Ne e R E a E O r rE ME a A gt ME This product includes cryptographic software written by Eric Young eay cryptsoft com This product includes software written by Tim e REO ay h erypilEsOnencOm E say Original SSLeay License 296 Python License Supplement Windows Only ee ee de ot ee ee oe eee ae ol E ee oe al en a ne es es ee a ee ie es ye el a es eg pn pe ee aire ale ihe alee 5 ale Sy Copyright C 1995 1998 Eric Young eay cryptsoft com All rights reserved This package is an SSL implementation written by Eric Young eay cryptsoft com he implementation was written so as to conform with Netscapes SSL This library is free for commercial and non commercial use as long as the following conditions are aheared to The following conditions apply to all code found in this distribution be it the RC4 RSA lhash DES etc code not just the SSL code The SSL documentation included with this distribution is covered by the same copyrig
168. _log_current_session ON audit_log_exclude_accounts audit_log_file audit log audit_log_flush OFF audit_log_format OLD audit_log_include_accounts audit_log_policy ALL audit_log_rotate_on_size 0 audit_log_statement_policy ALL audit_log_strategy ASYNCHRONOUS 4 4 To perform a manual rotation of the audit log file use the following command shell gt mysqlauditadmin server root localhost 3310 rotate 68 EXAMPLES Executing ROTATE command To display the audit log file statistics run the following command shell gt mysqlauditadmin file stats audit log name SERVER data audit log 4 4 File Size Created 4 4 audit log S258 audit log 13486539046497235 47317 4 4 Wed Sep 26 11 07 43 2012 Wed Sep 26 11 05 04 2012 Wed Sep 26 11 07 43 2012 Wed Sep 26 11 05 04 2012 To change the audit log policy to log only query events and show the system variables before and after the execution of the policy command use the following command shell gt mysqlauditadmin show options server root localhost 3310 policy Audit value QUERIES Log Variables and Optio log_buffer_size _log_connection_policy _log_current_session log_exclude_accounts _log_file _log_flush log_format _log_include_a
169. ables we wish to shard employees e Salary Assume further that both tables are to be sharded by the employee ID that is part of their columns Where a row is placed is based on the value in the employee ID column Hence in a range based sharding scheme a shard is nothing but a range of employee ID values 233 Example Fabric and Sharding 8 3 2 2 Sharding Scenario In the sections that follow we take the example of a employee table that must be sharded across three MySQL groups The following procedure lists the sequence of commands to run to perform each step Unsharded Data Assume that we have an unsharded table named employees that contains Employee IDs on which we want to create the following shards Each of these ranges is placed in one shard 1 99999 shard 1 e 100000 199999 shard 2 200000 shard 3 In addition to creating the ranges themselves in a range based sharding scheme we also must define where this range of values should reside Starting Multiple MySQL Servers MySQL Servers must be started on the directories that were copied Assume that MySQL servers are started on the following hosts and ports localhost 3307 lecalhose 3308 lecalhose 3309 lecalhose 3310 Localhost lt 3311 Localhost 23312 localhost 3313 localhost 3314 Creating the Groups in Fabric A logical group in Fabric maps to a shard So as a first step to sharding we mu
170. abric import import import import import import java java java java java sgl sql sgl sgl sql Connection DriverManager PreparedStatement Statement ResultSet he GNU General Public License for more details hould have received a copy of the GNU General Public License along with this Software Foundation Inc 51 Franklin St Fifth com mysql fabric jdbc FabricMySQLConnection Demonstrate working with employee data in MySQL Fabric with Connector J and the JDBC APIs ey public class EmployeesJdbc public static void main String args SEEN SEEI SEEI Strin SEEI SEEN throws Exception g hostname System getProperty com mysql fabric testsuite hostname g port System getProperty com mysql fabric testsuite port g database System getProperty com mysql fabric testsuite database g user g baseUrl System getProperty com mysql fabric testsuite username g password System getProperty com mysql fabric testsuite password jdocimysql fabric hostname 2 Integer valueOt port U W Load the driver if running under Java 5 if com mysql jdbc Util isJdbc4 267 A Complete Example Working with Employee Data Class forName com mysql fabric jdbc FabricMySQLDriver 1 Create database and table for our demo Connection rawConnection DriverManager getConnection baseUrl mysql fabricServerGrou
171. age engine on the source server use the default storage engine option The new storage engine option takes precedence over default storage engine if both are given 103 OPTIONS If the new storag ngine or default storage engine option is given and the destination server does not support the specified storage engine a warning is issued and the server s default storage engine setting is used instead You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects in the operation For details see NOTES If you attempt to import databases on a server with GTIDs enabled GTID_MODE ON a warning will be generated if the import file did not include the GTID statements generated by mysqidbexport The utility will also generate a warning if you import databases on a server without GTIDs enabled and there are GTID statements present in the file Use the skip gtid option to ignore the GTID statements To make the most use of GTIDs and export import you should export all of the databases on the server with the a11 option This will generate an export file with all of the databases and the GTIDs executed to that point Importing this file on another server will ensure that server has all of the data as well as all of the GTIDs recorded correctly in its logs OPTIONS mysqldbimport accepts the following command line options
172. ailover A return code of 0 indicates failover should not take place A return code other than 0 indicates failover should take place This is checked at the start of each interval if a script is supplied The timeout option is not used in this case and the script is run once at the start of each interval 118 mysql failover Automatic replication health monitoring and failover The utility permits the user to log all actions taken during the commands The 1o0g option requires a valid path and file name of the file to use for logging operations The log is active only when this option is specified The option 1og age specifies the age in days that log entries are kept The default is seven 7 days Older entries are automatically deleted from the log file but only if the 10g option is specified The format of the log file includes the date and time of the event the level of the event informational INFO warning WARN error ERROR critical failure CRITICAL and the message reported by the utility The interface provides a number of options for displaying additional information You can choose to view the replication health report default or choose to view the list of GTIDs in use the UUIDs in use or view the log file contents if logging is enabled Each of these reports is described below e health Display the replication health of the topology This report is the default view for the interface By default th
173. all govern this License Agreement only as to issues arising under or with respect to Paragraphs 4 5 and 7 of this License Agreement Nothing in this 288 Python License License Agreement shall be deemed to create any relationship of agency partnership or joint venture between CNRI and Licensee This License Agreement does not grant permission to use CNRI trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee or any third party 8 By clicking on the ACCEPT button where indicated or by copying installing or otherwise using Python 1 6 1 Licensee agrees to be bound by the terms and conditions of this License Agreement ACCEPT CWI LICENSE AGREEMENT FOR PYTHON 0 9 0 THROUGH 1 2 Copyright c 1991 1995 Stichting Mathematisch Centrum Amsterdam The Netherlands All rights reserved Permission to use copy modify and distribute this software and its documentation for any purpose and without fee is hereby granted provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of Stichting Mathematisch Centrum or CWI not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF ME
174. alues option To run the utility as a daemon use the daemon option There are four commands that can be used in daemon option These include e start Starts the daemon The 1og option is required e stop Stops the daemon If you used the option pidfile the value must be the same when starting the daemon e restart Restarts the daemon If you used the option pidfile the value must be the same when starting the daemon nodetach Starts the daemon but it will not detach the process from the console The 10g option is required OPTIONS mysqlfailover accepts the following command line options e help Display a help message and exit e license Display license information and exit e candidates lt candidate slave connections gt Connection information for candidate slave servers Valid only with failover command List multiple slaves in comma separated list To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e
175. am In October of the same year the PythonLabs team moved to Digital Creations now Zope Corporation see http www zope com In 2001 the Python Software Foundation PSF see http www python org psf was formed a non profit organization created specifically to own Python related Intellectual Property Zope Corporation is a sponsoring member of the PSE All Python releases are Open Source see http www opensource org for the Open Source Definition Historically most but not all Python releases have also been GPL compatible the table below summarizes the various releases Release Derived Year Owner GPL from compatible 1 029 0 clarwn 1 2 TIHE 1 VSS CWI yes MESE NE ey ile IVYS 1 Seis CNRI yes LG S62 2000 CNRI no 2 amp 0 eG 2000 BeOpen com no W WG 200 CNRI yes 2 2i ees Ole Sites an 200 PSF no 2 6 0 Al 26 Oar Goi 200 PSE yes 2g he Al 2 se 0 iL 200 PSF yes 22 2e ea 200 PSF yes 2a l2 2e 2002 PSF yes Qo ed 2 oe 2002 PSF yes 285 Python License zj Dav 2 eth 2002 PS yes Dees Td 530 Din Ore 2002 PSF yes Zit gt BaD 2003 PSE yes 2a ZONDERD 2002 2003 PSF yes Da Sicd Dos 2002 2003 PSF yes Cee Din Syn 2002 2003 PSF yes Dm Shey Deora 2002 2003 PSF yes 234 Dex See 2004 PSF yes A OS 2 3 4 2005 PSF yes 2 4 ZnS 2004 PSF yes DAs 2 4 2005 PSF yes De Aver Dede 2005 PSF yes 2P AI De Ave 2006 PSF yes DEX 2A 2006 PSF yes Deora Bais 2007 PSF yes DeLorean Zaye 2008 PSE yes ZIO
176. ame csv mysqldbexport server root serverl 3306 format csv dbl export data By default the operation uses a consistent snapshot to read the source databases To change the locking mode use the locking option with a locking type value Use a value of no locks to turn off locking altogether or lock all to use only table locks The default value is snapshot Additionally the utility uses WRITE locks to lock the destination tables during the copy You can include replication statements for exporting data among a master and slave or between slaves The rp1 option permits you to select from the following replication statements to include in the export master Include the CHANGE MASTER statement to make the destination server a slave of the server specified in the server option This places the appropriate STOP and START slave statements in the export whereby the STOP SLAVE statement is placed at the start of the export and the CHANGE MASTER followed by the START SLAVE statements are placed after the export stream slave Include the CHANGE MASTER statement to make the destination server a slave connected to the same master as the server specified in the server option It only works if the current server is a slave This places the appropriate STOP and START slave statements in the export whereby the STOP SLAVE statement is placed at the start of the export and the CHANGE MASTER followed by the START SLAVE statements are pla
177. ame lookup the replication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqli rplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu net you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine Similarly if you use localhost to connect to the master the health report may not show all of the slaves It is best to use the actual hostname of the master when connecting or setting up replication If the user does not specify the rpl user and the user has specified the switchover or failover command the utility will check to see if the slaves are using master info repository TABLE If they are not the utility will stop with an error All the commands require either the s laves or discover slaves login option but both cannot be used at the same time In fact some commands only allow the use of the slaves option which is 160 EXAMPLES safer to specify the list slaves because discover slaves login might not provide an up to date list of available slaves The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will
178. ant to do this on a single database server without resorting to messy file copies and or stopping the server In this case we want to copy the world_innodb database in its entirety and rename the copy to word_innodb_ clone The utility of choice here is named mysqldbcopy and it is capable of copying databases from server to another or on the same server Lets have a look at the execution Example Execution shell gt mysqldbcopy source root root localhost destination root root localhost world_innodb world_innodb_clone Source on localhost connected Destination on localhost connected Copying database world_innodb renamed as world_innodb_clone Copying TABLE world_innodb City Copying TABLE world_innodb Country Copying TABLE world_innodb CountryLanguage Copying data for TABLE world_innodb City Copying data for TABLE world_innodb Country Copying data for TABLE world_innodb CountryLanguage 16 How Can Make a Copy of a Database and Change the Storage Engine doner shell gt mysql uroot proot e SHOW DATABASES 4 Database 4 information_schema employees mysql world_innodb world_innodb_clone Discussion Notice we specified the source of the database we wanted to copy as well as the destination In this case they are the same server You must specify it this way so that it is clear we are operating on the same server Noti
179. ants indicate actions to perform on processes that match the search conditions mysql utilities command proc KILL_QUERY Kill the process query mysql utilities command proc KILL_CONNECTION Kill the process connection 205 Parsers mysql utilities command proc PRINT_PROCESS Print the processes Classes class mysql utilities command proc ProcessGrep matches actions use_regexp False This class searches the INFORMATION_SCHEMA PROCESSLIST table for processes on MySQL servers and optionally kills them It can both be used to actually perform the search or kill operation or to generate the SQL statement for doing the job To kill all queries with user mats the following code can be used gt gt gt from mysql utilities command proc import gt gt gt grep ProcessGrep matches USER mats actions KILL_QUERY gt gt gt grep execute root server l example com root server 2 example com Parameters matches List of var pat pairs Sequence of field comparison conditions In each condition var is one of the constants listed earlier that specify PROCESSLIST table fields and patis a pattern For a process to match all field conditions must match sql only_body False Return the SQL code for executing the search and optionally the kill If only_body is True only the body of the function is shown This is useful if the SQL code is to be used with other utilities that
180. any character except a x or y a Match a sequence of zero or more a 137 OPTIONS a Match a sequence of one or more a a Match zero or one a ab cd Match ab or cd a 5 Match five instances of a a 2 5 Match from two to five instances of a abc Match one or more repetitions of abc OPTIONS mysqimetagrep accepts the following command line options help Display a help message and exit license Display license information and exit body b Search the body of stored programs procedures functions triggers and events The default is to match only the name character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client database lt pattern gt Look only in databases matching this pattern format lt format gt f lt format gt Specify the output display format Permitted format values are grid default csv tab and vertical object types lt types gt search objects lt types gt Search only the object types named in types which is a comma separated list of one or more of the values database trigger user routine column table partition event and view 138 OPTIONS The default is to search in objects of all types pattern lt pattern gt e lt pattern gt The pattern to use when matching This is required when the first non option argument looks like a connection specifi
181. aracters Use this mode when the default mode cannot read the file or if a MySQL server is not installed on the host To read frm files list each file as a separate argument for the utility as shown in the following examples You will need to specify the path for each frm file you want to read or supply a path to a directory and all of the frm files in that directory will be read 128 OPTIONS You can specify the database name to be used in the resulting CREATE statement by prepending the frm file with the name of the database followed by a colon For example oltp t1 frm will use oltp for the database name in the CREATE statement The optional database name can also be used with paths For example home me oltp t1 frm will use oltp as the database name If you leave off the optional database name and include a path the last folder will be the database name For example home me aata1 t1 frm will use data1 as the database name If you do not want to use the last folder as the database name simply specify the colon like this home me data1 t1 frm In this case the database will be omitted from the CREATE statement OPTIONS e help show the program s help page e license Display license information and exit e basedir lt basedir gt The base directory for the server installed Use this or server for the default mode e diagnostic Turn on diagnostic mode to read frm files byte by byte and generate best ef
182. as secondary Setting a different server as primary can be done through the same command which demotes the current primary and elects a new one If the current primary has failed the same command which can be triggered either manually or automatically can be used to elect a new one Note R A server marked as faulty cannot be promoted to a secondary or primary status in one step The faulty server status must first be changed to the spare status 230 Example Fabric and Replication For example use mysqlfabric server set_status lt server address gt spare Activating or Deactivating a Failure Detector If the primary fails you may want to automatically promote one of the secondaries as primary and redirect the remaining secondaries to the new primary To do this execute the following command shell gt mysqlfabric group activate my group If the failure detector discovers that a primary has crashed it marks it as faulty and triggers a failover routine This is not done automatically because there may be users who prefer to use an external failure detector or want to do things manually To deactivate the failure detector execute the following command shell gt mysqlfabric group deactivate my group Executing Updates and Queries Executing queries with a Fabric aware connector is easy The following example shows a section of code designed to add employees and search for them Notice that we simply import the fabric pa
183. ase identifier specified as an argument contains special characters or is a reserved word then it must be appropriately quoted with backticks In turn names quoted with backticks must also be quoted with single or double quotes depending on the operating system i e in Windows or in non Windows systems in order for the utilities to read backtick quoted identifiers as a single argument For example to copy a database with the name weird db name with other weird db name the database pair must be specified using the following syntax in non Windows weird db name other weird db name Keep in mind that you can only take advantage of multiprocessing if your system has multiple CPUs available for concurrent execution Also note that multiprocessing is applied at a different level according to the operating system where the mysqldbcopy utility is executed due to python limitations In particular itis applied at the database level for Windows i e different databases are concurrently copied and at the table level for Non Windows POSIX systems i e different tables within the same database are concurrently copied EXAMPLES The following example demonstrates how to use the utility to copy a database named util_test toa new database named util_test_copy on the same server shell gt mysqldbcopy source root pass localhost 3310 test123 mysql sock destination root pass localhost 3310 test123 mysql
184. atabase and tables We show the resulting tables in the new fabric database below shell gt mysqlfabric manage setup param storage user fabric param storage password secret INFO 1379444563 457977 MainThread Initializing persister user fabric server localhost 3306 database fabric shell gt mysqlshow ufabric psecret fabric 4 Tables 4 checkpoints error_log group_replication groups permissions role_permissions roles servers shard_maps shard_ranges shard_tables shards user_roles users Note R The tables described here are subject to change in future versions of Fabric 8 5 1 Backing Store Tables The checkpoints table stores information on procedures executions and is used to safely resume executing a procedure after a crash and recovery Table 8 3 checkpoints Column Type Comment proc_uuid varchar 40 The procedure s unique identification 241 Backing Store Tables Column Comment lockable_objects Set of objects locked by the procedure job_uuid varchar 60 The job s unique identification sequence int 11 The job s sequence in the execution action_fqn text Reference to the fully qualified name of the function to be executed on behalf of the job param_args blob Positional arguments to the job s function param_kwargs blob Keyword arguments to the job s function started double 16
185. ate code like the following Set connection values dest_values CS haan EODD passwd secret POSE 3 IloyeeliVavoishe ll 7 Mexorsie ge 3 3108 5 unix s cket None The connection specification is a string the form lt user gt lt passwd gt lt host gt lt port gt lt socket gt where the passwd port and socket are optional This string is parsed using the options parse_connection function You can also specify an existing instance of the Server class In this case the new class will copy the connection information 10 Chapter 3 MySQL Utilities Administrative Tasks Table of Contents 8 1 Binary Log Operations i is ease anka aieea ins a aa aaa aA a EA r Aa a raa aA kea 11 3 1 1 How To Change The Location For Binary Log Files cccceeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaeaaeeees 12 3 1 2 How do you purge obsolete binary log files safely cccececeeeeeeeaeeeeeeeeeeeeeeeaeaaeeeeeeeeeees 13 3 2 Database Operations 220 20 cece cece ce cece cece ee ae ete a e ee aaa a area ee aed ate ee Aaaa aaah E EAA na SA TAEA NASRA 15 3 2 1 How Do Provision a Slave srsti n aaa aan re aiaa a aeaa hiasan 15 3 2 2 How Do Make a Copy of a Database on the Same Server ccccseeeeeeeeeeeeeeseeeeeeeeeeees 16 3 2 3 How Can Make a Copy of a Database and Change the Storage Engine a se 17 3 2 4 How Can Tell If a Table on Server X has the same Structure as the Same Table on Ea e A E PAI
186. ately the mysqirplsync utility allows us to perform this task in an easy and optimized way with a controlled impact on the running system limiting the execution time of all operations Let s assume that a replication topology with one master server1 3310 and two slaves server2 3311 server3 331 2 was previously setup and it is running without errors 44 How To Check If Data Is Correctly Replicated Example Execution Start the mysqlrplsync utility specifying the servers you want to check shell gt mysqlrplsync master user pass localhost 3310 slaves rpl1 pass localhost 3311 rpl pass localhost 3312 GTID differences between Master and Slaves Slave localhost 3311 is 15 transactions behind Master Slave localhost 3312 is 12 transactions behind Master Checking data consistency Using Master localhost 3310 as base server for comparison Checking test_rplsync_db database Checking G0 table data OK test_rplsync_db t0 checksum for server localhost 3311 OK test_rplsync_db t0 checksum for server localhost 3312 Checking Gl table data OK test_rplsync_db tl checksum for server localhost 3311 OK test_rplsync_db tl checksum for server localhost 3312 Checking test_db database Checking t0 table data OK test_db t0 checksum for server localhost 3311 OK test_db t0 checksum for server localhost 3312 Ch
187. behind the master The utility runs each test in turn unless there is a fatal error preventing further testing such as a loss of connection to the servers Each test can complete with one of the following states pass the prerequisites are met fail the prerequisites were met but one or more errors occurred or there are exceptions to consider or warn the test found some unusual settings that should be examined further but may not be in error Use the verbose option to see additional information such as server IDs lower_case_table_nam settings and the contents of the master information file on the slave To see the values from the SHOW SLAVE STATUS statement use the show slave status option OPTIONS mysqlrplicheck accepts the following command line options e help Display a help message and exit e license Display license information and exit e master lt source gt Connection information for the master server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure 165 OPTIONS e Use login paths from your mylogin cnf file encrypted not visible Exam
188. ble Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt suppress Suppress warning messages ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection ssl 166 NOTES Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required e verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit e width lt number gt Change the display width of the test report The default is 75 characters NOTES The login user must have the appropriate permissions to execute SHOW SLAVE STATUS SHOW MASTER STATUS and SHOW VARIABLES on the appropriate servers Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation
189. calhost connected Checking for binary logging on master Setting up replication pe One The following command starts replication from the beginning of a specific master binary log file n hell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl master log file my_log 000003 Master on localhost connected slave on localhost connected Checking for binary logging on master Setting up replication me done The following command starts replication from specific master binary log coordinates specific log file and position shell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl master log file my_log 000001 master log pos 96 148 RECOMMENDATIONS master on localhost connected slave on localhost connected Checking for binary logging on master Setting up replication done RECOMMENDATIONS You should set read_only 1 in the my cnf file for the slave to ensure that no accidental data changes such as INSERT DELETE UPDATE and so forth are permitted on the slave other than those produced by events read from the master Use the pedantic and vv options for setting up replication on production servers to avoid possible problems with differing storage engines PERMISSIONS REQUIRED The users on the master need the following privileges SELECT and
190. calhost 3306 export both all gt data sql shell gt mysqldbimport server root root localhost 3307 import both data sql Source on localhost connected Importing definitions and data from data sql done Discussion As you can see this is a multiple step process We saw examples of using the mysqliserverinfo mysqlserverclone mysqlidbexport and mysqldbimport utilities Notice in the example we used port 3307 for the clone which is reflected in the mysqidbimport utility server option Permissions Required The user must have permission to read all databases Since we are using the root account for these examples and you typically would permissions are not generally a problem You also need permissions to create the new data directory and write data to it Tips and Tricks If you want to copy all of the users and their permissions check out the mysqluserclone utility 3 5 2 How Can I Find What MySQL Servers Are Running One of the challenges for a database administrator or database developer when working with a development server that has multiple instances of MySQL running is knowing exactly how many are running In some cases this may have come about by accident but mostly having multiple instances of MySQL running is intentional Whichever the case it would be nice to be able to use a single command to find all of the MySQL processes Objectives Use the mysqlserverinfo utility to locate all of the My
191. case the range value 1 9999 was used to include all MySQL error codes and display all unsuccessful commands To retrieve only successful command no errors simply use the value 0 for the status option Permissions Required The user must have permissions to read the audit log on disk Tips and Tricks The value specified for the query type option are case insensitive therefore you can mix lower and upper case to specify the list of query types For example insert Update will produce the same result as using INSERT UPDATE Of course the use of non supported values will raise an appropriate error Many other options and search criteria are provided by the mysqlauditgrep utility check them in order to use the more appropriate one to meet your needs Note that the utility provides the pattern option to search entries in the audit log file using regular expressions like common grep tools By default this option will use standard SQL pattern matching used by LIKE comparison operator unless the regexp option is used to allow more powerful standard regular expressions POSIX extended 3 6 4 How Do Display Connections by the User root and Show the Result in CSV Format The audit log plugin can be used to record information about different type of events which one might need to monitor or keep a record in a different format For example a security record with the list of all logins performed to the database serve might nee
192. cation rather than a pattern If the pattern option is given the first non option argument is treated as a connection specifier not as a pattern regexp basic regexp G Perform pattern matches using the REGEXP operator The default is to use LIKE for matching This affects the database and pattern options server lt source gt Connection information for a server Use this option multiple times to search multiple servers To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt sql print sql p Print rather than executing the SQL code that would be executed to find all matching objects This can be useful to save the statement for later execution or to
193. ccounts mlogmpolmiay Showing options before command Showing options after command ns NONE ON OFF OLD ALL 1048576 audit Audit Log Variables and Options 4 4 Variable_name Value 4 audit_log_buffer_size 1048576 audit_log_connection_policy ALL audit_log_current_session ON audit_log_exclude_accounts audit_log_file audit log mwee los ae bee OFF audit_log_format OLD audit_log_include_accounts audit_log_policy ALL audit_log_rotate_on_size 0 audit_log_statement_policy ALL audit_log_strategy ASYNCHRONOUS 4 4 Executing POLICY command log 69 EXAMPLES audit log rotate on size audit_log_statement_policy ALL audit_log_strategy ASYNCHRONOUS 4 4 To change the audit log automatic file rotation size audit_log_rotate_on_size to 32535 and show the system variables before and after the execution of the rotate_on_size command use the following command Notice that the value set is actually 28672 because the specified rotate_on_size value is truncated to a multiple of 4096 shell gt mysqlauditadmin show options server root localhost 3310 rotate_on_size value 32535 Showing options before command Audit Log Variables and Options 4
194. ce how we specified the new name We used the lt old_name gt lt new_name gt syntax You can do this for as many databases as you want to copy That s right you can copy multiple databases with a single command renaming each along the way To copy a database without renaming it if the destination is a different server you can omit the lt new_name gt portion Permissions Required The user must have SELECT privileges for the database s on the source server and have CREATE INSERT UPDATE on the destination server Tips and Tricks Risks You can copy all of the databases on a source server to the destination by using the a11 option although this option does not permit rename actions To rename you must specify the databases one at a time You can specify certain objects to exclude skip in the copy Use the skip option to omit the type of objects For example you may want to exclude copying of triggers procedures and functions In this case use the option skip TRIGGERS PROCEDURES FUNCTIONS The values are case insensitive and uppercased for emphasis The copy is replication and GTID aware and will take actions to preserve the binary log events during the copy You can set the locking type with the Locking option Possible values include no locks do not use any table locks lock all use table locks but no transaction and no consistent read snapshot default consistent read using a single transaction
195. ced after the export stream both 95 OPTIONS Include both the master and slave information for CHANGE MASTER statements for either spawning a new slave with the current server s master or using the current server as the master All statements generated are labeled and commented to enable the user to choose which to include when imported To include the replication user in the CHANGE MASTER statement use the rpl user option to specify the user and password If this option is omitted the utility attempts to identify the replication user In the event that there are multiple candidates or the user requires a password these statements are placed inside comments for the CHANGE MASTER statement You can also use the comment rp1 option to place the replication statements inside comments for later examination If you specify the rp1 file option the utility writes the replication statements to the file specified instead of including them in the export stream If you attempt to export databases on a server with GTIDs enabled GTID_ MODE ON a warning will be generated if the export does not include all databases This is because the GTID statements generated include the GTIDs for all databases and not only those databases in the export The utility will also generate a warning if you export databases on a GTID enabled server but use the skip gtid option To make the most use of GTIDs and export import you should e
196. cepts one or more database pairs on the command line To name a database pair use db_name new_db_name syntax to specify the source and destination names explicitly If the source and destination database names are the same db_name can be used as shorthand for db_name db_name By default the operation copies all objects tables views triggers events procedures functions and database level grants and data to the destination server There are options to turn off copying any or all of the objects as well as not copying the data To exclude specific objects by name use the exclude option with a name in db obj format or you can supply a search pattern For example exclude db1 trig1 excludes the single trigger and exclude trig_ excludes all objects from all databases having a name that begins with trig and has a following character By default the utility creates each table on the destination server using the same storage engine as the original table To override this and specify the storage engine to use for all tables created on the destination server use the new storage engine option If the destination server supports the new engine all tables use that engine To specify the storage engine to use for tables for which the destination server does not support the original storage engine on the source server use the default storage engine option The new storage engine option takes precedence over default storage engin
197. ces running MySQL 5 6 10 or higher One server is required for the backing store and at least one server must be added to a group To use the replication features of MySQL Fabric a replication topology of a master and at least one slave is required To use the sharding features of MySQL Fabric you should have the number of servers corresponding to the depth of the shards the number of segments documented here and the old system is documented under Section 8 2 5 Old Note R The configuration system changed in Fabric 1 5 5 The current system is Configuration System For instructions on how to download and install MySQL server see the online MySQL reference manual Installing and Upgrading MySQL 8 2 1 Downloading MySQL Fabric Download a version of MySQL Fabric from the MySQL Developer Zone website http dev mysql com downloads utilities Packaged downloads are available for a variety of servers Download the package that matches your platform and extract the files 215 Installing MySQL Fabric Note R MySQL Fabric will become its own product and own download as of the 1 6 2 release The above website also provides the MySQL Fabric aware connectors Download the connector you want to use with a MySQL Fabric application For more information about how to install and get started using MySQL Fabric in your applications see the appropriate connector specific section Section 8 7 Using Connector Python with My
198. cess to specific methods FabricMySQLConnection connection FabricMySQLConnection rawConnection example data used to create employee records Integer ids new Integer 1 2 10001 10002 String firstNames new String John Jane Andy Alice String lastNames new String Doe Doe Wiley Wein insert employee data PreparedStatement ps connection prepareStatement INSERT INTO employees employees VALUES je re Aane ak Sp ah eas eBay A choose the shard that handles the data we interested in connection setShardKey ids i toString perform insert in standard fashion See lite il aleler fah yp ps setString 2 firstNames i ps setString 3 lastNames i ps executeUpdate 3 Query the data from employees 268 A Complete Example Working with Employee Data System out printiln Querying employees System out format 37s 30s 30s n emp_no first_name last_name SHAS Sul TOWNE _joue aLiove thin US Sa aS GPSS SSS SSS SS SSS SS SS SSS SS SS SS SSS SSS pe SS SS SS SS SS SS SSS SS SS SSS SS SSS Wys ps connection prepareStatement select emp_no first_name last_name from employees where emp_no tor Gite a Oe a lt 4b FEA we need to specify the shard key before accessing the data connection setShardKey ids i toString posset t P RAS ResultSet rs ps executeQuery rs next HySicom ole cori SV
199. ckage demo fabric import java sql Connection import java sql DriverManager import java sql PreparedStatement import java sql ResultSet import java sql Statement import com mysql fabric jdbc FabricMySQLConnection import com mysql fabric jdbc FabricMySOQLDataSource Demonstrate working with employee data in MySQL Fabric with Connector J and the JDBC APIs via a DataSou a 269 A Complete Example Working with Employee Data public class EmployeesDataSource public static void main String args throws Exception String hostname System getProperty com mysql fabric testsuite hostname String port System getProperty com mysql fabric testsuite port String database System getProperty com mysql fabric testsuite database credentials to authenticate with the SQL nodes String user System getProperty com mysql fabric testsuite username String password System getProperty com mysql fabric testsuite password credentials to authenticate to the Fabric node String fabricUsername System getProperty com mysql fabric testsuite fabricUsername String fabricPassword System getProperty com mysql fabric testsuite fabricPassword setup the Fabric datasource to create connections FabricMySQLDataSource ds new FabricMySQLDataSource ds setServerName hostname ds setPort Integer valueOf port ds setDatabaseName database ds setFabricUsername fabricUsername
200. ckage from the Connector Python library and provide the Fabric connection parameters such as the location of the Fabric node as specified in the protocol xmlrpc configuration file section and user credentials import mysql connector from mysql connector import fabric def add_employee conn emp_no first_name last_name conn set_property group my_group mode fabric MODE_READWRITE cur scConn CUr Sor cur execute USE employees cur execute INSERT INTO employees VALUES s S s emp_no first_name last_name We need to keep track of what we have executed in order to at least read our own updates from a slave cur execute SELECT global gtid_executed ore Tey ain Cie Y print Transactions executed on the master row 0 return row 0 def find_employee conn emp_no gtid_executed conn set_property group my_group mode fabric MODE_READONLY Cur conn cursor Guarantee that a slave has applied our own updates before reading anything cur execute SELECT WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS s 0 4 gtid_executed EO Wey aie ible Y joao Wiee co eine omaat mony Sereneco lenas cur execute USE employees cur execute SELECT first_name last_name FROM employees WHERE emp_no s emp_no EOS Toy aiel ible gy print Retrieved row Address of the Fabric not the host we are going to connect to 231 Example Fabric and Replication conn mysql connect
201. configuration file location Platform Package Location Microsoft Windows mysql utilities 1 5 6 win32 msi UTILITIES_INSTALLDIR etc mysq l fabric cfg Ubuntu Linux 14 04 mysql etc mysql fabric cfg utilities 1 5 6 1ubuntu14 04_all deb 227 Old Configuration System Red Hat Enterprise Linux 6 Oracle Linux 6 mysql utilities 1 5 6 1 el6 noarch rpm Platform Package Location Debian Linux 6 0 mysql etc mysql fabric cfg utilities 1 5 6 1debian6 0_all deb etc mysql fabric cfg The following shows the content of the configuration file and the modifications necessary In the logdir var log storage address localhost 3306 user fabric password secret database fabric DEFAULT prefix usr local sysconfdir usr local etc auth_plugin mysql_native_password connection_timeout 6 connection_attempts 6 connection_delay 1 servers user fabric password backup_user fabric backup_password restore_user fabric restore_password unreachable_timeout 5 protocol xmlrpc address localhost 32274 threads 5 user admin password disable_authentication no realm MySQL Fabric ssl_ca ssl_cert ssl_key protocol mysql address localhost 32275 user admin password disable_authentication no ssl_ca ssl_cert ssl_key executor executors 5 logging level INFO sharding url
202. copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of Lance Ellinghouse not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission LANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO 293 Python License THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS IN NO EVENT SHALL LANCE ELLINGHOUSE CENTRUM BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE Modified by Jack Jansen CWI July 1995 Use binascii module to do the actual line by line conversion between ascii and binary This results in a 1000 fold speedup The C version is still 5 times faster though Arguments more compliant with Python standard XML Remote Procedure Calls The xmlrpclib module contains the following notice The XML RPC client interface is Copyright c 1999 2002 by Secret Labs AB Copyright c 1999 2002 by Fredrik Lundh By obtaining using and or copying this software and or its associated documentation you agree that you have read understood and will comply with the following terms and conditions Permission to use copy modify and distribute th
203. count name password optional and host When the utility finishes you will have two new user accounts that have the same privileges as the original user joe localhost Permissions Required On the source server the user must have the SELECT privilege for the mysql database On the destination server the user must have the global CREATE USER privilege or the INSERT privilege for the mysql database as well as the GRANT OPTION privilege and the privileges that the original user has you grant a privilege you do not have yourself Tips and Tricks You can use dest ination option to specify a different server to copy a user account to another server Use the dump option with only the source option to see all user accounts Use the include global privileges option to include GRANT statements that the user host combination matches This is useful for copying user accounts from one server to another where there are global privileges in effect 3 3 4 What Options Are Used With Each Utility There are many utilities and it is not always easy to remember all of the options and parameters associated with each Sometimes we need to run several utilities using nearly the same options For example you may want to run several utilities logging into a particular server Rather than retype the connection information each time you would like to save the option value some way and reuse it 25 What Options Are Used With Each Utility
204. crypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt rpl timeout lt rpl_timeout_in_seconds gt Maximum timeout in seconds to wait for synchronization More precisely the time to wait for the replication process on a slave to reach a sync point GTID set Default 300 seconds Slaves lt slaves_connections gt Connection information for slave servers List multiple slaves in comma separated list To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line
205. ct localhost success Replication Topology Health F SE SE OSE SE OSE H KEKEKKKKKKKKKKKKKKKKKKKEKEK all DOSLE POCE role state gtid_mode health version master_log_file row KKEKEKKKKKKKEKKEKKEKKKAKKKK KK KK Ieee LIMOsie 333 MASTER UE ON OK 5 6 5 m8 debug log mysql bin 000001 master_log_ pos 571 I0 Thread SQL_Thread Secs_Behind Remaining_Delay TO Error Num TO Eror KKEKKKKKKKK KKK KKKKKKKKKKKK lie row KREKEKKKKKKKKKKKKKKKKKKKKEKEK ose lheeellinesic Dorce SIS role SLAVE state UIE gtid_mode ON health OK version 5 6 5 m8 debug log master_log_file mysql bin 000001 master_log_ pos 571 I0 Thread Yes SQL_Thread Yes Secs_Behind 0 Remaining_Delay No IO Error Num 0 TO Hirst Gees KKEKKKKKKKKKKKKKKKKKKKKKKK 35 row KREKEKKKKKKKKKKKKKKKKKKKKEEK nose lheeellinesic PORER SSS role SLAVE state UPF gtid_mode ON health OK version 5 6 5 m8 debug log master_log_file mysql bin 000001 master_log_ pos 571 I0_Thread Yes SQL_Thread Yes Secs_Behind 0 Remaining_Delay No IO_Error_Num 0 TO EFroOr kkkkkkxk xkxkxk k xk kx xkxk kx xkxk kxk kxk kxk kxk kxk kxk k ak row KEKEKKKKKKKKKKKKKKKKKKKKEEKEK 162 EXAMPLES boste Ilocellingsic joes F554 role SLAVE state UP gtid_mode ON health OK version 5 6 5 m8 debug log master_log_file mysql bin 000001 master_log_pos 571 IQ _Thread Yes SQL_Thread Yes Secs_Behind 0 Remaining_Delay No IO_Error_
206. ction for Connector Python to download the appropriate repository 1 2 Source Code The source code repository for MySQL Utilities includes all of the utility code as well as the MySQL Utilities library and manual pages It is available as an architecture independent distribution in either Zip archive format zip file or compressed tar archive format tar gz file You can use this repository to install on any platform that has Python 2 6 installed For example you can use the tar gz version of the repository to install MySQL Utilities on OS X or Ubuntu After you download and unpack the repository distribution open a terminal window and navigate to the directory containing the file Then unpack the file and install MySQL Utilities using the setup py script as shown below shell gt unzip mysql utilities 1 5 6 zip shell gt cd mysql utilities 1 5 6 shell gt python setup py build shell gt sudo python setup py install Note R Using this repository requires that you have Connector Python installed or install it separately For additional information see Section 1 1 Prerequisites Oracle Linux and Red Hat Linux 6 1 3 Oracle Linux and Red Hat Linux 6 This repository is available as an architecture independent RPM package rpm file After you download the package install it using the following command or similar depending on your platform configuration shell gt sudo rpm i mysql utilities 1 5 6 el6 noarch
207. ction is a READ_WRITE connection to the global group of the shard mapping for the database and table association with this connection provider 273 Using Hibernate with MySQL Fabric 2 public Connection getAnyConnection throws SQLException return getReadWriteConnectionFromServerGroup this globalGroup e Connection getConnection String tenantIdentifier throws SQLException This method must use the tenant Identifier to determine which server to access We can look up the ServerGroup from the ShardMapping like this Get a connection to access data association with the provided tenantIdentifier or shard key in Fabric speak The returned connection is a READ _WRITE connection 2 public Connection getConnection String tenantIdentifier throws SQLException String serverGroupName this shardMapping getGroupNameForKey tenantIdentifier try ServerGroup serverGroup this fabricConnection getServerGroup serverGroupName return getReadWriteConnectionFromServerGroup serverGroup catch FabricCommunicationException ex throw new RuntimeException ex Finally our trivial implementation to release connections Release a non shard specific connection a public void releaseAnyConnection Connection connection throws SQLException connection close Release a connection specific to tenantIdentifier if public void releaseConnection String tenantIdentifier Connection connection
208. cumentation for such software THIS SOFTWARE IS BEING PROVIDED AS IS WITHOUT ANY EXPRESS OR IMPLIED WARRANTY IN PARTICULAR NEITHER THE AUTHOR NOR LUCENT MAKES ANY REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE FERRARA e E A RA AR I k k k A I AA IA k k k k I kk k k S 9 2 3 Python License Supplement Windows Only The Windows installers for Python may include a copy of the OpenSSL libraries The terms of the Oracle license do NOT apply to the OpenSSL libraries it is licensed under the following license separately from the Oracle programs you receive If you do not wish to install this program you may uninstall this 295 Python License Supplement Windows Only version of Python The OpenSSL toolkit stays under a dual license i e both the conditions of he OpenSSL License and the original SSLeay license apply to the toolkit ct See below for the actual license texts Actually both licenses are BSD style pen Source licenses In case of any license issues related to OpenSSL please contact openssl core openssl org O OpenSSL License Copyright c 1998 2011 The OpenSSL Project All rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met 1 Redistributions of source code must retain the above c
209. custom Mult iTenantConnectionProvider itis access data related to shard key 40 Session session sessionFactory withOptions tenantIdentifier 40 openSession connection to an appropriate server This cannot be changed for the duration of the Session 8 8 10 Connector J Fabric Support Reference 8 8 10 1 Connection Properties The following connection properties are recognized by Connector J for dealing with MySQL Fabric e fabricShardKey The initial shard key used to determine which server group to send queries to The fabricShardTable property must also be specified e fabricShardTable The initial shard mapping used to determine a server group to send queries to e fabricServerGroup The initial server group to direct queries to fabricProtocol Protocol used to communicate with the Fabric node XML RPC over HTTP is currently the only supported protocol and is specified with a value of http e fabricUsername Username used to authenticate with the Fabric node e fabricPassword Password used to authenticate with the Fabric node 275 MySQL Fabric Frequently Asked Questions fabricReportErrors default false Determines whether or not errors are reported to Fabric s distributed failure detector Only connection errors those with a SQL state beginning with 08 are reported 8 8 10 2 Fabric MySQLConnection API The following methods are available in the com mysql fabric jdbc FabricMySQLConnection int
210. d configuration parameters for the XML RPC protocol on the server address threads user password disable_authentication realm ssl_ca ssl_cert ssl_key Section protocol mysql Host and port of XML RPC server The host is only used by the client when connecting to the MySQL Fabric node but the port is used by the server when starting the protocol server and by the client when reading how to connect to the XML RPC server The port number is typically 32274 and the host is typically Localhost Number of threads that the XML RPC server uses for processing requests This determines the number of concurrent requests that MySQL Fabric accepts User that the client uses to connect to the XML RPC server Password used when the client connects to the server If no password is provided the client requests a password on the command line Whether to disable authentication or not Disabling authentication can be useful when experimenting in a closed environment it is not recommended for normal usage Alternatives are yes or no and are case insensitive The realm as defined in RFC 2617 the XML RPC server identifies as when authenticating Path to a file containing a list of trusted SSL certification authorities CAs The name of the SSL certificate file to use for establishing a secure connection The name of the SSL key file to use for establishing a secure connection This section contains information about how
211. d its associated commands 8 4 1 Getting mysql mysql mysql mysql Help lfabric help lfabric help Show syntax information and the help commands lfabric help commands List the available commands and their description lfabric help groups List the available groups group command Provide detailed information on a command shell gt mysqlfabric help group create group create group_id Create a group description NONE synchronous shell gt mysqlfabric help Usage mysqlfabric sara cemirile lt eige gt lt em gt are aoada MySQL Fabric 1 5 6 MySQL server farm management framework Options version n laella param CONFIG_PARAMS config FILE Basic commands show program s version number and exit show this help message and exit Override a configuration parameter Read configuration from FILE 238 Group Commands help lt grp gt lt cmd gt Show help for command help commands List all commands help groups iaist all groups 8 4 2 Group Commands The group commands are used to define modify and control groups group activate Activate failure detector for server s in a group By default the failure detector is deactivated group group group group group group group group group group group deactivate Deactivate the failure detector for server s in a group create Create a group remove Remove a server
212. d line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl key The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required timeout lt seconds gt 123 NOTES Maximum timeout in seconds to wait for each replication command to complete For example timeout for slave waiting to catch up to master Default 3 verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES The logi
213. d master but not sent The utility accepts a list of slaves to be considered the candidate slave If no slave is found to meet the requirements the operation will search the list of known slaves The utility also provides a number of useful commands for managing a replication topology including the following elect This command is available to only those servers supporting global transaction identifiers GTIDs perform best slave election and report best slave to use in the event a switchover or failover is required Best slave election is simply the first slave to meet the prerequisites GTIDs are supported in version 5 6 5 and higher This command requires the options master and either slaves or discover slaves login failover This command is available to only those servers supporting GTIDs Conduct failover to the best slave The command will test each candidate slave listed for the prerequisites Once a candidate slave is elected it is made a slave of each of the other slaves thereby collecting any transactions executed on other slaves but not the candidate In this way the candidate becomes the most up to date slave This command requires the slaves option The discover slaves login option is not allowed 154 mysqlrpladmin Administration utility for MySQL replication because for failover the master is presumed to be offline or otherwise unreachable so there is no way to discover the slaves The master
214. d restore user but in this case the user would have the sum of privileges of the three users This would result in a very powerful user and is therefore not recommended for production use However for a quick and simple temporary trial it may be easiest to set the users for all accounts using the same user name and password such as root 216 Configuring MySQL Fabric The Backing Store Fabric User The first thing you must have is a user account on the MySQL server that you plan to use for your backing store The user account information is stored in the Fabric configuration file The backing store database and its associated user are defined under the storage using user for the user name and password as the password The Fabric user account on the backing store requires the following privileges on the backing store database ALTER alter some database objects CREATE create most database objects CREATE VIEW create view DELETE delete rows DROP drop most database objects EVENT manage events INDEX create index INSERT insert rows REFERENCES foreign keys SELECT select rows UPDATE update rows Example statements to create this user to be executed on the backing store MySQL server example is named mysq _fabric In other words do not execute CREATE database Note MySQL Fabric will create this database based on fabric cfg which in our mysql_fabric here CREA
215. d the audit log plugin is available as of MySQL Server versions 5 5 28 and 5 6 10 This utility requires Python version 2 6 or higher but does not support Python 3 The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf This feature exists as of MySQL Server 5 6 6 see mysql_config_editor MySQL Configuration Utility Changes to MySQL Enterprise Audit Log Plug in are not documented in this documentation so your output might be different than the examples here For example a new or removed MySQL Enterprise Audit Log Plug in option might affect the output LIMITATIONS The remote login option is not supported on Microsoft Windows platforms For Microsoft Windows use UNC paths and perform a local copy operation omitting the remote login option EXAMPLES To display the audit log system variables run the following command shell gt mysqlauditadmin show options server root localhost 3310 Showing options after command Audit Log Variables and Options 4 4 Variable_name Value 4 4 audit_log_buffer_size 1048576 audit_log_connection_policy NONE audit
216. d the slave is reading the correct master log file and slave delay is not more than the seconds behind threshold option reset Execute the STOP SLAVE and RESET SLAVE commands on all slaves This command requires the slaves option The discover slaves login option is not allowed because it might not provide the expected result excluding slaves with the IO thread stopped Optionally the master option can also be used and in this case the utility will perform an additional check to verify if the specified slaves are associated replication is configured to the given master start Execute the START SLAVE command on all slaves This command requires the slaves option The discover slaves login option is not allowed because it might not provide the expected result excluding slaves with the IO thread stopped Optionally the master option can also be used and in this case the utility will perform an additional check to verify if the specified slaves are associated replication is configured to the given master stop Execute the STOP SLAVE command on all slaves This command requires the slaves option The discover slaves login option is not allowed because it might not provide the expected result excluding slaves with the IO thread stopped Optionally the master option can also be used and in this case the utility will perform an additional check to verify if the specified slaves are associated replication is conf
217. d to be kept to later track the responsible for some change Moreover the retrieved information might need to be converted to a specific format such as CSV to feed another application Objectives The goal of this task is to retrieve from the audit log the information of all the connections established by the root user to the MySQL Server and display the resulting information in the comma separated value CSV format 59 How Do Display Connections by the User root and Show the Result in CSV Format Besides the search filter functionalities using different criteria the mysqlauditgrep utility also provides a feature to display the resulting information in different formats including CSV This allows this task to be performed easily with in a single step It is assumed that the audit 1log file exists and is located in the directory MySQL SERVER data Example Execution shell gt mysqlauditgrep user root event type Connect format CSV MySQL SERVER data audit log STATUS NAME TIMESTAMP CONNECTION_ID HOST USER PRIV_USER IP 0 Connect 20T Uc OITIS AiG sa localhost OO tao Oty elon ie ORnConnect ONS Uc OMS rel AGr SA localhost Ol Toot 2E 0 Connect 01S 03 OITIS 24 26 Shs localhost iekene EGOE 1277 5 0 Connect 2013 O3 Oil wis 3 24s OG She Ocal host Eoo EOE la 0 Connect 2013 03 OITIS 18 4327 localhost root TeeKeNe p12 7 5 0 Connect ACI SO O wil se ols Shs ietorere root 19
218. d to do so subject to the following conditions The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software THE SOFTWARE IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND EXPRESS R IMPLIED INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY ITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM DAMAGES OR OTHER LIABILITY WHETHER IN AN ACTION OF CONTRACT TORT OR OTHERWISE ARISING ROM OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS N THE SOFTWARE Fal lt O J 7 9 2 2 Python License The following software may be included in this product Python Programming Language This is the official license for the Python 2 7 release A HISTORY OF THE SOFTWARE Python was created in the early 1990s by Guido van Rossum at Stichting Mathematisch Centrum CWI see http www cwi nl in the Netherlands as a successor of a language called ABC Guido remains Python s principal author although it includes many contributions from others In 1995 Guido continued his work on Python at the Corporation for National Research Initiatives CNRI see http www cnri reston va us in Reston Virginia where he released several versions of the software In May 2000 Guido and the Python core development team moved to BeOpen com to form the BeOpen PythonLabs te
219. d using RANGE_STRING Causes a ValueError to be raised e For RANGE_DATETIMB a datetime or date object key is required For example to get the shard which holds employees hired after the year 2000 you could do the following with lower bounds set as group1 1980 01 01 group2 2000 01 01 cnx set_property tables employees employees key datetime date 2000 1 1 mode fabric MODE_READONLY If the lower bounds included a time it would have been like this cnx set_property tables employees employees key datetime datetime 2000 1 1 12 0 0 mode fabric MODE_READONLY Only datetime datetime and datetime date values are supported Any other type given when using a shard defined using RANGE_DATETIME causes a ValueError to be raised 8 8 Using Connector J with MySQL Fabric 261 Installing Connector J with MySQL Fabric Support MySQL Fabric provides data distribution and high availability features for a set of MySQL database servers Developers using Connector J can take advantage of its features to work with a set of servers managed by MySQL Fabric Connector J supports the following MySQL Fabric capabilities e Reporting errors to the Fabric node as part of the distributed failure detector The Fabric Client library for Java which is included with Connector J is comprised of the following packages e src com mysql fabric xmlrpc Classes for core implementation of the XML RPC protocol e src com my
220. deaceciedecisdveuceteteecee dds dea padeeceetalveuconseveceed dled ndbeneisleecaeieineceenates 245 8 6 3 Target Configuration 2 00 00 ceceee cece cece ee eee nannaa nanaawan i sees aae Naa RAE a 246 8 6 4 Setting up and testing your System 200 00 eee ceeeceee cece cece cece ce etee eect tees ee aaaaeneeeeeeeeeaaaaeeneeeeeeees 247 8 6 5 Key administrative tasks 2 0 0 ccc ee eeee tree eee e eee aa eee ki aaa eeeeaaaeeeeeaaaeeeeeaaaeeeeeaaaeeeeeaaeeees 255 8 7 Using Connector Python with MySQL Fabric ccccceeeeeee ee ceeceeeeeeeeee ee ae eae eeeeeeeeeeaaaaeeeeeeeeeeeeaaa 256 8 7 1 Installing Connector Python with MySQL Fabric Support c cceeeeeeeeeeeeeeeeeeeeeaeaaeees 257 8 7 2 Requesting a Fabric Connection ccccece ce eeeeeeeeeeeeee ceca ee ee eeeeeeeeeeaaaaeeeeeeeeeeaeaaeeeeeeeeees 257 8 7 3 Providing Information to Choose a MySQL Server ccceceeeeeeeeeeeeeeeeeeeeeeeaaaaeeeeeeeeeeeeaea 259 8 8 Using Connector J with MySQL Fabric 0 0 0 0 cccceeeee cece cece ee ee cette eter eres ee ae aaa teeeeeeeeeaeaaaaeeeeeeeeeeeeaaa 261 8 8 1 Installing Connector J with MySQL Fabric Support ceceeeeeeeeeeeeeeeeeeaeaaeteeeeeeeeeaeaaea 262 8 8 2 Loading the Driver and Requesting a Fabric Connection cccccceeeeeeeeeeeeeeeeeeeeeeeeeeeeaea 262 8 8 3 Providing Information to Choose a MySQL Server cceeeeeeeeeeeeeeeeeeeeeeeeeaaaaeeeeeeeeeeeeaaa 263 8 8 4 MySQL Fabric Configuration for R
221. ded data set e Default location 127 0 0 1 3403 e Config properties com mysql fabric testsuite shard2 host com mysql fabric testsuite shard2 port All except mysql fabric config should have server id set to a distinct value and the following entries added to my cnf log bin mysg bin log slave updates true enforce gtid consistency true gtid mode on Set up sharding The user name and password of the account used to manage Fabric Section 8 2 3 1 Create the Associated MySQL Users must be specified in Fabric s configuration file Section 8 2 3 2 Configuration File e Create the global group shell gt mysqlfabric group create fabric_test1_global shell gt mysqlfabric group add fabric_test1_global 127 0 0 1 3401 shell gt mysqlfabric group promote fabric_test1_global e Create shard groups shell gt mysqlfabric group create fabric_test1_shardl shell gt mysqlfabric group add fabric_testl_shardl 127 0 0 1 3402 shell gt mysqlfabric group promote fabric_test1_shard1 shell gt mysqlfabric group create fabric_test1_shard2 shell gt mysqlfabric group add fabric_testl_shard2 127 0 0 1 3403 shell gt mysqlfabric group promote fabric_test1_shard2 e Create the sharding definition shell gt mysqlfabric sharding create_definition RANGE fabric_test1_global Notice the return value in the command output for example return 1 This is the S MAPPING_ID used in the following commands shell gt mysqlfabric shardin
222. defaults This example shows how you can fine tune the scripts to meet your specific needs without having to specify a lot of additional options in your script We enable the quiet option on so as not to clutter the screen with messages and tell the copy to skip databases that do not exist in case we supply the databases option and provide a database that does not exist options vouiret t r DTrUS force True The actual copy of the databases is easy Just call the method and supply the list of databases from mysql utilities command import dbcopy Gives dbcopy copy_db conn dest_values db_list options except exception UtilError e print ERROR e errmsg sys exit Lastly we copy the user accounts Once again we must provide a dictionary of options and call the command module directly In this case the userclone module provides a method that clones one user to one or more users so we must loop through the users and clone them one at a time from mysql utilities command import userclone options overwrite True quiet True glopars True geoi Weer abe CSa STe EEN res userclone clone_user conn dest_values user user options except exception UtilError e print ERROR e errmsg sys exit We are done As you can see constructing new solutions from the MySQL utility command and common modules is easy and is limited only by your imagination 200 Enhancing the E
223. der 9 1 1 Can the frm reader read a frm file without the associated data files cccsecseeseeeeeeeeeeeeees 283 9 1 2 Will the frm reader modify my original frm file eee cece cece cece ee ee ae ee eeeeeeeeeeeeaeaaeeneeeeeeeeeaeaaea 284 9 1 3 What is diagnostic mode and why doesn t it produce the same output as the default mode 284 9 1 4 If the diagnostic mode is only a best effort compilation why use it eeeeeeeeeeeeeeeeeeeeeeeeeeees 284 9 1 5 Why does the default mode require a server cceceeeeeee ee eeeeeeeeeeeeee ee aeeeeeeeeeeeeeeeaaaaeeeeeeeeeeeeaaa 284 9 1 6 Can the frm reader read any frm file cece eee ceee cece ce eeeeeeeeeee tees aaaaeeeeeeeeeeeeaaaaaeneeeeeeeeeaeaaeeaes 284 9 1 7 My frm files are tucked away in a restricted folder How do get access to them to run the frm reader without copying or modifying file privileges eeeeceeeeeeeeeeeee ee aeaeeeeeeeeeeeeeaeaaeateneeeees 284 9 1 8 Will the default mode display a 100 accurate CREATE statement cccceccceceaseeeeeeeaeeees 284 9 1 1 Can the frm reader read a frm file without the associated data files Yes The frm reader was designed to read the contents of an frm file without requiring the data files 283 Third Party Licenses 9 1 2 Will the frm reader modify my original frm file No it does not modify the original frm file in either default or diagnostic mode 9 1 3
224. des a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt show defaults d Display default settings for mysqid from the local configuration file It uses my_print_defaults to obtain the options show servers Display information about servers running on the local host The utility examines the host process list to determine which servers are running ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required Start S Start the server in read only mode
225. disk usage for databases mysqlfailover automatic replication health monitoring and failover mysqlfrm show CREATE TABLE from frm files mysqlindexcheck check for duplicate or redundant indexes mysqlmetagrep search metadata mysqlprocgrep search process information mysqlreplicate establish replication with a master mysqlrpladmin administration utility for MySQL replication mysqlrplcheck check replication mysqlrplms establish multi source replication mysqlrplshow show slaves attached to a master mysqlrplsync replication synchronization checker utility mysqlserverclone start another instance of a running server mysqlserverinfo show server information mysqluserclone clone a MySQL user account to one or more new users mysqluc gt help mysqldb TAB TAB Weale Description mysqldbcompare compare databases for consistency mysqldbcopy copy databases from one server to another mysqldbexport export metadata and data from databases mysqldbimport import metadata and data from files mysqluc gt mysqlrplshow m TAB TAB Option Description master MASTER connection information for master server in the form lt user gt lt password gt lt host gt lt port gt lt socket gt Oe lt ikosain jeeielns eGeroiic gt o lt seelasic gt lt max depth MAX_DEPTH limit the traversal to this depth Valid only with the recurse option Valid values are non negative integers mysqluc gt mysqlrplshow mast TAB er
226. e enum RANGE HASH Shard map type 243 Protecting the Backing Store Column Type Comment global_ group varchar 64 Name of the global group likely to go away in the next revision The shard_ranges table is the sharding index and is used to map a sharding key to a shard Table 8 12 shard_ranges Column Type Comment shard_mapping_id int 11 Shard map identifier lower_bound varbinary 16 Lower bound for the range encoded as a binary string shard_id lint 1 1 Shard identifier a number The shard_tables table lists all tables that are sharded and what sharding map each belongs to It also names the column by which it is sharded Table 8 13 shard_tables Column Type Comment shard_mapping_id linci 1 Shard map identifier table_name varchar 64 Fully qualified table name column_name varchar 64 Column name that is the sharding key The shards table names the groups where each shard identifier is stored Table 8 14 shards Column Type Comment shard_id int 11 Shard identifier group_id varchar 64 Group identifier a dotted name state enum DISABLED ENABLED Status of the shard DISABLED means that it is not available for use The users table identifies the users that might have permission to access the functions in the different sub systems Table 8 15 users
227. e help Display a help message and exit e license Display license information and exit e autocommit Enable autocommit for data import By default autocommit is off and data changes are only committed once at the end of each imported file e bulk insert b Use bulk insert statements for data character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client e default storage engine lt def_engine gt The engine to use for tables if the destination server does not support the original storage engine on the source server drop first d Drop each database to be imported if exists before importing anything into it e dryrun Import the files and generate the statements but do not execute them This is useful for testing input file validity 104 OPTIONS format lt format gt f lt format gt Specify the input format Permitted format values are sql default grid tab csv raw_csv and vertical e import lt import_type gt i lt import_type gt Specify the import format Permitted format values are Table 5 1 mysqldbimport Import Types Import Type Definition definitions default Only import the definitions metadata for the objects in the database list data Only import the table data for the tables in the database list both Import both the definitions metadata and data If you attempt to i
228. e If not specified a notification concerning this requirement will be printed And if format is passed in without search parameters then all the records of the audit log are displayed in the specified format The file stats option is not considered a search criteria and is used to display the file statistics of a specified audit log Other search options will be ignored when the file stats option is used except the format option will continue to format the results accordingly To specify the format of the generated results use one of the following values with the format option GRID default Display output in grid or table format like that of the mysql client command line tool e CSV Display output in comma separated values format e VERTICAL Display output in single column format like that of the G command for the mysq1 client command line tool e RAW Display output results in the original raw format of the audit log records which is written in XML 71 Standard SQL Pattern Matching Standard SQL Pattern Matching The simple patterns defined by the SQL standard enables users to use two characters with special meanings percent matches zero or more characters and _ underscore matches exactly one arbitrary character In standard SQL these types of patterns are used with the LIKE comparison operator and they are case insensitive by default This utility assumes that they are case insensi
229. e use the option or read the appropriate manual page The utilities are designed to work on MySQL systems with grants enabled but can also operate on servers started with the skip grant tables Startup option However this practice is strongly discouraged and should be used only in situations where it is appropriate or deemed a last resort 2 2 Connecting to MySQL Servers This section describes the ways you can connect to a MySQL server via a MySQL Utility or via the MySQL Utilities library methods 2 2 1 Connection Parameters To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt conftiguration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwd gt lt host gt lt port gt lt socket Connection Parameters 2 2 1 1 Use login paths mylogin cnf The best way to
230. e GTIDs executed to that point e verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects in the operation To export all objects from a source database the user must have these privileges SELECT and SHOW VIEW on the database as well as SELECT on the mysq1 database Actual privileges needed may differ from installation to installation depending on the security privileges present and whether the database contains certain objects such as views or events Some combinations of the options may result in errors when the export is imported later For example eliminating tables but not views may result in an error when a view is imported on another server For the format export and display options the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utili
231. e MySQL server with the new value for the 1log bin option log bin mysql server binlogs server bin 12 How do you purge obsolete binary log files safely Discussion The above example illustrates how to change the binary log directory in an effortless way using the mysqlbinlogmove utility to move existing binary log files to the desired location Changing the 1og bin option requires the restart of the MySQL server Moreover the server also needs to be stopped in order to move all binary log files correctly otherwise an error might occur while moving files currently in use by the server In this case to relocate all available binary log files with the mysqlbinlogmove utility we simply need to specify their source directory using the binlog dir option and the target directory as an argument The binary log files are identified based on the default filename format i e with a base filename ending with bin If a custom basename is used not ending with bin then the bin log basename option must be used to specify it In the above example this option is not required because the binary log basename server bin matches the default format Similarly if a custom location or name is used for the binary log index file it must be specified using the bin log index option By default the binary log index file is assumed to be located in the specified source binary log directory and to use the default filename f
232. e Regents of the University of California All rights reserved any purpose without fee is hereby granted provided that this en tire notice is included in all copies of any software which is or includes a copy or modification of this software and in all Permission to use copy modify and distribute this software for 290 Python License copies of the supporting documentation for such software This work was produced at the University of California Lawrence Livermore National Laboratory under contract no W 7405 ENG 48 between the U S Department of Energy and The Regents of the University of California for the operation of UC LLNL DISCLAIMER This software was prepared as an account of work sponsored by an agency of the United States Government Neither the United States Government nor the University of California nor any of their em ployees makes any warranty express or implied or assumes any liability or responsibility for the accuracy completeness or usefulness of any information apparatus product or process disclosed or represents that its use would not infringe privately owned rights Reference herein to any specific commer cial products process or service by trade name trademark manufacturer or otherwise does not necessarily constitute or imply its endorsement recommendation or favoring by the United States Government or the University of California The views and opinions of au
233. e and re type For these reasons and more the MySQL Utilities were created to help both beginners and experienced database administrators perform common tasks What are the internals of the MySQL Utilities MySQL Utilities are designed as a collection of easy to use Python scripts that can be combined to provide more powerful features Internally the scripts use the mysql utilities module library to perform its various tasks Since a library of common functions is available it is easy for a database administrator to create scripts for common tasks These utilities are located in the scripts folder of the installation or source tree If you have a task that is not met by these utilities or one that can be met by combining one or more of the utilities or even parts of the utilities you can easily form your own custom solution The following sections present an example of a custom utility discussing first the anatomy of a utility and then what the mysql utilities module library has available Anatomy of a MySQL Utility MySQL Utilities use a three tier module organization At the top is the command script which resides in the scripts folder of the installation or source tree Included in the script is a command module designed to encapsulate and isolate the bulk of the work performed by the utility The command module resides in the mysql utilities commana folder of the source tree Command modules have names similar to the script A command m
234. e contents of the log file which only visible if the 10g option is specified This can be helpful to see when failover occurred and which actions or messages were recorded at the time The user interface is designed to match the size of the terminal window in which it is run A refresh option is provided to permit users to resize their terminal windows or refresh the display at any time However the interface will automatically resize to the terminal window on each interval The interface will display the name of the utility the master s status including binary log file position and filters as well as the date and time of the next interval event The interface will also permit the user to scroll up or down through a list longer than what the terminal window permits When a long list is presented the scroll options become enabled The user can scroll the list up with the up arrow key and down with the down arrow key Use the verbose option to see additional information in the health report and additional messages during failover 119 OPTIONS The utility supports two modes of operation The default mode running as a console works as described above An additional mode that permits you to run the utility as a daemon is provided for POSIX platforms When run as a daemon the utility does not have interactivity However all events are written to the log file You can control what is written to the log by using the report v
235. e delay is NNN seconds 192 leo Ss asne SLAVE we ON Slave delay is NNN seconds 4 4 4 4 done shell gt mysqlrpladmin master m_account 192 168 1 1 13001 slaves slave_acc1 192 168 1 2 13001 slave_acc2 192 168 1 3 3306 health Checking privileges Replication Topology Health 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 WO 5 Seaia AL ESO MASTER UP ON OK O Aen aA ESO SLAVE UP ON l OK SA aeae 3206 SLAVE UP ON OR 4 4 4 4 4 done shell gt mysqlrpladmin master m_account 192 168 1 1 13001 slaves slave_acc1 192 168 1 2 13001 slave_acc2 192 168 1 3 3306 new master slave_accl localhost 13002 demote master switchover Checking privileges Performing switchover from master at 192 168 1 1212001 toe slave at 192 1 68 1 22123001 Checking candidate slave prerequisites Checking slaves configuration to master Waiting for slaves to catch up to old master Stopping slaves Performing STOP on all slaves Demoting old master to be a slave to the new master Switching slaves to new master Starting all slaves Performing START on all slaves Check
236. e display format Permitted format values are unified context differ and sql The default is unified disable binary logging If binary logging is enabled disable it during the operation to prevent comparison operations from being written to the binary log Note Disabling binary logging requires the SUPER privilege exclude lt exclude gt x lt exclude gt Exclude one or more databases from the operation using either a specific name such as db1 or a search pattern Use this option multiple times to specify multiple exclusions By default patterns use database patterns such as LIKE With the regexp option patterns use regular expressions for matching names Added in release 1 4 0 format lt format gt f lt format gt Specify the display format for changed or missing rows Permitted format values are grid csv tab and vertical The default is grid compact Compacts the output by reducing the number of control lines that are displayed in the diff results This option should be used together with one of the following difference types unified or context It is most effective when used with the unified difference type and the grid format quiet q Do not print anything Return only an exit code of success or failure regexp basic regexp G Perform pattern matches using the REGEXP operator The default is to use LIKE for matching Added in release 1 4 0 run all tests t Do not halt at the first diff
237. e here we created a copy of the database and changed all tables in the destination database to use the InnoDB storage engine with the new storage engine option 18 How Can Tell If a Table on Server X has the same Structure as the Same Table on Server Y We show proof of the change by displaying the CREATE statement for one of the tables on the destination server Notice we also renamed the database by using the lt old_name gt lt new_name gt syntax Permissions Required The user must have SELECT privileges for the database s on the source server and have CREATE INSERT UPDATE on the destination server Tips and Tricks Risks You can exclude specific options by using the exclude option specifying a SQL pattern expression For example to exclude objects that start with xy use exclude xy You can use REGEXP patterns in the exclude option by specifying regexp in addition to the exclude option Should the copy fail in the middle the destination databases may be incomplete or inconsistent Should this occur drop the destination database in question repair the cause of the failure and restart the copy 3 2 4 How Can Tell If a Table on Server X has the same Structure as the Same Table on Server Y Multiple database servers that are kept synchronized manually or are compartmentalized for security purposes but are by practice kept up to date manually are prone to unintentional and sometimes inten
238. e if both are given If the new storag ngine or default storage engine option is given and the destination server does not support the specified storage engine a warning is issued and the server s default storage engine setting is used instead By default the operation uses a consistent snapshot to read the source databases To change the locking mode use the locking option with a locking type value Use a value of no locks to turn off locking altogether or lock all to use only table locks The default value is snapshot Additionally the utility uses WRITE locks to lock the destination tables during the copy You can include replication statements for copying data among a master and slave or between slaves The rp1 option permits you to select from the following replication statements to include in the export master Create and execute a CHANGE MASTER statement to make the destination server a slave of the server specified in the source option This executes the appropriate STOP and START slave statements The STOP SLAVE statement is executed at the start of the copy and the CHANGE MASTER followed by the START SLAVE statements are executed after the copy slave Create and execute a CHANGE MASTER statement to make the destination server a slave connected to the same master as the server specified in the source option This executes the appropriate STOP and START slave statements The STOP SLAVE statement is execut
239. e is experienced How many machines are needed in order to use MySQL Fabric 277 MySQL Fabric Frequently Asked Questions 8 9 7 8 9 8 8 9 9 For development the MySQL Fabric node and all of the managed MySQL Servers can be hosted on a single machine For deployment the minimal HA configuration would need 3 or more machines e 2 to host MySQL Servers e 1 to host the MySQL Fabric process that machine could also be running application code Do need to run an agent for each MySQL Server No The MySQL Fabric node is the only additional process and does not need to be co located with any of the MySQL Servers that are being managed What interface is available to manage MySQL Fabric and its server farm A Command Line Interface CLI is provided as well as an XML RPC API that can be used by connectors and or applications to make management changes or retrieve the routing information in this way an application could use MySQL Fabric without a Fabric aware connector How does MySQL Fabric Compare with MySQL Cluster MySQL Cluster is a mature well proven solution for providing very high levels of availability and scaling out of both reads and writes Some of the main extra capabilities that MySQL Cluster has over MySQL Fabric are e Synchronous replication e Faster automated fail over resulting in higher availability e Transparent sharding e Cross shard joins and Foreign Keys e In memory real time perfor
240. e or more sections with connection information Saving the data in configuration files is more secure than specifying the data on the command line but since the file is text the data can still be read by anyone who can access the file To reference the configuration file specify the path and file name followed by a section name in square brackets The path is optional If you do not specify it the utility will attempt to use your local my cnf if available For example if you wanted to create a configuration file in dev env test1 my cnf and you created a section named server1 you would specify it as server dev env test1 my cnf server1 The corresponding section in the file may look like the following server1 port 3308 user root password other pass host localhost The following shows the execution of a utility using a configuration file shell gt mysqlserverinfo py server dev env test1 my cnf serverl format vertical Source on localhost connected KKKKKKKKKKKKKKKKKKKKKKKKK Ik row KKKKKKKKKKKKKKKKKKKKKKKKK server localhost 13001 config_file etc my cnf etc mysql my cnf binary log clone bin 000001 Dinary Iheyep joreysyg eal relay_log relay_log_pos Weesdeime Bp il y lherg datadir Volumes Source source temp_13001 basedir Volumes Source source bzr mysql 5 6 plugin_dir Volumes Source source bzr mysql 5 6 lib plugin general_log OFF general_log_file general_log_file_size log_error l
241. e to execute on start Fora complete list of options see MySQL Users Console manual page The next command help shows the help for the users console itself As you can see there are a number of options available You can set user defined variables discover the help for other utilities display the latest error and see the options used to start the console The help utilities command shows you a list of the available utilities and a short description of each Next we decide we want to get help for one of the database utilities but we do not remember the name We know it starts with mysqldb but we are not sure of the rest In this case if we type mysqldb then hit TAB twice the users console will show us a list of all of the utilities that begin with mysqldb Now let s say we want to see a graph of our replication topology but we are not sure what the option for specifying the master In this case we type the command to launch the mysqirplshow utility and type the start of the option m then press TAB twice What we see is there are two options that match that prefix Notice we also see a short description help for each This is a real time saving feature for the users console Notice in the next segment we do not have to type the entire name of the option In this case we typed mast TAB which the users console completed with master This is tab completion for option names Notice the result of the command we ent
242. eads frm files and produces equivalent CREATE statements from the table definition data found in the file In most cases the generated CREATE statement is usable for recreating the table on another server or for extended diagnostics However some features are not saved in the frm files and therefore will be omitted The exclusions include but are not limited to e foreign key constraints e auto increment number sequences The mysqifrm utility has two modes of operation The default mode is designed to spawn an instance of an installed server by referencing the base directory using the basedir option or by connecting to the server with the server option The process will not alter the original frm file s This mode also requires the port option to specify a port to use for the spawned server It must be different than the port for the installed server and no other server must be using the port The spawned server will be shutdown and all temporary files removed after the frm files are read A diagnostic mode is available by using the diagnostic option This switches the utility to read the frm files byte by byte to recover as much information as possible The diagnostic mode has additional limitations in that it cannot decipher character set or collation values without using an existing server installation specified with either the server or basedir option This can also affect the size of the columns if the table uses multibyte ch
243. eat IPaddr2 params ip 192 168 1 200 cidr_netmask 24 nic eth0o primitive p_mysql ocf heartbeat mysql params binary usr sbin mysqld contig var lib mysql_drod my cni datadir var lib mysql_drbd data pid var run mysqld mysqld pid socket var run mysqld mysqld sock user mysql group mysql additional_parameters bind address localhost op start timeout 120s interval 0 op stop timeout 120s interval 0 op monitor interval 20s timeout 30s groupo mysql p_fs_mysql pip mysql p mysql p_fabric_mysql ms ms_drbd_mysql p_drbd_mysql meta master max 1 master node max 1 clone max 2 clone node max 1 notify true colocation c_mysql_on_drbd inf g_mysql ms_drbd_mysql Master order o_drbd_before_mysql inf ms_drbd_mysql promote g_mysql start primitive p_ping ocf pacemaker ping params name ping multiplier 1000 host_list 192 168 1 1 op monitor interval 15s timeout 60s start timeout 60s clone cl_ping p_ping meta interleave true location 1_drbd_master_on_ping ms_drbd_mysql rule role Master inf not defined ping or ping number lte 0 As the MySQL service group has a dependency on the host it is running on being the DRBD master that relationship is added by defining a co location and an ordering constraint to ensure that the MySQL group is co located with the DRBD master and that the DRBD promotion of the host to the master must happen before the MySQL
244. eceeeeeeeeeenteeeeeeeeeeees 108 5 8 mysqldiskusage Show Database Disk Usage cccccsseeeeeeaeeeeeeeaaeeeeeeaaeeeeeeaaeeees 114 5 9 mysql failover Automatic replication health monitoring and failover cceeeee 117 5 10 mysql frm File reader for frm files icecccccisecccctssecccactadeceatticeceaactecteacstieeescnsadecaasnaeeeaess 128 5 11 mysqlindexcheck ldentify Potentially Redundant Table Indexes c sseceeeenneeees 132 5 12 mysqlmetagrep Search Database Object Definitions cccceeeeeeeeeeeeeeeeeeeeeeeeeeees 136 5 13 mysqlprocgrep Search Server Process Lists s cccccesseeeeceseeeeeeaueeeeeeaueeeeesangees 140 5 14 mysqlreplicate Set Up and Start Replication Between Two Servers 0 ccceseeeeees 144 5 15 mysql rplms Set Up and Start Replication Among a Slave and Multiple Masters 149 5 16 mysqlrpladmin Administration utility for MySQL replication ccecceeeeeeeeeeeeeeeneeees 154 5 17 mysqlrplcheck Check Replication Prerequisites ccccceeesseeeeeeaeeeeeeaaeeeeeeaaeneees 164 5 18 mysqlrplshow Show Slaves for Master Server cccccceseeceeesseeeeeeeeeeeeeeaaeeeeeeeaeeeeees 169 5 19 mysqlrplsync Replication synchronization checker c ccceeeeeeeeeeeeeeeeeaeeeeeeeeeaaes 174 5 20 mysqlserverclone Clone Existing Server to Create New Server cc sseeeeeenneeeees 181 5 21 mysqlse
245. ecking Gl table date OK test_db tl1 checksum for server localhost 3311 OK test_db tl1 checksum for server localhost 3312 done SUMMARY No data consistency issue found Discussion The above example illustrates how to start the mysqlrplsync utility to check if all data on the specified replication topology is synchronized To do this we simply need to specify the master s connection with the master 175 option and the list of slaves with the slaves 176 option As an alternative to the slaves 176 option one can use the discover slaves login 175 specifying a user and password or login path to connect to the slaves and the utility will attempt to discover all the slaves connected with the master using the specified login For example discover slaves login root secret is used to discover all of the slaves and login to each using the root user id and the password secret The discover slaves login 175 can be very handy especially if there is a huge number of slaves in the topology but bear in mind that the explicit specification of slaves is safer and that discovery can fail to find some servers In particular it is important to note that in order for slaves to be discovered they must be started with the report host and report port options with appropriate values and they must be correctly connected to the master IO thread running otherwise d
246. ecomes the new master Objectives The goal is to execute the mysqlslavet rx utility to skip errant transactions on slaves making sure that those transaction will not be replicated if slave that originated them becomes the new master Skipping errant transactions is done by injecting an empty transaction one by one for each corresponding GTID on every slave This can be a very tedious task when performed manually especially if many transactions need to be skipped Thankfully the mysqlslavetrx utility allows us to skip multiple transactions on multiple slaves in a single step Let s assume that we have three slaves slave1 3311 slave2 3312 and slave3 3313 and that one of the slaves slave1 3311 has five errant transactions that need to be skipped on the other slaves The GTID set of those transactions is ce969d18 7b10 11e4 aaae 606720440b68 1 5 Example Execution Execute the mysqislavetrx utility specifying the GTID set of the transaction to skip and the target slaves shell gt mysqlslavetrx gtid set ce969d18 7b10 11e4 aaae 606720440b68 1 5 slaves dba pass slave2 3312 dba pass slave3 3313 WARNING Using a password on the command line interface can be insecure GTID set to be skipped for each server slave2 3312 ce969d18 7b10 1le4 aaae 606720440b68 1 5 slave3 3313 ce969d18 7b10 1le4 aaae 606720440b68 1 5 Injecting empty transactions for slave2 3312 Injecting empty transactions for slave3 3313
247. ection protocol to use optional default xml rpc Permitted values are xml rpc use XML RPC protocol and mysql use MySQL client server protocol If a value of mysq1 is specified the default port becomes 32275 although that can be changed with an explicit port value The username password report_errors ssl_ca ssl_cert and ssl_key options were added in Connector Python 1 2 1 It is possible to establish an SSL connection using only the ss1_ca argument The ssl1_key and ss1_cert arguments are optional However when either is given both must be given oran AttributeError is raised The protocol option was added in Connector Python2 2 1 2 It is also possible to pass a Fabric object instance as the fabric argument fabric_config host fabric example com fabinst Fabric fabric_config fonx mysql connector connect fabric fabinst user webuser password webpass database employees Or fabric_config host fabric example com fabinst Fabric fabric_config config Neglorese g EEN ASEA user webuser password webpass database employees fonx mysql connector connect config Once a Fabric object is used it is cached and reused Another less preferred way to establish a Fabric connection is pass configuration information to the MySQLFabricConnection Class to create a connection with a Fabric node This is similar to using the 258 Providing Information to Ch
248. ecute the desired utilities in the tests Generally MUT is run with a root user 211 212 Chapter 8 MySQL Fabric Table of Contents 8 1 Introduction to Fabite secs cece scvecccevesseecc vans cacetastence savseece RERNA ENA seed cece RANE CENE EEKE ENAERE ENNEA RE SANER RETENE EERE 214 8 1 1 Fabric Prerequisites osnan n E EERE AE 214 8 1 2 Fabric CONCEP S sreninir cde aetesiernad ree senntctesenne iaa i naaa Ea EN O Naaa REA 215 8 2 Installing and Configuring MySQL Fabric cc ccceceeeeeee cece ee eeee cece esse eae eeteeeeeeeeeaeaaeaneeeeeeeeeaeaaea 215 8 2 1 Downloading MySQL Fabric 0 cccececeeeeeeeeeeeee sete ee aaa ee eeeeeeee ee aaeaeeeeeeeeeeeaaaaeeeeeeeeeeeeaea 215 8 2 2 Installing MySQL Fabric sccswensiiieeeieleds ide seneedadeindeadlenieddans ded nui aleadiealenaens 216 8 2 3 Configuring MySQL Fabric cccceeeceeeeeeeee cece ener eee ete teee sees ee aaaaeeeeeeeeeeeeaeaaeaneeeeeeeeeaeaaed 216 8 2 4 Starting and Stopping MySQL Fabric Nodes 0 ccccecceeeeeeeeee ee eeeneeeeeeeeeeeeaeaaeeeeeeeeeeeeaea 226 8 2 5 Old Configuration System cccceceeeeceeeeeeeee tees ee aa ee eeeeeeeeeeeeaaeaeeeeeeeeeeeaaaaaeeeeeeeeeeeaeaaeaees 227 8 3 QUICK Stafar TE a E lied etc EA E E abet ieee a lai 229 8 3 1 Example Fabric and Replication ccceceeccceeeeeecneeeeeeaaeeeeeeaaeeeeeeaaeeeeeeaaeeeeeaaeeeeeeaaeeees 229 8 3 2 Example Fabric and Sharding cccceceeeeeeeeee
249. ed at the start of the copy and the CHANGE MASTER followed by the START SLAVE statements after the copy To include the replication user in the CHANGE MASTER statement use the rpl user option to specify the user and password If this option is omitted the utility attempts to identify the replication user In the event that there are multiple candidates or the user requires a password the utility aborts with an error 88 OPTIONS If you attempt to copy databases on a server with GTIDs enabled GTID_MODE ON a warning will be generated if the copy does not include all databases This is because the GTID statements generated include the GTIDs for all databases and not only those databases in the export The utility will also generate a warning if you copy databases on a GTID enabled server but use the skip gtid option To make the most use of GTIDs you should copy all of the databases on the server with the a11 option OPTIONS mysqldbcopy accepts the following command line options help Display a help message and exit license Display license information and exit character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client default storage engine lt def_engine gt The engine to use for tables if the destination server does not support the original storage engine on the source server destination lt destination gt Connection
250. ed in Fabric A high availability group or simply group is a collection of servers It is used to associate the servers in a set This association may be a set of replication enabled servers the servers participating in a sharding solution and so forth A group identifier is the name we give a group or members of the group A group identifier is a name that matches the regular expression a zA Z0 9_ Examples of legal identifiers are my_group employees and shardl A global group stores all updates that must be propogated to all shards that are part of a sharding scheme A node or fabric node is an instance of the Fabric system running on a server To use the features of Fabric at least one Fabric node must be running Sharding refers to the Fabric feature that permits the distribution of data across several servers There are many uses of sharding but the most effective use of sharding enables distributing the work of writing across several servers for improved write speeds A shard is a horizontal partition or segment of data in a table Primary refers to a member of a group that is designated as master in the sense that it can accept read write transactions Secondary refers to a member of a group that can be a candidate to replace the master during switchover or failover and that can accept read only transactions 8 2 Installing and Configuring MySQL Fabric To use MySQL Fabric you must have a set of MySQL server instan
251. eeeeeeeeeeaeaaeeeeeeeeeeeeseaaaaneeeeeeeseaeaaeaeeeees 233 8 4 The mysglfabrre Utility nex ctecevcaccivessves sees cnceatesaytaewsecndatesvyus aan a ai aie a i da aaah a Riean 238 8 4 1 Getting Help imomni aa a a E E eae 238 8 4 2 Group Commands sss srivesiskuna inondik ninne aaa aae Na a A aaia ao aan a aiaa 239 8 4 3 Manage Commands sorascorimiariiienidina diania n a A aaa Te Eaa 239 8 4 4 Server Commands ecc cece ceceee cece cent ee ee eet tenet eee eater eee eee ee acted anes ee ae aaeneeeeeeeeeaeaaea 239 8 4 5 Sharding Commands i see sided Gofecioinadetinie eed ieddedeabeiedeadielaenedisceiaseldedecieiedes ielanieciess 240 814 6 Event Commands succoiecsaedex sctcianedsnavexcesnccausdaneleansietiensasanecensssudindt dadadea i EERE 240 8 4 7 D Mp COMMANA Sorsien anan inana an res ao aa o Ea EE AAA ATANN E 240 8 5 Backing Stole es saxiecdececetwsanssenctiechdatysauapegchechichslacaseaatechsetncasapeectsahsetadaeadeaca shdsthysadanedcbashivnelanadeces 241 8 5 1 Backing Store Tables siririn niiina airaa id a dee 241 8 5 2 Protecting the Backing Store 0 ccccceceeeeeeeeeeeeeeeee ee aeeaeeeeeeeeeeeeaaeaeeeeeeeeeeeeaaaaeeeeeseeeseeaea 244 8 6 Using MySQL Fabric with Pacemaker and Corosync cccceeeeeeeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaeaees 245 8 6 1 IMtPOGUCHON ics fects ssy ceetaian envia k ecu ab en ENE ocubatagencedseceneudasenousdanceut thas eagtaasnccueusenchguaayeceuees 245 8 6 2 Pr6 reQuiSiteS x ccssse2eelivv
252. ege is only required when working with MySQL 5 7 and above MySQL Fabric does not check for this privilege on earlier versions The Restore User If you want to use sharding or clone a server with the intention to add it to a High Availability HA group then you must define restore and backup users Like the server user these users must be created on all managed servers The restore account is defined under the servers using restore_user for the user name and restore_password as the password The restore account on the managed MySQL servers requires the following privileges in global scope if mysqldump is used ALTER ALTER DATABASE ALTER ROUTINE ALTER PROCEDURE FUNCTION CREATE CREATE TABLE CREATE ROUTINE CREATE PROCEDURE FUNCTION CREATE TABLESPACE CREATE TABLESPACE CREATE VIEW CREATE VIEW DROP DROP TABLE used before CREATE TABLE EVENT DROP CREATE EVENT INSERT write data LOCK TABLES LOCK TABLES single transaction REFERENCES Create tables with foreign keys SELECT LOCK TABLES single transaction SUPER SET SESSION SQL_LOG_BIN 0 TRIGGER CREATE TRIGGER Note R Although the CREATE TABLESPACE and REFERENCES privileges are only required when working with MySQL 5 7 and above MySQL Fabric still checks for them to help simplify the upgrade process to MySQL 5 7 Example statements to create the restore user to be executed on each managed MySQL ser
253. egular expression for the search above would be pattern cost basic regex 31 How Can I Run a Process Every Night To Kill Certain Connections 3 3 7 How Can I Run a Process Every Night To Kill Certain Connections Some database administrators use nightly routines to perform maintenance on their databases or servers Sometimes these routines can be blocked by long running queries or applications that hang onto locks for longer than expected Naturally priority is given to the application and maintenance routines are often cancelled rather than interfere with an application Should it happen that you subscribe to this notion and you have a routine that is still being blocked or for some reason hasn t completed by a certain time you need a quick way to generate an event to kill the connection involved This is where the mysqlprocgrep utility can help Objectives The objective is to generate an event that will kill all connections based on a user login msaladin but only if that connection is trying to run a custom administration script named my_admin_thingy Example Execution shell gt mysqlprocgrep sql body match command my_admin_thingy match user msaladin kill connection DECLARE kill_done INT DECLARE kill_cursor CURSOR FOR SELECT Id User Host Db Command Time State Info FROM INFORMATION_SCHEMA PROCESSLIST WHERE COMMAND LIKE my_admin_thingy AND USER LIKE msaladins OPEN kill cu
254. emp_no first_name last_name from employees where emp_no core ipt sh We ah lt lt Abe shears if we need to specify the shard key before accessing the data connection setShardKey ids i toString ps setint 1 idaili ResultSet rs ps executeQuery rs next Systemsoub formati sid 2 30s S s0sen rse getint 1 re getString 2 re getString 3 rs close ps close 4 Connect to the global group and clean up connection setServerGroupName fabric_testl_global statement executeUpdate drop table if exists employees statement close connection close 8 8 8 How Connector J Chooses a MySQL Server Before a database server can be chosen a server group must be chosen The following values are taken into account e server group name If a server group name is specified directly it is used used The server group name can be accessed by con getCurrentServerGroup getName Once a server group is chosen an individual database server is chosen based on the read only state of the connection A read only server is chosen if one is available Otherwise a read write server is chosen The server weight is not currently taken into account None of these values can be changed while a transaction is in progress 271 Using Hibernate with MySQL Fabric 8 8 9 Using Hibernate with MySQL Fabric It is possible to use Hibernate 4 s multi tenancy support to work with a set
255. er slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt server2 lt source gt Connection information for the second server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt show reverse Produce a transformat
256. er is complete the interface returns to monitoring replication health after 5 seconds The following shows an example of failover occurring Failover starting Candidate slave localhost 3332 will become the new master Preparing candidate for failover Creating replication user if it does not exist Stopping slaves Performing STOP on all slaves Switching slaves to new master Starting slaves Performing START on all slaves Checking slaves for errors Failover complete Discovering slaves for master at localhost 3332 Failover console will restart in 5 seconds After the failover event the new topology is shown in the replication health report MySQL Replication Monitor and Failover Utility Failover Mode auto Next Interval Mon Mar 19 16 05 12 2012 Master Information Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB iyo lsoain OOWOOiL wWibiy GTID Executed Set ZAG DEOOS2Z DAIS IMEZ Ay it OO G42 BE ORS 7 ile UUIDs 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 4 4 Misesa lhos i2 MASTER UP ON IOK Docalhose 3333 SLAVE UP ON OK licocallhosta ssss4 SLAVE UP ON OK 4 4 4 4 4 4 Q quit R refresh H health G GTID Lists U UUIDs L log entries Pressing L with
257. erations These utilities are those designed to work at the database level They include utilities that can used to adminster databases on one or more servers mysqldbcompare Compare databases on two servers or the same server Compare definitions and data e Generate a difference report e Generate SQL transformation statements mysqldbcopy Copy databases between servers e Clone databases on the same server e Supports rename e mysqldbexport e Export metadata and or data from one or more databases e Formats SQL CSV TAB Grid Vertical mysqldbimport e Import metadata and data from one or more files e Reads all formats from mysqldbexport mysqldiff Compare object definitions e Generate a difference report 61 General Operations 4 2 General Operations These utilities are those designed to perform general operations such as reporting and searching mysqldiskusage e Show disk usage for databases e Generate reports in SQL CSV TAB Grid Vertical mysgqlfrm e Reads frm files optionally in byte by byte diagnostic mode e Generates CREATE statements from table definition data mysqlindexcheck e Read indexes for one or more tables e Check for redundant and duplicate indexes e Generate reports in SQL CSV TAB Grid Vertical mysqlmetagrep e Search metadata e Regexp database search e Generate SQL statement for search query mysqlprocgrep Search process information e Generate SQL statement for
258. ered mysqli rplshow master lt user gt lt password gt localhost 13001 There was an error here We can see the error with the show errors command We see in the error we failed to provide any connection information for the slaves Once we correct that omission the last command shows how the users console executes a utility and displays the results in the same stream as the console much like the mysq1 client command line tool Permissions Required There are no special permissions required to run mysqluc however you must have the necessary privileges to execute the desired utilities 3 3 5 I ve Got Too Many Indexes How Do I Know Which Ones to Drop MySQL allows its users to create several indexes that might be the same duplicate indexes or partially similar redundant indexes in its structure Although duplicate indexes have no advantages there are some cases where redundant indexes might be helpful However both have disadvantages Duplicate and redundant indexes slow down update and insert operations As a result it is usually a good idea to find and remove them 28 I ve Got Too Many Indexes How Do Know Which Ones to Drop Doing this manually would be a time consuming task especially for big databases and that is why there is a utility to automate this type of task mysqlindexcheck Objectives Our goal is to use the mysql indexcheck utility to help us find duplicate and redundant indexes For that we are go
259. erence found Process all objects server1 lt source gt Connection information for the first server 82 OPTIONS To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt server2 lt source gt Connection information for the second server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from yo
260. erface void clearServerSelectionCriteria Clear all the state that is used to determine which server to send queries to void setShardKey String shardKey throws SQLException Set the shard key for the data being accessed String getShardKey Get the shard key for the data being accessed void setShardTable String shardTable throws SQLException Set the table being accessed Can be a table name or a database and table name pair in the form db_name tb1_name The table must be known by Fabric as a sharded table String getShardTable Get the name of the table being accessed void setServerGroupName String serverGroupName throws SQLException Set the server group name to connect to Direct server group selection is mutually exclusive of sharded data access String getServerGroupName Get the server group name when using direct server group selection ServerGroup getCurrentServerGroup Get the current server group if sufficient server group selection has been provided Otherwise null void clearQueryTables throws SQLException Clear the list of tables for the last query This also clears the shard mapping table and must be given again for the next query via setShardTable or addQueryTable void addQueryTable String tableName throws SQLException Specify that the given table is intended to be used in the next query Set lt String gt getQueryTables Get the list of tables intended to be used in the next quer
261. errant transactions and will issue a warning or error before executing some of their respective operations This is done because this kind of transaction can lead to an unstable replication topology after a failover or switchover 46 How To Fix Errant Transactions on the Replication Topology What are errant transactions Errant transactions are transactions directly applied by a client on a slave and that do not exist on all the slaves connected to the master By nature these transactions should not be replicated and can lead to replication errors if the slave that possesses them is promoted to master In practice this can happen for example if the errant transaction corresponds to a data insert or delete on a table that only exists on that slave This kind of transactions usually result from a mistake or poor practice with data being changed directly on the slave without turning off logging to the binary log The best way to deal with errant transaction is to avoid them making sure that every transaction ona slave even if needed for example to add data for reporting or execute local administrative commands must be applied with binary logging disabled See SET sql_log_bin Syntax for more information about how to control logging to the binary log However in case errant transaction are found we still need to be able to deal with them in a easy and quick way skipping those transactions and avoiding them from being replicated if the slave b
262. erresa h hdeselltim CCID bistre UWS Llog eme ies Discussion The above example illustrates how to start the mysql failover utility to check the health of the replication topology and shows the displayed output when failover occurs Basically we simply need to specify the master s connection with the master option the list of slaves with the slaves option and the replication user login and password using the rpl user options In alternative to the slaves options one can use the discover slaves login specifying a user and password or login path to connect to the slaves and the utility will attempt to discover all the slaves connected with the master using the specified login For the above example discover slaves login root could be used The discover slaves login can be very handy especially if there is a huge number of slaves in the topology but bear in mind that the explicit specification of slaves is safer and that discovery can fail to find some servers In particular it is important to note that in order for slaves to be discovered they must be started with the report host and report port options with a appropriate values and they must be correctly connected to the master I O thread running otherwise discovery will fail It is also recommended to use the 10g options to specify a file to register all events warning and errors This is useful to keep a record of what happened for example to later d
263. ers on the source no destination is needed To control how to display list output use one of the following values with the format option e grid default Display output in grid or table format like that of the mysql1 client command line tool e CSV Display output in comma separated values format e tab Display output in tab separated format e vertical Display output in single column format like that of the G command for the mysqli client command line tool 191 OPTIONS OPTIONS mysqluserclone accepts the following command line options help Display a help message and exit license Display license information and exit destination lt destination gt Connection information for the destination server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the co
264. ersion 2 6 or higher but does not support Python 3 Single or double quote characters respectively or can be used around option values In fact quotes are required to set some options values correctly such as values with whitespace For example to specify the event types Create DB and Drop DB for the event type option the following syntax must be used vent type Create DB Drop DB or vent type Create DB Drop DB EXAMPLES To display the audit log file statistics and output the results in CSV format run the following command shell gt mysqlauditgrep file stats format CSV SERVER data audit log Audit Log File Statistics File Size Created Last Modified 75 EXAMPLES aumchic leg SlOl tau Seo 27 Isssseiil 2Oil2 Wow Cte il W sstOsss Zoe Audit Log Startup Entries SERVER_ID STARTUP_OPTIONS NAME TIMESTAMP MYSQL_VERSION OS_VERSION VERSION 1 SERVER sql mysqld defaults file SERVER my cnft Audit 2012 09 27713 33 11 5 5 29 log x86_64 Linux 1 To display the audit log entries of specific users use the following command shell gt mysqlauditgrep users testerl tester2 SERVER data audit log To display the audit log file statistics run the following command shell gt mysqlauditgrep users testerl tester2 SERVER data audit log jG Se IG ce IG ce IG ce Ge ee ee O 1046 1046 To display the audit log
265. erver for security purposes then only the login events need to be recorded The mysqlauditadmin utility allows us to perform such change in a simple way as well as changes other settings 55 How Do Record Only Login Events Objectives The goal is to set the audit log plugin to only write the login events to the log file It is assumed that the audit log plugin is enabled and running with the default settings logging all auditable events on the localhost and default port 3306 Example Execution shell gt mysqlauditadmin server root localhost 3306 policy value LOGINS show options Showing options before command Audit Log Variables and Options 4 4 Variable_name Value 4 4 audit_log_buffer_size 1048576 audit_log_file audit log audit_log_flush OFF aweh log policy ALL audit_log_rotate_on_size 0 audit_log_strategy ASYNCHRONOUS 4 4 Executing POLICY command Showing options after command Audit Log Variables and Options 4 4 Variable_name Value 4 4 audit_log_buffer_size 1048576 audit_log_file audit log audit_log_flush OFF avci hee galie LOGINS audit_log_rotate_on_size 0 audit_log_strategy ASYNCHRONOUS 4 4
266. erver2 3312 will not be demoted and set as a slave of the new master server 3311 The mysqlrpladmin utility will execute and display information about all required action to perform switchover After completing the switchover process an health report is displayed where it is possible to confirm the successful execution of the command and verify that the topology has changed as expected After completing these simple steps the replication topology is back to its initial structure before failover with its old master Therefore mysqli failover is ready to be executed again to monitor the topology and enable automatic failover Note that in this particular situation the use of the force option might be required because it is likely that some registration data from the previous failover instance execution might have be left on the recovered master due to its previous crash The mysqifailover utility registers its execution on the servers in order to avoid concurrent executions of the utility which will likely lead to errors and inconsistent state during failover If the utility detects that another instance might be running it will be started in fail mode not taking any action when it detects that the master failed The mysql failover instance registration is cleared when the utility exits and it is expected that unregistration fails on crashed or not accessible servers The force option overwrite the instance execution check al
267. erverl Data differences found among rows UPDATE cbi tl SkT b Test 123 WHERE a 1 UPDATE db1 tl SET b Test 789 WHERE a 3 DELETE FROM dbl tl WHERE a 4 NSERT INTO dbl tl a b VALUES 5 New row db2 Database consistency check failed Tone Similarly when the same command is run with changes for server2 and difftype sq1 the following report is generated shell gt mysqldbcompare serverl root root localhost server2 root root localhost db1 db2 changes for server2 a difftype sql Defn Row Data Type Object Name DHE Count Check a a Fe a O A EON TABLE el pass pass Compare table checksum FAIL 86 PERMISSIONS REQUIRED Find row differences FAIL Transformation for changes for server2 Data differences found among rows UPDATE de2 tl SET b Test 789 WHERE a Ti UPDATE db2 tl SET b Test 1237 WHERE a 3 DELETE FROM db2 t1 WHERE a 5 NSERT INTO db2 t1 a D VALUES 4 New row dbl Database consistency check failed done With the difftype sq1 SQL generation option set show reverse shows the object transformations in both directions Here is an excerpt of the results shell gt mysqldbcompare serverl root root localhost server2 root root localhost db1 db2 changes for serverl1 show reverse a difftype sql TABLE ieee pass pass Compare table checksum
268. es in each release see the MySQL Utilities Release Notes This is the MySQL Fabric 1 5 manual If you require a different version such as 1 6 then select that version s documentation from the Fabric Doc Index 8 1 Introduction to Fabric To take advantage of Fabric an application requires an augmented version of a MySQL connector which accesses Fabric using the XML RPC protocol Currently Connector Python and Connector J are fabric aware Fabric manages sets of MySQL Servers that have Global Transaction Identifiers GTIDs enabled to check and maintain consistency among servers Sets of servers are called high availability groups Information about all of the servers and groups is managed by a separate MySQL instance which cannot be a member of the Fabric high availability groups This server instance is called the backing store Fabric organizes servers in groups called high availability groups for managing different shards or simply for providing high availability For example if standard asynchronous replication is in use Fabric may be configured to automatically monitor the status of servers in a group If the current master in a group fails it elects a new one if a server in the group can become a master Besides the high availability operations such as failover and switchover Fabric also permits shard operations such as shard creation and removal Fabric is written in Python and includes a special library that implements all of
269. es sacadeen O EO 284 92 1 Doctrine DBA 2 3 24 pices sues vonseec an an aa aaa a ea E a Aa a aa aa Aea aa EAA AA 284 922 PYythOmiLICONSe aer T E RT 285 9 2 3 Python License Supplement Windows Only eceeeeeee eee eeeeeeeeeeeeaeaaeaneeeeeeeeeaeaaed 295 This chapter includes additional information about MySQL Utilities including a list of frequently asked questions and third party license information 9 1 MySQL Utilities Frequently Asked Questions FAQ Categories General Questions 283 Storage Engine Questions 283 The mysqlfrm Utility frm File Reader Questions 283 General 9 1 1 Are these utilities present in the community version of MySQL 0 ccceceeeeeeeeeeeeeeeaeeeeeeeeeeeees 283 9 1 1 Are these utilities present in the community version of MySQL Yes see Chapter 1 How to Install MySQL Utilities Storage Engines 9 1 1 Can the utilities be used with MyISAM or CSV ccecccccseeeececeeeeeeeceeeeeeeseeeeseeseeeeseeseaneseeaaees 283 9 1 1 Can the utilities be used with MyISAM or CSV Yes There are no storage engine specific limitations in using the utilities There are some features written specifically for InnoDB so those may not apply but in general no utility is storage engine specific For example the mysqidiskusage utility shows exact sizes for MyISAM and InnoDB files but uses estimated sizes for any other storage engine based on number of rows and row size The mysqlfrm Utility frm File Rea
270. et options are being used Add difftype option Add engine default storage engine options Check whether storage engines listed in options exist Parse connection values Establish replication connection between a master and a slave Return list of replication test function pointers Get connection dictionary Check whether any servers are running on tire jlocal host Connect to source and destination server Connect to running MySQL server and perform server level operations Encapsulate index for a given table as defined by SHOW INDEXES Encapsulate table for given database to perform table level operations Search for MySQL tool and return its 196 General Interface Specifications and Code Practices EL oeei delete_directory Remove directory folder and contents user parse_user_host Parse user passwd host port from user passwd host User Clone user and its grants to another user and perform user level operations General Interface Specifications and Code Practices The MySQL Utilities are designed and coded using mainstream coding practices and techniques common to the Python community Effort has been made to adhere to the most widely accepted specifications and techniques This includes limiting the choice of libraries used to the default libraries found in the Python distributions This ensures easier installation enhanced portability and fewer problems with missing libraries Similarly external librar
271. etermine when failover occurred and if the process occurred without any errors An important matter to discuss is the order in which the server are select as candidates for failover No distinction is made in terms of the number of transaction to select the most up to date slave to become the new master The reason is very simple this criteria is non deterministic as many circumstances i e network load server maintenance operations can temporarily influence the performance of a slave and could lead to an incorrect selection of the most appropriate candidate For example the slave with the best hardware should be in the long run the most appropriate candidate to become the new master but 39 Restore the Previous Master After Failover for some unanticipated reason it might actually had less transactions than other servers when the master crashed Therefore a more deterministic criteria based on the order in which the servers are specified is used allowing the user to control the order in which the candidates are selected The first server that will meet the required election criteria consisting on simple sanity checks server reachable and running with the required options GTID ON and binary logging enabled will be chosen More specifically the section of the new master will follow this order first sequentially check the list of servers specified by the candidates option then the servers listed in the s1laves option finally
272. event log that is used to present statistics Number of issues before the server is considered unstable Number of different sources that should report issues on a server before it is considered unstable Amount of time in seconds that is used when deciding if a server is unstable Issues older than this are not considered when deciding Minimum time in seconds between subsequent automatic promotions This parameter is used to prevent the system entering a sequence of promotions that could disable the system Number of successive failed attempts to contact the server after which the built in failure detector considers the server unstable Timeout in seconds used when contacting the server If the server does not respond within this time period it is recorded as a failed attempt to contact the server Maximum age in seconds for reported issues in the error log Issues older than this are removed from the error log This section contains parameters for the failure management system notifications notification_clients notification_interval failover_interval detections The notification threshold If more than this number of notifications arrive in the notification interval and the number of distinct notification clients are over the notification client threshold the server is considered dead and failover is triggered The number of distinct notification clients that need to report If more than this number of distinct not
273. example Ge Gh Gh 06 Test Basic database example 07 Test Basic database example Data for table util _test 2 NSERT INTO util_test t2 VALUES 11 Test Basic database example 12 Test Basic database example 13 Test Basic database example Date for table util test 22 NSERT INTO util_test t3 VALUES il ald test fkeys y ZS test tikes a 16 test fkeys Data for table util_test t4 NSERT INTO util_test t4 VALUES Sp 2 z w D 100 EXAMPLES If the database to be exported does not contain only InnoDB tables and you want to ensure data integrity of the exported data by locking the tables during the read step add a 1ocking lock al11 option to the command shell gt mysqldbexport server root pass localhost export DATA bulk insert util_test locking lock all Source on localhost connected USE util_test Exporting data from util_test Data for table util_test tl INSERT INTO util_test t1l VALUES 01 Test Basic database example 02 Test Basic database example 03 Test Basic database example 04 Test Basic database example 05 Test Basic database example 06 Test Basic database example 07 Test Basic database example Data for table util_test t2 NSERT INTO util_test t2 VALUES 11 Test Basic database example 12 Test Basic database example 13 Test Basic database example Date for table util_tes
274. example doner Connecting to the current server as master CHANGE MASTER TO MASTER_HOST localhost 101 PERMISSIONS REQUIRED MASTER_USER rpl MASTER_PASSWORD rpl MASTER PORT 3311 MASTER_LOG_FILE clone bin 000001 MASTER_LOG_POS 106 Starting slave START SLAVE Similarly to export a database and include the replication commands to use the current server s master for example to start a new slave using the same the master use the following command shell gt mysqldbexport server root localhost 3311 util_test export both rpl user rpl rpl rpl slave v spource on localhost coo connected Stopping slave STOP SLAVE Source on localhost connected Exporting metadata from util_test DROP DATABASE IF EXISTS util_test CREATE DATABASE util_test USE util_test TABLES UEI beest el CREATE TABLE CI la char 30 DEFAULT NULL ENGINE MEMORY DEFAULT CHARSET latinl done Source on localhost connected USE util_test Exporting data from util_test Data for table util test ci NSERT INTO util_test t1 VALUES 01 Test Basic database example NSERT INTO util_test t1 VALUES 02 Test Basic database example NSERT INTO util_test t1 VALUES 03 Test Basic database example NSERT INTO util_test t1 VALUES 04 Test Basic database example Eia r UN a r r NSERT INTO util_test t1l1 VALUES 05 Test Basic database exa
275. f there are more than one command that matches the prefix and the user presses TAB twice a list of all possible matches is displayed Tab completion for options is similar The user must first type a valid MySQL Utility command then types the first N characters of a command and presses TAB for example verb lt TAB gt In this case the console will complete the option For the cases where an option requires a value the console will complete the option name and append the character Tab completion for options works for both the full name and the alias if available If the user presses TAB twice the console will display a list of matching options Pressing TAB twice immediately after typing the name of a MySQL Utility will display a list of all options for that utility Tab completion for variables works the same as that for options In this case the user must first type the character then press TAB For example if a variable SERVER1 exists when the user types server SER lt TABs the console will complete the SERVER variable name For cases where there are multiple variables pressing TAB twice will display a list of all matches to the first N characters Pressing TAB twice after typing only the character will display a list of all variables Note R The mysq prefix is optional in the console For example typing disku lt TAB gt in the console will complete the command as diskusage Executing utilities is accom
276. faults tools which is required to read the login path values from the login configuration file mylogin cnf If any database identifier specified as an argument contains special characters or is a reserved word then it must be appropriately quoted with backticks In turn names quoted with backticks 84 EXAMPLES must also be quoted with single or double quotes depending on the operating system i e in Windows or in non Windows systems in order for the utilities to read backtick quoted identifiers as a single argument For example to compare a database with the name weird db name with other weird db name the database pair must be specified using the following syntax in non Windows weird db name other weird db name EXAMPLES Use the following command to compare the emp1 and emp2 databases on the local server and run all tests even if earlier tests fail shell gt mysqldbcompare serverl root localhost empl emp2 run all tests serverl on localhost connected Checking databases empl on serverl and emp2 on server2 WARNING Objects in server2 emp2 but not in serverl emp1 TRIGGER CEG PROCEDURE p1 TABLE t1 VIEW v1 Defn Row Data Type Object Name DREE Coun Check FUNCTION 1 pass TABLE departments pass pass Compare table checksum FAIL Find row differences FAIL Data differences found among rows empl departments emp2 departments EE 1 4 1 4 ee KKEKE
277. figures where they can be found mysqldump_program Path to the mysqldump program mysalclient_program Path to the mysql program Section statistics Connectors and other external entities log any errors while accessing servers so that MySQL Fabric can monitor server health and act accordingly For example MySQL Fabric promotes a new master after receiving notifications from the number of clients configured in notification_clients within the time interval configured in not ification_interval Ifa server is considered unstable but it is not a master it is simply marked as faulty To avoid making the system unstable a new master can only be 224 Configuring MySQL Fabric automatically promoted after the fai Llover_interval has been elapsed since the last promotion In order to ease the adoption of MySQL Fabric a built in failure detector is provided If the failure detector is enabled to monitor a group a new master is promoted after 3 failed successive attempts to access the current master within the time interval configured in failover_interval The failure detection routine tries to connect to servers in a group and uses the value configured in detection_timeout as timeout prune_time notifications notification_ clients notification_interval failover_interval detections detection_ timeout prune_time Section failure_tracking How often the internal event log is pruned in seconds and also the age of events in the
278. file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required e stats Show index performance statistics e verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit e worst lt N gt If stats is also passed in limit index statistics to the worst N indexes The default value of Nis 5 if omitted NOTES You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to read all objects accessed during the operation For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To check all tables i
279. formation for the server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt 115 NOTES e ssl ca The path to a file that contains a list of trusted SSL CAs e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example
280. fort CREATE statement e new storage engine lt engine gt Set the ENGINE option for all frm files read e port lt port gt The port to use for the spawned server in the default mode Must be a free port Required for default mode e Server lt server gt Connection information for a server Use this option or basedir for the default mode If provided with the diagnostic mode the storage engine and character set information will be validated against this server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt e ssl ca 129 NOTES The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate fi
281. from bootstrap onward Look for errors in these statements as to why the spawned server will not start Risks The utility performs a best effort approximation of the CREATE statement when run in diagnostic mode As such if you read a frm file that uses character sets or collations other than the default and you do not use a server option to connect to a server to read the character sets this can result in miscalculated column sizes For example suppose your default character set is latini which uses 1 byte per character Lets also suppose you are attempting to read a frm file that uses a character set that uses 3 bytes per character Furthermore we have no server to connect In this case the column sizes may be off by a factor of 3 A case in point would be a field such as col_a char 3 would appear in the output of the mysqli frm utility as col_a char 9 To mitigate risks such as this and to produce the most accurate CREATE statement in diagnostic mode always use the server option 3 3 3 Creating a New User With The Same Privileges as Another User The MySQL privilege system permits you to create a set of permissions for each user Sometimes the set of permissions are complex and may require multiple GRANT statements to effect Other times the user may acquire privileges over time Regardless of how it came about you may find yourself needing to create a new user that has the same privileges as another user Objectives
282. g add table MAPPING_ID employees employees emp_no e Create the shard index 265 Running Tests shell gt mysqlfabric sharding add_shard SMAPPING_ID fabric _test1_shard1 0 fabric_test1_shard2 10000 state ENABLED 8 8 5 Running Tests The test fabric target in the build xml file runs JUnit tests on these servers It requires only the setup described in Section 8 8 4 MySQL Fabric Configuration for Running Samples All necessary tables and data are created during test run The parameters for the servers must be provided to Ant to verify that the correct information is received from the Fabric node This includes server host names and ports This data can be provided on the command line with D arguments to Ant or in a build properties file This file should be placed in the root of the source directory where build xml is located Based on the information given so far this file would contain the following entries com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql com mysql TEIDE IOR TEIDE IGS TEIDE SFIDE PEIDER TEDER sageis Ne TEID LOR pieeleeie 5 TEADE srela es srela Nee SEIDEN testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite testsuite hostname localhost port 32274 fabricUsername admin fabricPassword secret username
283. generate the routine declaration If only_body is False a complete procedure will be generated if there is any kill action supplied and just a select statement if it is a plain search Parameters only_body boolean Show only the body of the procedure If this is False a complete procedure is returned Returns SQL code for executing the operation specified by the options Return type string execute connections output sys stdout connector mysql connector Execute the search on each of the connections supplied If oufout is not None the value is treated as a file object and the result of the execution is printed on that stream Note that the output and connector arguments must be supplied as keyword arguments All other arguments are treated as connection specifiers Parameters connections Sequence of connection specifiers to send the search to output File object to use for writing the result connector Connector to use for connecting to the servers 6 4 Parsers 6 4 1 mysql utilities parser Parse MySQL Log Files This module provides classes for parsing MySQL log files Currently Slow Query Log and General Query Log are supported 206 mysq utilities parser Parse MySQL Log Files class mysql utilities parser GeneralQueryLog stream This class parses the MySQL General Query Log Instances are iterable but the class does not provide multiple independent iterators Fo
284. ges such as su or sudo to execute the frm reader You must use the user option to specify a user to launch the spawned server however This will permit the frm reader to read the original frm file and copy it to the spawned server and access the copy without requiring additional privileges 9 1 8 Will the default mode display a 100 accurate CREATE statement For most tables and all views yes However there are at least two features that are not stored in the frm file and therefore will not be included These are autoincrement values and foreign keys That being said the CREATE statement produced will be syntactically correct 9 2 Third Party Licenses Use of any of this software is governed by the terms of the licenses that follow MySQL Utilities 1 5 e Section 9 2 1 Doctrine DBAL 2 3 4 Section 9 2 2 Python License e Section 9 2 3 Python License Supplement Windows Only 9 2 1 Doctrine DBAL 2 3 4 The following software may be included in this product Copyright c 2006 2012 Doctrine Project 284 Python License Permission is hereby granted free of charge to any person obtaining a copy of this software and associated documentation files the Software to deal in the Software without restriction including without limitation the rights to use copy modify merge publish distribute sublicense and or sell copies of the Software and to permit persons to whom the Software is furnishe
285. group can be started colocation c_mysql_on_drbd inf g_mysql ms_drbd_mysql Master order o_drbd_before_mysql inf ms_drbd_mysql promote g_mysql start In order to prevent a split brain scenario in the event of network partitioning Pacemaker can ping independent network resources Such as a network router and then prevent the host from being the DRBD master in the event that it becomes isolated primitive p_ping ocf pacemaker ping params name ping multiplier 1000 Host lh abet t ale 6s mIRE 254 Key administrative tasks op monitor interval 15s timeout 60s start timeout 60s clone cl_ping p_ping meta interleave true location 1_drbd_master_on_ping ms_drbd_mysql rule role Master inf not_defined ping or ping number lte 0 Check if everything is running fine using the following command root host1 crm mon one shot V Ensure the correct daemons are started at system boot At this point a reliable MySQL service is in place but it is also important to check that the correct cluster services are started automatically as part of the servers system startup It is necessary for the Linux startup to start the Corosync and Pacemaker services but not DRBD or MySQL Process and MySQL Server as those services will be started on the correct server by Pacemaker To this end execute the following commands on each host root host1 sysv rc conf drbd off root host1 sysv rc conf corosync on root host1 sysv rc conf mysq
286. gtid_executed None cur execute SELECT global gtid_executed Oe ew alia CWIES gtid_executed row 0 return gtid_executed def synchronize cur gtid_executed Guarantee that a slave has created the employees employees table before reading anything cur execute SELECT WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS s 0 gtid_executed Guns Site chicane def add_employee conn emp_no first_name last_name gtid_executed conn set_property tables employees employees key emp_no mode fabric MODE_READWRITE cur conn cursor synchronize cur gtid_executed cur execute USE employees cur execute INSERT INTO employees VALUES s sS S emp_no first_name last_name def find_employee conn emp_no gtid_executed conn set_property tables employees employees key emp_no mode fabric MODE_READONLY cur conn cursor synchronize cur gtid_executed cur execute USE employees Or Wey alia CMIES joie eel eC Gynec ie esemeacicacime cur execute SELECT first_name last_name FROM employees WHERE emp_no s emp_no for row in cur print row def pick_shard_key shard random randint 0 2 shard_range shard 100000 shard_range shard_range if shard 0 else shard_range 1 shift_within_shard random randint 0 99999 return shard_range shift_within_shard Address of the Fabric not the host we are going to connect to 236 Example Fabric and Sharding
287. guration file mylogin cnf 124 EXAMPLES The console creates a special table in the mysql database that is used to keep track of which instance is communicating with the master If you use the force option the console will remove the rows in this table The table is constructed with CREATE TABLE IF NOT EXISTS mysql failover_console host char 30 port char 10 When the console starts a row is inserted containing the hostname and port of the master On startup if a row matches these values the console will not start If you use the force option the row is deleted When running the utility using the daemon nodetach option the pidfile option can be omitted It will be ignored if used When using the external scripts the following parameters are passed in the order shown For example suppose you have a script named run_before sh and you specify that you want it executing before the failover is performed using the xec before option Further let us assume the master MySQL Server is using port 3306 on the host host1 and the MySQL Server that will become the new master is using port 3308 on host can_host2 The script would therefore be invoked in the following manner amp run_before sh hostl 3306 can_host2 3308 Table 5 2 External Script Parameters MySQL Failover Option Parameters Passed to External Script xec befor master host master port candidate host candidate port
288. hard unions or joins Not at present all queries are limited to the data within a single shard the Global Table data If data from multiple shards is required then it is currently the application s responsibility to collect and aggregate the data 8 9 12 ls the routing of queries and transactions transparent to my application Partially For HA the application simply needs to specify whether the operations are read only or involve writes or consistent reads For sharding the application must specify the sharding key a column from one or more tables but this is independent of the topology of the MySQL Servers and where the data is held and it is unaffected when data is moved from one server to another Consistency 8 9 1 What do do if need immediately consistent reads cccceeeeeeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaes 282 8 9 1 What do do if need immediately consistent reads Because replication from the Primary master to the Secondaries slaves is not synchronous you cannot guarantee that you will retrieve the most current data when reading from a secondary To force a read to be sent to the Primary the application may set the mode property for the connection to read write rather than read 282 Chapter 9 Appendix Table of Contents 9 1 MySQL Utilities Frequently Asked Questions cceceeeeeee ee eeeeeeeeeee esse aa ee eeeeeeeeeeeeaaaaeeneeeeeeeeeaaa 283 92 Imad Pant yVaLICONS OS i
289. harding COMMANAS siririna natai aa ea a a EEE a EE Aa A Eaa NE 240 8 4 6 Event COMMMANS siraremiormsaeima a a a a A ES 240 8 4 7 DUMP COMMANAS wascssesevedand egaccasennencsieseasicebaukeocait aa a a a aa EAA 240 8 5 BACKING Stole eai E E A E E A 241 8 5 1 Backing Store Tables vorsins noaea ena A EEE Ea R EAA a AN AEA 241 8 5 2 Protecting the Backing Store c ccceceeeeeeeeeeneeeeeeeeeeeeeeaaeaeeeeeeeeeeeaaaaeeneeeeeeeeeaeaaea 244 8 6 Using MySQL Fabric with Pacemaker and CoroSyn ccccceceeeeeeeeeeeeee ce eaeeeeeeeeeeeeseaaaaeeees 245 BG IMNOGUCHON saine r a a wiasecat eanvencisadwnode dans fae awoaceasaveecnsadenece N 245 8 6 2 PLe FQuISILGS isissesdeiueFeedescayedenneevecnslaeddebandeebeandvedeat NAA KARRA atu SORAAN RASKAAN 245 8 6 3 Target Configuration ss sassari iea a aa AA TESA aaa 246 8 6 4 Setting up and testing your system sssesssessirsisrssssrrirtitrsstsnrrnrittustsnnrnnnnnasnennn 247 8 6 5 Key administrative tasks seersirrrioiannna nR NE EAEE AARS 255 8 7 Using Connector Python with MySQL Fabric cccceeeeeeeeeee sees ae eeeeteeeeeeeeeeeaaaaeeeeeeeeeeeeaea 256 8 7 1 Installing Connector Python with MySQL Fabric Support ccceceeeeeeeeeeeeeeeneeeees 257 8 7 2 Requesting a Fabric Connection 0 cccceceeeeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaaaeeeeeeeeeeeeaaa 257 8 7 3 Providing Information to Choose a MySQL Server ccccceeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaea 259 8 8 Usi
290. he InnoDB Plugin and InnoDB on both servers must have the same major and minor version numbers and enabled state 144 OPTIONS By default the utility issues warnings for mismatches between the sets of storage engines the default storage engine and the InnoDB storage engine To produce errors instead use the pedantic option which requires storage engines to be the same on the master and slave The vv option displays any discrepancies between the storage engines and InnoDB values with or without the pedantic option Replication can be started using one of the following strategies e Start from the current position default Start replication from the current master binary log file and position The utility uses the SHOW MASTER STATUS statement to retrieve this information Start from the beginning Start replication from the first event recorded in the master binary log To do this use the start from beginning option Start from a binary log file Start replication from the first event in a specific master binary log file To do this use the master log file option Start from a specific event Start replication from specific event coordinates specific binary log file and position To do this use the master log file and master log pos options OPTIONS mysqlreplicate accepts the following command line options e help Display a help message and exit e license Display license info
291. he following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt report values lt report_values gt Report values used in mysaqlrplms It can be health gtid or uuid Multiple values can be used separated by commas e health Display the replication health of the topology e gtid Display the master s list of executed GTIDs contents of the GTID variables GLOBAL GTID_EXECUTED GLOBAL GTID_PURGED and GLOBAL GTID_OWNED e uuid 150 OPTIONS Display universally unique identifiers UUIDs for all servers Default health rpl user lt replication_user gt The user and password for the replication user in the format lt user gt lt passworad gt or lt login path gt slave lt slave gt Connection information for the slave server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your ch
292. hing_wonderful 1l obj_1l obj_2 opt_dictionary The documentation block for the preceding method is the style used throughout the library Example Now that you are familiar with the MySQL utilities and the supporting library modules let us take a look at an example that combines some of these modules to solve a problem Suppose that you want to develop a new database solution and need to use real world data and user accounts for testing The mysqlserverclone MySQL utility looks like a possibility but it makes only an 197 Example instance of a running server It does not copy data However mysqldbcopy makes a copy of the data and mysqluserclone clones the users You could run each of these utilities in sequence and that would work but we are lazy at heart and want something that not only copies everything but also finds it for us That is we want a one command solution The good news is that this is indeed possible and very easy to do Let us start by breaking the problem down into its smaller components In a nutshell we must perform these tasks e Connect to the original server e Find all of the databases e Find all of the users e Make a clone of the original server Copy all of the databases e Copy all of the users If you look at the utilities and the modules just listed you see that we have solutions and primitives for each of these operations So you need not even call the MySQL utilities directly although
293. hip or joint venture between PSF and Licensee This License Agreement does not grant permission to use PSF trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee or any third party 8 By copying installing or otherwise using Python Licensee agrees to be bound by the terms and conditions of this License Agreement BEOPEN COM LICENSE AGREEMENT FOR PYTHON 2 0 BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1 1 This LICENSE AGREEMENT is between BeOpen com BeOpen having an office at 160 Saratoga Avenue Santa Clara CA 95051 and the Individual or Organization Licensee accessing and otherwise using this software in source or binary form and its associated documentation the Software 2 Subject to the terms and conditions of this BeOpen Python License Agreement BeOpen hereby grants Licensee a non exclusive royalty free world wide license to reproduce analyze test perform and or display publicly prepare derivative works distribute and otherwise use the Software alone or in any derivative version provided however that the BeOpen Python License is retained in the Software alone or in any derivative version prepared by Licensee 3 BeOpen is making the Software available to Licensee on an AS IS basis BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES EXPRESS OR IMPLIED BY WAY OF EXAMPLE BUT NOT LIMITATION BEOPEN MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY
294. hon alone or in any derivative version provided however that PSF s License Agreement and PSF s notice of copyright i e Copyright c 2001 2002 2003 2004 2005 2006 Python Software Foundation All Rights Reserved are retained in Python alone or in any derivative version prepared by Licensee 3 In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof and wants to make the derivative work available to others as provided herein then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python 286 Python License 4 PSF is making Python available to Licensee on an AS IS basis PSF MAKES NO REPRESENTATIONS OR WARRANTIES EXPRESS OR IMPLIED BY WAY OF EXAMPLE BUT NOT LIMITATION PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT INFRINGE ANY THIRD PARTY RIGHTS 5 PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON FOR ANY INCIDENTAL SPECIAL OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING DISTRIBUTING OR OTHERWISE USING PYTHON OR ANY DERIVATIVE THEREOF EVEN IF ADVISED OF THE POSSIBILITY THEREOF 6 This License Agreement will automatically terminate upon a material breach of its terms and conditions 7 Nothing in this License Agreement shall be deemed to create any relationship of agency partners
295. host database joe localhost util_test ig Ol IkeceULnoysne Saldliv localhost 4E MESSE sam localhost util_test joe user util_test PERMISSIONS REQUIRED The permissions required include the ability to read the mysql database and to have read access to the data directory 194 Chapter 6 Extending MySQL Utilities Table of Contents 6 1 Introduction to extending the MySQL Utilities 2 2 0 eee cece eect ee ee ceed eeteeeeeeeeeaeaaaeneeeeeeeeeaeaaea 195 6 2 MySQL Utilities copy_server py sample cccceceeeceeeeeee cece tees aa ee eeeeeeeeeeeeaaaaeeeeeeeeeeeeaaaaenneeeeeeees 201 6 3 Specialized Operations cc23 2 0 savseiveee amiata a e aaaea aaa iaaea 204 6 3 1 mysql utilities command grep Search Databases for Objects cceeees 204 6 3 2 mysql utilities command proc Search Processes on Servers ccceeeceeeeeees 205 6 4 ParselS ninesi e ee aa a a Ea a ea aa a e a a e aa naaa i a ian 206 6 4 1 mysql utilities parser Parse MySQL Log Files 0 cc ccceceeeeeeeee eee eeeeeeeeeeeeeeaeaaeeeeeeeeeees 206 This chapter introduces the architecture for the MySQL Utilities library and demonstrates how to get started building your own utilities 6 1 Introduction to extending the MySQL Utilities Administration and maintenance on the MySQL server can at times be complicated Sometimes tasks require tedious or even repetitive operations that can be time consuming to typ
296. hread and write operations are scheduled and executed through the executor executors The number of executor threads that the executor uses when processing requests Section logging MySQL Fabric logs information about its activities to the standard output when started as a regular process However when started as a daemon it writes information to a file configured by the the option Fabric URL used for logging level The log level to use when generating the log Acceptable values are CRITICAL ERROR WARNING INFO and DEBUG The default is INFO url The URL to use for logging Supported protocols are currently file and syslog The file protocol creates a rotating file handler while the syslog protocol logs messages using the system logger syslogd The file handler accepts either a relative path or an absolute path If a relative path is provided it is relative to Configure default log directory The syslog handler accepts either a path for example syslog dev log or a hostname and an optional port for example syslog localhost 555 and syslog my example con If no port is provided it defaults to 541 which is the default port for the syslog daemon Section sharding To perform operations such as moving and splitting shards MySQL Fabric relies on the mysqldump and mysql client programs These programs can be installed in different locations and if they are not in the path for the MySQL Fabric node this section con
297. ht terms except that the holder is Tim Hudson tjh cryptsoft com Copyright remains Eric Young s and as such any Copyright notices in the code are not to be removed If this package is used in a product Eric Young should be given attribution as the author of the parts of the library used This can be in the form of a textual message at program startup or in documentation online or textual provided with the package Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions arte meti 1 Redistributions of source code must retain the copyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution 3 All advertising materials mentioning features or use of this software must display the following acknowledgement This product includes cryptographic software written by Eric Young eay cryptsoft com The word cryptographic can be left out if the rouines from the library being used are not cryptographic related 4 If you include any Windows specific code or a derivative thereof from the apps directory application code you must include an acknowledgement This product includes software written by Tim Hudson tjh cryptsoft com HIS SOFTW
298. ications Oracle and Java are registered trademarks of Oracle and or its affiliates Other names may be trademarks of their respective owners Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc AMD Opteron the AMD logo and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices UNIX is a registered trademark of The Open Group This software or hardware and documentation may provide access to or information about content products and services from third parties Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third party content products and services unless otherwise set forth in an applicable agreement between you and Oracle Oracle Corporation and its affiliates will not be responsible for any loss costs or damages incurred due to your access to or use of third party content products or services except as set forth in an applicable agreement between you and Oracle Documentation Accessibility For information about Oracle s commitment to accessibility visit the Oracle Accessibility Program website at http www oracle com pls topic lookup ctx acc amp id docacc Access to Oracle Support Oracle customers that have purchased support have access to electronic support through My
299. ices to avoid a split brain scenario This is achieved by having each host ping an external not one part of the cluster IP addresses in this case the network router 192 168 1 1 All the necessary software i e binaries must be installed in a regular partition independent on each node MySQL socket mysql sock and process id mysql pid files are stored in a regular partition as well The MySQL Server configuration file my cn the database files data and the MySQL Fabric configuration file fabric cfg are stored in a DRBD controlled file system that at any point in time is only available on one of the two hosts Figure 8 1 MySQL Fabric Setup using DRBD and Pacemaker Primary Secondary Node Node DRBD Replication 246 Setting up and testing your system 8 6 4 Setting up and testing your system 8 6 4 1 Configure Network It is recommended that you do not rely on DNS to resolve host names and so the following configuration files should be updated Example 8 1 etc hosts Host 1 127 20 Oelhocalhostmlocalhostelkocalcomanm 1 localhost localhost localdomain 192 LES l Oe aosta most lo alleclomansn Example 8 2 etc hosts Host 2 2710 Ollocalihostmlocalhostlocalcomanmn 1 localhost localhost localdomain LIZ LES L LOL iro Stelios eulemlom aleclomansn 8 6 4 2 Install all packages Install the necessary packages through the apt get repositories root host1 apt get install drbd8 utils coros
300. ie Cini 10 4 4 4 4 To display the audit log entries of specific query types use the following command shell gt mysqlauditgrep query type show SET SERVER data audit log e ee ee PEN NEEN ey ac a yr E ag E ee yn ee en ee eh Sy eee ae ay an yoy ay ne Sa A a Se ay er ay ey Sha a Sg Sa Sa eam an Se ae Se a Se GD Ah ey Sa ey eh STATUS TIMESTAMP NAME SOLTEXT CONNECTION 6 a nee a a a a ey ap a ay ye E hw eta eed ae a oh hi a aye ay Seen yeh anya Say ee ag a an ay Sen am Se aah aay ah Sg a a a ay Say a ae ya Sa a Ba Se a AG ey Sar ey Ey 0 2012 08 27 1133335 39 Query SET NAMES latinl COLLATE latinl_swedish_ci 7 0 AVIZ O9 2 7 933 335 58 Query SET session autocommit OFF 7 0 ADIL S09 2 7 33 335 38 Query SHOW VARIABLES LIKE READ_ONLY 7 0 ADIL S09 27 Til 33335 38 Query SHOW VARIABLES LIKE datadir 7 0 AVIZ 09 2 7 933 335 58 Query SHOW VARIABLES LIKE basedir 7 0 ADIL SO9 27 Til 33335 58 Query SET NAMES latinl COLLATE latinl_swedish_ci 8 0 AQIZ09 2 7 33 335 38 Query SET session autocommit OFF 8 0 ADVIL S09 27 Ti 33335 38 Query SHOW VARIABLES LIKE READ_ONLY 8 0 2012 09 27 711 3333539 Query SHOW VARIABLES LIKE basedir 8 0 ADIL S09 Z sri ig 26 950 Query SET NAMES latinl COLLATE latinl_swedish_ci 9 0 ZOUZS09 gt 2 30 26 50 Query SET session autocommit OFF 9
301. ies that resort to platform specific native code are also not used The class method and function signatures are designed to make use of a small number of required parameters and all optional parameters as a single dictionary Consider the following method def do_something_wonderful position objl obj2 options Does something wonderful A fictional method that does something to object 2 based on the location of something in object 1 Coste Lon in Poste Lon Am GoL eiai ara First object to manipulate obj2 in Second object to manipulate options in Option dictionary width width of printout default 75 iter max iterations default 2 ok to fail if True do not throw exception default True Returns bool True success Fail failed This example is typical of the methods and classes in the library Notice that this method has three required parameters and a dictionary of options that may exist Each method and function that uses this mechanism defines its own default values for the items in the dictionary A quick look at the method documentation shows the key names for the dictionary This can be seen in the preceding example where the dictionary contains three keys and the documentation lists their defaults To call this method and pass different values for one or more of the options the code may look like this opt_dictionary twiciint OOF uiteen g all ok_to_fail False result do_somet
302. if it is offline With this option you must also give the basedir and datadir options Start timeout Number of seconds to wait for the server to be online when started in read only mode using the start option The default value is 10 seconds The start timeout option is available as of MySQL Utilities 1 2 4 1 3 3 verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug 186 EXAMPLES e version Display version information and exit For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To display the server information for the local server and the settings for mysqld in the configuration file with the output in a vertical list use this command shell gt mysqlserverinfo server root pass localhost d format vertical Source on localhost Teonnectedk
303. ification clients are over the notification client threshold and the number of notifications above the notification threshold arrive in the notification interval the server is considered dead and failover is triggered The notification interval in seconds Only notifications arriving within this time frame can trigger a failover The minimum interval between failover operations in seconds In order to avoid making the system unstable failover operations are not triggered unless at least this much time has expired since the last failover This parameter is for the built in failure detector If more than this number of failures to contact the server occurs during the detection interval the server is considered unstable and a failover is triggered 225 Starting and Stopping MySQL Fabric Nodes detection_interval This parameter configures the detection interval for the built in failure detector in seconds detection_timeout This parameter configures the detection timeout used when attempting to contact the servers in the group prune_time This is the maximum age of events in the failure detector s error log and is also the interval for how often the error log is pruned Section connector Connectors that are MySQL Fabric aware contact MySQL Fabric to fetch information on groups shards and servers and then cache the results locally for a time period to improve performance This section contains configuration parameters passed
304. igured to the given master switchover Perform slave promotion to a specified candidate slave as designated by the new master option This command is available for both gtid enabled servers and non gtid enabled scenarios This command requires one of the following combinations master new master and slaves master new master and discover slaves login 155 mysql rpladmin Administration utility for MySQL replication Detection of a downed master is performed as follows If the connection to the master is lost wait ping seconds and check again If the master connection is lost and the master cannot be pinged or reconnected the failover event occurs For all commands that require specifying multiple servers the options require a comma separated list of connection parameters in the following form where the password port and socket are optional Slee k n passw Cx woogie e lt jeroiaie S lt SOCcleic S Ore lt lt legin paa gt 8 lt jsiowie S lt oes The utility permits users to discover slaves connected to the master In order to use the discover slaves feature all slaves must use the report host and report port startup variables to specify the correct hostname and ip port of the slave If these are missing or report the incorrect information the slaves health may not be reported correctly or the slave may not be listed at all The discover slaves feature ign
305. ile and pstats modules contain the following notice Copyright 1994 by InfoSeek Corporation all rights reserved Written by James Roskind 292 Python License Permission to use copy modify and distribute this Python software and its associated documentation for any purpose subject to the restriction in the following sentence without fee is hereby granted provided that the above copyright notice appears in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of InfoSeek not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission This permission is explicitly restricted to the copying and modification of the software to remain in Python compiled Python or other languages such as C wherein the modified or derived code is exclusively imported into a Python module INFOSEEK CORPORATION DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS IN NO EVENT SHALL INFOSEEK CORPORATION BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE Execution tracing The trace module contains the following notice
306. ilities command import serverclone CEN res serverclone clone_server conn opt new_data opt new_port opt new_id root None False True except exception UtilError e print ERROR e errmsg sys exit 199 Example As you can see the operation is very simple We just added a few options we needed like new data new port and new id much like mysqlserverclone and supplied some default values for the other parameters Next we need to copy the databases Once again we use the command module for mysqidbcopy to do all of the work for us First we need the connection parameters for the new instance This is provided in the form of a dictionary We know the instance is a clone so some of the values are going to be the same and we use a default root password so that is also known Likewise we specified the data directory and since we are running on a Linux machine we know what the socket path is For Windows machines you can leave the socket value None We pass this dictionary to the copy method dest_values user conn get user passwd Troot host conn get host pore opt new_port unix_socket os path join opt new_data mysql sock In this case a number of options are needed to control how the copy works for example if any objects are skipped For our purposes we want all objects to be copied so we supply only the minimal settings and let the library use the
307. ility has a number of options each explained in more detail below Some of the options are specific to certain commands Warning messages are issued whenever an option is used that does not apply to the command requested A brief overview of each command and its options is presented in the following paragraphs The start stop and reset commands require the slaves option to list all of the slaves in the topology Optionally the master option can be specified for the utility to check if the specified slaves are associated to the given master before executing the command making sure that the command is only applied to slaves connected to the right replication master The options required for the elect health and gtid commands include the master option to specify the existing master and either the slaves option to list all of the slaves in the topology or the discover slaves login option to provide the user name and password to discover any slaves in the topology that are registered and connected to the master The options required for switchover include the master option to specify the existing master the new master option to specify the candidate slave the slave to become the new master and either the slaves option to list the considered slaves in the topology or the discover slaves login option to provide the user name and password to discover any slaves in the topology that are registered and connected to the maste
308. ility pass Checking lower_case_table_names settings pass Checking slave delay seconds behind master FAIL Slave is NNN seconds behind master i oo cele shell gt mysqlrplcheck master m_account 192 168 1 1 13001 slave slave_acc2 192 168 1 3 3306 master on 192 168 1 1 connected slavicmOnm 2r ieor a Connected Test Description status Checking for binary logging on master pass Are there binlog exceptions pass Replication user exists pass Checking server_id values pass Checking server_uuid values pass Is slave connected to master pass Check master information file pass Checking InnoDB compatibility pass Checking storage engines compatibility pass Checking lower_case_table_names settings pass Checking slave delay seconds behind master FAIL Slave is N seconds behind master done Discussion In the above example we made use of the mysqlreplicate utility to setup a two layer replication topology where the existing server is now the master for the two new servers which will act as slaves Notice how we used the address of the old existing server in the master option and in the slave option we used the addresses of the new servers Also notice the use of the b flag this makes replication start from the first event recorded in the master s binary log Also notice how we used the mysql rplcheck utility to check the health of the replication In this case the failing test Check slave dela
309. imple steps to restore the initial topology as shown below 40 Restore the Previous Master After Failover Example Execution There are several steps involved in solving this problem We walk through each in turn You must first stop running the mysql failover utility instance and start the old master to be restored i e server1 3311 Next set the old master Server1 3311 as a slave of the current new master server2 331 2 shell gt mysqlreplicate master root server2 3312 slave root server1 3311 rpl user rpl rpl master on localhost connected slave on localhost connected Checking for binary logging on master Setting up replication dae done Next switchover to the previous master to restore the initial replication topology shell gt mysqlrpladmin master root server2 3312 slaves root server2 3313 root server4 3314 root server5 3315 rpl user rpl rpl new master root serverl1 3311 demote master switchover Checking privileges Performing switchover from master at server2 3312 to slave at serverl1 3311 Checking candidate slave prerequisites Checking slaves configuration to master Waiting for slaves to catch up to old master Stopping slaves Performing STOP on all slaves Demoting old master to be a slave to the new master Switching slaves to new master Starting all slaves Performing START on all slaves Checking slaves for errors Switchover co
310. inary forms with or without modification are permitted provided that the following conditions are mets 1 Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution 3 Neither the name of the project nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Floating point exception control Copyright c 1996 Th
311. ing EVENT util_test el Copying GRANTS from util_test gt COne To copy one or more databases from a master to a slave you can use the following command to copy the databases Use the master as the source and the slave as the destination shell gt mysqldbcopy source root localhost 3310 destination root localhost 3311 test123 rpl master rpl user rpl VOURC CMON MEO cose Ee COMMeOCECC Destination on localhost connected Source on localhost connected Stopping slave Copying database test123 Copying TABLE test123 t1l Copying data for TABLE test123 t1l Connecting to the current server as master Starting slave TEONE To copy a database from one slave to another attached to the same master you can use the following command using the slave with the database to be copied as the source and the slave where the database needs to copied to as the destination shell gt mysqldbcopy source root localhost 3311 destination root localhost 3312 test123 rpl slave rpl user rpl 93 PERMISSIONS REQUIRED Source on localhost connected Destination on localhost connected SOUBsCCHOnm LOG anos EC Onmo crocs Stopping slave Copying database test123 Copying TABLE test123 t1 Copying data for TABLE test123 tl Connecting to the current server s master Starting slave done PERMISSIONS REQUIRED The user must have SELECT SHOW VIEW EVENT and TRIGGER privileges for
312. ing slaves for errors Switchover complete Replication Topology Health 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 4 12 6 Lee aA ESO MASTER wie ON OK WO 5 AL a AL soil SLAVE UP ON OR TOZO NRS 3306 SLAVE wie ON OK 4 4 4 Discussion As with our previous scenario we used the mysql replicate utility to set up a replication topology between the existing server and the two new servers Notice the use of the b flag which this replication start from the first event recorded in the master s binary log 36 Setup Automatic Failover After setting our replication topology we made use of the mysqirpladmin utility specifying both the master and slave servers and using the health command to check the status of the replication Since our master server had lots of information it is normal for the new slaves to take some time to catch up thus the slave delay message on the health column of the output However if all goes well after some time the slaves will eventually catch up and when that happens the health column will show an OK status When this happened we used the mysql rpladmin utility yet again this time with switchover command Using the new master opti
313. ing to use the following table as an example CREATE TABLE test_db indexcheck_test emp_id INT 11 NOT NULL fiscal_number int 11 NOT NULL name VARCHAR 50 NOT NULL surname VARCHAR 50 NOT NULL Jay cicle VARCHAR 20 hire_date DATE default NULL birthday DATE default NULL PRIMARY KEY emp_id KEY idx _fnumber fiscal number UNIQUE KEY idx_unifnumber fiscal_number UNIQUE KEY idx_uemp_id emp_id KEY idx_full_nam nam surname KEY idx_full_name_dup name surname KEY idx_name name KEY idx_surname surname KEY idx_reverse_name surname name KEY dx_id_name emp_id name KEY idx_id_hdate emp_id hire_date Kev lh clixquna Clan Clay Maene nidi bir eNnday ENGINE InnoDB DEFAULT CHARSET utf8 Example Execution shell gt mysqlindexcheck server test_user localhost 13010 test_db indexcheck_test Source on localhost oos Connected The following indexes are duplicates or redundant for table test_db indexcheck_test CREATE INDEX idx_uemp_id gt ON test_db indexcheck_test emp_id USING BTREE may be redundant or duplicate of ALTER TABLE test_db indexcheck_test ADD PRIMARY KEY emp_id CREATE INDEX idx_fnumber ON test_db indexcheck_test fiscal_number USING BTREE may be redundant or duplicate of CREATE INDEX idx_unifnumber ON test_db indexcheck_test fiscal number USING B
314. instead stored in a separate MySQL server To provide a highly available solution both components must be redundant From a failure viewpoint these two components are treated as a single unit meaning that each pair will be collocated on a machine and if one of them fails the stack on that machine fails There will be two MySQL Fabric instances working in active and stand by mode The data stored in the MySQL server will be replicated through the Distributed Replicated Block Device or simply DRBD The MySQL Fabric process is stateless though and must simply be started in the stand by node in the event of a failure Pacemaker and CoroSync will be used to monitor whether the instances are running properly and to automate the failover and switchover operations Pacemaker will be responsible for monitoring components such as the MySQL Fabric Process MySQL server and DRBD and also for executing the failover and switchover operations CoroSync will be the communication infrastructure used by Pacemaker to exchange massages between the two nodes Applications accessing this cluster will do so through a Virtual IP address that is assigned to the active node specifically to the MySQL Fabric process and it will be automatically migrated to the stand by node during a failover or switchover operation This section aims to describe how to set up this highly available cluster 8 6 2 Pre requisites Two servers or virtual machines with A Linux
315. ion depending on the security privileges present and whether the database contains certain objects such as views or events and whether binary logging is enabled The new storag ngine and default storage engine options apply to all destination tables in the operation Some option combinations may result in errors during the operation For example eliminating tables but not views may result in an error a the view is copied The rp1 option is not valid for copying databases on the same server An error will be generated When copying data and including the GTID commands you may encounter an error similar to GTID_PURGED can only be set when GTID_EXECUTED is empty This occurs because the destination server is not in a clean replication state To aleviate this problem you can issue a RESET MASTER command on the destination prior to executing the copy Cloning databases that contain foreign key constraints does not change the constraint in the cloned table For example if table db1 t1 has a foreign key constraint on table db1 t2 when db1 is cloned to db2 table db2 t1 will have a foreign key constraint on db1 t2 The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf If any datab
316. ion report containing the SQL statements to conform the object definitions specified in reverse For example if changes for is set to server1 also generate the transformation for server2 Note 9 The reverse changes are annotated and marked as comments skip table options 110 NOTES NOTES Ignore the differences between all table options such as AUTO_INCREMENT ENGINE CHARSET etc A warning is issued if the skip table options option is used and table option differences are found ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug version Display version information and exit width lt number gt Change the display width of the test report The default is 75 characters You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects to be compared
317. ionProvider class or related demos compatible with Java 5 8 8 7 A Complete Example Working with Employee Data This document demonstrates two possible ways of working with sharded data relating to employees To run this program you must set up a shard mapping for the employees table in MySQL Fabric as described in Section 8 8 4 MySQL Fabric Configuration for Running Samples This code can be found in the distribution package in src demo fabric EmployeesJdbc java Copy eighe 3 J IA I Gr This his c program is free software 2013 2014 Oracle and or its affiliates All rights reserved he MySQL Connector J is licensed under the terms of the GPLv2 http www gnu org licenses old licenses gp1 2 0 html gt like most MySQL Connectors here are special exceptions to the terms and conditions of the GPLv2 as it is applied to his software see the FLOSS License Exception http www mysql com about legal licensing foss exception html gt you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation version 2 of the License program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See t You s progr Floor z am if not write to the Free 7 BOStON MA 02110 1301 USA package demo f
318. ions for connecting to a server shell gt mysqlserverinfo py server root other pass localhost 3308 format vertical Source on localhost connected KEK KKK KKK KKK KKEKEKKEKKKKKKKK als row KEK KKK KKK KEK KKK KEKKKKKEK KK KK server localhost 13001 config_file etc my cnf etc mysql my cnf binary_log clone bin 000001 binary_log_pos 341 relay_log relay log pos vorcion monon i llog datadir Volumes Source source temp_13001 basedir Volumes Source source bzr mysql 5 6 plugin_dir Volumes Source source bzr mysql 5 6 lib plugin general_llog OFF general_log_file general_log_file_size log_error log_error_file_size slow_query_log OFF slow_query_log_file slow_query_log_file_size 1 row done As of MySQL Utilities 1 4 4 this deprecated connection method issues a warning if you use this connection method 2 2 2 Specifying Connections in Python Library If you build your own utilities using the MySQL Utilities library you will encounter various methods for connecting to MySQL servers Methods that deal with connecting to servers can accept the following mechanisms for supplying the data Specifying Connections in Python Library As a Python dictionary containing the connection parameters e As a connection specification string containing the connection parameters e As a Server instance The dictionary lists the values by name as described above For example you would cre
319. ions require a comma separated list of connection parameters in the following form where the password port and socket are optional lt e eters eas sva es nesters lear Oommen lle aS OC keiec 1 mors za lheyeplinaaicla gt esmo re S lt Soelkeeics The utility permits users to discover slaves connected to the master In order to use the discover slaves feature all slaves must use the report host and report port startup variables to specify the correct hostname and port of the slave If these are missing or report the incorrect information the slave s health may not be reported correctly or the slave may not be listed at all The discover slaves feature ignores any slaves to which it cannot connect The discover slaves feature is run automatically on each interval The utility permits the user to specify an external script to execute before and after the switchover and failover commands The user can specify these with the xec before and xec after options The return code of the script is used to determine success thus each script must report 0 Success to be considered successful If a script returns a value other than 0 the result code is presented in an error message The utility also permits the user to specify a script to be used for detecting a downed master or an application level event to trigger failover This can be specified using the xec fail check option The return code for the script is used to invoke f
320. is includes the host name port role MASTER or SLAVE of the server state of the server UP is connected WARN not connected but can ping DOWN not connected and cannot ping the GTID_MODE and health state The master health state is based on the following if GTID_ MODE ON the server must have the binary log enabled and a user must exist with the REPLICATE SLAVE privilege The seconds behind option is used to detect when a slave is behind the master It allows users to set a threshold for reporting purposes only It does not apply to slave candidacy or selection during failover The slave health state is based on the following the IO_THREAD and SQL_THREADS must be running it must be connected to the master there are no errors the slave delay for non GTID enabled scenarios is not more than the threshold provided by the max position and the slave is reading the correct master log file and slave delay is not more than the seconds behind threshold option At each interval if the discover slaves option was specified at startup and new slaves are discovered the health report is refreshed e gtid Display the master s list of executed GTIDs contents of the GTID variables GLOBAL GTID_EXECUTED GLOBAL GTID_PURGED and GLOBAL GTID_OWNED Thus you can toggle through the four screens by pressing the c key e UUID Display universally unique identifiers UUIDs for all servers e Log This option displays th
321. is software and its associated documentation for any purpose and without fee is hereby granted provided that the above copyright notice appears in all copies and that both that copyright notice and this permission notice appear in supporting documentation and that the name of Secret Labs AB or the author not be used in advertising or publicity pertaining to distribution of the software without specific written prior permission SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT ABILITY AND FITNESS IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE DATA OR PROFITS WHETHER IN AN ACTION OF CONTRACT NEGLIGENCE OR OTHER TORTIOUS ACTION ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE test_epoll The test_epoll contains the following notice Copyright c 2001 2006 Twisted Matrix Laboratories Permission is hereby granted free of charge to any person obtaining a copy of this software and associated documentation files the Software to deal in the Software without restriction including without limitation the rights to use copy modify merge publish distribute sublicense and or sell copies of the Software and to permit persons to whom the Software is furnished to do so subject to the following conditions The
322. iscovery may fail to identify the slave In the above example no data consistency issues were found In case any data difference are found each is clearly identified by the DIFF prefix followed by concise information of where and what is the difference Additionally at the end the utility displays a summary of the number of issues found The utility also allows users to check slaves without specifying the master However be advised that only checking the slaves will not guarantee that there is no data consistency issue between the master and the slaves Also keep in mind that the results provided by the utility are valid at the time the checks are 45 How To Fix Errant Transactions on the Replication Topology actually performed for each table This is because in an active system with data continuously changing inconstancy issues might be introduced in the immediate instance after the check is completed Permissions Required The user for the master must have permissions to lock tables perform the checksum and get information about the master status Specifically the user used to connect to the master requires the following privileges SUPER or REPLICATION CLIENT LOCK TABLES and SELECT The user for the slaves must have permissions to start stop the slave perform the checksum and get information about the slave status More specifically the login user to connect to slaves requires the following privileges SUPER and SELECT
323. istency check identifies the missing rows To skip this test use the skip row count option 5 Check table data consistency This check identifies both changed rows as well as missing rows from one or another of the tables in the databases Changed rows are displayed as a diff style report with the format chosen GRID by 79 mysqldbcompare Compare Two Databases and Identify Differences default and missing rows are also displayed using the format chosen This check is divided in two steps first the full table checksum is compared between the tables then if this step fails or is skipped the algorithm to find rows differences is executed To skip the preliminary checksum table step in this test use the skip checksum table option To skip this full test use the skip data check option You may want to use the skip xxx options to run only one of the tests This might be helpful when working to bring two databases into synchronization to avoid running all of the tests repeatedly during the process Each test completes with one of the following states pass The test succeeded FAIL The test failed Errors are displayed following the test state line SKIP The test was skipped due to a missing prerequisite or a skip option WARN The test encountered an unusual but not fatal error The test is not applicable to this object To specify how to display diff style output use one of the following values w
324. it When the mode argument is applicable these values are permitted fabric MODE_READWRITE Connect to a master server This is the default e fabric MODE_READONLY Connect to a slave if one is available to the master otherwise If there are multiple secondary MySQL servers load balancing is used to obtain the server information When the scope argument is applicable these values are permitted e fabric SCOPE_LOCAL Local operation that affects the row with a given key This is the default e fabric SCOPE_GLOBAL Global operation that affects all rows Providing the name of a high availability group specifies that we know exactly the set of database servers that with which to interact To do this set the group property using the set_property method fcnx set_property group myGroup Connector Python to do so based on information from the MySQL Fabric server Whether operations use RANGE or HASH is transparent to the user The information is provided by Fabric and Connector Python uses the correct mechanism automatically can be specified the tables argument to set_property is specified as a tuple or list fconx set_property tables employees employees key 40 Cur FCN CUESOr do operations for employee with emp_no 40 260 Using Connector J with MySQL Fabric fcnx close By default operations occur in local scope or the scope property can be given to specify local or global scope explicitly
325. ith the difftype option unified default Display unified format output context Display context format output differ Display differ style format output sql Display SQL transformation statement output To specify how to display output for changed or missing rows use one of the following values with the format option grid default Display output in grid or table format like that of the mysql client command line tool csv Display output in comma separated values format 80 OPTIONS e tab Display output in tab separated format e vertical Display output in single column format like that of the G command for the mysq1 client command line tool The changes for option controls the direction of the difference by specifying the object to be transformed in either the difference report default or the transformation report designated with the difftype sql option Consider the following command shell gt mysqldbcompare serverl root hostl server2 root host2 difftype sql db1 dbx The leftmost database db1 exists on the server designated by the server1 option host1 The rightmost database dbx exists on the server designated by the server2 option host 2 changes for server1 Produce output that shows how to make the definitions of objects on server1 like the definitions of the corresponding objects on server2 changes for server2 Produce output that shows how t
326. ities 1 4 2 this option was named force locking lt locking gt Choose the lock type for the operation Permitted lock values are no locks do not use any table locks lock all use table locks but no transaction and no consistent read and snapshot consistent read using a single transaction The default is snapshot multiprocess Specify the number of processes to concurrently copy the specified databases Special values 0 number of processes equal to the number of detected CPUs and 1 default no concurrency Multiprocessing works at the database level for Windows and at the table level for Non Windows POSIX systems new storage engine lt new_engine gt The engine to use for all tables created on the destination server quiet q Turn off all messages for quiet execution regexp basic regexp G Perform pattern matches using the REGEXP operator The default is to use LIKE for matching rpl lt dump_option gt replication lt dump_option gt Include replication information Permitted values are master make destination a slave of the source server and slave make destination a slave of the same master as the source only works if the source server is a slave rpl user lt replication_user gt The user and password for the replication user requirement in the form lt user gt lt passwora gt or lt login path gt E g rpl passwd Default None skip gtid Skip creation and executio
327. ity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the replication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqlrplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu net you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To set up replication between two MySQL instances running on different ports of the same host using the default settings use this command shell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl master on localhost connected slave on localhost connected Checking for binary logging on master Setting up replication done 147 EXAMPLES The following command uses pedantic to ensure that re
328. ject created as shown in Section 8 8 2 Loading the Driver and Requesting a Fabric Connection Providing the name of a high availability group specifies that we know exactly the set of database servers with which to interact We can do this in two ways e The simplest method is to include the name of the server group in the connection string This is useful if a connection will need to access data only in that server group It is also possible to set the name of the server group in this way initially and to change it programatically later provide the server group as a connection property Connection conn DriverManager getConnection jdbc mysql fabric fabrichost 32274 database fabricServerGroup myGroup e If we connect without specifying a server group or want to change it later we can use the JDBC4FabricMySQLConnection interface to set the server group name 263 MySQL Fabric Configuration for Running Samples JDBC4FabricMySQLConnection conn connection initialization here conn setServerGroupName myGroup Connector J to do so based on information from the MySQL Fabric server e Shard tables and shard keys can also be given as connection properties if desirable Here we say that provide the shard table as a connection property Connection conn DriverManager getConnection Jdbc mysql fabric fabrichost 32274 database fabricShardTable employees employees example sets up the connection to perfo
329. l 5 168 1 101 77887 flexible meta disk internal on Hoste 4 device dev drbd0 isk leew esclop address 192 168 1 102 77887 meta disk internal The IP addresses and disk locations should be specific to the hosts that the cluster will be using In this example the device that DRBD will create will be located at dev drbao it is this device that will be swapped back and forth between the hosts by DRBD This resource configuration file should be copied over to the same location on the second host root host1 scp clusterdb_res res host2 etc drbd d 248 Setting up and testing your system The configuration file previously presented uses DRBD 8 3 dialect Although DRBD 8 4 is the newest version some distributions might still contain DRBD 8 3 If you have installed DRBD 8 4 do not worry though because it understands the DRBD 8 3 configuration file Before starting the DRBD daemon meta data must be created for the new resource clusterdb_res on each host using the command root host1 drbdadm create md clusterdb_res root host2 drbdadm create md clusterdb_res It is now possible to start the DRBD daemon on each host root host1 etc init d drbd start root host2 etc init d drbd start At this point the DRBD service is running on both hosts but neither host is the primary and so the resource block device cannot be accessed on either host this can be confirmed by querying the status of the service root
330. l gt mysqldbexport server root root localhost 13001 all export both rpl master rpl user rpl rp shell gt mysqldbimport server root root localhost 13002 data sql Source on localhost connected Importing definitions from data sql ERROR The import operation contains GTID statements that require the global gtid_executed system variable on the target to be empty no value The gtid_executed value must be reset by issuing a RESET MASTER command on the target prior to attempting the import operation Once the global gtid_executed value is cleared you may retry the import shell gt mysql uroot proot h 127 0 0 1 port 13002 e RESET MASTER shell gt mysqldbimport server root root localhost 13002 data sql Source on localhost connected Importing definitions from data sql CAUTION The following 1 warning messages were included in the import file WARNING A partial export from a server that has GTIDs enabled will by default include the GTIDs of all transactions even those that changed suppressed parts of the database f you don t want to generate the GTID statement use the skip gtid option To export all databases use the all and export both options aa CONS a Discussion There are several operations listed here The first one we see is the execution of the mysqldbexport utility to create a file that includes an export of all databases as designated with the a11 option We add the export
331. l off root host1 sysv rc conf pacemaker on root host2 sysv rc conf drbd off root host2 sysv rc conf corosync on root host2 sysv rc conf mysql off root host2 sysv rc conf pacemaker on Note R MySQL Fabric is not installed as a service so there is nothing to do here for it 8 6 5 Key administrative tasks The cluster management tool can then be used to migrate the resources between machines root host1 crm resource migrate g_mysql host2 Specifying the g_mysq1 group will migrate all resources in the group and implicitly any colocated resources as well If for any reason a resource cannot be properly started up or shut down it will become unmanaged In this case we will have to manually put it back to a managed state For example this could mean that we would have to fix an issue that blocked the shutdown kill or stop some processes and run the following command root host1 crm resource cleanup resource The components of this stack are designed to cope with component failures but there may be cases where a sequence of multiple failures could result in DRBD not being confident that the data on the two hosts is consistent In the event that this happens DRBD will break the connection Should this happen we need to identify which of the two hosts has the correct data and then have DRBD resynchronize the data for the steps below it is assumed that host1 has the correct data root host2 drbdadm secondary cluste
332. ldiff serverl root host1 server2 root host2 changes for server2 difftype sql db1 tablel1 dbx table3 serverl on hostli connected server2 on host2 connected Comparing dbl tablel to dbx table3 FAIL Transformation statements Sb Se Se SE ALTER TABLE dbx table3 DROP COLUMN notes CHANGE COLUMN misc misc char 30 Compare failed One or more differences found To generate a set of SQL statements that transform the definitions of dbx table3 and db1 tablel in both directions use this command shell gt mysqldiff serverl root host1 server2 root host2 show reverse difftype sql db1 tablel1 dbx table3 serverl on hostl connected server2 on host2 connected Comparing dbl tablel to dbx table3 FAIL Transformation statements destination serverl1 ALTER TABLE dbl tablel ADD COLUMN notes char 30 AFTER a CHANGE COLUMN misc misc char 55 destination server2 ALTER TABLE dbx table3 DROP COLUMN notes CHANGE COLUMN misc misc char 30 Se H Se OSE Compare failed One or more differences found 113 PERMISSIONS REQUIRED PERMISSIONS REQUIRED The user must have SELECT privileges for both objects on both servers as well as SELECT on the mysql database 5 8 mysqldiskusage Show Database Disk Usage This utility displays disk space usage for one or more databases The utility optionally displays disk usage for the binary log slow query log error log ge
333. le to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required show stats S Show file statistics and general table information for each frm file read start timeout lt timeout_in_seconds gt Number of seconds to wait for spawned server to start The default is 10 seconds user Execute the spawned server using this user account Permits the execution of the utility as one user but the spawned server as another Required if running the utility as the root user e g su or sudo e quiet Turn off all messages for quiet execution except CREATE statements and errors e verbose v Control how much information is displayed For example v verbose vv more verbose vvv debug e version Show program s version number and exit NOTES Tables with certain storage engines cannot be read in the default mode These include PARTITION PERFORMANCE _ SCHEMA You must read these with the diagnostic mode Use the diagnostic mode for tables that fail to open correctly in the default mode or if there is no server installed on the host To change the storage engine in the CREATE statement generated for all frm files read use the new storage engine option To tur
334. le_names 2 Slave lower_case_table_names 2 Checking slave delay seconds behind master pass Slave status Slave_IO_State Waiting for master to send event MEScer Kesic amp Inesie il Master_User rpl Master Port 2210 Connect_Retry 60 Master_Log_File clone bin 000001 Read_Master_Log_Pos 482 Relay_Log_File clone relay bin 000006 Relay_Log_Pos 251 Relay_Master_Log_File clone bin 000001 Slave_IO_Running Yes Slave_SQL_ Running Yes 168 PERMISSIONS REQUIRED Replicate_Do_DB Replicate_Ignore_DB Replicate_Do_Table Replicate_Ignore_Table Replicate_Wild_Do_Table Replicate_Wild_Ignore_Table Last_Errno 0 Last Error Skip Counter lt 7 0 Exec_Master_Log_Pos 482 Relay_Log_Space 551 Until_Condition None Untill Log File lt menil toe Bos 3 0 Master_SSL_Allowed No Master_SSL_CA_File Master_SSL_CA_Path Master_SSL_Cert Master_SSL_Cipher Master_SSL_Key Seconds_Behind_Master 0 Master_SSL_Verify_Server_Cert No Last_lO_Brrno 0 Last TO _ EFFOF Last_SOl_ Errno 2 0 Last SOU EFEOL S done PERMISSIONS REQUIRED The users on the master need the following privileges SELECT and INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION The slave users need the SUPER privilege Also when using GTIDs the slave users must also have SELECT privilege over the mysql database 5 18 mysql rplshow Show Slaves f
335. lfabric group add group_id MOVE localhost 3315 shell gt mysqlfabric group add group_id MOVE localhost 3316 shell gt mysqlfabric group promote group_id MOVE 2 Execute the shard move shell gt mysqlfabric sharding move_shard 1 group_id MOVE 3 Verify that the move has happened shell gt mysqlfabric sharding lookup_servers employees employees 4 237 The mysqlfabric Utility Shard Split If the shard becomes overloaded we may need to split the shard into another group The shard split feature can be used to split the data in a given shard into another shard The following demonstrates how to do this 1 Set up the required group or groups shell gt mysqlfabric shell gt mysqlfabric shell gt mysqlfabric shell gt mysqlfabric group create group_id SPLIT group add group_id SPLIT localhost 3317 group add group_id SPLIT localhost 3318 group promote group_id SPLIT 2 Execute the shard split shell gt mysqlfabric sharding split_shard 2 group_id SPLIT split_value 150 3 Verify that the shard Split happened shell gt mysqlfabric sharding lookup_servers employees employees 152 shell gt mysqlfabric sharding lookup_servers employees employees 103 8 4 The mysqlfabric Utility This section describes the mysql fabric utility including examples of all commands Fabric commands are organized in categories that include group dump manage server role user threat sharding and event We explain each group an
336. lication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqlrplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu net you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine Similarly in order to avoid issues mixing local IP 127 0 0 1 with localhost all the addresses 127 0 0 1 will be internally converted to localhost by the utility The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf LIMITATIONS This utility is designed to work exclusively for servers that support global transaction identifiers GTIDs and have gt id_mode oN Due to known server issues with some operations required for the synchronization process only MySQL Server versions 5 6 14 and higher are supported by this utility Some replication filtering options are not supported by this utility due to known issues on the server side namely replicate_do_db replicate_ignore_db and replicate_wild_do_table In
337. license information and exit e changes for lt direction gt Specify the server to show transformations to match the other server For example to see the transformation for transforming object definitions on server1 to match the corresponding definitions on server2 use changes for server1 Permitted values are serverl and server2 The default is server e character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client difftype lt difftype gt d lt difftype gt Specify the difference display format Permitted format values are unified default context differ and sql compact Compacts the output by reducing the control lines that are displayed in the diff results This option should be used together with one of the following difference types unified or context 109 OPTIONS force Do not halt at the first difference found Process all objects to find all differences quiet q Do not print anything Return only an exit code of success or failure server1 lt source gt Connection information for the first server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server mast
338. life easier shell gt cp etc corosync corosync conf example etc corosync corosync conf After editing it it should have a content similar to what follows totem version 2 crypto_cipher none crypto_hash none interface ringnumber 0 bindnetaddr 102 1763 1 0 Meastacar 239 255 mcastport 5405 tels logging to_syslog yes quorum provider corosync_votequorum two_node 1 eis see anika nodelist node met aces 102 162 1101 nodeid 1 node mimet aces I2 MEE sI O2 nodeid 2 Be careful while setting up the network address that the Corosync will bind to For example according to the Corosync documentation if the local interface is 192 168 5 92 with netmask 255 255 255 0 set bindnetaddr to 192 168 5 0 If the local interface is 192 168 5 92 with netmask 255 255 255 192 set bindnetaddr to 192 168 5 64 and so forth This will make Corosync automatically pick the network interface based on the network address provided It is also possible to set up a specific address such as 192 168 5 92 but in this case the configuration file will be different per machine Create the etc corosync service d pcmk file to tell the Corosync to load the Pacemaker plug in service 252 Setting up and testing your system Load the Pacemaker Cluster Resource Manager name pacemaker ver 1 Change the etc default corosync file as follows start corosync at boot yes no START y65S
339. list of console or base commands These allow the user to interact with the features of the console itself The list of base commands is shown below along with a brief description Command Description help utilities Display list of all gtilities supported help lt utility gt Display help for a specific utility help help commands Show this jist exit quit Exit the console set lt variable gt lt value gt Store a variable for recall in commands show options Display list of options specified by the user on launch show variables Display list of variables lt ENTER gt Press ENTER to execute command lt ESCAPE gt Press ESCAPE to clear the command entry lt DOWN gt Press DOWN to retrieve the previous command SURS Press UP to retrieve the next command in history lt TAB gt Press TAB for type completion of utility option or variable names lt TAB gt lt TAB gt Press TAB twice for list of matching type completion context sensitive One of the most helpful base commands is the ability to see the options for a given utility by typing help lt utility gt When the user types this command and presses ENTER the console will display a list of all of the options for the utility The console provides tab completion for all commands options for utilities and user defined variables Tab completion for commands allows users to specify the starting N characters of a command and press TAB to complete the command I
340. ll the servers with their process ID and data directory On Windows the utility shows only the process ID and port OPTIONS mysqlserverinfo accepts the following command line options help Display a help message and exit license Display license information and exit basedir lt basedir gt The base directory for the server This option is required for starting an offline server Is also used to access server tools such as my_print_defaults that is required to read the login path values from the login configuration file mylogin cnf datadir lt datadir gt The data directory for the server This option is required for starting an offline server format lt format gt f lt format gt Specify the output display format Permitted format values are grid csv tab and vertical The default is grid no headers h Do not display column headers This option applies only for grid csv and tab output port range lt start end gt The port range to check for finding running servers This option applies only to Windows and is ignored unless show servers is given The default range is 3306 3333 Server lt server gt Connection information for a server Use this option multiple times to see information for multiple servers 185 OPTIONS To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provi
341. ll gt mysqlrplsync master user pass localhost 3310 slaves rp1 pass localhost 3311 rpl1 pass localhost 3312 verbose Checking users permission to perform consistency check WARNING Replication filters found on checked servers This can lead data consistency issues depending on ho More information http dev mysql com doc en replication rules html Master localhost 3310 lpiinllee_clo_ clog test Zalleyjne_clioil Slave localhost Gs snl replicate do table test_rplsync_db1 t1l Slave localhost 3312 replicate_ignore_table test_rplsync_db1 t2 replicate_wild_ignore_table test _rplsync _db1 3 GTID differences between Master and Slaves Slave localhost 231i is up to date Slave localhost 2312 is up to date Checking data consistency Using Master localhost 3310 as base server for comparison Checking test_rplsync_db1 database SKIP Table t0 check for localhost 3311 filtered by replication rule 180 PERMISSIONS REQUIRED Checeime VicO iecalole Cleicas Setting data synchronization point for slaves Compute checksum on slaves wait to catch up and resume replication OK test_rplsync_dol t0 checksum for server localhoste s3i2 Checking tl table data Setting data synchronization point for slaves Compute checksum on slaves wait to catch up and resume replication OK test rplsync dbl n Cl checksum for server localhost 3311
342. ll utilities supported The console will execute the command then exit shell gt mysqluc e help utilities UEtlity Description mysqlindexcheck check for duplicate or redundant indexes mysqlrplcheck check replication mysqluserclone clone a MySQL user account to one or more new users mysqldbcompare compare databases for consistency mysqldiff compare object definitions among objects where the difference is how dbl objl differs from db2 obj2 mysqldbcopy copy databases from one server to another mysqlreplicate establish replication with a master mysqldbexport export metadata and data from databases mysqldbimport import metadata and data from files mysqlimetagrep search metadata mysqlprocgrep search process information mysqldiskusage show disk usage for databases mysqlserverinfo show server information mysqlserverclone start another instance of a running server The following demonstrates launching the console to run several commands using the execute option to including setting a variable for a server connection and executing a utility using variable substitution Note R It may be necessary to escape the on some platforms such as Linux The output below is an excerpt and is representational only shell gt mysqluc e set SERVER root host123 mysqldiskusage server SERVER Source on host123 connected NOTICE Your user account does not have read access to the datadir Data sizes will be calculated and actual
343. lobal shell gt mysqlfabric group promote group_id 1 shell gt mysqlfabric group promote group_id 2 shell gt mysqlfabric group promote group_id 3 Define a Shard Mapping When we define a shard mapping we basically do three things 1 Define the type of sharding scheme we want to use RANGE HASH LIST and so forth 2 Define a global group that stores all the updates that must be propagated to all the shards that are part of this sharding scheme 3 We generate a unique shard mapping id to which we can later associate database objects tables shell gt mysqlfabric sharding create definition RANGE group_id global Procedure uuid 195bcale c552 464b b4e3 1fal5e9b49d5 finished True success True return 1 activities Add Tables to Defined Shard Mappings Add the database table to the shard mapping defined previously shell gt mysqlfabric sharding add_table 1 employees employees emp_no 235 Example Fabric and Sharding Add Shards for the Registered Tables shell gt mysqlfabric sharding add shard 1 group_id 1 1 group id 2 100000 group id 2 200000 state ENABLED Executing Updates and Queries Now you can write some example code for querying the sharded system import random import mysql connector from mysql connector import fabric def prepare_synchronization cur We need to keep track of what we have executed so far to guarantee that the employees employees table exists at all shards
344. location user password will be prompted use the following command shell gt mysqlauditadmin audit log name audit log 13486539046497235 copy remote login user host copy to PERMISSIONS REQUIRED The user must have permissions to read the audit log on disk and write the file to the remove location 5 2 mysqlauditgrep Allows users to search the current or an archived audit log This utility allows you to search the current or archived audit logs allowing you to display data from the audit log file according to the defined search criterion It also allows you to output the results in different formats namely GRID default TAB CSV VERTICAL and RAW the original XML format This utility allows you to search and filter the returned audit log records by users users date and time ranges start date and nd date SQL query types query type logged event and record types event type status status and matching patterns pattern Any of these search options can be combined and used together with the retrieved records resulting from all passed in options being true The pattern supports two types of pattern matching standard SQL used with the SQL LIKE operator SQL patterns and standard REGEXP POSIX regular expression patterns This utility always requires an audit log file to be passed in so the AUD T_LOG_FILE argument is searched as a full path and file name for the audit log fil
345. log on disk Tips and Tricks The values for the vent type and format options are case insensitive therefore lower and upper cases can be mixed to specify these values as long as a supported event type name or format is used Unlike them the value specified for the users option is case sensitive so be careful not to mix upper and lower cases here It is possible to find some event type values with a space in the middle for example like Binlog Dump or Init DB If one of such values needs to be specified for the event t ype option then it must be surrounded by double or single quotes depending on the operating system 60 Chapter 4 Overview of MySQL Utilities Table of Contents 4 1 Database Operations 1 ceil aiiees dea tions alive nano eh aed nies 4 2 General Operations 2 0 0 0 cece cece e cece eee eee teeter ee eeni eee edariak dda iaki denii niaan iKi didi aaki dena EDHE da eraai donera 4 3 High Availability Operations meinir ee ee ee ee ee eect cere ee ee eda eane ee eeeeee ee aaaaeeeseeeeeaeaaaaenees 4A Server Operations ie sccccetistsee ces cnc dennavedeeacs iraani aa eeandelees aie Ea a e aaeeeiai ai aane 4 5 Specialized OperationS ssieis ionita ninatan ea eaaa an eiea esi aBa TEE A E SE de a Ea EEEE This chapter presents an brief overview of each of the available utilities The utilities are grouped into sections based on the type of administrative function that they perform 4 1 Database Op
346. log to standard output Thus this is the machine where you installed the MySQL Fabric and Connector Python software and is also the machine listed in the configuration file protocol xmlrpc section To follow the examples in the quick start section you must use localhost for the host name To put the MySQL Fabric node in the background add the daemoni ze option However this will divert the log to the syslog file While experimenting with MySQL Fabric you may find it more convenient not to use daemonize so that the log is written to your terminal Use this command to stop a MySQL Fabric node shell gt mysqlfabric manage stop 226 Old Configuration System This command contacts the MySQL Fabric server running at the address mentioned in the protocol xmlrpc section and instructs it to stop 8 2 5 Old Configuration System Important A This documentation describes the MySQL Fabric configuration before version 1 5 5 1 6 2 The previous configuration documentation is archived here for informational and upgrade purposes Configuring MySQL Fabric requires creating a MySQL user to access the backing store and editing the configuration file with the MySQL user details This section assumes you have already set up the backing store See Section 8 5 Backing Store for more information Create a MySQL User The first thing you must have is a user account on the MySQL server that you plan to use for your backing store
347. lowing to surpass unregistration failure on crashed old masters allowing the mysqlfailover utility to start in auto mode Permissions Required The user have permissions to configure replication 42 How Can Find All of the Slaves Attached to My Master Server Tips and Tricks It is important to wait for the old master to catch up with the new master in order to guaranty that no transactions are lost Depending on the time the old master was down or not accessible it might take a considerable time for the old master to execute all missing transactions MySQL Utilities provide tools that allow the visualizations of the slaves status namely the health command of the mysqli rpladmin utility An alternative set of steps could have been followed to perform the desired task using the failover command from mysqlrpladmin instead of switchover In this case the old master should be specified in the candidates list using the option candidates to be chosen as the preferred slave to become the new master no need for the master new master and demote master options However an additional step will be required to set the previous master server2 3312 as a slave of the old master server 331 1 using the mysqlreplicate utility because failover will not demote the previous master as it assumes that it is not available Notice that unlike switchover that will fail if the server specified by the new master option does not meet
348. lt user gt lt password gt localhost 13001 The console has detected that the utility mysqlrplshow ended with an error code You can get more information about the error by running the console command show last error mysqluc gt show last error Execution of utility mysqlrplshow master lt user gt lt password gt localhost 13001 returned errorcode 2 with error message Usage mysqlrplshow py master root localhost 3306 mysqlrplshow py error The discover slaves login is required to test slave connectivity mysqluc gt mysqlrplshow master lt user gt lt password gt localhost 13001 discover slaves login lt user gt lt password gt master on localhost connected Finding slaves for master localhost 13001 Replication Topology Graph localhost 13001 MASTER t Toca knosutisuu2 LeLaAves Toca those ta suns Lelaves 27 I ve Got Too Many Indexes How Do Know Which Ones to Drop T lieesellinogics ISO E EAE 7 lheezellinogics ISOOS Siw mysqluc gt Discussion There is a lot going on here in this example Let s look through the command entries as they occur in the text The first command mysqluc starts the users console Once the console starts you will see a welcome banner followed by a simple prompt mysqluc gt No additional options or parameters are necessary However it should be noted that you can pass commands to the consol
349. mance MySQL Fabric on the other hand allows the application to stick with the InnoDB storage engine which is better suited to many applications 8 9 10 How is MySQL Fabric licensed MySQL Fabric is available for use under the GPL v2 Open Source license or it can be commercially licensed as part of MySQL Enterprise Edition or MySQL Cluster Carrier Grade Edition 8 9 11 What if MySQL Fabric doesn t do what need it to There are a number of options e Raise feature requests or bug reports e Modify the code to customize the current services MySQL Fabric is written in Python and is designed to be easy to extend e Implement new modules that bind into the MySQL Fabric framework to implement new services High Availability 8 9 1 How is High Availability achieved with MySQL Fabric cccccceeeeceeeeeeeaeeeeeeeeeeeeeeeaeaaeaeeeeeeees 279 8 9 2 How are MySQL Server failures detected ccececeeeceaeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaaaeeeeeeeeeeaeaaes 279 278 MySQL Fabric Frequently Asked Questions 8 9 3 What happens when the primary master MySQL Server fails 0 ccccceeeeeeeeeeeeeeeeeeeeaeaaeeees 279 8 9 4 Does my application need to do anything as part of the failover cceeeeeeaeeeeeeeeeeeeeeeaeeaeeees 279 8 9 5 Is a recovered MySQL Server automatically put back into service ccceeeeeeeeeeeeeeeeeeeeeeeeeaes 279 8 9 6 Does MySQL Fabric work with semisynchronous replication
350. ment match command lt pattern gt Match all processes where the Command field matches the pattern match db lt pattern gt Match all processes where the Db field matches the pattern match host lt pattern gt Match all processes where the Host field matches the pattern match id lt pattern gt Match all processes where the ID field matches the pattern match info lt pattern gt Match all processes where the Info field matches the pattern match state lt pattern gt Match all processes where the State field matches the pattern match user lt pattern gt Match all processes where the User field matches the pattern print Print information about the matching processes This is the default if no kill connection or kill query option is given If a kill option is given print prints information about the processes before killing them regexp basic regexp G Perform pattern matches using the REGEXP operator The default is to use LIKE for matching This affects the mat ch xxx options Server lt source gt Connection information for a server Use this option multiple times to search multiple servers To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as se
351. meters connections Sequence of connection specifiers to send the query to output File object to use for writing the result connector Connector to use for connecting to the servers 6 3 2 mysql utilities command proc Search Processes on Servers This module searches processes on a server and optionally kills either the query or the connection for all matching processes Processes are matched by searching the fields of the INFORMATION_SCHEMA PROCESSLIST table which is available only for servers from MySQL 5 1 7 and later Internally the module operates by constructing a SELECT statement for finding matching processes and then sending it to the server Instead of performing the search the module can return the SQL code that performs the query This can be useful if you want to execute the query later or feed it to some other program that processes SQL queries further Constants The following constants correspond to columns in the INFORMATION_SCHEMA PROCESSLIST table They indicate which columns to examine when searching for processes matching the search conditions mysql utilities command proc ID mysql utilities command proc USER mysql utilities command proc HOST mysql utilities command proc DB e mysql utilities command proc COMMAND mysql utilities command proc TIME mysql utilities command proc STATE mysql utilities command proc INFO The following const
352. mmand line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt dump d Display the GRANT statements to create the account rather than executing them In this case the utility does not connect to the destination server and no destination option is needed format lt list_format gt f lt list_format gt Specify the user display format Permitted format values are grid csv tab and vertical The default is grid This option is valid only if l ist is given force Drop the new user account if it exists before creating the new account Without this option it is an error to try to create an account that already exists include global privileges Include privileges that match base_user as well as base_user host list List all users on the source server With this option a destination server need not be specified quiet q Turn off all messages for quiet execution Source lt source gt 192 NOTES Connection information for the source server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e
353. mple NSERT INTO util_test t1 VALUES 06 Test Basic database example NSERT INTO util_test t1 VALUES 07 Test Basic database example Te donek r r Connecting to the current server s master CHANGE MASTER TO MASTER HOST localhost MASTER_USER rpl MASTER_PASSWORD rpl MASTER_PORT 3310 MASTER_LOG_FILE clone bin 000001 MASTER_LOG_POS 1739 Starting slave START SLAVE PERMISSIONS REQUIRED The user must have permission to read all databases Since we are using the root account for these examples and you typically would permissions are not generally a problem 5 6 mysqldbimport Import Object Definitions or Data into a Database 102 mysqldbimport Import Object Definitions or Data into a Database This utility imports metadata object definitions or data or both for one or more databases from one or more files If an object exists on the destination server with the same name as an imported object it is dropped first before importing the new object To skip objects by type use the skip option with a list of the objects to skip This enables you to extract a particular set of objects say for importing only events by excluding all other types Similarly to skip creation of UPDATE statements for BLOB data specify the skip blobs option To specify the input format use one of the following values with the format option These correspond to the outpu
354. mplete Replication Topology Health 4 4 4 4 4 4 host jexouere role state gtid_mode health 4 4 4 4 4 4 serverl SSi MASTER WE ON OK server2 S32 SLAVE WE ON OR server3 S343 SLAVE WE ON OK server4 3314 SLAVE WE ON OK server5 S35 SLAVE WE ON OR 4 4 4 4 4 4 done The initial replication topology is now restored and mysql failover can be restarted but using force as initially shell gt mysqlfailover master root server1 3311 slaves root server2 3312 root server3 3313 root server4 3314 server5 3315 log log txt rpl user rpl rpl force Checking privileges MySQL Replication Failover Utility Failover Mode auto Next Interval Sat dul 27 O2s17s12 2013 Master Information 41 Restore the Previous Master After Failover Binary Log File Position Binleg_Deo_DB Binlog_Igqnore_DB master bin 000002 151 GTID Executed Set None Replication Health Status 4 4 4 4 host port role state gtid_mode health 4 4 4 serverl Ssialal MASTER UP ON OE server2 332 SLAVE UP ON OES server
355. mport objects into an existing database the result depends on the import format If the format is definitions or both an error occurs unless drop first is given If the format is data imported table data is added to existing table data e max bulk insert Specify the maximum number of INSERT statements to bulk by default 30000 This option is only used with bulk insert e multiprocess Specify the number of processes to concurrently import the specified files Special values 0 number of processes equal to the number of detected CPUs and 1 default no concurrency Multiprocessing works at the files level for any operating systems e new storage engine lt new_engine gt The engine to use for all tables created on the destination MySQL server e no headers h Input does not contain column headers This option only applies to the csv and tab file formats e quiet q Turn off all messages for quiet execution e server lt server gt Connection information for the server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file enc
356. n Windows POSIX systems no headers h Do not display column headers This option applies only for csv and tab output output file Specify the path and file name to store the generated export output By default the standard output is used no file quiet q Turn off all messages for quiet execution regexp basic regexp G 97 OPTIONS Perform pattern matches using the REGEXP operator The default is to use LIKE for matching rpl lt rpl_mode gt replication lt rpl_mode gt Include replication information Permitted values are master make destination a slave of the source server slave make destination a slave of the same master as the source only works if the source server is a slave and both include the master and slave options where applicable rpl file RPL_FILE replication file RPL_FILE The path and file name where the generated replication information should be written Valid only with the rp1 option rpl user lt replication_user gt The user and password for the replication user requirement in the format lt user gt lt password gt or lt login path gt For example rp1 passwd The default is None server lt server gt Connection information for the server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this informa
357. n every Shard eccecceeeeeeeeeeeeeeeaeaeeeeeeeeeeeeeaeaaes 280 8 9 4 How many MySQL Servers can scale tO ccceceeeee eee eeeeeeeeeeeeeae aaa neeeeeeeeeaeaaeaneeeeeeeeeaeaaea 281 8 9 5 Can MySQL Fabric introduce contention or deadlock ccccceceseeeeeeeeeeeeeeeaaeeeeeeaaeeeeeeaaeeeees 281 8 9 6 What happens when my data set or usage grows and a shard grows too big aace 281 8 9 7 Is there extra latency when using MySQL Fabric ccccceeeceeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaaaneneeeees 281 8 9 8 Why does MySQL Fabric route using connector logic rather than via a proxy essees 281 8 9 9 What is the difference between a shard key and a shard identifier ecceseeeeeeeeeeeeeee eee 281 8 9 10 Does my application need to change when a shard is moved to a different MySQL Server or split into multiple Shards ccccceceee cece cece eeeeee eee e ee aaa teeter eens ee aa eae ee eeeeeeeeaaaaaeeeeeeeeeeeaaaaeeeees 281 8 9 11 Is it possible to perform cross Shard UNIONS or JOINS ccceeececeeeeeeceeeeeeeeeaaeeeeeesaeeeeeeaaenees 282 8 9 12 Is the routing of queries and transactions transparent to my application c ceeeeeeeee eee 282 8 9 1 How is scaling achieved with MySQL Fabric Horizontal scaling is achieved by partitioning sharding the data from a table across multiple MySQL Servers or HA Groups In that way each server or group will contain a subset of the rows from a
358. n of GTID statements during the copy operation all Copy all of the databases on the server skip lt objects gt Specify objects to skip in the operation as a comma separated list no spaces Permitted values are CREATE_DB DATA EVENTS FUNCTIONS GRANTS PROCEDURES TABLES TRIGGERS and VIEWS Source lt SOurce gt 90 NOTES Connection information for the source server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt e ssl ca The path to a file that contains a list of trusted SSL CAs e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key file to use for establishi
359. n off all messages except the CREATE statement and warnings or errors use the quiet option Use the show stats option to see file statistics for each frm file If you need to run the utility with elevated privileges use the user option to execute the spawned server using a normal user account 130 EXAMPLES If you encounter connection or similar errors when running in default mode re run the command with the verbose option and view the output from the spawned server and repair any errors in launching the server If mysql frm fails in the middle you may need to manually shutdown the server on the port specified with port EXAMPLES The following example will read a single frm file in the default mode from the current working directory using the server installed in usr local bin mysql1 and port 3333 for the spawned server Notice the use of the db table frm format for specifying the database name for the table The database name appears to the left of and the frm name to the right In this case we have database test1 and table city so the CREATE statement reads CREATE TABLE testl city shell gt mysqlfrm basedir usr local bin mysql test1l city frm port 3333 Starting the spawned server on port 3333 done Reading frm files Reading the city frm file CREATE statement for city frm CREATE TABLE testli erity city_id smallint 5 unsigned NOT NULL AUTO_INCR
360. n the employees database on the local server to see the possible redundant and duplicate indexes use this command shell gt mysqlindexcheck server root localhost employees Source on localhost connected The following indexes are duplicates or redundant for table employees dept_emp CREATE INDEX emp_no ON employees dept_emp emp_no USING BTREE may be redundant or duplicate of ALTER TABLE employees dept_emp ADD PRIMARY KEY emp_no dept_no The following indexes are duplicates or redundant for table employees dept_manager 135 PERMISSIONS REQUIRED CREATE INDEX emp_no ON employees dept_manager emp_no USING BTREE may be redundant or duplicate of ALTER TABLE employees dept_manager ADD PRIMARY KEY emp_no dept_no The following indexes are duplicates or redundant for table employees salaries CREATE INDEX emp_no ON employees salaries emp_no USING BTREE may be redundant or duplicate of ALTER TABLE employees salaries ADD PRIMARY KEY emp_no from_date The following indexes are duplicates or redundant for table employees titles CREATE INDEX emp_no ON employees titles emp_no USING BTREE may be redundant or duplicate of ALTER TABLE employees titles ADD PRIMARY KEY emp_no title from_date PERMISSIONS REQUIRED Regarding the privileges needed to run this utility the user needs SELECT privilege on the mysql database as well as for the databases which tables are being checked
361. n the master fails auto automatically fail to best slave elect fail to candidate list or if no candidate meets criteria fail fail take no action and stop when master fails Default auto force Override the registration check on master for multiple instances of the console monitoring the same master See notes interval lt seconds gt i lt seconds gt Interval in seconds for polling the master for failure and reporting health Default 15 seconds Minimum is 5 seconds log lt log_file gt Specify a log file to use for logging messages log age lt days gt Specify maximum age of log entries in days Entries older than this will be purged on startup Default 7 days master lt connection gt Connection information for the master server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this 121 OPTIONS information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt c
362. n to MySQL Utilities What are the MySQL Utilities It is a package of utilities that are used for maintenance and administration of MySQL servers These utilities encapsulate a set of primitive commands and bundles them so they can be used to perform macro operations with a single command The utilities are written in Python available under the GPLv2 license and are extendable using the supplied library They are designed to work with Python versions 2 6 or later and there is no support yet for Python v3 1 How do we access the MySQL Utilities The MySQL Utilities are command line scripts which by default are available in your system s PATH Alternatively you can access their location from MySQL Workbench by selecting Tools from the main menu and then Start Shell for MySQL Utilities This opens a terminal shell window in the mysqluc utility shell Type help to list the available commands Connecting to MySQL Servers Figure 2 1 Starting MySQL Utilities from Workbench MySQL Workbench fe Local instance MySQL56_1 x File Edit View Query Database Server Tools Scripting Help E HEIENE z Browse Audit Log File Configuration Navigator SCHEMAS Utilities Start Shell for MySQL Utilities Q Filter objects gt sakila gt test gt world MEUG Schemas Information Object Info EESAN You can launch any of the utilities listed by typing the name of the command To find out what options are availabl
363. n user must have the appropriate permissions for the utility to check servers and monitor their status e g SHOW SLAVE STATUS SHOW MASTER STATUS The user must also have permissions to execute the failover procedure e g STOP SLAVE START SLAVE WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS CHANGE MASTER TO Lastly the user must have the REPLICATE SLAVE privilege for slaves to connect to their master The same permissions are required by the failover utility for master and slaves in order to run successfully In particular users connected to slaves candidates and master require SUPER GRANT OPTION REPLICATION SLAVE RELOAD DROP CREATE INSERT and SELECT privileges The DROP CREATE INSERT and SELECT privileges are required to register the failover instance on the initial master or the new master after a successful failover Therefore since any slave can become the new master slaves and candidates also require those privileges The utility checks permissions for the master slaves and candidates at startup At startup the console will attempt to register itself with the master If another console is already registered and the failover mode is auto or elect the console will be blocked from running failover When a console quits it unregisters itself from the master If this process is broken the user may override the registration check by using the force option Mixing IP and hostnames is not recommended The replication specific
364. names and the partition names if it is a partitioned table Constants The following constants denote the object types that can be searched e mysql utilities command grep ROUTINE e mysql utilities command grep EVENT e mysql utilities command grep TRIGGER mysql utilities command grep TABLE e mysql utilities command grep DATABASE e mysql utilities command grep VIEW e mysql utilities command grep USER The following constant is a sequence of all the object types that are available It can be used to generate a version independent list of object types that can be searched for example options and help texts mysql utilities command grep OBJECT_TYPES Classes class mysql utilities command grep ObjectGrep pattern database_pattern None types OBJECT_TYPES check_body False use_regexp False Search MySQL server instances for objects where the name or content for routines triggers or events matches a given pattern 204 mysql utilities command proc Search Processes on Servers sql string Return the SQL code for executing the search in the form of a SELECT statement Returns SQL code for executing the operation specified by the options Return type string execute connections output sys output connector mysql connector Execute the search on each of the connections in turn and print an aggregate of the result as a grid table Para
365. nd it will also automatically select and install the required prerequisites See the MySQL Installer manual for additional details e Standalone Download and execute the MySQL Utilities standalone MSI file Note R MySQL Utilities requires Connector Python to be installed For additional information see Section 1 1 Prerequisites OS X 1 6 OS X The dmg file available for OS X is built for x84 64 bit platforms and supports OS X version 10 7 Lion and newer After you download the dmg file install MySQL Utilities by opening it and double clicking the pkg file Note R MySQL Utilities requires Connector Python to be installed For additional information see Section 1 1 Prerequisites Chapter 2 Introduction Table of Contents 2 1 Introduction to MySQL Utilities 20 0 eect cece ee ee eee etter rete ee eae ete ee ee ee ae aaaaeeeeeeeeeaeaaaaeeeeees 5 2 2 Connecting to MySQL Servers ceceeceeee cece cette eect etter eee ee ee eee reer ee ee edad eeeeee seas aaaaneeeeeeeeeeeaaa 6 2 2 1 Connection Parameters cocirier aaia iea aE saadeds dade da TE das 6 2 2 2 Specifying Connections in Python Library 0 cccceceeeceeeeeeeeeeeee tees aa ee eeeeeeeeeeeeaaaaeeeeeeeeeeeeaea 9 This chapter introduces MySQL Utilities and presents information on how to access and download MySQL Utilities It also includes the basics of how to use the account login option common to all utilities 2 1 Introductio
366. nd parser parser optparse OptionParser version VERSION_FRM format program os path basename sys argv 0 description DESCRIPTION usage USAGE add_help_option False parser add_option help action help Setup utility specific options Connection information for the source server parser add_option server action store dest server type sString default root localhost 3306 help connection information for original server in the form lt user gt lt password gt lt host gt lt port gt lt socket gt Data directory for new instance parser add_option new data action store dest new_data type sString help the full path to the location of the data directory for the new instance Port for the new instance parser add_option new port action store dest new_port type sString default 3307 help the new port for the new instance default default Server id for the new instance parser add_option new id action store dest new_id type sString default 2 help the server_id for the new instance default default List of databases parser add_option d databases action store dest dbs_to_copy type string help comma separated list of databases to include in the copy omit for all databases default None List of users parser add_option u users action store dest users_to_copy t
367. nector Python with MySQL Fabric Support e Class fabric Fabric Manages the connection with a MySQL Fabric node used by MySQLFabricConnection e Other helper classes for caching information 8 7 1 Installing Connector Python with MySQL Fabric Support Fabric support in Connector Python requires version 1 2 0 or greater Downloads are available at http dev mysql com downloads connector python in various packages such as Zip archives compressed tar archives RPM packages Debian packages and Windows Installer packages Using the compressed t ar package you can install MySQL Connector Python as follows shell gt tar xzf mysql connector python 1 2 3 tar gz shell gt cd mysql connector python 1 2 3 shell gt python setup py install For more information see Connector Python Installation 8 7 2 Requesting a Fabric Connection The modules related to Fabric are under mysql connector fabric Importing fabric provides access to everything needed to use Fabric import mysql connector from mysql connector import fabric Traditionally a MySQL connection is set up using the mysql connector connect method using the connection arguments described at Connector Python Connection Arguments and the connection is established immediately A request for a Fabric connection by contrast does not immediately connect Instead pass a fabric argument to the connect call This argument must be a dictionary When Fabric connects to the MyS
368. neral query log relay log and InnoDB tablespaces The default is to only show database disk usage If the command line lists no databases the utility shows the disk space usage for all databases Sizes displayed without a unit indicator such as MB are in bytes The utility determines the location of the data directory by requesting it from the server For a local server the utility obtains size information directly from files in the data directory and InnoDB home directory In this case you must have file system access to read those directories Disk space usage shown includes the sum of all storage engine specific files such as the MYI and MYD files for MyISAM and the tablespace files for InnoDB If the file system read fails or if the server is not local the utility cannot determine exact file sizes It is limited to information that can be obtained from the system tables which therefore should be considered an estimate For information read from the server the account used to connect to the server must have the appropriate permissions to read any objects accessed during the operation If information requested requires file system access but is not available that way the utility prints a message that the information is not accessible This occurs for example if you request log usage but the server is not local and the log files cannot be examined directly To specify how to display output use one of the following values with the
369. ng Connector J with MySQL Fabric 0 ccccecceeeeeeeeeeeee ee ae eeeeeeeeeeeeeeseaaaaeeeeeeeeeaeaaaaenees 261 8 8 1 Installing Connector J with MySQL Fabric Support cccceeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaee 262 8 8 2 Loading the Driver and Requesting a Fabric Connection ccceeeeeeseeeeeeeeeeeeeeaee 262 8 8 3 Providing Information to Choose a MySQL Server ccccceeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaea 263 8 8 4 MySQL Fabric Configuration for Running Samples ceeeeeeeeeeeeeeeeeeeeeeeeeeeeaes 264 B 6 5 RUNNING T OSTS ren a tanec es Guedes toutes asus ded E E ad reunion ens 266 8 8 6 Running Demonstration Programs ccecceeeeeeeenneeeeee ae eeeeee aa eeee aa ERKAN 266 8 8 7 A Complete Example Working with Employee Data ccceeeeeeeeeeeeeeeeeeeeeeeeaaes 267 8 8 8 How Connector J Chooses a MySQL Server 00 cccceeeeeeeeeceeeeeeeeeeeeseaaeaeeeeeeeeeeeeaea 271 8 8 9 Using Hibernate with MySQL Fabric ccecceececenceeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaeaaeaneeeeeees 272 8 8 10 Connector J Fabric Support Reference 0 c ccceceeeaeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaaaenees 275 8 9 MySQL Fabric Frequently Asked QUESTIONS 0 cccceeeeeeeeeeeeeeeee tees ae eaeeeeeeeeeeeaaaaeeeeeeeeeeeeaaa 276 D ADPOMOIX E A A EAE E E E E E E beet sae cuee cian N T E A 283 9 1 MySQL Utilities Frequently Asked Questions 0 cceceeeeeeeeeeeee ee ee ee eeteeeeeeeeeaeaaeaeeneeeeeeeeeaea
370. ng a secure connection e ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required e verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects in the operation On the source to copy all objects from the database the user must have these privileges SELECT for tables SHOW VIEW for views EVENT for events and TRIGGER for triggers Additionally the SELECT privilege is also required for the mysq1 database On the destination to copy all objects the user must have these privileges CREATE ALTER SELECT INSERT UPDATE LOCK TABLES DROP if drop first option is used SUPER when binary logging is enabled CREATE VIEW for views CREATE ROUTINE EXECUTE for procedures and functions EVENT for events TRIGGER for triggers and GRANT OPTION to copy grants The SUPER privilege might also be required for some objects views procedures functions events and triggers depending on their DEFINER value 91 EXAMPLES Actual privileges required may differ from installation to installat
371. not process any kind of user s request server set_status Seta server s status which determines whether it is primary secondary spare or faulty 8 4 5 Sharding Commands The sharding commands are used to define modify and control sharding e sharding move_shard Move the shard represented by the shard_id to the destination group sharding sharding sharding sharding sharding sharding sharding sharding sharding sharding lookup_servers Lookup a shard based on the give sharding key disable_shard Disable a shard remove_definition Remove the shard mapping definition list_tables Return all the shard mapping definitions of a particular sharding type add_table Add a table to a shard mapping definition add_shard Add a shard list_definitions Lists all the shard mapping definitions enable_shard Enable a shard remove_shard Remove a shard prune_shard Given the table name prune the table according to the defined sharding specification for the table sharding sharding sharding sharding lookup_table Fetch the shard specification mapping for the given table split_shara Split the shard represented by the shard_id into the destination group create_definition Create a shard mapping definition remove_table Remove table from the shard mapping definition 8 4 6 Event Commands The event commands are used to define events for tailoring the Fabric system to your needs for controlling applicatio
372. ns event trigger Trigger an event event wait_for_procedures Wait until procedures finish executing Procedures are identified through their UUID in a list and separated by comma 8 4 7 Dump Commands The dump commands are designed to be used by the connectors to retrieve information on shards high availability groups and their servers dump shard_maps Return information about all shard mappings matching any of the provided patterns 240 Backing Store e dump shard_index Return information about the index for all mappings matching any of the patterns provided dump servers Return information about all servers e dump sharding_information Return all the sharding information about the tables passed as patterns e dump shard_tables Return information about all tables belonging to mappings matching any of the provided patterns dump fabric_nodes Return a list of Fabric nodes 8 5 Backing Store The backing store feature requires a MySQL instance This server should be the same version as your other servers and MySQL version 5 6 10 or later This section explains how to set up the backing store and provides information about some of the tables created To set up the backing store use the mysqli fabric command The param options specify the user and password we created in Section 8 2 3 1 Create the Associated MySQL Users so that the utility can connect to the backing store and create the d
373. ns or Data into a Database ce eececeeeeeeeeeaeeeees 102 5 7 mysqldiff Identify Differences Among Database Objects cccceceeceeeeeeeeeeeeeeaeaaeeeeeeeeees 108 5 8 mysqidiskusage Show Database Disk Usage cccccccceeececeeeeeeeeeeeeeeeeaeeeeeeeaaeeeeeeaaeeeeees 114 5 9 mysql failover Automatic replication health monitoring and failover seseeeeeeeee eee 117 5 10 mysqlfrm File reader for frm files 00 ccccecccccceeeeececeeeeececeeeeeeeaeaeeeeaaeeeeesageeeesaeeeeeeaaaees 128 5 11 mysqlindexcheck Identify Potentially Redundant Table Indexes ccccceeeeeeeeeeeeeeeeeees 132 5 12 mysqlmetagrep Search Database Object Definitions cece ecccceseeeeeeeaeeeeeeaaeeeeeeaaeeees 136 5 13 mysqlprocgrep Search Server Process Lists ccccseeeececeeeeeeeceeeeeeeceeeeeeeeeeeeeeeseeneeeesnees 140 5 14 mysqlreplicate Set Up and Start Replication Between Two Servers ccccsseeeeeeneeeees 144 5 15 mysqlrplms Set Up and Start Replication Among a Slave and Multiple Masters 149 5 16 mysqirpladmin Administration utility for MySQL replication ecceeeeeeceeeeeeeeeeeeeeeaeeaeeees 154 5 17 mysqirplcheck Check Replication Prerequisites cccccccecseeceeeaeeeeeeaaeeeeeeaaeeeeeeaaeneees 164 5 18 mysqlrplshow Show Slaves for Master Server c ccccceceeeececeeeeeeeseeeeeeeseeeeseeseeeeeeeeeees 169
374. nt of information For example v verbose vv more verbose vvv debug version Display version information and exit NOTES The login user must have the appropriate permissions to create new objects access read the mysql database and grant privileges If a database to be imported already exists the user must have read permission for it which is needed to check the existence of objects in the database 106 EXAMPLES Actual privileges needed may differ from installation to installation depending on the security privileges present and whether the database contains certain objects such as views or events and whether binary logging is enabled Some combinations of the options may result in errors during the operation For example excluding tables but not views may result in an error when a view is imported The new storag ngine and default storage engine options apply to all destination tables in the operation For the format and import options the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value When importing data and including the GTID commands you may encounter an error similar to GTID_PURGED can only be set when GTID_ EXECUTED is empty This occurs because the destination server is not in a clean replicati
375. o make the definitions of objects on server2 like the definitions of the corresponding objects on server1 The default direction is server1 You must provide connection parameters user host password and so forth for an account that has the appropriate privileges to access all objects in the operation If the utility is to be run on a server that has binary logging enabled and you do not want the comparison steps logged use the disable binary logging option OPTIONS mysqldbcompare accepts the following command line options all a Compare all database in common with the same name between two servers The a11 option ignores the following databases INFORMATION _SCHEMA PERFORMANCE_SCHEMA mysql and sys Note R The sys database is ignored as of Utilities 1 5 5 e help Display a help message and exit e license Display license information and exit e changes for lt direction gt 81 OPTIONS Specify the server to show transformations to match the other server For example to see the transformation for transforming object definitions on server1 to match the corresponding definitions on server2 use changes for server1 Permitted values are serverl and server2 The default is server character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client difftype lt difftype gt d lt difftype gt Specify the differenc
376. o see if the server is alive via a ping operation followed by a check of the connector to detect if the server is still reachable The ping operation can be controlled with the ping option see below If the master is found to be offline or unreachable the utility will execute one of the following actions based on the failover mode option value The available values are auto default Execute automatic failover to the list of candidates first and if no slaves are viable continue to locate a viable candidate from the list of slaves If no slaves are found to be a viable candidate the utility will generate and error and exit Once a candidate is found the utility will conduct failover to the best slave The command will test each candidate slave listed for the prerequisites Once a candidate slave is elected it is made a slave of each of the other slaves thereby collecting any transactions executed on other slaves but not the candidate In this way the candidate becomes the most up to date slave elect This mode is the same as auto except if no candidates specified in the list of candidate slaves are viable then it does not check the remaining slaves and instead generates an error and then exits fail This mode produces an error and does not failover when the master is downed This mode is used to provide periodic health monitoring without the failover action taken For all options that permit specifying multiple servers the opt
377. o the mysql database You should never use the root user to spawn the server nor should you use the mysql user when spawning the server or running the utility 5 11 mysqlindexcheck Identify Potentially Redundant Table Indexes This utility reads the indexes for one or more tables and identifies duplicate and potentially redundant indexes To check all tables in a database only specify the database name To check a specific table name the table in db table format It is possible to mix database and table names You can scan tables in any database except the internal databases mysql INFORMATION_SCHEMA and performance_schema 132 mysqlindexcheck Identify Potentially Redundant Table Indexes Depending on the index type the utility applies the following rules to compare indexes designated as idx_a and idx_b e BTREE idx_b is redundant to idx_a if and only if all the columns from idx_b are a prefix of idx_a Order and uniqueness count e HASH idx_a and idx_b are redundant if they are duplicates i e if and only if they contain the same columns in the same order e SPATIAL idx_a and idx_b are duplicates if and only if they contain the same column only one column is permitted e FULLTEXT idx_b is redundant to idx_a if and only if all columns in idx_b are included in idx_a Order does not count To see DROP statements drop redundant indexes specify the show drops option To examine the existing inde
378. ocalhost connected Database totals db_name db_dir_size data_size misc_files total test 070 070 elas 0 07 C A 602 0 070 eloi 0 0 0 0 backup_test 19410 1117 18293 19410 employees 242519463 205979648 242519463 448499111 mysql 867211 657669 191720 849389 t1 9849 1024 8825 9849 best 6162 4096 92066 56162 Wet rest er OSAS 2048 LIST L625 Ub ese mo lise Op leiS An ely saan Get rest Cr 19525 2048 MISTS L623 Total database disk usage 449 490 516 bytes or 428 00 MB Log information The general_log is turned off on the server The slow_query_log is turned off on the server binary log information Current binary log file mysql bin 000076 log_file size data mysql bin 000076 125 data mysql bin 000077 125 data mysql bin 000078 556 jdata mysoql bin 000079 168398223 data mysql bin index 76 Total size of binary logs 168 399 105 bytes or 160 00 MB Server is not an active slave no relay log information InnoDB tablespace information InnoDB_file size type specification data ib_logfile0 5242880 log file data ib_logfilel 5242880 log file data ibdatal 220200960 shared tablespace ibdatal 210M data ibdata2 10485760 shared tablespace ibdata2 10M autoextend data employees departments ibd 114688 file tablespace data employees dept_emp ibd 30408704 file tablespace data employees dept_manager ibd 131072 file tablespace data employees employees ibd 23068672 file
379. od 755 usr lib ocf resource d heartbeat mysql fabric 8 6 4 3 Configure DRBD If your nodes do not already have an empty partition that you plan to use for the DRBD then create one If you are using a Virtual Machine you can add a new storage to your machine These details go beyond the scope of this guide This partition will be used as a resource managed and synchronized between nodes by DRBD in order for DRBD to be able to do this a new configuration file in this case called clusterdb_res res must be created in the etc drbd d directory the contents should look similar to resource clusterdb_res prorocolie handlers pri on incon degr usr lib drbd notify pri on incon degr sh usr lib drbd notify emergency reboot sh ec pPri lost atter sb usr lib drbd notify pri lost atter so sh usr lib drod notify emergency reboot sh ec jecal io error usr lib drod notity io error sh usr lib drbd notify emergency shutdown sh echo o gt pr fence peer usr lib drbd crm fence peer sh startup degr wie timeout 120 2 minutes outdated wfc timeout 2 2 seconds disk on io error detach net cram nmacn a iome na 9 shared secret clusterdb after sb Opri disconnect aiter sb lpri disconnect after sb 2pri disconnect Pr coniiice cisconnect syncer rate 10M al extents 257 on no data accessible io error im ineysre il device dev drbd0 disk dev scp address A
380. odule includes classes and methods from one or more common modules where the abstract objects and method groups are kept The common modules reside in the mysql utilities common folder of the source tree The following illustrates this arrangement using the mysqlserverinfo utility scripts mysqlserverinfo py mysql utilities command serverinfo py PA mysql utilities common options py ON mysql utilities common server py S mysql utilities common tools py 195 The MySQL Utilities Library e mysql utilities common format py Each utility script is designed to process the user input and option settings and pass them on to the command module Thus the script contains only such logic for managing and validating options The work of the operation resides in the command module Command modules are designed to be used from other Python applications For example one could call the methods in the serverinfo py module from another Python script This enables developers to create their own interfaces to the utilities It also permits developers to combine several utilities to form a macro level utility tailored to a specified need For example if there is a need to gather server information as well as disk usage it is possible to import the serverinfo py and diskusage py modules and create a new utility that performs both operations Common modules are the heart of the MySQL Utilities library These modules contain classes that
381. of mysqlserverclone The following is an example of the running command shell gt mysqlserverclone basedir C MySQOL mysql 5 6 15 winx64 new data C MySQL instance_3307 new port 3307 root password pass mysqld ssl ca C newcerts cacert pem ssl cert C newcerts server cert pem ssl key C newcerts server key pem Cloning the MySQL server located at C MySQL mysql 5 6 15 winx6 4 Creating new data directory Configuring new instance mocaitan quem S cll oolksreess Setting up empty database and mysql tables Starting new instance of the server Testing connection to new instance Success Setting the root password Connection Information uroot ppass port 3307 me doner Now we have a new MySQL server instance and you can confirm the use of the given SSL certificates with the MySQL Command Line Tool by executing the command show variables like ssl shell gt mysql uroot ppass port 3307 e show variables like ss1 4 4 Variable_name Value 4 4 have_openssl YES have ssl ves ssl_ca C newcerts cacert pem 52 How Can Use a secure encrypted connection between Utilities and a MySQL Server ssl_capath ssl_cert ssl_cipher ssl cri ssl crlipath ssl_key C newcerts server key pem 4
382. of database servers managed by MySQL Fabric APIs necessary to implement MultiTenantConnectionProvider We can use internal APIs included with Connector J with MySQL Fabric support to implement Hibernate s MultiTenantConnectionProvider The following implementation is included in the package as the com mysql fabric hibernate FabricMultiTenantConnectionProvider class An example of how to use it is included as the class demo fabric HibernateFabric To implement MultiTenantConnectionProvider we use the com mysql fabric FabricConnection class This class connects to the MySQL Fabric manager to obtain information about servers and data sharding This is an internal API and subject to change The following FabricConnection methods can be used FabricConnection String url String username String password throws FabricCommunicationException Construct a new instance of a MySQL Fabric client and initiate the connection ServerGroup getServerGroup String serverGroupName throws FabricCommunicationException Retrieve an object representing the named server group This includes the list of servers in the group including their mode read only or read write ShardMapping getShardMapping String database String table throws FabricCommunicationException Retrieve an object represending a shard mapping for the given database and table The ShardMapping indicates the global group and the individual shards The following additional methods
383. og_error_file_size slow_query_log OFF slow_query_log_file slow_query_log_file_size T row odone 2 2 1 3 Command line Options The least secure way to provide connection information for MySQL servers is to specify the data on the command line option This is least secure because the data is visible on the command line and will also be visible in any log or redirection of the execution Specifying Connections in Python Library In this case we specify the data in the following order lt users lt passwa gt lt host gt lt port gt lt socket where the passwd port and socket are optional Each item is described in more detail below e user The name of the user to connect passwd The password to use when connecting The default if no password is supplied is the empty password e host The domain name of the host or the IP address This field accepts host names and IPv4 and IPv6 addresses It also accepts quoted values which are not validated and passed directly to the calling methods This enables users to specify host names and IP addresses that are outside of the supported validation mechanisms port The port to use when connecting to the server The default if no port is supplied is 3306 which is the default port for the MySQL server as well unix_socket The socket to connect to instead of using the host and port parameters The following demonstrates executing a utility using command line opt
384. oice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection SSl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required Start from beginning b Start replication at the beginning of events logged in the master binary log switchover interval lt seconds gt Interval in seconds for switching masters Default 60 seconds Minimum is 30 seconds pidfile lt pidfile gt Pidfile for running mysqlrplms as a daemon This file contains the PID process identifier that uniquely identify a process It is needed to identify and control the process forked by mysqlr
385. on demand command The available actions include the following copy This command copies the audit log specified by audit log name to the destination path specified by copy to The remote login option can be used to copy log files from a remote location Note the destination path must be locally accessible by the current user 65 OPTIONS policy The policy command is used to change the audit logging policy The accepted values are the following which are set using the value option Note R The server option is also required to execute this command Starting from MySQL server 5 6 20 and 5 7 5 the value is read only for the audit_log_policy variable Now the policy results from the combination of two new variables audit_log_connection_policy and audit_log_statement_policy This change is supported starting from MySQL Utilities 1 5 2 ALL log all events NONE log nothing LOGINS only log login events QUERIES only log query events DEFAULT sets the default log policy rotate_on_size This command sets the file size threshold for automatic rotation of the audit log the audit_log_rotate_on_size variable The value is set using the value option and must be in the range 0 4294967295 This command also requires the server option to be specified Note if the variable is set with a value that is not a multiple of 4096 then it is truncated to the nearest multiple
386. on state To solve this problem you can issue a RESET MASTER command on the destination prior to executing the import The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file my login cnf Keep in mind that you can only take advantage of multiprocessing if your system has multiple CPUs available for concurrent execution Also note that multiprocessing is applied at the file level for the mysqldbimport utility which means that only different files can be concurrently imported EXAMPLES To import the metadata from the ut il_test database to the server on the local host using a file in CSV format use this command shell gt mysqldbimport server root localhost import definitions format csv data csv Source on localhost connected Importing definitions from data csv done Similarly to import the data from the ut i1_test database to the server on the local host importing the data using bulk insert statements use this command shell gt mysqldbimport server root localhost import data bulk insert format csv data csv Source on localhost connected Importing data from data csv done To import both data and definitions from the ut il_test database im
387. on we specify the server that will become the new master We can not forget the demote master option which turns the old master into a slave otherwise it would still behave as a master just without any slaves Server will become a slave of Server2 After the switchover Server2 becomes the master server for both Server1 and Server3 which are now the slaves Permissions Required The m_account user needs the following privileges for the mysqlreplicate SELECT and INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION As for the slave_acc users they need the SUPER privilege The repl user used as the argument for the rpl user option is either created automatically or if it exists it needs the REPLICATION SLAVE privilege To run the mysqlrpladmin utility with the health command the m_account used on the master needs an extra SUPER privilege As for the switchover command all the users need the following privileges SUPER GRANT OPTION SELECT RELOAD DROP CREATE and REPLICATION SLAVE Tips and Tricks We can use the discover slaves login option for mysql rpladmin in order to detect the slaves automatically instead of manually specifying the slaves The mysgqirpladmin utility allows users to specify a script to execute before and after the failover and switchover operations using the xec before and xec after options respectively Note that the script specified using the exec after option
388. onfiguration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt max position lt position gt Used to detect slave delay The maximum difference between the master s log position and the slave s reported read position of the master A value greater than this means the slave is too far behind the master Default 0 pedantic p Used to stop failover if some inconsistencies are found such as errant transactions on slaves or SQL thread errors during server checks By default the utility only generates warnings if issues are found when checking a slave s status during failover and it will continue its execution unless this option is specified pidfile lt pidfile gt Pidfile for running mysql failover as a daemon This file contains the PID process identifier that uniquely identifies a process It is needed to identify and control the process forked by mysql failover ping lt number gt Number of ping attempts for detecting a downed server Default is 3 seconds Note R On some platforms this is the same as number of seconds to wait for ping to return report values lt report_values gt Report values used in mysqlfailover running as a daemon It can be health gtid or uuid Multiple values can be used separated by commas e health Display the replication health of the topology e gtid
389. only runs in case the switchover failover executes successfully We can use the mysqirpladmin utility to start and stop all the slaves with the start stop commands Using the stop command only stops servers that are actually slaves of the specified master thus preventing us from stopping unwanted servers 3 4 3 Setup Automatic Failover Once your replication topology is setup it is important to consider the possible occurrences of failures in order to maintain the high availability level of your system Several failures independently from their cause network connection issue hard drive crash cosmic lightning etc can stop the replication process by making the master no longer accessible by its slaves In this type of situation it is desirable to promote one of the slaves to master while the problem with the old master is being solve as it can take some considerable time It will be even better to have an application to monitor the replicate topology and automatically perform failover minimizing downtime and keeping replication running 37 Setup Automatic Failover Objectives The goal is to start the mysql failover utility to monitor a replication topology and perform failover automatically when required When the current master fails manually promoting a slave to the new master can be a very tedious and error prone task as all the remaining slave have to point out to the new master and the new master needs to catch up wi
390. oose a MySQL Server mysql connector connect method or MySOLConnection Class with the addition of the required fabric argument config fabricii host fabric example com TE user webuser password webpass fens fabric MySQLFabricConnection config Error Reporting Connector Python can report errors to Fabric that occur while accessing a MySQL instance The information can be used to update the backing store and trigger a failover operation provided that the instance is a primary server and Fabric has received a sufficient number of problem reports from different connectors e The fabric dictionary argument to the connect method accepts a report_errors value Its default value is False pass a value of True to enable error reporting to Fabric To define which errors to report use the ext ra_failure_report function from mysql connector fabric import extra_failure_report extra_failure_report error_code_0 error_code_l 8 7 3 Providing Information to Choose a MySQL Server If you create a Fabric connection without providing any information about which data to access the connection cannot function To access a database you must provide the driver with either of these types of information group The following discussion describes both ways of providing information You do this by setting one or more properties of the Fabric connection object using its set_property method so the
391. option is ignored for this command gtid This command is available to only those servers supporting GTIDs It displays the contents of the GTID variables GLOBAL GTID_EXECUTED GLOBAL GTID_PURGED and GLOBAL GTID_OWNED The command also displays universally unique identifiers UUIDs for all servers This command requires one of the following combinations master and slaves or master and discover slaves login health Display the replication health of the topology By default this includes the host name port role MASTER or SLAVE of the server state of the server UP is connected WARN not connected but can ping DOWN not connected and cannot ping the GTID_MODE and health state This command can be run with the following combination of options master and slaves master and discover slaves login e slaves Note R The health column will display no master specified when generating a health report for a collection of slaves and no master option specified The master health state is based on the following if GTID_MODE ON the server must have binary log enabled and there must exist a user with the REPLICATE SLAVE privilege The slave health state is based on the following the O_ THREAD and SQL_THREADS must be running it must be connected to the master there are no errors the slave delay for non gtid enabled scenarios is not more than the threshold provided by the max position an
392. opyright notice this list of conditions and the following disclaimer 2 Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution 3 All advertising materials mentioning features or use of this software must display the following acknowledgment This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit http www openssl org 4 The names OpenSSL Toolkit and OpenSSL Project must not be used to endorse or promote products derived from this software without prior written permission For written permission please contact openssl core openssl org 5 Products derived from this software may not be called OpenSSL nor may OpenSSL appear in their names without prior written permission of the OpenSSL Project 6 Redistributions of any form whatsoever must retain the following acknowledgment This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit http www openssl org THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT AS IS AND ANY EXPRESSED OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE MPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE OpenSSL PROJECT OR TS CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLAR
393. or more information on the sections and variables in the configuration file see Section 8 2 3 3 Configuration File Sections 8 2 3 3 Configuration File Sections The MySQL Fabric configuration file contains all the information necessary to run the MySQL Fabric utility In addition it serves as a configuration file for the utilities from within MySQL Fabric Each section has one or more variables defined that provide key information to the MySQL Fabric system libraries Note R The client section was removed in MySQL Fabric 1 5 5 1 6 2 Instead use the restore_user restore_password backup_user and backup_password under the servers section to configure users for the backup and restore utilities such as mysqldump and the mysql Client Section DEFAULT The DEFAULT section contains information on the installation paths for MySQL Fabric This section is generated as part of the installation and should normally not be modified prefix The installation prefix used when installing the mysql fabric package and the binaries sysconfdir The location of the system configuration files Normally located in the etc directory under the directory given in prefix but in some situations this might be different logdir Configures the directory where log files are located by default Normally the logging URL contains the absolute path but in the event that the path is relative it is relative to this directory Section storage
394. or Master Server This utility shows the replication slaves for a master It prints a graph of the master and its slaves labeling each with the host name and port number You must specify the discover slaves login option to provide the user name and password to discover any slaves in the topology To explore the slaves for each client use the recurse option This causes the utility to connect to each slave found and attempt to determine whether it has any slaves If slaves are found the process continues until the slave is found in the list of servers serving as masters a circular topology The graph displays the topology with successive indents A notation is made for circular topologies If you use the recurse option the utility attempts to connect to the slaves using the user name and password provided for the master By default if the connection attempt fails the utility throws an error and stops To change this behavior use the prompt option which permits the utility to prompt for the user name and password for each slave that fails to connect You can also use the num retries n option to reattempt a failed connection n times before the utility fails An example graph for a typical topology with relay slaves is shown here Replication Topology Graph localhost 3311 MASTER 169 OPTIONS T leeellaosicssSi Siw lheeallingstes ssl SLAVE t MASTER T locellsiosessSils Siw
395. or connect Tralee Vinge eg Wileeailaosie eerie g Sao 7d username admin password adminpass Ihe user webuser password webpass autocommit True conn set_property group my_group mode fabric MODE_READWRITE cur conn cursor cur execute CREATE DATABASE IF NOT EXISTS employees cur execute USE employees cur execute DROP TABLE IF EXISTS employees cur execute CREATE TABLE employees emp_no INT first_name CHAR 40 last_name CHAR 40 Wyn gtid_executed add_employee conn 12 John Doe find_employee conn 12 gtid_executed You can copy this code into a Python module named test_fabric_query py and execute it with the following command shell gt python test_fabric_query py wi Kola wi Dios Group Maintenance To find out which servers are in a group use this command shell gt mysqlfabric group lookup_servers my_group Command success True return status PRIMARY server_uuid bbe6f7cl b6c3 11le3 aaa2 58946b051f64 IMOde te READE WRITE uwenchit lO vaddress Nm localhosit So 0ii Ig ustatus TSECONDARY server _ uuid 0c9e67d0 8194 11e2 8a7c f0defl24dcc5 imode s EREADEONELY Vigealopye 2 lO Ysellaacs es Vile alnvocic oS S0le Ig tsta tus TSECONDARY server _ uuid 0c67e5b1 8194 1 1e2 8a7c f 0defl24dcc5 modere EREADEONEY Vigealeiye se lO stellascs es Vile alsvecic 9 oao I ac
396. or external scripts to trigger aborting the operation if result is greater than or equal to the threshold Default None no threshold checking seconds behind lt seconds gt Used to detect slave delay The maximum number of seconds behind the master permitted before slave is considered behind the master Default 0 slaves lt slave connections gt Connection information for slave servers List multiple slaves in comma separated list The list will be evaluated literally whereby each server is considered a slave to the master listed regardless if they are a slave of the master To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt ssl ca 159 NOTES
397. ores any slaves it cannot connect to or if the IO thread stopped it is not connected to the master The utility permits the user to demote a master to a slave during the switchover operation The demote master option tells the utility to once the new master is established make the old master a slave of the new master This permits rotation of the master role among a set of servers The utility permits the user to specify an external script to execute before and after the switchover and failover commands The user can specify these with the xec before and xec after options The return code of the script is used to determine success thus each script must report 0 Success to be considered successful If a script returns a value other than 0 the result code is presented in an error message The utility permits the user to log all actions taken during the commands The 1og option requires a valid path and file name of the file to use for logging operations The log is active only when this option is specified The option 1og age specifies the age in days that log entries are kept The default is seven 7 days Older entries are automatically deleted from the log file but only if the 10g option is specified The format of the log file includes the date and time of the event the level of the event informational INFO warning WARN error ERROR critical failure CRITICAL and the message reported by the utility The ut
398. ormat As we can see in the above example the mysqlbinlogmove utility displays the list of all files that are moved This helps to confirm that all files were relocated as expected Permissions Required The system user used to execute the utility must have read and write access to the source and destination directories in order to move the binary log files successfully Tips and Tricks By default the utility only moves binary log files Use the 10g t ype option with the value relay to move relay log files or all to move both binary log and relay log files The mysqibiniogmove utility can also be used to move some binary log files to a different location while the server is still running For example suppose you want to archive the binary log files or free some disk space on the server s partition In this case the server option should be used instead of the binlog dir option like in the above example The utility will ensure that the binary files that might be in use by the server those with the higher sequence number will not be moved The utility also provides options to filter the files to relocate based on their sequence number using the sequence option or their last modification date using the modified before option 3 1 2 How do you purge obsolete binary log files safely Important AN This documented task uses the mysqlbinlogpurge utility which was introduced in MySQL Utilities 1 6 0 At some poin
399. p help commands exit quit set lt variable gt lt value gt show options show variables Description Display list of alll utilities supported Display help for a specific utility Display errors captured during the execution of the utilities clear captured errors Display the last error captured during the execution of the utilities show titis list Exit the console Store a variable for recall in commands Display list of options specified by the user on launch Display list of variables lt ENTER gt Press ENTER to execute command lt ESCAPE gt Press ESCAPE to clear the command entry lt DOWN gt Press DOWN to retrieve the previous command lt UP gt Press UP to retrieve the next command in history lt TAB gt Press TAB for type completion of utility option or variable names lt TAB gt lt TAB gt Press TAB twice for list of matching type completion context sensitive mysqluc gt help utilities UEdlity Description mysqlauditadmin audit log maintenance utility mysqlauditgrep audit log search utility 26 What Options Are Used With Each Utility mysqldbcompare compare databases for consistency mysqldbcopy copy databases from one server to another mysqldbexport export metadata and data from databases mysqldbimport import metadata and data from files mysqldiff compare object definitions among objects where the difference is how dbl objl differs from db2 o0bj2 mysqldiskusage show
400. p fabric_testl_global user password Statement statement rawConnection createStatement statement executeUpdate create database if not exists employees statement close rawConnection close We should connect to the global group to run DDL statements they will be replicated to the server groups for all shards The 1 st way is to set its name explicitly via the fabricServerGroup connection property rawConnection DriverManager getConnection baseUrl database fabricServerGroup fabric_testl_global user password statement rawConnection createStatement statement executeUpdate create database if not exists employees statement close rawConnection close The 2 nd way is to get implicitly connected to global group when the shard key isn t provided ie set fabricShardTable connection property but don t set fabricShardKey rawConnection DriverManager getConnection baseUrl employees fabricShardTable employees employees user password At this point we have a connection to the global group for the employees employees shard mapping statement rawConnection createStatement statement executeUpdate drop table if exists employees statement executeUpdate create table employees emp_no int not null first_name varchar 50 last_name varchar 50 primary key emp_no i Za ANSert data Cast to a Fabric connection to have ac
401. ple lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt master info file lt file gt The name of the master information file on the slave The default is master info read from the data directory Note This option requires that you run the utility on the slave and that you have appropriate read access for the file quiet q Turn off all messages for quiet execution Note Errors and warnings are not suppressed show slave status s Display the values from SHOW SLAVE STATUS on the master slave lt source gt Connection information for the slave server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visi
402. plication between the master and slave is successful if and only if both servers have the same storage engines available the same default storage engine and the same InnoDB storage engine shell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl vv pedantic master on localhost connected slave on localhost connected master id 2 slave id 99 Checking InnoDB statistics for type and version conflicts Checking storage engines Checking for binary logging on master Setting up replication Flushing tables on master with read lock Connecting slave to master CHANGE MASTER TO MASTER_HOST omitted Starting slave status Waiting for master to send event error 0 Unlocking tables on master done The following command starts replication from the current position of the master which is the default shell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl master on localhost connected slave on localhost connected Checking for binary logging on master Setting up replication done Sh Sh SE OSE OH The following command starts replication from the beginning of recorded events on the master n hell gt mysqlreplicate master root localhost 3306 slave root localhost 3307 rpl user rpl rpl start from beginning master on localhost connected slave on lo
403. plished by typing the complete command and pressing ENTER The user does not have to type python or provide the py file extension The console will add these if needed The user can also run commands using the option execute The value for this option is a semi colon separated list of commands to execute These can be base commands or MySQL Utility commands The console will execute each command and display the output All commands to be run by the console must appear inside a quoted string and separated by semi colons Commands outside of the quoted string will be treated as arguments for the mysqluc utility itself and thus ignored for execution 188 OPTIONS commands at the point of failure Commands may also be piped into the console Note R In the console an error in the console or related code will stop executing using a mechanism such as echo lt commands gt mysqluc The console also allows users to set user defined variables for commonly used values in options The syntax is simply set VARNAME VALUE The user can see a list of all variables by entering the show variables command To use the values of these variables in utility commands the user must prefix the value with a For example server SERVER1 will substitute the value of the SERVER1 user defined variable when the utility is executed Note R User defined variables have a session lifetime They are not saved from one execution to another in
404. plms verbose v 151 NOTES Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES The login user for the master servers must have the appropriate permissions to grant access to all databases and have the ability to create user accounts For example the user accounts used to connect to each of the masters must have the WITH GRANT OPTION privilege The server IDs on the masters and slave must be nonzero and unique The utility reports an error if the server ID is 0 on either server or the same on the masters and slave Set these values before starting this utility Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the replication utilities could report a false negative that the slave is not connected to the master The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from
405. porting the data using bulk insert statements from a file that contains SQL statements use this command shell gt mysqldbimport server root localhost import both bulk insert format sql data sql Source on localhost connected Importing definitions and data from data sql done 107 PERMISSIONS REQUIRED PERMISSIONS REQUIRED You also need permissions to create the new data directory and write data to it 5 7 mysqldiff Identify Differences Among Database Objects This utility reads the definitions of objects and compares them using a diff like method to determine whether they are the same The utility displays the differences for objects that are not the same Use the notation db1 db2 to name two databases to compare or alternatively just db1 to compare two databases with the same name The latter case is a convenience notation for comparing same named databases on different servers The comparison may be executed against two databases of different names on a single server by specifying only the server1 option The user can also connect to another server by specifying the server2 option In this case db1 is taken from server1 and db2 from server2 When a database pair is specified all objects in one database are compared to the corresponding objects in the other Objects not appearing in either database produce an error To compare a specific pair of objects add an object name to each database
406. queries failed and you want to determine what might be the cause of those transaction failures All queries are recorded to the audit log file so you just need to get retrieve all queries of a given type that failed with a MySQL Error and analyze them This can be achieved using common grep command line tools but likely involves the use of very complex regular expression to filter the desired data Fortunately the mysqlauditgrep utility allows to perform this kind of task in a much easier and simple way taking advantage of the knowledge of the structure and semantics of the audit log files Objectives The goal is display all INSERT and UPDATE queries that failed independently of error from the current audit log file It is assumed that the audit 1log file exists and is located in the directory MySQL SERVER data The below example show how easy it is to perform the desired search with the mysqlauditgrep utility Example Execution shell gt mysqlauditgrep query type INSERT UPDATE status 1 9999 MySOL SERVER data audit log 4 4 STATUS TIMESTAMP NAME SOLTEXT CONNECTION_ID 4 a 4 4 1046 2OLS OF OlTIGseOsase Cera INSEE INC cloull_ineic_exaisie Aplopie WAUINS il 2 3 Sy 58
407. r 156 OPTIONS The failover command requires only the s laves option to explicitly list all of the slaves in the topology because it is expected that the master is down when this command is used Note E The option to pass in slaves without also passing in master was added in MySQL Utilities 1 6 0 Use the verbose option to see additional information in the health report and additional messages during switchover or failover OPTIONS mysqlrpladmin accepts the following command line options help Display a help message and exit license Display license information and exit candidates lt candidate slave connections gt Connection information for candidate slave servers for failover Valid only with failover command List multiple slaves in comma separated list To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration
408. r 19 15 56 03 2012 Master Information Binary Log File Position Manlog De DB Binlog ignores Ds Hye oii OOOOOW 5 Wil GTID Executed Set PAGTDEOO 20Al S11 ROSATI O0 1 Gare BEoOr 1 7 xe Replication Health Status 4 4 4 4 4 4 host Voort role state gtid_ mode health 4 4 4 4 4 belo calliho staan S sol MASTER UP ON OK Palo callihos t ieo SLAVE UE ON OK Polo calihostam mS Ssses SLAVE UB ON Ox localhost 3334 SLAVE WE ON OK 4 4 4 4 Q quit R refresh H health G GTID Lists U UUIDs L log entries Press Q to exit the utility R to refresh the current display and H returns to the replication health report Press G key to show a GTID report similar to the following The first page shown is the master s executed GTID set MySQL Replication Monitor and Failover Utility Failover Mode auto Next Interval Mon Mar 19 15 59 33 2012 Master Information Binary Log File Position Binlog pois Binlog tonere pe mysql bin 000001 571 GTID Executed Set ANG TOTOO ZADA k 2 SNL OO oan ear gil y ln oo Master GTID Executed Set 2A67DE00 2DA1 11E2 A711 00764F2BE90F 1 7 S508 D8 7R gt 2DB2 gt 1h2 Ay el Ov 7 DACE4B33 213 Q quit R refresh H health G GTID Lists U UUIDs L log entries Up Down scroll
409. r example to read the log and print the entries gt gt gt general_log open var lib mysql mysql log gt gt gt log GeneralQueryLog general_log eee 0F Emery tn log print entry Parameters e stream file type a valid file type for example the result of the built in Python function open version Returns Version of the MySQL server that produced the log Return type tuple program Returns Full path of the MySQL server executable Return type str port Returns TCP IP port on which the MySQL server was listening Return type int socket Returns Full path of the MySQL server Unix socket Return type str start_datetime Returns Date and time of the first read log entry Return type datetime datetime lastseen_datetime Returns Date and time of the last read log entry Return type datetime datetime class mysql utilities parser SlowQueryLog stream This class parses the MySQL Slow Query Log Instances are iterable but the class does not provide multiple independent iterators For example to read the log and print the entries 207 mysq utilities parser Parse MySQL Log Files gt gt gt slow_log open var lib mysql mysql slow log gt gt gt log SlowQueryLog slow_log zee for entry an log print entry
410. r example to specify 1log bin binlog and general log file mylogfile use If the option skip innodb is included when connecting to a MySQL server version 5 7 5 or higher the option is ignored and a warning is issued mysqld log bin binlog general log file my log file new data lt path_to_new_datadir gt The full path to the location of the data directory for the new instance The path size must be 200 characters or less and it requires at least 120 MB of free space new id lt server_id gt The server_id value for the new server instance The default is 2 new port lt port gt The port number for the new server instance The default is 3307 quiet q Turn off all messages for quiet execution root password lt password gt The password for the root user of the new server instance server lt source gt Connection information for the server to be cloned To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure 182 EXAMPLES e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt l
411. r password in the servers and client sections Finally create MySQL Fabric s state store as follows root host1 mysqlfabric config var 1lib mysql_drbd fabric cfg param protocol xmlrpc address localhost 32274 param protocol mysql address localhost 32275 manage setup From this point onwards all resources will be managed by the clustering software so they have to be stopped root host1 etc init d mysql stop root host1 umount var lib mysql_drbd root host1 drbdadm secondary clusterdb_res Then DRBD should be stopped as well root host1 etc init d drbd stop root host2 etc init d drbd stop 251 Setting up and testing your system 8 6 4 6 Configure Corosync and Pacemaker At this point the DRBD file system is configured and initialized and both MySQL Fabric and MySQL Server has been installed and the required files set up on the replicated DRBD file system Pacemaker and Corosync are installed but they are not yet managing the MySQL Fabric Process MySQL Server and DRBD resources to provide a clustered solution the next step is to set that up Firstly set up some network specific parameters from the Linux command line and also in the Corosync configuration file The multi cast address should be unique in your network but the port can be left at 5405 The IP address should be based on the IP addresses being used by the servers but should take the form of XX YY ZZ 0 Copy an example to make your
412. r process exec post failover to run a script after completing the failover process before displaying the health report 3 4 4 Restore the Previous Master After Failover After a successful failover it is sometime required to restore the initial topology and promote the crashed server to master again or even a new server with distinctive hardware characteristics Sometimes failover can be triggered by simple network issue not affecting the health of the initial master server and after being resolved it might be desirable to put the old master back in the replication topology Objectives The goal of this task is simply to replace the new master of a replication topology with the previous one that might have been demoted as result of successful automatic failover execution due to some failure It is assumed that the server to be restored as master is healthy and any previous issue that triggered failover have been resolved Let s consider the previous topology after failover now with a new master server2 3312 and three slaves server3 3313 server4 3314 server 3315 and that we want to promote the initial server Server1 3311 to master again Perform this task manually can be delicate as one wrong or missing step can lead to errors and incorrect replication topology or even to the lost of some transaction Once more MySQL Utilities can provide a precious assistance to perform this task in this case requiring the user to follow three s
413. r server localhost 3311 OK test_rplsync_db t0O checksum for server localhost 3312 Checking tl table data OK test_rplsync_do tl checksum for server localhost 3311 OK test_rplsync_do tl checksum for server localhost 3312 Checking test_db database Checking tl table data OK test_db tl1 checksum for server localhost 3311 OK test_db tl checksum for server localhost 3312 za a SS SUMMARY No data consistency issue found To check the data consistency on an active replication system excluding a specific database and table shell gt mysqlrplsync master user pass localhost 3310 slaves rp1 pass localhost 3311 rpl1 pass localhost 3312 exclude test_rplsync_db test_db t1 GTID differences between Master and Slaves Slave localhost 3311 is 15 transactions behind Master Slave localhost 3312 is 12 transactions behind Master Checking data consistency Using Master localhost 3310 as base server for comparison Checking test_db database Chieciciing VieO icaloile Cleicec lt lt OK test_db t0 checksum for server localhost 3311 OK testdb tO checksum for server localhost sai2 done 179 EXAMPLES SUMMARY No data consistency issue found The following is an example of a replication check that has data inconsistencies shell gt mysqlrplsync master user pass localhost 33
414. rage Engine 17 3 2 4 How Can Tell If a Table on Server X has the same Structure as the Same Table on SEVER V2 ici vive E TE ative cst ee E A ee Abate 19 3 2 5 How Can Synchronize a Table on Two Servers Where Neither is Up to date 20 3 3 General Operations 0 ccccecccecece cece cece cece cee a ete ee ee ee ee aaa aaae ee eeee ee aeaaaaaeeeeeeeeeaaaaaaeeseeeeeeaeaaaaees 21 3 3 1 How Can Find Out How Much Space My Data Uses cceeeeeeeeeeeeeeeeeeeeeeaeaaeeees 21 3 3 2 My Server Crashed Need to Know the Structure of a Table How Can Do That 23 3 3 3 Creating a New User With The Same Privileges as Another User assesses 24 3 3 4 What Options Are Used With Each Utility cece ceeeee cece eeeeeeeeeeeeeeeaaeaeeeeeeeeeeeaes 25 3 3 5 I ve Got Too Many Indexes How Do Know Which Ones to Drop eceeeeeeeeeeees 28 3 3 6 Need to Find an Object on My Server But All Have is a Partial Name How Do Find All Objects with That Name Prefix 00 ccccccceeeeaeceeeeeeeeeeeeeaeaaeeeeeeeeeeeeeeaaaaeeeeeeeeeeeaaa 30 3 3 7 How Can Run a Process Every Night To Kill Certain Connections n 32 3 4 High Availability Operations 0 ccc cece cece cece ee eeeeee cree seer ae aa eee eeeeeeee se aaeaaeeeeeeeeeeeaaaaeneeseeeees 33 3 4 1 How Can Use Replication 2 0 0 0 ccc eeeeee cree ee ee ae eeeeeeeeeeeeeeaaeaeeeeeeeeeeeaaaaeeeeeeeeees 33 3 4 2 How Do Add New Servers to My
415. rce on localhost connected KKKKKKKKKKKKKKKKKKKKKKKKK Ds row KKKKKKKKKKKKKKKKKKKKKKKKK server localhost 3307 config_file binary_log Dinary mloOgmpOs relay_log relay_log_pos 53 How Can Use a secure encrypted connection between Utilities and a MySQL Server version 5 6 15 datadir C MySQL instance_3307 basedir C MySQL mysql 5 6 15 winx64 plugin_dir C MySQL mysql 5 6 15 winx64 lib plugin general_log OFF general_log_file general_log_file_size log_error C MySQL instance_3307 clone err log_error_file_size 1569 bytes slow_query_log OFF slow_query_log_file slow_query_log_file_size 1 rov done Or you can indicate the SSL options by Using Option Files as is mentioned in the Section 2 2 Connecting to MySQL Servers documentation This is an example of how it may look for a group with the options in an options file for the command used above instance_3307 port 3307 uUSer root password pass host localhost ssl ca C newcerts cacert pem ssl cert C newcerts client cert pem ssl key C newcerts client key pem In this case the file is located at Cc MySQL instance 3307 cnf and by indicating this path and the group name in the server option the options for the mysqiserverinfo of the previous example takes this form shell gt mysqlserverinfo server c MySQL instance 3307 cnf instance_3307 format vertical Source on localhost connected KKK KKEKKKEKKEKKEK
416. rd work for us We then check the result using the mysql rpladmin utility Lets assume the existing server Server1 is running on port 13001 on the local machine with IP 192 168 1 1 and that we want to add 2 new servers Server2 running on 192 168 1 2 13001 and Server3 running on 192 168 1 3 3306 Example Execution shell gt mysqlreplicate master m_account 192 168 1 1 13001 slave slave_acc1 192 168 1 2 13001 rpl user repl slavepass b Master on 192 168 1213 2 connected slave on 197 166 1 23 2 connected Checking for binary logging on master Setting up replication TIC OMes shell gt mysqlreplicate master m_account 192 168 1 1 13001 slave slave_acc2 192 168 1 3 3306 rpl user repl slavepass b master on 192 168 1 13 connected slave on 197 16e 1 32 2 connected Checking for binary logging on master Setting up replication TIC OWes shell gt mysqlrplcheck master m_account 192 168 1 1 13001 slave slave_acc1 192 168 1 2 13001 master on 192 168 1 1 connected 33 How Can Use Replication slave on 192 168 1 2 connected Test Description status Checking for binary logging on master pass Are there binlog exceptions pass Replication user exists pass Checking server_id values pass Checking server_uuid values pass Is slave connected to master pass Check master information file pass Checking InnoDB compatibility pass Checking storage engines compatib
417. rdb_res root host2 drbdadm discard my data connect clusterdb_res root host1 drbdadm primary clusterdb_res 255 Using Connector Python with MySQL Fabric root host1 drbdadm connect clusterdb_res Before executing these steps it is advised to check the error log s and run the following command to identify the DRBD s status shell gt etc init d drbd status 8 7 Using Connector Python with MySQL Fabric MySQL Fabric provides data distribution and high availability features for a set of MySQL database servers Developers using Connector Python can take advantage of its features to work with a set of servers managed by MySQL Fabric Connector Python supports the following MySQL Fabric capabilities More specifically Connector Python Fabric support provides these features e Requesting a connection to a MySQL server managed by Fabric is as transparent as possible to users already familiar with Connector Python Connector Python is able to get a MySQL server connection given a high availability group and a mode specifying whether the connection is read only or also permits updates read write Connector Python supports sharding and is able to find the correct MySQL server for a given table or tables and key based on scope local or global and mode read only or read write RANGE and HASH mechanisms are supported transparently to the user e Among secondary MySQL servers in the same group read only connections a
418. re being used We get this from using the show servers option with mysqlserverinfo On POSIX systems we can use the ps command to find the command line options shell gt mysqlserverinfo server root root localhost format vertical show servers The following MySQL servers are active on this host Process id 2377 Data path usr local mysql data Process id 2478 Data path Volumes Source source temp_13001 Process id 2487 Data path Volumes Source source temp_13002 48 How Do Make A Temporary Copy of My Server For Testing Source on localhost connected KEK KKK KKK KKK KKK KKEKKKK KK KEK iy row KEK KKK KKK KKK KKK KKKKKEK KK KK server localhost 3306 vension S il 50 keg datadir usr local mysql data basedir usr local mysql 5 1 50 osx10 6 x86_64 plugin dir usr local mysql 5 1 50 o5x10 6 x86_64 lib plugin config_file etc my cnf binary_log my_log 000287 binary_log_pos 106 relay_log None relay_log_pos None 1 row done shell gt my_print_defaults mysqld etc my cnf pore 23306 basedir usr local mysql datadir usr local mysql data server_id 5 iog bin my log general_log slow_query_log innodb_data_file_path ibdatal 778M ibdata2 50M autoextend shell gt ps f 2377 UID PID PEITD C STIME IEG TIME CMD 74 2377 2300 0 10 56AM 0 02 04 usr local mysql bin mysqld basedir usr local mysql datadir usr local mysql data
419. re load balanced Load balancing is based on a weight set for each MySQL server using a Weighted Round Robin algorithm Faulty MySQL servers are reported to Fabric and failover is supported when failure occurs for a server in a group e To speed up operations Connector Python caches information obtained from Fabric such as group and sharding information Each time Connector Python looks up data it first checks its cache When information in the cache is too old it is invalidated By default the time to live TTL for cached information is 1 minute However Fabric itself can provide this TTL for its clients and this value is used instead if greater than zero Cache information is also invalidated when failures occur such as when a connection to a MySQL server fails invalidation applies to the group to which the server belongs Fabric support applies to versions of Python supported by Connector Python itself see Connector Python Versions In particular you can use Connector Python with Python 3 1 and later to establish Fabric connections even though Fabric does not support Python 3 Connector Python support for Fabric comprises the following module and classes e Module mysql connector fabric All classes functions and constants related to MySQL Fabric e Class fabric MySQLFabricConnection Similar to MySOLConnection it creates a connection to the MySQL server based on application provided information 256 Installing Con
420. rent active binlog file As we can see in the example the mysqlbinlogmove utility displays latest binlog file replicated by all slaves and if want the utility to display the current binlog file being read by the I O thread we can use the verbose option Permissions Required The user requires the SUPER and REPLICATION SLAVE privileges to purge the binlog files Tips and Tricks The dry run option can be used to display only the latest binlog file matched by all the slaves without actually purge the binlog files Use it along vv to display even more information as status of the SQL and I O threads of each slave 14 Database Operations 3 2 Database Operations The tasks described in this section relate to those that are performed on or with one or more databases 3 2 1 How Do I Provision a Slave When working with replication one of the common tasks is adding a new slave for scale out Although adding a new slave has been simplified with utilities like mysqlreplicate provisioning the slave copying data and getting replication started properly can be a challenge if done manually Fortunately we have two utilities mysqldbexport and mysqldbimport that have been designed to work with replication so that when the export is generated you can include the proper replication control statements in the output stream Objectives Perform slave provisioning using mysqldbexport and mysqlidbimport Example Execution shel
421. report port s opt new_port Clone the server print Cloning server instance EPY res serverclone clone_server conn options except UtilError e print ERROR e errmsg sys exit Set connection values dest_values user conn get user passwd r EO Ol Mlavoysye seon Gee unos ar port opt new_port unix_socket os path join opt new_data mysql sock Build dictionary of options options Tkepmaleye E UPre ioe force i Irus print Copying databases EEV dbcopy copy_db conn dest_values db_list options except UtilError e print ERROR e errmsg 203 Specialized Operations sys exit Build dictionary of options options overwrite True quiet lt Tew globals True print Cloning the users for user in user_list CEV res userclone clone_user conn dest_values user user options except UtilError e print ERROR e errmsg sys exit print Wis 5 5 done 6 3 Specialized Operations 6 3 1 mysql utilities command grep Search Databases for Objects This module provides utilities to search for objects on a server The module defines a set of object types that can be searched by searching the fields of each object The notion of an object field is very loosely defined and means any names occurring as part of the object definition For example the fields of a table include the table name the column
422. rm a join between the employees and departments tables Details on how to provide the shard key are given in the next step JDBC4FabricMySQLConnection conn provide the set of query tables to the connection conn addQueryTable departments conn addQueryTable employees JDBC4FabricMySQLConnection conn connection initialization here conn setShardKey 40 work with data related to shard key 40 to access a database server 8 8 4 MySQL Fabric Configuration for Running Samples To run JUnit tests from test suite fabric or demonstration examples from src demo fabric you must configure the MySQL Fabric test environment as follows 1 Setup MySQL servers e mysql fabric config The backing store for Fabric configuration Used internally by Fabric to store the server list shard mappings and so forth You set up this server instance during the MySQL Fabric setup procedure e mysql global The only server in the global group Used to send DDL commands and update any data not in the shard data set e Default location 127 0 0 1 3401 Config properties com mysql fabric testsuite global host com mysql fabric testsuite global port 264 MySQL Fabric Configuration for Running Samples e mysql shardl First shard of sharded data set e Default location 127 0 0 1 3402 e Config properties com mysql fabric testsuite shardl host com mysql fabric testsuite shardl port mysql shard2 Second shard of shar
423. rmation For example v verbose vv more verbose vvv debug To diagnose test execution problems use vvv to display the actual results of test cases and ignore result processing version 210 NOTES Display version information and exit e width lt number gt Specify the display width The default is 75 characters NOTES The connection specifier must name a valid account for the server Any test named _template py is skipped This enables the developer to create a base class to import for a collection of tests based on a common code base EXAMPLES The following example demonstrates how to invoke mut to execute a subset of the tests using an existing server which is cloned The example displays the test name status and relative time shell gt mut server root localhost do tests clone_user width 70 MySQL Utilities Testing MUT Parameters used Display Width 70 Sorted True Force False Test directory SA Utilities directory scripts SSeS joerc 3310 Test wildcard clone_users Servers Connecting to localhost as user root on port 3306 CONNECTED TEST NAME STATUS TIME main clone_user pass 54 main clone_user_errors pass an main clone_user_parameters pass y Testing completed Friday 03 December 2010 09 50 06 All 3 tests passed PERMISSIONS REQUIRED There are no special permissions required to run mysqluc however you must have the necessary privileges to ex
424. rmation and exit e master lt master gt Connection information for the master server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt 145 OPTIONS e login path mylogin cnf lt ogin path gt lt port gt lt socket gt e Configuration file lt configuration file path gt lt section gt Command line lt users lt passwa gt lt host gt lt port gt lt socket gt master log file lt master_log_file gt Begin replication from the beginning of this master log file master log pos lt master_log_pos gt Begin replication from this position in the master log file This option is not valid unless master log file is given
425. root password database employees alobal host il27 0 Onl global port 3401 shardl host 127 0 0 1 shardl port 3402 shand2host i27 0 0 i shard2 port 3403 Sample Ant calls are shown below If the parameters are specified in your build properties file it is not necessary to include them on the command line shell gt JAVA_HOME opt jdk1 5 ant Dcom mysql Dcom mysql Dcom mysql Dcom mysql test fabric fabric fabric fabric fabric testsuite password pwd testsuite global port 3401 testsuite shard1 port 3402 testsuite shard2 port 3403 8 8 6 Running Demonstration Programs To run the demo programs you must set up the MySQL Fabric environment as described in Section 8 8 4 MySQL Fabric Configuration for Running Samples After that you can use the demo fabric Ant targets shell gt JAVA_HOME opt jdk1 5 ant Dcom mysql fabric testsuite password pwd demo fabric These targets invoke all demo programs except Hibernate demos You should use the demo fabric hibernate target instead shell gt JAVA_HOME opt jdk1 6 ant Dcom mysql fabric testsuite password pwd demo fabric hibernate 266 A Complete Example Working with Employee Data N Note You need Java 6 to use Hibernate Fabric integration Multi tenancy is a feature specific to Hibernate 4 and higher which requires Java 6 That is why we do not provide the FabricMultiTenantConnect
426. rpm You can also use the RPM package manager that is part of your base operating system See your operating system documentation for more details Note R MySQL Utilities requires Connector Python to be installed For additional information see Section 1 1 Prerequisites 1 4 Debian Linux The deb repository is built for Debian 6 and is architecture independent Although built expressly for Debian 6 it can be installed on various ports such as amd64 i386 etc 2 6 and Debian 7 currently ships with Python 2 7 For Debian 7 use the source Note R The repository does not work for Debian 7 because MySQL Utilities requires Python code repository to install MySQL Utilities After you download the file install it using the following command or similar depending on your specific release or version of Debian shell gt sudo dpkg i mysql utilities 1 5 6 debian6 0_all deb Note R MySQL Utilities requires Connector Python to be installed For additional information see Section 1 1 Prerequisites 1 5 Microsoft Windows Either install MySQL Utilities using the MySQL Installer for Windows a system that manages installations and updates for all MySQL products on Windows or download and execute the standalone file The download links are as follows e MySQL Installer Download and execute the MySQL Installer MSI file Select the MySQL Utilities product and then proceed with the installation This is the recommended approach a
427. rsor BEGIN DECLARE id BIGINT DECLARE EXIT HANDLER FOR NOT FOUND SET kill_done 1 kaL logosz LOOP FETCH kili eurc or INIO 10 KILL CONNECTION id END LOOP kill_loop END CLOSER Kk MNRECURS OE Discussion Notice in the example above we did not connect to any server to get this information That is one of the great things about this utility you can generate all manner of SQL statements for finding processes and try them out on a test system before incorporating them into your events triggers and routines We specified the user with the match user option using a wildcard in case the user is logged in from a different system We also specified the name of the maintenance routine in the same manner in case it gets renamed with a version number or some such The output of this utility then is the SQL statement we need to use to find and kill the connections that meet these criteria Armed with this we can make a procedure we can call from an event and execute the SQL at a precise time every day Permissions Required The user must have the SELECT permission on the mysql database 32 High Availability Operations Tips and Tricks If you are familiar with using regular expressions you can use the regexp option to use regular expressions instead of database patterns 3 4 High Availability Operations The tasks described in this section include those for replication and general to specific high availability ta
428. running on port 13001 on the local machine with IP 192 168 1 1 that the two new machines with mysql server instances are Server2 running on 192 168 1 2 13001 and Server3 running on 192 168 1 3 3306 We want to make Server2 the new master Example Execution shell gt mysqlreplicate master m_account 192 168 1 1 13001 slave slave_acc1 192 168 1 2 13001 rpl user repl slavepass b master on 192 168 1 13 connected slave on 197 16e 1 23 2 connected Checking for binary logging on master Setting up replication done shell gt mysqlreplicate master m_account 192 168 1 1 13001 slave slave_acc2 192 168 1 3 3306 rpl user repl slavepass b master on 192 168 1 13 conmected 35 How Do Add New Servers to My Topology and Change Master Role a Blave om 192 1607138 ccoo Connected Checking for binary logging on master Setting up replication eonen shell gt mysqlrpladmin master m_account 192 168 1 1 13001 slaves slave_acc1 192 168 1 2 13001 slave_acc2 192 168 1 3 3306 health Checking privileges Replication Topology Health 4 4 4 4 4 host port role state gtid_mode health 4 4 4 4 2 Haea a AL soil MASTER wie ON OK IS Sonila LSOCil Sitydwaa UP ON Slav
429. rver master slave etc The methods include the following in order of most secure to least secure 142 Options e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt e sql print sql Q Instead of displaying the selected processes emit the SELECT statement that retrieves information about them If the kill connection or kill query option is given the utility generates a stored procedure named kill_processes for killing the queries rather than a SELECT statement e sql body Like sq1 but produces the output as the body of a stored procedure without the CREATE PROCEDURE part of the definition This could be used for example to generate an event for the server Event Manager When used with a kill option code for killing the matching queries is generated Note that it is not possible to execute the emitted code unless it is put in a stored routine event or trigger For example the following code could be generated to kill all idle connections for user www data shell gt mysqlprocgrep kill connection sql body match user www
430. rverinfo Display Common Diagnostic Information from a Server 05 184 5 22 mysqluc Command line client for running MySQL Utilities 22 eee eeeeeeeeeeeeaee 187 5 23 mysqluserclone Clone Existing User to Create New USE ccccseeceeeneeeeeeaneeeees 191 6 Extending MySQL Utilities esi ce setssesseeccanseauesseaeedand aevdeastaneceant anneedashaseeens eoveds bisastenseneteebnabeneendededes 195 6 1 Introduction to extending the MySQL Utilities 2 0 2 cece ee eeeeeeeeeeeeee ee aeaa ee eeeeeeeeeeaeaaeeees 195 6 2 MySQL Utilities copy_server py Sample cccceeeeececeeeeeeeeeeeeeaeaaeeteeeeeeeeeaeaaaaeeeeeeeeeeeeaaa 201 6 3 Specialized Operations isser munanna e ai a E aE R a aaia 204 6 3 1 mysql utilities command grep Search Databases for Objects 0 204 6 3 2 mysql utilities command proc Search Processes on Servers c 0000ee 205 6A PASES sc cassaneuivsneecnaanahnadivnsse aAA E a AAA A E senenedagoneecnavapeavansmancdcabenauanaannnes 206 6 4 1 mysql utilities parser Parse MySQL Log Files eccecseeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaes 206 7 MySQL Utilities Testing MUT scassi iturssantine esinaine n aaa aaa 209 7 1 mut MySQL Utilities Testing 2 0 0 0 ccceeeeeee cece cece ae aa tte ee nese ee ae aaaadaeeeeeeeeseaaaaaaeeeeeeeeeeaea 209 B MYSQL Fabrice sisirin aiana eaae TEONE tenet est E AA NAE a E a E T aE 213 8 1 Introd ction to FADING omissis renan E REE a
431. rypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt 105 NOTES e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt skip lt skip_objects gt Specify objects to skip in the operation as a comma separated list no spaces Permitted values for this list are CREATE_DB DATA EVENTS FUNCTIONS GRANTS PROCEDURES TABLES TRIGGERS and VIEWS skip blobs Do not import BLOB data skip gtid Skip execution of GTID_PURGED statements skip rpl Do not execute replication commands ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection e Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required table lt db gt lt table gt Specify the table for importing This option is required while using format raw_csv verbose v Specify how much information to display Use this option multiple times to increase the amou
432. rypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt skip long Exclude tests that require greater resources or take a long time to run skip suite lt name gt Exclude the named test suite Use this option multiple times to specify multiple suites skip test lt name gt Exclude the named test Use this option multiple times to specify multiple tests skip tests lt prefix gt Exclude all tests that begin with prefix sort Execute tests sorted by suite name either ascending asc or descending desc Default is ascending asc Start port lt port gt The first port to use for spawned servers If you run the entire test suite you may see up to 12 new instances created The default is to use ports 3310 to 3321 start test lt prefix gt Start executing tests that begin with prefix stop test lt prefix gt Stop executing tests at the first test that begins with prefix suite lt name gt Execute the named test suite Use this option multiple times to specify multiple suites testdir lt path gt The path to the test directory utildir lt path gt The location of the utilities verbose v Specify how much information to display Use this option multiple times to increase the amount of info
433. s to wait for the checksum query to complete Default 3 seconds e discover slaves login lt user_login gt Detect registered slaves at startup and use the user name and password specified to connect in the format lt user gt lt password gt or lt login path gt For example discover slaves login joe secret will use joe as the user and secret as the password for each discovered slave exclude lt databases tables to exclude gt Fully qualified name for the databases or tables to exclude lt db_name gt lt tb _name gt List multiple data objects in a comma separated list e help Display a help message and exit e interval lt interval_in_secondss i lt interval_in_seconds gt Interval in seconds for periodically polling the slaves sync status to verify if the sync point was reached Default 3 seconds e license Display license information and exit master lt master_connection gt Connection information for the master server 175 OPTIONS To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file en
434. s user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt 158 OPTIONS e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt e Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt no health Turn off health report after switchover or failover ping lt number gt Number of ping attempts for detecting downed server Note on some platforms this is the same as number of seconds to wait for ping to return This value is also used to check down status of master Failover will wait ping seconds to check master response If no response failover event occurs quiet q Turn off all messages for quiet execution rpl user lt replication_user gt The user and password for the replication user requirement in the format lt users lt password gt or lt login path gt E g rpl passwd Default None script threshold lt return_code gt Value f
435. se steps shell gt mysqlfabric group create my _group Procedure A wiel d4e60ed4 fd36 4df6 8004 d034202c3698 finished True success True return True activities shell gt mysqlfabric group add my group localhost 3307 Procedure uuid 6a33ed29 ccf8 437f b436 daf07db7alfc finished True success True return True activities shell gt mysqlfabric group add my_group localhost 3308 Procedure A owel 6892bc49 3ab7 4bc2 891d 57c4a1577081 finished True success True return True activities shell gt mysqlfabric group add my_group localhost 3309 Procedure A owel 7943b27f 2da5 4dcf ala4 24aed8066bb4 finished True success True return True activities To show information about the set of servers in a group use this command shell gt mysqlfabric group lookup_servers my_group To get detailed information about the group health use this command shell gt mysqlfabric group health my_group Promoting and Demoting Servers After executing the steps in setting up a high availability group Fabric does not become aware of any replication topology that was already in place It is necessary to promote one of them to primary that is master and make the remaining servers secondaries that is slaves To do so execute this command shell gt mysqlfabric group promote my_group If there is a primary in a group any server added subsequently is automatically set
436. server User name root Password master on localhost connected Pinding slaves for master localhost 32333 Server localhost 3334 is running on localhost master on localhost FAILED Connection to localhost 3334 has failed Please enter the following information to connect to this server User names root Password master on localhost connected Finding slaves for master localhost 3334 Replication Topology Graph localhost 3331 MASTER t o leeellnosessss2 SLAW T lieesllnxosics 335s SILA Am ap MAS EER T lo cellaomesassa Sila PERMISSIONS REQUIRED The user connected to the master must have the REPLICATION SLAVE privilege The user specified with the discover slaves login option that logs into each slave must have the REPLICATION CLIENT privilege 5 19 mysqlrplsync Replication synchronization checker This utility permits you to check replication servers for synchronization It checks data consistency between a master and slaves or between two slaves The utility reports missing objects as well as missing data The utility can operate on an active replication topology applying a synchronization process to check the data Those servers where replication is not active can still be checked but the synchronization process will be skipped In that case it is up to the user to manually synchronize the servers The user must provide connection parameters for
437. server to clone for testing purposes You must specify at least one server but you may specify multiple servers for tests designed to use additional servers The utility has a special test suite named performance where performance related tests are placed This suite is not included by default and must be specified with the suite option to execute the performance tests OPTIONS mut accepts the following command line options e help Display a help message and exit e do tests lt prefix gt Execute all tests that begin with prefix e force Do not abort when a test fails e record Record the output of the specified test if successful With this option you must specify exactly one test to run e Server lt server gt Connection information for the server to use in the tests Use this option multiple times to specify multiple servers To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt 209 OPTIONS e Use a configuration file unenc
438. sistency between servers in a replicated setup e Uses Global Transaction Identifiers GTID e Requires MySQL Server 5 6 14 and higher 4 4 Server Operations These utilities are used to perform server wide operations mysqlserverclone 63 Specialized Operations e Start a new instance of a running server mysqlserverinfo e Show server information e Can search for running servers on a host e Access online or offline servers 4 5 Specialized Operations These utilities are designed to be used with a specific commercial extension In this case these utilities require the Audit Log Plugin mysqlauditadmin e Monitor the audit log Copy rotate and configure the audit log e mysqlauditgrep e Search the audit log e Output results to different formats 64 Chapter 5 Manual Pages Table of Contents 5 1 mysqlauditadmin Allows users to perform maintenance action on the audit log 60 65 5 2 mysqlauditgrep Allows users to search the current or an archived audit log cc cseeeees 71 5 3 mysqldbcompare Compare Two Databases and Identify Differences ccccesseeeeeeeneeeees 78 5 4 mysqldbcopy Copy Database Objects Between Servers ccccccseececeneeeeeeaaeeeeeeaaeeeeeeaaneees 87 5 5 mysqldbexport Export Object Definitions or Data from a Database ccceeceeceeseeeeeeeeeeees 94 5 6 mysqldbimport Import Object Definitio
439. sks such as automatic failover 3 4 1 How Can I Use Replication MySQL has built in support for several types of replication Replication is usually employed with the purpose of increasing the performance and or the fault tolerance of a service However setting up replication can be a somewhat complicated and error prone process But fear not MySQL Utilities has tools that can help simplify and even automate several replication related tasks Lets see a possible scenario where replication is used to obtain scalability i e to increase the performance of a service Lets imagine an online shopping service The shop started out small so a single server was enough to handle all the requests however now it has become quite popular and as a result that single server is no longer able to handle all the requests Being an online store most of the operations are read operations checking existing products reviews stock availability etc Objectives Our goal is to use replication in order to improve the throughput of the service by adding more servers which will become replicas of the already existing server These replicas will allow scaling out of the service by taking up all the read requests leaving the old server now called the master only in charge of the writes Rather than doing everything by hand with the mysql command line we are going to setup this replication scenario using a single script mysql replicate which will do most of the ha
440. sl cert The name of the SSL key file to use for establishing a secure connection e Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug version Display version information and exit use indexes List the index to use Use this option to select the index to use if the table has no primary key or it has more than one unique index without null columns Use this option in the format use indexes lt table1 gt lt indexA gt lt table2 gt lt indexB gt width lt number gt Change the display width of the test report The default is 75 characters NOTES The login user must have the appropriate permissions to read all databases and tables listed For the difftype option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example di f ft ype d specifies the differ type An error occurs if a prefix matches more than one valid value The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_de
441. specify server connection information is with your mylogin cnf file Not only is this file encrypted but any logging of the utility execution will not expose the connection information Thus no user names passwords ports etc are visible in the log This is the preferred method for using MySQL Utilities to connect to servers Utilities support the use of login paths in the connection string provided they use the following format login path name port socket where the port and socket parameters are optional If used these optional parameters override the respective options from the specified login path When using login paths there are no default values except on Posix systems when specifying a socket In this case the host option defaults to localhost This means that combining the values specified in the login path with the two optional values port and socket one needs to specify at least a user a hostname and a port or socket Use the mysql_config_editor tool http dev mysql com doc en mysaql config editor html to add the connection information as follows shell gt mysql_config_editor set login path instance_13001 host localhost user root port 13001 pas Enter password lt Password is prompted to be inserted in a more secure way gt Next use the following command to confirm that the login path data was correctly added to mylogin cnf the encrypted file shell gt mysql_config_editor print login path instance_
442. splay output use one of the following values with the format option grid default Display output in grid or table format like that of the mysql client command line tool e CSV Display output in comma separated values format tab Display output in tab separated format 136 SQL Simple Patterns vertical Display output in single column format like that of the G command for the mysq1 client command line tool SQL Simple Patterns The simple patterns defined by the SQL standard consist of a string of characters with two characters that have special meaning percent matches zero or more characters and _ underscore matches exactly one character For example John Match any string that starts with john Match any string consisting of one or more characters POSIX Regular Expressions POSIX regular expressions are more powerful than the simple patterns defined in the SQL standard A regular expression is a string of characters optionally containing characters with special meaning Documenting these regular expressions goes beyond the scope of this manual but the full syntax is described in the MySQL manual and other locations such as executing man regex in your terminal Match any character oA Match the beginning of a string Match the end of a string axy Match a x or y gt a f Match any character in the range a to f that is a b c d e or f axy Match
443. sql fabric Classes for interacting with the MySQL Fabric management system using the XML RPC protocol src com mysql fabric jdbc Classes for JDBC access to MySQL servers based on shard information src com mysql fabric hibernate FabricMultiTenantConnectionProvider java Class enabling integration with Hibernate testsuite fabric JUnit tests src demo fabric Usage samples 8 8 1 Installing Connector J with MySQL Fabric Support Fabric support is available in Connector J 5 1 30 and later Please refer to Connector J documentation for installation instructions 8 8 2 Loading the Driver and Requesting a Fabric Connection When using Connector J with MySQL Fabric you must provide the host name and port of the MySQL Fabric server instead of the database server The connection string must follow the same form it normally does with the addition of the fabric keyword as follows jdbc mysql fabric fabrichost 32274 database The user name and password provided to the connection are used for authentication with the individual database servers Fabric authentication parameters can be given in the URL using the fabricUsername and fabricPassword properties e g jdbc mysql fabric fabrichost 32274 database fabricUsername admin amp fabricPassword secret authentication is used This should only be done when authentication has been Note R If the username and password to authenticate to the Fabric node are omitted no disabled on
444. sssssssssssrrsrssrrrrssrrrrssrrrissrrrrnsrnrnanrnenanns 40 3 4 5 How Can Find All of the Slaves Attached to My Master Server cseeeeeeeeeeeeeeeaeees 43 3 4 6 How To Check If Data Is Correctly Replicated cccceeeeeeeeeeeeeeeeeeeeeeeaaeaeeeeeeeeeeeeaeaaaeees 44 3 4 7 How To Fix Errant Transactions on the Replication Topology eeeeeeeeeeeeeeeeeeeeeeeaee 46 3 5 Server Operations avec seth ea te A Toes aalal E AA 48 3 5 1 How Do Make A Temporary Copy of My Server For Testing ccseeeeeeeeeseeteeteeeeeees 48 3 5 2 How Can Find What MySQL Servers Are RUNNING 0 ceeeeeeeeeneeeeeeeeeeeeaeaaeateeeeeees 50 3 5 3 How Can Use a secure encrypted connection between Utilities and a MySQL Server 51 3 6 Specialized Operations ccccccceccneeeeeeeee cece ee ae eect seen ee ee aa tn eae EASES ee aaaaaaaaeeeeeeeeaeaaaaageeeeeeeeaeaaaaees 55 3 6 1 How Do Record Only Login Events 0 cccccceeeeeeeeeee ee aeeeeeeeeeeeeeeeaeaaeeneeeeeeeeeaeaaeaneneeeees 55 3 6 2 How Do Copy Move The Audit Log cccccceeeeeeeeeeeeeeeeeeeeaeaaeeeeeeeeeeeeaeaaeaeeeseeeeeeaeaaeaees 57 3 6 3 How Do Show All INSERT and UPDATE Queries That Failed 0 ccceceecsseeeeseeeeeeees 58 3 6 4 How Do Display Connections by the User root and Show the Result in CSV Format 59 MySQL Utilities provides a command line set of tools for working with MySQL Servers and da
445. st implement the Fabric groups that store the shards This can be done as follows shell gt mysqlfabric group create group_id 1 shell gt mysqlfabric group create group_id 2 shell gt mysqlfabric group create group_id 3 The preceding commands create three high availability groups group_id 1 group_id 2 and group_id 3 Each group will store a shard Then we must define a global group which is used to propagate schema updates to all tables in the sharding setup and updates to global tables throughout the sharding scheme shell gt mysqlfabric group create group_id global 234 Example Fabric and Sharding Registering the Servers The MySQL servers must be registered with the groups Each group contains two servers 3307 3308 belong to group_id 1 shell gt mysqlfabric group add group_id 1 localhost 3307 shell gt mysqlfabric group add group_id 1 localhost 3308 3309 3310 belong to group_id 2 shell gt mysqlfabric group add group_id 2 localhost 3309 shell gt mysqlfabric group add group_id 2 localhost 3310 3311 3312 belong to group_id 3 shell gt mysqlfabric group add group_id 3 localhost 3311 shell gt mysqlfabric group add group_id 3 localhost 3312 3313 3314 belong to group_id global shell gt mysqlfabric group add group_id global localhost 3313 shell gt mysqlfabric group add group_id global localhost 3314 Then promote one server to master in each group shell gt mysqlfabric group promote group_id g
446. structs the utility to generate the SQL statements The show reverse option instructs the utility to generate the differences in both direction That is from the perspective of serveri as compared to server2 and server2 as compared to server1 By convention the second set is commented out should you wish to pipe the output to a consumer Lastly the quiet option simply turns off the verbosity of print statements that normally occur for communicating progress Permissions Required The user must have the SELECT privilege for the databases on both servers Tips and Tricks You can change the direction using the changes for option For example changes for server1 is the default direction and changes for server2 is the reverse In the second case the show revers displays the perspective of server commented out 3 3 General Operations The tasks described in this section include general tasks such as reporting information about a server and searching for objects or processes on a server 3 3 1 How Can Find Out How Much Space My Data Uses When preparing to create a backup or maintenance on a server related to storage it is often the case we need to know how much space is used by our data and the logs the server maintains Fortunately there is a utility for that Objectives Show the disk space used by the databases and all logs using the mysqidiskusage utility Example Execution shell gt sudo env PYTHONP
447. sts By default the utility stops on the first failed test but you can specify the run all tests option to cause the utility to run all tests regardless of their end state Note R Using run all tests may produce expected cascade failures For example if the row counts differ among two tables being compared the data consistency will also fail The tests include the following 1 Check database definitions A database existence precondition check ensures that both databases exist If they do not no further processing is possible and the run all tests option is ignored 2 Check existence of objects in both databases The test for objects in both databases identifies those objects missing from one or another database The remaining tests apply only to those objects that appear in both databases To skip this test use the skip object compare option That can be useful when there are known missing objects among the databases 3 Compare object definitions The definitions the CREATE statements are compared and differences are presented To skip this test use the skip diff option That can be useful when there are object name differences only that you want to ignore 4 Check table row counts This check ensures that both tables have the same number of rows This does not ensure that the table data is consistent It is merely a cursory check to indicate possible missing rows in one table or the other The data cons
448. t t3 NSERT INTO util_test t3 VALUES 1 14 test fkeys test fkeys test fkeys Data for table util_test t4 NSERT INTO util_test t4 VALUES Sp 2 done Ww N e To export a database and include the replication commands to use the current server as the master for example to start a new slave using the current server as the master use the following command shell gt mysqldbexport server root localhost 3311 util_test export both rpl user rpl rpl rpl master v Source on localhost oos Connected Stopping slave STOP SLAVE Source on localhost connected Exporting metadata from util_test DROP DATABASE IF EXISTS util_test CREATE DATABASE util_test USE util_test TABTE S wwiea ll VWeSgic sel CREATE TABLE EIl a gt char 30 DEFAULT NULL ENGINE MEMORY DEFAULT CHARSET latinl done Sourcekonilocallhost erm COMM ECE USE util_test Exporting data from util_test Data for table weil _test tl NSERT INTO util_test t1 VALUES O01 Test Basic database example NSERT INTO util_test tl1 VALUES 02 Test Basic database example NSERT INTO util_test t1 VALUES 03 Test Basic database example NSERT INTO util_test t1 VALUES 04 Test Basic database example NSERT INTO util_test t1 VALUES 05 Test Basic database example NSERT INTO util_test t1 VALUES 06 Test Basic database example NSERT INTO util_test t1 VALUES 07 Test Basic database
449. t we have a connection to the global group for the employees employees shard mappin statement rawConnection createStatement statement executeUpdate drop table if exists employees statement executeUpdate create table employees emp_no int not null first_name varchar 50 last_nam 2 Insert data Cast to a Fabric connection to have access to Fabric specific methods FabricMySQLConnection connection FabricMySQLConnection rawConnection example data used to create employee records Integer ids new Integer 1 2 10001 10002 String firstNames new String John Jane Andy Alice String lastNames new String Doe Doe Wiley Wein insert employee data PreparedStatement ps connection prepareStatement INSERT INTO employees employees VALUES 270 How Connector J Chooses a MySQL Server ore iot 1 0 a lt Ale stars i choose the shard that handles the data we interested in connection setShardKey ids i toString perform insert in standard fashion So Sic CPES ps setString 2 firstNames i ps setString 3 lastNames i ps executeUpdate 3 Query the data from employees System out println Querying employees System out format 37s 30s 30s n emp_no first_name last_name System Otice ine in We pra SSS SSS SS SSS SS SSS SSS SSS SSS SSS T Le ps connection prepareStatement Select
450. t you must have the ability to do a reverse name lookup to compare the IP 192 168 0 6 and the hostname ubuntu net to determine if they are the same machine The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES To show the slaves for a master running on port 3311 on the local host use the following command 172 EXAMPLES shell gt mysqlrplshow master root localhost 3311 discover slaves login root master on localhost connected Finding slaves for master localhost 3311 Replication Topology Graph localhost 3311 MASTER localhost 2a310 SLAVE to localhost 2gsl2 gt Suave As shown in the example you must provide valid login information for the master To show additional information about the IO thread status to confirm if the slaves are really connected to the master use the option verbose shell gt mysqlrplshow master root localhost 3311 discover slaves login root verbose master on localhost connected Finding slaves for master localhost 3311 Replication Topology Graph localhost 3311 MASTER Jloeallioesies S30 I0 wes SOL Yes SLAVE Jlloeellioesies Sse I0 Yes S0L Yes S
451. t MySQL servers may generate a significant number of binary log files that may consume considerable hard drive space and being a good reason a user might want to delete them But in a replication scenario is necessary to know which binary log files can be deleted without breaking replication since replication relies on the binary log Users need to check each slave to determine the latest position or GTID read This information is then used to determine which of the binary logs on the master may be removed That is which files have been read by all of the slaves 13 How do you purge obsolete binary log files safely Objectives The goal is to purge all unnecessary binlog files on the MySQL server in a replication topology Another important goal is to automate the operation Executing this task manually can be tedious and error prone since it requires verifying each slave The more slaves the more complicated this becomes Fortunately the mysqlbinlogpurge utility can help us to perform this task in an easy and safe manner by determining all the unnecessary files automatically on each slave and purging them on the master Let s assume that a master server is currently running on port 13001 and a few slaves are connected Example Execution 1 Run the mysqlbinlogpurge utility specifying the master connection string using the master option and either the slaves to indicate each slave connection string or the discover sla
452. t actually injecting empty transactions The dryrun option must be specified to use this read only mode 3 5 Server Operations The tasks described in this section are those used to perform server wide operations such as cloning a server instance 3 5 1 How Do Make A Temporary Copy of My Server For Testing When diagnosing a problem or needing to experiment with a server for developing new features or test modifications you often need a duplicate of your running server so that you can ensure your solution works for the actual server It would be really convenient if we had a process to make a copy of a running server for such processes Although it is possible and indeed popular to use replication to replicate all of your data to multiple slaves and use one of the slaves for these purposes cases where you are working with a particular server or if replication is not in use you will need some way to duplicate not only the data but also the server and its startup parameters Objectives Create a new instance of a running server complete with the same options and the same data Example Execution To meet this objective we need to use several utilities But before we get started we need to know what specific options the host server is using To do this we use the mysqliserverinfo utility to discover the configuration file and the my_print_defaults tool to print the defaults We can also show the process id to see what command line options a
453. t formats of the mysqldbexport utility e sql default Input consists of SQL statements For definitions this consists of the appropriate CREATE and GRANT statements For data this is an INSERT statement or bulk insert if the bulk insert option is specified grid Display output in grid or table format like that of the mysql client command line tool e cSV Input is formatted in comma separated values format e raw_csv Input is a simple CSV file containing uniform rows with values separated with commas The file can contain a header the first row that lists the table columns The option t able is required to use this format e tab Input is formatted in tab separated format e vertical Display output in single column format like that of the G command for the mysq1 client command line tool To indicate that input in csv or tab format does not contain column headers specify the no headers option To turn off all feedback information specify the quiet option By default the utility creates each table on the destination server using the same storage engine as the original table To override this and specify the storage engine to use for all tables created on the destination server use the new storage engine option If the destination server supports the new engine all tables use that engine To specify the storage engine to use for tables for which the destination server does not support the original stor
454. t socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt e ssl ca The path to a file that contains a list of trusted SSL CAs e ssl cert The name of the SSL certificate file to use for establishing a secure connection e ssl cert The name of the SSL key file to use for establishing a secure connection e Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required Start timeout lt timeout_in_seconds gt Number of seconds to wait for server to start Default 10 seconds e verbose v Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit e write command lt file_name gt w lt file_name gt Path name of file in which to write the command used to launch the new server instance EXAMPLES The following command demonstrates how to create a new instance of a running server set the root user password and enable binary logging shell gt mkdir source test123 shell gt mysqlserverclone
455. t tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES For each example assume that the root user on localhost has sufficient privileges to kill queries and connections Kill all queries created by user john that are less than 1 minute shell gt mysqlprocgrep server root localhost match user john age 1m kill query Kill all connections that have been idle for more than 1 hour shell gt mysqlprocgrep server root localhost match command sleep age 1h kill connection PERMISSIONS REQUIRED The user must have the SELECT privilege on the mysql database 5 14 mysqlreplicate Set Up and Start Replication Between Two Servers This utility permits an administrator to start replication from one server the master to another the slave The user provides login information for the slave and connection information for connecting to the master It is also possible to specify a database to be used to test replication The utility reports conditions where the storage engines on the master and the slave differ It also reports a warning if the InnoDB storage engine differs on the master and slave For InnoDB to be the same both servers must be running the same type of InnoDB built in or t
456. t under the terms of the GNU General Public License as published by the Free Software Foundation version 2 of the License This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General Public License for more details You should have received a copy of the GNU General Public License along with this program if not write to the Free Software Sil axceyaieilain Sie wWaieiela wiloor ABOS Eon AMATO MOm SOSA This file contains an example of how to build a customized utility using the MySQL Utilities scripts and libraries import optparse Impore os import sys from from from from from from mysql mysql mysql mysql mysql mysql utilities utilities utilities utilities utilities utilities import VERSION_FRM command import dbcopy command import serverclone command import userclone common server import Server common options import parse_connection 201 MySQL Utilities copy_server py sample from mysql utilities exception import UtilError Constants NAME example copy_server DESCRIPTION copy_server copy an existing server USAGE prog server user pass host port socket __new dir lt path gt new id lt server_id gt new port lt port gt databases lt db list gt users lt user list gt Setup the comma
457. tabases MySQL Utilities fully supports MySQL Server versions 5 1 and above It is also compatible with MySQL Server 5 0 but not every feature of 5 0 may be supported It does not support MySQL Server versions 4 x In this section we present a number of example administrative tasks Included in each is a description of the need goals example execution and a discussion about the specific options and techniques illustrated Also included is a description of the specific permissions required to execute the utilities demonstrated 3 1 Binary Log Operations The tasks described in this section relate to those that are performed on or with binary log files 11 How To Change The Location For Binary Log Files 3 1 1 How To Change The Location For Binary Log Files Important AN This documented task uses the mysqlbinlogmove utility which was introduced in MySQL Utilities 1 6 0 At some point a user might want to change the location where the binary log files are stored by default in the datadir There are many reasons for separating the binary log files from the database data These include fault tolerance performance disk management For example you may want to store the binary log files on a different device You can use the 1og bin startup option but simply changing this option is not enough and it will likely result on errors when starting the MySQL server In fact the existing binary log files also need to be moved to the
458. tables into synch Example Execution shell gt mysqldbcompare serverl root root localhost 13001 server2 root root localhost 13002 menagerie a difftype SQL show reverse quiet Checking databases menagerie on serverl and menagerie on server2 Row counts are not the same among menagerie pet and menagerie pet gt Transformation for changes for serverl DELETE FROM menagerie pet WHERE pet_num DELETE FROM menagerie pet WHERE pet_num NSERT INTO menagerie pet pet_num name owner species sex birth death WAIEUIGS VILL Wwarolleic Vaoimetctae Melee MEY 2OIO 10 20 T e 20 General Operations Transformation for reverse changes changes for server2 DELETE FROM menagerie pet WHERE pet_num 11 INSERT INTO menagerie pet pet_num name owner species sex birth death WALICS V0 Ydeomdcm AnS EES Velog Van VZ20LO0 10 20 VA0L2 07 Oil p INSERT INTO menagerie pet pet_num name owner species sex birth death WAS Vi2 Minoneti e WAnt a Veega Vey VAOTO O 20 O e Discussion In the example above we connected to two servers and compare the database named menagerie We enabled the transformation statements using a combination of options as follows The difftype SQL option in
459. ted data directory of the host server 23 Creating a New User With The Same Privileges as Another User The port option is always required for running the utility in default mode it is not needed for diagnostic mode You must supply a valid unused port The utility will check to see if the port is in use and if so will produce an error We use the basedir option instead of the server option because we were faced with a situation where the original server was offline down Note that you can use the basedir option for a running server if you do not want the utility to connect to the original server in any way Permissions Required The permissions for using mysql frm will vary and depend entirely on how you use it If you use the utility to read frm files in a protected folder like the example above in either mode you must have the ability to run the server as root If you use the utility with a server connection the user you use to connect must have the ability to read system variables at a minimum read access to the mysql database Tips and Tricks The utility is designed to work on the host where the frm files reside It does not permit connecting to a remote host to read frm files If something goes wrong during the spawning of the server use the verbosity option three times vvv to turn on maximum depth debug statements This will ensure you will see all of the messages from the start of the spawned server
460. tes to the same object from multiple masters e Replication is asynchronous OPTIONS mysqlrpims accepts the following command line options e daemon lt command gt Run as a daemon The command can be start start daemon stop stop daemon restart stop then start the daemon or nodetach start but do not detach the process This option is only available for POSIX systems 149 OPTIONS format lt format gt f lt format gt Display the replication health output in either grid default tab csv or vertical format help Display a help message and exit interval lt seconds gt i lt seconds gt Interval in seconds for reporting health Default 15 seconds Minimum is 5 seconds license Display license information and exit log lt log_file gt Specify a log file to use for logging messages log age lt days gt Specify maximum age of log entries in days Entries older than this will be purged on startup Default 7 days masters lt master connections gt Connection information for master servers List multiple masters in comma separated list To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include t
461. th all the slaves to make sure that no transactions is lost Fortunately the mysql failover utility is capable of executing this full process automatically and in a optimized way Let s assume that a replication topology with one master server1 3311 and four slaves server2 3312 server3 3313 server4 3314 server 3315 was previously setup Example Execution Start the mysql failover utility in console mode default shell gt mysqlfailover master root server1 3311 slaves root server2 3312 root server3 3313 root server4 3314 root server5 3315 log log txt rpl user rpl rpl NOTE Log file log txt does not exist Will be created Checking privileges MySQL Replication Failover Utility Failover Mode auto Next Interval Fri Jul 26 10 17 52 2013 Master Information Binary Log File Position Binlog_Do_DB Binlog_ignore_DB master bin 000001 151 GTID Executed Set None Replication Health Status 4 4 4 4 4 4 lawyer jevoRete role state gtid_mode health 4 4 4 4 4 4 serverl 3311 MASTER UP ON OK server2 sak SLAVE UP ON OK server3 T3313 SLAVE UP ON OR server4 3314 SLAVE UP ON OK server5 ISIS SLAVE UP ON OK 4 4 4 4 4 4 Q eMliie RWeicweisin lelneallicin eM Ias
462. the rp1 user option is either created automatically or if it exists it needs the REPLICATION SLAVE privilege To run the mysql rpladmin utility with the health command the account used on the master needs an extra SUPER privilege As for the switchover command all the users need the following privileges SUPER GRANT OPTION SELECT RELOAD DROP CREATE and REPLICATION SLAVE 5 17 mysqlrplcheck Check Replication Prerequisites 164 OPTIONS This utility checks the prerequisites for replication between a master and a slave These checks called tests are designed to ensure a healthy replication setup The utility performs the following tests 1 Is the binary log enabled on the master 2 Are there binary logging exceptions such as _do_db or _ignore_db settings If so display them 3 Does the replication user exist on the master with the correct privileges 4 Are there server _idconflicts on Is the slave connected to this master If not display the master host and port 6 Are there conflicts between the master info file on the slave and the values shown in SHOW SLAVE STATUS on the master 7 Are the InnoDB configurations compatible plugin vs native 8 Are the storage engines compatible have same on slave as master 9 Are the lower_case_tables_names settings compatible Warn if there are settings for lowercase uppercase table names that can cause problems See Bug 59240 10 Is the slave
463. the database s on the source server On the destination server the user must have the following privileges for the copied database s CREATE ALTER SELECT INSERT UPDATE LOCK TABLES DROP if drop first option is used and SUPER depending on the objects DEFINER value 5 5 mysqldbexport Export Object Definitions or Data from a Database This utility exports metadata object definitions or data or both from one or more databases By default the export includes only definitions mysqldbexport differs from mysqldump in that it can produce output in a variety of formats to make your data extraction transport much easier It permits you to export your data in the format most suitable to an external tool another MySQL server or other use without the need to reformat the data To exclude specific objects by name use the exclude option with a name in db obj format or you can supply a search pattern For example exclude db1 trig1 excludes the single trigger and exclude trig_ excludes all objects from all databases having a name that begins with trig and has a following character To skip objects by type use the skip option with a list of the objects to skip This enables you to extract a particular set of objects say for exporting only events by excluding all other types Similarly to skip creation of UPDATE statements for BLOB data specify the skip blobs option To specify how to display output use one of the
464. the users console User defined variables may also be set by passing them as arguments to the mysqluc command For example to set the SERVER1 variable and launch the console the user can launch the console using this command shell gt mysqluc SERVER1 root localhost The user can provide any number of user defined variables but they must contain a value and no spaces around the character Once the console is launched the user can see all variables using the show variables command OPTIONS e version show program s version number and exit e help show the program s help page e license Display license information and exit e verbose v control how much information is displayed For example v verbose vv more verbose vvv debug e quiet suppress all informational messages e execute lt commands gt e lt commands gt Execute commands and exit Multiple commands are separated with semi colons Note R Some platforms may require double quotes around the command list 189 NOTES NOTES e utildir lt path gt location of utilities e width lt number gt Display width Using the execute option or piping commands to the console may require quotes or double quotes for example on Windows EXAMPLES To launch the console use this command shell gt mysqluc The following demonstrates launching the console and running the console command help utilities to see a list of a
465. thors expressed herein do not necessarily state or reflect those of the United States Government or the University of California and shall not be used for advertising or product endorsement purposes The source code for the md5 module contains the following notice Copyright C 1999 2002 Aladdin Enterprises All rights reserved This software is provided as is without any express or implied warranty In no event will the authors be held liable for any damages arising from the use of this software Permission is granted to anyone to use this software for any purpose including commercial applications and to alter it and redistribute it freely subject to the following restrictions 1 The origin of this software must not be misrepresented you must not claim that you wrote the original software If you use this software in a product an acknowledgment in the product documentation would be appreciated but is not required 2 Altered source versions must be plainly marked as such and must not be misrepresented as being the original software 3 This notice may not be removed or altered from any source distribution L Peter Deutsch ghost aladdin com ndependent implementation of MD5 RFC 1321 This code implements the MD5 Algorithm defined in RFC 1321 whose text is available at Rete ry www lett sorg rie ricl3Z lr xt The code is derived from the text of the RFC including the test suite section A 5 but excluding the
466. tion All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required skip lt skip objects gt Specify objects to skip in the operation as a comma separated list no spaces Permitted values are CREATE_DB DATA EVENTS FUNCTIONS GRANTS PROCEDURES TABLES TRIGGERS and VIEWS 98 NOTES e skip blobs Do not export BLOB data e skip gtid Skip creation of GTID_ PURGED statements e all Generate an export file with all of the databases and th
467. tional divergence For example you may maintain a production server and a development server The development server may have the same databases and the same structures as the production server but maybe not the same data However the natural course of administrative tasks and maintenance can sometimes leave the development server behind When this happens you need to have a way to quickly check the schema for a table on the production server to see if the development server has the same structure The utility of choice for this operation is mysqldiff Objectives The goal is to compare a table schema on one machine to another and show they differ Example Execution shell gt mysqldiff serverl root root localhost server2 root root localhost 3307 world city world city changes for server2 serverl on localhost connected server2 on localhost connected Comparing world city to world city FAIL Object definitions differ changes for server2 yworllel liey sire Yworellel eey 4 6 4 7 CountryCode char 3 NOT NULL DEFAULT District char 20 NOT NULL DEFAULT 19 How Can Synchronize a Table on Two Servers Where Neither is Up to date Population int 11 NOT NULL DEFAULT 0 Clamate enumi tropialcal dry mild comtinental pollar DEFAULT NULL PRIMARY KEY IED KEY CountryCode CountryCode CONSTRAINT city_ibfk_1 FOREIGN KEY
468. tive For example e audits Match any string that starts with audit Match any string consisting of one or more characters For documentation about the standard SQL pattern matching syntax see Pattern Matching REGEXP Pattern Matching POSIX Standard REGEXP patterns are more powerful than the simple patterns defined in the SQL standard A regular expression is a string of ordinary and special characters specified to match other strings Unlike SQL Patterns REGEXP patterns are case sensitive The REGEXP syntax defines the following characters with special meaning Match any character Par Match the beginning of a string e Match the end of a string e Match zero or more repetitions of the preceding regular expression Match one or more repetitions of the preceding regular expression e Match zero or one repetition of the preceding regular expression l Match either the regular expressions from the left or right of e 72 REGEXP Pattern Matching POSIX Indicates a set of characters to match symbol acquires a different meaning if it is the first character of the set matching the complementary set i e all the characters that are not in the set will Note R Special characters lose their special meaning inside sets In particular the caret be matched m Match m repetitions of the preceding regular expression m n Match from m to n repetitions of the preceding regular e
469. tivities In this example there are three servers identified by their UUID values The server running at localhost 3307 is a primary whereas the other servers are secondaries It is sometimes necessary to take secondaries offline for maintenance However before stopping a server it is important to remove it from the group to avoid having the Fabric failure detector trigger any action This can be done through the following commands server_uuid should be replaced with a server UUID value a value of the form d2369bc9 2009 11e3 93c6 f0def14a00F 4 shell gt mysqlfabric group remove my_group server_uuid A primary cannot be removed from a group To disable any primary in a group execute this command 232 Example Fabric and Sharding shell gt mysqlfabric group demote my_ group If a group contains no servers it is empty and can be destroyed removed with this command shell gt mysqlfabric group destroy my_group It is also possible to force removal of a nonempty group by specifying the parameter force This command removes all servers from my_group and removes the group shell gt mysqlfabric group destroy my_group force 8 3 2 Example Fabric and Sharding This example explores sharding The essence of a sharding solution that uses MySQL involves partitioning the data into independent sets independent MySQL Servers and directing each client to the partition MySQL Server that has the data the client wants to
470. to the connectors ttl The Time To Live TTL measured in seconds is passed together with other information to the connector This is used by the connector to invalidate the caches and reload them from a MySQL Fabric node after the TTL has expired 8 2 3 4 The Configuration Parameter param The param option allows you to override configuration options at runtime The syntax is param section option value For example shell gt mysqlfabric manage setup param storage user fabric_store param storage password secret shell gt mysqlfabric param storage user fabric_store param storage password secret manage setup shell gt mysqlfabric manage setup param storage address localhost 13000 param storage user root param protocol xmlrpc password secret For additional information about the available options see Section 8 2 3 2 Configuration File 8 2 4 Starting and Stopping MySQL Fabric Nodes To start or stop MySQL Fabric nodes use the mysql fabric command see Section 8 4 The mysql fabric Utility This command requires that MySQL Fabric and Connector Python be installed and assumes that you have set up the backing store The following command starts a MySQL Fabric node and should be run on one of the servers listed in the protocol xmlrpc section in the configuration file shell gt mysqlfabric manage start This command starts a MySQL Fabric node on the machine where it is executed and prints the
471. to the master it also displayed the state of the IO and SQL thread for each slave We used the master server for the master option but for the slaves we provided the option discover slaves login which provides the user name and password for the account used to connect 43 How To Check If Data Is Correctly Replicated to each slave Without this we would not be able to determine if the slave is attached currently or the state of its threads The discover slaves login option applies to all slaves If you do not have the same user defined on all of your slaves you can use the prompt option to prompt for the user and password for each slave To get the state of the slave threads use the verbose option Permissions Required The user connected to the master must have the REPLICATION SLAVE privilege The user specified with the discover slaves login option that logs into each slave must have the REPLICATION CLIENT privilege Tips and Tricks You can also display multiple tiered topologies by providing the recurse option Notice in the example we used the option discover slaves login This is a shortcut feature built into every utility If you type the first N letters of a utility that uniquely identifies it among the options for said utility the utility accepts it as if you typed the entire string For example the full name of the option we used is discover slaves login 3 4 6 How To Check If
472. tor logic rather than via a proxy One reason is to reduce complexity rather than having a pool of proxy processes a single proxy would represent a single point of failure the logic is just part of each connector instance The second reason is to avoid the latency involved in all operations being diverted via a proxy process which is likely to be an a different machine What is the difference between a shard key and a shard identifier The shard key is simply the value of a column from one or more tables The shard key does not change if a row is migrated from one shard or server to another The shard key is mapped to a shard id using either a HASH or RANGE based mapping scheme the shard id represents the shard itself As an example if an existing shard were split in two then some of the rows would map to one shard s shard id and the rest to the other s any given row s shard key would not change as part of that split Very importantly shard keys are known to the application while shard ids are not and so any changes to the topology of the collection of servers is completely transparent to the application 8 9 10 Does my application need to change when a shard is moved to a different MySQL Server or split into multiple shards 281 MySQL Fabric Frequently Asked Questions No Because the application deals in shard keys and shard keys do not change during shard moves or splits 8 9 11 Is it possible to perform cross s
473. torage engines compatibility pass hecking lower_case_table_ names settings pass hecking slave delay seconds behind master pass OOQooroan PT a oO Q 167 EXAMPLES done As shown in the example you must provide valid login information for both the master and the slave To perform the same command but also display the contents of the master information file on the slave and the values of SHOW SLAVE STATUS as well as additional details use this command shell gt mysqlrplcheck master root host1 3310 slave root host2 3311 show slave status vv master on hostl connected slave on host2 connected Test Description status Checking for binary logging on master pass Are there binlog exceptions pass Replication user exists pass Checking server_id values pass master id 10 slave id 11 Is slave connected to master pass Check master information file pass Master information file Master_Log_File clone bin 000001 Read_Master_Log_Pos 482 MEScer_ Hosic 8 Inesicil Master_User rpl Master_Password XXXX Master Port 2210 Connect_Retry 60 Master_SSL_Allowed 0 Master_SSL_CA_File Master_SSL_CA_Path Masterisslucere Master_SSL_Cipher Master_ 55L Key i Master_SSL_Verify_Server_Cert 0 Checking InnoDB compatibility pass Checking storage engines compatibility pass Checking lower_case_table_names settings pass Master lower_case_tab
474. ts the server with a socket file You can optionally add a password for the login user account on the new instance If the user does not have read and write access to the folder specified by the new data option the utility shall issue an error Similarly if the folder specified by new data exists and is not empty the utility will not delete the folder and will issue an error message Users must specify the delete data option to permit the utility to remove the folder prior to starting the cloned server OPTIONS mysqlserverclone accepts the following command line options e help Display a help message and exit e license Display license information and exit delete data 181 OPTIONS Delete the folder specified by new data if it exists and is not empty basedir The base directory for the MySQL server source as an alternative to the server option shell gt mysqlserverclone basedir source mysql 5 6 new data source temp_3007 new port 3007 new id 101 root root mysqld log bin gtid mode on log slave updates enforce gtid consistency master info repository table report host localhost report port 3007 delete force Ignore the maximum path length and the low space checks for the new data option mysqld lt options gt Additional options for mysqld To specify multiple options separate them by spaces Use appropriate quoting as necessary Fo
475. tting itself down in the event that it loses connectivity with the independent host root host1 crm configure property stonith enabled false Roughly speaking STONITH refers to one node trying to kill another in the even that it believes the other has partially failed and should be stopped in order to avoid any risk of a split brain scenario To prevent a healthy resource from being moved around the cluster when a node is brought back on line Pacemaker has the concept of resource stickiness which controls how much a service prefers to stay running where it is 253 Setting up and testing your system root host1 crm configure rsc_defaults resource stickiness 100 In the next steps we will describe how to configure the different resources as a cluster root host1 crm configure edit This will open up your default text editor and you should use it to add the following lines into the cluster configuration primitive p_drbd_mysql ocf linbit drbd params drbd_resource clusterdb_res op monitor interval 15s primitive p_fabric_mysql ocf heartbeat mysql fabric params binary usr local bin mysqlfabric Contig var lib myeql drbd tabricl cig op start timeout 120s interval 0 op stop timeout 120s interval 0 op monitor interval 20s timeout 30s primitive p_fs_mysql ocf heartbeat Filesystem params device dev drbd0 directory var lib mysql_drbd fstype ext4 primitive p_ip_mysql ocf heartb
476. ty to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf If any database identifier specified as an argument contains special characters or is a reserved word then it must be appropriately quoted with backticks In turn names quoted with backticks must also be quoted with single or double quotes depending on the operating system i e in Windows or in non Windows systems in order for the utilities to read backtick quoted identifiers as a single argument For example to export a database with the name weird db name it must be specified as argument using the following syntax in non Windows weird db name Keep in mind that you can only take advantage of multiprocessing if your system has multiple CPUs available for concurrent execution Also note that multiprocessing is applied at a different level according to the operating system where the mysqldbexport utility is executed due to python limitations In particular it is applied at the database level for Windows i e different databases are concurrently exported and at the table level for Non Windows POSIX systems i e different tables within the same database are concurrently exported 99 EXAMPLES EXAMPLES To export the definitions of the database dev from a MySQL server on the local host via port 3306 producing output consisting of CREATE statements use this command
477. ual to get a quick solution to an administrative task complete with explanation of how to run the utilities involved and the options and parameters needed See the tasks chapter for this information You can use the manual to learn what utilities exist and how each fits into your own administrative needs See the utility overview chapter for this information You can also use the manual to get more information about each utility and what each option and parameter does via the manuals section The manual concludes with a look at extending the MySQL Utilities library a look at the developer testing environment and a list of frequently asked questions Legal Notices Copyright 2006 2015 Oracle and or its affiliates All rights reserved This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws Except as expressly permitted in your license agreement or allowed by law you may not use copy reproduce translate broadcast modify license transmit distribute exhibit perform publish or display any part in any form or by any means Reverse engineering disassembly or decompilation of this software unless required by law for interoperability is prohibited The information contained herein is subject to change without notice and is not warranted to be error free If you find any errors please report them to us in writing
478. ue is 0 all entries from the start of the log are displayed Accepted formats yyyy mm ddThh mm ss or yyyy mm dd e status lt STATUS gt Commaz separated list of status values or intervals to search for all audit log records with a matching status Status values are non negative integers corresponding to MySQL error codes Status intervals are closed i e include both endpoints and defined simply using a dash between its endpoints For Example 1051 1068 1075 1109 1146 The status option is available as of MySQL Utilities 1 2 4 1 3 3 e users lt USERS gt u lt USERS gt Commaz separated list of user names to search for their associated log entries For example dan jon john paul philip stefan e verbose V Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES This utility is available as of MySQL Utilities 1 2 0 This utility can only be applied to servers with the audit log plugin enabled And the audit log plugin is available as of MySQL Server versions 5 5 28 and 5 6 10 This utility support both of the existing audit log file formats old and new The new audit log format is supported as of MySQL Utilities 1 4 3 See The Audit Log File for more information about available file formats This utility requires the use of Python v
479. unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required verbose v 176 NOTES Specify how much information to display Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug e version Display version information and exit NOTES The data consistency check is performed per table using a checksum table If the calculated checksum differs it indicates the tables are not synchronized Nevertheless since the checksum operation is not collision free there is a very small probability that two different tables can produce the same checksum Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the rep
480. unning Samples cccccecceeeeeeeeeeeeeeaeeaeeeeeeeeeeeeaeaaea 264 88 5 R NAMO TESTE serpens pirena pirer EEn REEN EEEE donee cus EEEE E REANA EEEE REEE A EIRENE 266 8 8 6 Running Demonstration Programs cceeeeee nents cece tee eeeee ee eeeeeeeeeeeeseaaeeeeeeaeeeeeaeaeeeeeaaaes 266 8 8 7 A Complete Example Working with Employee Data ccccccceeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaed 267 8 8 8 How Connector J Chooses a MySQL Server ccccceeeeeeeeeee ee ae eeeeteeeeeeeeeaeaaeeneeeeeeeeeaeaaea 271 8 8 9 Using Hibernate with MySQL Fabric ccceeeececeeeeeeeee cece ee aa ea eeeeeeeeeeeeaaaaeeeeeseeeeeeaeaaaeees 272 8 8 10 Connector J Fabric Support Reference cccceccceeeeeeeeeeeee ee ae aeeeeeeeeeeeeeaeaaeaneeeeeeeeeaeaae 275 8 9 MySQL Fabric Frequently Asked Questions ccccceeeee cece eter ce eeeeee ener cess ae aaeaeeeeeeeeeaeaaaaneneeeess 276 MySQL Fabric is a system for managing a farm of MySQL servers MySQL Fabric provides an extensible and easy to use system for managing a MySQL deployment for sharding and high availability 213 Introduction to Fabric This document describes MySQL Fabric beginning with a short introduction providing instructions on how to download and install MySQL Fabric and a quick start guide to help you begin using and experimenting with MySQL Fabric Later sections provide details for MySQL Fabric aware connectors For notes detailing the chang
481. ups An example might be the department table from an employee database the contents of the department table being small enough to be stored in every server and where that data could be referenced by any record from one of the sharded employee tables Similarly any schema changes would be sent to the Global Group where they can be replicated to all of the HA Groups How many MySQL Servers can scale to There is no limit either to the number of HA Groups or the number of servers within a HA group Can MySQL Fabric introduce contention or deadlock No A single transaction can only access data from a single shard Global Table data and all writes are sent to the Primary server within that shard s HA Group In other words all writes to a specific row will be handled by the same MySQL Server and so InnoDB s row based locking will work as normal What happens when my data set or usage grows and a shard grows too big MySQL Fabric provides the ability to either e Move a shard to a new HA group containing larger or more powerful servers e Split an existing shard into two shards where the new shard will be stored in a new HA Group In the future different levels of granularity may be supported for shard splitting Is there extra latency when using MySQL Fabric No Because the routing is handled within the connector there is no need for any extra hops to route the request via a proxy process Why does MySQL Fabric route using connec
482. ur mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt show reverse Produce a transformation report containing the SQL statements to conform the object definitions specified in reverse For example if changes for is set to server1 also generate the transformation for server2 Note The reverse changes are annotated and marked as comments skip checksum table Skip the CHECKSUM TABLE step in the data consistency check Added in release 1 4 3 skip data check Skip the data consistency check skip diff Skip the object definition difference check Skip object compare Skip the object comparison check skip row count Skip the row count check 83 NOTES Span key size lt number of bytes to use for key gt Change the size of the key used for compare table contents A higher value can help to get more accurate results comparing large databases but may slow the algorithm Default value is 8 ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection s
483. uration file mylogin cnf If any database object identifier specified as an argument contains special characters or is a reserved word then it must be appropriately quoted with backticks In turn names quoted with backticks must also be quoted with single or double quotes depending on the operating system i e in Windows or in non Windows systems in order for the utilities to read backtick quoted identifiers as a single argument For example to show the difference between table weird table1 from database weird db name and table weird table2 from database other weird db name the objects pair must be specified using the following syntax in non Windows weird db name weird table1 other weird db name weird table2 EXAMPLES To compare the employees and emp databases on the local server use this command shell gt mysqldiff serverl root localhost employees emp1 serverl on localhost connected WARNING Objects in serverl employees but not in server2 emp1 EVENT el Compare failed One or more differences found shell gt mysqldiff serverl root localhost employees t1 emp1 t1 employees t3 emp1 t3 serverl on localhost connected Comparing employees tl to empl tl PASS serverl on localhost connected Comparing employees t3 to emp1 t3 PASS Success All objects are the same shell gt mysqldiff serverl root localhost employees salaries empl salaries
484. urrent audit log file i e without the timestamp extension For example shell gt mysqlauditadmin file stats audit log name MySQL SERVER data audit log 4 4 4 File Size Created Last Modified 4 4 4 4 audit log 1375370617938 78237 4716 The ane 1 les2os37 2013 Thu Aug 1 les2as37 2013 I anebhie Los 6062 ou Arey il aangaan BOs a Ayre il iss A one AOS audit log 13753705S495049727 335142503 Thu Aug 1 16 22 29 2013 Thu Aug 1 16 22 29 2013 4 4 4 4 Note R If an audit log file with the timestamp extension is specified in this example for the audit log name option only the information of the specified file will be displayed as opposed to the file statistics of all existing files 3 6 3 How Do I Show All INSERT and UPDATE Queries That Failed Useful information can be recorded in the audit log files and also a considerable amount of it However how can someone easily filter this information and search for specific events for instance in order to determine the possible cause of a problem For example suppose that someone reported that some data changes are missing INSERT or UPDATE
485. us if you ran a partial export that includes the replication commands and you have GTIDs enabled you should use the skip rp1 option to skip the replication commands and restart replication manually Should your data be large enough to make the use of mysqidbexport impractical you can use mysqldbexport to generate the correct replication commands anyway by using the export definitions option This will generate the SQL statements for the objects but not the data You can then use the replication commands generated with your own backup and restore tools You can use the option rp1 slave to generate a output stream that considers the source server a slave and uses the source servers master settings for generating the CHANGE MASTER command 3 2 2 How Do Make a Copy of a Database on the Same Server If you are working with a database and want to experiment with changes to objects or data either from direct manipulation SQL commands or as a result of interaction with an application it is prudent to always have a copy to fall back to if something should go wrong Naturally a full backup is key for any production server but what if you just want to do something as a test or as a prototype Sure you can restore from your backup when the test is complete but who has the time for that Why not just make a copy of the database in question and use it Objectives The goal is to make a copy of a database and rename it to another name We w
486. use it as input for other programs ssl ca The path to a file that contains a list of trusted SSL CAs ssl cert The name of the SSL certificate file to use for establishing a secure connection ssl cert The name of the SSL key file to use for establishing a secure connection Ssl Specifies if the server connection requires use of SSL If an encrypted connection cannot be established the connection attempt fails Default setting is 0 SSL not required version Display version information and exit 139 NOTES NOTES For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value The path to the MySQL client tools should be included in the PATH environment variable in order to use the authentication mechanism with login paths This will allow the utility to use the my_print_defaults tools which is required to read the login path values from the login configuration file mylogin cnf EXAMPLES Find all objects with a name that matches the pattern t_ the letter t followed by any single character shell gt mysqlmetagrep pattern t_ server john localhost 4 4 4 4 Connection Object Type Object Name Database 4
487. ut values The utility provides the flexibility to be executed separately for different set of servers only affecting different parts of the replication system at each time For example considering an heterogeneous system where slaves have a different performances composed by one master M and three slaves S1 S2 and S3 To minimize the impact on the master the user can run the utility first for the master M and the fastest slave S1 and then run it again only for the slaves S1 S2 and S3 If no consistency issues are found in the first execution M S1 nor in the second execution S1 S2 S3 then by transitivity and due to the inclusion of the same server S1 in both checks it can be said that there is no consistency issues in the all topology M S1 S2 S3 at the time the first check was completed This kind of execution must be performed sequentially and not concurrently otherwise the synchronization process of each instance will likely affect the other and it will not work properly 3 4 7 How To Fix Errant Transactions on the Replication Topology Important A This documented task uses the mysqlslavet rx utility which was introduced in MySQL Utilities 1 6 0 At some point in time when performing some maintenance administration operation or other task which verify your replication topology you may discover the existence of errant transactions Some utilities like mysqlfailover and mysqlrpladmin will detect
488. var log usr local storage address localhost 3306 fabric_store secret mysql_fabric user password database auth_plugin connection_timeout 6 connection_attempts 6 connection_delay 1 servers fabric_server secret user password backup_user backup_password restore _user restore_password secret unreachable_timeout 5 secret protocol xmlrpe address localhost 32274 threads 5 user admin password secret disable_authentication no realm ssl_ca ssl_cert ssl_key MySQL Fabric protocol mysql usr local etc fabric_backup fabric_restore mysql_native_password address localhost 32275 user admin password secret disable_authentication no ssl_ca ssl_cert ssl_key executor executors 5 logging level INFO url file var log fabric log 220 Configuring MySQL Fabric sharding mysqldump_program usr bin mysqldump mysqlclient_program usr bin mysql Seatistics prune_time 3600 failure_tracking motifications 300 notification_clients 50 notification_interval 60 failover_interval 0 detections 3 detection_interval 6 detection_timeout 1 prune_time 3600 connector cdl Sil Each section has one or more variables defined that provide key information to the MySQL Fabric system libraries You might not have to change any of these variables other than the users and passwords F
489. ver CREATE USER fabric_restore localhost TDENTIVIED BY secreti GRANT ALTER ALTER ROUTINE CREATE CREATE ROUTINE CREATE TABLESPACE CREATE VIEW DROP EVENT INSERT LOCK TABLES REFERENCES SELECT SUPER TRIGGER ON TO fabric_restore localhost 8 2 3 2 Configuration File The location of the MySQL Fabric configuration file varies depending on the operating system it is installed on and how you installed it If you are using the pre built packages from http dev mysq com downloads utilities the default configuration file locations are as follows Table 8 1 Default MySQL Fabric configuration file location Platform Package Location Microsoft Windows mysql utilities 1 5 6 win32 msi UTILITIES_INSTALLDIR etc mysql fabric cfg 219 Configuring MySQL Fabric Platform Package Location Ubuntu Linux 14 04 mysql etc mysql fabric cfg utilities 1 5 6 1ubuntu14 04_all de Debian Linux 6 0 mysql etc mysql fabric cfg utilities 1 5 6 1debian6 0_all deb Red Hat Enterprise Linux 6 mysql etc mysql fabric cfg Oracle Linux 6 utilities 1 5 6 1 el6 noarch rpm OS X mysql utilities 1 5 6 0sx10 9 dmg etc mysql fabric cfg Modify the configuration file and include the users and passwords defined in the previous step Section 8 2 3 1 Create the Associated MySQL Users here is an example Fabric configuration file DEFAULT prefix sysconfdir logdir
490. ves having an office at 1895 Preston White Drive Reston VA 20191 CNRI and the Individual or Organization Licensee accessing and otherwise using Python 1 6 1 software in source or binary form and its associated documentation 2 Subject to the terms and conditions of this License Agreement CNRI hereby grants Licensee a nonexclusive royalty free world wide license to reproduce analyze test perform and or display publicly prepare derivative works distribute and otherwise use Python 1 6 1 alone or in any derivative version provided however that CNRI s License Agreement and CNRI s notice of copyright i e Copyright c 1995 2001 Corporation for National Research Initiatives All Rights Reserved are retained in Python 1 6 1 alone or in any derivative version prepared by Licensee Alternately in lieu of CNRI s License Agreement Licensee may substitute the following text omitting the quotes Python 1 6 1 is made available subject to the terms and conditions in CNRI s License Agreement This Agreement together with Python 1 6 1 may be located on the Internet using the following unique persistent identifier known as a handle 1895 22 1013 This Agreement may also be obtained from a proxy server on the Internet using the following URL http hdl handle net 1895 22 1013 3 In the event Licensee prepares a derivative work that is based on or incorporates Python 1 6 1 or any part thereof and wants to make the derivative work
491. ves login lt slave login gt 170 OPTIONS Supply the user and password in the form lt user gt lt passwd gt or lt login path gt for discovering slaves and relay slaves in the topology For example discover joe secret will use joe as the user and secret as the password for each discovered slave format lt format gt f lt format gt Specify the display format for column list output Permitted format values are grid csv tab and vertical The default is grid This option applies only if show list is given master lt source gt Connection information for the master server To connect to a server it is necessary to specify connection parameters such as user name host name password and either a port or socket MySQL Utilities provides a number of ways to supply this information All of the methods require specifying your choice via a command line option such as server master slave etc The methods include the following in order of most secure to least secure e Use login paths from your mylogin cnf file encrypted not visible Example lt ogin path gt lt port gt lt socket gt e Use a configuration file unencrypted not visible Note available in release 1 5 0 Example lt configuration file path gt lt section gt Specify the data on the command line unencrypted visible Example lt user gt lt passwa gt lt host gt lt port gt lt socket gt max depth lt
492. ves login to query the master for all registered slaves and use the user name and password specified to connect and determine the binlog files that can be purge shell gt mysqlbinlogpurge master root root localhost 13001 discover slaves login rpl1 rpl Discovering slaves for master at localhost 13001 Discovering slave at localhost 13002 Found slave localhost 13002 Discovering slave at localhost 13003 Found slave localhost 13003 Discovering slave at localhost 13004 Found slave localhost 13004 Latest binlog file replicated by all slaves mysql bin 000005 Purging binary logs prior to mysql bin 0O00UG Discussion The previous example illustrates how to purge the binlog files from a master in a replication scenario We used the master option for the master server but for the slaves we provided the option discover slaves login and the utility used the specified information to determinate the available binlog files on the server and for each slave verified the latest binlog file that has been loaded finally purges the latest binlog that is not required for any of the slaves In example all binlog files that the utility determinate that were not required by any of the slaves were purged but the option binlog can be used to specify the first binlog file to keep not to purge from the not required binlog files The binlog files that will be keep available on the master are from the indicated file to the cur
493. vestigation Whatever the case without a running MySQL server it is not possible to know the structure of a table unless you keep meticulous notes and or use some form of high availability redundancy Fortunately there is a utility for situations like this The mysql frm utility can be used to discover the structure of a table Objectives With a downed or offline server discover the structure of a table More specifically generate the CREATE TABLE SQL command Example Execution shell gt sudo env PYTHONPATH SPYTHONPATH mysqlfrm basedir usr local mysql port 3333 user lt user gt usr local mysql data welford_kindle books frm Spawning server with user lt user gt Starting the spawned server on port 3333 done Reading frm files Reading the books frm file CREATE statement for usr local mysql data kindle books frm CREATE TABLE welford_kindle books ISBN char 32 NOT NULL PRIMARY KEY title char 128 DEFAULT NULL purchase_date date DEFAULT NULL cost float 10 2 DEFAULT NULL ENGINE MyISAM DEFAULT CHARSET latinl Hoo oCleme Discussion For this example we used three required parameters the base directory for the offline server basedir a new port to use for the spawned server port and a user name to use to run the spawned server port The later is necessary since we must launch the mysql frm utility as root Sudo in order to be able to read copy files from the protec
494. vseeacitscigaatavan tansuueunusunnenassecusanstadsvaismardsteatenasnnantedisaatavansunres 1 1 3 Oracle Linux and Red Hat LINUX 6 cece cecccscceeceueceaeeeeeeeeeeeeeeeeeeceueeeacecaceeaeeuaeeaeeeaeeeaeeeanaeass 2 TA DepiamMi LINUX Sa sncsencieatscnas tonic ans ested anne Paaneunseaeinedsanteduaslaitedenpad r outed amed nbnauaceatedantyentadnnsnitenat 2 1 5 MicroSoft WINKOWS iiciscicccccssccciaescoseceascedectaeensadeanseavdaascetvcaasouiactiacustdeaseedvdaadsadedadesvinadsaansadeaarsaudaads 2 LOOS A aa a A E E 3 MySQL Utilities is available in a number of repository formats Although you may not see your specific operating system or platform listed we provide general repository formats for most platforms If none of the available repositories are applicable to your platform you can use the source code repository and install MySQL Utilities from the command line The latest MySQL Utilities downloads are available at http dev mysql com downloads utilities 1 5 html The following sections discuss each repository For information specific to Fabric see Section 8 2 Installing and Configuring MySQL Fabric 1 1 Prerequisites MySQL Utilities requires Python 2 6 All of the Python code is written to conform to this version of Python For connecting to MySQL MySQL Utilities requires a MySQL Connector Python General Availability GA release version 2 0 4 2 1 2 or later If you do not have Connector Python installed see the download se
495. xample Enhancing the Example A complete solution for the example named copy_server py is located in the appendix It is complete in so far as this document explains but it can be enhanced in a number of ways The following briefly lists some of the things to consider adding to make this example utility more robust e Table locking Currently databases are not locked when copied To achieve a consistent copy of the data on an active server you may want to add table locking or use transactions for example if you are using InnoDB for a more consistent copy e Skip users not associated with the databases being copied e Do not copy users with only global privileges e Start replication after all of the users are copied makes this example a clone and replicate scale out solution e Stop new client connections to the server during the copy Conclusion If you find some primitives missing or would like to see more specific functionality in the library or scripts please contact us with your ideas or better still write them yourselves We welcome all suggestions in code or text To file a feature request or bug report visit http ougs mysql com For discussions visit http forums mysql com list php 155 6 2 MySQL Utilities copy_server py sample Copyright 200 Foundation Wiel 2013 Oracle and or its affiliates All rights reserved This program is free software you can redistribute it and or modify i
496. xes use the verbose option which prints the equivalent CREATE INDEX or ALTER TABLE for primary keys To display the best or worst non primary key indexes for each table use the best or worst option This causes the output to show the best or worst indexes from tables with 10 or more rows By default each option shows five indexes To override that provide an integer value for the option To change the format of the index lists displayed for the show indexes best and worst options use one of the following values with the format option grid default Display output in grid or table format like that of the mysql client command line tool e cSV Display output in comma separated values format e tab Display output in tab separated format sql Print SQL statements rather than a list e vertical Display output in single column format like that of the G command for the mysq1 client command line tool Note R The best and worst lists cannot be printed as SQL statements 133 OPTIONS OPTIONS mysqlindexcheck accepts the following command line options help Display a help message and exit license Display license information and exit best lt N gt If stats is given limit index statistics to the best N indexes The default value of Nis 5 if omitted format lt index_format gt f lt index_format gt Specify the index list display format for output produced by stats
497. xport all of the databases on the server with the a11 option This will generate an export file with all of the databases and the GTIDs executed to that point Importing this file on another server will ensure that server has all of the data as well as all of the GTIDs recorded correctly in its logs OPTIONS mysqldbexport accepts the following command line options e help Display a help message and exit license Display license information and exit e bulk insert b Use bulk insert statements for data e character set lt charset gt Sets the client character set The default is retrieved from the server variable character_set_client comment rpl Place the replication statements in comment statements Valid only with the rp1 option e display lt display gt d lt display gt 96 OPTIONS Control the number of columns shown Permitted display values are brief minimal columns for object creation full all columns and names only object names not valid for format sql The default is brief exclude lt exclude gt x lt exclude gt Exclude one or more objects from the operation using either a specific name such as db1 t1 ora search pattern Use this option multiple times to specify multiple exclusions By default patterns use LIKE matching With the regexp option patterns use REGEXP matching This option does not apply to grants export lt export gt e lt export gt
498. xpressed by indicating the group name in the form config path group name suchas C MySQL instances cnf instance_3307 port 3307 user root password pass host localhost ssl ca C newcerts cacert pem ssl cert C newcerts client cert pem ssl key C newcerts client key pem instance_3308 port 3308 user root password other pass host localhost ssl ca C newcerts cacert_2 pem ssl cert C newcerts client cert_2 pem ssl key C newcerts client key_2 pem shell gt mysqlreplicate master c MySQL instances cnf instance_3307 slave C MySQL instances cnf instance_3308 3 6 Specialized Operations The tasks described in this section relate to specific situations or configurations and may not apply in the general case For example some tasks require a specific commercial plugin More specifically the following tasks are those for use with the Audit Log Plugin 3 6 1 How Do Record Only Login Events The audit log plugin records MySQL servers activity By default it is set to write all auditable events to the log file which can represent a considerable amount of information Fortunately it is possible to control the type of information that is written to the audit log file changing the audit log plugin s policy The policy should be set to log only the events of interest avoiding wasting resources to log unnecessary events In particular if the audit log plugin is only used to monitor access to the database s
499. xpression e0 Define a matching group and matches the regular expression inside the parentheses For example e na Match a sequence of zero or more a 2 ap Match a sequence of one or more a Nap Match zero or one a taboa Match ab or cd axy Match a x or y lt fa f Match any character in the range a to f that is a b c d e or axy Match any character except a x or y lt faf Match exactly five copies of a Nad2 op Match from two to five copies of a e m abe 73 OPTIONS Match one or more repetitions of abc This is a brief overview of regular expressions that can be used to define this type of patterns The full syntax is described in the Python re module docs supporting the definition of much more complex pattern matching expression OPTIONS mysqlauditgrep accepts the following command line options end date lt END_DATE gt End date time to retrieve log entries until the specified date time range If not specified or the value is 0 all entries to the end of the log are displayed Accepted formats yyyy mm ddThh mm ss or yyyy mm dd event type lt EVENT_TYPE gt Commaz separated list of event types to search in all audit log records matching the specified types Supported values are Audit Binlog Dump Change user Close stmt Connect Out Connect Create DB Daemon Debug Delayed insert Drop DB Execute Fetch Field List Init DB Kill
500. y 8 9 MySQL Fabric Frequently Asked Questions 276 MySQL Fabric Frequently Asked Questions FAQ Categories e General Questions 277 e High Availability Questions 278 e Sharding Questions 280 e Consistency Questions 282 General 8 9 1 What is MySQL Fabric sissies ves casinee seauetacon cupties aaaea EEEO tes TARESTE Aaa 277 8 9 2 Is it necessary to use a MySQL Fabric specific Storage Engine 0ccccceeeeeeeteeeeeeeeeeeeeeeaaes 277 8 9 3 What versions of MySQL are supported by MySQL Fabric cccceceeeeeeeeeeeeeeeaeeeeeeeeeeeeeeeaes 277 8 9 4 What connectors support MySQL Fabric 0 ccccceceecceeeeeeeeeee cece aeeeeeeeeeeeeeeeaaaaeeseeeeeeeeeaeaaaeaes 277 8 9 5 Are transactions 1 Dee 277 8 9 6 How many machines are needed in order to use MySQL Fabric cceeeeeeeeeeeeeeeeeeneneeeeeeees 277 8 9 7 Do need to run an agent for each MySQL Server cceeeeeeeeeeee ee eeeeeeeeeeeeeaeaaeeneeeeeeeeeaeaaea 278 8 9 8 What interface is available to manage MySQL Fabric and its server farm accese 278 8 9 9 How does MySQL Fabric Compare with MySQL Cluster 0 cccccececseeeeeeeeeeeeeeaaeaeeeeeeeeeeeeaea 278 8 9 10 How is MySQL Fabric licensed ccceeeccceeeeeee cece eee e eee ete ee nese ee ae aa eet eeeeeeeeaeaaaaeeeeeeeeeaeaaee 278 8 9 11 What if MySQL Fabric doesn t do what need it t0 ce ecceeeececeeeeeeeceeeeeeeeseeeeeeseeaeeeeeaees 278 8 9 1 What
501. y is expected since the slaves are catching up with the master When the slaves have read and applied all the transactions from the master s binary log the Check slave delay test will pass Also in case the slave wasn t properly configured and pointing to the master specified the Is slave connect to master test would notify us of that with a FAIL or WARN status Permissions Required As for the required privileges to run these utilities The m_account user needs the following privileges for the mysqlreplicate SELECT and INSERT privileges on mysql database REPLICATION SLAVE REPLICATION CLIENT and GRANT OPTION As for the slave_acc users they need the SUPER privilege The repl user used as the argument for the 34 How Do Add New Servers to My Topology and Change Master Role rpl user option is either created automatically or if it exists it needs the REPLICATION SLAVE privilege Also when using GTIDs the slave_acc users must also have SELECT privilege over the mysql database in order to run the mysqirplcheck utility successfully Tips and Tricks In the mysqlreplicate utility we could have also used the test db option which creates a dummy database to test the replication setup However the mysql rplcheck provides more detailed information in that regard As previously stated the b option tells the utility to start replication from the first event recorded in the master s binary log Omitting this flag in turn
502. yed Use this option multiple times to increase the amount of information For example v verbose vv more verbose vvv debug If you use vwv the output will contain the state of the IO and SQL threads for each slave version Display version information and exit NOTES The login user must have the REPLICATE SLAVE and REPLICATE CLIENT privileges to successfully execute this utility Specifically the login user must have appropriate permissions to execute SHOW SLAVE STATUS SHOW MASTER STATUS and SHOW SLAVE HOSTS For the format option the permitted values are not case sensitive In addition values may be specified as any unambiguous prefix of a valid value For example format g specifies the grid format An error occurs if a prefix matches more than one valid value Mixing IP and hostnames is not recommended The replication specific utilities will attempt to compare hostnames and IP addresses as aliases for checking slave connectivity to the master However if your installation does not support reverse name lookup the comparison could fail Without the ability to do a reverse name lookup the replication utilities could report a false negative that the slave is not connected to the master For example if you setup replication using MASTER_HOST ubuntu net on the slave and later connect to the slave with mysqlrplcheck and have the master specified as master 192 168 0 6 using the valid IP address for ubuntu ne
503. ync pacemaker sysv rc conf libaiol root host2 apt get install drbd8 utils corosync pacemaker sysv rc conf libaiol Download and install the common server and client components on both hosts Our example downloads and installs the bundled binaries from dev mysql com Download the latest MySQL Server release bundle and install it on both machines using the following commands root host1 dpkg i mysql common_ ubuntul14 04_amd64 deb root host1 dpkg i mysql community server_ ubuntul4 04_amd64 deb root host1 dpkg i mysql community client_ ubuntul4 04_amd64 deb root host2 dpkg i mysql common_ ubuntul14 04_amd64 deb root host2 dpkg i mysql community server_ ubuntul4 04_amd64 deb root host2 dpkg i mysql community client_ ubuntul4 04_amd64 deb Next install MySQL Fabric Download MySQL Fabric by downloading the MySQL Utilities and install it using the following commands on each machine shell gt unzip mysql utilities zip shell gt cd mysql utilities shell gt python setup py install The script required to run MySQL Fabric with Pacemaker is not distributed with the packages and you need to manually download and install the script on each machine root host1 cp mysql fabric usr lib ocf resource d heartbeat root host1 chmod 755 usr lib ocf resource d heartbeat mysql fabric 247 Setting up and testing your system root host2 cp mysql fabric usr lib ocf resource d heartbeat root host2 chm
504. you could Now let us dive into the code for this example The first task is to connect to the original server We use the same connection mechanism as the other MySQL utilities by specifying a server option like this parser add_option server action store dest server type string default root localhost 3306 help connection information for original server in the form lt user gt lt password gt lt host gt lt port gt lt socket gt Once we process the options and arguments connecting to the server is easy Use the parse_connection method to take the server option values and get a dictionary with the connection values All of the heavy diagnosis and error handling is done for us so we just need to check for exceptions from mysql utilities common options import parse_connection Ery conn parse_connection opt server except parser error Server connection values invalid or cannot be parsed Now that we have the connection parameters we create a class instance of the server using the Server class from the server module and then connect Once again we check for exceptions from mysql utilities common server import Server server_options Pecina O E CCN role source serverl Server server_options Erys serverl connect except UtilError e Orsi RRO a me eraiamis G The next item is to get a list of all of the databases on the server We use the new server class instance
505. ype string help comma separated list of users to include in the copy omit for all users default None Now we process the rest of the arguments opt args parser parse_args Parse source connection values CENS conn parse_connection opt server except parser error Server connection values invalid or cannot be parsed Get a server class instance print Connecting to server server_options Verein Lairo E CCm role SS OUR uy 202 MySQL Utilities copy_server py sample serverl Server server_options EEV serverl connect except UtilError e print ERROR e errmsg Get list of databases from the server if not specified in options print Getting databases db_list if opt dbs_to_copy is None for db in serverl get_all_databases db_list append db 0 None else ico Clo aia Cj clos ire _Cooy sodas 1 2 db_list append db None Get list of all users from the server print Getting users user_list if opt users_to_copy is None users serverl exec_query SELECT user host FROM mysql user WHERE user root and user for user in users user_list append user 0 user 1 else OM WISE ain Cie Wises 6 Cojo Spiate E user_list append user Build options options new_data opt new_data new_port opt new_port new_id opt new_id root pass Gy Sscreyouey IY mysqld_options IEE report host localhost

MySQL Utilities - Oracle Documentation

Contents

Download Pdf Manuals

Related Search

Related Contents