Class: Jetpants::Pool
- Includes:
- CallbackHandler
- Defined in:
- lib/jetpants/pool.rb
Overview
a Pool represents a group of database instances (Jetpants::DB objects).
The default implementation assumes that a Pool contains:
-
1 master
-
0 or more slaves, falling into one of these categories:
-
active slaves (actively taking production read queries)
-
standby slaves (for HA, promotable if a master or active slave fails + used to clone new replacements)
-
backup slaves (dedicated for backups and background jobs, never put into prod, potentially different hardware spec)
-
Plugins may of course override this extensively, to support different topologies, such as master-master trees.
Many of these methods are only useful in conjunction with an asset-tracker / configuration-generator plugin
Direct Known Subclasses
Instance Attribute Summary collapse
-
#active_slave_weights ⇒ Object
readonly
Hash mapping DB object => weight, for active (read) slaves.
-
#aliases ⇒ Object
readonly
Array of strings containing other equivalent names for this pool.
-
#master ⇒ Object
readonly
Jetpants::DB object that is the pool’s master.
-
#master_read_weight ⇒ Object
If the master also receives read queries, this stores its weight.
-
#name ⇒ Object
readonly
human-readable String name of pool.
-
#slave_name ⇒ Object
Can be used to store a name that refers to just the active_slaves, for instance if your framework isn’t smart enough to know about master/slave relationships.
Instance Method Summary collapse
-
#active_slaves ⇒ Object
Returns an array of Jetpants::DB objects.
-
#add_alias(name) ⇒ Object
Informs this pool that it has an alias.
-
#backup_slaves ⇒ Object
Returns an array of Jetpants::DB objects.
-
#before_sync_configuration ⇒ Object
Callback to ensure that a sync’ed pool is already in Topology.pools.
-
#has_active_slave(slave_db, weight = 100) ⇒ Object
Informs Jetpants that slave_db is an active slave.
-
#initialize(name, master) ⇒ Pool
constructor
A new instance of Pool.
-
#mark_slave_active(slave_db, weight = 100) ⇒ Object
Turns a standby slave into an active slave, giving it the specified read weight.
-
#mark_slave_standby(slave_db) ⇒ Object
Turns an active slave into a standby slave.
-
#master_promotion!(promoted) ⇒ Object
Demotes the pool’s existing master, promoting a slave in its place.
-
#method_missing(name, *args, &block) ⇒ Object
Jetpants::Pool proxies missing methods to the pool’s @master Jetpants::DB instance.
-
#nodes ⇒ Object
returns a flat array of all Jetpants::DB objects in the pool: the master and all slaves of all types.
-
#output(str) ⇒ Object
Displays the provided output, along with information about the current time, and self (the name of this Pool).
-
#remove_slave!(slave_db) ⇒ Object
Remove a slave from a pool entirely.
- #respond_to?(name, include_private = false) ⇒ Boolean
-
#slaves(type = false) ⇒ Object
Returns all slaves, or pass in :active, :standby, or :backup to receive slaves just of a particular type.
-
#standby_slaves ⇒ Object
Returns an array of Jetpants::DB objects.
-
#summary(extended_info = false) ⇒ Object
Displays a summary of the pool’s members.
-
#sync_configuration ⇒ Object
Informs your asset tracker about any changes in the pool’s state or members.
-
#to_s ⇒ Object
Returns the name of the pool.
Methods included from CallbackHandler
Constructor Details
#initialize(name, master) ⇒ Pool
Returns a new instance of Pool.
52 53 54 55 56 57 58 59 |
# File 'lib/jetpants/pool.rb', line 52 def initialize(name, master) @name = name @slave_name = false @aliases = [] @master = master.to_db @master_read_weight = 0 @active_slave_weights = {} end |
Dynamic Method Handling
This class handles dynamic methods through the method_missing method
#method_missing(name, *args, &block) ⇒ Object
Jetpants::Pool proxies missing methods to the pool’s @master Jetpants::DB instance.
309 310 311 312 313 314 315 |
# File 'lib/jetpants/pool.rb', line 309 def method_missing(name, *args, &block) if @master.respond_to? name @master.send name, *args, &block else super end end |
Instance Attribute Details
#active_slave_weights ⇒ Object (readonly)
Hash mapping DB object => weight, for active (read) slaves. Default weight is 100. Safe to leave at default if your app framework doesn’t support different weights for individual read slaves. Weights have no effect inside Jetpants, but any asset tracker / config generator plugin can carry them through to the config file.
44 45 46 |
# File 'lib/jetpants/pool.rb', line 44 def active_slave_weights @active_slave_weights end |
#aliases ⇒ Object (readonly)
Array of strings containing other equivalent names for this pool
30 31 32 |
# File 'lib/jetpants/pool.rb', line 30 def aliases @aliases end |
#master ⇒ Object (readonly)
Jetpants::DB object that is the pool’s master
27 28 29 |
# File 'lib/jetpants/pool.rb', line 27 def master @master end |
#master_read_weight ⇒ Object
If the master also receives read queries, this stores its weight. Set to 0 if the master does not receive read queries (which is the default). This has no effect inside of Jetpants, but can be used by an asset tracker / config generator plugin to carry the value through to the config file.
50 51 52 |
# File 'lib/jetpants/pool.rb', line 50 def master_read_weight @master_read_weight end |
#name ⇒ Object (readonly)
human-readable String name of pool
24 25 26 |
# File 'lib/jetpants/pool.rb', line 24 def name @name end |
#slave_name ⇒ Object
Can be used to store a name that refers to just the active_slaves, for instance if your framework isn’t smart enough to know about master/slave relationships. Safe to leave as nil otherwise. Has no effect in Jetpants, but an asset tracker / config generator plugin may include this in the generated config file.
37 38 39 |
# File 'lib/jetpants/pool.rb', line 37 def slave_name @slave_name end |
Instance Method Details
#active_slaves ⇒ Object
Returns an array of Jetpants::DB objects. Active slaves are ones that receive read queries from your application.
75 76 77 |
# File 'lib/jetpants/pool.rb', line 75 def active_slaves @master.slaves.select {|sl| @active_slave_weights[sl]} end |
#add_alias(name) ⇒ Object
Informs this pool that it has an alias. A pool may have any number of aliases.
143 144 145 146 147 148 149 150 |
# File 'lib/jetpants/pool.rb', line 143 def add_alias(name) if @aliases.include? name false else @aliases << name true end end |
#backup_slaves ⇒ Object
Returns an array of Jetpants::DB objects. Backup slaves are never promoted to active or master. They are for dedicated backup purposes. They may be a different/cheaper hardware spec than other slaves.
90 91 92 |
# File 'lib/jetpants/pool.rb', line 90 def backup_slaves @master.slaves.reject {|sl| @active_slave_weights[sl] || !sl.for_backups?} end |
#before_sync_configuration ⇒ Object
Callback to ensure that a sync’ed pool is already in Topology.pools
285 286 287 288 289 |
# File 'lib/jetpants/pool.rb', line 285 def before_sync_configuration unless Jetpants.topology.pools.include? self Jetpants.topology.pools << self end end |
#has_active_slave(slave_db, weight = 100) ⇒ Object
Informs Jetpants that slave_db is an active slave. Potentially used by plugins, such as in Topology at start-up time.
102 103 104 105 106 |
# File 'lib/jetpants/pool.rb', line 102 def has_active_slave(slave_db, weight=100) slave_db = slave_db.to_db raise "Attempt to mark a DB as its own active slave" if slave_db == @master @active_slave_weights[slave_db] = weight end |
#mark_slave_active(slave_db, weight = 100) ⇒ Object
Turns a standby slave into an active slave, giving it the specified read weight. Syncs the pool’s configuration afterwards. It’s up to your asset tracker plugin to actually do something with this information.
111 112 113 114 115 |
# File 'lib/jetpants/pool.rb', line 111 def mark_slave_active(slave_db, weight=100) raise "Attempt to make a backup slave be an active slave" if slave_db.for_backups? has_active_slave slave_db, weight sync_configuration end |
#mark_slave_standby(slave_db) ⇒ Object
Turns an active slave into a standby slave. Syncs the pool’s configuration afterwards. It’s up to your asset tracker plugin to actually do something with this information.
119 120 121 122 123 124 |
# File 'lib/jetpants/pool.rb', line 119 def mark_slave_standby(slave_db) slave_db = slave_db.to_db raise "Cannot call mark_slave_standby on a master" if slave_db == @master @active_slave_weights.delete(slave_db) sync_configuration end |
#master_promotion!(promoted) ⇒ Object
Demotes the pool’s existing master, promoting a slave in its place.
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 |
# File 'lib/jetpants/pool.rb', line 194 def master_promotion!(promoted) demoted = @master raise "Demoted node is already the master of this pool!" if demoted == promoted raise "Promoted host is not in the right pool!" unless demoted.slaves.include?(promoted) output "Preparing to demote master #{demoted} and promote #{promoted} in its place." # If demoted machine is available, confirm it is read-only and binlog isn't moving, # and then wait for slaves to catch up to this position if demoted.running? demoted.enable_read_only! unless demoted.read_only? raise "Unable to enable global read-only mode on demoted machine" unless demoted.read_only? coordinates = demoted.binlog_coordinates raise "Demoted machine still taking writes (from superuser or replication?) despite being read-only" unless coordinates == demoted.binlog_coordinates demoted.slaves.concurrent_each do |s| while true do sleep 1 break if s.repl_binlog_coordinates == coordinates output "Still catching up to coordinates of demoted master" end end # Demoted machine not available -- wait for slaves' binlogs to stop moving else demoted.slaves.concurrent_each do |s| progress = s.repl_binlog_coordinates while true do sleep 1 break if s.repl_binlog_coordinates == progress s.output "Still catching up on replication" end end end # Stop replication on all slaves replicas = demoted.slaves.dup replicas.each do |s| s.pause_replication if s.replicating? end raise "Unable to stop replication on all slaves" if replicas.any? {|s| s.replicating?} user, password = promoted.replication_credentials.values log, position = promoted.binlog_coordinates # reset slave on promoted, and make sure read_only is disabled promoted.disable_replication! promoted.disable_read_only! # gather our new replicas replicas.delete promoted replicas << demoted if demoted.running? # perform promotion replicas.each do |r| r.change_master_to promoted, user: user, password: password, log_file: log, log_pos: position end # ensure our replicas are configured correctly by comparing our staged values to current values of replicas promoted_replication_config = { master_host: promoted.ip, master_user: user, master_log_file: log, exec_master_log_pos: position.to_s } replicas.each do |r| promoted_replication_config.each do |option, value| raise "Unexpected slave status value for #{option} in replica #{r} after promotion" unless r.slave_status[option] == value end r.resume_replication unless r.replicating? end # Update the pool # Note: if the demoted machine is not available, plugin may need to implement an # after_master_promotion! method which handles this case in configuration tracker @active_slave_weights.delete promoted # if promoting an active slave, remove it from read pool @master = promoted sync_configuration Jetpants.topology.write_config output "Promotion complete. Pool master is now #{promoted}." replicas.all? {|r| r.replicating?} end |
#nodes ⇒ Object
returns a flat array of all Jetpants::DB objects in the pool: the master and all slaves of all types.
96 97 98 |
# File 'lib/jetpants/pool.rb', line 96 def nodes [master, slaves].flatten.compact end |
#output(str) ⇒ Object
Displays the provided output, along with information about the current time, and self (the name of this Pool)
298 299 300 301 302 303 304 305 306 |
# File 'lib/jetpants/pool.rb', line 298 def output(str) str = str.to_s.strip str = nil if str && str.length == 0 str ||= "Completed (no output)" output = Time.now.strftime("%H:%M:%S") + " [#{self}] " output << str print output + "\n" output end |
#remove_slave!(slave_db) ⇒ Object
Remove a slave from a pool entirely. This is destructive, ie, it does a RESET SLAVE on the db. Note that a plugin may want to override this (or implement after_remove_slave!) to actually sync the change to an asset tracker, depending on how the plugin implements Pool#sync_configuration. (If the implementation makes sync_configuration work by iterating over the pool’s current slaves to update their status/role/pool, it won’t see any slaves that have been removed, and therefore won’t update them.)
133 134 135 136 137 138 139 140 |
# File 'lib/jetpants/pool.rb', line 133 def remove_slave!(slave_db) raise "Slave is not in this pool" unless slave_db.pool == self slave_db.disable_monitoring slave_db.stop_replication slave_db.repl_binlog_coordinates # displays how far we replicated, in case you need to roll back this change manually slave_db.disable_replication! sync_configuration # may or may not be sufficient -- see note above. end |
#respond_to?(name, include_private = false) ⇒ Boolean
317 318 319 |
# File 'lib/jetpants/pool.rb', line 317 def respond_to?(name, include_private=false) super || @master.respond_to?(name) end |
#slaves(type = false) ⇒ Object
Returns all slaves, or pass in :active, :standby, or :backup to receive slaves just of a particular type
63 64 65 66 67 68 69 70 71 |
# File 'lib/jetpants/pool.rb', line 63 def slaves(type=false) case type when :active then active_slaves when :standby then standby_slaves when :backup then backup_slaves when false then @master.slaves else [] end end |
#standby_slaves ⇒ Object
Returns an array of Jetpants::DB objects. Standby slaves do not receive queries from your application. These are for high availability. They can be turned into active slaves or even the master, and can also be used for cloning additional slaves.
83 84 85 |
# File 'lib/jetpants/pool.rb', line 83 def standby_slaves @master.slaves.reject {|sl| @active_slave_weights[sl] || sl.for_backups?} end |
#summary(extended_info = false) ⇒ Object
156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 |
# File 'lib/jetpants/pool.rb', line 156 def summary(extended_info=false) probe alias_text = @aliases.count > 0 ? ' (aliases: ' + @aliases.join(', ') + ')' : '' data_size = @master.running? ? "[#{master.data_set_size(true)}GB]" : '' state_text = (respond_to?(:state) && state != :ready ? " (state: #{state})" : '') print "#{name}#{alias_text}#{state_text} #{data_size}\n" if extended_info details = {} nodes.concurrent_each do |s| if !s.running? details[s] = {coordinates: ['unknown'], lag: 'N/A'} elsif s == @master details[s] = {coordinates: s.binlog_coordinates(false), lag: 'N/A'} else lag = s.seconds_behind_master lag_str = lag.nil? ? 'NULL' : lag.to_s + 's' details[s] = {coordinates: s.repl_binlog_coordinates(false), lag: lag_str} end end end binlog_pos = extended_info ? details[@master][:coordinates].join(':') : '' print "\tmaster = %-13s %-30s %s\n" % [@master.ip, @master.hostname, binlog_pos] [:active, :standby, :backup].each do |type| slave_list = slaves(type) slave_list.sort.each_with_index do |s, i| binlog_pos = extended_info ? details[s][:coordinates].join(':') : '' slave_lag = extended_info ? "lag=#{details[s][:lag]}" : '' print "\t%-7s slave #{i + 1} = %-13s %-30s %-26s %s\n" % [type, s.ip, s.hostname, binlog_pos, slave_lag] end end true end |
#sync_configuration ⇒ Object
Informs your asset tracker about any changes in the pool’s state or members. Plugins should override this, or use before_sync_configuration / after_sync_configuration callbacks, to provide an implementation of this method.
281 282 |
# File 'lib/jetpants/pool.rb', line 281 def sync_configuration end |
#to_s ⇒ Object
Returns the name of the pool.
292 293 294 |
# File 'lib/jetpants/pool.rb', line 292 def to_s @name end |