Class: SVMKit::LinearModel::SVC
- Inherits:
-
Object
- Object
- SVMKit::LinearModel::SVC
- Includes:
- Base::BaseEstimator, Base::Classifier
- Defined in:
- lib/svmkit/linear_model/svc.rb
Overview
SVC is a class that implements Support Vector Classifier with the Pegasos algorithm.
Reference
-
Shalev-Shwartz and Y. Singer, “Pegasos: Primal Estimated sub-GrAdient SOlver for SVM,” Proc. ICML’07, pp. 807–814, 2007.
-
Instance Attribute Summary collapse
-
#bias_term ⇒ Float
readonly
Return the bias term (a.k.a. intercept) for SVC.
-
#rng ⇒ Random
readonly
Return the random generator for performing random sampling in the Pegasos algorithm.
-
#weight_vec ⇒ Numo::DFloat
readonly
Return the weight vector for SVC.
Attributes included from Base::BaseEstimator
Instance Method Summary collapse
-
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
-
#fit(x, y) ⇒ SVC
Fit the model with given training data.
-
#initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 100, batch_size: 50, random_seed: nil) ⇒ SVC
constructor
Create a new classifier with Support Vector Machine by the Pegasos algorithm.
-
#marshal_dump ⇒ Hash
Dump marshal data.
-
#marshal_load(obj) ⇒ nil
Load marshal data.
-
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
-
#score(x, y) ⇒ Float
Claculate the mean accuracy of the given testing data.
Constructor Details
#initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 100, batch_size: 50, random_seed: nil) ⇒ SVC
Create a new classifier with Support Vector Machine by the Pegasos algorithm.
41 42 43 44 45 46 47 48 49 50 51 52 53 |
# File 'lib/svmkit/linear_model/svc.rb', line 41 def initialize(reg_param: 1.0, fit_bias: false, bias_scale: 1.0, max_iter: 100, batch_size: 50, random_seed: nil) @params = {} @params[:reg_param] = reg_param @params[:fit_bias] = fit_bias @params[:bias_scale] = bias_scale @params[:max_iter] = max_iter @params[:batch_size] = batch_size @params[:random_seed] = random_seed @params[:random_seed] ||= srand @weight_vec = nil @bias_term = 0.0 @rng = Random.new(@params[:random_seed]) end |
Instance Attribute Details
#bias_term ⇒ Float (readonly)
Return the bias term (a.k.a. intercept) for SVC.
27 28 29 |
# File 'lib/svmkit/linear_model/svc.rb', line 27 def bias_term @bias_term end |
#rng ⇒ Random (readonly)
Return the random generator for performing random sampling in the Pegasos algorithm.
31 32 33 |
# File 'lib/svmkit/linear_model/svc.rb', line 31 def rng @rng end |
#weight_vec ⇒ Numo::DFloat (readonly)
Return the weight vector for SVC.
23 24 25 |
# File 'lib/svmkit/linear_model/svc.rb', line 23 def weight_vec @weight_vec end |
Instance Method Details
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
109 110 111 |
# File 'lib/svmkit/linear_model/svc.rb', line 109 def decision_function(x) @weight_vec.dot(x.transpose) + @bias_term end |
#fit(x, y) ⇒ SVC
Fit the model with given training data.
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 |
# File 'lib/svmkit/linear_model/svc.rb', line 60 def fit(x, y) # Generate binary labels negative_label = y.to_a.uniq.sort.shift bin_y = y.to_a.map { |l| l != negative_label ? 1 : -1 } # Expand feature vectors for bias term. samples = x if @params[:fit_bias] samples = Numo::NArray.hstack( [samples, Numo::DFloat.ones([x.shape[0], 1]) * @params[:bias_scale]] ) end # Initialize some variables. n_samples, n_features = samples.shape rand_ids = [*0..n_samples - 1].shuffle(random: @rng) weight_vec = Numo::DFloat.zeros(n_features) # Start optimization. @params[:max_iter].times do |t| # random sampling subset_ids = rand_ids.shift(@params[:batch_size]) rand_ids.concat(subset_ids) target_ids = subset_ids.map { |n| n if weight_vec.dot(samples[n, true]) * bin_y[n] < 1 }.compact n_subsamples = target_ids.size next if n_subsamples.zero? # update the weight vector. eta = 1.0 / (@params[:reg_param] * (t + 1)) mean_vec = Numo::DFloat.zeros(n_features) target_ids.each { |n| mean_vec += samples[n, true] * bin_y[n] } mean_vec *= eta / n_subsamples weight_vec = weight_vec * (1.0 - eta * @params[:reg_param]) + mean_vec # scale the weight vector. norm = Math.sqrt(weight_vec.dot(weight_vec)) scaler = (1.0 / @params[:reg_param]**0.5) / (norm + 1.0e-12) weight_vec *= [1.0, scaler].min end # Store the learned model. if @params[:fit_bias] @weight_vec = weight_vec[0...n_features - 1] @bias_term = weight_vec[n_features - 1] else @weight_vec = weight_vec[0...n_features] @bias_term = 0.0 end self end |
#marshal_dump ⇒ Hash
Dump marshal data.
134 135 136 |
# File 'lib/svmkit/linear_model/svc.rb', line 134 def marshal_dump { params: @params, weight_vec: @weight_vec, bias_term: @bias_term, rng: @rng } end |
#marshal_load(obj) ⇒ nil
Load marshal data.
140 141 142 143 144 145 146 |
# File 'lib/svmkit/linear_model/svc.rb', line 140 def marshal_load(obj) @params = obj[:params] @weight_vec = obj[:weight_vec] @bias_term = obj[:bias_term] @rng = obj[:rng] nil end |
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
117 118 119 |
# File 'lib/svmkit/linear_model/svc.rb', line 117 def predict(x) Numo::Int32.cast(decision_function(x).map { |v| v >= 0 ? 1 : -1 }) end |
#score(x, y) ⇒ Float
Claculate the mean accuracy of the given testing data.
126 127 128 129 130 |
# File 'lib/svmkit/linear_model/svc.rb', line 126 def score(x, y) p = predict(x) n_hits = (y.to_a.map.with_index { |l, n| l == p[n] ? 1 : 0 }).inject(:+) n_hits / y.size.to_f end |