Class: SVMKit::PolynomialModel::FactorizationMachineClassifier
- Inherits:
-
Object
- Object
- SVMKit::PolynomialModel::FactorizationMachineClassifier
- Includes:
- Base::BaseEstimator, Base::Classifier
- Defined in:
- lib/svmkit/polynomial_model/factorization_machine_classifier.rb
Overview
FactorizationMachineClassifier is a class that implements Factorization Machine with stochastic gradient descent (SGD) optimization. For multiclass classification problem, it uses one-vs-the-rest strategy.
Reference
-
Rendle, “Factorization Machines with libFM,” ACM Transactions on Intelligent Systems and Technology, vol. 3 (3), pp. 57:1–57:22, 2012.
-
-
Rendle, “Factorization Machines,” Proceedings of the 10th IEEE International Conference on Data Mining (ICDM’10), pp. 995–1000, 2010.
-
Instance Attribute Summary collapse
-
#bias_term ⇒ Numo::DFloat
readonly
Return the bias term for Factoriazation Machine.
-
#classes ⇒ Numo::Int32
readonly
Return the class labels.
-
#factor_mat ⇒ Numo::DFloat
readonly
Return the factor matrix for Factorization Machine.
-
#rng ⇒ Random
readonly
Return the random generator for random sampling.
-
#weight_vec ⇒ Numo::DFloat
readonly
Return the weight vector for Factorization Machine.
Attributes included from Base::BaseEstimator
Instance Method Summary collapse
-
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
-
#fit(x, y) ⇒ FactorizationMachineClassifier
Fit the model with given training data.
-
#initialize(n_factors: 2, loss: 'hinge', reg_param_bias: 1.0, reg_param_weight: 1.0, reg_param_factor: 1.0, init_std: 0.1, max_iter: 1000, batch_size: 10, random_seed: nil) ⇒ FactorizationMachineClassifier
constructor
Create a new classifier with Factorization Machine.
-
#marshal_dump ⇒ Hash
Dump marshal data.
-
#marshal_load(obj) ⇒ nil
Load marshal data.
-
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
-
#predict_proba(x) ⇒ Numo::DFloat
Predict probability for samples.
Methods included from Base::Classifier
Constructor Details
#initialize(n_factors: 2, loss: 'hinge', reg_param_bias: 1.0, reg_param_weight: 1.0, reg_param_factor: 1.0, init_std: 0.1, max_iter: 1000, batch_size: 10, random_seed: nil) ⇒ FactorizationMachineClassifier
Create a new classifier with Factorization Machine.
59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 59 def initialize(n_factors: 2, loss: 'hinge', reg_param_bias: 1.0, reg_param_weight: 1.0, reg_param_factor: 1.0, init_std: 0.1, max_iter: 1000, batch_size: 10, random_seed: nil) @params = {} @params[:n_factors] = n_factors @params[:loss] = loss @params[:reg_param_bias] = reg_param_bias @params[:reg_param_weight] = reg_param_weight @params[:reg_param_factor] = reg_param_factor @params[:init_std] = init_std @params[:max_iter] = max_iter @params[:batch_size] = batch_size @params[:random_seed] = random_seed @params[:random_seed] ||= srand @factor_mat = nil @weight_vec = nil @bias_term = nil @classes = nil @rng = Random.new(@params[:random_seed]) end |
Instance Attribute Details
#bias_term ⇒ Numo::DFloat (readonly)
Return the bias term for Factoriazation Machine.
38 39 40 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 38 def bias_term @bias_term end |
#classes ⇒ Numo::Int32 (readonly)
Return the class labels.
42 43 44 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 42 def classes @classes end |
#factor_mat ⇒ Numo::DFloat (readonly)
Return the factor matrix for Factorization Machine.
30 31 32 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 30 def factor_mat @factor_mat end |
#rng ⇒ Random (readonly)
Return the random generator for random sampling.
46 47 48 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 46 def rng @rng end |
#weight_vec ⇒ Numo::DFloat (readonly)
Return the weight vector for Factorization Machine.
34 35 36 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 34 def weight_vec @weight_vec end |
Instance Method Details
#decision_function(x) ⇒ Numo::DFloat
Calculate confidence scores for samples.
113 114 115 116 117 118 119 120 121 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 113 def decision_function(x) linear_term = @bias_term + x.dot(@weight_vec.transpose) factor_term = if @classes.size <= 2 0.5 * (@factor_mat.dot(x.transpose)**2 - (@factor_mat**2).dot(x.transpose**2)).sum else 0.5 * (@factor_mat.dot(x.transpose)**2 - (@factor_mat**2).dot(x.transpose**2)).sum(1).transpose end linear_term + factor_term end |
#fit(x, y) ⇒ FactorizationMachineClassifier
Fit the model with given training data.
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 84 def fit(x, y) @classes = Numo::Int32[*y.to_a.uniq.sort] n_classes = @classes.size _n_samples, n_features = x.shape if n_classes > 2 @factor_mat = Numo::DFloat.zeros(n_classes, @params[:n_factors], n_features) @weight_vec = Numo::DFloat.zeros(n_classes, n_features) @bias_term = Numo::DFloat.zeros(n_classes) n_classes.times do |n| bin_y = Numo::Int32.cast(y.eq(@classes[n])) * 2 - 1 factor, weight, bias = binary_fit(x, bin_y) @factor_mat[n, true, true] = factor @weight_vec[n, true] = weight @bias_term[n] = bias end else negative_label = y.to_a.uniq.sort.first bin_y = Numo::Int32.cast(y.ne(negative_label)) * 2 - 1 @factor_mat, @weight_vec, @bias_term = binary_fit(x, bin_y) end self end |
#marshal_dump ⇒ Hash
Dump marshal data.
152 153 154 155 156 157 158 159 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 152 def marshal_dump { params: @params, factor_mat: @factor_mat, weight_vec: @weight_vec, bias_term: @bias_term, classes: @classes, rng: @rng } end |
#marshal_load(obj) ⇒ nil
Load marshal data.
163 164 165 166 167 168 169 170 171 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 163 def marshal_load(obj) @params = obj[:params] @factor_mat = obj[:factor_mat] @weight_vec = obj[:weight_vec] @bias_term = obj[:bias_term] @classes = obj[:classes] @rng = obj[:rng] nil end |
#predict(x) ⇒ Numo::Int32
Predict class labels for samples.
127 128 129 130 131 132 133 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 127 def predict(x) return Numo::Int32.cast(decision_function(x).ge(0.0)) * 2 - 1 if @classes.size <= 2 n_samples, = x.shape decision_values = decision_function(x) Numo::Int32.asarray(Array.new(n_samples) { |n| @classes[decision_values[n, true].max_index] }) end |
#predict_proba(x) ⇒ Numo::DFloat
Predict probability for samples.
139 140 141 142 143 144 145 146 147 148 |
# File 'lib/svmkit/polynomial_model/factorization_machine_classifier.rb', line 139 def predict_proba(x) proba = 1.0 / (Numo::NMath.exp(-decision_function(x)) + 1.0) return (proba.transpose / proba.sum(axis: 1)).transpose if @classes.size > 2 n_samples, = x.shape probs = Numo::DFloat.zeros(n_samples, 2) probs[true, 1] = proba probs[true, 0] = 1.0 - proba probs end |