Module: Appium::Core::Base::Device::ImageComparison

Defined in:
lib/appium_lib_core/common/device/image_comparison.rb

Constant Summary collapse

MODE = 
[:matchFeatures, :getSimilarity, :matchTemplate].freeze
MATCH_FEATURES = 
{
  detector_name: %w(AKAZE AGAST BRISK FAST GFTT KAZE MSER SIFT ORB),
  match_func: %w(FlannBased BruteForce BruteForceL1 BruteForceHamming BruteForceHammingLut BruteForceSL2),
  goodMatchesFactor: nil, # Integer
  visualize: [true, false]
}.freeze
MATCH_TEMPLATE = 
{
  visualize: [true, false]
}.freeze
GET_SIMILARITY = 
{
  visualize: [true, false]
}.freeze

Instance Method Summary collapse

Instance Method Details

#compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil) ⇒ Hash

Performs images comparison using OpenCV framework features. It is expected that both OpenCV framework and opencv4nodejs module are installed on the machine where Appium server is running.

Parameters:

  • mode (Symbol) (defaults to: :matchFeatures) —

    One of possible comparison modes: :matchFeatures, :getSimilarity, :matchTemplate. +:matchFeatures is by default.

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • options (Hash, nil) (defaults to: nil) —

    The content of this dictionary depends on the actual mode value. See the documentation on appium-support module for more details.

Returns:

  • (Hash) —

    The content of the resulting dictionary depends on the actual mode and options values. See the documentation on appium-support module for more details.



160
161
162
163
164
165
166
167
168
169
170
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 160

def compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil)
  raise "content_type should be #{MODE}" unless MODE.member?(mode)

  params = {}
  params[:mode] = mode
  params[:firstImage] = Base64.strict_encode64 first_image
  params[:secondImage] = Base64.strict_encode64 second_image
  params[:options] = options if options

  execute(:compare_images, {}, params)
end

#find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil, multiple: nil, match_neighbour_threshold: nil) ⇒ Object

Performs images matching by template to find possible occurrence of the partial image in the full image with default options. Read template_matching for more details on this topic.

Examples:

@driver.find_image_occurrence full_image: "image data 1", partial_image: "image data 2"

visual = @@driver.find_image_occurrence full_image: image1, partial_image: image2, visualize: true
File.write 'find_result_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • full_image (String) —

    A full image data.

  • partial_image (String) —

    A partial image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.

  • threshold (Float, nil) (defaults to: nil) —
    0.5

    At what normalized threshold to reject

  • multiple (bool, nil) (defaults to: nil) —

    Whether to enable the support of multiple image occurrences @since Appium 1.21.0.

  • match_neighbour_threshold (integer, nil) (defaults to: nil) —

    The pixel distance between matches we consider to be part of the same template match @since Appium 1.21.0. This option is only considered if multiple matches mode is enabled. 10 pixels by default.



110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 110

def find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil,
                          multiple: nil, match_neighbour_threshold: nil)
  raise "visualize should be #{MATCH_TEMPLATE[:visualize]}" unless MATCH_TEMPLATE[:visualize].member?(visualize)

  options = {}
  options[:visualize] = visualize
  options[:threshold] = threshold unless threshold.nil?
  options[:multiple] = multiple unless multiple.nil?
  options[:matchNeighbourThreshold] = match_neighbour_threshold unless match_neighbour_threshold.nil?

  compare_images(mode: :matchTemplate, first_image: full_image, second_image: partial_image, options: options)
end

#get_images_similarity(first_image:, second_image:, visualize: false) ⇒ Object

Performs images matching to calculate the similarity score between them with default options. The flow there is similar to the one used in find_image_occurrence but it is mandatory that both images are of equal size.

Examples:

@driver.get_images_similarity first_image: "image data 1", second_image: "image data 2"

visual = @@driver.get_images_similarity first_image: image1, second_image: image2, visualize: true
File.write 'images_similarity_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



138
139
140
141
142
143
144
145
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 138

def get_images_similarity(first_image:, second_image:, visualize: false)
  raise "visualize should be #{GET_SIMILARITY[:visualize]}" unless GET_SIMILARITY[:visualize].member?(visualize)

  options = {}
  options[:visualize] = visualize

  compare_images(mode: :getSimilarity, first_image: first_image, second_image: second_image, options: options)
end

#match_images_features(first_image:, second_image:, detector_name: 'ORB', match_func: 'BruteForce', good_matches_factor: nil, visualize: false) ⇒ Object

Performs images matching by features with default options. Read py_matcher for more details on this topic.

Examples:

@driver.match_images_features first_image: "image data 1", second_image: "image data 2"

visual = @@driver.match_images_features first_image: image1, second_image: image2, visualize: true
File.write 'match_images_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • detector_name (String) (defaults to: 'ORB') —

    Sets the detector name for features matching algorithm. Some of these detectors (FAST, AGAST, GFTT, FAST, SIFT and MSER) are not available in the default OpenCV installation and have to be enabled manually before library compilation. The default detector name is ‘ORB’.

  • match_func (String) (defaults to: 'BruteForce') —

    The name of the matching function. The default one is ‘BruteForce’.

  • good_matches_factor (String, nil) (defaults to: nil) —

    The maximum count of “good” matches (e. g. with minimal distances). The default one is nil.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 61

def match_images_features(first_image:,
                          second_image:,
                          detector_name: 'ORB',
                          match_func: 'BruteForce',
                          good_matches_factor: nil,
                          visualize: false)
  unless MATCH_FEATURES[:detector_name].member?(detector_name.to_s)
    raise "detector_name should be #{MATCH_FEATURES[:detector_name]}"
  end

  unless MATCH_FEATURES[:match_func].member?(match_func.to_s)
    raise "match_func should be #{MATCH_FEATURES[:match_func]}"
  end

  raise "visualize should be #{MATCH_FEATURES[:visualize]}" unless MATCH_FEATURES[:visualize].member?(visualize)

  options = {}
  options[:detectorName] = detector_name.to_s.upcase
  options[:matchFunc] = match_func.to_s
  options[:goodMatchesFactor] = good_matches_factor.to_i unless good_matches_factor.nil?
  options[:visualize] = visualize

  compare_images(mode: :matchFeatures, first_image: first_image, second_image: second_image, options: options)
end