Module: Appium::Core::Base::Device::ImageComparison

Defined in:
lib/appium_lib_core/common/device/image_comparison.rb

Constant Summary collapse

MODE = 
[:matchFeatures, :getSimilarity, :matchTemplate].freeze
MATCH_FEATURES = 
{
  detector_name: %w(AKAZE AGAST BRISK FAST GFTT KAZE MSER SIFT ORB),
  match_func: %w(FlannBased BruteForce BruteForceL1 BruteForceHamming BruteForceHammingLut BruteForceSL2),
  goodMatchesFactor: nil, # Integer
  visualize: [true, false]
}.freeze
MATCH_TEMPLATE = 
{
  visualize: [true, false]
}.freeze
GET_SIMILARITY = 
{
  visualize: [true, false]
}.freeze

Instance Method Summary collapse

Instance Method Details

#compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil) ⇒ Hash

Performs images comparison using OpenCV framework features. It is expected that both OpenCV framework and opencv4nodejs module are installed on the machine where Appium server is running.

Parameters:

  • mode (Symbol) (defaults to: :matchFeatures) —

    One of possible comparison modes: :matchFeatures, :getSimilarity, :matchTemplate. +:matchFeatures is by default.

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • options (Hash, nil) (defaults to: nil) —

    The content of this dictionary depends on the actual mode value. See the documentation on appium-support module for more details.

Returns:

  • (Hash) —

    The content of the resulting dictionary depends on the actual mode and options values. See the documentation on appium-support module for more details.

Raises:



169
170
171
172
173
174
175
176
177
178
179
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 169

def compare_images(mode: :matchFeatures, first_image:, second_image:, options: nil)
  raise ::Appium::Core::Error::ArgumentError, "content_type should be #{MODE}" unless MODE.member?(mode)

  params = {}
  params[:mode] = mode
  params[:firstImage] = Base64.strict_encode64 first_image
  params[:secondImage] = Base64.strict_encode64 second_image
  params[:options] = options if options

  execute(:compare_images, {}, params)
end

#find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil, multiple: nil, match_neighbour_threshold: nil) ⇒ Object

Performs images matching by template to find possible occurrence of the partial image in the full image with default options. Read template_matching for more details on this topic.

Examples:

@driver.find_image_occurrence full_image: "image data 1", partial_image: "image data 2"

visual = @@driver.find_image_occurrence full_image: image1, partial_image: image2, visualize: true
File.write 'find_result_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • full_image (String) —

    A full image data.

  • partial_image (String) —

    A partial image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.

  • threshold (Float, nil) (defaults to: nil) —
    0.5

    At what normalized threshold to reject

  • multiple (bool, nil) (defaults to: nil) —

    Whether to enable the support of multiple image occurrences @since Appium 1.21.0.

  • match_neighbour_threshold (integer, nil) (defaults to: nil) —

    The pixel distance between matches we consider to be part of the same template match @since Appium 1.21.0. This option is only considered if multiple matches mode is enabled. 10 pixels by default.



113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 113

def find_image_occurrence(full_image:, partial_image:, visualize: false, threshold: nil,
                          multiple: nil, match_neighbour_threshold: nil)
  unless MATCH_TEMPLATE[:visualize].member?(visualize)
    raise ::Appium::Core::Error::ArgumentError,
          "visualize should be #{MATCH_TEMPLATE[:visualize]}"
  end

  options = {}
  options[:visualize] = visualize
  options[:threshold] = threshold unless threshold.nil?
  options[:multiple] = multiple unless multiple.nil?
  options[:matchNeighbourThreshold] = match_neighbour_threshold unless match_neighbour_threshold.nil?

  compare_images(mode: :matchTemplate, first_image: full_image, second_image: partial_image, options: options)
end

#get_images_similarity(first_image:, second_image:, visualize: false) ⇒ Object

Performs images matching to calculate the similarity score between them with default options. The flow there is similar to the one used in find_image_occurrence but it is mandatory that both images are of equal size.

Examples:

@driver.get_images_similarity first_image: "image data 1", second_image: "image data 2"

visual = @@driver.get_images_similarity first_image: image1, second_image: image2, visualize: true
File.write 'images_similarity_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 144

def get_images_similarity(first_image:, second_image:, visualize: false)
  unless GET_SIMILARITY[:visualize].member?(visualize)
    raise ::Appium::Core::Error::ArgumentError,
          "visualize should be #{GET_SIMILARITY[:visualize]}"
  end

  options = {}
  options[:visualize] = visualize

  compare_images(mode: :getSimilarity, first_image: first_image, second_image: second_image, options: options)
end

#match_images_features(first_image:, second_image:, detector_name: 'ORB', match_func: 'BruteForce', good_matches_factor: nil, visualize: false) ⇒ Object

Performs images matching by features with default options. Read py_matcher for more details on this topic.

Examples:

@driver.match_images_features first_image: "image data 1", second_image: "image data 2"

visual = @@driver.match_images_features first_image: image1, second_image: image2, visualize: true
File.write 'match_images_visual.png', Base64.decode64(visual['visualization']) # if the image is PNG

Parameters:

  • first_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • second_image (String) —

    An image data. All image formats, that OpenCV library itself accepts, are supported.

  • detector_name (String) (defaults to: 'ORB') —

    Sets the detector name for features matching algorithm. Some of these detectors (FAST, AGAST, GFTT, FAST, SIFT and MSER) are not available in the default OpenCV installation and have to be enabled manually before library compilation. The default detector name is ‘ORB’.

  • match_func (String) (defaults to: 'BruteForce') —

    The name of the matching function. The default one is ‘BruteForce’.

  • good_matches_factor (String, nil) (defaults to: nil) —

    The maximum count of “good” matches (e. g. with minimal distances). The default one is nil.

  • visualize (Bool) (defaults to: false) —

    Makes the endpoint to return an image, which contains the visualized result of the corresponding picture matching operation. This option is disabled by default.



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
 
# File 'lib/appium_lib_core/common/device/image_comparison.rb', line 61

def match_images_features(first_image:,
                          second_image:,
                          detector_name: 'ORB',
                          match_func: 'BruteForce',
                          good_matches_factor: nil,
                          visualize: false)
  unless MATCH_FEATURES[:detector_name].member?(detector_name.to_s)
    raise ::Appium::Core::Error::ArgumentError, "detector_name should be #{MATCH_FEATURES[:detector_name]}"
  end

  unless MATCH_FEATURES[:match_func].member?(match_func.to_s)
    raise ::Appium::Core::Error::ArgumentError, "match_func should be #{MATCH_FEATURES[:match_func]}"
  end

  unless MATCH_FEATURES[:visualize].member?(visualize)
    raise ::Appium::Core::Error::ArgumentError,
          "visualize should be #{MATCH_FEATURES[:visualize]}"
  end

  options = {}
  options[:detectorName] = detector_name.to_s.upcase
  options[:matchFunc] = match_func.to_s
  options[:goodMatchesFactor] = good_matches_factor.to_i unless good_matches_factor.nil?
  options[:visualize] = visualize

  compare_images(mode: :matchFeatures, first_image: first_image, second_image: second_image, options: options)
end