Module: TestCaseModifications
- Defined in:
- lib/files/testcase_modifications.rb
Overview
Test case parsing and modifications
Class Method Summary collapse
-
.add_tag(tag, line, type_string, prepend_colon: true) ⇒ Object
PreCondition: This is only called on examples that are verified to have test id’s attached to them.
-
.add_tags_to_rspec_tests(tag_priority: true, tag_device: true) ⇒ Object
Modifies the Rspec files to add metadata to each test example block.
- .browsers_to_testrail(browser_array) ⇒ Object
-
.check_duplicates ⇒ Object
checks for duplicate test case ID’s in the rspec tests.
-
.get_example_device_type(line, test_cases) ⇒ Object
Gets a description of the minimum screen size given a line of text from the rspec.
-
.get_example_priority(line, test_cases) ⇒ Object
Get the priority of the test case given a line of text from the rspec.
-
.get_example_testrail_ids(line) ⇒ Object
Given a line of code extracted from an RSPEC example, it returns an array of numbers (integers) that correspond to the testrail_id’s.
-
.lowest_priority_of(ids, test_cases) ⇒ Object
Finds the lowest priority for an array of test cases ids - array of test cases integer ID’s.
-
.parse_specs ⇒ Object
Parses the rspec files and reports how many test cases are skipped, and a percentage of the test cases that are executed.
-
.push_to_testrail ⇒ Object
Parses all the test cases as found in the rspec files.
-
.update_automated_status(suite_ids, dryrun: false) ⇒ Object
This takes the file name associated with each testrail_id and posts it to the associated test rail ID on testrail.com.
-
.update_run_once(suite_ids) ⇒ Object
Inspects all request specs and checks if their associated testcase is marked as run once on testrail.
Class Method Details
.add_tag(tag, line, type_string, prepend_colon: true) ⇒ Object
PreCondition: This is only called on examples that are verified to have test id’s attached to them. Removes and adds in a tag to the string string parameter ‘line’.
146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 |
# File 'lib/files/testcase_modifications.rb', line 146 def self.add_tag(tag, line, type_string, prepend_colon:true) tag_index = line.index(tag) modified_line = "0xDEADBEAF" if tag_index # Found an existing tag:, remove the existing one first. next_comma = line.index(",", tag_index) if next_comma modified_line = line[0, tag_index - 2] + line[next_comma..-1] else # It's at the end, Look for the 'do' keyword last_do = line.index("do\n") first_part = line[0, tag_index - 2] second_part = line[last_do..-1] modified_line = first_part + " " + second_part end line = modified_line end colon = ":" if prepend_colon tag_index = line.index(tag) unless tag_index # None found, add one after testrail_id: testrail_index = line.index("testrail_id:") if testrail_index next_comma = line.index(",", testrail_index) if next_comma modified_line = line.insert(next_comma, ", #{tag} #{colon}#{type_string.downcase}") else # It was at the end already. last_bracket = line.index("]", testrail_index) modified_line = line.insert(last_bracket + 1, ", #{tag} #{colon}#{type_string.downcase}") end end end modified_line end |
.add_tags_to_rspec_tests(tag_priority: true, tag_device: true) ⇒ Object
Modifies the Rspec files to add metadata to each test example block. This will pull down data from testrail.com for each test case. It will then extract the priority and the desktop size that the test case can run on, and then add those as metadata to the example. This will modify all the rspec files in the regression_spec folder.
188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 |
# File 'lib/files/testcase_modifications.rb', line 188 def self.(tag_priority: true, tag_device: true) test_cases = TestRailOperations.get_test_rail_cases spec_files = Dir["regression_spec/**/*_spec.rb"] spec_files.each do |file| puts "Rspec file: #{file}" changes = [] # The new lines of code for the file # Read the file File.open(file).each do |line| new_line = line if line.match("testrail_id") if tag_device type = get_example_device_type(line, test_cases) if type new_line = add_tag("device:", line, type) end end if tag_priority priority = get_example_priority(line, test_cases) if priority new_line = add_tag("priority:", new_line, priority.to_s, prepend_colon: false) end end end changes << new_line end # Write the file out File.open(file, "w") do |f| changes.each do |line| f.puts line end end end end |
.browsers_to_testrail(browser_array) ⇒ Object
408 409 410 411 412 413 414 415 416 417 418 419 |
# File 'lib/files/testcase_modifications.rb', line 408 def self.browsers_to_testrail(browser_array) result = [] browser_array.each do |name| if name == "allbrowsers" result = [1, 2, 3, 5] return result else result << @browsers[name] end end result end |
.check_duplicates ⇒ Object
checks for duplicate test case ID’s in the rspec tests
336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 |
# File 'lib/files/testcase_modifications.rb', line 336 def self.check_duplicates regression_files = Dir["regression_spec/**/*_spec.rb"] spec_files = regression_files + Dir["spec/**/*_spec.rb"] # For keeping a running list of found ID's # Key is integer ID # Value is the file ids = {} # parse all the files looking for examples spec_files.each do |file| # puts "Rspec file: #{file}" File.open(file).each do |line| if line.match("testrail_id") testrail_ids = get_example_testrail_ids(line) testrail_ids.each do |id| # puts " id: #{id}" if ids[id] puts "Found duplicate: #{id}" puts "Other File: #{ids[id]}" puts "This File: #{file}" else ids[id] = file end end end end end end |
.get_example_device_type(line, test_cases) ⇒ Object
Gets a description of the minimum screen size given a line of text from the rspec. Given a line of code that was extracted from the test case contains a testrail_id tag, this function will extract the test rail ID, look it up in a list of test cases and return the type of screen size this minimally works on.
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
# File 'lib/files/testcase_modifications.rb', line 41 def self.get_example_device_type(line, test_cases) ids = get_example_testrail_ids(line) # puts ids if (ids.size == 1) id = ids[0] tc = test_cases[id] if tc tc.screen_size else # It could be in some other test rail project nil end else # Multiple test rail id's specified for the example. desktop_count = 0 tablet_count = 0 phone_count = 0 ids.each do |id| tc = test_cases[id] next if tc.nil? case tc.screen_size when "Desktop" desktop_count += 1 when "Tablet" tablet_count += 1 when "Phone" phone_count += 1 end end # Always go with the smallest minimum size specified in the test rail id list if phone_count != 0 "Phone" elsif tablet_count != 0 "Tablet" elsif desktop_count != 0 "Desktop" else "<-- ========================== ERROR ============================== -->" end end end |
.get_example_priority(line, test_cases) ⇒ Object
Get the priority of the test case given a line of text from the rspec. Given something like this: example “I make a course”, testrail_id: %w, priority: 1 do it will return a numeric value of the test priority
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 |
# File 'lib/files/testcase_modifications.rb', line 127 def self.get_example_priority(line, test_cases) ids = get_example_testrail_ids(line) if (ids.size == 1) id = ids[0] tc = test_cases[id] if tc tc.priority else # It could be in some other test rail project nil end else # Multiple test rail id's specified for the example. lowest_priority_of(ids, test_cases) end end |
.get_example_testrail_ids(line) ⇒ Object
Given a line of code extracted from an RSPEC example, it returns an array of numbers (integers) that correspond to the testrail_id’s.
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
# File 'lib/files/testcase_modifications.rb', line 13 def self.get_example_testrail_ids(line) # puts line testrail_id = "testrail_id:" length = testrail_id.size index = line.index(testrail_id) substr = line[(index + length)..-1] comma = substr.index(",") if comma substr = substr[0, comma] end substr = substr.gsub(" do", "") substr = substr.gsub("%w[", "") substr = substr.gsub("]", "") numbers = substr.split(" ") result = [] numbers.each do |str| result << str.to_i end result end |
.lowest_priority_of(ids, test_cases) ⇒ Object
Finds the lowest priority for an array of test cases ids - array of test cases integer ID’s. test_cases - array of TestCase instances returns the lower priority for a given set of test cases
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
# File 'lib/files/testcase_modifications.rb', line 88 def self.lowest_priority_of(ids, test_cases) p0_count = 0 p1_count = 0 p2_count = 0 p3_count = 0 ids.each do |id| tc = test_cases[id] next if tc.nil? case tc.priority when 0 p0_count += 1 when 1 p1_count += 1 when 2 p2_count += 1 when 3 p3_count += 1 end end # Always go with the lowest priority specified in the test rail id list if p0_count != 0 0 elsif p1_count != 0 1 elsif p2_count != 0 2 elsif p3_count != 0 3 else "<-- ========================== ERROR ============================== -->" end end |
.parse_specs ⇒ Object
Parses the rspec files and reports how many test cases are skipped, and a percentage of the test cases that are executed. returns a hash of test cases, where the key is an integer of the test case ID (as found in test rail), and the value is an instance of TestCase, containing all the skip information and everything.
369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 |
# File 'lib/files/testcase_modifications.rb', line 369 def self.parse_specs rspec_examples = [] spec_files = Dir["regression_spec/**/*_spec.rb"] example_count = 0 skip_count = 0 file_count = 0 # parse all the files looking for examples spec_files.each do |file| file_count += 1 parser = RSpecParser.new(file) parser.parse parser.test_cases.each do |tc| tc.file = file if tc.skip.count > 0 skip_count += 1 end end rspec_examples += parser.test_cases # puts "%5d in %s" % [parser.test_cases.count, file] example_count += parser.test_cases.count end puts "total files: #{file_count}" puts "total examples: #{example_count}" puts "total examples skipped: #{skip_count}" puts "total executed: #{example_count - skip_count}" puts "total coverage: %.2f %" % [(1.0 - (skip_count / example_count.to_f)) * 100] result = {} rspec_examples.each do |tc| next if tc.id.nil? tc.id.each do |id| result[id.to_i] = tc end end result end |
.push_to_testrail ⇒ Object
Parses all the test cases as found in the rspec files. then pushes information about teach test case up to testrail. This will update 3 fields about each test case:
-
the spec file location
-
if the test is skipped, and on which browser
-
if the test is automated (by default is true)
427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 |
# File 'lib/files/testcase_modifications.rb', line 427 def self.push_to_testrail rspec_examples = parse_specs trclient = TestRailOperations.get_test_rail_client # First set the fields to blank in test rail test_cases = TestRailOperations.get_test_rail_cases # test_cases.each do |key_id, val_tc| # url = "update_case/#{key_id}" # data = { "custom_spec_location" => nil, "custom_browser_skip" => [], "custom_automated" => false } # trclient.send_post(url, data) # end # Then upload the information contain in our rspec files rspec_examples.each do |id_key, tc_val| if test_cases.key?(id_key) url = "update_case/#{id_key}" puts "updating test case: " browser_skips = browsers_to_testrail(tc_val.skip) data = { "custom_spec_location" => tc_val.file, "custom_browser_skip" => browser_skips, "custom_automated" => true } trclient.send_post_retry(url, data) end end end |
.update_automated_status(suite_ids, dryrun: false) ⇒ Object
This takes the file name associated with each testrail_id and posts it to the associated test rail ID on testrail.com. For example, if an rspec test example has a testrail_id of 123123 in file foo_spec.rb, it will update the test case on test rail and update the spec location field with foo_spec.rb. This will iterate over all the files in the regression_spec folder.
230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 |
# File 'lib/files/testcase_modifications.rb', line 230 def self.update_automated_status(suite_ids, dryrun:false) test_cases = TestRailOperations.get_test_rail_cases_for_all_suites(suite_ids) regression_files = Dir["regression_spec/**/*_spec.rb"] spec_files = regression_files + Dir["spec/**/*_spec.rb"] # For keeping test cases that actually changed changed_cases = {} orphaned_ids_count = 0 # parse all the files looking for examples spec_files.each do |file| # puts "Rspec file: #{file}" File.open(file).each do |line| if line.match("testrail_id") testrail_ids = get_example_testrail_ids(line) testrail_ids.each do |id| # puts " id: #{id}" tc = test_cases.delete(id) if tc if file != tc.file puts "\r\nID: #{id} - #{tc.title[0,50]}#{(tc.title.length > 50) ? '...' : ''}" puts " Old File: #{tc.file}" puts " New File: #{file}" tc.file = file changed_cases[id] = tc end # if the current testcase is marked run once, leave it. otherwise, mark it as # run once if the file it is found to be in is a request spec tc.run_once ||= (file =~ /^spec\/requests\/.*_spec\.rb$/i).nil? ? false : true # assuming it never becomes un-automated unless tc.automated puts " ID: #{id} Marking automated" tc.automated = true changed_cases[id] = tc end else puts "Test CaseID: #{id} not found in any testrail suite for project: #{TestRailOperations.project_id}" orphaned_ids_count += 1 end end end end end # remaining testcases should be unautomated, ensure they are check_unautomated(test_cases, changed_cases) # update testrail if needed if changed_cases.count > 0 puts "\nTest Cases that will get modified" trclient = TestRailOperations.get_test_rail_client changed_cases.each do |id_key, tc_val| puts "\t#{id_key} -> " + (!!tc_val.file ? tc_val.file : "no spec location") url = "update_case/#{id_key}" data = { "custom_spec_location" => tc_val.file, "custom_automated" => tc_val.automated, "custom_run_once" => tc_val.run_once } unless dryrun trclient.send_post_retry(url, data) end end else puts "Nothing to update \\o/" end puts "Number of orphaned testcase IDs: #{orphaned_ids_count}" unless orphaned_ids_count.zero? end |
.update_run_once(suite_ids) ⇒ Object
Inspects all request specs and checks if their associated testcase is marked as run once on testrail. If they are not, then the testcases on testrail get marked as run once
302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 |
# File 'lib/files/testcase_modifications.rb', line 302 def self.update_run_once(suite_ids) test_cases = TestRailOperations.get_test_rail_cases_for_all_suites(suite_ids) spec_files = Dir["spec/**/*_spec.rb"].reject { |f| f.include? "spec/features/" } request_specs = [] # parse all the files looking for examples spec_files.each do |file| # puts "Rspec file: #{file}" File.open(file).each do |line| if line.match("testrail_id") testrail_ids = get_example_testrail_ids(line) testrail_ids.each do |id| # puts " id: #{id}" tc = test_cases[id] if tc && tc.file unless tc.run_once request_specs << tc end end end end end end trclient = TestRailOperations.get_test_rail_client request_specs.each do |tc| puts "Marking Test Case as run_once: #{tc.id}" url = "update_case/#{tc.id}" data = { "custom_run_once" => true } trclient.send_post_retry(url, data) end end |