Class: CsvPiper::PreProcessors::RemoveNilColumns

Inherits:
Object
  • Object
show all
Defined in:
lib/csv_piper/pre_processors/remove_extra_columns.rb

Overview

Just removing nil is significanlty faster for large csvs (100M+ cells) (10M cells = ~20x faster on 10 col row [3.8s vs 0.19s], ~100x faster on 1000 col row [3.8s vs 0.035s]). Even better is your processors don’t need this pre-processing.

Instance Method Summary collapse

Instance Method Details

#process(origin, errors) ⇒ Object 



14
15
16
17
 
# File 'lib/csv_piper/pre_processors/remove_extra_columns.rb', line 14

def process(origin, errors)
  origin.delete(nil)
  [origin, errors]
end