Class: CsvPiper::PreProcessors::RemoveNilColumns

Inherits:

Object

• Object
• CsvPiper::PreProcessors::RemoveNilColumns

Defined in:

lib/csv_piper/pre_processors/remove_extra_columns.rb

Overview

Just removing nil is significanlty faster for large csvs (100M+ cells) (10M cells = ~20x faster on 10 col row [3.8s vs 0.19s], ~100x faster on 1000 col row [3.8s vs 0.035s]). Even better is your processors don’t need this pre-processing.

Instance Method Summary

• #process(origin, errors) ⇒ Object

Instance Method Details

#process(origin, errors) ⇒ Object

# File 'lib/csv_piper/pre_processors/remove_extra_columns.rb', line 14

def process(origin, errors)
  origin.delete(nil)
  [origin, errors]
end