Class: CSV
- Inherits:
-
Object
- Object
- CSV
- Extended by:
- Forwardable
- Includes:
- Enumerable
- Defined in:
- lib/csv.rb,
lib/csv/row.rb,
lib/csv/table.rb,
lib/csv/parser.rb,
lib/csv/writer.rb,
lib/csv/match_p.rb,
lib/csv/version.rb,
lib/csv/fields_converter.rb
Overview
This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.
The most generic interface of the library is:
csv = CSV.new(string_or_io, **)
# Reading: IO object should be open for read
csv.read # => array of rows
# or
csv.each do |row|
# ...
end
# or
row = csv.shift
# Writing: IO object should be open for write
csv << row
There are several specialized class methods for one-statement reading or writing, described in the Specialized Methods section.
If a String is passed into ::new, it is internally wrapped into a StringIO object.
options
can be used for specifying the particular CSV flavor (column separators, row separators, value quoting and so on), and for data conversion, see Data Conversion section for the description of the latter.
Specialized Methods
Reading
# From a file: all at once
arr_of_rows = CSV.read("path/to/file.csv", **)
# iterator-style:
CSV.foreach("path/to/file.csv", **) do |row|
# ...
end
# From a string
arr_of_rows = CSV.parse("CSV,data,String", **)
# or
CSV.parse("CSV,data,String", **) do |row|
# ...
end
Writing
# To a file
CSV.open("path/to/file.csv", "wb") do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
# To a String
csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
csv << ["another", "row"]
# ...
end
Shortcuts
# Core extensions for converting one line
csv_string = ["CSV", "data"].to_csv # to CSV
csv_array = "CSV,String".parse_csv # from CSV
# CSV() method
CSV { |csv_out| csv_out << %w{my data here} } # to $stdout
CSV(csv = "") { |csv_str| csv_str << %w{my data here} } # to a String
CSV($stderr) { |csv_err| csv_err << %w{my data here} } # to $stderr
CSV($stdin) { |csv_in| csv_in.each { |row| p row } } # from $stdin
Data Conversion
CSV with headers
CSV allows to specify column names of CSV file, whether they are in data, or provided separately. If headers specified, reading methods return an instance of CSV::Table, consisting of CSV::Row.
# Headers are part of data
data = CSV.parse(<<~ROWS, headers: true)
Name,Department,Salary
Bob,Engineering,1000
Jane,Sales,2000
John,Management,5000
ROWS
data.class #=> CSV::Table
data.first #=> #<CSV::Row "Name":"Bob" "Department":"Engineering" "Salary":"1000">
data.first.to_h #=> {"Name"=>"Bob", "Department"=>"Engineering", "Salary"=>"1000"}
# Headers provided by developer
data = CSV.parse('Bob,Engeneering,1000', headers: %i[name department salary])
data.first #=> #<CSV::Row name:"Bob" department:"Engineering" salary:"1000">
Typed data reading
CSV allows to provide a set of data converters e.g. transformations to try on input data. Converter could be a symbol from CSV::Converters constant’s keys, or lambda.
# Without any converters:
CSV.parse('Bob,2018-03-01,100')
#=> [["Bob", "2018-03-01", "100"]]
# With built-in converters:
CSV.parse('Bob,2018-03-01,100', converters: %i[numeric date])
#=> [["Bob", #<Date: 2018-03-01>, 100]]
# With custom converters:
CSV.parse('Bob,2018-03-01,100', converters: [->(v) { Time.parse(v) rescue v }])
#=> [["Bob", 2018-03-01 00:00:00 +0200, "100"]]
CSV and Character Encodings (M17n or Multilingualization)
This new CSV parser is m17n savvy. The parser works in the Encoding of the IO or String object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the Encoding it is in. Thus CSV will return Arrays or Rows of Strings in the Encoding of your data. This is accomplished by transcoding the parser itself into your Encoding.
Some transcoding must take place, of course, to accomplish this multiencoding support. For example, :col_sep
, :row_sep
, and :quote_char
must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV’s defaults should just magically work for your data. However, you can set these values manually in the target Encoding to avoid the translation.
It’s also important to note that while all of CSV’s core parser is now Encoding agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It’s just too hard for me to support native conversions in all of Ruby’s Encodings.
Anyway, the practical side of this is simple: make sure IO and String objects passed into CSV have the proper Encoding set and everything should just work. CSV methods that allow you to open IO objects (CSV::foreach(), CSV::open(), CSV::read(), and CSV::readlines()) do allow you to specify the Encoding.
One minor exception comes when generating CSV into a String with an Encoding that is not ASCII compatible. There’s no existing data for CSV to use to prepare itself and thus you will probably need to manually specify the desired Encoding for most of those cases. It will try to guess using the fields in a row of output though, when using CSV::generate_line() or Array#to_csv().
I try to point out any other Encoding issues in the documentation of methods as they come up.
This has been tested to the best of my ability with all non-“dummy” Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.
Defined Under Namespace
Modules: MatchP Classes: FieldInfo, FieldsConverter, MalformedCSVError, Parser, Row, Table, Writer
Constant Summary collapse
- DateMatcher =
A Regexp used to find and convert some common Date formats.
/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} | \d{4}-\d{2}-\d{2} )\z /x
- DateTimeMatcher =
A Regexp used to find and convert some common DateTime formats.
/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} | \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} | # ISO-8601 \d{4}-\d{2}-\d{2} (?:T\d{2}:\d{2}(?::\d{2}(?:\.\d+)?(?:[+-]\d{2}(?::\d{2})|Z)?)?)? )\z /x
- ConverterEncoding =
The encoding used by all converters.
Encoding.find("UTF-8")
- Converters =
This Hash holds the built-in converters of CSV that can be accessed by name. You can select Converters with CSV.convert() or through the
options
Hash passed to CSV::new().:integer
-
Converts any field Integer() accepts.
:float
-
Converts any field Float() accepts.
:numeric
-
A combination of
:integer
and:float
. :date
-
Converts any field Date::parse() accepts.
:date_time
-
Converts any field DateTime::parse() accepts.
:all
-
All built-in converters. A combination of
:date_time
and:numeric
.
All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.
To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.
{ integer: lambda { |f| Integer(f.encode(ConverterEncoding)) rescue f }, float: lambda { |f| Float(f.encode(ConverterEncoding)) rescue f }, numeric: [:integer, :float], date: lambda { |f| begin e = f.encode(ConverterEncoding) e.match?(DateMatcher) ? Date.parse(e) : f rescue # encoding conversion or date parse errors f end }, date_time: lambda { |f| begin e = f.encode(ConverterEncoding) e.match?(DateTimeMatcher) ? DateTime.parse(e) : f rescue # encoding conversion or date parse errors f end }, all: [:date_time, :numeric], }
- HeaderConverters =
This Hash holds the built-in header converters of CSV that can be accessed by name. You can select HeaderConverters with CSV.header_convert() or through the
options
Hash passed to CSV::new().:downcase
-
Calls downcase() on the header String.
:symbol
-
Leading/trailing spaces are dropped, string is downcased, remaining spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.
All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.
This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.
To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.
{ downcase: lambda { |h| h.encode(ConverterEncoding).downcase }, symbol: lambda { |h| h.encode(ConverterEncoding).downcase.gsub(/[^\s\w]+/, "").strip. gsub(/\s+/, "_").to_sym } }
- DEFAULT_OPTIONS =
The options used when no overrides are given by calling code. They are:
:col_sep
-
","
:row_sep
-
:auto
:quote_char
-
'"'
:field_size_limit
-
nil
:converters
-
nil
:unconverted_fields
-
nil
:headers
-
false
:return_headers
-
false
:header_converters
-
nil
:skip_blanks
-
false
:force_quotes
-
false
:skip_lines
-
nil
:liberal_parsing
-
false
:quote_empty
-
true
{ col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, quote_empty: true, }.freeze
- VERSION =
The version of the installed library.
"3.0.5"
Instance Attribute Summary collapse
-
#encoding ⇒ Object
readonly
The Encoding CSV is parsing or writing in.
Class Method Summary collapse
-
.filter(input = nil, output = nil, **options) ⇒ Object
:call-seq: filter( **options ) { |row| … } filter( input, **options ) { |row| … } filter( input, output, **options ) { |row| … }.
-
.foreach(path, **options, &block) ⇒ Object
This method is intended as the primary interface for reading CSV files.
-
.generate(str = nil, **options) {|csv| ... } ⇒ Object
:call-seq: generate( str, **options ) { |csv| … } generate( **options ) { |csv| … }.
-
.generate_line(row, **options) ⇒ Object
This method is a shortcut for converting a single row (Array) into a CSV String.
-
.instance(data = $stdout, **options) ⇒ Object
This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same
data
object (tested by Object#object_id()) with the sameoptions
. -
.open(filename, mode = "r", **options) ⇒ Object
:call-seq: open( filename, mode = “rb”, **options ) { |faster_csv| … } open( filename, **options ) { |faster_csv| … } open( filename, mode = “rb”, **options ) open( filename, **options ).
-
.parse(*args, &block) ⇒ Object
:call-seq: parse( str, **options ) { |row| … } parse( str, **options ).
-
.parse_line(line, **options) ⇒ Object
This method is a shortcut for converting a single line of a CSV String into an Array.
-
.read(path, *options) ⇒ Object
Use to slurp a CSV file into an Array of Arrays.
-
.readlines(*args) ⇒ Object
Alias for CSV::read().
-
.table(path, **options) ⇒ Object
A shortcut for:.
Instance Method Summary collapse
-
#<<(row) ⇒ Object
(also: #add_row, #puts)
The primary write method for wrapped Strings and IOs,
row
(an Array or CSV::Row) is converted to CSV and appended to the data source. - #binmode? ⇒ Boolean
-
#col_sep ⇒ Object
The encoded
:col_sep
used in parsing and writing. -
#convert(name = nil, &converter) ⇒ Object
:call-seq: convert( name ) convert { |field| … } convert { |field, field_info| … }.
-
#converters ⇒ Object
Returns the current list of converters in effect.
-
#each ⇒ Object
Yields each row of the data source in turn.
- #eof? ⇒ Boolean (also: #eof)
-
#field_size_limit ⇒ Object
The limit for field size, if any.
- #flock(*args) ⇒ Object
-
#force_quotes? ⇒ Boolean
Returns
true
if all output fields are quoted. -
#header_convert(name = nil, &converter) ⇒ Object
:call-seq: header_convert( name ) header_convert { |field| … } header_convert { |field, field_info| … }.
-
#header_converters ⇒ Object
Returns the current list of converters in effect for headers.
-
#header_row? ⇒ Boolean
Returns
true
if the next row read will be a header row. -
#headers ⇒ Object
Returns
nil
if headers will not be used,true
if they will but have not yet been read, or the actual headers after they have been read. -
#initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) ⇒ CSV
constructor
This constructor will wrap either a String or IO object passed in
data
for reading and/or writing. -
#inspect ⇒ Object
Returns a simplified description of the key CSV attributes in an ASCII compatible String.
- #ioctl(*args) ⇒ Object
-
#liberal_parsing? ⇒ Boolean
Returns
true
if illegal input is handled. -
#line ⇒ Object
The last row read from this file.
-
#lineno ⇒ Object
The line number of the last row read from this file.
- #path ⇒ Object
-
#quote_char ⇒ Object
The encoded
:quote_char
used in parsing and writing. -
#read ⇒ Object
(also: #readlines)
Slurps the remaining rows and returns an Array of Arrays.
-
#return_headers? ⇒ Boolean
Returns
true
if headers will be returned as a row of results. -
#rewind ⇒ Object
Rewinds the underlying IO object and resets CSV’s lineno() counter.
-
#row_sep ⇒ Object
The encoded
:row_sep
used in parsing and writing. -
#shift ⇒ Object
(also: #gets, #readline)
The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).
-
#skip_blanks? ⇒ Boolean
Returns
true
blank lines are skipped by the parser. -
#skip_lines ⇒ Object
The regex marking a line as a comment.
- #stat(*args) ⇒ Object
- #to_i ⇒ Object
- #to_io ⇒ Object
-
#unconverted_fields? ⇒ Boolean
Returns
true
if unconverted_fields() to parsed results. -
#write_headers? ⇒ Boolean
Returns
true
if headers are written in output.
Constructor Details
#initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) ⇒ CSV
This constructor will wrap either a String or IO object passed in data
for reading and/or writing. In addition to the CSV instance methods, several IO methods are delegated. (See CSV::open() for a complete list.) If you pass a String for data
, you can later retrieve it (after writing to it, for example) with CSV.string().
Note that a wrapped String will be positioned at the beginning (for reading). If you want it at the end (for writing), use CSV::generate(). If you want any other positioning, pass a preset StringIO object instead.
You may set any reading and/or writing preferences in the options
Hash. Available options are:
:col_sep
-
The String placed between each field. This String will be transcoded into the data’s Encoding before parsing.
:row_sep
-
The String appended to the end of each row. This can be set to the special
:auto
setting, which requests that CSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next"\r\n"
,"\n"
, or"\r"
sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found,data
isARGF
,STDIN
,STDOUT
, orSTDERR
, or the stream is only available for output, the default$INPUT_RECORD_SEPARATOR
($/
) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead. This String will be transcoded into the data’s Encoding before parsing. :quote_char
-
The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use
'
as the quote character instead of the correct"
. CSV will always consider a double sequence of this character to be an escaped quote. This String will be transcoded into the data’s Encoding before parsing. :field_size_limit
-
This is a maximum size CSV will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit CSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to
nil
, or off, by default. :converters
-
An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn’t have to be in an Array. All built-in converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.
:unconverted_fields
-
If set to
true
, an unconverted_fields() method will be added to all returned rows (Array or CSV::Row) that will return the fields as they were before conversion. Note that:headers
supplied by Array or String were not fields of the document and thus will have an empty Array attached. :headers
-
If set to
:first_row
ortrue
, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of CSV::parse_line() with the same:col_sep
,:row_sep
, and:quote_char
as this instance to produce an Array of headers. This setting causes CSV#shift() to return rows as CSV::Row objects instead of Arrays and CSV#read() to return CSV::Table objects instead of an Array of Arrays. :return_headers
-
When
false
, header rows are silently swallowed. If set totrue
, header rows are returned in a CSV::Row object with identical headers and fields (save that the fields do not go through the converters). :write_headers
-
When
true
and:headers
is set, a header row will be added to the output. :header_converters
-
Identical in functionality to
:converters
save that the conversions are only made to header rows. All built-in converters try to transcode headers to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the header unchanged. :skip_blanks
-
When set to a
true
value, CSV will skip over any empty rows. Note that this setting will not skip rows that contain column separators, even if the rows contain no actual data. If you want to skip rows that contain separators but no content, consider using:skip_lines
, or inspecting fields.compact.empty? on each row. :force_quotes
-
When set to a
true
value, CSV will quote all CSV fields it creates. :skip_lines
-
When set to an object responding to
match
, every line matching it is considered a comment and ignored during parsing. When set to a String, it is first converted to a Regexp. When set tonil
no line is considered a comment. If the passed object does not respond tomatch
,ArgumentError
is thrown. :liberal_parsing
-
When set to a
true
value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields. :nil_value
-
When set an object, any values of an empty field are replaced by the set object, not nil.
:empty_value
-
When set an object, any values of a blank string field is replaced by the set object.
:quote_empty
-
TODO
:write_converters
-
TODO
:write_nil_value
-
TODO
:write_empty_value
-
TODO
:strip
-
TODO
See CSV::DEFAULT_OPTIONS for the default settings.
Options cannot be overridden in the instance methods for performance reasons, so be sure to set what you want here.
898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 |
# File 'lib/csv.rb', line 898 def initialize(data, col_sep: ",", row_sep: :auto, quote_char: '"', field_size_limit: nil, converters: nil, unconverted_fields: nil, headers: false, return_headers: false, write_headers: nil, header_converters: nil, skip_blanks: false, force_quotes: false, skip_lines: nil, liberal_parsing: false, internal_encoding: nil, external_encoding: nil, encoding: nil, nil_value: nil, empty_value: "", quote_empty: true, write_converters: nil, write_nil_value: nil, write_empty_value: "", strip: false) raise ArgumentError.new("Cannot parse nil as CSV") if data.nil? # create the IO object we will read from @io = data.is_a?(String) ? StringIO.new(data) : data @encoding = determine_encoding(encoding, internal_encoding) @base_fields_converter_options = { nil_value: nil_value, empty_value: empty_value, } @write_fields_converter_options = { nil_value: write_nil_value, empty_value: write_empty_value, } @initial_converters = converters @initial_header_converters = header_converters @initial_write_converters = write_converters @parser_options = { column_separator: col_sep, row_separator: row_sep, quote_character: quote_char, field_size_limit: field_size_limit, unconverted_fields: unconverted_fields, headers: headers, return_headers: return_headers, skip_blanks: skip_blanks, skip_lines: skip_lines, liberal_parsing: liberal_parsing, encoding: @encoding, nil_value: nil_value, empty_value: empty_value, strip: strip, } @parser = nil @writer_options = { encoding: @encoding, force_encoding: (not encoding.nil?), force_quotes: force_quotes, headers: headers, write_headers: write_headers, column_separator: col_sep, row_separator: row_sep, quote_character: quote_char, quote_empty: quote_empty, } @writer = nil writer if @writer_options[:write_headers] end |
Instance Attribute Details
#encoding ⇒ Object (readonly)
The Encoding CSV is parsing or writing in. This will be the Encoding you receive parsed data in and/or the Encoding data will be written in.
1091 1092 1093 |
# File 'lib/csv.rb', line 1091 def encoding @encoding end |
Class Method Details
.filter(input = nil, output = nil, **options) ⇒ Object
:call-seq:
filter( **options ) { |row| ... }
filter( input, **options ) { |row| ... }
filter( input, output, **options ) { |row| ... }
This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output
altered or not.
The input
and output
arguments can be anything CSV::new() accepts (generally String or IO objects). If not given, they default to ARGF
and $stdout
.
The options
parameter is also filtered down to CSV::new() after some clever key parsing. Any key beginning with :in_
or :input_
will have that leading identifier stripped and will only be used in the options
Hash for the input
object. Keys starting with :out_
or :output_
affect only output
. All other keys are assigned to both objects.
The :output_row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
).
468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 |
# File 'lib/csv.rb', line 468 def self.filter(input=nil, output=nil, **) # parse options for input, output, or both , = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR} .each do |key, value| case key.to_s when /\Ain(?:put)?_(.+)\Z/ [$1.to_sym] = value when /\Aout(?:put)?_(.+)\Z/ [$1.to_sym] = value else [key] = value [key] = value end end # build input and output wrappers input = new(input || ARGF, ) output = new(output || $stdout, ) # read, yield, write input.each do |row| yield row output << row end end |
.foreach(path, **options, &block) ⇒ Object
This method is intended as the primary interface for reading CSV files. You pass a path
and any options
you wish to set for the read. Each row of file will be passed to the provided block
in turn.
The options
parameter can be anything CSV::new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
507 508 509 510 511 512 |
# File 'lib/csv.rb', line 507 def self.foreach(path, **, &block) return to_enum(__method__, path, ) unless block_given? open(path, ) do |csv| csv.each(&block) end end |
.generate(str = nil, **options) {|csv| ... } ⇒ Object
:call-seq:
generate( str, **options ) { |csv| ... }
generate( **options ) { |csv| ... }
This method wraps a String you provide, or an empty default String, in a CSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.
Note that a passed String is modified by this method. Call dup() before passing if you need a new String.
The options
parameter can be anything CSV::new() understands. This method understands an additional :encoding
parameter when not passed a String to set the base Encoding for the output. CSV needs this hint if you plan to output non-ASCII compatible data.
532 533 534 535 536 537 538 539 540 541 542 543 544 545 |
# File 'lib/csv.rb', line 532 def self.generate(str=nil, **) # add a default empty String, if none was given if str str = StringIO.new(str) str.seek(0, IO::SEEK_END) else encoding = [:encoding] str = +"" str.force_encoding(encoding) if encoding end csv = new(str, ) # wrap yield csv # yield for appending csv.string # return final String end |
.generate_line(row, **options) ⇒ Object
This method is a shortcut for converting a single row (Array) into a CSV String.
The options
parameter can be anything CSV::new() understands. This method understands an additional :encoding
parameter to set the base Encoding for the output. This method will try to guess your Encoding from the first non-nil
field in row
, if possible, but you may need to use this parameter as a backup plan.
The :row_sep
option
defaults to $INPUT_RECORD_SEPARATOR
($/
) when calling this method.
560 561 562 563 564 565 566 567 568 569 |
# File 'lib/csv.rb', line 560 def self.generate_line(row, **) = {row_sep: $INPUT_RECORD_SEPARATOR}.merge() str = +"" if [:encoding] str.force_encoding([:encoding]) elsif field = row.find {|f| f.is_a?(String)} str.force_encoding(field.encoding) end (new(str, ) << row).string end |
.instance(data = $stdout, **options) ⇒ Object
This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same data
object (tested by Object#object_id()) with the same options
.
If a block is given, the instance is passed to the block and the return value becomes the return value of the block.
428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 |
# File 'lib/csv.rb', line 428 def self.instance(data = $stdout, **) # create a _signature_ for this method call, data object and options sig = [data.object_id] + .values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s }) # fetch or create the instance for this signature @@instances ||= Hash.new instance = (@@instances[sig] ||= new(data, )) if block_given? yield instance # run block, if given, returning result else instance # or return the instance end end |
.open(filename, mode = "r", **options) ⇒ Object
:call-seq:
open( filename, mode = "rb", **options ) { |faster_csv| ... }
open( filename, **options ) { |faster_csv| ... }
open( filename, mode = "rb", **options )
open( filename, **options )
This method opens an IO object, and wraps that with CSV. This is intended as the primary interface for writing a CSV file.
You must pass a filename
and may optionally add a mode
for Ruby’s open(). You may also pass an optional Hash containing any options
CSV::new() understands as the final argument.
This method works like Ruby’s open() call, in that it will pass a CSV object to a provided block and close it when the block terminates, or it will return the CSV object when no block is provided. (Note: This is different from the Ruby 1.8 CSV library which passed rows to the block. Use CSV::foreach() for that behavior.)
You must provide a mode
with an embedded Encoding designator unless your data is in Encoding::default_external(). CSV will check the Encoding of the underlying IO object (set by the mode
you pass) to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read just as you can with a normal call to IO::open(). For example, "rb:UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
An opened CSV object will delegate to many IO methods for convenience. You may call:
-
binmode()
-
binmode?()
-
close()
-
close_read()
-
close_write()
-
closed?()
-
eof()
-
eof?()
-
external_encoding()
-
fcntl()
-
fileno()
-
flock()
-
flush()
-
fsync()
-
internal_encoding()
-
ioctl()
-
isatty()
-
path()
-
pid()
-
pos()
-
pos=()
-
reopen()
-
seek()
-
stat()
-
sync()
-
sync=()
-
tell()
-
to_i()
-
to_io()
-
truncate()
-
tty?()
634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 |
# File 'lib/csv.rb', line 634 def self.open(filename, mode="r", **) # wrap a File opened with the remaining +args+ with no newline # decorator file_opts = {universal_newline: false}.merge() begin f = File.open(filename, mode, file_opts) rescue ArgumentError => e raise unless /needs binmode/.match?(e.) and mode == "r" mode = "rb" file_opts = {encoding: Encoding.default_external}.merge(file_opts) retry end begin csv = new(f, ) rescue Exception f.close raise end # handle blocks like Ruby's open(), not like the CSV library if block_given? begin yield csv ensure csv.close end else csv end end |
.parse(*args, &block) ⇒ Object
:call-seq:
parse( str, **options ) { |row| ... }
parse( str, **options )
This method can be used to easily parse CSV out of a String. You may either provide a block
which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block
is given).
You pass your str
to read from, and an optional options
containing anything CSV::new() understands.
678 679 680 681 682 683 684 685 686 687 688 689 |
# File 'lib/csv.rb', line 678 def self.parse(*args, &block) csv = new(*args) return csv.each(&block) if block_given? # slurp contents, if no block is given begin csv.read ensure csv.close end end |
.parse_line(line, **options) ⇒ Object
This method is a shortcut for converting a single line of a CSV String into an Array. Note that if line
contains multiple rows, anything beyond the first row is ignored.
The options
parameter can be anything CSV::new() understands.
698 699 700 |
# File 'lib/csv.rb', line 698 def self.parse_line(line, **) new(line, ).shift end |
.read(path, *options) ⇒ Object
Use to slurp a CSV file into an Array of Arrays. Pass the path
to the file and any options
CSV::new() understands. This method also understands an additional :encoding
parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8"
would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.
713 714 715 |
# File 'lib/csv.rb', line 713 def self.read(path, *) open(path, *) { |csv| csv.read } end |
.readlines(*args) ⇒ Object
Alias for CSV::read().
718 719 720 |
# File 'lib/csv.rb', line 718 def self.readlines(*args) read(*args) end |
.table(path, **options) ⇒ Object
729 730 731 732 733 |
# File 'lib/csv.rb', line 729 def self.table(path, **) read( path, { headers: true, converters: :numeric, header_converters: :symbol }.merge() ) end |
Instance Method Details
#<<(row) ⇒ Object Also known as: add_row, puts
The primary write method for wrapped Strings and IOs, row
(an Array or CSV::Row) is converted to CSV and appended to the data source. When a CSV::Row is passed, only the row’s fields() are appended to the output.
The data source must be open for writing.
1185 1186 1187 1188 |
# File 'lib/csv.rb', line 1185 def <<(row) writer << row self end |
#binmode? ⇒ Boolean
1122 1123 1124 1125 1126 1127 1128 |
# File 'lib/csv.rb', line 1122 def binmode? if @io.respond_to?(:binmode?) @io.binmode? else false end end |
#col_sep ⇒ Object
The encoded :col_sep
used in parsing and writing. See CSV::new for details.
979 980 981 |
# File 'lib/csv.rb', line 979 def col_sep parser.column_separator end |
#convert(name = nil, &converter) ⇒ Object
:call-seq:
convert( name )
convert { |field| ... }
convert { |field, field_info| ... }
You can use this method to install a CSV::Converters built-in, or provide a block that handles a custom conversion.
If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a CSV::FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.
1207 1208 1209 |
# File 'lib/csv.rb', line 1207 def convert(name = nil, &converter) parser_fields_converter.add_converter(name, &converter) end |
#converters ⇒ Object
Returns the current list of converters in effect. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.
1014 1015 1016 1017 1018 1019 |
# File 'lib/csv.rb', line 1014 def converters parser_fields_converter.map do |converter| name = Converters.rassoc(converter) name ? name.first : converter end end |
#each ⇒ Object
Yields each row of the data source in turn.
Support for Enumerable.
The data source must be open for reading.
1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 |
# File 'lib/csv.rb', line 1235 def each return to_enum(__method__) unless block_given? enumerator = parser_enumerator begin while true yield enumerator.next end rescue StopIteration end self end |
#eof? ⇒ Boolean Also known as: eof
1158 1159 1160 1161 1162 1163 1164 1165 |
# File 'lib/csv.rb', line 1158 def eof? begin parser_enumerator.peek false rescue StopIteration true end end |
#field_size_limit ⇒ Object
The limit for field size, if any. See CSV::new for details.
1000 1001 1002 |
# File 'lib/csv.rb', line 1000 def field_size_limit parser.field_size_limit end |
#flock(*args) ⇒ Object
1130 1131 1132 1133 |
# File 'lib/csv.rb', line 1130 def flock(*args) raise NotImplementedError unless @io.respond_to?(:flock) @io.flock(*args) end |
#force_quotes? ⇒ Boolean
Returns true
if all output fields are quoted. See CSV::new for details.
1078 1079 1080 |
# File 'lib/csv.rb', line 1078 def force_quotes? @writer_options[:force_quotes] end |
#header_convert(name = nil, &converter) ⇒ Object
:call-seq:
header_convert( name )
header_convert { |field| ... }
header_convert { |field, field_info| ... }
Identical to CSV#convert(), but for header rows.
Note that this method must be called before header rows are read to have any effect.
1222 1223 1224 |
# File 'lib/csv.rb', line 1222 def header_convert(name = nil, &converter) header_fields_converter.add_converter(name, &converter) end |
#header_converters ⇒ Object
Returns the current list of converters in effect for headers. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.
1062 1063 1064 1065 1066 1067 |
# File 'lib/csv.rb', line 1062 def header_converters header_fields_converter.map do |converter| name = HeaderConverters.rassoc(converter) name ? name.first : converter end end |
#header_row? ⇒ Boolean
Returns true
if the next row read will be a header row.
1263 1264 1265 |
# File 'lib/csv.rb', line 1263 def header_row? parser.header_row? end |
#headers ⇒ Object
Returns nil
if headers will not be used, true
if they will but have not yet been read, or the actual headers after they have been read. See CSV::new for details.
1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 |
# File 'lib/csv.rb', line 1033 def headers if @writer @writer.headers else parsed_headers = parser.headers return parsed_headers if parsed_headers raw_headers = @parser_options[:headers] raw_headers = nil if raw_headers == false raw_headers end end |
#inspect ⇒ Object
Returns a simplified description of the key CSV attributes in an ASCII compatible String.
1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 |
# File 'lib/csv.rb', line 1288 def inspect str = ["<#", self.class.to_s, " io_type:"] # show type of wrapped IO if @io == $stdout then str << "$stdout" elsif @io == $stdin then str << "$stdin" elsif @io == $stderr then str << "$stderr" else str << @io.class.to_s end # show IO.path(), if available if @io.respond_to?(:path) and (p = @io.path) str << " io_path:" << p.inspect end # show encoding str << " encoding:" << @encoding.name # show other attributes ["lineno", "col_sep", "row_sep", "quote_char"].each do |attr_name| if a = __send__(attr_name) str << " " << attr_name << ":" << a.inspect end end ["skip_blanks", "liberal_parsing"].each do |attr_name| if a = __send__("#{attr_name}?") str << " " << attr_name << ":" << a.inspect end end _headers = headers str << " headers:" << _headers.inspect if _headers str << ">" begin str.join('') rescue # any encoding error str.map do |s| e = Encoding::Converter.asciicompat_encoding(s.encoding) e ? s.encode(e) : s.force_encoding("ASCII-8BIT") end.join('') end end |
#ioctl(*args) ⇒ Object
1135 1136 1137 1138 |
# File 'lib/csv.rb', line 1135 def ioctl(*args) raise NotImplementedError unless @io.respond_to?(:ioctl) @io.ioctl(*args) end |
#liberal_parsing? ⇒ Boolean
Returns true
if illegal input is handled. See CSV::new for details.
1083 1084 1085 |
# File 'lib/csv.rb', line 1083 def liberal_parsing? parser.liberal_parsing? end |
#line ⇒ Object
The last row read from this file.
1108 1109 1110 |
# File 'lib/csv.rb', line 1108 def line parser.line end |
#lineno ⇒ Object
The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.
1097 1098 1099 1100 1101 1102 1103 |
# File 'lib/csv.rb', line 1097 def lineno if @writer @writer.lineno else parser.lineno end end |
#path ⇒ Object
1140 1141 1142 |
# File 'lib/csv.rb', line 1140 def path @io.path if @io.respond_to?(:path) end |
#quote_char ⇒ Object
The encoded :quote_char
used in parsing and writing. See CSV::new for details.
995 996 997 |
# File 'lib/csv.rb', line 995 def quote_char parser.quote_character end |
#read ⇒ Object Also known as: readlines
Slurps the remaining rows and returns an Array of Arrays.
The data source must be open for reading.
1252 1253 1254 1255 1256 1257 1258 1259 |
# File 'lib/csv.rb', line 1252 def read rows = to_a if parser.use_headers? Table.new(rows, headers: parser.headers) else rows end end |
#return_headers? ⇒ Boolean
Returns true
if headers will be returned as a row of results. See CSV::new for details.
1048 1049 1050 |
# File 'lib/csv.rb', line 1048 def return_headers? parser.return_headers? end |
#rewind ⇒ Object
Rewinds the underlying IO object and resets CSV’s lineno() counter.
1169 1170 1171 1172 1173 1174 |
# File 'lib/csv.rb', line 1169 def rewind @parser = nil @parser_enumerator = nil @writer.rewind if @writer @io.rewind end |
#row_sep ⇒ Object
The encoded :row_sep
used in parsing and writing. See CSV::new for details.
987 988 989 |
# File 'lib/csv.rb', line 987 def row_sep parser.row_separator end |
#shift ⇒ Object Also known as: gets, readline
The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).
The data source must be open for reading.
1274 1275 1276 1277 1278 1279 1280 |
# File 'lib/csv.rb', line 1274 def shift begin parser_enumerator.next rescue StopIteration nil end end |
#skip_blanks? ⇒ Boolean
Returns true
blank lines are skipped by the parser. See CSV::new for details.
1073 1074 1075 |
# File 'lib/csv.rb', line 1073 def skip_blanks? parser.skip_blanks? end |
#skip_lines ⇒ Object
The regex marking a line as a comment. See CSV::new for details
1005 1006 1007 |
# File 'lib/csv.rb', line 1005 def skip_lines parser.skip_lines end |
#stat(*args) ⇒ Object
1144 1145 1146 1147 |
# File 'lib/csv.rb', line 1144 def stat(*args) raise NotImplementedError unless @io.respond_to?(:stat) @io.stat(*args) end |
#to_i ⇒ Object
1149 1150 1151 1152 |
# File 'lib/csv.rb', line 1149 def to_i raise NotImplementedError unless @io.respond_to?(:to_i) @io.to_i end |
#to_io ⇒ Object
1154 1155 1156 |
# File 'lib/csv.rb', line 1154 def to_io @io.respond_to?(:to_io) ? @io.to_io : @io end |
#unconverted_fields? ⇒ Boolean
Returns true
if unconverted_fields() to parsed results. See CSV::new for details.
1024 1025 1026 |
# File 'lib/csv.rb', line 1024 def unconverted_fields? parser.unconverted_fields? end |
#write_headers? ⇒ Boolean
Returns true
if headers are written in output. See CSV::new for details.
1053 1054 1055 |
# File 'lib/csv.rb', line 1053 def write_headers? @writer_options[:write_headers] end |