Class: SecApi::Collections::Filings

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/sec_api/collections/filings.rb

Overview

A collection of SEC filings with Enumerable support and pagination.

Pagination Design (Architecture ADR-6): Uses cursor-based pagination via “from” offset rather than page numbers. Why cursor-based? More efficient for large datasets - the server doesn’t need to calculate page boundaries, and the client can stop early without fetching unnecessary data. Supports both manual (fetch_next_page) and automatic (auto_paginate) iteration patterns.

Filings collections are returned from query operations and support iteration, pagination metadata, total count from API response, and fetching subsequent pages of results.

Examples:

Iterating through filings

filings = client.query.ticker("AAPL").search
filings.each { |f| puts f.form_type }

Using Enumerable methods

filings.map(&:ticker)           #=> ["AAPL", "AAPL", ...]
filings.select { |f| f.form_type == "10-K" }
filings.first                   #=> Filing

Accessing total count from API

filings.count     #=> 1250 (total results, not just current page)
filings.to_a.size #=> 50 (current page size)

Pagination

filings = client.query.ticker("AAPL").search
while filings.has_more?
  filings.each { |f| process(f) }
  filings = filings.fetch_next_page
end

See Also:

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(data, client: nil, query_context: nil) ⇒ Filings

Initialize a new Filings collection.

Parameters:

  • data (Hash) —

    API response data containing filings array

  • client (SecApi::Client, nil) (defaults to: nil) —

    client instance for pagination requests

  • query_context (Hash, nil) (defaults to: nil) —

    original query parameters for pagination



53
54
55
56
57
58
59
60
 
# File 'lib/sec_api/collections/filings.rb', line 53

def initialize(data, client: nil, query_context: nil)
  @_data = data
  @_client = client
  @_query_context = query_context
  build_objects
  build_metadata
  freeze_collection
end

Instance Attribute Details

#next_cursor ⇒ Integer (readonly)

Returns offset position for fetching next page of results.

Returns:

  • (Integer) —

    offset position for fetching next page of results



46
47
48
 
# File 'lib/sec_api/collections/filings.rb', line 46

def next_cursor
  @next_cursor
end

#total_count ⇒ Object (readonly)

Returns the value of attribute total_count.



46
 
# File 'lib/sec_api/collections/filings.rb', line 46

attr_reader :next_cursor, :total_count

Instance Method Details

#auto_paginate ⇒ Enumerator::Lazy

Note:

Memory Efficiency: Only the current page is held in memory. Previous pages become eligible for garbage collection as iteration proceeds.

Note:

Retry Behavior: Transient errors (503, timeouts) during page fetches are automatically retried by the middleware. Permanent errors (401, 404) will be raised to the caller.

Returns a lazy enumerator that automatically paginates through all results.

Memory Efficiency Design: Why Enumerator::Lazy? For backfill operations with 100k+ results, we can’t load all filings into memory. Lazy enumeration fetches pages on-demand:

  • Only current page in memory (~50 filings)

  • Previous pages become GC-eligible as we iterate

  • Early termination via .take(N) avoids fetching unnecessary pages

  • Enumerable chaining (.select, .map) works naturally

Each iteration yields a single Objects::Filing object. Pages are fetched on-demand as the enumerator is consumed, keeping memory usage constant regardless of total result count. Only the current page is held in memory; previous pages become eligible for garbage collection as iteration proceeds.

Examples:

Backfill with early termination

client.query
  .ticker("AAPL")
  .date_range(from: 5.years.ago, to: Date.today)
  .search
  .auto_paginate
  .each { |f| process(f) }

Collect all results (use cautiously with large datasets)

all_filings = filings.auto_paginate.to_a

With filtering (Enumerable methods work with lazy enumerator)

filings.auto_paginate
  .select { |f| f.form_type == "10-K" }
  .take(100)
  .each { |f| process(f) }

Returns:

Raises:

See Also:



181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
 
# File 'lib/sec_api/collections/filings.rb', line 181

def auto_paginate
  raise PaginationError, "Cannot paginate without client reference" if @_client.nil?

  Enumerator.new do |yielder|
    current_page = self

    loop do
      # Yield each filing from current page
      current_page.each { |filing| yielder << filing }

      # Stop if no more pages
      break unless current_page.has_more?

      # Fetch next page (becomes new current, old page eligible for GC)
      next_page = current_page.fetch_next_page

      # Guard against infinite loop if API returns empty page mid-pagination
      # (defensive coding against API misbehavior)
      break if next_page.to_a.empty? && current_page.next_cursor == next_page.next_cursor

      current_page = next_page
    end
  end.lazy
end

#count ⇒ Integer #count(item) ⇒ Integer #count {|filing| ... } ⇒ Integer

Note:

When filtering, only filings in the current page are counted. For total filtered count across all pages, use auto_paginate.

Returns total count of results from API metadata, or delegates to Enumerable#count when filtering.

When called without arguments, returns the total number of matching filings across all pages (from API metadata), not just the count of filings in the current page.

When called with a block or argument, delegates to Enumerable#count to count filings in the current page matching the condition.

Examples:

Total count from API

filings.count     #=> 1250 (total matching filings across all pages)

Filtered count in current page

filings.count { |f| f.form_type == "10-K" }  #=> 5 (in current page)

Overloads:

  • #count ⇒ Integer

    Returns total count from API metadata

    Returns:

    • (Integer) —

      total count from API, or current page size if unavailable

  • #count(item) ⇒ Integer

    Counts occurrences of item in current page (delegates to Enumerable)

    Parameters:

    • item (Object) —

      the item to count

    Returns:

    • (Integer) —

      count of matching items in current page

  • #count {|filing| ... } ⇒ Integer

    Counts filings matching block in current page (delegates to Enumerable)

    Yields:

    • (filing) —

      each filing to test

    Returns:

    • (Integer) —

      count of filings where block returns true



111
112
113
114
115
116
117
118
119
120
121
122
123
124
 
# File 'lib/sec_api/collections/filings.rb', line 111

def count(*args, &block)
  if block || args.any?
    super
  else
    case @total_count
    when Hash
      @total_count[:value] || @total_count["value"] || @objects.size
    when Integer
      @total_count
    else
      @objects.size
    end
  end
end

#each {|filing| ... } ⇒ Enumerator

Yields each Filing to the block. Required for Enumerable support.

Yields:

  • (filing) —

    each filing in the collection

Yield Parameters:

Returns:

  • (Enumerator) —

    if no block given



75
76
77
 
# File 'lib/sec_api/collections/filings.rb', line 75

def each(&block)
  @objects.each(&block)
end

#fetch_next_page ⇒ Filings

Fetch the next page of results.

Makes an API request using the stored query context with the next cursor offset. Returns a new immutable Filings collection containing the next page of results.

Examples:

Manual pagination

filings = client.query.ticker("AAPL").search
if filings.has_more?
  next_page = filings.fetch_next_page
  next_page.each { |f| puts f.accession_number }
end

Returns:

  • (Filings) —

    new collection with the next page of filings

Raises:



221
222
223
224
225
226
227
 
# File 'lib/sec_api/collections/filings.rb', line 221

def fetch_next_page
  raise PaginationError, "No more pages available" unless has_more?

  payload = @_query_context.merge(from: @next_cursor.to_s)
  response = @_client.connection.post("/", payload)
  Filings.new(response.body, client: @_client, query_context: @_query_context)
end

#filings ⇒ Array<Objects::Filing>

Returns the array of Filing objects.

Returns:



65
66
67
 
# File 'lib/sec_api/collections/filings.rb', line 65

def filings
  @objects
end

#has_more? ⇒ Boolean

Returns true if more pages of results are available.

More pages are available when:

  • A client reference exists (pagination requires API access)

  • The next_cursor is less than the total count

Returns:

  • (Boolean) —

    true if more pages can be fetched



133
134
135
136
 
# File 'lib/sec_api/collections/filings.rb', line 133

def has_more?
  return false if @_client.nil?
  @next_cursor < extract_total_value
end