Class: Kotoshu::Cache::BaseCache Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/kotoshu/cache/base_cache.rb

Overview

This class is abstract.

Subclass must implement #download_resource, #load_cached

Abstract base class for all cache implementations.

Provides common functionality for:

  • HTTP downloads with metadata
  • Cache validation (exists, expired)
  • Statistics tracking (hits, misses, hit rate)
  • TTL management

Subclasses implement specific download and loading logic.

Direct Known Subclasses

FrequencyCache, LanguageCache, ModelCache

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_path: nil, url_base: nil, cache_ttl: nil, github_url: nil, resource_pin: nil, manifest_url: nil, audit_log: nil, source_registry: nil, max_cache_size: nil) ⇒ BaseCache

Create a new cache.



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/kotoshu/cache/base_cache.rb', line 61

def initialize(cache_path: nil, url_base: nil, cache_ttl: nil, github_url: nil,
               resource_pin: nil, manifest_url: nil, audit_log: nil,
               source_registry: nil, max_cache_size: nil)
  @cache_path = cache_path || default_cache_path
  @source_registry = source_registry || source_registry_for(url_base)
  @url_base = url_base || @source_registry.base_url
  @cache_ttl = cache_ttl || default_cache_ttl
  @github_url = github_url || default_github_url
  @resource_pin = resource_pin || @source_registry.pin_for_source(:spelling)
  @manifest_url = manifest_url
  @audit_log = audit_log || default_audit_log
  @max_cache_size = max_cache_size || default_max_cache_size
  @manifest = nil
  @manifest_loaded = false
  @hits = 0
  @misses = 0

  # Ensure cache directory exists
  FileUtils.mkdir_p(@cache_path)
  FileUtils.mkdir_p(File.join(@cache_path, "tmp"))
end

Instance Attribute Details

#cache_path ⇒ String (readonly) 



25
26
27
 
# File 'lib/kotoshu/cache/base_cache.rb', line 25

def cache_path
  @cache_path
end

#cache_ttl ⇒ Integer (readonly) 



31
32
33
 
# File 'lib/kotoshu/cache/base_cache.rb', line 31

def cache_ttl
  @cache_ttl
end

#github_url ⇒ String (readonly) 



40
41
42
 
# File 'lib/kotoshu/cache/base_cache.rb', line 40

def github_url
  @github_url
end

#max_cache_size ⇒ Integer (readonly) 



37
38
39
 
# File 'lib/kotoshu/cache/base_cache.rb', line 37

def max_cache_size
  @max_cache_size
end

#source_registry ⇒ Kotoshu::SourceRegistry (readonly) 



45
46
47
 
# File 'lib/kotoshu/cache/base_cache.rb', line 45

def source_registry
  @source_registry
end

#url_base ⇒ String (readonly) 



28
29
30
 
# File 'lib/kotoshu/cache/base_cache.rb', line 28

def url_base
  @url_base
end

Instance Method Details

#available?(resource_id) ⇒ Boolean

Check if a resource is available in cache.



87
88
89
90
91
92
93
94
95
 
# File 'lib/kotoshu/cache/base_cache.rb', line 87

def available?(resource_id)
  return false unless supports_resource?(resource_id)

  metadata_path = metadata_path_for(resource_id)
  return false unless File.exist?(metadata_path)
  return false if expired?(metadata_path)

  resource_files_exist?(resource_id)
end

#cache_kind ⇒ String

Short label for error messages ("frequency", "language", "model") derived from the nearest named class — anonymous subclasses (test recorders) fall back to their parent's name.



290
291
292
293
294
 
# File 'lib/kotoshu/cache/base_cache.rb', line 290

def cache_kind
  klass = self.class
  klass = klass.superclass while klass && klass.name.nil?
  klass&.name.to_s[/(\w+)Cache\z/, 1]&.downcase || "resource"
end

#cached_data?(resource_id) ⇒ Boolean

Whether readable cached data exists for resource_id, TTL ignored (plan 117). Expiry is a refresh signal for the setup / download paths, never a reason for readers to behave as if the data were absent: the bytes are checksummed at write time, so present-but-expired data is exactly the dataset it was.



105
106
107
108
109
 
# File 'lib/kotoshu/cache/base_cache.rb', line 105

def cached_data?(resource_id)
  return false unless supports_resource?(resource_id)

  File.exist?(metadata_path_for(resource_id)) && resource_files_exist?(resource_id)
end

#cached_resources ⇒ Array<String>

List all cached resources.

Raises:

  • (NotImplementedError) 


224
225
226
 
# File 'lib/kotoshu/cache/base_cache.rb', line 224

def cached_resources
  raise NotImplementedError, "Subclass must implement"
end

#clean ⇒ Hash

Clean expired cache entries.



187
188
189
190
191
192
193
194
195
 
# File 'lib/kotoshu/cache/base_cache.rb', line 187

def clean
  expired_count = clean_expired
  size_reclaimed = clean_by_size

  {
    expired_entries_removed: expired_count,
    bytes_reclaimed: size_reclaimed
  }
end

#clear(resource_id) ⇒ Boolean

Clear a specific resource from cache.



134
135
136
137
138
139
140
141
142
143
144
 
# File 'lib/kotoshu/cache/base_cache.rb', line 134

def clear(resource_id)
  return false unless supports_resource?(resource_id)

  resource_dir = resource_dir_for(resource_id)
  if File.exist?(resource_dir)
    FileUtils.rm_rf(resource_dir)
    return true
  end

  false
end

#clear_all

This method returns an undefined value.

Clear all cached resources.



149
150
151
152
153
154
155
 
# File 'lib/kotoshu/cache/base_cache.rb', line 149

def clear_all
  @hits = 0
  @misses = 0
  FileUtils.rm_rf(@cache_path)
  FileUtils.mkdir_p(@cache_path)
  FileUtils.mkdir_p(File.join(@cache_path, "tmp"))
end

#download(resource_id) ⇒ Object?

Download a resource from GitHub.

Raises:

  • (Kotoshu::DictionaryNotFoundError) —

    when the fetch fails (unreachable source, HTTP error); the documented cache API boundary raises the public error class rather than leaking transport errors

  • (Kotoshu::ResourceNotCachedError) —

    in offline mode — only explicit paths (Kotoshu.setup, kotoshu cache download) reach here, so fail loudly instead of silently skipping



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
 
# File 'lib/kotoshu/cache/base_cache.rb', line 247

def download(resource_id)
  return nil unless supports_resource?(resource_id)

  language, explicit_type = resource_id.to_s.split(":", 2)
  raise Kotoshu::ResourceNotCachedError.new(language, explicit_type || cache_kind) if offline?

  resource_dir = resource_dir_for(resource_id)
  FileUtils.mkdir_p(resource_dir)

  begin
    download_resource(resource_id, resource_dir)
  rescue Kotoshu::DictionaryNotFoundError
    raise
  rescue StandardError => e
    warn "Error downloading #{resource_id}: #{e.message}" if $VERBOSE
    nil
  end
end

#download_resource(resource_id, dest_path) ⇒ Object

This method is abstract.

Subclass must implement

Abstract: Download a specific resource.

Raises:

  • (NotImplementedError) 


272
273
274
 
# File 'lib/kotoshu/cache/base_cache.rb', line 272

def download_resource(resource_id, dest_path)
  raise NotImplementedError, "Subclass must implement"
end

#evict(dry_run: false) ⇒ Hash

Enforce the cache size cap by evicting the oldest entries (LRU by cached_at). When dry_run: true, returns the plan without modifying the disk; otherwise removes the evicted entries via FileUtils.rm_rf and returns the same plan.

The plan is computed by EvictionPolicy — a pure value object that performs no IO. This method is the only place that collects on-disk entries and the only place that mutates the cache directory based on the plan, keeping the policy testable in isolation.



212
213
214
215
216
217
218
219
 
# File 'lib/kotoshu/cache/base_cache.rb', line 212

def evict(dry_run: false)
  policy = EvictionPolicy.new(max_size: @max_cache_size)
  plan = policy.plan(collect_eviction_entries)
  return plan if dry_run

  plan[:evict].each { |e| FileUtils.rm_rf(e[:path]) }
  plan
end

#get(resource_id, force_download: false) ⇒ Object?

Get a resource from cache or download it.



116
117
118
119
120
121
122
123
124
125
126
127
128
 
# File 'lib/kotoshu/cache/base_cache.rb', line 116

def get(resource_id, force_download: false)
  return nil unless supports_resource?(resource_id)

  metadata_path = metadata_path_for(resource_id)

  if !force_download && cached?(metadata_path) && !expired?(metadata_path)
    @hits += 1
    return load_cached(resource_id)
  end

  @misses += 1
  download(resource_id)
end

#load_cached(resource_id) ⇒ Object?

This method is abstract.

Subclass must implement

Abstract: Load cached resource data.

Raises:

  • (NotImplementedError) 


301
302
303
 
# File 'lib/kotoshu/cache/base_cache.rb', line 301

def load_cached(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#metadata_path_for(resource_id) ⇒ String

This method is abstract.

Subclass must implement

Abstract: Get metadata file path for a resource.

Raises:

  • (NotImplementedError) 


310
311
312
 
# File 'lib/kotoshu/cache/base_cache.rb', line 310

def metadata_path_for(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#offline? ⇒ Boolean

True when downloads are forbidden (config.offline / KOTOSHU_OFFLINE=1). Checked in #download so every cache subclass inherits the enforcement.



281
282
283
 
# File 'lib/kotoshu/cache/base_cache.rb', line 281

def offline?
  Kotoshu.configuration.offline
end

#read_metadata(path) ⇒ Hash?

Read metadata from file.

Public because the CLI cache subcommand and other inspection paths consume it to display cache state without going through the full download-and-load pipeline.



770
771
772
773
774
775
776
 
# File 'lib/kotoshu/cache/base_cache.rb', line 770

def read_metadata(path)
  return nil unless File.exist?(path)

  JSON.parse(File.read(path))
rescue JSON::ParserError
  nil
end

#reset_stats ⇒ self

Reset statistics counters.



178
179
180
181
182
 
# File 'lib/kotoshu/cache/base_cache.rb', line 178

def reset_stats
  @hits = 0
  @misses = 0
  self
end

#resource_dir_for(resource_id) ⇒ String

This method is abstract.

Subclass must implement

Abstract: Get resource directory path.

Raises:

  • (NotImplementedError) 


319
320
321
 
# File 'lib/kotoshu/cache/base_cache.rb', line 319

def resource_dir_for(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#resource_files_exist?(resource_id) ⇒ Boolean

This method is abstract.

Subclass must implement

Abstract: Check if all resource files exist.

Raises:

  • (NotImplementedError) 


328
329
330
 
# File 'lib/kotoshu/cache/base_cache.rb', line 328

def resource_files_exist?(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#stats ⇒ Hash

Get cache statistics.



160
161
162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/kotoshu/cache/base_cache.rb', line 160

def stats
  total = @hits + @misses
  hit_rate = total.positive? ? (@hits.to_f / total) : 0.0

  {
    hits: @hits,
    misses: @misses,
    total: total,
    hit_rate: hit_rate,
    cached_resources: cached_resources,
    size_bytes: cache_size,
    oldest_entry: oldest_entry
  }
end

#supports_resource?(resource_id) ⇒ Boolean

Check if a resource type is supported.

Raises:

  • (NotImplementedError) 


232
233
234
 
# File 'lib/kotoshu/cache/base_cache.rb', line 232

def supports_resource?(resource_id)
  raise NotImplementedError, "Subclass must implement"
end