Class: Air18n::Backend
- Inherits:
-
Object
- Object
- Air18n::Backend
- Includes:
- I18n::Backend::Base
- Defined in:
- lib/air18n/backend.rb
Constant Summary collapse
- T_LAST_LOADED_AT =
Define constants used as cache keys
'Air18n::translations_last_loaded_at_%s'
- T_LAST_UPDATED_AT =
'Air18n::translations_last_updated_at_%s'
- T_DATA =
'Air18n::translation_data_%s'
- RACE_CONDITION_TTL =
This value allows one app instance to make a DB call while other still pull slightly stale data from the cache. The unit is seconds.
5
- APPROX_MAX_SCREENSHOTS_PER_KEY =
We will only take 2 screenshots per Rails instance per key. This is to avoid having to moderate 200 screenshots for every new site-wide phrase. Because this is a per-Rails-thread cap, it is a lower bound for how many screenshot we will end up with per widely-used phrase.
3
Instance Attribute Summary collapse
-
#default_text_change_observer ⇒ Object
-
translation_data: map of locale to key to translation - phrase_screenshots: map of phrase key to the routes context that we have screenshots for - default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
-
-
#phrase_screenshots ⇒ Object
-
translation_data: map of locale to key to translation - phrase_screenshots: map of phrase key to the routes context that we have screenshots for - default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
-
-
#translation_data ⇒ Object
-
translation_data: map of locale to key to translation - phrase_screenshots: map of phrase key to the routes context that we have screenshots for - default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
-
Instance Method Summary collapse
- #available_locales ⇒ Object
- #check_for_new_translations(locale) ⇒ Object
- #check_last_timestamps(locale) ⇒ Object
- #get_from_cache_or_reload(locale) ⇒ Object
- #guess_translation(text, orig_locale, other_locale) ⇒ Object
-
#has_translation?(key, locale) ⇒ Boolean
If there is a translation of given key in given, or a less-specific fallback locale, returns the most specific locale that has a translation.
- #init_translations(locale) ⇒ Object
-
#lookup(locale, key, scope = [], options = {}) ⇒ Object
This method does the meat of Air18n functionality.
- #reload_translations(locales) ⇒ Object
- #rescreenshot(routes_context) ⇒ Object
- #reset_phrase_screenshots ⇒ Object
-
#store_translations(locale, data, options = {}) ⇒ Object
Stores translations for a given locale.
Instance Attribute Details
#default_text_change_observer ⇒ Object
-
translation_data: map of locale to key to translation
-
phrase_screenshots: map of phrase key to the routes context that we have screenshots for
-
default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
19 20 21 |
# File 'lib/air18n/backend.rb', line 19 def default_text_change_observer @default_text_change_observer end |
#phrase_screenshots ⇒ Object
-
translation_data: map of locale to key to translation
-
phrase_screenshots: map of phrase key to the routes context that we have screenshots for
-
default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
19 20 21 |
# File 'lib/air18n/backend.rb', line 19 def phrase_screenshots @phrase_screenshots end |
#translation_data ⇒ Object
-
translation_data: map of locale to key to translation
-
phrase_screenshots: map of phrase key to the routes context that we have screenshots for
-
default_text_change_observer: set this to an implementation of DefaultTextChangeObserver to be alerted to changes of default text, and be able to guard against flipflops.
19 20 21 |
# File 'lib/air18n/backend.rb', line 19 def translation_data @translation_data end |
Instance Method Details
#available_locales ⇒ Object
39 40 41 42 43 44 45 46 47 48 |
# File 'lib/air18n/backend.rb', line 39 def available_locales if @translation_data @translation_data.inject([]) do |carry, (locale, translations)| carry << locale unless translations.empty? carry end else [] end end |
#check_for_new_translations(locale) ⇒ Object
123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
# File 'lib/air18n/backend.rb', line 123 def check_for_new_translations(locale) # When TranslateController makes a new translation, it sets # translations_last_updated_at to the current time in the cache. # If we haven't reset the i18n backends since then, we take the opportunity to reset them. # No-op if locale is the default locale; its translations never change. if locale != I18n.default_locale translation_last_loaded_at, last_updated_at = (locale) if (translation_last_loaded_at.nil?) || (last_updated_at && last_updated_at.to_i >= translation_last_loaded_at.to_i) I18n.cache.write(T_LAST_LOADED_AT % locale, Time.now + RACE_CONDITION_TTL) if I18n.cache reload_translations([locale]) end end end |
#check_last_timestamps(locale) ⇒ Object
90 91 92 93 94 95 96 97 98 99 100 101 |
# File 'lib/air18n/backend.rb', line 90 def (locale) # Potential race condition here but the order of operations will at worst # cause an extra DB call. if I18n.cache translation_last_loaded_at = I18n.cache.read(T_LAST_LOADED_AT % locale) last_updated_at = I18n.cache.read(T_LAST_UPDATED_AT % locale) else translation_last_loaded_at = @translations_last_loaded_at[locale] last_updated_at ||= Time.now end [translation_last_loaded_at, last_updated_at] end |
#get_from_cache_or_reload(locale) ⇒ Object
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
# File 'lib/air18n/backend.rb', line 103 def get_from_cache_or_reload(locale) # This function will check the cache if available to see if there is data. # If it is available, read it and store it in a instance variable. # If it is not available, increment the last_loaded_at cache key by # a few seconds in the future. (This will prevent the majority of application # processes from doing a database lookup by serving them slightly stale data # from the cache. If multiple instances start up when the cache is empty, then multiple # call to the database are unavoidable unless a locking and wait style is used. # Refer to the specs 'I18n cache' for more information. cache_results = nil cache_results = ChunkCache::get(I18n.cache, T_DATA % locale) if I18n.cache if !cache_results.nil? @translation_data ||= {} @translation_data[locale.to_sym] = cache_results else I18n.cache.write(T_LAST_LOADED_AT % locale, Time.now) if I18n.cache reload_translations([locale]) end end |
#guess_translation(text, orig_locale, other_locale) ⇒ Object
145 146 147 148 |
# File 'lib/air18n/backend.rb', line 145 def guess_translation(text, orig_locale, other_locale) @prim_and_proper ||= PrimAndProper.new @prim_and_proper.guess(text, orig_locale, other_locale) end |
#has_translation?(key, locale) ⇒ Boolean
If there is a translation of given key in given, or a less-specific fallback locale, returns the most specific locale that has a translation. Returns nil otherwise.
For “guessed” specific locales like British English, returns the specific locale only if there is a manually-written translation for that locale.
This is useful for knowing whether or not there is a translation for a key in a locale. It can also be used to determine whether a translation comes from a fallback locale or not.
306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 |
# File 'lib/air18n/backend.rb', line 306 def has_translation?(key, locale) if key.blank? || !key.is_a?(String) return false end init_translations(locale) for fallback_locale in I18n.fallbacks_for(locale, :exclude_default => true) if @translation_data.include?(fallback_locale) && @translation_data[fallback_locale].include?(key) return fallback_locale end if fallback_locale == I18n.default_locale return fallback_locale end end return nil end |
#init_translations(locale) ⇒ Object
79 80 81 82 83 84 |
# File 'lib/air18n/backend.rb', line 79 def init_translations(locale) reset_phrase_screenshots unless @phrase_screenshots if @translation_data.nil? || !@translation_data.include?(locale) get_from_cache_or_reload(locale) end end |
#lookup(locale, key, scope = [], options = {}) ⇒ Object
This method does the meat of Air18n functionality. 1) Finds the translations for specified key and locale, falling back to
appropriate locales if necessary based on I18n.fallbacks.
2) Queues up screenshot-taking jobs if we don’t have a screenshot for a
(key, options[:routes_context])
3) Populates the phrases database if key and options aren’t
already in there.
4) Updates the phrases database if options doesn’t match the
English text in the database. Alternatively, you can use
options[:default_is_low_priority] if you don't want this behavior.
5) Makes the :xx locale translations. 6) Asks PrimAndProper to make en-GB translations and other best-guess
translations if appropriate.
7) Chooses the correct translation form based on options if present.
172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 |
# File 'lib/air18n/backend.rb', line 172 def lookup(locale, key, scope = [], = {}) # Useful i18n logging for debugging translation lookup problems. # LoggingHelper.info "Lookup! key is #{key.inspect}, options are #{options.inspect}" # caller.each { |l| LoggingHelper.info " " + l } # Sometimes translate() is called with an array of keys. We don't handle that case. if key.blank? || !key.is_a?(String) return nil end # Force locale to symbol to allow e.g. # I18n.t('foo', :default => 'Foo', :locale => @current_user.preferred_locale) locale = locale.to_sym default = if [:default] && [:default].is_a?(String) [:default] else nil end overrides_previous_default = ![:default_is_low_priority] I18n.fallbacks_for(locale).each do |l| init_translations(l) end # Only create new screenshots while using default locale, and ignore keys # that come in hash format or with a wacky namespace if [:routes_context] && locale == I18n.default_locale # Check to see if we have screenshot for this phrase/routes context combo unless @phrase_screenshots[key] && (@phrase_screenshots[key].include?([:routes_context]) || @phrase_screenshots[key].size >= APPROX_MAX_SCREENSHOTS_PER_KEY) I18n.phrase_needs_screenshot([:routes_context], {key => @translation_data[I18n.default_locale][key] || default || key}) @phrase_screenshots[key] = (@phrase_screenshots[key] || []) << [:routes_context] end end fallback_chain = I18n.fallbacks_for(locale) result = nil locale_fallen_back_to = nil for fallback_locale in fallback_chain if @translation_data.include?(fallback_locale) result = @translation_data[fallback_locale][key] locale_fallen_back_to = fallback_locale break if result end end if result && locale_fallen_back_to != locale # See if we can guess a translation instead of using the fallback # translation. guess = guess_translation(result, locale_fallen_back_to, locale) result = guess if guess end # If there was a default-language translation, check if it matches the default. # If the phrase is not in the phrases table, it will be created. if locale == I18n.default_locale && default != nil && default != "" && result != default # If it doesn't, we need to update the 'phrases' table so that # the 'value' column reflects the latest English default text. store_default = true if result.present? if @default_text_change_observer.present? @default_text_change_observer.default_text_changed(locale, key, result, default) store_default = @default_text_change_observer.allow_default_text_change?(locale, key, result, default) end end if store_default phrase = Phrase.where(:key => key).first if phrase.present? # A phrase with this key already exists; change its default text. if (overrides_previous_default && phrase.value != default) || (!overrides_previous_default && phrase.value != default && phrase.value.blank?) phrase.value = default begin phrase.save rescue Exception => e # If many requests happen simultaneously for a page with a new # phrase, a "duplicate entry" exception will happen naturally. # And if phrase creation fails for some other reason, we will try # again to create it next time automatically. end end else # Create the phrase for the first time. p = Phrase.create do |p| p.key = key p.value = default end end @translation_data[locale][key] = default end result = default end # Helps debug translation loading and default setting. # LoggingHelper.info "Airbnb Backend looking up key #{key.inspect}. result is #{result.inspect}. Default is #{default.inspect}" # if a default is given, use it here if result.nil? && default result = default end unless [:disable_xss_check] xss_detection = XssDetector::safe?(default, result, I18n.default_locale, locale_fallen_back_to) if !xss_detection[:safe] # Kill the translation if the result is unsafe. LoggingHelper.error "Killing unsafe translation! Default is #{default.inspect}, result is #{result.inspect}, reason for kill is #{xss_detection[:result]}" result = default end end # Strip whitespace from both sides. For fun? result = result.strip if result.present? # Handle pseudo-locales. result = PseudoLocales.translate(locale, key, result) # Handle smart counts. result = SmartCount.choose(result, locale_fallen_back_to, [SmartCount::INTERPOLATION_VARIABLE_NAME]) result end |
#reload_translations(locales) ⇒ Object
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
# File 'lib/air18n/backend.rb', line 50 def reload_translations(locales) if defined?(STATSD) STATSD.increment("Air18n.backend.reload_translations") end # We don't want to load translations of unused phrases, so we exclude # unused from the set of translations we initialize. translations_hash = PhraseTranslation.translations_for_locales(locales, :exclude_unused => true) # In development, people might not be running the counter server to keep # track of which phrases are used. In this ase, load all phrases, except UGC. if translations_hash.all? { |locale, translations| translations.empty? } translations_hash = PhraseTranslation.translations_for_locales(locales, :exclude_ugc => true) end num_translations = translations_hash.inject(0) { |carry, (locale, translations)| carry + translations.size } LoggingHelper.info "Loaded #{num_translations} translations for locales #{locales.inspect}." number_of_translations = 0 translations_hash.each_pair do |locale, data| store_translations(locale, data) number_of_translations += data.size @translations_last_loaded_at ||= {} @translations_last_loaded_at[locale] = Time.now end LoggingHelper.info "Translation data size: #{Marshal.dump(@translation_data).size}" end |
#rescreenshot(routes_context) ⇒ Object
138 139 140 141 142 143 |
# File 'lib/air18n/backend.rb', line 138 def rescreenshot(routes_context) @phrase_screenshots.each do |key, routes| routes.delete(routes_context) end nil end |
#reset_phrase_screenshots ⇒ Object
86 87 88 |
# File 'lib/air18n/backend.rb', line 86 def reset_phrase_screenshots @phrase_screenshots = PhraseScreenshot.all_phrase_urls end |
#store_translations(locale, data, options = {}) ⇒ Object
Stores translations for a given locale.
30 31 32 33 34 35 36 37 |
# File 'lib/air18n/backend.rb', line 30 def store_translations locale, data, = {} @translation_data ||= {} @translation_data[locale.to_sym] = data # We pass in nil for the expiry because we do not want the translations to # be expired. Rather we prefer to explicitily overwrite them when reload # translations is called. ChunkCache::set(I18n.cache, T_DATA % locale, data, nil) if I18n.cache end |