Class: PEROBS::FlatFileDB

Inherits:
DataBase show all
Defined in:
lib/perobs/FlatFileDB.rb

Overview

The FlatFileDB is a storage backend that uses a single flat file to store the value blobs.

Constant Summary collapse

VERSION =

This version number increases whenever the on-disk format changes in a way that requires conversion actions after an update.

4

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from DataBase

#check_option, #deserialize, #serialize

Constructor Details

#initialize(db_name, options = {}) ⇒ FlatFileDB

Create a new FlatFileDB object.

Parameters:

  • db_name (String) —

    name of the DB directory

  • options (Hash) (defaults to: {}) —

    options to customize the behavior. Currently only the following options are supported: :serializer : Can be :marshal, :json, :yaml :progressmeter : Reference to a ProgressMeter object :log : IO that should be used for logging :log_level : Minimum Logger level to log



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
 
# File 'lib/perobs/FlatFileDB.rb', line 57

def initialize(db_name, options = {})
  super(options)

  @db_dir = db_name
  # Create the database directory if it doesn't exist yet.
  ensure_dir_exists(@db_dir)
  PEROBS.log.level = options[:log_level] if options[:log_level]
  PEROBS.log.open(options[:log] || File.join(@db_dir, 'log'))
  check_version_and_upgrade

  # Read the existing DB config.
  @config = get_hash('config')
  check_option('serializer')

  put_hash('config', @config)
end

Instance Attribute Details

#max_blob_size ⇒ Object (readonly)

Returns the value of attribute max_blob_size.



47
48
49
 
# File 'lib/perobs/FlatFileDB.rb', line 47

def max_blob_size
  @max_blob_size
end

Class Method Details

.delete_db(db_name) ⇒ Object 



94
95
96
97
 
# File 'lib/perobs/FlatFileDB.rb', line 94

def FlatFileDB::delete_db(db_name)
  close
  FileUtils.rm_rf(db_name)
end

Instance Method Details

#check(id, repair) ⇒ TrueClass/FalseClass

Check if the stored object is syntactically correct.

Parameters:

  • id (Integer) —

    Object ID

  • repair (TrueClass/FalseClass) —

    True if an repair attempt should be made.

Returns:

  • (TrueClass/FalseClass) —

    True if the object is OK, otherwise false.



196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'lib/perobs/FlatFileDB.rb', line 196

def check(id, repair)
  begin
    return get_object(id) != nil
  rescue PEROBS::FatalError => e
    PEROBS.log.error "Cannot read object with ID #{id}: #{e.message}"
    if repair
      begin
        PEROBS.log.error "Discarding broken object with ID #{id}"
        @flat_file.delete_obj_by_id(id)
      rescue PEROBS::FatalError
      end
    end
  end

  return false
end

#check_db(repair = false) ⇒ Object

Basic consistency check.

Parameters:

  • repair (TrueClass/FalseClass) (defaults to: false) —

    True if found errors should be repaired.

Returns:

  • number of errors found



186
187
188
 
# File 'lib/perobs/FlatFileDB.rb', line 186

def check_db(repair = false)
  @flat_file.check(repair)
end

#clear_marks ⇒ Object

This method must be called to initiate the marking process.



157
158
159
 
# File 'lib/perobs/FlatFileDB.rb', line 157

def clear_marks
  @flat_file.clear_all_marks
end

#close ⇒ Object

Close the FlatFileDB.



82
83
84
85
86
 
# File 'lib/perobs/FlatFileDB.rb', line 82

def close
  @flat_file.close
  @flat_file = nil
  PEROBS.log.info "FlatFile '#{@db_dir}' closed"
end

#delete_database ⇒ Object

Delete the entire database. The database is no longer usable after this method was called.



90
91
92
 
# File 'lib/perobs/FlatFileDB.rb', line 90

def delete_database
  FileUtils.rm_rf(@db_dir)
end

#delete_unmarked_objects ⇒ Integer

Permanently delete all objects that have not been marked. Those are orphaned and are no longer referenced by any actively used object.

Returns:

  • (Integer) —

    Number of the removed objects from the DB.



164
165
166
 
# File 'lib/perobs/FlatFileDB.rb', line 164

def delete_unmarked_objects
  @flat_file.delete_unmarked_objects
end

#get_hash(name) ⇒ Hash

Load the Hash with the given name.

Parameters:

  • name (String) —

    Name of the hash.

Returns:

  • (Hash) —

    A Hash that maps String objects to strings or numbers.



121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/perobs/FlatFileDB.rb', line 121

def get_hash(name)
  file_name = File.join(@db_dir, name + '.json')
  return ::Hash.new unless File.exist?(file_name)

  begin
    json = File.read(file_name)
  rescue => e
    PEROBS.log.fatal "Cannot read hash file '#{file_name}': #{e.message}"
  end
  JSON.parse(json, :create_additions => true)
end

#get_object(id) ⇒ Hash

Load the given object from the filesystem.

Parameters:

  • id (Integer) —

    object ID

Returns:

  • (Hash) —

    Object as defined by PEROBS::ObjectBase or nil if ID does not exist



143
144
145
146
147
148
149
 
# File 'lib/perobs/FlatFileDB.rb', line 143

def get_object(id)
  if (raw_obj = @flat_file.read_obj_by_id(id))
    return deserialize(raw_obj)
  else
    nil
  end
end

#include?(id) ⇒ Boolean

Return true if the object with given ID exists

Parameters:

  • id (Integer)

Returns:

  • (Boolean) 


101
102
103
 
# File 'lib/perobs/FlatFileDB.rb', line 101

def include?(id)
  !@flat_file.find_obj_addr_by_id(id).nil?
end

#is_marked?(id, ignore_errors = false) ⇒ Boolean

Check if the object is marked.

Parameters:

  • id (Integer) —

    ID of the object to check

  • ignore_errors (Boolean) (defaults to: false) —

    If set to true no errors will be raised for non-existing objects.

Returns:

  • (Boolean) 


178
179
180
 
# File 'lib/perobs/FlatFileDB.rb', line 178

def is_marked?(id, ignore_errors = false)
  @flat_file.is_marked_by_id?(id)
end

#item_counter ⇒ Integer

Returns Number of objects stored in the DB.

Returns:

  • (Integer) —

    Number of objects stored in the DB.



152
153
154
 
# File 'lib/perobs/FlatFileDB.rb', line 152

def item_counter
  @flat_file.item_counter
end

#mark(id) ⇒ Object

Mark an object.

Parameters:

  • id (Integer) —

    ID of the object to mark



170
171
172
 
# File 'lib/perobs/FlatFileDB.rb', line 170

def mark(id)
  @flat_file.mark_obj_by_id(id)
end

#open ⇒ Object

Open the FlatFileDB for transactions.



75
76
77
78
79
 
# File 'lib/perobs/FlatFileDB.rb', line 75

def open
  @flat_file = FlatFile.new(@db_dir, @progressmeter)
  @flat_file.open
  PEROBS.log.info "FlatFile '#{@db_dir}' opened"
end

#put_hash(name, hash) ⇒ Object

Store a simple Hash as a JSON encoded file into the DB directory. numbers.

Parameters:

  • name (String) —

    Name of the hash. Will be used as file name.

  • hash (Hash) —

    A Hash that maps String objects to strings or



109
110
111
112
113
114
115
116
 
# File 'lib/perobs/FlatFileDB.rb', line 109

def put_hash(name, hash)
  file_name = File.join(@db_dir, name + '.json')
  begin
    RobustFile.write(file_name, hash.to_json)
  rescue IOError => e
    PEROBS.log.fatal "Cannot write hash file '#{file_name}': #{e.message}"
  end
end

#put_object(obj, id) ⇒ Object

Store the given object into the cluster files.

Parameters:

  • obj (Hash) —

    Object as defined by PEROBS::ObjectBase



135
136
137
 
# File 'lib/perobs/FlatFileDB.rb', line 135

def put_object(obj, id)
  @flat_file.write_obj_by_id(id, serialize(obj))
end

#put_raw_object(raw, id) ⇒ Object

Store the given serialized object into the cluster files. This method is for internal use only!

Parameters:

  • raw (String) —

    Serialized Object as defined by PEROBS::ObjectBase

  • id (Integer) —

    Object ID



217
218
219
 
# File 'lib/perobs/FlatFileDB.rb', line 217

def put_raw_object(raw, id)
  @flat_file.write_obj_by_id(id, raw)
end