Class: Rufus::Tokyo::Table

Inherits:
Object
  • Object
show all
Includes:
CabinetConfig, HashMethods, Transactions
Defined in:
lib/rufus/tokyo/cabinet/table.rb

Overview

A ‘table’ a table database.

http://alpha.mixi.co.jp/blog/?p=290
http://tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

A short example :

require 'rubygems'
require 'rufus/tokyo/cabinet/table'

t = Rufus::Tokyo::Table.new('table.tdb', :create, :write)
  # '.tdb' suffix is a must

t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
t['pk1'] = { 'name' => 'bob', 'age' => '18' }
t['pk2'] = { 'name' => 'charly', 'age' => '45' }
t['pk3'] = { 'name' => 'doug', 'age' => '77' }
t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

p t.query { |q|
  q.add_condition 'age', :numge, '32'
  q.order_by 'age'
  q.limit 2
}
  # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
  #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

t.close

Direct Known Subclasses

TyrantTable

Constant Summary collapse

INDEX_TYPES = 
{
  :lexical => 0,
  :decimal => 1,
  :token => 2,
  :qgram => 3,
  :opt => 9998,
  :optimized => 9998,
  :void => 9999,
  :remove => 9999,
  :keep => 1 << 24
}

Instance Attribute Summary

Attributes included from HashMethods

#default_proc

Instance Method Summary collapse

Methods included from Transactions

#abort, #transaction

Methods included from HashMethods

#[], #default, #default=, #each, #merge, #merge!, #to_a, #to_h, #values

Constructor Details

#initialize(path, params = {}) ⇒ Table

Creates a Table instance (creates or opens it depending on the args)

For example,

t = Rufus::Tokyo::Table.new('table.tdb')
  # '.tdb' suffix is a must

will create the table.tdb (or simply open it if already present) and make sure we have write access to it.

parameters

Parameters can be set in the path or via the optional params hash (like in Rufus::Tokyo::Cabinet)

* :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
           'e' non locking, 'f' non blocking lock), default is 'wc'
* :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
           (usually empty or something like 'ld' or 'lb')

* :bnum    number of elements of the bucket array
* :apow    size of record alignment by power of 2 (defaults to 4)
* :fpow    maximum number of elements of the free block pool by
           power of 2 (defaults to 10)
* :mutex   when set to true, makes sure only 1 thread at a time
           accesses the table (well, Ruby, global thread lock, ...)

* :rcnum   specifies the maximum number of records to be cached.
           If it is not more than 0, the record cache is disabled.
           It is disabled by default.
* :lcnum   specifies the maximum number of leaf nodes to be cached.
           If it is not more than 0, the default value is specified.
           The default value is 2048.
* :ncnum   specifies the maximum number of non-leaf nodes to be
           cached. If it is not more than 0, the default value is
           specified. The default value is 512.

* :xmsiz   specifies the size of the extra mapped memory. If it is
           not more than 0, the extra mapped memory is disabled.
           The default size is 67108864.

* :dfunit  unit step number. If it is not more than 0,
           the auto defragmentation is disabled. (Since TC 1.4.21)

Some examples :

t = Rufus::Tokyo::Table.new('table.tdb')
t = Rufus::Tokyo::Table.new('table.tdb#mode=r')
t = Rufus::Tokyo::Table.new('table.tdb', :mode => 'r')
t = Rufus::Tokyo::Table.new('table.tdb#opts=ld#mode=r')
t = Rufus::Tokyo::Table.new('table.tdb', :opts => 'ld', :mode => 'r')



125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 125

def initialize (path, params={})

  conf = determine_conf(path, params, :table)

  @db = lib.tctdbnew

  #
  # tune table

  libcall(:tctdbsetmutex) if conf[:mutex]

  libcall(:tctdbtune, conf[:bnum], conf[:apow], conf[:fpow], conf[:opts])

  # TODO : set indexes here... well, there is already #set_index
  #conf[:indexes]...

  libcall(:tctdbsetcache, conf[:rcnum], conf[:lcnum], conf[:ncnum])

  libcall(:tctdbsetxmsiz, conf[:xmsiz])

  libcall(:tctdbsetdfunit, conf[:dfunit]) \
    if lib.respond_to?(:tctdbsetdfunit) # TC >= 1.4.21

  #
  # open table

  @path = conf[:path]

  libcall(:tctdbopen, @path, conf[:mode])
end

Instance Method Details

#[]=(pk, h_or_a) ⇒ Object

Inserts a record in the table db

table['pk0'] = [ 'name', 'fred', 'age', '45' ]
table['pk1'] = { 'name' => 'jeff', 'age' => '46' }

Accepts both a hash or an array (expects the array to be of the form [ key, value, key, value, … ] else it will raise an ArgumentError)

Raises an error in case of failure.



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 240

def []= (pk, h_or_a)

  pk = pk.to_s
  h_or_a = Rufus::Tokyo.h_or_a_to_s(h_or_a)

  m = Rufus::Tokyo::Map[h_or_a]

  r = lib.tab_put(@db, pk, Rufus::Tokyo.blen(pk), m.pointer)

  m.free

  (r == 1) || raise_error # raising potential error after freeing map

  h_or_a
end

#clear ⇒ Object

Removes all records in this table database



274
275
276
277
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 274

def clear

  libcall(:tab_vanish)
end

#close ⇒ Object

Closes the table (and frees the datastructure allocated for it), returns true in case of success.



173
174
175
176
177
178
179
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 173

def close

  result = lib.tab_close(@db)
  lib.tab_del(@db)

  (result == 1)
end

#delete(k) ⇒ Object

Removes an entry in the table

(might raise an error if the delete itself failed, but returns nil if there was no entry for the given key)



261
262
263
264
265
266
267
268
269
270
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 261

def delete (k)

  k = k.to_s

  v = self[k]
  return nil unless v
  libcall(:tab_out, k, Rufus::Tokyo.blen(k))

  v
end

#delete_keys_with_prefix(prefix) ⇒ Object

Deletes all the entries whose key begin with the given prefix.



333
334
335
336
337
338
339
340
341
342
343
344
345
346
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 333

def delete_keys_with_prefix (prefix)

  # TODO : use ...searchout

  ks = lib.tab_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1)
    # -1 for no limit

  begin
    ks = Rufus::Tokyo::List.new(ks)
    ks.each { |k| self.delete(k) }
  ensure
    ks && ks.free
  end
end

#difference(*queries) ⇒ Object

Returns the difference of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second but not both.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



512
513
514
515
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 512

def difference (*queries)

  search(:difference, *queries)
end

#do_query(&block) ⇒ Object

Prepares and runs a query, returns a ResultSet instance (takes care of freeing the query structure)



379
380
381
382
383
384
385
386
387
388
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 379

def do_query (&block)

  q = prepare_query(&block)
  rs = q.run

  return rs

ensure
  q && q.free
end

#generate_unique_id ⇒ Object Also known as: genuid

Generates a unique id (in the context of this Table instance)



183
184
185
186
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 183

def generate_unique_id

  lib.tab_genuid(@db)
end

#intersection(*queries) ⇒ Object

Returns the intersection of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query AND the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



490
491
492
493
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 490

def intersection (*queries)

  search(:intersection, *queries)
end

#keys(options = {}) ⇒ Object

Returns an array of all the primary keys in the table

With no options given, this method will return all the keys (strings) in a Ruby array.

:prefix --> returns only the keys who match a given string prefix

:limit --> returns a limited number of keys

:native --> returns an instance of Rufus::Tokyo::List instead of
  a Ruby Hash, you have to call #free on that List when done with it !
  Else you're exposing yourself to a memory leak.



292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 292

def keys (options={})

  outlen = nil

  if pre = options[:prefix]

    l = lib.tab_fwmkeys(
      @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

    l = Rufus::Tokyo::List.new(l)

    options[:native] ? l : l.release

  else

    limit = options[:limit] || -1
    limit = nil if limit < 1

    l = options[:native] ? Rufus::Tokyo::List.new : []

    lib.tab_iterinit(@db)

    outlen = FFI::MemoryPointer.new(:int)

    loop do
      break if limit and l.size >= limit
      out = lib.tab_iternext(@db, outlen)
      break if out.address == 0
      l << out.get_bytes(0, outlen.get_int(0))
    end

    l
  end

ensure

  outlen && outlen.free
end

#lget(keys) ⇒ Object Also known as: mget

No ‘misc’ methods for the table library, so this lget is equivalent to calling get for each key. Hoping later versions of TC will provide a mget method.



352
353
354
355
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 352

def lget (keys)

  keys.inject({}) { |h, k| k = k.to_s; v = self[k]; h[k] = v if v; h }
end

#lib ⇒ Object

Using the cabinet lib



158
159
160
161
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 158

def lib

  CabinetLib
end

#path ⇒ Object

Returns the path to the table.



165
166
167
168
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 165

def path

  @path
end

#pointer ⇒ Object

Returns the actual pointer to the Tokyo Cabinet table



446
447
448
449
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 446

def pointer

  @db
end

#prepare_query(&block) ⇒ Object

Prepares a query instance (block is optional)



368
369
370
371
372
373
374
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 368

def prepare_query (&block)

  q = TableQuery.new(self)
  block.call(q) if block

  q
end

#query(&block) ⇒ Object

Prepares and runs a query, returns an array of hashes (all Ruby) (takes care of freeing the query and the result set structures)



393
394
395
396
397
398
399
400
401
402
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 393

def query (&block)

  rs = do_query(&block)
  a = rs.to_a

  return a

ensure
  rs && rs.free
end

#query_delete(&block) ⇒ Object

Prepares a query and then runs it and deletes all the results.



406
407
408
409
410
411
412
413
414
415
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 406

def query_delete (&block)

  q = prepare_query(&block)
  rs = q.delete

  return rs

ensure
  q && q.free
end

#search(type, *queries) ⇒ Object

A #search a la ruby-tokyotyrant (github.com/actsasflinn/ruby-tokyotyrant/tree)

r = table.search(
  :intersection,
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

Accepts the symbols :union, :intersection, :difference or :diff as first parameter.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

Raises:

  • (ArgumentError) 


536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 536

def search (type, *queries)

  run_query = true
  run_query = queries.pop if queries.last == false

  raise(
    ArgumentError.new("pass at least one prepared query")
  ) if queries.size < 1

  raise(
    ArgumentError.new("pass instances of Rufus::Tokyo::TableQuery only")
  ) if queries.find { |q| q.class != TableQuery }

  t = META_TYPES[type]

  raise(
    ArgumentError.new("no search type #{type.inspect}")
  ) unless t

  qs = FFI::MemoryPointer.new(:pointer, queries.size)
  qs.write_array_of_pointer(queries.collect { |q| q.pointer })

  r = lib.tab_metasearch(qs, queries.size, t)

  qs.free

  pks = Rufus::Tokyo::List.new(r).release

  run_query ? lget(pks) : pks
end

#set_index(column_name, *types) ⇒ Object

Sets an index on a column of the table.

Types maybe be :lexical or :decimal.

Recently (TC 1.4.26 and 1.4.27) inverted indexes have been added, they are :token and :qgram. There is an :opt index as well.

Sorry couldn’t find any good doc about those inverted indexes apart from :

http://alpha.mixi.co.jp/blog/?p=1147
http://www.excite-webtl.jp/world/english/web/?wb_url=http%3A%2F%2Falpha.mixi.co.jp%2Fblog%2F%3Fp%3D1147&wb_lp=JAEN&wb_dis=2&wb_submit=+%96%7C+%96%F3+

Use :keep to “add” and :remove (or :void) to “remove” an index.

If column_name is :pk or “”, the index will be set on the primary key.

Returns true in case of success.



220
221
222
223
224
225
226
227
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 220

def set_index (column_name, *types)

  column_name = column_name == :pk ? '' : column_name.to_s

  i = types.inject(0) { |i, t| i = i | INDEX_TYPES[t]; i }

  (lib.tab_setindex(@db, column_name, i) == 1)
end

#size ⇒ Object

Returns the number of records in this table db



361
362
363
364
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 361

def size

  lib.tab_rnum(@db)
end

#tranabort ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.



440
441
442
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 440

def tranabort
  libcall(:tctdbtranabort)
end

#tranbegin ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.



422
423
424
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 422

def tranbegin
  libcall(:tctdbtranbegin)
end

#trancommit ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.



431
432
433
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 431

def trancommit
  libcall(:tctdbtrancommit)
end

#union(*queries) ⇒ Object

Returns the union of the listed queries

r = table.union(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



468
469
470
471
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 468

def union (*queries)

  search(:union, *queries)
end