Class: Rufus::Tokyo::Table

Inherits:
Object
  • Object
show all
Extended by:
Openable
Includes:
CabinetConfig, HashMethods, Transactions
Defined in:
lib/rufus/tokyo/cabinet/table.rb

Overview

A ‘table’ a table database.

http://alpha.mixi.co.jp/blog/?p=290
http://tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

A short example :

require 'rubygems'
require 'rufus/tokyo/cabinet/table'

t = Rufus::Tokyo::Table.new('table.tct')

t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
t['pk1'] = { 'name' => 'bob', 'age' => '18' }
t['pk2'] = { 'name' => 'charly', 'age' => '45' }
t['pk3'] = { 'name' => 'doug', 'age' => '77' }
t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

p t.query { |q|
  q.add_condition 'age', :numge, '32'
  q.order_by 'age'
  q.limit 2
}
  # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
  #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

t.close

Direct Known Subclasses

TyrantTable

Constant Summary collapse

INDEX_TYPES = 
{
  :lexical => 0,
  :decimal => 1,
  :token => 2,
  :qgram => 3,
  :opt => 9998,
  :optimized => 9998,
  :void => 9999,
  :remove => 9999,
  :keep => 1 << 24
}

Instance Attribute Summary

Attributes included from HashMethods

#default_proc

Instance Method Summary collapse

Methods included from Openable

open

Methods included from Transactions

#abort, #transaction

Methods included from HashMethods

#[], #default, #default=, #each, #merge, #merge!, #to_a, #to_h, #values

Constructor Details

#initialize(path, params = {}) ⇒ Table

Creates a Table instance (creates or opens it depending on the args)

For example,

t = Rufus::Tokyo::Table.new('table.tct')
  # '.tct' suffix is a must

will create the table.tct (or simply open it if already present) and make sure we have write access to it.

parameters

Parameters can be set in the path or via the optional params hash (like in Rufus::Tokyo::Cabinet)

* :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
           'e' non locking, 'f' non blocking lock), default is 'wc'
* :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
           (usually empty or something like 'ld' or 'lb')

* :bnum    number of elements of the bucket array
* :apow    size of record alignment by power of 2 (defaults to 4)
* :fpow    maximum number of elements of the free block pool by
           power of 2 (defaults to 10)
* :mutex   when set to true, makes sure only 1 thread at a time
           accesses the table (well, Ruby, global thread lock, ...)

* :rcnum   specifies the maximum number of records to be cached.
           If it is not more than 0, the record cache is disabled.
           It is disabled by default.
* :lcnum   specifies the maximum number of leaf nodes to be cached.
           If it is not more than 0, the default value is specified.
           The default value is 2048.
* :ncnum   specifies the maximum number of non-leaf nodes to be
           cached. If it is not more than 0, the default value is
           specified. The default value is 512.

* :xmsiz   specifies the size of the extra mapped memory. If it is
           not more than 0, the extra mapped memory is disabled.
           The default size is 67108864.

* :dfunit  unit step number. If it is not more than 0,
           the auto defragmentation is disabled. (Since TC 1.4.21)

Some examples :

t = Rufus::Tokyo::Table.new('table.tct')
t = Rufus::Tokyo::Table.new('table.tct#mode=r')
t = Rufus::Tokyo::Table.new('table.tct', :mode => 'r')
t = Rufus::Tokyo::Table.new('table.tct#opts=ld#mode=r')
t = Rufus::Tokyo::Table.new('table.tct', :opts => 'ld', :mode => 'r')



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 126

def initialize (path, params={})

  conf = determine_conf(path, params, :table)

  @db = lib.tctdbnew

  #
  # tune table

  libcall(:tctdbsetmutex) if conf[:mutex]

  libcall(:tctdbtune, conf[:bnum], conf[:apow], conf[:fpow], conf[:opts])

  # TODO : set indexes here... well, there is already #set_index
  #conf[:indexes]...

  libcall(:tctdbsetcache, conf[:rcnum], conf[:lcnum], conf[:ncnum])

  libcall(:tctdbsetxmsiz, conf[:xmsiz])

  libcall(:tctdbsetdfunit, conf[:dfunit]) \
    if lib.respond_to?(:tctdbsetdfunit) # TC >= 1.4.21

  #
  # open table

  @path = conf[:path]

  libcall(:tctdbopen, @path, conf[:mode])

  #
  # no default

  @default_proc = nil
end

Instance Method Details

#[]=(pk, h_or_a) ⇒ Object

Inserts a record in the table db

table['pk0'] = [ 'name', 'fred', 'age', '45' ]
table['pk1'] = { 'name' => 'jeff', 'age' => '46' }

Accepts both a hash or an array (expects the array to be of the form [ key, value, key, value, … ] else it will raise an ArgumentError)

Raises an error in case of failure.



246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 246

def []= (pk, h_or_a)

  pk = pk.to_s
  h_or_a = Rufus::Tokyo.h_or_a_to_s(h_or_a)

  m = Rufus::Tokyo::Map[h_or_a]

  r = lib.tab_put(@db, pk, Rufus::Tokyo.blen(pk), m.pointer)

  m.free

  r || raise_error # raising potential error after freeing map

  h_or_a
end

#clear ⇒ Object

Removes all records in this table database



280
281
282
283
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 280

def clear

  libcall(:tab_vanish)
end

#close ⇒ Object

Closes the table (and frees the datastructure allocated for it), returns true in case of success.



179
180
181
182
183
184
185
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 179

def close

  result = lib.tab_close(@db)
  lib.tab_del(@db)

  result
end

#delete(k) ⇒ Object

Removes an entry in the table

(might raise an error if the delete itself failed, but returns nil if there was no entry for the given key)



267
268
269
270
271
272
273
274
275
276
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 267

def delete (k)

  k = k.to_s

  v = self[k]
  return nil unless v
  libcall(:tab_out, k, Rufus::Tokyo.blen(k))

  v
end

#delete_keys_with_prefix(prefix) ⇒ Object

Deletes all the entries whose key begin with the given prefix.



312
313
314
315
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 312

def delete_keys_with_prefix (prefix)

  query_delete { |q| q.add('', :strbw, prefix) }
end

#difference(*queries) ⇒ Object

Returns the difference of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second but not both.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



501
502
503
504
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 501

def difference (*queries)

  search(:difference, *queries)
end

#do_query(&block) ⇒ Object

Prepares and runs a query, returns a ResultSet instance (takes care of freeing the query structure)



353
354
355
356
357
358
359
360
361
362
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 353

def do_query (&block)

  q = prepare_query(&block)
  rs = q.run

  return rs

ensure
  q && q.free
end

#generate_unique_id ⇒ Object Also known as: genuid

Generates a unique id (in the context of this Table instance)



189
190
191
192
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 189

def generate_unique_id

  lib.tab_genuid(@db)
end

#intersection(*queries) ⇒ Object

Returns the intersection of the listed queries

r = table.intersection(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query AND the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



479
480
481
482
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 479

def intersection (*queries)

  search(:intersection, *queries)
end

#keys(options = {}) ⇒ Object

Returns an array of all the primary keys in the table

With no options given, this method will return all the keys (strings) in a Ruby array.

:prefix --> returns only the keys who match a given string prefix

:limit --> returns a limited number of keys

:native --> returns an instance of Rufus::Tokyo::List instead of
  a Ruby Hash, you have to call #free on that List when done with it !
  Else you're exposing yourself to a memory leak.



298
299
300
301
302
303
304
305
306
307
308
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 298

def keys (options={})

  pre = options.fetch(:prefix, "")

  l = lib.tab_fwmkeys(
    @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

  l = Rufus::Tokyo::List.new(l)

  options[:native] ? l : l.release
end

#lget(*keys) ⇒ Object Also known as: mget

No ‘misc’ methods for the table library, so this lget is equivalent to calling get for each key. Hoping later versions of TC will provide a mget method.



321
322
323
324
325
326
327
328
329
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 321

def lget (*keys)

  keys.flatten.inject({}) { |h, k|
    k = k.to_s
    v = self[k]
    h[k] = v if v
    h
  }
end

#lib ⇒ Object

Using the cabinet lib



164
165
166
167
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 164

def lib

  CabinetLib
end

#path ⇒ Object

Returns the path to the table.



171
172
173
174
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 171

def path

  @path
end

#pointer ⇒ Object

Returns the actual pointer to the Tokyo Cabinet table



435
436
437
438
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 435

def pointer

  @db
end

#prepare_query(&block) ⇒ Object

Prepares a query instance (block is optional)



342
343
344
345
346
347
348
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 342

def prepare_query (&block)

  q = TableQuery.new(self)
  block.call(q) if block

  q
end

#query(&block) ⇒ Object

Prepares and runs a query, returns an array of hashes (all Ruby) (takes care of freeing the query and the result set structures)



367
368
369
370
371
372
373
374
375
376
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 367

def query (&block)

  rs = do_query(&block)
  a = rs.to_a

  return a

ensure
  rs && rs.free
end

#query_count(&block) ⇒ Object

Prepares a query and then runs it and deletes all the results.



393
394
395
396
397
398
399
400
401
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 393

def query_count (&block)

  q = prepare_query { |q|
    q.pk_only  # improve efficiency, since we have to do the query
  }
  q.count
ensure
  q.free if q
end

#query_delete(&block) ⇒ Object

Prepares a query and then runs it and deletes all the results.



380
381
382
383
384
385
386
387
388
389
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 380

def query_delete (&block)

  q = prepare_query(&block)
  rs = q.delete

  return rs

ensure
  q && q.free
end

#search(type, *queries) ⇒ Object

A #search a la ruby-tokyotyrant (github.com/actsasflinn/ruby-tokyotyrant/tree)

r = table.search(
  :intersection,
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

Accepts the symbols :union, :intersection, :difference or :diff as first parameter.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

Raises:

  • (ArgumentError) 


525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 525

def search (type, *queries)

  run_query = true
  run_query = queries.pop if queries.last == false

  raise(
    ArgumentError.new("pass at least one prepared query")
  ) if queries.size < 1

  raise(
    ArgumentError.new("pass instances of Rufus::Tokyo::TableQuery only")
  ) if queries.find { |q| q.class != TableQuery }

  t = META_TYPES[type]

  raise(
    ArgumentError.new("no search type #{type.inspect}")
  ) unless t

  qs = FFI::MemoryPointer.new(:pointer, queries.size)
  qs.write_array_of_pointer(queries.collect { |q| q.pointer })

  r = lib.tab_metasearch(qs, queries.size, t)

  qs.free

  pks = Rufus::Tokyo::List.new(r).release

  run_query ? lget(pks) : pks
end

#set_index(column_name, *types) ⇒ Object

Sets an index on a column of the table.

Types maybe be :lexical or :decimal.

Recently (TC 1.4.26 and 1.4.27) inverted indexes have been added, they are :token and :qgram. There is an :opt index as well.

Sorry couldn’t find any good doc about those inverted indexes apart from :

http://alpha.mixi.co.jp/blog/?p=1147
http://www.excite-webtl.jp/world/english/web/?wb_url=http%3A%2F%2Falpha.mixi.co.jp%2Fblog%2F%3Fp%3D1147&wb_lp=JAEN&wb_dis=2&wb_submit=+%96%7C+%96%F3+

Use :keep to “add” and :remove (or :void) to “remove” an index.

If column_name is :pk or “”, the index will be set on the primary key.

Returns true in case of success.



226
227
228
229
230
231
232
233
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 226

def set_index (column_name, *types)

  column_name = column_name == :pk ? '' : column_name.to_s

  ii = types.inject(0) { |i, t| i = i | INDEX_TYPES[t]; i }

  lib.tab_setindex(@db, column_name, ii)
end

#size ⇒ Object

Returns the number of records in this table db



335
336
337
338
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 335

def size

  lib.tab_rnum(@db)
end

#tranabort ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.



428
429
430
431
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 428

def tranabort

  libcall(:tctdbtranabort)
end

#tranbegin ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.



408
409
410
411
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 408

def tranbegin

  libcall(:tctdbtranbegin)
end

#trancommit ⇒ Object

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.



418
419
420
421
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 418

def trancommit

  libcall(:tctdbtrancommit)
end

#union(*queries) ⇒ Object

Returns the union of the listed queries

r = table.union(
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'es'
  },
  @t.prepare_query { |q|
    q.add 'lang', :includes, 'li'
  }
)

will return a hash { primary_key => record } of the values matching the first query OR the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.



457
458
459
460
 
# File 'lib/rufus/tokyo/cabinet/table.rb', line 457

def union (*queries)

  search(:union, *queries)
end