Class: Token

Inherits:
Object
  • Object
show all
Defined in:
lib/tsql_shparser/tsql_tokenizer.rb

Overview

Defines the Token type. A token is an array of three elements (triplet):

line_number, column_number, token_string

Some tokens may span over multiple lines (e.g. quoted strings), in which case the line number and column number is that of the end of the token.

Constant Summary collapse

VERSION = 
"0.0.1"
DATA_TYPES = 
%w{
  SQL_VARIANT DATETIME    SMALLDATETIME    FLOAT    REAL    DECIMAL    MONEY
  SMALLMONEY    BIGINT    INT    SMALLINT    TINYINT    BIT    NTEXT    TEXT
  IMAGE    TIMESTAMP    UNIQUEIDENTIFIER     NVARCHAR    NCHAR    VARCHAR    CHAR
  VARBINARY BINARY
}
KEY_WORDS = 
%w{
  ADD EXCEPT PERCENT 
  ALL EXEC PLAN 
  ALTER EXECUTE PRECISION 
  AND EXISTS PRIMARY 
  ANY EXIT PRINT 
  AS FETCH PROC 
  ASC FILE PROCEDURE 
  AUTHORIZATION FILLFACTOR PUBLIC 
  BACKUP FOR RAISERROR 
  BEGIN FOREIGN READ 
  BETWEEN FREETEXT READTEXT 
  BREAK FREETEXTTABLE RECONFIGURE 
  BROWSE FROM REFERENCES 
  BULK FULL REPLICATION 
  BY FUNCTION RESTORE 
  CASCADE GOTO RESTRICT 
  CASE GRANT RETURN 
  CHECK GROUP REVOKE 
  CHECKPOINT HAVING RIGHT 
  CLOSE HOLDLOCK ROLLBACK 
  CLUSTERED IDENTITY ROWCOUNT 
  COALESCE IDENTITY_INSERT ROWGUIDCOL 
  COLLATE IDENTITYCOL RULE 
  COLUMN IF SAVE 
  COMMIT IN SCHEMA 
  COMPUTE INDEX SELECT 
  CONSTRAINT INNER SESSION_USER 
  CONTAINS INSERT SET 
  CONTAINSTABLE INTERSECT SETUSER 
  CONTINUE INTO SHUTDOWN 
  CONVERT IS SOME 
  CREATE JOIN STATISTICS 
  CROSS KEY SYSTEM_USER 
  CURRENT KILL TABLE 
  CURRENT_DATE LEFT TEXTSIZE 
  CURRENT_TIME LIKE THEN TIES
  CURRENT_TIMESTAMP LINENO TO 
  CURRENT_USER LOAD TOP 
  CURSOR NATIONAL  TRAN 
  DATABASE NOCHECK TRANSACTION 
  DBCC NONCLUSTERED TRIGGER 
  DEALLOCATE NOT TRUNCATE 
  DECLARE NULL TSEQUAL 
  DEFAULT NULLIF UNION 
  DELETE OF UNIQUE 
  DENY OFF UPDATE 
  DESC OFFSETS UPDATETEXT 
  DISK ON USE 
  DISTINCT OPEN USER 
  DISTRIBUTED OPENDATASOURCE VALUES 
  DOUBLE OPENQUERY VARYING 
  DROP OPENROWSET VIEW 
  DUMMY OPENXML WAITFOR 
  DUMP OPTION WHEN 
  ELSE OR WHERE 
  END ORDER WHILE 
  ERRLVL OUTER WITH 
  ESCAPE OVER WRITETEXT 
}

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(line, col, val) ⇒ Token

Returns a new instance of Token.



92
93
94
95
96
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 92

def initialize(line,col,val)
  @line = line
  @col  = col 
  @token_value = val
end

Instance Attribute Details

#col ⇒ Object (readonly)

Returns the value of attribute col.



89
90
91
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 89

def col
  @col
end

#line ⇒ Object (readonly)

Returns the value of attribute line.



88
89
90
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 88

def line
  @line
end

#token_value ⇒ Object (readonly)

Returns the value of attribute token_value.



90
91
92
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 90

def token_value
  @token_value
end

Instance Method Details

#==(tok) ⇒ Object 



116
117
118
119
120
121
122
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 116

def ==(tok)
  return nil unless tok

  (self.line == tok.line) && 
  (self.col == tok.col) && 
  (self.token_value == tok.token_value)
end

#token_type ⇒ Object 



98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
 
# File 'lib/tsql_shparser/tsql_tokenizer.rb', line 98

def token_type
  case 
 #     when @token_value =~ /^#.*/ then :TempTable
    when @token_value =~ /^[&|~^\/<>+%*!=-]/ then :Operator 
    when @token_value =~ /^@.*/            then :HostVariable
    when @token_value =~ /^[.]$/           then :Dot
    when @token_value =~ /^[()]$/          then :Paran
    when @token_value =~ /^[;]$/           then :SemiColon      
    when @token_value =~ /^[,]$/           then :Comma
    when @token_value =~ /.+:$/            then :Label
    when @token_value =~ /^N?'/            then :String
    when @token_value =~ /^([$])?\d+(\.\d+)?$/   then :Number
    when DATA_TYPES.include?(@token_value) then :DataType
    when KEY_WORDS.include?(@token_value)  then :KeyWord
    else :Id
  end
end