Class: MdToNotion::Lexer
- Inherits:
-
Object
- Object
- MdToNotion::Lexer
- Includes:
- Tokens
- Defined in:
- lib/md_to_notion/lexer.rb
Defined Under Namespace
Classes: InvalidTokenSyntaxError
Constant Summary collapse
- ALLOWED_VIDEO_EMBED_URLS =
[ "https://user-images.githubusercontent.com/" ]
Constants included from Tokens
Tokens::BULLET_LIST, Tokens::CODE_BLOCK, Tokens::EMBED_FILE_REGEXES, Tokens::GH_EMBED_FILE, Tokens::HEADING_1, Tokens::HEADING_2, Tokens::HEADING_3, Tokens::IMAGE, Tokens::NUMBERED_LIST, Tokens::QUOTE
Instance Method Summary collapse
-
#initialize(markdown) ⇒ Lexer
constructor
A new instance of Lexer.
- #tokenize ⇒ Object
Methods included from Tokens
#bold, #bullet_list, #code, #code_block, #embeded_file, #heading_1, #heading_2, #heading_3, #image, #italic, #numbered_list, #paragraph, #quote, #strikethrough, #text, #tokenize_rich_text
Constructor Details
#initialize(markdown) ⇒ Lexer
Returns a new instance of Lexer.
15 16 17 18 19 20 |
# File 'lib/md_to_notion/lexer.rb', line 15 def initialize(markdown) @markdown = markdown @tokens = [] @stack = [] @index = 0 end |
Instance Method Details
#tokenize ⇒ Object
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
# File 'lib/md_to_notion/lexer.rb', line 22 def tokenize while @index < @markdown.length begin next_char = @markdown[@index] if next_char == " " @index += 1 next @stack << " " elsif next_char == "#" tokenize_heading elsif next_char == "[" tokenize_link elsif next_char == "!" tokenize_image elsif ALLOWED_VIDEO_EMBED_URLS.join("").include?(peek(41)) elsif next_char == ">" tokenize_quote elsif next_char == "-" tokenize_bullet_list elsif next_char =~ /\d+/ tokenize_numbered_list elsif next_char == "`" && peek(2) == "```" tokenize_block_code elsif next_char == "\n" @index += 1 else tokenize_paragraph end rescue InvalidTokenSyntaxError tokenize_paragraph end @stack = [] end @tokens end |