Class: Bones::Preprocessor

Inherits:
Common
  • Object
show all
Defined in:
lib/bones/preprocessor.rb

Overview

This is the C99 pre-processor for Bones. It has two tasks:

  • To remove all lines starting with ‘#’ (pre-processor directives).

  • To detect all pragma’s forming algorithm classes from the source.

Attributes:

  • header_code - All the code that was removed by the pre-processor but was not relevant to Bones. This contains for example includes and defines.

  • algorithms - An array of identified algorithms, each of class Bones::Algorithm.

  • target_code - The processed code containing no Bones directives nor other pre-processor directives (such as includes and defines).

Constant Summary collapse

IDENTIFIER =

Denotes the start of an algorithmic species.

'#pragma species'
WHITESPACE =

Regular expression to identify whitespaces (tabs, spaces).

'\s*'
SPECIES_START =

This directive denotes the start of a algorithm. It is based on the IDENTIFIER constant.

IDENTIFIER+' kernel'
SPECIES_END =

This directive denotes the end of a algorithm. It is based on the IDENTIFIER constant.

IDENTIFIER+' endkernel'
SCOP_START =

Start of the scop

'#pragma scop'
SCOP_END =

Enf of the scop

'#pragma endscop'
SYNC =

Synchronise directive.

IDENTIFIER+' sync'
COPYIN =

Copy in directive.

IDENTIFIER+ ' copyin'
COPYOUT =

Copy out directive.

IDENTIFIER+ ' copyout'
REGEXP_PREFIX =

A regular expression captures a prefix in a algorithm (e.g. unordered/multiple).

/^[a-z]+ /
DEFAULT_NAME =

Providing a default name in case a algorithm is not named.

'algorithm'

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from Common

#flatten_hash, #from, #replace_defines, #search_and_replace, #search_and_replace!, #sum, #sum_and_from, #to

Constructor Details

#initialize(source_code, directory, filename, scheduler) ⇒ Preprocessor

This is the method which initializes the preprocessor. Initialization requires the target source code to process, which is then set as the class variable @source_code.



47
48
49
50
51
52
53
54
55
56
57
58
59
 
# File 'lib/bones/preprocessor.rb', line 47

def initialize(source_code,directory,filename,scheduler)
	@source_code = source_code
	@target_code = []
	@header_code = ''
	@device_header = ''
	@directory = directory
	@filename = filename
	@algorithms = Array.new
	@copies = Array.new
	@defines = {}
	@found_algorithms = 0
	@scheduler = scheduler
end

Instance Attribute Details

#algorithms ⇒ Object (readonly)

Returns the value of attribute algorithms.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def algorithms
  @algorithms
end

#copies ⇒ Object (readonly)

Returns the value of attribute copies.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def copies
  @copies
end

#defines ⇒ Object (readonly)

Returns the value of attribute defines.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def defines
  @defines
end

#device_header ⇒ Object (readonly)

Returns the value of attribute device_header.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def device_header
  @device_header
end

#header_code ⇒ Object (readonly)

Returns the value of attribute header_code.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def header_code
  @header_code
end

#scop ⇒ Object (readonly)

Returns the value of attribute scop.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def scop
  @scop
end

#target_code ⇒ Object (readonly)

Returns the value of attribute target_code.



12
13
14
 
# File 'lib/bones/preprocessor.rb', line 12

def target_code
  @target_code
end

Instance Method Details

#process ⇒ Object

This is the method to perform the actual preprocessing. This method takes care of all the pre-processor tasks. The output is stored in the three attributes header_code, algorithms, and target_code.



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
 
# File 'lib/bones/preprocessor.rb', line 65

def process
	algorithm_code = ''
	species = nil
	found = 0
	alloc_index, free_index = 0, 0
	block_comment = false
	a_scop_was_found = false
	scop = 0
	scop_copies = []
	
	# Process the file line by line
	@source_code.each_line.with_index do |line,index|

		# Don't consider one-line comments
		if !(line =~ /^#{WHITESPACE}\/\//)

			# Found the start of a block comment
			if line =~ /\/\*/
				block_comment = true
			end

			# Search for the end of the block comment
			if block_comment
				if line =~ /\*\//
					block_comment = false
				end
				@target_code << line

			# Not in a block-comment
			else

				if line =~ /^#{WHITESPACE}#/
					
					# Keep 'include' statements as header code
					if line =~ /^#{WHITESPACE}#include/
						@header_code += line
						if line =~ /"(.*)"/
							process_header($1)
						end
					
					# Process 'define' statements for the algorithm code, but also keep as header code
					elsif line =~ /^#{WHITESPACE}#define/
						@header_code += line
						@device_header += line
						match = line.split(/\/\//)[0].scan(/^#{WHITESPACE}#define\s+(\w+)\s+(\S*)/)
						@defines[match.first[0].to_sym] = match.first[1]
					
					# Found the start of algorithm marker
					elsif line =~ /^#{WHITESPACE}#{SPECIES_START}/
						if found == 0
							line = replace_defines(line,@defines)
							prefix, input, output = marker_to_algorithm(line)
							puts MESSAGE+'Found algorithm "'+(prefix+' '+input+' '+ARROW+' '+output).lstrip+'"' if VERBOSE
							species = Bones::Species.new(prefix,input,output)
							@found_algorithms = @found_algorithms + 1
						end
						found = found + 1
						#@target_code << "int bones_temp_species_start = '#{line.gsub(NL,'')}';"+NL
					
					# Found the end of algorithm marker
					elsif line =~ /^#{WHITESPACE}#{SPECIES_END}/
						if found == 1
							name = line.strip.scan(/^#{WHITESPACE}#{SPECIES_END} (.+)/).join
							name = DEFAULT_NAME if name == ''
							@algorithms.push(Bones::Algorithm.new(name,@filename,index.to_s,species,algorithm_code))
							algorithm_code = ''
						end
						found = found - 1
						#@target_code << "int bones_temp_species_end = '#{line.gsub(NL,'')}';"+NL
						
					# Found a sync marker
					elsif @scheduler && line =~ /^#{WHITESPACE}#{SYNC}/
						sync = line.strip.scan(/^#{WHITESPACE}#{SYNC} (.+)/).join
						@target_code << "bones_synchronize(#{sync});"+NL
						
					# Found a copyin marker
					elsif @scheduler && line =~ /^#{WHITESPACE}#{COPYIN}/
						copies = line.strip.scan(/^#{WHITESPACE}#{COPYIN} (.+)/).join.split(WEDGE).map{ |c| c.strip }
						copies.each_with_index do |copy,copynum|
							name = copy.split('[').first
							domain = copy.scan(/\[(.+)\]/).join.split(DIM_SEP)
							deadline = copy.split('|').last
							@copies.push(Bones::Copy.new(scop,name,domain,deadline,'in',"#{index*100+copynum}"))
							scop_copies.push(@copies[-1])
							@target_code << "bones_copyin_#{index*100+copynum}_#{name}(#{name});"+NL
						end
						
					# Found a copyout marker
					elsif @scheduler && line =~ /^#{WHITESPACE}#{COPYOUT}/
						copies = line.strip.scan(/^#{WHITESPACE}#{COPYOUT} (.+)/).join.split(WEDGE).map{ |c| c.strip }
						copies.each_with_index do |copy,copynum|
							name = copy.split('[').first
							domain = copy.scan(/\[(.+)\]/).join.split(DIM_SEP)
							deadline = copy.split('|').last
							@copies.push(Bones::Copy.new(scop,name,domain,deadline,'out',"#{index*100+copynum}"))
							scop_copies.push(@copies[-1])
							@target_code << "bones_copyout_#{index*100+copynum}_#{name}(#{name});"+NL
						end
					end
					
					# Check if it was a 'pragma scop' / 'pragma endscop' line
					if line =~ /^#{WHITESPACE}#{SCOP_START}/
						scop += 1
						scop_copies = []
						alloc_index = @target_code.length
						a_scop_was_found = true
						@target_code << 'bones_timer_start();'+NL
					elsif line =~ /^#{WHITESPACE}#{SCOP_END}/
						free_index = @target_code.length
						@target_code << 'bones_timer_stop();'+NL
						
						# Add frees and mallocs
						if @scheduler
							alloc_code, free_code = '', ''
							included_copies = []
							scop_copies.each do |copy|
								if !included_copies.include?(copy.name)
									alloc_code += copy.get_function_call('alloc')+NL
									free_code += copy.get_function_call('free')+NL
									included_copies << copy.name
								end
							end
							@target_code.insert(alloc_index, alloc_code)
							@target_code << free_code
						end
					end
					
				else
					if found > 0
						algorithm_line = replace_defines(line,@defines)
						@target_code << algorithm_line
						algorithm_code += algorithm_line if line !~ /^#{WHITESPACE}#/
					else
						@target_code << line
					end
				end
			end
		else
			@target_code << line
		end
	end
	puts WARNING+'Begin/end kernel mismatch ('+@found_algorithms.to_s+' versus '+@algorithms.length.to_s+'), probably missing a "'+SPECIES_END+'"' unless @algorithms.length == @found_algorithms
	
	# Print warning if there is no SCoP found
	if !a_scop_was_found
		puts WARNING+'No "#pragma scop" and "#pragma endscop" found!'
	end
	
	# Join the array
	@target_code = @target_code.join('')
end

#process_header(filename) ⇒ Object

This is the method to preprocess a header file. Currently, it only searches for defines and adds those to a list. In the meanwhile, it also handles ifdef’s.



220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
 
# File 'lib/bones/preprocessor.rb', line 220

def process_header(filename)
	ifdefs = [true]
	
	# Process the file line by line
	block_comment = false
	File.read(File.join(@directory,filename)).each_line.with_index do |line,index|
		
		# Don't consider one-line comments
		if !(line =~ /^#{WHITESPACE}\/\//)

			# Found the start of a block comment
			if line =~ /\/\*/
				block_comment = true
			end

			# Search for the end of the block comment
			if block_comment
				if line =~ /\*\//
					block_comment = false
				end

			# Not in a block-comment
			else
				if line =~ /^#{WHITESPACE}#/
					
					# Process 'include' statements
					if line =~ /^#{WHITESPACE}#include/ && ifdefs.last
						if line =~ /"(.*)"/
							process_header($1)
						end
					
					# Process 'define' statements
					elsif line =~ /^#{WHITESPACE}#define/ && ifdefs.last
						match = line.split(/\/\//)[0].scan(/^#{WHITESPACE}#define\s+(\w+)\s+(\S*)/)
						@defines[match.first[0].to_sym] = match.first[1].strip
					
					# Process 'ifdef' statements
					elsif line =~ /^#{WHITESPACE}#ifdef#{WHITESPACE}(\w+)/
						valid = (ifdefs.last) ? @defines.has_key?($1.to_sym) : false
						ifdefs.push(valid)
						
					# Process 'endif' statements
					elsif line =~ /^#{WHITESPACE}#endif/
						ifdefs.pop
					end
				end
			end
		end
	end
end