Class: Rake::Pipeline

Inherits:
Object
  • Object
show all
Defined in:
lib/rake-pipeline.rb,
lib/rake-pipeline/cli.rb,
lib/rake-pipeline/dsl.rb,
lib/rake-pipeline/error.rb,
lib/rake-pipeline/graph.rb,
lib/rake-pipeline/filter.rb,
lib/rake-pipeline/server.rb,
lib/rake-pipeline/matcher.rb,
lib/rake-pipeline/project.rb,
lib/rake-pipeline/railtie.rb,
lib/rake-pipeline/version.rb,
lib/rake-pipeline/manifest.rb,
lib/rake-pipeline/middleware.rb,
lib/rake-pipeline/file_wrapper.rb,
lib/rake-pipeline/manifest_entry.rb,
lib/rake-pipeline/reject_matcher.rb,
lib/rake-pipeline/dsl/project_dsl.rb,
lib/rake-pipeline/sorted_pipeline.rb,
lib/rake-pipeline/dsl/pipeline_dsl.rb,
lib/rake-pipeline/dynamic_file_task.rb,
lib/rake-pipeline/filters/gsub_filter.rb,
lib/rake-pipeline/filters/concat_filter.rb,
lib/rake-pipeline/filters/ordering_concat_filter.rb,
lib/rake-pipeline/filters/pipeline_finalizing_filter.rb,
lib/generators/rake/pipeline/install/install_generator.rb

Overview

A Pipeline is responsible for taking a directory of input files, applying a number of filters to the inputs, and outputting them into an output directory.

The normal way to build and configure a pipeline is by using Pipeline.build. Inside the block passed to Pipeline.build, all methods of DSL are available.

Examples:

Rake::Pipeline.build do
  # process all js, css and html files in app/assets
  input "app/assets", "**/*.{js,coffee,css,scss,html}"

  # processed files should be outputted to public
  output "public"

  # process all coffee files
  match "*.coffee" do
    # compile all CoffeeScript files. the output file
    # for the compilation should be the input name
    # with the .coffee extension replaced with .js
    filter(CoffeeCompiler) do |input|
      input.sub(/\.coffee$/, '.js')
    end
  end

  # specify filters for js files. this includes the
  # output of the previous step, which converted
  # coffee files to js files
  match "*.js" do
    # first, wrap all JS files in a custom filter
    filter ClosureFilter
    # then, concatenate all JS files into a single file
    concat "application.js"
  end

  # specify filters for css and scss files
  match "*.{css,scss}" do
    # compile CSS and SCSS files using the SCSS
    # compiler. if an input file has the extension
    # scss, replace it with css
    filter(ScssCompiler) do |input|
      input.sub(/\.scss$/, 'css')
    end
    # then, concatenate all CSS files into a single file
    concat "application.css"
  end

  # the remaining files not specified by a matcher (the
  # HTML files) are simply copied over.

  # you can also specify filters here that will apply to
  # all processed files (application.js and application.css)
  # up until this point, as well as the HTML files.
end

See Also:

Direct Known Subclasses

Matcher, SortedPipeline

Defined Under Namespace

Modules: DSL Classes: CLI, ConcatFilter, DynamicFileTask, EncodingError, Error, FileWrapper, Filter, Graph, GsubFilter, InstallGenerator, Manifest, ManifestEntry, Matcher, Middleware, OrderingConcatFilter, PipelineFinalizingFilter, Project, Railtie, RejectMatcher, Server, SortedPipeline, TmpInputError, UnopenedFile

Constant Summary collapse

VERSION =

Version:

  • 0.8.0

"0.8.0"

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Pipeline

Returns a new instance of Pipeline.

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :inputs (Hash) —

    set the pipeline’s #inputs.

  • :tmpdir (String) —

    set the pipeline’s #tmpdir.

  • :output_root (String) —

    set the pipeline’s #output_root.

  • :rake_application (Rake::Application) —

    set the pipeline’s #rake_application.



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'lib/rake-pipeline.rb', line 158

def initialize(options={})
  @filters         = []
  @invoke_mutex    = Mutex.new
  @clean_mutex     = Mutex.new
  @tmp_id          = 0
  @inputs          = options[:inputs] || {}
  @tmpdir          = options[:tmpdir] || File.expand_path("tmp")
  @project         = options[:project]

  if options[:output_root]
    self.output_root = options[:output_root]
  end

  if options[:rake_application]
    self.rake_application = options[:rake_application]
  end
end

Instance Attribute Details

#filters ⇒ Array (readonly)

Returns this pipeline’s filters.

Returns:

  • (Array) —

    this pipeline’s filters.



142
143
144
 
# File 'lib/rake-pipeline.rb', line 142

def filters
  @filters
end

#input_files ⇒ Array<FileWrapper> Also known as: eligible_input_files

If you specify #inputs, this method will calculate the input files for the directory. If you supply input_files directly, this method will simply return the input_files you supplied.

Returns:

  • (Array<FileWrapper>) —

    An Array of file wrappers that represent the inputs for the current pipeline.



261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
 
# File 'lib/rake-pipeline.rb', line 261

def input_files
  return @input_files if @input_files

  assert_input_provided

  result = []

  @inputs.each do |root, glob|
    expanded_root = File.expand_path(root)
    files = Dir[File.join(expanded_root, glob)].sort.select { |f| File.file?(f) }

    files.each do |file|
      relative_path = file.sub(%r{^#{Regexp.escape(expanded_root)}/}, '')
      wrapped_file = FileWrapper.new(expanded_root, relative_path)

      raise TmpInputError, file if wrapped_file.in_directory?(tmpdir)

      result << wrapped_file
    end
  end

  result.sort
end

#inputs ⇒ Hash[String, String]

Returns the directory paths for the input files and their matching globs.

Returns:

  • (Hash[String, String]) —

    the directory paths for the input files and their matching globs.



124
125
126
 
# File 'lib/rake-pipeline.rb', line 124

def inputs
  @inputs
end

#output_files ⇒ Array<FileWrapper> (readonly)

A list of the output files that invoking this pipeline will generate.

Returns:



139
140
141
 
# File 'lib/rake-pipeline.rb', line 139

def output_files
  @output_files
end

#output_root ⇒ String

Returns the directory path for the output files.

Returns:

  • (String) —

    the directory path for the output files.



127
128
129
 
# File 'lib/rake-pipeline.rb', line 127

def output_root
  @output_root
end

#project ⇒ Project

Returns the Project that created this pipeline.

Returns:

  • (Project) —

    the Project that created this pipeline



147
148
149
 
# File 'lib/rake-pipeline.rb', line 147

def project
  @project
end

#rake_tasks ⇒ Array (readonly)

Returns an Array of Rake::Task objects. This property is populated by the #generate_rake_tasks method.

Returns:

  • (Array) —

    an Array of Rake::Task objects. This property is populated by the #generate_rake_tasks method.



135
136
137
 
# File 'lib/rake-pipeline.rb', line 135

def rake_tasks
  @rake_tasks
end

#tmpdir ⇒ String

Returns the directory path for temporary files.

Returns:

  • (String) —

    the directory path for temporary files



130
131
132
 
# File 'lib/rake-pipeline.rb', line 130

def tmpdir
  @tmpdir
end

Class Method Details

.build(options = {}, &block) ⇒ Rake::Pipeline

Build a new pipeline taking a block. The block will be evaluated by the Rake::Pipeline::DSL class.

Examples:

Rake::Pipeline.build do
  input "app/assets"
  output "public"

  concat "app.js"
end

Returns:

See Also:



194
195
196
197
 
# File 'lib/rake-pipeline.rb', line 194

def self.build(options={}, &block)
  pipeline = new(options)
  pipeline.build(options, &block)
end

Instance Method Details

#add_filters(*filters) ⇒ void Also known as: add_filter

This method returns an undefined value.

Add one or more filters to the current pipeline.

Parameters:

  • filters (Array<Filter>) —

    a list of filters



308
309
310
311
312
313
314
 
# File 'lib/rake-pipeline.rb', line 308

def add_filters(*filters)
  filters.each do |filter|
    filter.rake_application = rake_application
    filter.pipeline = self
  end
  @filters.concat(filters)
end

#add_input(root, pattern = nil) ⇒ Object

Add an input directory, optionally filtering which files within the input directory are included.

Parameters:

  • root (String) —

    the input root directory; required

  • pattern (String) (defaults to: nil) —

    a pattern to match within root; optional; defaults to “*/”



249
250
251
252
 
# File 'lib/rake-pipeline.rb', line 249

def add_input(root, pattern = nil)
  pattern ||= "**/*"
  @inputs[root] = pattern
end

#build(options = {}, &block) ⇒ Rake::Pipeline

Evaluate a block using the Rake::Pipeline DSL against an existing pipeline.

Returns:

  • (Rake::Pipeline) —

    this pipeline with any modifications made by the given block.

See Also:



206
207
208
209
 
# File 'lib/rake-pipeline.rb', line 206

def build(options={}, &block)
  DSL::PipelineDSL.evaluate(self, options, &block) if block
  self
end

#copy(target_class = self.class, &block) ⇒ Pipeline

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Copy the current pipeline’s attributes over.

Parameters:

  • target_class (Class) (defaults to: self.class) —

    the class to create a new instance of. Defaults to the class of the current pipeline. Is overridden in Matcher

  • block (Proc) —

    a block to pass to the DSL

Returns:



219
220
221
222
223
224
225
226
227
 
# File 'lib/rake-pipeline.rb', line 219

def copy(target_class=self.class, &block)
  pipeline = target_class.new
  pipeline.inputs = inputs
  pipeline.tmpdir = tmpdir
  pipeline.rake_application = rake_application
  pipeline.project = project
  pipeline.build &block
  pipeline
end

#finalize ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Add a final filter to the pipeline that will copy the pipeline’s generated files to the output.



388
389
390
 
# File 'lib/rake-pipeline.rb', line 388

def finalize
  add_filter(Rake::Pipeline::PipelineFinalizingFilter.new)
end

#fingerprint ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

A unique fingerprint. It’s used to generate unique temporary directory names. It must be unique to the pipeline. It must be the same across processes.

Returns:

  • (String) 


398
399
400
401
402
403
404
 
# File 'lib/rake-pipeline.rb', line 398

def fingerprint
  if project 
    project.pipelines.index self
  else
    1
  end
end

#invoke ⇒ void

This method returns an undefined value.

Invoke the pipeline, processing the inputs into the output.



320
321
322
323
324
325
326
327
328
329
330
 
# File 'lib/rake-pipeline.rb', line 320

def invoke
  @invoke_mutex.synchronize do
    @tmp_id = 0

    self.rake_application = Rake::Application.new

    setup

    @rake_tasks.each { |task| task.invoke }
  end
end

#last_manifest ⇒ Object

the Manifest used in this pipeline



412
413
414
 
# File 'lib/rake-pipeline.rb', line 412

def last_manifest
  project.last_manifest
end

#manifest ⇒ Object

the Manifest used in this pipeline



407
408
409
 
# File 'lib/rake-pipeline.rb', line 407

def manifest
  project.manifest
end

#rake_application ⇒ Rake::Application

Returns The Rake::Application to install rake tasks onto. Defaults to Rake.application.

Returns:

  • (Rake::Application) —

    The Rake::Application to install rake tasks onto. Defaults to Rake.application



291
292
293
 
# File 'lib/rake-pipeline.rb', line 291

def rake_application
  @rake_application || Rake.application
end

#rake_application=(rake_application) ⇒ void

This method returns an undefined value.

Set the rake_application on the pipeline and apply it to filters.



298
299
300
301
302
 
# File 'lib/rake-pipeline.rb', line 298

def rake_application=(rake_application)
  @rake_application = rake_application
  @filters.each { |filter| filter.rake_application = rake_application }
  @rake_tasks = nil
end

#setup ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Set up the filters and generate rake tasks. In general, this method is called by invoke.



337
338
339
340
341
 
# File 'lib/rake-pipeline.rb', line 337

def setup
  setup_filters
  generate_rake_tasks
  record_input_files
end

#setup_filters ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Set up the filters. This will loop through all of the filters for the current pipeline and wire up their input_files and output_files.

Because matchers implement the filter API, matchers will also be set up as part of this process.



351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
 
# File 'lib/rake-pipeline.rb', line 351

def setup_filters
  last = @filters.last

  @filters.inject(eligible_input_files) do |current_inputs, filter|
    filter.input_files = current_inputs

    # if filters are being reinvoked, they should keep their roots but
    # get updated with new files.
    filter.output_root ||= begin
      output = if filter == last
        output_root
      else
        generate_tmpdir
      end

      File.expand_path(output)
    end

    filter.setup_filters if filter.respond_to?(:setup_filters)

    filter.output_files
  end
end