Flak Architecture

Targets

When you type rake, the Rakefile is executed. It builds Flak::Target objects, one for the project, and one for each tool. The process is started by reading the contents of project.yml and any tool.yml file it finds. There is a project.yml at the project root and a tool.yml at the root of each tool. We call these files target files.

  • The name of a target yml file is either tool.yml or project.yml. It is used to build a Target object.
  • A Target extends itself with settings, methods, and tasks from templates specified in the target yml file.
  • Each template is a ruby module that reads a yml file.
  • Template order is important, as settings are merged in such a way that new values override old values.
  • The Environment template is implicitly read first and does not appear in the template list.
  • The target yml file is read again at the end to specify file-lists and to allow the opportunity to override any other settings.
  • The Target makes 2 passes over the templates so that all settings are resolved before any tasks are generated.

Templates

A Target can be thought of as a task builder that inherits functionality from a set of objects that we call templates. If you look in a target file such as tool.yml you'll see a templates section.

templates:
- cpp
- maya
- maya_plugin
- gl

This specifies that a Target will be built and will be extended with settings and methods for writing a c++ maya plugin with openGL.

The environment template is read implicitly by the Target before any other templates.

The order of templates in a target file is important. Later templates override earlier ones. If for example the cpp template specifies that the extension for shared libraries under windows is dll and the maya_plugin template specifies mll, then mll will be used because maya_plugin is listed after cpp.

When a Target specifies a template, it means three things:

  1. The target's settings hash is infused with the values from the settings specified in the template
  2. Instance methods defined in the template become instance methods of the target object.
  3. Rake tasks defined in the template are added to the list tasks that can be run on the command line by rake.

How the Settings Hash is built

When a Target is created the first step is to build the settings Hash. Here's the process:

  • Get the OS and configuration keys from environment.yml
  • Get the list of templates from the target file. e.g tool.yml
  • For each template:
    • Replace any environment variables found in the pattern: ${ENV_VAR_NAME}
    • Flatten the hash based on the os and configuration settings. In other words, if a key is nested in configuration_something, or os_something, only use it if the somethings in question are the current configuration and current os respectively.
    • Merge this Hash into the target's settings Hash, overwriting any string values with the new value, and merging any array values.
  • Finally read the target file itself, again, flattening out the OS and configuration, and merging it into the target's settings hash.
  • Expand all FileList globs in the settings hash.

This resolved Hash is what you see when you type 

rake <target_name>:inspect

Some settings keys are required for building other settings - for example :product_revision is required to build some path settings. If these are not present or are the wrong type, then rake tasks will not be generated and you will see a bright red error. In this case not even the rake inspect tasks will be available to help you track down the error. The error does however show the key name and the file it was raised in. 

settings[:product_revision] is not a String. It is a NilClass. (environment.rb)

Other keys are not required until build time - for example :include_flag is used when rake tasks are run, in this case during compilation. If it is missing you will still get a red error, but you will also be able to run rake -T to see available rake tasks, and rake <target_name>:inspect to view the entire settings hash. You might see there is a typo such as :includ_flag

Methods

A Target object also inherits instance methods from the templates it extends. These instance methods provide functionality to the tasks that could not be provided by settings values alone. An example is building a C compile command. As these extended instance methods are only used by tasks when they are run, all the settings will be in place and therefore these methods have access to the fully resolved settings via the @settings instance variable.

Tasks

Tasks are the things that actually build the product. There are rake tasks for compiling 3delight shaders, c++ applications and DSOs, converting images, copying files, compressing files and even building documentation. In flak, we generate Rake tasks in the task_factory method of each template module. Rake tasks may be generated dynamically which considerably reduces the amount of work needed to add file dependencies to a project. Unlike Make, Rake is a Ruby DSL so you get the power of a first class programming language as opposed to the arcane squiggles of Make that have no application anywhere else. See Rake rationale or Martin Fowler's Rake page

The tasks for copying files or processing with erb allow you to add your own rules. You simply specify the source files in a file list under a special key and specify a destination directory with an associated destination key.

Copy Tasks

For example if you wanted to add some some shake files to the source and release them to a shake directory at build time, you could do the following.

In the tool.yml file specify the following file lists and put some files in those directories:

shake_script_copy_files:
- shake/*
- shake/init/*

The fact that the key ends in copy_files is a signal to flak that these files should be copied. It will look for a matching key called shake_script_destination, so tool.yml should also contain

shake_script_destination: 'shake/scripts'

This will create rake tasks for files in the specified shake source directories to be copied to the relative destination when you type rake. Possibly, ~/tools/flintstone/0.0.3/shake/scripts.

NOTE: Like many tasks generated by flak, these tasks are not named tasks, so they don't appear when you type rake -T. To display all tasks type rake -P. These tasks can still be executed individually.

Erb Tasks

Erb tasks are similar to copy tasks, the difference being that erb tokens in the file are replaced with the values from the settings hash at build time.

Suppose you wanted to add a file containing meta information about your product and provide it with the release.

In the project.yml (the project level target file) file you might specify the following file lists and destination:

metafile_erb_files:
- script/meta.yml.erb
metafile_destination: ''

The .erb extension is optional. If it is found it will be stripped away in the destination.

A key ending in erb_files signals that before moving to the destination, its files should be filtered with erb in the context of the target. In other words, you can enter ruby code to be evaluated at build time and it will have access to the settings hash owned by the target. The contents of the file might look like this:

---
date:                      '<%= Time.now.strftime("%d %B %Y") %>'
product_title:             '<%= @settings[:product_name] %>'
version_number:            '<%= @settings[:product_revision] %>'
owner:                     '<%= @settings[:author_name] %>'

When you run rake the file meta.yml will appear at the root of the release and will contain something like:

---
date:                      '10 March 2012'
product_title:             'flintstone'
version_number:            '0.0.1'
owner:                     'Julian Mann'

Admittedly, owner and title are unlikely to change much, so you could type them explicitly. However using values from the settings hash where possible helps to eliminate mistakes and keeps the project DRY. See the ERB Docs for more info on ERB syntax.

Maya Icon Tasks

If you simply want to copy image icons to the destination you could use the formula above for copy tasks. However, Maya node icons should be released in a specific format and sometimes at a specific size. Node icons for example should have 2 versions, one 20X20 with the prefix out_ for the outliner, and one 32x32 with no prefix for the dependency graph. They should both have the extension xpm. If you make your icons in photoshop, the ideal workflow is simply to save one image at any size, in any format, and have the different versions created for the outliner and dependency graph when you build the project. This is exactly what flak does.

In the tool.yml for a maya tool you'll see the keys:

maya_node_icon_files:
- maya/icons/node/*
maya_icon_files:
- maya/icons/*.jpg
maya_icon_copy_files:
- maya/icons/*.png
- maya/icons/*.xbm

Files specified under maya_node_icon_files are converted to 2 versions as detailed above. Files specified under maya_icon_files are left the same size, but converted to xpm. Files specified under maya_icon_copy_files are copied using the copy rules above.

You must have image_magick installed for image conversions to work. To see if it is installed type:

convert