sexp_cli_tools

Educational project exploring the utility in searching and manipulating codebases using S-expressions.

Inspiration

I once found wide spread use of a "magical" Ruby method which was unnecessary. The intent of this method was to relieve the developer from the repetition of setting instance variables from method parameters. How this magic method did this was difficult to understand for most. Upon examining this method, I noticed it made costly calls to do at run time what could have been done by a developer with a keyboard, or if you wanted to, with code at load time.

I was new to the team and project, and because the use of this method was wide spread, I wanted a systematic and repeatable approach to refactoring it out of existence so that I and my new colleagues could trust the widespread change.

Concrete Examples We Can All Learn From

Decoupling Subclasses Using Hook Messages

In Chapter 6 of Practical Object-Oriented Design in Ruby by Sandi Metz, part of the discussion is focused on finding the ideal coupling between child and parent classes. One proposal introduced in that chapter is to use hook methods, instead of calling super.

Lets imagine a scenario where we have achieved total consensus in an organization, and the new direction is to dogmatically use hook methods, instead of calling super.

Goal

  • Replace methods that call super with a hook method
  • Modify parent class' implementation of the supered method to call hook methods

Initial state

We will begin with the classes Bicycle, RoadBike and MountainBike. We will build them up to the state from Managing Coupling Between Superclasses and Subclasses until we can recognize the important parts of the "discernible pattern."

Milestones

Things we must be able to interogate about this code:

  • Which are the children, and which is the parent class?
  • Which methods call super, and which is the method that responds to super?
  • What in each method that calls super needs to be in the respective hook method?
  • What change needs to occur in the method responding to super to leverage the hook methods?

Finding the parent of child classes

An s-expression for an empty class in Ruby, as parsed by ruby_parser, looks like this:

class Bicycle

end

s(:class, :Bicycle, nil)

An s-expression has a type and a body. The above s-expression's type is :class and the body is s(:Bicycle, nil).

An s-expression for an empty class with a parent looks like this:

class MountainBike < Bicycle

end

s(:class, :MountainBike, s(:const, Bicycle))

This s-expression's type is still :class, but the body is: s(:MountainBike, s(:const, :Bicycle)).

An s-expression is a representation of the abstract syntax tree, and the s-expressions generated by ruby_parser use this sexp_body recursion to create that tree.

Matching a class

ruby_parser comes with a class Sexp::Matcher which provides a terse syntax that we can use to select nodes from the s-expression tree.

The Sexp::Matcher expression that matches any class definition is: (class ___). That expression uses the triple underscore ___ wildcard to match anything following a class type s-expression.

Matching a class with an explicit parent

The Sexp::Matcher expression that matches any class with an explicit parent is: (class _ (const _) ___). This uses the single underscore _ positional wild card match, and then matches the constant s-expression containing the parent class.

Matching a class with an implicit parent

It is also possible to include negation in Sexp::Matcher. A class with an implicit parent does not have the constant s-expression (const _). Right now, our class s-expression matcher, (class ___) matches all our classes. To match only Bicycle we must use negation. That s-expression is (class _ [not? (const _)] ___).

Capturing what we've learned in a tool that people can use

Knowing the syntax for Sexp::Matcher expressions gives us some confidence that we can start iterating on a tool to help us achieve our goal. The implicit expectation in the project name is that a command line interface is provided. To complete an initial release of a command line tool, we'll use the rubygem aruba to help with test setup and teardown.

The sexp command offers a convenient shortcut to the Sexp::Matcher expressions we'll develop. As we figure out the s-expression matchers along the way, we can add to the list of known matchers to create simple shortcuts, like with the builtin sexp find child-class or sexp find parent-class.

Methods that call super, and methods that are super

Iterating on figuring out Sexp::Matcher patterns

What isn't shown in the commit which added the Sexp::Matcher is the trial and error in the console trying to remember the terse rules.

Setting up a unit test can help close that iteration loop. Consider the unit test for SexpCliTools::Matchers::SuperCaller

Allowing users to experiment with s-expressions might enable exploration and discovery. The sexp find command also supports inline s-expressions. Try these in your projects:

  • sexp find '(class ___)' to find class definitions
  • sexp find '[child (class ___)]' to find class definitions nested in a namespace
  • sexp find '[child (case ___)]' to find case statements

So having test driven the development of the super-caller matcher next we have to find the methods that respond to super.

Finding super implementations

So far we've been using Sexp::Matcher strings to find quite abstract parts of our code. But, it's completely possible to fill in what in the parts we know that we'd like to find.

  • sexp find '(class :Bicycle ___)' from my working copy of this project turns up the test fixture file for Bicycle, as well as the copy of it aruba makes in the tmp/ directory for testing purposes.
  • sexp find '[child (defn :initialize ___)]' only turns up the test fixture file for RoadBike. I guess it is time to fill in more of our Bicycle class!

Finding the super implementation will involve finding a class that contains a method defintion. So far, our matchers haven't taken any parameters. A (naive) matcher for a super implementation might have two parameters, the name of the class we expect to define the method, and the name of the method.

Passing matcher parameters

Early on I chose to have the second sequence argument to the command line interface sexp find the glob pattern of files to include in the search. However, I want to prioritize matcher parameters for that position now. Although my test coverage didn't include tests for that glob pattern, I did document it.

So, when I moved that out into the --include command line option, that was a breaking change to the public interface. That would necessitate incrementing the major version number according to semantic versioning. I have a hunch that because I'm still in the 0 major release, I could get away with not bumping it. But, I think the --include is something I can stick to.

What I remember about semantic versioning is that additions can just be minor version bumps. So, as long as I don't make a backwards incompatible change to the find command or the --include option I should be good.

Following merge of: ✨ sexp find method-implementation passed_method lists files that define the passed method I'll release v1.0.0! In that PR I chose to do inside-out testing because the aruba tests are a bit slow.

I found it helpful to run just the CLI command tests I was working on using the TEST and TESTOPTS options to the rake test tast, like so:

rake TEST='test/sexp_cli_tools/cli_test.rb' TESTOPTS="--name=/method-implementation/"
Capturing the Superclass name

Hook methods from super callers

Hook calls from super methods