Cucumber Messages is a message protocol for representing results and other information from Cucumber.
Cucumber Messages are currently implemented in the following versions of Cucumber (using the
- Cucumber-JVM 6.0.0 and greater
- Cucumber-Ruby 4.0.0 and greater
- Cucumber.js 7.0.0 and greater
json formatter is now in maintenance mode for these these implementations, and Messages is the preferred standard.
See utilities for a list of tools that may help with backward and forward compatibility
Why Cucumber Messages
Cucumber needs to produce results in a machine-readable format so that other tools can generate reports.
+----------+ messages +-----------+ | Cucumber |---m-m-m-m-m-m-->| Formatter | +----------+ +-----------+
Historically, Cucumber has done this with the
These formats have several shortcomings that are addressed by cucumber messages:
High memory footprint
JSON and XML production/consumption is done by serialising/deserialising an object graph. For "big" Cucumber runs this graph may consume a considerable amount of RAM, in particular if several large attachments (screenshots) are attached.
This can cause out of memory errors, aborting Cucumber runs or reporting jobs. It also means that no results can be consumed by downstream processors until the last scenario has finished.
With Cucumber Messages, several messages containing smaller pieces of information are emitted continuously to a stream, avoiding high memory consumption and enabling real-time processing of results.
Lack of a schema
The JSON report does not have a formal schema. This has led to slightly inconsistent implementations of the JSON formatter in various Cucumber implementations. Consumers of the JSON format have to anticipate and detect these inconsistencies and try to cope with them.
junit XML format can only contain very limited information such as test case name and status.
While there isn't an official schema for JUnit XML, there are a few defacto ones around, and they
are very limited.
json format represents the following information:
- Gherkin source (as a rough and lossy representation of a Gherkin document's abstract syntax tree)
- Attachments (formerly called
- Path and line for step definitions
- Results for hooks
However, it does not contain the following information (but Cucumber Messages does):
- Original source code of the Gherkin document
- Gherkin document abstract syntax tree
- Step definitions
- Parameter types
The protocol aims to decouple various components of the Cucumber platform so that:
- Each component only needs to know about a subset of messages
- Gherkin is decoupled from the Cucumber execution component
- This is part of a strategy to support other formats such as Markdown and Excel
See messages.md for a detailed description of each message type.
When Cucumber Messages are stored in a file or sent over a network, they are encoded as NDJSON. We call this a message stream.
Each message in a message stream is of type Envelope.
Each subdirectory defines language-specific implementations of these messages, generated from the JSON schemas.
You will find examples of Cucumber Messages in the compatibility-kit project.