X12Parser - a library to manipulate X12 structures using native Ruby syntax
$Id: README 40 2008-11-13 19:51:31Z ikk $
WARNING The project is in development. Contributors are welcome.
Project home is at rubyforge.org/projects/x12parser/. Please note, this is a different project from Chris Parker’s port of X12::Parser Perl module.
The goal
The idea is to access X12 messages directly from Ruby, i.e., using a syntax like
.L1000.L1010[1].AK4.DataElementReferenceNumber
This syntax can be used to get and set any field of an X12 message and it makes X12 parsing much more straightforward and self-documenting.
The problem
X12 is a set of “standards” possessing all the elegance of an elephant designed by committee, and quite literally so, see www.x12.org. X12 defines rough syntax for specifying text messages, but each of more than 300 specifications defines its own message structure. While messages themselves are easy to parse with a simple tokenizer, their semantics is heavily dependent on the domain. For example, this is X12/997 message conveying “Functional Acknowledgment”:
ST*997*2878~AK1*HS*293328532~AK2*270*307272179~AK3*NM1*8*L1010_0*8~
AK4*0:0*66*1~AK4*0:1*66*1~AK4*0:2*66*1~AK3*NM1*8*L1010_1*8~AK4*1:0*
66*1~AK4*1:1*66*1~AK3*NM1*8*L1010_2*8~AK4*2:0*66*1~AK5*R*5~AK9*R*1*
1*0~SE*8*2878~
I.e., X12 defines an alphabet and somewhat of a dictionary - not a grammar or semantics for each particular data interchange conversation. Because of many entrenched implementations and government mandates, the X12 is not going to die anytime soon, unfortunately.
The message above can be easily represented in Ruby as a nested array:
m = [
['ST', '997', '2878'],
['AK1', 'HS', '293328532'],
['AK2', '270', '307272179'],
['AK3', 'NM1', '8', 'L1010_0', '8'],
['AK4', '0:0', '66', '1'],
['AK4', '0:1', '66', '1'],
['AK4', '0:2', '66', '1'],
['AK3', 'NM1', '8', 'L1010_1', '8'],
['AK4', '1:0', '66', '1'],
['AK4', '1:1', '66', '1'],
['AK3', 'NM1', '8', 'L1010_2', '8'],
['AK4', '2:0', '66', '1'],
['AK5', 'R', '5'],
['AK9', 'R', '1', '1', '0'],
['SE', '8', '2878'],
]
but it will not help any since, say, segment ‘AK4’ is ambiguously defined and its meaning not at all obvious until the message’s structure is interpreted and correct ‘AK4’ segment is found.
The solution
Message structure
Each participant in EDI has to know the structure of the data coming across the wire - X12 or no X12. The X12 structures are defined in so-called Implementation Guides - thick books with all the data pieces spelled out. There is no other choice, but to invent a computer-readable definition language that will codify these books. For example, the X12/997 message can be defined as
loop 997 1:1
{
segment ST 1:1
segment AK1 1:1
loop L1000 0:999999
{
segment AK2 0:1
loop L1010 0:999999
{
segment AK3 0:1
segment AK4 0:99
} # L1010
segment AK5 1:1
} # L1000
segment AK9 1:1
segment SE 1:1
} # 997
Namely, the 997 is a ‘loop’ containing segments ST (only one - ‘1:1’), AK1 (also only one), another loop L1000 (zero or many repeats), segments AK9 and SE. The loop L1000 can contain a segment AK2 (optional - ‘0:1’) and another loop L1010 (zero or many), and so on.
The segments’ structure can be further defined as, for example,
segment AK2 {
TransactionSetIdentifierCode S R 3-3 Tbl143
TransactionSetControlNumber S R 4-9
} # AK2
wihch defines a segment AK2 as having to fields: TransactionSetIdentifierCode and TransactionSetControlNumber. The field TransactionSetIdentifierCode is defined as having a type of string (‘S’), begin required (‘R’), having length of minimum 3 and maximum 3 characters (‘3-3’), and being validated against a table Tbl143. The validation table is defined as
table Tbl143 {
100 Insurance Plan Description
101 Name and Address Lists
...
997 Functional Acknowledgment
998 Set Cancellation
} # Tbl143
where required values are first tokens on each line, i.e., 100, 101, …, 997, 998.
This message is fully flashed out in an example ‘misc/997.d12’ file, copied from the ASC X12N 276/277 (004010X093) “Health Care Claim Status Request and Response” National Electronic Data Interchange Transaction Set Implementation Guide.
Now expressions like
.L1000.L1010[1].AK4.DataElementReferenceNumber
start making sense of sorts, overall X12’s idiocy notwithstanding - it’s a field called ‘DataElementReferenceNumber’ of a first of possibly many segments ‘AK4’ found in the second repeat of the loop ‘L1010’ inside the enclosing loop ‘L1000’. The meaning of the value ‘66’ found in this field is still in the eye of the beholder, but, at least its location is clearly identified in the message.
X12 Structure Definition Language (d12)
The syntax of the X12 structure definition language should be apparent from the ‘997.d12’ file enclosed with the package. The strict definition is formalized in ‘lib/X12/x12syntax.treetop’ file.
Parsing
Here is how to parse an X12/997 message (the source is in example/parse.rb):
require 'x12'
# Read message definition and create an actual parser
# by compiling .d12 file
parser = X12::Parser.new('misc/997.d12')
# Define a test message to parse
m997='ST*997*2878~AK1*HS*293328532~AK2*270*307272179~'\
'AK3*NM1*8*L1010_0*8~AK4*0:0*66*1~AK4*0:1*66*1~AK4*0:2*'\
'66*1~AK3*NM1*8*L1010_1*8~AK4*1:0*66*1~AK4*1:1*66*1~AK3*'\
'NM1*8*L1010_2*8~AK4*2:0*66*1~AK5*R*5~AK9*R*1*1*0~SE*8*2878~'
# Parse the message
r = parser.parse('997', m997)
# Access components of the message as desired
# Whole ST segment: -> ST*997*2878~
puts r.ST
# One filed, Group Control Number of AK1 -> 293328532
puts r.AK1.GroupControlNumber
# Individual loop, namely, third L1010 sub-loop of
# L1000 loop: -> AK3*NM1*8*L1010_2*8~AK4*2:0*66*1~
puts r.L1000.L1010[2]
# First encounter of Data Element Reference Number of the
# first L1010 sub-loop of L1000 loop -> 66
puts r.L1000.L1010.AK4.DataElementReferenceNumber
# Number of L1010 sub-loops in L1000 loop -> 3
puts r.L1000.L1010.size
Generating
Here is how to perform a reverse operation and generate a well-formed 997 message (the source is in example/factory.rb):
require 'x12'
# Read message definition and create an actual parser
# by compiling .d12 file
parser = X12::Parser.new('misc/997.d12')
# Make a new 997 message
r = parser.factory('997')
#
# Set various fields as desired
#
# Set fields directly
r.ST.TransactionSetIdentifierCode = 997
r.ST.TransactionSetControlNumber = '2878'
# Set fields inside a segment (AK1 in this case)
r.AK1 { |ak1|
ak1.FunctionalIdentifierCode = 'HS'
ak1.GroupControlNumber = 293328532
}
# Set fields deeply inside a segment inside
# nested loops (L1000/L1010/AK4 in this case)
r.L1000.L1010.AK4.DataElementSyntaxErrorCode = 55
r.L1000.AK2.TransactionSetIdentifierCode = 270
# Set nested loops
r.L1000.L1010 {|l|
l.AK3 {|s|
s.SegmentIdCode = 'NM1'
s.LoopIdentifierCode = 'L1000D'
}
l.AK4 {|s|
s.CopyOfBadDataElement = 'Bad element'
}
}
# Add loop repeats
r.L1000.repeat {|l1000|
(0..1).each {|loop_repeat| # Two repeats of the loop L1010
l1000.L1010.repeat {|l1010|
l1010.AK3 {|s|
s.SegmentIdCode = 'DMG'
s.SegmentPositionInTransactionSet = 0
s.LoopIdentifierCode = 'L1010'
s.SegmentSyntaxErrorCode = 22
} if loop_repeat == 0 # AK3 only in the first repeat of L1010
(0..1).each {|ak4_repeat| # Two repeats of the segment AK4
l1010.AK4.repeat {|s|
s.PositionInSegment = loop_repeat
s.DataElementSyntaxErrorCode = ak4_repeat
} # s
} # ak4_repeat
} # l1010
} # loop_repeat
l1000.AK5{|a|
a.TransactionSetAcknowledgmentCode = 666
a.TransactionSetSyntaxErrorCode4 = 999
} # a
} # l1000
# Print the message as a string -> ST*997*2878~AK1*HS*293328532~
# AK2*270*~AK3*NM1**L1000D~AK4***55*Bad element~AK5*~AK3*DMG*0*
# L1010*22~AK4*0**0~AK4*0**1~AK4*1**0~AK4*1**1~AK5*666****999~
# AK9****~SE**~
puts r.render
Download
The latest X12 library version can be downloaded from rubyforge.org/frs/?group_id=7297
Installation
You can install X12 library with the following command.
% gem install X12
If you install directly from the X12*.gem file, it requires these packages to be installed first:
License
X12 library is released under the Lesser GPL license, see www.gnu.org/licenses/lgpl.txt
Major deficiencies
-
Validation is not implemented.
-
Field types and sizes are ignored.
-
No access methods for composites’ fields.
Wish list
-
.d12 files should have an ‘include’ facility, so data definitions can be reused for different messages.
-
It would be nice to codify all popular X12 messages in .d12 format.
Support
Please use the following:
-
forums on Rubyforge for general discussions, rubyforge.org/forum/?group_id=7297
-
trackers to submit bugs or feature requests, rubyforge.org/tracker/?group_id=7297
-
to contact the author, send mail to prelude rubyforge org
Acknowledgments
The authors of the project were inspired by the following works:
-
The Perl X12 parser by Prasad Poruporuthan, search.cpan.org/~prasad/X12-0.09/lib/X12/Parser.pm
-
The Ruby port of the above by Chris Parker, rubyforge.org/projects/x12-parser/
-
Treetop Ruby parser, treetop.rubyforge.org