The graph read_and_write_multiple_files.mp reads the contents of the same three input files as read_multiple_files_simple.mp (described on page 140). However, rather than simply writing the reformatted records to a single output file, it uses READ MULTIPLE FILES and WRITE MULTIPLE FILES components in combination to get the filenames and write the records to three appropriately renamed files. It shows how to:
AB INITIO CONFIDENTIAL AND PROPRIETARY Processing sets of files 145
z Include data from the input to a READ MULTIPLE FILES component in its output records. In this example, the filenames are added to each output record, and the output format is different from the record format of the files that the component reads.
z Write multiple files, using a different record format than that of the data flowing into the WRITE MULTIPLE FILES component. In this example, the output records of the REFORMAT include the filename, but the records written by the WRITE MULTIPLE FILES component do not. Instead, the component uses that information to name the files it writes to.
To read multiple files from a list and write reformatted data to renamed output files:
1.Add an INPUT FILE component whose URL specifies a file containing a list of the files to read and whose record format specifies how the filenames are separated.
In read_and_write_multiple_files.mp, the URL of Input File List is file:$AI_SERIAL/
list_of_files.dat, and the referenced file has the following content:
trans-2006-01.dat trans-2006-02.dat trans-2006-03.dat
146 Processing sets of files AB INITIO CONFIDENTIAL AND PROPRIETARY
The record format for list_of_files.dat is:
record
string('\n') filename;
end
2.Connect a READ MULTIPLE FILES component to the INPUT FILE and configure it as follows:
a. Hold down the Shift key and double-click the READ MULTIPLE FILES component to generate a default transform. (If the Transform Editor is displayed in Package View, switch to Text View, and you will be prompted to generate the transform.)
b. Specify the record format for reading the records from the input files:
include "/~$AI_DML/transaction_type.dml";
type input_type = transaction;
c. Code the get_filename function such that it will generate the full path to each input file and then retrieve its content. For example:
filename :: get_filename(in) = begin
filename :: string_concat($AI_SERIAL, '/', in.filename);
end;
d. Code the reformat function to output the name of each input file and all desired fields.
This graph outputs the fields defined by transaction in the output record format as described in Step e:
out::reformat(in, filename) = begin
out.filename :: filename;
out.trans :: in;
end;
AB INITIO CONFIDENTIAL AND PROPRIETARY Processing sets of files 147 e.On the Ports tab, select Embed and specify the record format for the output.
This graph includes the file transaction_type.dml, which contains the type definition of transaction (page 141), and creates a metadata type consisting of fields for it and the filename:
include "/~$AI_DML/transaction_type.dml";
metadata type = record
string(",") filename;
transaction trans;
end;
3.Connect a REFORMAT to the READ MULTIPLE FILES component and configure it as follows:
a.On the Ports tab, select Embed and specify the record format for the output.
This graph includes the file processed_type.dml, which contains the type definition of processed (page 143), and creates a metadata type consisting of fields for it and the filename:
include "/~$AI_DML/processed_type.dml";
metadata type = record
string(",") filename;
processed trans;
end;
b.Code the transform as follows to output the filenames and the processed fields.
This graph replaces the description field of the transaction type with the trans_kind field of the processed type as follows:
include "/~$AI_XFR/map_trans_kind.xfr";
148 Processing sets of files AB INITIO CONFIDENTIAL AND PROPRIETARY out::reformat(in) =
begin
out.filename :: in.filename;
out.trans.* :: in.trans.*;
out.trans.trans_kind :: map_trans_kind(in.trans.description);
end;
4.Connect a WRITE MULTIPLE FILES to the REFORMAT component and code its transform as follows:
a. Specify the record format of the output type.
This graph includes the DML file containing the definition of the desired output type, processed:
include "/~$AI_DML/processed_type.dml";
type output_type = processed;
b. Name the output files appropriately:
This graph replaces the string ‘trans’ from the input filenames with ‘processed’ in the output filenames:
filename::get_filename(in) = begin
filename :: string_replace(in.filename, 'trans', 'processed');
end;
c. Write the reformatted transaction data to each output file:
write::reformat(in) = begin
write :: in.trans;
end;
The WRITE MULTIPLE FILES component uses the result of the reformat function as the data to write to the file.
AB INITIO CONFIDENTIAL AND PROPRIETARY Processing sets of files 149 5.If you want to be able to view the contents of one or more of the input files listed in Input File
List, add an INPUT FILE component and configure it as follows:
a.Set its URL to the location of the input file.
In read_and_write_multiple_files.mp, this is file:$AI_SERIAL/trans-2006-01.dat. b.Disable it to prevent others from thinking it is an actual input.
6.If you want to be able to view the contents of one or more of the output files written by the WRITE MULTIPLE FILES component, add an OUTPUT FILE component and configure it as follows:
a.Set its URL to the location of the output file.
In read_and_write_multiple_files.mp, this is file:$AI_SERIAL/processed-2006-01.dat. b.Disable it to prevent others from thinking it is an actual output.
In read_and_write_multiple_files.mp, the data for the first output file looks like this:
AB INITIO CONFIDENTIAL AND PROPRIETARY Header and trailer record processing 151