ForkSim’s generation process begins with a base subject system, which is duplicated (forked) a specified
number of times. Continued development of the individual forks is simulated by repeatedly injecting source
code into the forks. Specifically, ForkSim injects a user specified number of functions, source files, and source
directories. Instances of source code of these types are mined from a repository of software systems, which
ensures the injected code is realistic and varied.
Each of the chosen functions, files and directories are injected into one or more of the forks. The number
and particular forks to inject a source artifact into are selected at random. Injection into a single fork creates
code unique to that fork, while injection into a subset of the forks creates code shared amongst those forks.
Injection locations are selected randomly, but only amongst the code inherited from the base system, i.e., not
inside previously injected code. This prevents the injected code from interacting, which makes the generation
process much easier to track and thereby simplifies tool evaluation using the generated dataset. When code is
injected into multiple forks, the injection location may be kept uniform or varied, given a specified probability.
Forks may share code, but that code may be positioned differently within the individual forks.
Table 7.1: Taxonomy of Fork Development Activities
ID Development Activity
DA1 New source code is added. DA2 Existing source code is removed.
DA3 Existing source code is modified and/or evolved. DA4 Existing source code is moved.
DA5 Source code is copied from another fork. It may be copied into a different position than in the source fork, and it may be modified and/or evolved independently of the source. DA6 A fork may itself be forked.
Before code is injected into a fork it may be mutated, i.e., automatically modified in a defined way, given
a specified probability. This causes code shared by the forks to contain differences, simulating that shared
code may be modified or evolved independently for the needs of a particular fork.
Files and directories may be renamed before injection, given a specified probability. While forks may
share code at the file and directory level, they may not have the same name. For example, they may have
been renamed to match conventions used in the fork.
Each injection operation (injection of a function, a file, or a directory) is logged. This includes recording
the forks the code was injected into, the injection locations used, and if and how the code was mutated and/or
renamed before injection. A copy of the code and any of its mutants are kept separately and referenced by the log. ForkSim also maintains a copy of the original subject system. From the log and the referenced
data the directory, file and function code similarities and differences inherited from the original system and
introduced by injection, can be procedurally deduced.
Fig. 7.1 depicts this generation process. On the left side are the inputs: the subject system, source
repository and user-specified generation parameters. The subject system duplicates (forks) are modified by
the boxed process, which repeats for each of the source files, directories and functions to be introduced. The
figure shows an example of this process, in which a randomly selected function from the source repository
is introduced into the forks. ForkSim randomly decided to inject this function into three of the four forks
using non-uniform injection locations, and to mutate the function before injection into the latter two forks.
On the right side are the outputs: the generated fork dataset and the generation log.
ForkSim supports the generation of Java, C and C# fork datasets. It is implemented in Java 7 (main
architecture and simulation logic) and TXL [27] (source code analysis, extraction and mutation). ForkSim’s
generation parameters are summarized in Table 7.2.
Injection Directory and file injection is accomplished by copying the directory or file into a randomly
selected directory in the fork. In order to prevent injected directories from dominating a fork’s directory
tree, only leaf directories (those containing no subdirectories) are selected for injection. Function injection is
performed by selecting a random source file from the fork, and copying the function’s content directly after
a randomly selected function in the chosen file. The generation process can be parametrized to only select functions for injection which fall within a specified size range measured in lines before mutation.
Mutation ForkSim uses mutation operators to mutate code before injection. Fifteen mutation operators,
named and described in Table 7.3, were implemented in TXL [27]. Each operator applies a single random modification of its defined type to input code. These mutation operators are based upon a comprehensive
taxonomy of the types of edits developers make on cloned code [108], which makes them suitable for simulating
how developers modify shared code duplicated between forks.
Files and functions are mutated by applying one of the mutation operators a random number of times
Subject System
Repos- itoryForks
Dataset
1.Select Code
2. Select Forks
3. Select Injection
Locations
4. Mutate Code
6. Log
5. Inject Code
Repeat for X source files, Y source directories and Z functions.
Generation
Parameters
Log
Figure 7.1: Fork Generation Process
measured in lines after pretty printing. This provides an upper limit on how much simulated development is
allowed to occur on a source file or function in a particular fork. Pretty printing (one statement per line, no
empty lines, comments removed) the source artifact before measurement ensures the measure is consistently
proportional to the amount of actual source code contained. This ratio can be specified separately for files
and functions.
A small mutation ratio is recommended (10-15%) as too many changes may cause the variants of a
file/function injected into multiple forks to become so dissimilar that they may no longer be a clone. Detection
tools would be correct not to report them. ForkSim datasets are only useful if the elements declared as similar
are indeed similar.
When directories are injected, each of the source files in the directory may be mutated using the same
process as used for injected files. The directory mutation probability parameter defines how likely a file in
an injected directory is mutated.
As a principle of mutation analysis, ForkSim does not mix mutation operators. This makes it easier
to discover if a similarity analysis tool struggles to detect similar code with particular types of differences.
ForkSim cycles through the mutation operators to ensure that each is represented in the generated forks roughly evenly. When it is not possible to apply a given operator to the file or function, another operator is
Table 7.2: ForkSim Generation Parameters
Parameter Description
Subject System The base system which is forked during the generation process. Source Repository A collection of systems from which the source files, directories and
functions are mined.
Language Language of forks to generate (Java, C or C#). # Forks Number of forks to generate.
# Files Number of files to inject. # Directories Number of directories to inject.
# Functions Number of functions to inject.
Function Size Maximum/Minimum size of functions to inject.
Max Injections Maximum number of forks to inject a particular function/file/directory into.
Uniform Injection Rate Probability of uniform injection of a source artifact.
Mutation Rate Probability of source mutation before injection. Specified separately for function, file and directory injections.
Rename Rate Probability of renaming before injection.
Max Mutations Maximum number of mutations to apply to injected code. Specified as a ratio of the code’s size in lines.
Table 7.3: Mutation Operators from a Code Editing Taxonomy for Cloning
ID Description
mCW A Change in whitespace (addition). mCW R Change in whitespace (removal).
mCC BT Change in between token (/* */) comments. mCC EOL Change in end of line (//) comments.
mCF A Change in formatting (addition of a newline). mCF R Change in formatting (removal of a newline).
mSRI Systematic renaming of an identifier. mARI Arbitrary renaming of a single identifier. mRL N Change in value of a single numeric literal. mRL S Change in value of a single string literal.
mSIL Small insertion within a line (function parameter). mSDL Small deletion within a line (function parameter).
mIL Insertion of a line. mDL Deletion of a line.
mML Modification of a whole line.
Renaming The probability of a file or directory being renamed before injection is specified separately from
that of source code mutation, and both are allowed to occur on the same injection. Renamed source files keep their original extensions.
Usage ForkSim operation is very simple. The user makes a copy of the default generation parameters file,
and tweaks it for the dataset they wish to generate. This includes specifying paths to the subject system
and source repository to use. Once the parameters file and systems are prepared, the user executes ForkSim
and specifies the location of the parameters file and a directory to output the forks and generation log into. Once execution is complete, the forks and generation log are ready for use in experiments involving similarity