In this section, the conceptual design of an ECP with the focus on task assignment strategies is presented. Section3.4.1 provides the basic structure, discussing briefly the relationships of basic elements on the ECP derived from the literature and the qualitative study. Section 3.4.2 focuses on the design of assignment strategies and presents the corresponding workflows for the management of the respective crowd- sourcing processes motivated by the identified scenarios.
3.4.1 Basic Structure of Enterprise Crowdsourcing Platforms
Figure9shows schematically how the elements of an ECP are structured. At the cen- ter of an ECP, there are the tasks and the actors, as well as the contribution towards the result of the crowdsourcing process. Depending on the interaction of the actor on the task, an actor can either be the worker of a task or the requester of a task. In a managed ECP, it could be necessary to define a manager, who needs to review created tasks before they are published on the platform. A user of an ECP can act in different ways on different tasks and is therefore seen as an actor, who can perform different actions, depending on his view on the task. Actors have three main attributes attached that describe their profile: the qualifications, the roles, and the motivators.
The qualifications describe the skills, knowledge, and characteristics of the respec- tive actor and can be matched against the requirements of a task. The qualifications can describe characteristics such as age, department, confidentiality level, etc., which can be integrated from existing systems or queried during registration. Skills and knowledge, on the other hand, are more difficult to grasp and are subject to a con-
Assignment Strategy Requirements Incentive Mechanism Task Subtask Contribution Actor Worker Requester Manager Roles Motivators Qualifications
Figure 9: Basic structure of elements on ECPs.
stant change as a result of experience at work and in everyday life. They may be recorded at the time of registration, while a distinction should be made between non-verified data and verified data, e.g., via certificates. In addition, the actor’s ac- tivity on the ECP results in changes in skills and knowledge. Executed tasks and corresponding feedback can serve to prove the skills and the level at which they are mastered.
Besides the qualifications, an actor’s profile consists of motivators describing the motives of the user on the platform. Depending on those motivators, the incentive mechanism of a task can be adapted. In addition, at least one role is assigned to each actor, which defines the general rights of the users on the platform. Such roles include the rights to work on tasks, request tasks, manage tasks or provide adminis- tration rights for certain parts of the platform.
A task can be broken down into further subtasks as shown in Figure 9. In order
to perform the composition of results from a decomposed task, this relationship has to be managed carefully. A task contains information about the specific requirements that are necessary to fulfill the task. In addition, each task adheres to exactly one assignment strategy with which it is offered to the crowd. The incentive mechanism can be dependent on the task and also adapted to the motivators of its target workers.
The core task of the platform is to provide the actors with the tasks that are ap- propriate for them. For this purpose, various filter and matching processes are used, which are described in Section3.4.2. These reduce the crowd to the actors who are suitable for a task, based on the qualifications that match the requirements of the task but also based on the motivators of an actor. However, it is necessary not to restrict possible assignments too much, as one main motivator in ECPs is to gather further skills [83], also described in one of the interviews:
“. . . you want to expand your skills in some way [. . . ] you take on tasks that perhaps go beyond your own qualifications, to get this competency into your profile.”
3.4.2 Design of Task Assignment Strategies and Workflows
The scenarios identified and the information collected serve as a basis for the distinc- tion of task assignment strategies and for the design of corresponding workflows. An assignment strategy describes general conditions and goals in which a task is offered to possible workers. An assignment workflow describes different steps and dependencies to assign a task to one or many workers. Different scenarios demand identical assignment strategies, and therefore, workflows can be used in different scenarios depending on the required assignment strategy or application.
This section describes the two different assignment strategies direct assignment and open application, which differ from the self-selection assignment strategy used widespread in micro-task markets and the corresponding workflows. From the coded interview results, task assignment strategies were often found to include a crowd se- lection workflow. As this is a shared process between the different task assignment strategies, crowd selection is described as an independent process since this work- flow is identified as part of different assignment strategies. Therefore, the crowd selection workflow is presented first, while the descriptions of the direct assignment and open application strategies follows afterward.
3.4.2.1 The Crowd Selection Workflow
In enterprise crowdsourcing, the set of workers is equal to all employees. In par- ticular cases, it is extended by members of partners or clients. Previous sections discussed the relevance of qualification, confidentiality, and motivation in enter- prise crowdsourcing. In many cases, the general set of all workers is restricted to assignable workers. This means, before an assignment of a task to one or many workers can be executed, the possible crowd has to be filtered. This step of selecting workers is called crowd selection. Selecting assignable workers from the crowd is described as a three-step process, which is visualized in Figure10.
0. At first, all workers are judged to be relevant for selection.
1. In a first filtering step, the qualifications of the workers are matched with the requirements of the task. To not restrict the selection too much, a task can include must-have and nice-to-have requirements. The result of the first step is a list of workers that are capable of solving the task without further restrictions. 2. A second filter takes the availability and the confidentiality level of the workers
into account. This optional step returns a list of workers that are capable of, available for, and have the clearance for solving the task.
3. The third and last filter is concerned with the motivation of the selected worker. The target is to identify workers that are not only capable of working on the task but have a particular interest in solving the task. This filtering step leads to the final set of chosen workers.
Though the last two filtering steps are optional, they are considered a crucial element in order to maintain a sustainable platform, as they provide driving factors for the acceptance of the platform.
Figure 10: Crowd selection process.
3.4.2.2 Assignment Workflow for the Direct Assignment of Workers
The first assignment strategy, which is related to the scenario of GUI tests is called direct assignment (see Figure11). The goal of the direct assignment strategy is to reach many users without caring too much about single contributions. Therefore, the cor- responding workflow assigns the tasks to the selected crowd (see Section3.4.2.1) and mainly focuses on the concern to reach a critical number of contributions. As long as this number can still be reached, there is nothing to do but wait for the contributions. When this number cannot be reached anymore, the requester needs to take action in order to get the task done (i.e., adapt the crowd selection or allow the system to publish the task also externally). The selected crowd is also applicable to form a user group in order to provide the tests to the same crowd a second time. This can also benefit the efforts of building proper community management.
The workflow for this strategy does not require any application of workers. After creating the tasks and filtering the crowd, the selected workers are directly assigned to the task. That means they will find a new task assigned to them on the platform for execution.
Figure 11: Direct assignment workflow to reach a broad crowd.
3.4.2.3 Assignment Workflow for Open Applications of Workers
The second assignment strategy open application, is related to the scenario of creat- ing presentation slides, described before. The goal of the open application strategy is to find a single qualified worker. The scenario requires an iterative approach for reviewing applications and republishing the task (see Figure 12). After creating the task and filtering the crowd, workers can apply for the task. The requester is directly involved in the selection process, by reviewing the workers that applied for the task.
The selected worker submits the solution, and the requester either accepts the solu- tion or reopens the task with additional feedback for the worker. This process can be repeated until the expected result is reached. If there are no applications or ev- ery application has been rejected, the requester needs to lower the requirement and constraints of the task or find other means to solve it.
Figure 12: Open application workflow to assign a specific worker.