7.1.
Introduction
The chapter discusses methods of validation, verification, and evaluation. It aims to give the SSA developers the proper assessment tools to insure they are developing the right sociological requirements (validation), are developing them right (verification), and are on the right path to achieving desired sociological goals and objectives (“G&Os”) (evaluation). The third case studies section discusses applications used for the development of sociology of software architecture and reviews candidate case studies. This “how to” manual aims to empower SSA application development. We ask and answer the following questions: What are SSA requirements? How do we define, qualify, and/or quantify SSA groups (stakeholders) using group models and/or sociological theory frameworks? How do we apply methods, approaches, models, and techniques to map SSA requirements? Then we discuss some applications including ones that we have researched, developed, and examined ourselves exploring and developing SSA methodology. This paper’s goal is to be a practical training and demonstration manual on how to develop, apply, and validate SSA requirements.
Assessment is “the action or an instance of making a judgment about something” (MWD, 2019). The Cambridge dictionary defines it as “the act of judging or deciding the amount, value, quality, or importance of something, or the judgment or decision that is made” (CED,
2019) We define SSA Assessment as the action of making an approval, decision, or judgement on whether we are developing the right SRs, whether we are developing them the right way, and whether the SSA development is meeting its G&Os. The G&Os include quality production, value generation, increased return on investment, optimization, cost effectiveness, prioritization, importance, etc. We break SSA Assessment into three assessment types: validation, verification, and evaluation.
7.1.1.
Validation
Validation (Kirk, Miller, and Miller, 1986; Carmines and Zeller, 1979; Wainwright, 1997; and Heise and Bohrnstedt, 1970) is “the act or process of making something officially or legally acceptable or approved,” or the “proof that something is correct” (CED, 2019). By SSA Validation we mean the action of making an approval, decision, or judgement on whether we are developing the right SRs. SSA Validation is a method to confirm that we are developing the correct/right SSA application and/or product.
7.1.2.
Verification
Verification is “the process of testing or finding out if something is true, real, accurate, etc.”(CED, 2019). We define SSA Verification as the action of making an approval, decision, or judgement on whether we are developing the SRs accurately or the right way. SSA Verification is a method to confirm that we are developing the correct/right SSA application and/or product right.
7.1.3.
Evaluation
Evaluation is “the process of judging something's quality, importance, or value, or a report that includes this information” (CED, 2019). We define evaluation as the action of making an approval, decision, or judgement on whether the SSA development is meeting its G&Os including quality, importance, and value. SSA Evaluation is a measurement method to confirm our progress towards achieving our SSA development G&Os with optimal results.
Assessing social behavior is much more complex and challenging than assessing IT behavior; social behavior is much more complex, has apparent and hidden variables (such as intent), and generates more qualitative than quantitative data. This is why we focus on statistics for sociology (i.e. SPSS). IT developers have the advantages and experience of having a toolbox of continuous user feedback data collection; we advocate using it for a better assessment of social requirements. There is a Cambrian explosion of consumer data and attributes. It is easier and cheaper to acquire. It can be very powerful in discovering sociological trends, and behavior elasticity and propensity. With the exponential growth and advancement of data science, many data transformation techniques allow for better transformation of qualitative to quantitative data. This is a promising era in the rapid development of assessment techniques that can keep the SSA developer’s promise of efficiency, productivity, and optimization high and attractive. And the benefits could be mammoth.
Sociological research methods encourage to pay attention to gender, ethnic, cultural, and generational variations. There is a strong sociological momentum to promote and support consumer empowerment through distributed technology. This is an opportunity for SSA developers to exploit the wave and generate appreciable added value and benefits. We should be aware of whether we are promoting social cohesion or division, conflict or resolution, social interactionism or rejectionism, progressiveness or backwardness, etc. Regarding SRs, the law is often not as technical and clear as it is for SA technical and operational requirements; dealing with the FDA or FCC on IT requirements is much easier that dealing with the CFPB on consumer protection requirements. In this case, professional legal opinion from one or more experts in the specific law domain becomes very important for proper legal assessment.
How do we make the best and most optimal judgements and/or decisions?
The best and most optimal judgements and/or decisions are based on empirical science; they are quantitative, measurable, and testable. And they are easier to confirm through
methodical assessments. However, not all decisions can be based on empirical science. The social sciences are challenged with the complexity of animate human behavior, a large number of unknown and known variables (more unknowns than knowns), and the complexity and variation of intelligent behavior (Savage and Burrows, 2007). Hence, not all sociology is quantitative. It is often qualitative and subjective; but it has to be rational knowledge, methodical, and well grounded and supported by the cumulative body of knowledge development. Johnson, Burke, Russo, and Schoonenboom call it “the meeting of philosophy, science, and practice (Johnson, Russo, and Schoonenboom, 2019). In SSA development, our decision making depends on a spectrum of methodical rational knowledge. This ranges from objective and replicable scientific knowledge based on empirical, quantitative, measurable, and testable data to subjective and not easily replicable near-scientific knowledge based on qualitative data and subjective theoretical frameworks. This spectrum of knowledge helps us contemplate, deliberate, and make optimal decisions.
How do we assess our SSA development?
We identify three forms of assessment: structural, component, and operations assessment. This is represented by a triangular diagram. The three sides include: “team and testing,” “data and analytics,” and “opinion and advice.” Having the right team and doing things right especially conducting tests and making decisions on empirical evidence whenever possible is the most important factor to achieving the SSA G&Os. Data and analytics give us empirical evidence on which we can make better judgements and sounder decisions. Opinion and
advice are necessary to insure that non-empirical qualitative variables are assessed best using expert and professional opinion and advice.
7.1.4.
Structural Assessment
We start by assessing if we have the right team, if the team is operating the right way, and if we can measure members’ contribution to progress towards SSA development G&Os.
SSA Structural Assessment Checklist
Validation Verification Evaluation
Team & Testing
Team Do we have the
right team? Is the team operating the right way? Can we measure team members contribution to progress? Testing, prototypes, & beta versions
Are we using the right tests?
Are we testing right?
Does the test measure progress?
Theoretical and Empirical
Theoretical Empirical
Data & Analytics
Own project data, knowledge, & experience Are we collecting the right data? Are we collecting it the right way?
Can we measure data value and
contribution to progress?
Public domain data Right data? Right way? Measure? 199 of 389
Academic data, knowledge, and research
Right data? Right way? Measure?
Commercial Right data? Right way? Measure?
Industry circles data and Knowledge Right circles, data, and knowledge? Using resources right?
Measure cost and contribution to progress Government data, knowledge, and guidelines Right data, knowledge, & guidelines Using resources right?
Measure cost and contribution to progress.
Opinion & Advice
Advisory board Do we have the
right mix of advisors? Are they the right advisors?
Do we use the advisory board the right way to produce good advice?
Can we measure advisors and advice value and
contribution to progress?
Expert opinion & consulting
Are we hiring the right expert or consultant? Are we using their resources right? Can we measure their contribution to progress and conduct a meaningful cost benefit analysis?
Professional opinion Right pro? Using pro right? Measure contribution?
Table 20 SSA Assessment checklist