Gestión de Calidad

Christopher J. Mackie

He who does not live in some degree for others, hardly lives for himself.

—Montaigne

Openness begets openness; that is the hope. But can openness sustain itself? That is an essential question. Both open educational content (OEC) and open source software (OSS) have generated cultures of open- ness among the institutions and individuals who have embraced them. The members of those cultures have discovered new effi ciencies, new opportunities for productivity (“social production”), new forms of orga- nization (“coopetition”), new markets (“the long tail”), new pathways to learning, and new models for engaging with their colleagues and others around the sharing and collaborative construction of intellectual property. However, the jury is still out on whether these two cultures are each sustainable, and whether they can help each other to achieve sustainability.

This chapter discusses some of the challenges facing OSS development and attempts to draw lessons from them for the OEC community. It makes a few key points: that multiple models of both OEC and OSS are possible; that each model has different strengths and weaknesses; and that the approaches one takes to sustainability will depend in important ways on the models of openness that one wishes to encourage.

My program’s grantees and other partners inform the analysis in this chapter; however, the views expressed here are my own. Openness is a controversial topic. Even people who agree on its desirability can dis- agree over what openness really means and how best to achieve it. My colleagues and grantees hold diverse views on the various openness

movements, as refl ected in the diversity of the Foundation’s openness initiatives, which range from the development of OSS through Mellon/ RIT, to the information accessibility and digitization initiatives of the Mellon Scholarly Communication Program, to the subscription models of Mellon-supported organizations such as JSTOR, ARTstor, Aluka, and Portico, to Mellon’s extensive support of the OpenCourseWare initiative. Many of us hope that openness can be achieved broadly and deeply throughout the educational sphere, but the analyses and evaluations on which the recommendations in this chapter are based do not necessarily refl ect the views of my colleagues or the Mellon Foundation.

OSS Limitations and CSS

For all that the OSS movement has produced some runaway successes, including projects like Perl, Linux, and Mozilla Firefox, there appear to be certain types of challenges that are diffi cult for OSS to tackle. Most notably, voluntaristic OSS projects struggle to launch products whose primary customers are institutions rather than individuals: fi nancial or HR systems rather than Web servers or browsers; or uniform, manage- able desktop environments rather than programming languages or oper- ating systems. This limitation may trace to any of several factors: the number of programmers having the special expertise required to deliver an enterprise information system may be too small to sustain a commu- nity; the software may be inherently too unglamorous or uninteresting to attract volunteers; the benefi ts of the software may be too diffuse to encourage benefi ciaries to collaborate to produce it; the software may be too complex for its development to be coordinated on a purely vol- unteer basis; the software may require the active, committed participa- tion of specifi c fi rms or institutions having strong disincentives to participate in OSS; and so on. Any of these factors might be enough to prevent the successful formation of an OSS project, and there are many useful types of enterprise software—including much of the enterprise software needed by higher education institutions—to which several of them apply. In short, however well a standard OSS approach may work for many projects, there is little reason to believe that the same model can work for every conceivable software project.

Some higher education and other nonprofi t institutions, recognizing the limitations of the classic OSS strategy, have evolved a variant that attempts to launch OSS projects for environments in which the classic model appears impossible. This variant brings to the fore the patronage dynamics, governance supports, and incentive structures obscured in the classic model and makes them the centerpiece of the strategy. Under this new model, several institutions contract together to build software for a common need, with the intent of releasing that software as open source. The institutions form a virtual development organization consisting of employees seconded from each of the partners. This entity is governed cooperatively by the partners and managed as if it were an enterprise software development organization, with project and team leads, archi- tects, developers, and usability specialists, and all the trappings of orga- nizational life, including reporting relationships and formal incentive structures. During and after the initial construction phase, the consortial partners open the project and invite in anyone who cares to contribute; over time the project evolves into a more ordinary OSS project, albeit one in which institutions rather than individual volunteers usually con- tinue to play a major role. This variant is sometimes called “directed open source” but is becoming better known as “community source soft- ware” (CSS).

The relative deprecation of altruism as a value by CSS, and its replace- ment with an emphasis on intelligently structured incentives, has implica- tions for how CSS projects organize themselves. CSS products typically take several years to reach market, and partners need to be confi dent that each participant organization will continue to contribute for the duration. As one CSS partner put it, “if we get a new Provost tomorrow who wants to know why our people are being paid by us and taking up our space but reporting to someone at another university to build soft- ware that we’re going to give away for free, I can’t respond by talking about altruistic benefi ts to higher education or the virtues of open source: I need to have a spreadsheet that shows clearly why it is in our institu- tional interest to keep doing this.” Statements like that strike some OSS supporters as mercenary, but essentially the same calculus—perhaps better hidden from view—informs institutional patronage of OSS projects as well. CSS is not fundamentally different from OSS; rather,

it recognizes the hidden sustainability dynamics of OSS and harnesses them more explicitly.

CSS has demonstrated its ability to build enterprise-scale software that can compete on features with commercial offerings; examples include the Sakai collaborative learning environment and the newer Kuali Financial System. Sakai is now in use or in pilot at more than 100 institutions and is growing more rapidly in adoption than many commercial systems. Kuali Financials offered its fi rst public release only a few months ago, and a few smaller institutions are already adopting it; the rest of the projects are still pre-release. The initial successes of CSS have been visible enough that other projects are currently underway. Kuali, in particular, has already spawned projects to produce research administration tools and student information systems, and there is reportedly some interest in building a Kuali HR system and a library automation system as well.

The institutional orientation of CSS may be a powerful stabilizing and sustainability factor. CSS as a concept is substantially newer than OSS, however, and the jury remains out. Advocates argue that the institutional commitment is itself a guarantee of stability. Institutions do not replace these systems quickly, so a community of supporters is guaranteed for at least the medium term, making CSS potentially even a safer bet than a commercial product from a vendor that may be bought-out at any moment. The advocates have logic on their side, but many higher educa- tion CIOs understandably want to see empirical evidence that the logic is sound before they commit to joining these projects.

OEC Differences from OSS, CSS

If we have learned one thing about sustaining openness through the work of Mellon/RIT, it is that project sustainability results from the careful alignment of producers’ and consumers’ interests, so that each party gains something of value from the cooperative project. Traditional OSS usually aligns those interests straightforwardly, because the producers of the software are generally also its users. In cases where producers and consumers are not the same people but are both members of the same organizations, CSS may prove to work better because it makes the orga- nizations, rather than the individuals, both producers and consumers. Other models for sustainability have been attempted, but if one cannot

align producer and consumer interests effectively, then the only other demonstrated way to keep a project viable is by means of an ongoing, outside cash infusion, as when a commercial vendor adopts an OSS project. This makes the donor a single point of failure for the entire project, and if the donor is a commercial entity, it may also introduce important confl icts of interest.

One signifi cant difference between OSS and OEC is that the alignment of producer with consumer interests in OSS production is voluntary, while the alignment of interests in CSS and OEC production usually is not. Once an institution chooses to join a CSS initiative, it assigns staff to the project as it would to any other software project; some voluntary choice may be involved, but staffi ng a CSS project is a job, not a hobby. Similarly, an institution may choose to join an open content initiative and may even ask for faculty volunteers. So far, at least, institutions usually end up compensating or subsidizing faculty in some way for participation—and I know of no institution that gives students the choice of whether or not to have open content in their curricula, any more than institutions ask students to choose their textbooks.

Both CSS and OEC production therefore require institutions to mobi- lize and align internal incentives for their stakeholders while also aligning their organizational interests with those of their consortial partners. This makes CSS particularly interesting for people seeking a sustainability model for OEC. With its leveraging of organizations to cut through the voluntary alignment problem, CSS points to at least one possible way to address OEC’s similar dilemma. However, the interests surrounding OEC production are, if anything, even more challenging to align. Faculty may lose some revenue or career rewards by diverting content out of traditional publishing channels and into OEC; the content usually requires some refi nement or additional processing to be useful as OEC; some portion of its processing, such as copyright clearance, is tedious and requires expertise that most content creators do not possess; and little, if any, of the work required is of tangible benefi t to the content creator or his or her employer. An effective alignment of incentives therefore requires that producers be compensated for some portion of the OEC creation process, along with additional compensation of others whose time and expertise is required to shepherd content through the OEC production process. Perhaps most important, if the creator’s

employer is going to broker all of this organized activity, there should be something in it for the organization as well. Not all compensation need be monetary, but some adequate incentive must be found for each contributor and each piece of content.

A second difference between OSS and OEC involves community dynamics and the way the products are used. Software grows more powerful and capable through continued refi nement, so there are signifi - cant benefi ts, as well as signifi cant community pressures, to contribute derivative work back to the main project. Content, on the other hand, is highly contextual. An instructor will tend to acquire a piece of OEC and adapt it to his or her localized needs via language translation or more substantive amendment. The instructor can then contribute back the derivative work, but it is not usually merged into the ongoing OEC project in the same way that a software patch is merged into an OSS project, because the original creator is unlikely to need modifi cations targeted at different contexts. In fact, it is diffi cult to know how one might merge derivative content productively back into an original. If one encrusts an object with all of its possible contextualizations, the com- pound object will quickly become unwieldy. On the other hand, if one uses the new contextual information to remove context-specifi c depen- dencies in the original, the resulting streamlined object will tend to become so abstracted as to be unusable. As David Wiley notes, in OSS, “forking” the project into independent derivatives is usually considered a disaster; in OEC, it is the central value proposition (personal commu- nication, May 26, 2007).

Context dependency also introduces diffi culties in measurement for OEC that OSS/CSS projects do not face. The need to compile or interpret computer program code simplifi es context, making relatively universal judgments about performance or quality easier to reach. The contextual- ity of teaching and learning styles and situations has the opposite effect, making it diffi cult to evaluate a piece of content without detailed refer- ence to how it will be used, when, and by whom. This distinction can easily be overdrawn, because many aspects of software development (such as usability and programming style) are also highly contextual, and many aspects of content quality, like grammar and spelling, transcend local context to at least some degree. Notwithstanding those similarities, however, the lack of any OEC equivalent to a software compiler at the

heart of the project makes it very diffi cult to develop broadly acontextual quality standards for OEC.

The absence of shared measures of quality in OEC matters for several reasons. First, it complicates the creation and alignment of incentives for producing high-quality OEC. It is diffi cult to structure incentives to produce consistently excellent work if the standard of excellence is ambiguous; instead, one must either create incentives for all producers, paying only limited attention to quality, or else one must pick some particular defi nition of quality, create incentives around it, and hope that enough producers and consumers share the defi nition to make the incen- tives worthwhile. Second, the absence of objective measures of quality confuses consumers as well—particularly in the realm of OEC, where consumers are looking for content precisely because they don’t feel that they know enough about a subject. Finally, the lack of objective quality standards removes one foundation of the meritocratic culture upon which OSS/CSS communities depend. The example of Wikipedia, where confl ict over the merits of anonymous editing have created schism, sug- gests that this can encourage not just the forking of content but also the fragmentation of the entire community.

All of these differences appear to work to the disadvantage of OEC versus OSS/CSS, at least in terms of sustainability. At the same time, however, OEC has at least one profound advantage over OSS/CSS: The skills required to create open content are broadly, even universally, dis- tributed among higher education faculty, staff, and students. The pool of potential OEC contributors is orders of magnitude larger than the pool of potential contributors to a software project. It is diffi cult to judge the relative weights of these advantages and disadvantages, but as someone who works very hard to identify fi ve or six institutions out of thousands that are prepared to join together to build a particular piece of software at some mutually agreeable moment, I envy OEC advocates the opportunities afforded by their large pool of potential participants. OSS Lessons for OEC

The lessons that one may wish to draw from OSS to help OEC initiatives grow and become sustainable depend greatly on one’s mental model of OEC production: Is OEC more like OSS, or more like CSS? Before

reading on, answer that question for yourself—and then refl ect on your answer in order to understand better your own assumptions about OEC. Do you think about OEC primarily as a voluntaristic project in the col- lective pursuit of maximal human freedom to learn? Or do you think about OEC primarily as an institutional effort to achieve reliable educa- tional returns by reaping the benefi ts of cooperation, albeit in a way that shares the resulting value with the world rather than hoarding it? The difference is important in many ways, but real-world OEC initiatives sometimes jumble both visions in ways that make it diffi cult to think incisively about either.

Among other consequences, one’s answer to the question has powerful implications for sustainability. Those interested in pursuing voluntaristic OEC will want to study OSS projects for strategies to reduce the need for patronage and create the greatest quantity and diversity of non- monetary, ego/status incentives for individual participants. Those inter- ested in pursuing institutional OEC strategies will want to study OSS to get below the surface and look hard at the patronage, governance, and incentive arrangements that have aligned complex interests and permit- ted sustainability at scale—and should probably look even harder at CSS projects for examples of how those lessons can be incorporated explicitly into formal institutional models of open production. Institutional OEC advocates will need to pay a great deal of attention to the questions of what type(s) of virtual organization(s) make sense for OEC, and how those organization(s) should be governed and sustained.

Whatever the sustainability model they adopt, OEC advocates must also fi nd a way to reward the production of high-quality content. This is no small challenge under any approach to OEC production. High quality, broadly usable OEC demands production values that exceed those applied to most educational content in regular use today, as evi- denced by the sums spent refi ning existing course content to produce the material on MIT’s OpenCourseWare site. Content must be edited to replace cryptic or institution-specifi c information; it may need interstitial information that was provided by other means in the actual course; it must be checked for copyright; it must be provided in a location and a format that permits easy access by others; and metadata must be added to help others fi nd more precisely the content that they are seeking. For greatest effi ciency, these procedures should be standardized across

disciplines and institutions, so that users need not master parochial