This thesis has investigated a small portion of the design space in the distributed transactional memory execution. There is a significant number of areas for future investigation. Those areas concern:
• Infrastructure Optimizations: DiSTM is a clustering JVM solution written in Java. The remote communication system is based on the ProAc- tive framework which is a high level API wrapper of Java RMI. Concerning ProActive, there is a variety of optimizations that can be exploited such as Group Communications and Immediate Services. With group commu- nications a stream is serialized once and sent to a group of nodes limiting serialization costs. DiSTM currently uses group communications but there is still room for further exploitation. With immediate services, active ob- jects can serve a request immediately bypassing the wait queue. Although
this might introduce consistency risks, if carefully used, it can speedup ex- ecution. DiSTM currently does not use immediate services as its primary focus was consistency. Concerning RMI which is inherently slow due to object marshaling and serialization, a variety of solutions exists which can boost performance significantly. For example, Java Fast Sockets (JFS) [95] bypass object serialization by directly mapping memory locations to the network interface. Furthermore, a more aggressive optimization would be the total replacement of Java’s network communication with MPI. In this way the modularity and flexibility of the programming interface of DiSTM would be maintained while increasing performance. ProActive currently supports wrapping of MPI code. In addition, moving to MPI could lead to network interconnects with higher bandwidth such as Myrinet or InfiniBand as it can support them.
• Tuning: Tuning DiSTM can be achieved in many ways. The most impor- tant tuning aspect is the Garbage Collector (GC). GC can lead to significant bottlenecks to Java execution. Especially in multi-threaded environments, the stop-the-world stages of the GC, can negatively affect performance. Currently DiSTM does not support any form of distributed GC. The GC process is performed per node with the standard HotSpot’s GCs. Extra care has been taken in order to avoid collection of vital transactional metadata. This resulted in DiSTM’s significant memory footprint (between 6GB and 15GB) which consequently led to increased stall times due to GCs. An- other important tuning parameter is the scheduling of the transactions. As shown, high abort rates in a distributed environment are fatal for per- formance. Techniques tackling pathologies of transactional memory (for example repeat conflicts, Chapter 5) exist [10, 17] and their application is orthogonal to DiSTM.
• Transactional Protocols: Concerning optimistic distributed transactional protocols, which was the main focus of this thesis, there is still a vast de- sign space for exploration. objects’ multiversioning, time-stamp ordering, contention managers and resolution policies are some of them.
• Applications: The applicability of systems and protocols such as the one proposed in this thesis can take place in the enterprise domain. Terra- cotta’s main target is the clustering of enterprise applications with minimal
CHAPTER 6. CONCLUSIONS & FUTURE WORK 125
alterations of the existing code. In addition, existing enterprise Java-based servers such as JBoss, WebSphere and GlassFish provide clustering of their instances in order to speed-up execution. Consequently, applying transac- tional protocols to existing products or porting existing enterprise applica- tions to DiSTM could provide a better insight of the key problems hurting performance as it would allow the evaluation against real-life workloads. Overall, the application of TM to distributed systems appears to be a promis- ing solution especially for those applications that are difficult to parallelize with lock based solutions. Although the distributed TM protocols, presented in this thesis, do not cope well with certain workloads, the applicability of the optimiza- tions described may enhance performance significantly. Furthermore, enterprise applications usually have loosely coupled datasets and therefore it is uncommon for highly contented workloads to exist. However, it is vital for any distributed TM system to be competitive with lock based systems in any workload. This might be impossible for certain cases as research in TM for ChipMultiprocessors has demonstrated until now. The ease in programmability of TM along with the continuous performance improvement may close that gap and finally TM will prevail against locks in multithreaded programming, distributed or not.
[1] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sun- deram. PVM 3: User’s guide and reference manual. Technical Report Oak Ridge National Laboratory, 1994.
[2] Marcos K. Aguilera, Arif Merchant, Mehul Shah, Alistair Veitch, and Chris- tos Karamanolis. Sinfonia: A new paradigm for building scalable dis- tributed systems. In SOSP ’07: Proceedings of 21st ACM SIGOPS Sympo-
sium on Operating Systems Principles, 2007.
[3] E. Allen, D. Chase, J. Hallett, V. Luchangco, J. Maessen, S. Ryu, G. Steele, and S. Tobin-Hochstadt. The Fortress language specification version 1.0.
Technical report, SUN Microsystems. 2008.
[4] C. Scott Ananian, Krste Asanovic, Bradley C. Kuszmaul, Charles E. Leis- erson, and Sean Lie. Unbounded transactional memory. In HPCA ’05: Pro-
ceedings of the 11th International Symposium on High-Performance Com- puter Architecture, 2005.
[5] Marcelo Lobosco Anderson and Claudio L. De Amorim. A New Distributed JVM for Cluster Computing. In Europar ’03: Proceedings of the 9th Euro-
par Conference, 2003.
[6] Mohammad Ansari. Efficient execution of concurrent applications using transactional memory. PhD Thesis, 2008.
[7] Mohammad Ansari, Behram Khan, Mikel Luj´an, Christos Kotselidis, Chris C. Kirkham, and Ian Watson. Steal-on-Abort: Improving Trans- actional Memory Performance through Dynamic Transaction Reordering. In HIPEAC ’10: Proceedings of the 5th International Conference on High
Performance and Embedded Architectures and Compilers, 2010.
BIBLIOGRAPHY 127
[8] Mohammad Ansari, Christos Kotselidis, Kim Jarvis, Mikel Luj´an, Chris Kirkham, and Ian Watson. Advanced concurrency control for transactional memory using transaction commit rate. In EUROPAR ’08: Proceedings of
the 14th European Conference on Parallel Processing, 2008.
[9] Mohammad Ansari, Christos Kotselidis, Kim Jarvis, Mikel Luj´an, Chris Kirkham, and Ian Watson. Lee-TM: A non-trivial Benchmark for Trans- actional Memory. In ICA3PP ’08: Proceedings of the 7th International
Conference on Algorithms and Architectures for Parallel Processing, 2008.
[10] Mohammad Ansari, Mikel Luj´an, Christos Kotselidis, Kim Jarvis, Chris Kirkham, and Ian Watson. Steal-on-abort: Improving transactional mem- ory performance through dynamic transaction reordering. In HIPEAC ’09:
Proceedings of the 4th International Conference on High Performance and Embedded Architectures and Compilers. LNCS, Springer, January 2009.
[11] Mohammad Ansari, Mikel Luj´an, Christos Kotselidis, Kim Jarvis, Chris Kirkham, and Ian Watson. Transaction reordering to reduce aborts in software transactional memory. Transactions on High Performance and
Embedded Architectures and Compilers, 4(4), 2009.
[12] Gabriel Antoniu, Luc Boug´e, Philip Hatcher, Mark MacBeth, Keith McGuigan, and Raymond Namyst. The Hyperion system: Compiling multi- threaded Java bytecode for distributed execution. Parallel Comput., 27(10), 2001.
[13] Apache Foundation. BCEL: Bytecode Engineering Library.
http://jakarta.apache.org/bcel/.
[14] Yariv Aridor, Michael Factor, and Avi Teperman. cJVM: A Single Sys- tem Image of a JVM on a Cluster. In ICPP ’09: Proceedings of the 28th
International Conference on Parallel Processing, 1999.
[15] AzulSystems. RMI: Remote Method Invocation.
[16] Laurent Baduel, Fran¸coise Baude, Denis Caromel, Arnaud Contes, Fabrice Huet, Matthieu Morel, and Romain Quilici. Programming, deploying, com- posing, for the grid. In Grid Computing: Software Environments and Tools, 2006.
[17] Tongxin Bai, Xipeng Shen, Chengliang Zhang, William N. Scherer, Iii Chen, Ding Michael, and L. Scott. A key-based adaptive transactional memory executor. In IPDPS ’07: Proceedings of the 21th International Parallel and
Distributed Symposium, 2007.
[18] Lee Baugh, Naveen Neelakantam, and Craig Zilles. Using hardware mem- ory protection to build a high-performance, strongly atomic hybrid transac- tional memory. In ISCA ’08: Proceedings of the 35th Annual International
Symposium on Computer Architecture, 2008.
[19] E.R. Berlekamp, J. H. Conway, and R. K. Guy. Winning Ways for your mathematical plays. New York, 1982. Academic Press.
[20] P. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. 1987.
[21] Annette Bieniusa and Thomas Fuhrmann. A Description Analysis of “dstm”, a Fully Decentralized Software Transactional Memory Algorithm. Technical report, Technical University of Munich, 2009.
[22] B. H. Bloom. Space/time trade-offs in hash coding with allowable errors. In Communications of the ACM, 13(7), 1970.
[23] Jayaram Bobba, Neelam Goyal, Mark D. Hill, Michael M. Swift, and David A. Wood. TokenTM: Efficient Execution of Large Transactions with Hardware Transactional Memory. In ISCA ’08: Proceedings of the 35th
Annual International Symposium on Computer Architecture, 2008.
[24] Robert L. Bocchino, Vikram S. Adve, and Bradford L. Chamberlain. Soft- ware transactional memory for large scale clusters. In PPoPP ’08: Proceed-
ings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming, 2008.
[25] D. Bonachea. GASNet specification, v1.1. Technical report, University of California at Berkeley, 2002.
[26] Jonas Bon´er and Eugene Kuleshov. Clustering the Java virtual machine using aspect-oriented programming. In AOSD ’07: Proceedings of the 6th
BIBLIOGRAPHY 129
[27] Bryan Carpenter, Guansong Zhang, Geoffrey Fox, Xinying Li, and Yuhong Wen. HPJava: Data parallel extensions to Java. 1998.
[28] J. Cachopo and A. Rito-Silva. Versioned boxes as the basis of transactional memory systems. In Sci. Comput. Program, vol 63, no. 2, 2006.
[29] David Callahan, Bradford L. Chamberlain, and Hans P. Zima. The Cascade High Productivity Language. In HIPS 2004: 9th International Workshop
on High-Level Programming Models and Supportive Environments, 2004.
[30] N. Carvalho, J. Pereira, and L. Rodrigues. Towards a generic group com- munication service. In DOA ’06: Proceedings of the 8th International Sym-
posium on Distributed Objects and Applications, 2006.
[31] Philippe Charles, Christian Grothoff, Vijay Saraswat, Christopher Donawa, Allan Kielstra, Kemal Ebcioglu, Christoph von Praun, and Vivek Sarkar. X10: An object-oriented approach to non-uniform cluster computing. In
OOPSLA ’05: Proceedings of the 20th Annual ACM SIGPLAN Conference on Object Oriented Programming, Systems, Languages, and Applications,
2005.
[32] M. Couceiro, P. Romano, N. Carvalho, and L. Rodrigues. D2STM: Depend- able distributed software transactional memory. In PRDC ’09: Proceedings
of the 15th Pacific Rim International Symposium on Dependable Comput- ing, 2009.
[33] Peter Damron, Alexandra Fedorova, Yossi Lev, Victor Luchangco, Mark Moir, and Daniel Nussbaum. Hybrid Transactional Memory. In ASPLOS-
XII: Proceedings of the 12th International Conference on Architectural Sup- port for Programming Languages and Operating Systems, 2006.
[34] Alokika Dash and Brian Demsky. Software Transactional Distributed Shared Memory. Technical report, University of California at Irvine, 2009. [35] Xavier D´efago, Andr´e Schiper, and P´eter Urb´an. Total order broadcast and multicast algorithms: Taxonomy and survey. ACM Comput. Surv., 2004. [36] Dave Dice, Ori Shalev, and Nir Shavit. Transactional locking II. In DISC
’06: Proceedings of the 20th International Symposium on Distributed Com- puting, 2006.
[37] T. El-Ghazawi, W. Carlson, T. Sterling, and K. Yelick. UPC: Distributed shared-memory programming. Wiley-Interscience, 2005.
[38] Robert Ennals. Efficient software transactional memory. Technical report, Intel Research Cambridge, 2005.
[39] EPCC. Java Grande Benchmark Suite.
http://www.epcc.ed.ac.uk/research/activities/java-grande/.
[40] Michael Factor, Assaf Schuster, and Konstantin Shagin. JavaSplit: A Run- time for Execution of Monolithic Java Programs on Heterogeneous Collec- tions of Commodity Workstations. In Cluster Computing, IEEE Interna-
tional Conference on, 2003.
[41] Marc Fleury and Franscisco Reverbel. The JBoss extensible server. In Mid-
dleware ’03: Proceedings of the ACM/IFIP/USENIX 2003 International Conference on Middleware, 2003.
[42] M.J. Flynn and P. Hung. Microprocessor design issues: Thoughts on the road ahead. IEEE Micro, 25(3):16–31, 2005.
[43] Message Passing Interface Forum. MPI: A message-passing interface stan- dard. Technical Report UT-CS-94-230, 1994.
[44] Apache Foundation. Apache Tomcat.
[45] Keir Fraser and Tim Harris. Concurrent programming without locks. ACM
Trans. Comput. Syst., 2007.
[46] James Gosling, Bill Joy, and Guy Steele. The Java language specification. Addison-Wesley, 2000.
[47] C. Gray and D. Cheriton. Leases: An efficient fault-tolerant mechanism for distributed file cache consistency. In SOSP ’89: Proceedings of the 12th
ACM Symposium on Operating Systems Principles, 1989.
[48] Rachid Guerraoui, Maurice Herlihy, Michal Kapalka, and Bastian Pochon. Robust Contention Management in Software Transactional Memory. In
SCOOL ’05: Workshop on Synchronization and Concurrency in Object- Oriented Languages, 2005.
BIBLIOGRAPHY 131
[49] Rachid Guerraoui, Maurice Herlihy, and Bastian Pochon. Toward a Theory of Transactional Contention Managers. In PODC ’05: Proceedings of the
24th Annual Symposium on Principles of Distributed Computing, 2005.
[50] Rachid Guerraoui, Micha l Kapa lka, and Jan Vitek. STMBench7: A bench- mark for software transactional memory. In EuroSys ’07: Proceedings of
the 2nd European Systems Conference, 2007.
[51] Theo Haerder and Andreas Reuter. Principles of transaction-oriented database recovery. In ACM Computing Surveys, 1983.
[52] Lance Hammond, Vicky Wong, Mike Chen, Brian D. Carlstrom, John D. Davis, Ben Hertzberg, Manohar K. Prabhu, Honggo Wijaya, Christos Kozyrakis, and Kunle Olukotun. Transactional memory coherence and consistency. In ISCA ’04: Proceedings of the 31st Annual International
Symposium on Computer Architecture, 2004.
[53] Tim Harris and Keir Fraser. Language support for lightweight transac- tions. In OOPSLA ’03: Proceedings of the 18th Annual ACM SIGPLAN
conference on Object-Oriented Programing, Systems, Languages, and Ap- plications, 2003.
[54] Rutger Hofman Ceriel Jacobs Koen Langendoen Tim Ruhl Henri E. Bal, Raoul Bhoedjang and M. Frans Kaashoek. Performance Evaluation of the Orca Shared-Object System. 1998.
[55] M. Herlihy and Y. Sun. Distributed Transactional Memory for Metric-Space Networks. In DISC ’05: Proceedings of the 19th International Symposium
on Distributed Computing, 2005.
[56] Maurice Herlihy, Victor Luchangco, and Mark Moir. A flexible framework for implementing software transactional memory. In OOPSLA ’06: Pro-
ceedings of the 18th Annual ACM SIGPLAN conference on Object-Oriented Programing, Systems, Languages, and Applications, 2006.
[57] Maurice Herlihy, Victor Luchangco, Mark Moir, and William Scherer III. Software transactional memory for dynamic-sized data structures. In PODC
’03: Proceedings of the 22nd annual symposium on Principles of distributed computing, 2003.
[58] Maurice Herlihy and J. Eliot B. Moss. Transactional Memory: Architectural Support for Lock-free Data Structures. In ISCA ’93: Proceedings of the 20th
Annual International Symposium on Computer Architecture, 1993.
[59] High Performance Fortran Forum. High Performance Fortran language specification, version 1.0. Scientific Programming, 1993.
[60] IBM. Websphere application server.
[61] Pete Keleher, Alan L. Cox, Hya Dwarkadas, and Willy Zwaenepoel. Tread- marks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the 1994 Winter Usenix Conference, 1994. [62] Christos Kotselidis, Mohammad Ansari, Kim Jarvis, Mikel Luj´an, Chris
Kirkham, and Ian Watson. DiSTM: A Software Transactional Memory Framework for Clusters. In ICPP ’08: Proceedings of the 37th International
Conference on Parallel Processing, 2008.
[63] Christos Kotselidis, Mohammad Ansari, Kim Jarvis, Mikel Luj´an, Chris Kirkham, and Ian Watson. Investigating Software Transactional Memory on Clusters. In IWJPDC ’08: 10th International Workshop on Java and
Components for Parallelism, Distribution and Concurrency, 2008.
[64] Christos Kotselidis, Mikel Luj´an, Mohammad Ansari, Konstantinos Malakasis, Behram Khan, Chris Kirkham, and Ian Watson. Clustering JVMs with Software Transactional Memory Support. In IPDPS ’10: Pro-
ceedings of the 24th International Parallel and Distributed Symposium,
2010.
[65] James R. Larus and Ravi Rajwar. Transactional Memory. Morgan and Claypool, 2006.
[66] Yossi Lev and Jan-Willem Maessen. Split hardware transactions: True nesting of transactions using best-effort hardware transactional memory. In PPoPP ’08: Proceedings of the 13th ACM SIGPLAN Symposium on
Principles and practice of parallel programming, 2008.
[67] Yossi Lev, Mark Moir, and Dan Nussbaum. PhTM: Phased transactional memory. In TRANSACT ’07: Proceedings of the 2nd ACM SIGPLAN
BIBLIOGRAPHY 133
[68] D. B. Lomet. Process structuring, synchronization, and recovery using atomic actions. In Proceedings of an ACM conference on Language design
for reliable software, 1977.
[69] M. Baker, B. Carpenter, and A. Shafi. MPJ Express: Towards Thread Safe Java HPC. In 2006 IEEE International Conference on Cluster Computing, 2006.
[70] M. Baker, B. Carpenter, G.Fox, and S.H. Koo. mpiJava: An Object- Oriented Java interface to MPI. In Workshop on Java for Parallel and
Distributed Computing, IPPS/SPDP, 1999.
[71] Matchy J. M. Ma, Cho-Li Wang, and Francis C. M. Lau. JESSICA: Java- Enabled Single-System-Image Computing Architecture. In Journal of Par-
allel and Distributed Computing, 2000.
[72] Kaloian Manassiev, Madalin Mihailescu, and Cristiana Amza. Exploit- ing distributed version concurrency in a transactional memory cluster. In
PPoPP ’06: Proceedings of the 11th ACM SIGPLAN symposium on Prin- ciples and practice of parallel programming, 2006.
[73] Virendra Marathe, Michael Spear, Christopher Herio, Athul Acharya, David Eisenstat, William Scherer III, and Michael L. Scott. Lowering the overhead of software transactional memory. In TRANSACT ’06: 1st ACM
SIGPLAN Workshop on Transactional Computing, June 2006.
[74] D. Matzke. Will physical scalability sabotage performance gains? IEEE
Computer, 30(9):37–39, 1997.
[75] Sun Microsystems. CORBA: Common Object Request Broker Architecture. [76] Sun Microsystems. http://www.azulsystems.com.
[77] Chi Cao Minh, Martin Trautmann, JaeWoong Chung, Austen McDonald, Nathan Bronson, Jared Casper, Christos Kozyrakis, and Kunle Olukotun. An effective hybrid transactional memory system with strong isolation guar- antees. In ISCA ’07: Proceedings of the 34th Annual International Sympo-
[78] G.E. Moore. Cramming more components onto integrated circuits. Elec-
tronics, pages 114–117, 1965.
[79] R.W. Numrich and J.K. Reid. co-Array Fortran for Parallel Programming. ACM Fortran Forum, volum 17, 1998.
[80] Kunle Olukotun and Lance Hammond. The future of microprocessors.
Queue, 2005.
[81] Kunle Olukotun, Basem A. Nayfeh, Lance Hammond, Ken Wilson, and Kunyung Chang. The case for a single-chip multiprocessor. In ASPLOS-
VII: Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, 1996.
[82] Michael Philippsen and Matthias Zenger. JavaParty: Transparent Remote Objects in Java. Concurrency: Practice and Experience, 1997.
[83] Veldema R., Bhoedjang R., and Bal H. Distributed Shared Memory for Java. Technical report.
[84] D.P. Reed. Implementing atomic actions on decentralized data. ACM
Trans. Comput. Syst., 1983.
[85] P. Romano, N. Carvalho, L. Rodrigues, and J. Cachopo. Cloud-TM: Har- nessing the Cloud with Distributed Transactional Memories. In LADIS ’09:
Proceedings of the 3rd International Workshop on Large-Scale Distributed Systems and Middleware, 2009.
[86] F. Rubin. The Lee path connection algorithm. IEEE Transactions on
Computers, 1974.
[87] Bratin Saha, Ali-Reza Adl-Tabatabai, Richard L. Hudson, Chi Cao Minh, and Benjamin Hertzberg. McRT-STM: A high performance software trans- actional memory system for a multi-core runtime. In PPoPP ’06: Proceed-
ings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2006.
[88] William Scherer III and Michael L. Scott. Contention Management in Dy- namic Software Transactional Memory. In CSJP ’04: Workshop on Con-
BIBLIOGRAPHY 135
[89] William Scherer III and Michael L. Scott. Advanced contention manage- ment for dynamic software transactional memory. In PODC ’05: Proceed-
ings of the 24th Annual Symposium on Principles of Distributed Computing,
2005.
[90] Nir Shavit and Dan Touitou. Software transactional memory. In PODC
’95: Proceedings of the 14th Annual ACM Symposium on Principles of Distributed Computing, 1995.
[91] SPEC. SPECjbb2000. http://www.spec.org/jbb2000/.
[92] SPLASH-2. Stanford Parallel Applications for Shared Memory.
http://www-flash.stanford.edu/apps/SPLASH/.
[93] S. Sridharan, J.S Vetter, and P.M Kogger. Scalable Software Transactional Memory for Global Address Space Architectures. Technical report, Oak Ridge National Laboratory, 2009.
[94] Fuad Tabba, Cong Wang, James R. Goodman, and Mark Moir. NZTM: Nonblocking, Zero-Indirection Transactional Memory. In TRANSACT ’07:
Second ACM SIGPLAN Workshop on Transactional Computing, 2007.
[95] Guillermo L. Taboada, Juan Touri no, and Ram´on Doallo. Java fast sockets: Enabling high-speed java communications on high performance clusters. In
Comput. Commun., volume 31, 2008.
[96] Ian Watson, Chris Kirkham, and Mikel Luj´an. A study of a transactional parallel routing algorithm. In PACT ’07: Proceedings of the 16th Inter-
national Conference on Parallel Architectures and Compilation Techniques,
2007.
[97] Kathy Yelick, Luigi Semenzato, Geoff Pike, Carleton Miyamoto, Ben Liblit, Arvind Krishnamurthy, Paul Hilfinger, Susan Graham, David Gay, Phil Colella, and Alex Aiken. Titanium: A high-performance Java dialect. 1998. [98] Luke Yen, Jayaram Bobba, Michael M. Marty, Kevin E. Moore, Haris Volos, Mark D. Hill, Michael M. Swift, and David A. Wood. LogTM-SE: Decou- pling hardware transactional memory from caches. In HPCA ’07: Proceed-
ings of the 13th International Symposium on High-Performance Computer Architecture, 2007.
[99] Weimin Yu and Alan Cox. Java/DSM: A platform for heterogeneous com- puting.
[100] Yuanyuan Zhou, Liviu Iftode, and Kai Li. Performance evaluation of two home-based lazy release consistency protocols for shared virtual memory systems. In OSDI ’96: Proceedings of the second USENIX symposium on
Operating systems design and implementation, 1996.
[101] Wenzhang Zhu, Cho-Li Wang, and Francis C.M. Lau. JESSICA2: A Dis-