International Journal of High Performance Computing Applications recent issues

The International Exascale Software Project: a Call To Cooperative Action By the Global High-Performance Community

Thu, 22 Oct 2009 02:11:07 PDT

Over the last 20 years, the open-source community has provided more and more software on which the world’s high-performance computing systems depend for performance and productivity. The community has invested millions of dollars and years of effort to build key components. Although the investments in these separate software elements have been tremendously valuable, a great deal of productivity has also been lost because of the lack of planning, coordination, and key integration of technologies necessary to make them work together smoothly and efficiently, both within individual petascale systems and between different systems. A repository gatekeeper and an email discussion list can coordinate open-source development within a single project, but there is no global mechanism working across the community to identify critical holes in the overall software environment, spot opportunities for beneficial integration, or specify requirements for more careful coordination. It seems clear that this completely uncoordinated development model will not provide the software needed to support the unprecedented parallelism required for peta/exascale computation on millions of cores, or the flexibility required to exploit new hardware models and features, such as transactional memory, speculative execution, and GPUs. We believe the community must work together to prepare for the challenges of exascale computing, ultimately combing their efforts in a coordinated International Exascale Software Project.

Summary of the IESP White Papers

Mohr, B. — Thu, 22 Oct 2009 02:11:07 PDT

As part of the planning and development process for the IESP workshops, we asked participants to prepare white papers on various topics. We have grouped and summarized the content of the white papers that were contributed here. The list of the white papers relevant to a given group is given at the end of the summary for that group.

Programmability Issues

Chapman, B., Labarta, J., Sarkar, V., Sato, M. — Thu, 22 Oct 2009 02:11:07 PDT

Programming models are at the very center of our effort to address the exascale challenge. They are the key interface that will allow the separation of the programmers’ concerns from those of system designers, potentially at different levels of granularity. Any such model must meet the extensive needs of application developers and be supported by the entire software stack. Considerable research is needed to define and implement the programming and execution models for such systems. Whereas evolutionary approaches may best support the migration of existing application software, revolutionary models may be best suited to providing extreme-scale performance for new applications on emerging architectures. Both approaches should be explored.

Models of Computation -- Enabling Exascale

Sterling, T. — Thu, 22 Oct 2009 02:11:07 PDT

A model of computation provides a set of governing principles and a discipline determining the objects, their interrelationships, parallel flow control, and distribution. In so doing it provides a conceptual framework for the co-design and operation for all layers of the software and hardware architecture system stack from programming model through system software to processor core instruction set and structures. As enabling technologies advance changing capabilities, balance, and design challenges, the response has been a phase change in how HPC systems are realized reflected by the development and adoption of new models of computation to best fit the emerging underlying technology conditions. This has occurred at least through five technology epochs in the past and the field is entering a sixth such with the heterogeneous multicore Petaflops decade. A brief presentation of the nature and characteristics of models of computation is offered here to contribute to the current community discussions on how to proceed towards the realization of the promise of practical Exascale computing by the end of the next decade.

The Biggest Need: a New Model of Computation

Sterling, T. — Thu, 22 Oct 2009 02:11:07 PDT

The development of HPC systems capable of exascale performance will demand innovations in hardware architecture and system software as well as programming models and methods. The combination of a vast increase in scale of such systems combined with the emergence of heterogeneous multicore structures is forcing future systems to be organized and operate on principles very different from the conventional practices of the last two decades. The biggest need for achieving exascale computing is the development of a new model of computation that will provide the new paradigm exploiting the opportunities and addressing the challenges of the emerging technologies and advanced architectures that together will comprise future exascale systems. This brief paper discusses the need for a new model of computation and the properties such a model should embody.

Slouching Towards Exascale

Lusk, E. — Thu, 22 Oct 2009 02:11:07 PDT

One question before the high-performance computing community is "How will application developers write code for exascale machines?" At this point it looks like they might be riding a rough beast indeed. This paper is a brief assessment of where we stand now with respect to writing programs for our largest supercomputers and what we should do next. MPI is likely to remain a critical part of the programming infrastructure as we move towards exascale, but more is needed, in particular a robust, portable, and effective standard for parallel programming within a single address space, perhaps for heterogeneous processors. Formal methods provide the only truly scalable approach to developing correct code in this complex programming environment.

BSC Vision Towards Exascale

Labarta, J., Ayguade, E., Valero, M. — Thu, 22 Oct 2009 02:11:07 PDT

This paper provides the vision of the Barcelona Supercomputing Center towards exascale computing. We believe that it is key to have unified views of future computer systems, looking at the good ideas, developments, and practices from the past and applying them at the scalability levels we want to consider. The programming model is of Alexander’s sword used to break the Gordian knot of exascale systems based on massive multicore architectures. The implementation of the programming model should decouple the way programs are written by the user (parallelism, address spaces, etc.) and executed by the runtime (execution vehicles, memory containers, malleability and load balancing, fault tolerance, etc.) on a specific target architecture. At the application level, it will be crucial to ensure that application porting is going to guarantee their survival for some decades or their clean upgrade to the foreseeable explosion of hardware platforms. Performance tools and analysis practices are in their infancy with regard to providing the required exascale support. BSC would like to contribute with this vision and ongoing efforts to the holistic exascale initiative.

Programming Models at Exascale: Adaptive Runtime Systems, Incomplete Simple Languages, and Interoperability

Kale, L. — Thu, 22 Oct 2009 02:11:07 PDT

Applications running on exascale machines will be complex in many ways. They will involve dynamic and adaptive refinements, and will be composed of multiple, independently developed modules, often involving a multiphysics simulation. The programming models of this era must have several characteristics. First, they need to do away with the notion of processors, and automate resource management via adaptive runtime systems. Data structure-specific frameworks and domain-specific environments will be needed to further simplify programming. More importantly, parallel mini-languages need to be developed, such that each language captures only a restricted subset of possible parallel interactions, but allows for a simple expression of them. Coupled with interoperability and parallel composition, which must be supported in many ways, including message-driven runtime systems, this will create a productive ecosystem of parallel programming models for the exascale era.

Resource Management

Maccabe, A., Falter, H., Kramer, W. — Thu, 22 Oct 2009 02:11:07 PDT

Application scalability is directly related to the ability of the application developer to effectively use the resources provided by a computing system. As we start to address the development of exascale platforms, we must engage in a dialog to define the terms related to resource management. Approaches to resource management can be categorized in two dimensions: static/dynamic and explicit/ implicit. The static/dynamic dimension refers to when resource management decisions are made: prior to program execution or during program execution. The implicit/ explicit dimension refers to the object that implements the decision making: the tools that implement the programming environment or the application developer. The development of applications that can scale to the resources provided by an exascale system will require tools that allow programmers to move easily and seamlessly between these dimensions as they express resource management decisions.

The Case for a Hierarchical System Model for Linux Clusters

Seager, M., Gorda, B. — Thu, 22 Oct 2009 02:11:07 PDT

Performance at Exascale

Mohr, B., Muller, M. S., Nagel, W. E. — Thu, 22 Oct 2009 02:11:07 PDT

Exascale systems will consist of complex configurations with a huge number of potentially heterogeneous components. Deep software hierarchies of large, complex software components will be required to make use of such systems. While the software layers are designed to be transparent, they are typically not transparent with respect to performance. This performance non-transparency will result in escalation of unforeseen problems to higher layers, including the application. Altogether this will require an integrated and collaborative approach to handling performance issues and correctly detecting and analyzing performance problems. Contributions to this integrated approach are required from areas such as system design, operating software, and programming models, as well as performance modeling, monitoring, and analysis.

On the Importance of End-to-End Application Performance Monitoring and Workload Analysis at the Exascale

Skinner, D., Choudary, A. — Thu, 22 Oct 2009 02:11:07 PDT

This paper sets out to examine the future of performance monitoring on exascale HPC systems. In particular we put forth the idea that such machines will be sufficiently complex that performance monitoring of individual applications and the workload as a whole will change from being a beneficial option to being a necessity. This complexity arises from the number of components and concurrencies expected for such systems. We see the need for a shift from performance monitoring being a useful add-on toward it being a core requirement for basic operation and suggest some first steps toward meeting that need.

XXL Simulation for XXIst Century Power Systems Operation

Berthou, J.-Y., Hamelin, J.-F., de Rocquigny, E. — Thu, 22 Oct 2009 02:11:07 PDT

As an industrial user with very high stakes in the operation and maintenance of complex systems such as nuclear power plants, EDF has been engaged in simulation for many years. We feel that exaflops software should not only be thought of as a way of tackling daunting research problems but should also take into account the sometimes equally daunting requirements that stem from an industrial usage perspective. We do feel that, whatever the hard changes that will probably have to be made to various software aspects, we should not lose sight of the fact that continuity paths also have to be found in order to make those big changes acceptable and profitable to many. We identify in this paper what we think are priority research themes that could benefit from an international collaboration.

Partial Differential Equation-Based Applications and Solvers At Extreme Scale

Keyes, D. — Thu, 22 Oct 2009 02:11:07 PDT

Partial differential equation-based applications of multi-scale, multiphysics phenomena have driven the quest for extreme architectural scales since the foundation of modern digital computing and will continue to be principal among a broader set of science drivers for the foreseeable future. However, scientific and engineering drivers ceased long ago to dominate the computing industry and any commercially viable path to the petascale would seem to be through architectures that are assembled from components designed without the balance of resources required by scientific and engineering simulations foremost in mind. Concurrency will be massive and will involve many cores sharing common memory at the finest scales and severely dividing available memory bandwidth. As a result, algorithm designers will have to look beyond the message-passing-based SPMD paradigm that dominates today’s most successful large-scale applications and solver frameworks, with stronger than ever emphasis on locality or operands and synchronization avoidance.

Application Analysis and Porting in the PRACE Project

Michielse, P. — Thu, 22 Oct 2009 02:11:07 PDT

This paper reports on the construction of a benchmark suite, to be used both within the current PRACE project and beyond, when actual Tier-0 systems will be purchased. PRACE targets toward a European research infrastructure, ideally consisting of various hardware architectures. This implicitly means that some applications are more suited to certain architectures than others. This needs to be reflected in the final benchmark suite, with the idea that potentially subsets of the overall benchmark suite may be used for benchmarking different architectures. The paper gives insight into the ideas behind the construction of the benchmark suite, including the current status of it.

Toward Exascale Resilience

Cappello, F., Geist, A., Gropp, B., Kale, L., Kramer, B., Snir, M. — Thu, 22 Oct 2009 02:11:07 PDT

Over the past few years resilience has became a major issue for high-performance computing (HPC) systems, in particular in the perspective of large petascale systems and future exascale systems. These systems will typically gather from half a million to several millions of central processing unit (CPU) cores running up to a billion threads. From the current knowledge and observations of existing large systems, it is anticipated that exascale systems will experience various kind of faults many times per day. It is also anticipated that the current approach for resilience, which relies on automatic or application level checkpoint/ restart, will not work because the time for checkpointing and restarting will exceed the mean time to failure of a full system. This set of projections leaves the community of fault tolerance for HPC systems with a difficult challenge: finding new approaches, which are possibly radically disruptive, to run applications until their normal termination, despite the essentially unstable nature of exascale systems. Yet, the community has only five to six years to solve the problem. This white paper synthesizes the motivations, observations and research issues considered as determinant of several complimentary experts of HPC in applications, programming models, distributed systems and system management.

An Exascale Approach to Software and Hardware Design

Kramer, W., Skinner, D. — Thu, 22 Oct 2009 02:11:07 PDT

For the past 10—15 years, horizontal layers of software and hardware design and development have been the de facto standard of creating high-performance computing (HPC) software. The horizontal design approach leads to the development of discrete components in the software stack and independent hardware components — all developed with different methods, requirements and quality dominated by plug-and-play componentization that is focused on horizontal functionality and portability. The horizontal software paradigm will break down at the exascale due to the system scale and complexity. The vertical approach needed for the exascale should include resilience (reliability and fault tolerance); performance; programmability; computational models; I/O; consistency and verification; resource management; and power management/total cost of ownership. To make the exascale an effective reality, instead of thinking of integration as the final step in defining and developing an exascale system, it will have to be the first step.

Consistent Application Performance at the Exascale

Kramer, W., Skinner, D. — Thu, 22 Oct 2009 02:11:07 PDT

We examine the future of application performance consistency on exascale parallel computing systems. By performance consistency we mean the regularity of wall clock times to complete a fixed amount of application progress. Exascale systems will have dramatically increased complexity along with their capability. Contributors to inconsistency include architectural choices, software functions and subtle interactions, and inconsistency will lead to lost potential. The challenge is how to maintain consistency at the exascale. In order for exascale systems to exhibit the consistency that is required to make the applications and systems productive, a new understanding of the causes and solutions to inconsistency is required, along with new ways of measuring the impact that design, implementation and operational choices have on consistency.

A Collaboration and Commercialization Model for Exascale Software Research

Seager, M., Gorda, B. — Thu, 22 Oct 2009 02:11:07 PDT

We propose a coordinated strategy for exascale software development that includes the incorporation of successful research and development (R&D) into development and engineering (D&E) projects and harvesting the successful D&E projects into products with vendor support (P&S). This allows the most flexible R&D agenda while at the same time providing a commercialization path. This process is described as a natural extension of current focus areas and funding agents for R&D, D&E and P&S, but adds stake holders from the next stage in the process in the upstream processes. This model allows the flexibility to encourage development and competition of ideas in the research, development and productization phases. We anticipate that multiple iterations through this process from R&D through P&S are required to achieve appropriate software for Exascale systems.

Towards Exascale Distributed Data Management

Aloisio, G., Fiore, S. — Thu, 22 Oct 2009 02:11:07 PDT

"Exascale eScience infrastructures" will face important and critical challenges, both from computational and data perspectives. Increasingly complex and parallel scientific codes will lead to the production of a huge amount of data. The large volume of data and the time needed to locate, access, analyze and visualize data will greatly impact on the scientific productivity of scientists and researchers in several domains. Significant improvements in the data management field will increase research productivity in solving complex scientific problems. Next-generation eScience infrastructures will start from the assumption that exascale high-performance computing (HPC) applications (running on million of cores) will generate data at a very high rate (terabytes/s). Hundreds of exabytes of data (distributed across several centers) are expected, by 2020, to be available through heterogeneous storage resources for access, analysis, post-processing and other scientific activities.

IESP Exascale Challenge: Co-Design of Architectures and Algorithms

Geist, A., Dosanjh, S. — Thu, 22 Oct 2009 02:11:07 PDT

There is a large gap between the peak performance of supercomputers and the actual performance realized by today’s algorithms. This architecture—algorithm performance gap will get even wider with the increase in computing power being driven by a rapid escalation in the number of cores incorporated into a single chip rather than increases in the clock rate. In order to improve the effectiveness of peta and exascale systems we need to have a paradigm shift where architectures and algorithms are co-designed.

The Application Perspective: Seeking Productivity and Performance

Barkai, D. — Thu, 22 Oct 2009 02:11:07 PDT

In this note we propose two projects: (1) creating a hierarchical programming model from current models; and (2) extracting application primitives from the "13 dwarfs". The first topic addresses the need for a unified and manageable framework for very large-scale concurrent execution. This is the productivity part: less complexity will drive better mapping of algorithms to architecture, which will also contribute to better performance. The second topic focuses mostly on the processor and the node with the aim of laying the groundwork for software and silicon optimized kernels. While it is understood that applications primitives are outside the scope of IESP, the motivation for introducing it here is that it is a companion issue and that increasing the efficiency of each processor provides high return for science, at all levels of system size.

Musings on the Path Forward to Exascale

Lucas, R. F — Thu, 22 Oct 2009 02:11:07 PDT

Early Application Development/Tuning and Application Characterization/ Segmentation

Kale, L. — Thu, 22 Oct 2009 02:11:07 PDT

This position paper makes two separate, only tangentially related points:

1. A research program is necessary to produce tools and techniques that allow application developers to develop and tune applications for future exascale machines, well before the machines are deployed.

2. In order to cope with, or facilitate, segmentation of exascale architectures, it is necessary to carry out an extensive study characterizing the needs and behaviors of applications that are expected to run on exascale machines.

On the Need for a Consortium of Capability Centers

Gropp, W., Snir, M. — Thu, 22 Oct 2009 02:11:07 PDT

Users of high-performance computing systems face many challenges, particularly as they design and develop their software to run at multiple facilities. This can lead to a "greatest common denominator" strategy that slows innovation and the adoption of newer techniques. In addition, these systems typically push the limits — leading to problems with reliability and functionality of the system and software. We propose a consortium of HPC centers to collaborate on raising both the capability and the quality of the software for HPC systems.

Exascale Software: Some Questions to Drive the Development

Patra, A., Pennington, R., Seidel, E. — Thu, 22 Oct 2009 02:11:07 PDT

Developing a High-Performance Computing/Numerical Analysis Roadmap

Trefethen, A., Higham, N., Duff, I., Coveney, P. — Thu, 22 Oct 2009 02:11:07 PDT

A roadmap activity in the UK has leveraged US and European efforts for identifying the challenges and barriers in the development of high-performance computing (HPC) algorithms and software. The activity has identified the Grand Challenge to provide:

1. Algorithms and software that application developers can reuse in the form of high-quality, high performance, sustained software components, libraries and modules

2. A community environment that allows the sharing of software, communication of interdisciplinary knowledge and the development of appropriate skills.

Through a series of workshops and discussions with UK HPC application groups and numerical analysts, five areas of challenge have emerged.

Major Computer Science Challenges At Exascale

Geist, A., Lucas, R. — Thu, 22 Oct 2009 02:11:07 PDT

Exascale systems will provide an unprecedented opportunity for science, one that will make it possible to use computation not only as a critical tool along with theory and experiment in understanding the behavior of the fundamental components of nature, but also for critical advances for the nation’s energy needs and security. To create exascale systems and software that will enable the US Department of Energy (DOE) to meet the science goals critical to the nation’s energy, ecological sustainability, and global security, we must focus on major architecture, software, algorithm, and data challenges, and build on newly emerging programming environments. Only with this new infrastructure will applications be able to scale up to the required levels of parallelism and integrate technologies into complex coupled systems for real-world multidisciplinary modeling and simulation. Achieving this goal will likely involve a shift from current static approaches for application development and execution to a combination of new software tools, algorithms, and dynamically adaptive methods.

Software Challenges for Extreme Scale Computing: Going From Petascale to Exascale Systems

Heroux, M. A. — Thu, 22 Oct 2009 02:11:07 PDT

Preparing applications for a transition from petascale to exascale systems will require a very large investment in several areas of software research and development. The introduction of manycore nodes, the abundance of parallelism, an increase in system faults (including soft errors) and a complicated, multi-component software environment are some of the most challenging issues we face. In this paper we address four topics we believe to be the most the challenging issues and therefore the greatest opportunities for making effective next-generation scalable applications. First and foremost is the need to transform existing applications to run on manycore platforms and properly design new applications. This is particularly challenging in the absence of a standard, portable manycore programming environment, but we can make progress in this direction while manycore programming models are developed. Second is promoting advanced modeling and simulation capabilities such as embedded optimization and uncertainty quantification that lead to higher quality results and orders of magnitude more parallelism. Third is progress toward fault resilience in applications, a critical need as system reliability degrades. Fourth and finally is a qualitative improvement in software design, including the social aspects, as exascale software systems will be increasingly multi-team and multi-faceted efforts.

Editorial

Dongarra, J., Tourancheau, B. — Mon, 20 Jul 2009 04:07:45 PDT

Hybrid Message-Passing and Shared-Memory Programming in a Molecular Dynamics Application On Multicore Clusters

Chorley, M. J., Walker, D. W., Guest, M. F. — Mon, 20 Jul 2009 04:07:45 PDT

Hybrid programming, whereby shared-memory and message-passing programming techniques are combined within a single parallel application, has often been discussed as a method for increasing code performance on clusters of symmetric multiprocessors (SMPs). This paper examines whether the hybrid model brings any performance benefits for clusters based on multicore processors. A molecular dynamics application has been parallelized using both MPI and hybrid MPI/OpenMP programming models. The performance of this application has been examined on two high-end multicore clusters using both Infiniband and Gigabit Ethernet interconnects. The hybrid model has been found to perform well on the higher-latency Gigabit Ethernet connection, but offers no performance benefit on low-latency Infiniband interconnects. The changes in performance are attributed to the differing communication profiles of the hybrid and MPI codes.

Fault Tolerance in Petascale/ Exascale Systems: Current Knowledge, Challenges and Research Opportunities

Cappello, F. — Mon, 20 Jul 2009 04:07:45 PDT

The emergence of petascale systems and the promise of future exascale systems have reinvigorated the community interest in how to manage failures in such systems and ensure that large applications, lasting several hours or tens of hours, are completed successfully. Most of the existing results for several key mechanisms associated with fault tolerance in high-performance computing (HPC) platforms follow the rollback—recovery approach. Over the last decade, these mechanisms have received a lot of attention from the community with different levels of success. Unfortunately, despite their high degree of optimization, existing approaches do not fit well with the challenging evolutions of large-scale systems. There is room and even a need for new approaches. Opportunities may come from different origins: diskless checkpointing, algorithmic-based fault tolerance, proactive operation, speculative execution, software transactional memory, forward recovery, etc. The contributions of this paper are as follows: (1) we summarize and analyze the existing results concerning the failures in large-scale computers and point out the urgent need for drastic improvements or disruptive approaches for fault tolerance in these systems; (2) we sketch most of the known opportunities and analyze their associated limitations; (3) we extract and express the challenges that the HPC community will have to face for addressing the stringent issue of failures in HPC systems.

Adaptive Fault Tolerance for Scalable Cluster Computing in Space

James, M. L., Shapiro, A. A., Springer, P. L., Zima, H. P. — Mon, 20 Jul 2009 04:07:45 PDT

Future missions of deep-space exploration face the challenge of building more capable autonomous spacecraft and planetary rovers. Given the communication latencies and bandwidth limitations for such missions, the need for increased autonomy becomes mandatory, along with the requirement for enhanced on-board computational capabilities while in deep-space or time-critical situations. This will result in dramatic changes in the way missions are conducted and supported by on-board computing systems. Specifically, the traditional approach of relying exclusively on radiation-hardened hardware and modular redundancy will not be able to deliver the required computational power. As a consequence, such systems are expected to include high-capability low-power components based on emerging commercial-off-the-shelf (COTS) multi-core technology. In this paper we describe the design of a generic framework for introspection that supports runtime monitoring and analysis of program execution as well as a feedback-oriented recovery from faults. Our focus is on providing flexible software fault tolerance matched to the requirements and properties of applications by exploiting knowledge that is either contained in an application knowledge base, provided by users, or automatically derived from specifications. A prototype implementation is currently in progress at the Jet Propulsion Laboratory, California Institute of Technology, targeting a cluster of cell broadband engines.

The Raid-6 Liber8Tion Code

Plank, J. S. — Mon, 20 Jul 2009 04:07:45 PDT

Large centralized and networked storage systems have grown to the point where the single fault tolerance provided by RAID-5 is no longer enough. RAID-6 storage systems protect k disks of data with two parity disks so that the system of k + 2 disks may tolerate the failure of any two disks. Coding techniques for RAID-6 systems are varied, but an important class of techniques are those with minimum density, featuring an optimal combination of encoding, decoding and modification complexity. The word size of a code has an impact on both how the code is laid out on each disk's sectors and how large k can be. Word sizes which are powers of two are especially important, since they fit precisely into file system blocks. Minimum density codes exist for many word sizes with the notable exception of eight. This paper fills that gap by describing a new code called The RAID-6 Liber8tion Code for this important word size. The description includes performance properties as well as details of the discovery process.

HPC and Grid Computing for Integrative Biomedical Research

Mon, 20 Jul 2009 04:07:45 PDT

Integrative biomedical research projects query, analyze, and integrate many different data types and make use of datasets obtained from measurements or simulations of structure and function at multiple biological scales. With the increasing availability of high-throughput and high-resolution instruments, the integrative biomedical research imposes many challenging requirements on software middleware systems. In this paper, we look at some of these requirements using example research pattern templates. We then discuss how middleware systems, which incorporate Grid and high-performance computing, could be employed to address the requirements.

Energy Profiling and Analysis of the HPC Challenge Benchmarks

Shuaiwen Song, , Rong Ge, , Xizhou Feng, , Cameron, K. W. — Mon, 20 Jul 2009 04:07:45 PDT

Future high performance systems must use energy efficiently to achieve petaFLOPS computational speeds and beyond. To address this challenge, we must first understand the power and energy characteristics of high performance computing applications. In this paper, we use a power-performance profiling framework called Power-Pack to study the power and energy profiles of the HPC Challenge benchmarks. We present detailed experimental results along with in-depth analysis of how each benchmark's workload characteristics affect power consumption and energy efficiency. This paper summarizes various findings using the HPC Challenge benchmarks, including but not limited to: 1) identifying application power profiles by function and component in a high performance cluster; 2) correlating applications' memory access patterns to power consumption for these benchmarks; and 3) exploring how energy consumption scales with system size and workload.

Parallel Programming in MATLAB

Luszczek, P. — Mon, 20 Jul 2009 04:07:45 PDT

A visit to the neighborhood PC retail store provides ample proof that we are in the multi-core era. The key differentiator among manufacturers today is the number of cores that they pack onto a single chip. The clock frequency of commodity processors has reached its limit, however, and is likely to stay below 4 GHz for years to come. As a result, adding cores is not synonymous with increasing computational power. To take full advantage of the performance enhancements offered by the new multi-core hardware, a corresponding shift must take place in the software infrastructure — a shift to parallel computing.

Hierarchical Task-Based Programming With StarSs

Planas, J., Badia, R. M., Ayguade, E., Labarta, J. — Mon, 20 Jul 2009 04:07:45 PDT

Programming models for multicore and many-core systems are listed as one of the main challenges in the near future for computing research. These programming models should be able to exploit the underlying platform, but also should have good programmability to enable programmer productivity. With respect to the heterogeneity and hierarchy of the underlying platforms, the programming models should take them into account but they should also enable the programmer to be unaware of the complexity of the hardware. In this paper we present an extension of the StarSs syntax to support task hierarchy. A motivation for such a hierarchical approach is presented through experimentation with CellSs. A prototype implementation of such a hierarchical task-based programming model that combines a first task level with SMPSs and a second task level with CellSs is presented. The preliminary results obtained when executing a matrix multiplication and a Cholesky factorization show the viability and potential of the approach and the current issues raised.

Accurate and Efficient Estimation of Parameters of Heterogeneous Communication Performance Models

Lastovetsky, A., Rychkov, V. — Wed, 20 May 2009 04:33:32 PDT

Analytical predictive communication models play an important role in the optimization of communication operations in scientific applications running on computational clusters. The effectiveness of this model-based optimization strongly depends on the accuracy of the estimation of the parameters of these models. The task of accurate estimation of the model is particularly challenging for heterogeneous communication models that use a much larger number of point-to-point parameters than their homogeneous counterparts. One particular challenge occurs when the number of point-to-point parameters describing communication between a pair of processors becomes larger than the number of independent point-to-point communication experiments traditionally used for estimation of the parameters. In this paper, we address this and other related issues and propose an approach that allows us to design a set of communication experiments sufficient for the accurate and efficient estimation of the parameters of a heterogeneous communication performance model. The experiments on heterogeneous clusters demonstrate the accuracy and efficiency of the proposed solution.

A Parallel Algorithm To Solve Large Stiff ODE Systems On Grid Systems

Bahi, J. M., Charr, J.-C., Couturier, R., Laiymani, D. — Wed, 20 May 2009 04:33:32 PDT

This paper introduces a parallel algorithm to solve large stiff ODE systems on distributed clusters, with computing nodes geographically distant from each other. This algorithm is based on the waveform relaxation method coupled with a sequential solver for differential equations systems. With respect to the standard PVODE algorithm (Parallel Variable-coefficient Ordinary Differential Equations solver; Byrne, George, and Hindmars 1999), it drastically reduces the number of messages exchanged between nodes which makes it less sensitive to slow communications. Thus, it is a coarse-grained algorithm well suited for grid environments connected via high latency networks. In this paper, we present various experiments which compare the PVODE solver and our algorithm and which show the benefits brought by this work.

Parallel Algorithms for the Execution of Relational Database Operations Revisited On Grids

Mach, W., Schikuta, E. — Wed, 20 May 2009 04:33:32 PDT

This paper presents an analytical discussion of algorithms for relational database operations in a grid environment, compares the findings with the classical generalized multiprocessor framework, and describes an optimization algorithm to maximize performance for a heterogeneous environment.

We develop a concise but comprehensive analytical model of parallel algorithms for sorting, joining, and aggregation. In our approach we focus on a limited number of characteristic parameters to keep the analytical model clear. It is shown that an expressive model can be built upon just three characteristic parameter sets, namely the node processing performance and the network and the disk bandwidths. These parameters are the input for the optimization process for the orchestration of the execution workflow on the grid. Based on these results the paper proves that using smart enhancement to exploit the heterogeneity of the grid, the performance of the algorithms for database operations can be increased remarkably.

Multi-Criteria Scheduling of Pipeline Workflows (and Application To the JPEG Encoder)

Benoit, A., Kosch, H., Rehn-Sonigo, V., Robert, Y. — Wed, 20 May 2009 04:33:32 PDT

Mapping workflow applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline graphs. Several antagonist criteria should be optimized, such as throughput and latency (or a combination). A typical application class is digital image coding, where images are processed in steady-state mode. In this paper, we study the general bi-criteria mapping problem (minimizing period and latency) for pipeline graphs on communication homogeneous platforms. We present an integer linear programming formulation for this NP-hard problem. Furthermore, we provide several efficient polynomial bi-criteria heuristics, whose relative performance is evaluated through extensive simulations. As a case study, we provide simulations and MPI experimental results for the JPEG encoder application pipeline on a cluster of workstations.

Thank You To Reviewers

Wed, 18 Feb 2009 07:55:26 PST

The Problem With the Linpack Benchmark 1.0 Matrix Generator

Dongarra, J. J., Langou, J. — Wed, 18 Feb 2009 07:55:27 PST

We characterize the matrix sizes for which the Linpack Benchmark 1.0 matrix generator constructs a matrix with identical columns.

Performance Modeling and Analysis of a Massively Parallel Direct--Part 1

Jian He, , Verstak, A., Watson, L.T., Sosonkina, M. — Wed, 18 Feb 2009 07:55:27 PST

Modeling and analysis techniques are used to investigate the performance of a massively parallel version of DIRECT, a global search algorithm widely used in multidisciplinary design optimization applications. Several high-dimensional benchmark functions and real world problems are used to test the design effectiveness under various problem structures. Theoretical and experimental results are compared for two parallel clusters with different system scales and network connectivities. The present work aims at studying the performance sensitivity to important parameters for problem configurations, parallel schemes, and system settings. The performance metrics include the memory usage, load balancing, parallel efficiency, and scalability. An analytical bounding model is constructed to measure the load balancing performance under different schemes. Additionally, linear regression models are used to characterize two major overhead sources, interprocessor communication and processor idleness, and also applied to the isoefficiency functions in scalability analysis. For a variety of high-dimensional problems and large-scale systems, the massively parallel design has achieved reasonable performance. The results of the performance study provide guidance for efficient problem and scheme configuration. More importantly, the generalized design considerations and analysis techniques are beneficial for transforming many global search algorithms into effective large-scale parallel optimization tools.

Performance Modeling and Analysis of a Massively Parallel Direct--Part 2

Jian He, , Verstak, A., Sosonkina, M., Watson, L.T. — Wed, 18 Feb 2009 07:55:27 PST

Modeling and analysis techniques are used to investigate the performance of a massively parallel version of DIRECT, a global search algorithm widely used in multidisciplinary design optimization applications. Several high-dimensional benchmark functions and real world problems are used to test the design effectiveness under various problem structures. In this second part of a two-part work, theoretical and experimental results are compared for two parallel clusters with different system scales and network connectivities. The first part studied performance sensitivity to important parameters for problem configurations and parallel schemes, using performance metrics such as memory usage, load balancing, and parallel efficiency. Here linear regression models are used to characterize two major overhead sources, interprocessor communication and processor idleness, and also applied to the isoefficiency functions in scalability analysis. For a variety of high-dimensional problems and large-scale systems, the massively parallel design has achieved reasonable performance. The results of the performance study provide guidance for efficient problem and scheme configuration. More importantly, the design considerations and analysis techniques generalize to the transformation of other global search algorithms into effective large-scale parallel optimization tools.

Opendda: a Novel High-Performance Computational Framework for the Discrete Dipole Approximation

Donald, J. M., Golden, A., Jennings, S. G. — Wed, 18 Feb 2009 07:55:27 PST

This work presents a highly optimized computational framework for the Discrete Dipole Approximation, a numerical method for calculating the optical properties associated with a target of arbitrary geometry that is widely used in atmospheric, astrophysical and industrial simulations. Core optimizations include the bit-fielding of integer data and iterative methods that complement a new Discrete Fourier Transform (DFT) kernel, which efficiently calculates the matrix— vector products required by these iterative solution schemes. The new kernel performs the requisite 3-D DFTs as ensembles of 1-D transforms, and by doing so, is able to reduce the number of constituent 1-D transforms by 60% and the memory by over 80%. The optimizations also facilitate the use of parallel techniques to further enhance the performance. Complete OpenMP-based shared-memory and MPI-based distributed-memory implementations have been created to take full advantage of the various architectures. Several benchmarks of the new framework indicate extremely favorable performance and scalability.

Optimization With High-Cost Objective Function Evaluations in a Computing Grid and an Application To Simulation-Based Design

Jin Woo Park, , Si Hyong Park, , Seung Jo Kim, — Wed, 18 Feb 2009 07:55:27 PST

The emerging grid computing technologies are aimed at the creation of virtual supercomputers to conduct huge scale scientific computations by harvesting computing resources on the Internet. This paper introduces a grid-enabled implementation of an optimization program for large-scale optimization problems requiring high-cost, black-box objective function evaluations. Adopting grid computing can be particularly beneficial for building surrogates such as response surfaces and carrying out large-scale optimizations using stochastic optimization algorithms. However, several problems have to be resolved for effective utilization of grid resources because of heterogeneity in computer capacity among grid resources and limited network conditions inherent in grid systems. This paper identifies some of the problems and introduces algorithms to effectively carry out large-scale optimizations on a grid. Specifically, asynchronous genetic and particle swarm optimization algorithms are developed for grid computing environments. The performance and characteristics of the grid-enabled implementations are assessed via extensive numerical tests. Finally, structural design based on high-fidelity simulations is carried out using the proposed algorithm in a computing grid system.

Dynamic Component Extension: a Strategy for Performance Improvement in Multicomponent Applications

Sivagama, S. M., Vadhiyar, S. S., Nanjundiah, R. S. — Wed, 18 Feb 2009 07:55:27 PST

Multicomponent application paradigms have gained prominence in many significant multidisciplinary scientific applications. In this work, we propose a software strategy called dynamic component extension for multicomponent applications to improve application performance by minimizing processor idling. In this strategy, the processor space of a component is dynamically extended to include the processors of other components during certain computationally intensive phases of the component. We demonstrate its use in improving the performance of one of the most prominent multicomponent applications, the community climate system model (CCSM). In this application, we dynamically extend the atmosphere component to minimize the idling in other components caused by large periodic temporal load imbalances in the atmosphere component. By means of experiments on different parallel platforms with different numbers of processors, we show that using our strategy can lead to about 15% reduction and savings of several days in execution times of CCSM for 1000-year simulation runs.

How to Balance the Load on Heterogeneous Clusters

Beltran, M., Guzman, A. — Wed, 18 Feb 2009 07:55:27 PST

The problem of computing a large set of different tasks on a set of heterogeneous resources connected by a network is very common nowadays in very different environments and load balancing is indispensable for achieving high performance and high throughput in systems such as clusters. Cluster heterogeneity increases the difficulty of balancing the load across the system nodes and, although the relationship between heterogeneity and load balancing is difficult to describe analytically, in this paper different models and performance metrics are proposed to describe heterogeneous cluster behavior and to perform an exhaustive analysis of the effects of heterogeneity on load balancing algorithm performance. This analysis allows us to propose efficient solutions capable of dealing with heterogeneity for all the load balancing algorithm stages. Furthermore, a load balancing algorithm has been implemented following these solutions to demonstrate, with experimental results, its efficiency on real heterogeneous clusters.