Evaluating the Combined Effect of Memory Capacity and Concurrency for Many-Core Chip Design

Modern memory systems are structured under hierarchy and concurrency. The combined impact of hierarchy and concurrency, however, is application dependent and difficult to describe. In this article, we introduce C ² -Bound, a data-driven analytical model that serves the purpose of optimizing many-core design. C ² -Bound considers both memory capacity and data access concurrency. It utilizes the combined power of the newly proposed latency model, concurrent average memory access time, and the well-known memory-bounded speedup model (Sun-Ni’s law) to facilitate computing tasks. Compared to traditional chip designs that lack the notion of memory capacity and concurrency, the C ² -Bound model finds that memory bound factors significantly impact the optimal number of cores as well as their optimal silicon area allocations, especially for data-intensive applications with a non-parallelizable sequential portion. Therefore, our model is valuable to the design of next-generation many-core architectures that target big data processing, where working sets are usually larger than the conventional scientific computing. These findings are evidenced by our detailed simulations, which show, with C ² -Bound, the design space of chip design can be narrowed down significantly up to four orders of magnitude. C ² -Bound analytic results can be either used in reconfigurable hardware environments or, by software designers, applied to scheduling, partitioning, and allocating resources among diverse applications.