In-Cache Streaming: Morphable Infrastructure for Many-Core Processing Systems

Transcrição

1 In-Cache Streaming: Morphable Infrastructure for Many-Core Processing Systems Nuno Neves Adrien Mussio Fabien Gonçalves Pedro Tomás Nuno Roma 9th Workshop on UnConventional High Performance Computing 2016 Signal Processing Systems Group INESC-ID / IST Portugal 1

2 Outline Motivation Data Streaming with Compiler-Assisted Prefetching In-Cache Streaming Infrastructure System Overview In-Cache Stream Controller Hybrid Shared Memory Controller Experimental Evaluation Conclusions 2

3 Motivation: Adaptive hardware for data communication Runtime hardware adaptation has been increasingly viable to improve processing performance and reduce energy consumption. Reconfigurable processing architectures allow tuning their hardware resources, in runtime, to better suit a running application s requirements and cope with system energy constraints. The underlying communication infrastructures are left static, since in most cases runtime reconfiguration often imposes overheads instead of improving throughput. Although highly efficient in their domain, most established data communication paradigms for many-core architectures present significant drawbacks in other types of applications. 3

4 Motivation: Efficient data communication in many-core systems Conventional approaches usually rely on cache-based communication structures: - Compute-bound applications - Memory accesses with high data and temporal locality - Very large data-sets - Memory-bound applications - Memory accesses with poor datalocality Large data-sets and memory-bound applications are usually tackled with prefetching techniques. However, the prefetching complexity is bound by its resource overhead and increased energy consumption. Moreover, prefetching units cannot deal with complex data-patterns. 4

5 Motivation: Efficient data communication in many-core systems Stream-based communication is a viable approach to tackle these drawbacks: - Very large data-sets and memory-bound applications Stream prefetching and data reutilization (buffering) - Complex memory access patterns (i.e. with poor datalocality) Dedicated memory access generators and explicit data-pattern description However, stream-based techniques can hardly deal with dynamically indexed data or with non-deterministic/runtime generated data access patterns. Goal: Combine the advantages of the two approaches in a single and adaptable communication infrastructure that is capable of in-time switching its paradigm to better suit a running application. 5

6 Outline Motivation Data Streaming with Compiler-Assisted Prefetching In-Cache Streaming Infrastructure System Overview In-Cache Stream Controller Hybrid Shared Memory Controller Experimental Evaluation Conclusions 6

7 Data Streaming with Compiler-Assisted Prefetching Stream-based communication with static prefetching 1. compile-time procedures pre-analyse the application to extract/model its memory access pattern; 2. information is fed to on-chip address generation units, which autonomously generate the required memory address sequence and send data streams to processing elements (PEs). Requires far simpler hardware structures than other prefetching approaches. Data streaming promotes the exploitation of highly efficient communication means: Complex memory access pattern generation On-the-fly data reorganization Persistent data reutilization Data broadcast and point-to-point communication (e.g. data-flow graphs) 7

8 Memory Access Pattern Description Most memory access patterns can be described as a n-dimensional affine function: These representations are commonly used by DMAs and other data-fetch controllers encoded in data-pattern descriptors. 3D Data-pattern Descriptor Tree Specification [1] [1] Nuno Neves et al., Efficient Data-Stream Management for Shared-Memory Many- Core Systems, th International Conference on Field Programmable Logic and Applications (FPL), IEEE, pages 1-8,

9 3D Data-pattern Descriptor Tree Three-dimensional description function (example): Descriptor combination in a tree-like hierarchical scheme 9

10 Outline Motivation Data Streaming with Compiler-Assisted Prefetching In-Cache Streaming Infrastructure System Overview In-Cache Stream Controller Hybrid Shared Memory Controller Experimental Evaluation Conclusions 10

11 In-Cache Streaming Infrastructure Hybrid Communication Unification New communication paradigm simultaneously supporting address-based and streambased memory accesses: Dynamic adaptation of cache memories without relying on partial reconfiguration; Each individual way of a n-way set associative cache memory can be converted to an individual stream buffer; Hybrid main memory access that combines a conventional DMA with a dedicated stream management controller (SMC). 11

12 In-Cache Streaming Infrastructure System Overview Adaptive Infrastructure Overview In-Cache Stream Controller: Associated to each PE private cache memory. 12

13 In-Cache Streaming Infrastructure System Overview Adaptive Infrastructure Overview In-Cache Stream Controller: Associated to each PE private cache memory. Low-footprint NoC interconnection: Bidirectional ring topology; Dedicated Message-Passing Protocol; Point-to-point and Broadcast Communication. 12

14 In-Cache Streaming Infrastructure System Overview Adaptive Infrastructure Overview In-Cache Stream Controller: Associated to each PE private cache memory. Low-footprint NoC interconnection: Bidirectional ring topology; Dedicated Message-Passing Protocol; Point-to-point and Broadcast Communication. Hybrid Shared Memory Controller: Conventional DMA; Stream Management Controller (SMC); Programmable Pattern Descriptor Memory. 12

15 In-Cache Streaming Infrastructure In-Cache Stream Controller In-Cache Stream Controller 13

16 In-Cache Streaming Infrastructure In-Cache Stream Controller In-Cache Stream Controller Hybrid Cache Controller deploys conventional memoryaddressed communication to the PE and maintains the configuration of the hybrid cache memory. 13

17 In-Cache Streaming Infrastructure In-Cache Stream Controller In-Cache Stream Controller Hybrid Cache Controller deploys conventional memoryaddressed communication to the PE and maintains the configuration of the hybrid cache memory. Stream Controller deploys stream-based data accesses to each cache way (used as a stream buffer). 13

18 In-Cache Streaming Infrastructure In-Cache Stream Controller In-Cache Stream Controller Hybrid Cache Controller deploys conventional memoryaddressed communication to the PE and maintains the configuration of the hybrid cache memory. Stream Controller deploys stream-based data accesses to each cache way (used as a stream buffer). Stream Table holds the information and the state of every stream currently stored and handled by the stream controller. 13

19 In-Cache Streaming Infrastructure In-Cache Stream Controller In-Cache Stream Controller Cache memory hybridization Seamless transformation of a n-way set-associative memory in m independent stream buffers; Stream access performed with a dedicated set of read and write pointers to the memory region where a stream is stored (according to the Stream Table); Configuration can be performed according to PE requests, through a dedicated memory-mapped interface, or scheduled by a system controller. 14

20 In-Cache Streaming Infrastructure Main Memory Access Hybrid Shared Memory Controller Conventional DMA handles memory access for the conventional cache protocol 15

21 In-Cache Streaming Infrastructure Main Memory Access Hybrid Shared Memory Controller Conventional DMA handles memory access for the conventional cache protocol Stream Management Controller (SMC) generates and saves streams, according to the patterns stored in the pattern descriptor memory; a Descriptor Tree Controller (DTC) [1] deploys the adopted 3D data-pattern descriptor tree specification; stream data is stored in a local stream buffer before being sent to the PEs, promoting data reutilization. 15

22 In-Cache Streaming Infrastructure Main Memory Access Hybrid Shared Memory Controller Conventional DMA handles memory access for the conventional cache protocol Stream Management Controller (SMC) generates and saves streams, according to the patterns stored in the pattern descriptor memory; a Descriptor Tree Controller (DTC) [1] deploys the adopted 3D data-pattern descriptor tree specification; stream data is stored in a local stream buffer before being sent to the PEs, promoting data reutilization. 15

23 Outline Motivation Data Streaming with Compiler-Assisted Prefetching In-Cache Streaming Infrastructure System Overview In-Cache Stream Controller Hybrid Shared Memory Controller Experimental Evaluation Conclusions 16

24 Experimental Evaluation Experimental Setup Xilinx Virtex-7 VC707 (XC7VX485T) 1GB DDR3 SODIMM 800MHz/1600Mbps (MT8JTF12864HZ-1G6G1) P&R values obtained with Xilinx ISE 14.5 System simulations performed with isim Simulator Evaluation Hardware resource requirements; Comparison with a conventional cache-based system; Performance speedup and data transfer latency reduction results for three benchmarks applications; System configurations of up to 16 PEs: Each PE is composed of a MB-LITE processor, a private scratchpad memory and a memory-mapped interface to its associated In-Cache Stream controller. 17

25 Experimental Evaluation Hardware Resources System Configuration 8KB 4-way set-associative cache memory with 64-Byte cache lines 3D descriptor size: 128 bits SMC pattern descriptor memory: 256x128 bits When compared to the baseline cache controller, the in-cache stream controller imposes a small increase of the hardware resources, and a small impact (of 28 MHz) in the maximum operating frequency. Each of the devised components requires less than 2% of the FPGA resources. 18

26 Experimental Evaluation Performance Evaluation Benchmark setup Benchmark Dataset Feature Demonstration Blocked Matrix Multiplication Biological Sequence Alignment 128x128 matrices 8x8 blocks Sequences w/ 1024 symbols (1.) - Data prefetching - PE local data reutilization - Complex data pattern generation - On-the-fly data reorganization Histogram Equalization 256x256 image - Complex communication (2.) 1. All randomly generated. 2. Complex communication with multiple data transfer schemes: Reduction operations w/ point-to-point communication; Intermediate result broadcast; Image loading and storing with cache-based communication. 19

27 Experimental Evaluation Performance Evaluation Significant reduction of the data transfer and manipulation overheads. Overall performance speedups of up to 14x, 5x and 12x, when compared to the baseline conventional setup. 20

28 Outline Motivation Data Streaming with Compiler-Assisted Prefetching In-Cache Streaming Infrastructure System Overview In-Cache Stream Controller Hybrid Shared Memory Controller Experimental Evaluation Conclusions 21

29 Conclusions A novel in-cache streaming communication model and adaptive infrastructure for manycore systems was proposed. The devised controller deploys both conventional memory-addressed and stream-based communication paradigms, offering: In-time and seamless switching between communication paradigms without any significant impact on the data-transfer performance; Stream prefetching; Complex memory access generation and data reutilization and reorganization; Significant reduction in data communication, manipulation and total memory accesses. 22

30 Future Research and Work in Progress Addition of efficient scheduling schemes to efficiently manage the system s execution and dataflow. Extension of the adopted 3D data pattern descriptor tree specification, to further exploit pattern regularity and advance towards the representation of non-deterministic memory access patterns. Exploration of compile-time techniques for automatic data-pattern extraction. Further developments in the main memory access scheme, accounting for the characteristics of the adopted DDR memory and current access optimization techniques (including bank activation sequence, access reordering and burst transfers). 23

31 24