PGA HPC Implementation of Microtubule Brownian Dynamics Simulations

This paper presents high performance simulation of microtubule molecular dynamics implemented on Xilinx Virtex-7 FPGA using high level synthesis tool Vivado HLS. FPGA implementation is compared to multicore Intel Xeon CPU and Nvidia K40 GPU implementations in terms of performance and energy efficiency. Algorithm takes into account Brownian motion thus heavily uses normally distributed random numbers. Original sequential code was optimized for different platforms using OpenMP for CPU, OpenCL for GPU and Vivado HLS for FPGA. We show that in terms of performance FPGA achieved 17x speed up against CPU and 11x speedup against GPU for our best optimized CPU and GPU versions. As to power efficiency, FPGA outperformed CPU 227 times and GPU 75 times. FPGA application is developed using SDK, which has Board Support Package including FPGA project framework where accelerator kernel (designed in Vivado HLS) IP core is to be integrated, and host-side libraries used to communicate with FPGA via PCI Express. Developed flow does not require expert FPGA skills and can be used by programmer with little knowledge of hardware design methodology that could use C\C++ language for complete development of FPGA accelerated solution.

HPC, FPGA, Microtubule, HLS, Brownian Dynamics

1. B. Liu, D. Zydek, H. Selvaraj, and L. Gewali. «Accelerating High Performance Computing Applications: Using CPUs, GPUs, Hybrid CPU/GPU, and FPGAs». In 2012 13th International Conference on Parallel and Distributed Computing, Applications and Technologies, 2012, pp. 337–342.

2. Wim Vanderbauwhede and K. Benkrid. High-Performance Computing Using FPGAs. Springer, 2013.

3. J. Fowers, G. Brown, P. Cooke, and G. Stitt. «A Performance and Energy Comparison of FPGAs, GPUs, and Multicores for Sliding-window Applications». In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, New York, NY, USA, 2012, pp. 47–56.

4. K. Sano, Y. Hatsuda, and S. Yamamoto. «Multi-FPGA Accelerator for Scalable Stencil Computation with Constant Memory Bandwidth». IEEE Trans. Parallel Distrib. Syst. 2014, vol. 25, no. 3, pp. 695–705.

5. K. Benkrid, A. Akoglu, C. Ling, Y. Song, Y. Liu, and X. Tian. «High Performance Biological Pairwise Sequence Alignment: FPGA Versus GPU Versus Cell BE Versus GPP». Int. J. Reconfig. Comput., vol. 2012, 2012.

6. B. G. Fitch, A. Rayshubskiy, M. Eleftheriou, T. J. C. Ward, M. Giampapa, M. C. Pitman, and R. S. Germain. «Blue Matter: Approaching the Limits of Concurrency for Classical Molecular Dynamics». In Proceedings of the ACM/IEEE SC 2006 Conference, 2006, pp. 44–44.

7. K. J. Bowers, D. E. Chow, H. Xu, R. O. Dror, M. P. Eastwood, B. A. Gregersen, J. L. Klepeis, I. Kolossvary, M. A. Moraes, F. D. Sacerdoti, J. K. Salmon, Y. Shan, and D. E. Shaw, «Scalable Algorithms for Molecular Dynamics Simulations on Commodity Clusters». In Proceedings of the ACM/IEEE SC 2006 Conference, 2006, pp. 43–43.

8. Y. Komeiji, M. Uebayasi, R. Takata, A. Shimizu, K. Itsukashi, and M. Taiji. «Fast and accurate molecular dynamics simulation of a protein using a special-purpose computer». J. Comput. Chem., vol. 18, no. 12, pp. 1546–1563, 1997.

9. D. E. Shaw, J. P. Grossman, J. A. Bank, B. Batson, J. A. Butts, J. C. Chao, M. M. Deneroff, R. O. Dror, A. Even, C. H. Fenton, A. Forte, J. Gagliardo, G. Gill, B. Greskamp, C. R. Ho, D. J. Ierardi, L. Iserovich, J. S. Kuskin, R. H. Larson, T. Layman, L.-S. Lee, A. K. Lerer, C. Li, D. Killebrew, K. M. Mackenzie, S. Y.-H. Mok, M. A. Moraes, R. Mueller, L. J. Nociolo, J. L. Peticolas, T. Quan, D. Ramot, J. K. Salmon, D. P. Scarpazza, U. Ben Schafer, N. Siddique, C. W. Snyder, J. Spengler, P. T. P. Tang, M. Theobald, H. Toma, B. Towles, B. Vitale, S. C. Wang, and C. Young. «Anton 2: Raising the Bar for Performance and Programmability in a Special-purpose Molecular Dynamics Supercomputer». In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Piscataway, NJ, USA, 2014, pp. 41–53.

10. M. Taiji, T. Narumi, Y. Ohno, N. Futatsugi, A. Suenaga, N. Takada, and A. Konagaya. «Protein Explorer: A Petaflops Special-Purpose Computer System for Molecular Dynamics Simulations». In Supercomputing, 2003 ACM/IEEE Conference, 2003, pp. 15–15.

11. C. Rodrigues, D. Hardy, J. Stone, K. Schulten, and W.-Mei Hwu. «GPU acceleration of cutoff pair potentials for molecular modeling applications». Proceedings of the 5th conference on Computing frontiers, 2008, pp. 273–282.

12. S. R. Alam, P. K. Agarwal, M. C. Smith, J. S. Vetter, and D. Caliga. «Using FPGA Devices to Accelerate Biomolecular Simulations». Computer, vol. 40, no. 3, 2007, pp. 66–73.

13. N. Azizi, I. Kuon, A. Egier, A. Darabiha, and P. Chow. «Reconfigurable molecular dynamics simulator». In 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, 2004. FCCM 2004, pp. 197–206.

14. Y. Gu, T. VanCourt, and M. C. Herbordt. «Improved Interpolation and System Integration for FPGA-Based Molecular Dynamics Simulations». In 2006 International Conference on Field Programmable Logic and Applications, 2006, pp. 1–8.

15. V. Kindratenko and D. Pointer. «A case study in porting a production scientific supercomputing application to a reconfigurable computer». 2006, pp. 13–22.

16. R. Scrofano, M. B. Gokhale, F. Trouw, and V. K. Prasanna. «Accelerating Molecular Dynamics Simulations with Reconfigurable Computers». IEEE Trans. Parallel Distrib. Syst., vol. 19, no. 6, 2008, pp. 764–778.

17. M. Chiu and M. C. Herbordt. «Molecular Dynamics Simulations on High-Performance Reconfigurable Computing Systems». ACM Trans Reconfigurable Technol Syst, vol. 3, no. 4, 2010, pp. 23:1–23:37.

18. T. Mitchison and M. Kirschner. «Dynamic instability of microtubule growth». Nature, vol. 312, no. 5991, 1984, pp. 237–242.

19. A. Desai and T. J. Mitchison. «Microtubule Polymerization Dynamics». Annu. Rev. Cell Dev. Biol., vol. 13, no. 1, 1997, pp. 83–117.

20. P. Zakharov, N. Gudimchuk, V. Voevodin, A. Tikhonravov, F. I. Ataullakhanov, and E. L. Grishchuk. «Molecular and Mechanical Causes of Microtubule Catastrophe and Aging». Biophys. J., vol. 109, no. 12, , 2015, pp. 2574–2591.

21. M. K. Gardner, M. Zanic, C. Gell, V. Bormuth, and J. Howard, «Depolymerizing Kinesins Kip3 and MCAK Shape Cellular Microtubule Architecture by Differential Control of Catastrophe». Cell, vol. 147, no. 5, , 2011, pp. 1092–1103.

22. D. L. Ermak and J. A. McCammon, «Brownian dynamics with hydrodynamic interactions». J. Chem. Phys., v. 69, issue 4, , 1978, pp. 1352–1360.

23. M. Matsumoto and T. Nishimura, «Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator». ACM Trans. Model. Comput. Simul., vol. 8, 1998, pp. 3–30.

24. S. Kasap and K. Benkrid, «Parallel processor design and implementation for molecular dynamics simulations on a FPGA-Based supercomputer». J. Comput., vol. 7, no. 6, 2012, pp. 1312–1328.

25. R. Baxter, S. Booth, M. Bull, G. Cawood, J. Perry, M. Parsons, A. Simpson, A. Trew, A. McCormick, G. Smart, R. Smart, A. Cantle, R. Chamberlain, and G. Genest, «Maxwell - a 64 FPGA Supercomputer». In Second NASA/ESA Conference on Adaptive Hardware and Systems (AHS 2007), 2007, pp. 287–294.

26. J. M. P. Cardoso, P. C. Diniz, and M. Weinhardt, «Compiling for reconfigurable computing: A survey». ACM Comput. Surv. CSUR, vol. 42, no. 4, p. 13, 2010.

27. Y. Liang, K. Rupnow, Y. Li, D. Min, M. N. Do, and D. Chen, «High-level synthesis: productivity, performance, and software constraints». J. Electr. Comput. Eng., vol. 2012, 2012, p. 1.

28. J. Monson, M. Wirthlin, and B. L. Hutchings, «Implementing high-performance, low-power FPGA-based optical flow accelerators in C». IEEE 24th International Conference on Application-Specific Systems, Architectures and Processors, 2013, pp. 363–369.

29. T. Hussain, M. Pericàs, N. Navarro, and E. Ayguadé, «Implementation of a Reverse Time Migration kernel using the HCE High Level Synthesis tool». Field-Programmable Technology (FPT), 2011 International Conference, 2011, pp. 1–8.