Scheduling and routing algorithm for aggregating large data files from distributed databases to super-computers on lambda grid

Shen Sun; Wei Guo; Weiqiang Sun; Yaohui Jin; Weisheng Hu

doi:10.1117/12.803604

19 November 2008 Scheduling and routing algorithm for aggregating large data files from distributed databases to super-computers on lambda grid

Shen Sun, Wei Guo, Weiqiang Sun, Yaohui Jin, Weisheng Hu

Author Affiliations +

Proceedings Volume 7137, Network Architectures, Management, and Applications VI; 71372D (2008) https://doi.org/10.1117/12.803604
Event: Asia-Pacific Optical Communications, 2008, Hangzhou, China

Abstract

These days while the traditional Internet cannot meet the requirement of data-intensive communications in large scale escience grid applications, Optical network which is also referred to as Lambda Grid provide a simple means of achieving guaranteed high bandwidth, guaranteed latency and deterministic connection. Lots of e-science applications like e-VLBI and GTL require aggregating several hundred GB data files from distributed databases to super-computers frequently at real time. Thus minimizing the aggregation time can improve the overall system performance. We consider the problem of aggregating large data files from distributed databases to distributed computational resources on lambda grid. We modify the model of Time-Path Scheduling Problem (TPSP) which has been proposed and propose a new N-destination TPSP (NDTPSP) model. We present the proof of NDTPSP's NP-completeness. We also propose a list scheduling algorithm and a modified list scheduling algorithm for our problem. The performance of different algorithms will be compared and analyzed by simulations.

Citation Download Citation

Shen Sun, Wei Guo, Weiqiang Sun, Yaohui Jin, and Weisheng Hu "Scheduling and routing algorithm for aggregating large data files from distributed databases to super-computers on lambda grid", Proc. SPIE 7137, Network Architectures, Management, and Applications VI, 71372D (19 November 2008); https://doi.org/10.1117/12.803604

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available