Slurm (Simple Linux Utility for Resource Management) is a popular open-source cluster job scheduling system. In scenarios involving multiple queues and a large number of scientific computing tasks, the Slurm scheduling system faces challenges such as improper queue depth settings and uneven job loads in queues. This research aims to optimize the scheduling performance of the Slurm scheduling system in scenarios where resources are shared among multiple queues. By forecasting future CPU load increases based on the historical CPU utilization of queues, a window period is identified. Using the forecasted results, a dynamic adjustment method for queue priorities is introduced. This method aims to elevate job priorities in specific queues when a significant number of jobs are queued, ensuring prompt execution. Additionally, this study involves a dynamic adjustment strategy for queue depth to address issues arising from inappropriate queue depth settings, where resources in the queue are idle, but queued jobs face delays in scheduling. Experimental results demonstrate that this approach enhances the scheduling performance of the Slurm system in multi-queue scenarios, improving cluster resource utilization and better meeting the diverse demands of users.
With the vigorous development of Internet technology, the scale of web users is gradually expanding, and the problem of high concurrency caused by huge visits has become a pain point that large-scale websites need to solve. Among them, Nginx as a reverse proxy for the back-end Web server cluster to provide load balancing technology solutions are widely used in large-scale websites. This paper designs an adaptive dynamic load strategy that includes an optimized dynamic load balancing algorithm for Nginx's weighted Round Robin which can periodically read the parameters according to the running state of the back-end server, dynamically modify the weights and determine the efficiency of load balancing through many experiments.
Convolutional Neural Network (CNN) has always been a hot topic in deep learning. With the increasing demand for network models in daily production, the optimization of convolution calculation process is very important. This paper starts with the process of back propagation in convolutional neural network, introduces the derivation of convolutional neural network back propagation and the conversion process of im2col, uses implicit to convert the calculation of convolution on the domestic acceleration platform, and optimizes the convolution back propagation operator through a variety of general matrix multiplication optimization strategies. The final performance reaches more than 70% of the performance of NVIDIA operator, which meets the expectation of the experiment under the performance bottleneck of the platform.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.