开发者

How to use Bash to parallelise my PHP script

开发者 https://www.devze.com 2023-01-25 11:31 出处:网络
I have a PHP script that loops through some urls. The urls are stored in urls.txt, and it needs to call the php script li开发者_如何学Goke php urlcheck.php LINE_FROM_urls.txt

I have a PHP script that loops through some urls. The urls are stored in urls.txt, and it needs to call the php script li开发者_如何学Goke php urlcheck.php LINE_FROM_urls.txt

When it reaches a set number of processes, I want it to sleep for 5 seconds, like this

COMCHECK=`ps aux | grep -c php`

while [ $COMCHECK -ge 150 ];do
COMCHECK=`ps aux | grep -c php`
echo "Sleeping"
sleep 5

and then I want it to check again. If its -le 150, run the next url etc.

I have no idea how to do this. My Bash is quite limited, and frankly, the language weirds me out.


You can try something like:

limit=150  

# read lines from the file urls.txt one by one.
while read line; do

        # check number of running php processes.
        while [ $(ps aux | grep -c php) -ge $limit ];do
                # if it's >= limit..sleep.
                echo "Sleeping"       
                sleep 5
        done

        # limit not reached..run another process..in background. 
        php urlcheck.php  $line &

done < urls.txt


while read URL; do
    while (( $(ps aux | grep -c php) >= 150 )); do
        sleep 5
    done

    php urlcheck.php "$URL" &
done < urls.txt

# Optional: wait for all PHP processes to finish.
wait
0

精彩评论

暂无评论...
验证码 换一张
取 消