2

Rake creating a rule that uses multitasking

 2 years ago
source link: https://www.codesd.com/item/rake-creating-a-rule-that-uses-multitasking.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Rake creating a rule that uses multitasking

advertisements

If I create a simple rule like

rule '.o' => ['.c'] do |t|
  sh "cc #{t.source} -c -o #{t.name}"
end

How can I tell Rake that I want the auto-generated tasks to be parallelizable?


as stated by Dennis Rake rules are implemented as Tasks

therefore the -m flag which converts tasks to multitasks also converts rules to 'multirules'

require 'rake/clean'

rule '.ext2' => '.ext1' do |t|
    sh "cp #{t.source} #{t.name}"
    sleep(1)
end

def dependencies(input_file)
    base, is, ext = input_file.split('.')
    _from, _to = is.split('-')
    files = []
    Integer(_from).upto(Integer(_to)) do |i|
        files << "#{base}_#{i}.#{ext}"
    end
    return files
end

rule ".ext2" => lambda { |i| dependencies(i) } do |t|
    sh "touch #{t.source}"
end

task :make_files do |t|
    1.upto(20) do |i|
        sh "touch file_#{i}.ext1"
    end
end

CLEAN = FileList['*.ext2']

running the following commands (not threaded):

rake make_files
time rake file.1-20.ext2

real    0m20.232s
user    0m0.134s
sys     0m0.082s

with 5 threads:

rake clean
time rake -m -j 5 file.1-20.ext2

real    0m4.152s
user    0m0.122s
sys     0m0.071s

with 20 threads:

rake clean
time rake -m -j 20 file.1-20.ext2

real    0m1.167s
user    0m0.130s
sys     0m0.065s

I use sleep(1) to emulate 'work/io', if your processes block a lot or if you have many cores this might be useful to you :)

EDIT: As pointed out by Shadow in comments, the follow toggles 'multitask' to be always on

Rake.application.options.always_multitask = true


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK