Rake creating a rule that uses multitasking
source link: https://www.codesd.com/item/rake-creating-a-rule-that-uses-multitasking.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Rake creating a rule that uses multitasking
If I create a simple rule like
rule '.o' => ['.c'] do |t|
sh "cc #{t.source} -c -o #{t.name}"
end
How can I tell Rake that I want the auto-generated tasks to be parallelizable?
as stated by Dennis Rake rules are implemented as Tasks
therefore the -m flag which converts tasks to multitasks also converts rules to 'multirules'
require 'rake/clean'
rule '.ext2' => '.ext1' do |t|
sh "cp #{t.source} #{t.name}"
sleep(1)
end
def dependencies(input_file)
base, is, ext = input_file.split('.')
_from, _to = is.split('-')
files = []
Integer(_from).upto(Integer(_to)) do |i|
files << "#{base}_#{i}.#{ext}"
end
return files
end
rule ".ext2" => lambda { |i| dependencies(i) } do |t|
sh "touch #{t.source}"
end
task :make_files do |t|
1.upto(20) do |i|
sh "touch file_#{i}.ext1"
end
end
CLEAN = FileList['*.ext2']
running the following commands (not threaded):
rake make_files
time rake file.1-20.ext2
real 0m20.232s
user 0m0.134s
sys 0m0.082s
with 5 threads:
rake clean
time rake -m -j 5 file.1-20.ext2
real 0m4.152s
user 0m0.122s
sys 0m0.071s
with 20 threads:
rake clean
time rake -m -j 20 file.1-20.ext2
real 0m1.167s
user 0m0.130s
sys 0m0.065s
I use sleep(1) to emulate 'work/io', if your processes block a lot or if you have many cores this might be useful to you :)
EDIT: As pointed out by Shadow in comments, the follow toggles 'multitask' to be always on
Rake.application.options.always_multitask = true
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK