Patch the Ragel lexer after generating it.

This further increases throughput of the lexer. On MRI this seems to save
around one second or so. It now sits at ~6,8 seconds in the big XML benchmark.

On JRuby, combined with some JIT options and invoke dynamic enabled, this can
reduce the average lexing time to around 3,5 seconds.  Rubinius, also with a
few aggressive JIT options, seems to stick around 9 seocnds.
This commit is contained in:
Yorick Peterse 2014-05-02 00:40:10 +02:00
parent 9dfdefee47
commit 57255012b7
1 changed files with 14 additions and 0 deletions

View File

@ -2,6 +2,20 @@ rule '.rb' => '.rl' do |task|
Cliver.assert('ragel', '~> 6.7') Cliver.assert('ragel', '~> 6.7')
sh "ragel -F1 -R #{task.source} -o #{task.name}" sh "ragel -F1 -R #{task.source} -o #{task.name}"
puts "Applying patch http://git.io/ow6e1A to #{task.name}"
# Patches the lexer based on http://git.io/ow6e1A.
input = File.read(task.source)
output = File.read(task.name)
getkey = input.match(/getkey\s+(.+);/)[1]
output = output.gsub(getkey, '_wide')
output = output.gsub('_trans = if', "_wide = #{getkey}\n _trans = if")
File.open(task.name, 'w') do |handle|
handle.write(output)
end
end end
desc 'Generates the lexer' desc 'Generates the lexer'