core
/
oga
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
oga/Rakefile

41 lines
674 B
Ruby
Raw Normal View History

Basic project layout.
2014-02-26 18:50:16 +00:00
require 'bundler/gem_tasks'
require 'digest/sha2'
require 'rake/clean'
Initial setup using a C extension. While I've tried to keep Oga pure Ruby for as long as possible the performance of Ragel's Ruby output was not worth the trouble. For example, lexing 10MB of XML would take 5 to 6 seconds at least. Nokogiri on the other hand can parse that same XML into a DOM document in about 300 miliseconds. Such a big performance difference is not acceptable. To work around this the XML/HTML lexer will be implemented in C for MRI/Rubinius and Java for JRuby. For now there's only a C extension as I haven't read up yet on the JRuby API. The end goal is to provide some sort of Ragel "template" that can be used to generate the corresponding C/Java extension code. This would remove the need of duplicating the grammar and associated code. The native extension setup is a hybrid between native and Ruby. The raw Ragel stuff happens in C/Java while the actual logic of actions happens in Ruby. This adds a small amount of overhead but makes it much easier to maintain the lexer. Even with this extra overhead the performance is much better than pure Ruby. The 10MB of XML mentioned above is lexed in about 600 miliseconds. In other words, it's 10 times faster.
2014-05-04 22:25:34 +00:00
require 'rake/extensiontask'
Basic project layout.
2014-02-26 18:50:16 +00:00
require 'cliver'
GEMSPEC = Gem::Specification.load('oga.gemspec')
Renamed HTML_PARSER to PARSER_OUTPUT. This keeps it consistent with the lexer.
2014-03-25 08:35:39 +00:00
PARSER_OUTPUT = 'lib/oga/xml/parser.rb'
Basic project layout.
2014-02-26 18:50:16 +00:00
Don't rely on Git for generating the MANIFEST. When using Git the resulting Gem will contain far too many useless files. For example, the profile/ and spec/ directories are not needed when building Gems.
2014-04-29 18:39:20 +00:00
CLEAN.include(
Benchmark for lexer lines/second. This benchmark uses a fixture file that is automatically downloaded.
2014-04-17 18:06:24 +00:00
'coverage',
'yardoc',
PARSER_OUTPUT,
Remove sample files when running rake clean.
2014-05-01 18:57:17 +00:00
'benchmark/fixtures/big.xml',
Initial setup using a C extension. While I've tried to keep Oga pure Ruby for as long as possible the performance of Ragel's Ruby output was not worth the trouble. For example, lexing 10MB of XML would take 5 to 6 seconds at least. Nokogiri on the other hand can parse that same XML into a DOM document in about 300 miliseconds. Such a big performance difference is not acceptable. To work around this the XML/HTML lexer will be implemented in C for MRI/Rubinius and Java for JRuby. For now there's only a C extension as I haven't read up yet on the JRuby API. The end goal is to provide some sort of Ragel "template" that can be used to generate the corresponding C/Java extension code. This would remove the need of duplicating the grammar and associated code. The native extension setup is a hybrid between native and Ruby. The raw Ragel stuff happens in C/Java while the actual logic of actions happens in Ruby. This adds a small amount of overhead but makes it much easier to maintain the lexer. Even with this extra overhead the performance is much better than pure Ruby. The 10MB of XML mentioned above is lexed in about 600 miliseconds. In other words, it's 10 times faster.
2014-05-04 22:25:34 +00:00
'profile/samples/**/*.txt',
'lib/liboga.*',
'tmp',
'ext/liboga/lexer.c'
Don't rely on Git for generating the MANIFEST. When using Git the resulting Gem will contain far too many useless files. For example, the profile/ and spec/ directories are not needed when building Gems.
2014-04-29 18:39:20 +00:00
)
Basic project layout.
2014-02-26 18:50:16 +00:00
Don't rely on Git for generating the MANIFEST. When using Git the resulting Gem will contain far too many useless files. For example, the profile/ and spec/ directories are not needed when building Gems.
2014-04-29 18:39:20 +00:00
FILE_LIST = FileList.new(
'checkum/**/*.*',
'doc/**/*.*',
Initial setup using a C extension. While I've tried to keep Oga pure Ruby for as long as possible the performance of Ragel's Ruby output was not worth the trouble. For example, lexing 10MB of XML would take 5 to 6 seconds at least. Nokogiri on the other hand can parse that same XML into a DOM document in about 300 miliseconds. Such a big performance difference is not acceptable. To work around this the XML/HTML lexer will be implemented in C for MRI/Rubinius and Java for JRuby. For now there's only a C extension as I haven't read up yet on the JRuby API. The end goal is to provide some sort of Ragel "template" that can be used to generate the corresponding C/Java extension code. This would remove the need of duplicating the grammar and associated code. The native extension setup is a hybrid between native and Ruby. The raw Ragel stuff happens in C/Java while the actual logic of actions happens in Ruby. This adds a small amount of overhead but makes it much easier to maintain the lexer. Even with this extra overhead the performance is much better than pure Ruby. The 10MB of XML mentioned above is lexed in about 600 miliseconds. In other words, it's 10 times faster.
2014-05-04 22:25:34 +00:00
'lib/**/*.rb',
Don't rely on Git for generating the MANIFEST. When using Git the resulting Gem will contain far too many useless files. For example, the profile/ and spec/ directories are not needed when building Gems.
2014-04-29 18:39:20 +00:00
'LICENSE',
'MANIFEST',
'*.gemspec',
Include .yardopts in the Gem.
2014-04-29 18:42:09 +00:00
'README.md',
Initial setup using a C extension. While I've tried to keep Oga pure Ruby for as long as possible the performance of Ragel's Ruby output was not worth the trouble. For example, lexing 10MB of XML would take 5 to 6 seconds at least. Nokogiri on the other hand can parse that same XML into a DOM document in about 300 miliseconds. Such a big performance difference is not acceptable. To work around this the XML/HTML lexer will be implemented in C for MRI/Rubinius and Java for JRuby. For now there's only a C extension as I haven't read up yet on the JRuby API. The end goal is to provide some sort of Ragel "template" that can be used to generate the corresponding C/Java extension code. This would remove the need of duplicating the grammar and associated code. The native extension setup is a hybrid between native and Ruby. The raw Ragel stuff happens in C/Java while the actual logic of actions happens in Ruby. This adds a small amount of overhead but makes it much easier to maintain the lexer. Even with this extra overhead the performance is much better than pure Ruby. The 10MB of XML mentioned above is lexed in about 600 miliseconds. In other words, it's 10 times faster.
2014-05-04 22:25:34 +00:00
'.yardopts',
'ext/**/*.*'
Don't rely on Git for generating the MANIFEST. When using Git the resulting Gem will contain far too many useless files. For example, the profile/ and spec/ directories are not needed when building Gems.
2014-04-29 18:39:20 +00:00
)
Basic project layout.
2014-02-26 18:50:16 +00:00
Initial setup using a C extension. While I've tried to keep Oga pure Ruby for as long as possible the performance of Ragel's Ruby output was not worth the trouble. For example, lexing 10MB of XML would take 5 to 6 seconds at least. Nokogiri on the other hand can parse that same XML into a DOM document in about 300 miliseconds. Such a big performance difference is not acceptable. To work around this the XML/HTML lexer will be implemented in C for MRI/Rubinius and Java for JRuby. For now there's only a C extension as I haven't read up yet on the JRuby API. The end goal is to provide some sort of Ragel "template" that can be used to generate the corresponding C/Java extension code. This would remove the need of duplicating the grammar and associated code. The native extension setup is a hybrid between native and Ruby. The raw Ragel stuff happens in C/Java while the actual logic of actions happens in Ruby. This adds a small amount of overhead but makes it much easier to maintain the lexer. Even with this extra overhead the performance is much better than pure Ruby. The 10MB of XML mentioned above is lexed in about 600 miliseconds. In other words, it's 10 times faster.
2014-05-04 22:25:34 +00:00
Rake::ExtensionTask.new('liboga', GEMSPEC)
Basic project layout.
2014-02-26 18:50:16 +00:00
Dir['./task/*.rake'].each do |task|
import(task)
end
task :default => :test