This allows for more fine grained control over when to close certain
elements. For example, an unclosed <tr> element should be closed first
when bumping into any element other than <td> or <th>. Using the old
NodeNameSet this would mean having to list every possible HTML element
out there. Using this new setup one can just create a whitelist of the
<td> and <th> elements.
While the MIT license is a fantastic license for those too lazy (or
unable) to understand more complex licenses it's too lax when it comes
to protecting authors (= me). For example, there are no clauses regarding
patents or ownership of source code. This means that patent trolls
could, in theory, drag me to court.
Of course one can still do that when using the MPL, but at least it has
an explicit clause regarding patents. The MPL also provides a nice
balance between the MIT license and the Apache license. I don't like the
Apache license as it requires listing any significant changes in every
changed file.
In short, I don't really care what people do with my software (they
could sell it for millions for all I care), as long as they don't drag
me to court or otherwise hold me accountable for something.
When closing certain HTML elements the lexer should also close whatever
parent elements remain. For example, consider the following HTML:
<table>
<thead>
<tr>
<th>Foo
<th>Bar
<tbody>
...
</tbody>
</table>
Here the "<tbody>" element shouldn't only close the "<th>Bar" element
but also the parent "<tr>" and "<thead>" elements. This ensures we'd end
up with the following HTML:
<table>
<thead>
<tr>
<th>Foo</th>
<th>Bar</th>
</tr>
</thead>
<tbody>
...
</tbody>
</table>
Instead of garbage along the lines of this:
<table>
<thead>
<tr>
<th>Foo</th>
<th>Bar</th>
<tbody>
...
</tbody>
</table></tr></thead>
Fixes#99 (hopefully for good this time)
By using AST::Node#children directly with a splat we save ourselves an
extra method call. This in turn speeds up both the
xpath/evaluator/big_xml_average_bench.rb and
xpath/evaluator/node_matches_bench.rb benchmarks a little bit.
Using the benchmark xpath/evaluator/node_matches_bench.rb the results
prior to this commit were as following for 3 cases:
name only: 737633 i/s
namespace wildcard: 612196 i/s
name wildcard: 516030 i/s
With this commit said numbers have changed to the following:
name only: 746086 i/s
namespace wildcard: 1097168 i/s
name wildcard: 1151255 i/s
This results in the following increase of performance for each case:
name only: 1,011x (insignificant)
namespace wildcard: 1,79x
name wildcard: 2,23x
In the benchmark xpath/evaluator/big_xml_average_bench.rb the difference
isn't really noticable as said benchmark only queries elements by names,
of which the performance hasn't really improved.
This ensures that HTML such as this:
<li>foo
<li>bar
is parsed as this:
<li>foo</li>
<li>bar</li>
and not as this:
<li>
foo
<li>bar</li>
</li>
Fixes#97
This makes it easier to automatically insert preceding tokens when
starting a new element as we now have access to the name. Previously
on_element_start would be invoked first which doesn't receive an
argument.
The XML/HTML lexer is now capable of processing most invalid XML/HTML
(that I can think of at least). This is achieved by inserting missing
closing tags (where needed) and/or ignoring excessive closing tags. For
example, HTML such as this:
<a></a></p>
Results in the following tokens:
[:T_ELEM_START, nil, 1]
[:T_ELEM_NAME, 'a', 1]
[:T_ELEM_CLOSE, nil, 1]
In turn this HTML:
<a>
Results in these tokens:
[:T_ELEM_START, nil, 1]
[:T_ELEM_NAME, 'a', 1]
[:T_ELEM_CLOSE, nil, 1]
Fixes#84
Previously a single Ragel machine was used for processing HTML
script and style tags. This had the unfortunate side-effect that the
following was not parsed correctly (while being valid HTML):
<script>
var foo = "</style>";
</script>
The same applied to style tags:
<style>
/* </script> */
</style>
By using separate machines we can work around the above issue. The
downside is that this can produce multiple T_TEXT nodes, which have to
be stitched back together in the parser.
This adds lexing support for HTML/XML such as:
<foo bar="""></foo>
While technically invalid, some websites (e.g. yahoo.com) contain HTML
just like this.
The lexer handles this as following:
1. When we're in the "element_head" machine, do business as usual until
we bump into a "=".
2. Call (using Ragel's "fcall") the machine to use for processing the
attribute value (if any).
3. In this machine quoted strings are processed. The moment a string has
been processed the lexer jumps right back in to the "element_head"
machine. This ensures that any stray quotes are ignored instead of
being processed as extra attribute values (eventually leading to
parsing errors due to unbalanced quotes).
Yorick Peterse
Andrei Botalov