Commit Graph

564 Commits

Author SHA1 Message Date
Tero Tasanen 0b4791b277 Ability to replace a node with another node or string
```
element = Oga::XML::Element.new(:name => 'div')
some_node.replace(element)
```

You can also pass a `String` to  `replace` and it will be replaced with
a `Oga::XML::Text` node

```
some_node.replace('this will replace the current node with a text node')
```

closes #115
2015-06-17 21:27:50 +03:00
Yorick Peterse fe7e2e4f74 Use ternary "if" when encoding attribute values 2015-06-16 22:48:42 +02:00
Yorick Peterse 074b53c18c Fix entity encoding of attribute values
This ensures that single and double quotes are also encoded, previously
they would be left as is.

Fixes #113
2015-06-16 22:47:10 +02:00
Yorick Peterse e0837fd44e Fixed YARD parameter name 2015-06-16 11:14:22 +02:00
Yorick Peterse b701a9bdd4 Release 1.0.3 2015-06-16 11:13:06 +02:00
Yorick Peterse b6d34a406d Removed redundant returns 2015-06-16 00:51:51 +02:00
Yorick Peterse bcffd86c50 Removed usage of YARD @!attribute macros 2015-06-16 00:04:10 +02:00
Yorick Peterse 2c18a51ba9 Support for strict parsing of XML documents
Currently this only disabled the automatic insertion of closing tags, in
the future this may also disable other features if deemed worth the
effort.

Fixes #107
2015-06-15 23:53:11 +02:00
Yorick Peterse 4031c4f843 Nuked Oga::XML::Lexer#html
This method was rather pointless since there's already a "html?" method.
2015-06-15 23:45:20 +02:00
Yorick Peterse 020cd34083 Mark the XML lexer class as private
I was supposed to mark this as private for 1.0 but completely forgot.
2015-06-15 23:18:11 +02:00
Yorick Peterse af7f2674af Decoding of entities with numbers
This ensures that entities such as "½" are decoded properly.
Previously this would be ignored as the regular expression used for this
only matched [a-zA-Z].

This was adapted from PR #111.
2015-06-07 17:42:24 +02:00
Yorick Peterse 5951a6f187 Release 1.0.2 2015-06-03 06:53:31 +02:00
Yorick Peterse 4bfeea2590 Use require vs require_relative
See ruby-ll commit b27fe7cc109a39184ac984405a1e452868f3fac9 for a more
in-depth explanation of this.
2015-06-03 06:42:30 +02:00
Yorick Peterse d0d597e2d9 Allow script/template in various table elements
Fixes #105
2015-05-23 10:46:49 +02:00
Yorick Peterse 5182d0c488 Correct closing of unclosed, nested HTML elements
Previous HTML such as this would be lexed incorrectly:

    <div>
        <ul>
            <li>foo
        </ul>
        inside div
    </div>
    outside div

The lexer would see this as the following instead:

    <div>
        <ul>
            <li>foo</li>
            inside div
        </ul>
    outside div
    </div>

This commit exposes the name of the closing tag to
XML::Lexer#on_element_end (omitted for self closing tags). This can be
used to automatically close nested tags that were left open, ensuring
the above HTML is lexer correctly.

The new setup ignores namespace prefixes as these are not used in HTML,
XML in turn won't even run the code to begin with since it doesn't allow
one to leave out closing tags.
2015-05-23 09:59:50 +02:00
Yorick Peterse 3c6263d8de Updated list of elements that close <p> tags 2015-05-21 21:11:41 +02:00
Yorick Peterse 04948eb211 Release 1.0.1 2015-05-21 11:42:06 +02:00
Yorick Peterse 2766d5f27f Pack HTML entities using "U*"
See https://github.com/YorickPeterse/oga/issues/90#issuecomment-89859273
for more details, apparently I didn't fix this before.
2015-05-21 11:26:01 +02:00
Yorick Peterse c97c1b6899 Do not encode single/double quotes as entities
By encoding single/double quotes we can potentially break input, so lets
stop doing this. This now ensures that this:

    <foo>a"b</foo>

Is actually serialized back into the exact same instead of being
serialized into:

    <foo>a&quot;b</foo>
2015-05-21 11:23:44 +02:00
Yorick Peterse cab47f155c Release 1.0.0 2015-05-20 22:44:59 +02:00
Yorick Peterse 688a1fff0e Use blacklists/whitelists for HTML closing rules
This allows for more fine grained control over when to close certain
elements. For example, an unclosed <tr> element should be closed first
when bumping into any element other than <td> or <th>. Using the old
NodeNameSet this would mean having to list every possible HTML element
out there. Using this new setup one can just create a whitelist of the
<td> and <th> elements.
2015-05-18 00:32:29 +02:00
Yorick Peterse 132d112f5f Removed NodeNameSet class 2015-05-17 21:59:43 +02:00
Yorick Peterse cec8798694 Mark HTML_VOID_ELEMENTS as private 2015-05-17 21:59:14 +02:00
Yorick Peterse bcc101b819 Use Whitelist for HTML_VOID_ELEMENTS 2015-05-17 21:59:00 +02:00
Yorick Peterse 596a9b18d6 Use Whitelist for LITERAL_HTML_ELEMENTS 2015-05-17 21:56:38 +02:00
Yorick Peterse ca16a2976e Added Blacklist/Whitelist classes
These will be used in favour of the NodeNameSet class.
2015-05-17 21:55:06 +02:00
Yorick Peterse 1c095ddaff Added more HTML closing rules for colgroup/caption 2015-05-12 23:14:48 +02:00
Yorick Peterse 1e0b7feb02 Recursively closing of parent HTML elements
When closing certain HTML elements the lexer should also close whatever
parent elements remain. For example, consider the following HTML:

    <table>
        <thead>
            <tr>
                <th>Foo
                <th>Bar
        <tbody>
            ...
        </tbody>
    </table>

Here the "<tbody>" element shouldn't only close the "<th>Bar" element
but also the parent "<tr>" and "<thead>" elements. This ensures we'd end
up with the following HTML:

    <table>
        <thead>
            <tr>
                <th>Foo</th>
                <th>Bar</th>
            </tr>
        </thead>
        <tbody>
            ...
        </tbody>
    </table>

Instead of garbage along the lines of this:

    <table>
        <thead>
            <tr>
                <th>Foo</th>
                <th>Bar</th>
        <tbody>
            ...
        </tbody>
    </table></tr></thead>

Fixes #99 (hopefully for good this time)
2015-05-12 00:35:00 +02:00
Yorick Peterse df96b3d3bb Define the public API using YARD/semver 2015-05-11 21:34:34 +02:00
Yorick Peterse ecdeeacd76 Use Symbol#equal? instead of Symbol#==
At least on JRuby and Rubinius this can be quite a bit faster. On MRI
the difference is not really significant.
2015-05-07 23:03:03 +02:00
Yorick Peterse 5c7c4a6110 Don't use a splat with AST::Node#to_a
By using AST::Node#children directly with a splat we save ourselves an
extra method call. This in turn speeds up both the
xpath/evaluator/big_xml_average_bench.rb and
xpath/evaluator/node_matches_bench.rb benchmarks a little bit.
2015-05-07 01:23:27 +02:00
Yorick Peterse 0298e7068c Don't use Namespace#to_s when matching namespaces
This is a waste of time as it allocates a new String on every call.
2015-05-07 01:04:03 +02:00
Yorick Peterse b9145d83f8 Less html? calls in Element#available_namespaces
Previously it would always call the "html?" method, even if the
available namespaces were already set.
2015-05-07 01:02:59 +02:00
Yorick Peterse b5e63dc50e Improved perf of XPath::Evaluator#node_matches?
Using the benchmark xpath/evaluator/node_matches_bench.rb the results
prior to this commit were as following for 3 cases:

    name only:          737633 i/s
    namespace wildcard: 612196 i/s
    name wildcard:      516030 i/s

With this commit said numbers have changed to the following:

    name only:          746086  i/s
    namespace wildcard: 1097168 i/s
    name wildcard:      1151255 i/s

This results in the following increase of performance for each case:

    name only:          1,011x (insignificant)
    namespace wildcard: 1,79x
    name wildcard:      2,23x

In the benchmark xpath/evaluator/big_xml_average_bench.rb the difference
isn't really noticable as said benchmark only queries elements by names,
of which the performance hasn't really improved.
2015-05-07 00:05:25 +02:00
Yorick Peterse 69180ff686 Extra closing rules for caption/colgroup/head/body
Fixes #99
2015-05-03 01:09:07 +02:00
Yorick Peterse 4b1c296936 Automatically closing of certain HTML tags
This ensures that HTML such as this:

    <li>foo
    <li>bar

is parsed as this:

    <li>foo</li>
    <li>bar</li>

and not as this:

    <li>
        foo
        <li>bar</li>
    </li>

Fixes #97
2015-04-27 18:43:26 +02:00
Yorick Peterse 4b21a2fadc Added NodeNameSet class
This class can be used to more easily create a Set containing both
lowercase and uppercase element names.
2015-04-22 00:54:29 +02:00
Yorick Peterse 8135074a62 Merged on_element_start with on_element_name
This makes it easier to automatically insert preceding tokens when
starting a new element as we now have access to the name. Previously
on_element_start would be invoked first which doesn't receive an
argument.
2015-04-21 23:38:06 +02:00
Yorick Peterse 853d804f34 Decoding of zero padded XML entities
This would previously fail due to the lack of an explicit base to use
for Integer().
2015-04-20 00:13:15 +02:00
Yorick Peterse 13e2c3d82f Better handling of incorrect XML/HTML tags
The XML/HTML lexer is now capable of processing most invalid XML/HTML
(that I can think of at least). This is achieved by inserting missing
closing tags (where needed) and/or ignoring excessive closing tags. For
example, HTML such as this:

    <a></a></p>

Results in the following tokens:

    [:T_ELEM_START, nil, 1]
    [:T_ELEM_NAME, 'a', 1]
    [:T_ELEM_CLOSE, nil, 1]

In turn this HTML:

    <a>

Results in these tokens:

    [:T_ELEM_START, nil, 1]
    [:T_ELEM_NAME, 'a', 1]
    [:T_ELEM_CLOSE, nil, 1]

Fixes #84
2015-04-19 23:19:02 +02:00
Yorick Peterse 84e1bfc955 Release 0.3.4 2015-04-19 22:19:02 +02:00
Yorick Peterse da62fcd75d Decode XML/HTML entities in the SAX parser
This was broken when decoding was moved out of the Lexer class into
XML::Text and XML::Attribute.

Fixes #92
2015-04-18 22:03:44 +02:00
Yorick Peterse 611beb78c7 Release 0.3.3 2015-04-18 20:49:40 +02:00
Yorick Peterse 73fbbfbdbd Use separate Ragel machines for script/style tags
Previously a single Ragel machine was used for processing HTML
script and style tags. This had the unfortunate side-effect that the
following was not parsed correctly (while being valid HTML):

    <script>
    var foo = "</style>";
    </script>

The same applied to style tags:

    <style>
    /* </script> */
    </style>

By using separate machines we can work around the above issue. The
downside is that this can produce multiple T_TEXT nodes, which have to
be stitched back together in the parser.
2015-04-16 01:45:39 +02:00
Yorick Peterse a08829add5 Release 0.3.2 2015-04-15 11:38:09 +02:00
Yorick Peterse 23a441933a Use recursion for parsing string bodies
This is a little bit faster than using the * operator combined with
Array#inject.
2015-04-15 00:49:40 +02:00
Yorick Peterse b2ea20ba61 Lex processing instructions in chunks
Similar to comments (ea8b4aa92f) and CDATA
tags (8acc7fc743) processing instructions
are now lexed in separate chunks _with_ proper support for streaming
input.

Related issue: #93
2015-04-15 00:11:57 +02:00
Yorick Peterse ea8b4aa92f Lex comments in chunks
Similar to this being added for CDATA tags in
8acc7fc743 comments are now also lexed in
chunks.

Related issue: #93
2015-04-14 23:11:22 +02:00
Yorick Peterse 8acc7fc743 Lex CDATA tags in chunks
Instead of using a single token (T_CDATA) for a CDATA tag the lexer now
uses 3 tokens:

1. T_CDATA_START
2. T_CDATA_BODY
3. T_CDATA_END

The T_CDATA_BODY token can occur multiple times and is turned into a
single value in the XML parser. This is similar to the way strings are
lexed.

By changing the way CDATA tags are lexed Oga can now lex CDATA tags
containing newlines when using an IO as input. For example, this would
previously fail:

    Oga.parse_xml(StringIO.new("<![CDATA[\nfoo]]>"))

Because IO input reads input per line the input for the lexer would be
as following:

    "<![CDATA[\n"
    "foo]]>"

Related issues: #93
2015-04-14 22:45:55 +02:00
Yorick Peterse 739e3b474c Optimize Traversal#each_node allocations
This removes the need for allocating an Array for every set of child
nodes.
2015-04-12 22:45:42 +02:00