Commit Graph

504 Commits

Author SHA1 Message Date
Yorick Peterse d8fbaf75d8 Ruby generator support for while loops 2015-08-19 20:14:20 +02:00
Yorick Peterse 7fdf8d7460 Rewrote XPath compiler predicate specs 2015-08-19 20:14:20 +02:00
Yorick Peterse 6f6151fd52 Added Ruby generator support for Symbols 2015-08-19 20:14:20 +02:00
Yorick Peterse cf2405998b Ruby::Generator support for #[] methods 2015-08-19 20:14:20 +02:00
Yorick Peterse 2c1b4e7cbc Support for generating "else" statements 2015-08-19 20:14:20 +02:00
Yorick Peterse ac6c0d806e Updated XPath variables spec to use the compiler 2015-08-19 20:14:20 +02:00
Yorick Peterse 94f7f85dc3 Added XML::Document#root_node 2015-08-19 20:14:20 +02:00
Yorick Peterse a7744b7a5c Use the XPath compiler for XPath/CSS specs 2015-08-19 20:14:20 +02:00
Yorick Peterse 3300a6df49 Added XPath::Compiler.compile_with_cache 2015-08-19 20:14:20 +02:00
Yorick Peterse 6d01adafc7 XPath compiler now actually returns a Proc 2015-08-19 20:14:20 +02:00
Yorick Peterse 6daff674d9 Use "parse" instead of "parse_xml" 2015-08-19 20:14:20 +02:00
Yorick Peterse 337d126264 Added Ruby::Generator class
This class will be used to serialize a Ruby AST back to valid Ruby
source code (as a String).
2015-08-19 20:14:20 +02:00
Yorick Peterse 6673f176d8 Added Oga::Ruby::Node
This class will be used for building Ruby ASTs that will be generated
based on XPath expressions.
2015-08-19 20:14:20 +02:00
Yorick Peterse 08c965bfbc Basic specs for the XPath compiler 2015-08-19 20:14:19 +02:00
Daniel Fockler 496811a23f Fixes #127 2015-08-14 16:15:49 -07:00
Jakub Pawlowicz ed3cbe7975 Fixes #129 - lexing superfluous end tags.
Prevents a superfluous end tag of a self-closing HTML tag from
closing its parent element prematurely, for example:

```html
<object><param></param><param></param></object>
```

(note <param> is self closing) being turned into:

```html
<object><param/></object><param/>
```
2015-07-23 13:16:11 +01:00
Jakub Pawlowicz 6fc3ef425b Fixes #118 - decoding invalid entities.
Previous regular expression was too greedy in terms of matching
letters from outside of A-F hex scope, and matching letters when
not in hex mode.
2015-06-30 17:56:26 +02:00
Yorick Peterse 565e3da176 Added encoding comment in elements_spec.rb
This ensures that older Ruby versions don't poop their pants when
running these specs.
2015-06-29 21:09:33 +02:00
Yorick Peterse dde644cd79 Support for Unicode XML/HTML identifiers
Technically HTML only allows for ASCII names but restricting that
actually requires more work than just allowing it.
2015-06-29 21:08:01 +02:00
Laurence Lee 139985612b Lexer test for elements with inline dots. 2015-06-29 20:55:48 +02:00
Yorick Peterse 71960fff87 Added CSS :nth() pseudo class
This is a Nokogiri extension (as far as I'm aware) but it's useful
enough to also include in Oga. Selectors such as "foo:nth(2)" are simply
compiled to XPath "descendant::foo[position() = 2]".

Fixes #123
2015-06-29 20:51:38 +02:00
Yorick Peterse d26b48feb4 CSS parsing support for commas
The lexer already had the basic plumbing in place but apparently I
completely forgot to also implement the required bits in the parser.

Fixes #121
2015-06-29 18:59:01 +02:00
Yorick Peterse 3b633ff41c Relax support for HTML unquoted attribute values
This allows for parsing of HTML such as:

    <a href=lol("javascript")></a>

Here the "href" attribute would have its value set to:

    lol("javascript")

Fixes #119
2015-06-29 16:35:48 +02:00
Tero Tasanen 0b4791b277 Ability to replace a node with another node or string
```
element = Oga::XML::Element.new(:name => 'div')
some_node.replace(element)
```

You can also pass a `String` to  `replace` and it will be replaced with
a `Oga::XML::Text` node

```
some_node.replace('this will replace the current node with a text node')
```

closes #115
2015-06-17 21:27:50 +03:00
Yorick Peterse 074b53c18c Fix entity encoding of attribute values
This ensures that single and double quotes are also encoded, previously
they would be left as is.

Fixes #113
2015-06-16 22:47:10 +02:00
Yorick Peterse 2c18a51ba9 Support for strict parsing of XML documents
Currently this only disabled the automatic insertion of closing tags, in
the future this may also disable other features if deemed worth the
effort.

Fixes #107
2015-06-15 23:53:11 +02:00
Yorick Peterse fd307a0fcc Support HTML attributes without starting quotes
This allows the lexer to process input such as:

    <a href=foo"></a>

For XML input the lexer still expects properly opened/closed attribute
values.

Fixes #109
2015-06-08 06:46:08 +02:00
Yorick Peterse a76286b973 Support for spaces around attribute equal signs
This also takes care of making sure line numbers are incremented
properly.

Fixes #112
2015-06-08 06:34:49 +02:00
Yorick Peterse af7f2674af Decoding of entities with numbers
This ensures that entities such as "&frac12;" are decoded properly.
Previously this would be ignored as the regular expression used for this
only matched [a-zA-Z].

This was adapted from PR #111.
2015-06-07 17:42:24 +02:00
Yorick Peterse d2523a1082 Support whitespace in element closing tags
Fixes #108
2015-05-25 13:41:17 +02:00
Yorick Peterse d0d597e2d9 Allow script/template in various table elements
Fixes #105
2015-05-23 10:46:49 +02:00
Yorick Peterse 5182d0c488 Correct closing of unclosed, nested HTML elements
Previous HTML such as this would be lexed incorrectly:

    <div>
        <ul>
            <li>foo
        </ul>
        inside div
    </div>
    outside div

The lexer would see this as the following instead:

    <div>
        <ul>
            <li>foo</li>
            inside div
        </ul>
    outside div
    </div>

This commit exposes the name of the closing tag to
XML::Lexer#on_element_end (omitted for self closing tags). This can be
used to automatically close nested tags that were left open, ensuring
the above HTML is lexer correctly.

The new setup ignores namespace prefixes as these are not used in HTML,
XML in turn won't even run the code to begin with since it doesn't allow
one to leave out closing tags.
2015-05-23 09:59:50 +02:00
Yorick Peterse 8172de192c Dropped html_ prefix from HTML lexer specs 2015-05-23 09:48:45 +02:00
Yorick Peterse f587b49406 Move HTML lexer specs into spec/oga/html/lexer 2015-05-23 09:47:49 +02:00
Yorick Peterse c97c1b6899 Do not encode single/double quotes as entities
By encoding single/double quotes we can potentially break input, so lets
stop doing this. This now ensures that this:

    <foo>a"b</foo>

Is actually serialized back into the exact same instead of being
serialized into:

    <foo>a&quot;b</foo>
2015-05-21 11:23:44 +02:00
Yorick Peterse dc2e31e35b Added remaining HTML closing specs 2015-05-19 23:41:06 +02:00
Yorick Peterse 2f182a65fe HTML closing specs for <dd>/<dd> elements 2015-05-19 00:22:04 +02:00
Yorick Peterse 1ba801370f HTML closing specs for the <li> element 2015-05-18 21:49:36 +02:00
Yorick Peterse efeb38699a HTML closing specs for the "body" element 2015-05-18 21:44:00 +02:00
Yorick Peterse 5a74571536 Added HTML head closing specs 2015-05-18 00:32:19 +02:00
Yorick Peterse 2a1c5646f3 Reworked HTML colgroup closing specs 2015-05-18 00:32:09 +02:00
Yorick Peterse 81cf7ba9b6 Reworked HTML caption closing specs 2015-05-18 00:32:01 +02:00
Yorick Peterse 541fb2d5c3 Removed generated HTML closing specs 2015-05-18 00:31:48 +02:00
Yorick Peterse 132d112f5f Removed NodeNameSet class 2015-05-17 21:59:43 +02:00
Yorick Peterse ca16a2976e Added Blacklist/Whitelist classes
These will be used in favour of the NodeNameSet class.
2015-05-17 21:55:06 +02:00
Yorick Peterse 1c095ddaff Added more HTML closing rules for colgroup/caption 2015-05-12 23:14:48 +02:00
Yorick Peterse 1e0b7feb02 Recursively closing of parent HTML elements
When closing certain HTML elements the lexer should also close whatever
parent elements remain. For example, consider the following HTML:

    <table>
        <thead>
            <tr>
                <th>Foo
                <th>Bar
        <tbody>
            ...
        </tbody>
    </table>

Here the "<tbody>" element shouldn't only close the "<th>Bar" element
but also the parent "<tr>" and "<thead>" elements. This ensures we'd end
up with the following HTML:

    <table>
        <thead>
            <tr>
                <th>Foo</th>
                <th>Bar</th>
            </tr>
        </thead>
        <tbody>
            ...
        </tbody>
    </table>

Instead of garbage along the lines of this:

    <table>
        <thead>
            <tr>
                <th>Foo</th>
                <th>Bar</th>
        <tbody>
            ...
        </tbody>
    </table></tr></thead>

Fixes #99 (hopefully for good this time)
2015-05-12 00:35:00 +02:00
Yorick Peterse 4b1c296936 Automatically closing of certain HTML tags
This ensures that HTML such as this:

    <li>foo
    <li>bar

is parsed as this:

    <li>foo</li>
    <li>bar</li>

and not as this:

    <li>
        foo
        <li>bar</li>
    </li>

Fixes #97
2015-04-27 18:43:26 +02:00
Yorick Peterse 4b21a2fadc Added NodeNameSet class
This class can be used to more easily create a Set containing both
lowercase and uppercase element names.
2015-04-22 00:54:29 +02:00
Yorick Peterse 8135074a62 Merged on_element_start with on_element_name
This makes it easier to automatically insert preceding tokens when
starting a new element as we now have access to the name. Previously
on_element_start would be invoked first which doesn't receive an
argument.
2015-04-21 23:38:06 +02:00