Commit Graph

698 Commits

Author SHA1 Message Date
Yorick Peterse d408989499 Added expanded_name for Element and Attribute 2015-08-19 20:14:23 +02:00
Yorick Peterse 25e2f57a8d XPath compiler support for local-name() 2015-08-19 20:14:23 +02:00
Yorick Peterse 49a7c2c782 Fixed incorrect spelling of "predicate" 2015-08-19 20:14:23 +02:00
Yorick Peterse db93f845f3 Renamed on_predicate methods
This ensures they all start with "on_predicate".
2015-08-19 20:14:23 +02:00
Yorick Peterse c026244f6e XPath compiler support for id() in predicates 2015-08-19 20:14:23 +02:00
Yorick Peterse 49196c285f XPath compiler support for lang() 2015-08-19 20:14:23 +02:00
Yorick Peterse eb6cf68140 Tidied up YARD documentation of the XPath compiler 2015-08-19 20:14:23 +02:00
Yorick Peterse 54473d9865 Allow followed_by to take a block
This removes the need for a lot of local variables in the Compiler
class, at the cost of some extra indentation levels.
2015-08-19 20:14:22 +02:00
Yorick Peterse 7bcd462e22 XPath compiler support for id()
This has been largely ported over from the Evaluator implementation.
2015-08-19 20:14:22 +02:00
Yorick Peterse 2ff1f9ab4f XPath compiler support for a bucket of functions
This includes the following functions:

* boolean()
* ceiling()
* floor()
* round()
* concat()
* contains()
* count()
2015-08-19 20:14:22 +02:00
Yorick Peterse e3b45fddfc to_float support for non String values 2015-08-19 20:14:22 +02:00
Yorick Peterse f3f3c7d31c Cleaned up literal usage in the XPath compiler 2015-08-19 20:14:22 +02:00
Yorick Peterse 44bd0751bc Removed commented out code
Something something I should just get a decent debugging mode.
2015-08-19 20:14:22 +02:00
Yorick Peterse e4e777ac4a XPath compiler support for preceding-sibling 2015-08-19 20:14:22 +02:00
Yorick Peterse 7362a783b8 XPath compiler support for the preceding axis 2015-08-19 20:14:22 +02:00
Yorick Peterse 67bc338474 XPath compiler support for the namespace axis 2015-08-19 20:14:22 +02:00
Yorick Peterse 07b52fb48a Added Ruby::Node#not
This is a shortcut for "!foo". Using this method one doesn't have to
worry about how the "!" operator binds. For example, this:

    !foo.or(bar)

would be parsed/evaluated as this:

    !(foo.or(bar))

when instead we want it to be this:

    (!foo).or(bar)

Using explicit parenthesis leads to ugly code, so now we can do this
instead:

    foo.not.or(bar)
2015-08-19 20:14:22 +02:00
Yorick Peterse 4698e98632 XPath compiler support for the following axis 2015-08-19 20:14:22 +02:00
Yorick Peterse e04ca9be35 XPath compiler Support for following-sibling
This is a direct port of the same handler used in the Evaluator class.
The code is a bit rough on the edges but this will be cleaned up in
upcoming commits.
2015-08-19 20:14:22 +02:00
Yorick Peterse 02ee35122a XPath compiler support for the self axis 2015-08-19 20:14:22 +02:00
Yorick Peterse bc49af02a2 XPath compiler support for the parent axis 2015-08-19 20:14:22 +02:00
Yorick Peterse ca7930cbf4 XPath compiler support for "descendant" 2015-08-19 20:14:22 +02:00
Yorick Peterse f8671a96b7 Fixed compiler support for descendant-or-self 2015-08-19 20:14:22 +02:00
Yorick Peterse 23379d6467 Fixed node() type test for the XPath compiler
This ensures the handler behaves the same as in the old XPath evaluator.
2015-08-19 20:14:22 +02:00
Yorick Peterse 8a49d9c0ee Basic compiler support for descendant-or-self
The generated code isn't entirely correct which considering the tests do
pass means the tests need to be fixed too.
2015-08-19 20:14:22 +02:00
Yorick Peterse d50f89cdf1 Use "node" for axis variables
This ensures that any nested code uses the right variables.
2015-08-19 20:14:22 +02:00
Yorick Peterse 045cfe4ab8 XPath compiler support for operators in predicates
Previously the operator methods would ignore any blocks set by the
on_predicate family of methods.
2015-08-19 20:14:22 +02:00
Yorick Peterse aa8386e6f3 Cleaned up compiler handling of axes/calls
Function call handlers don't receive a single AST node, instead they
receive the XPath arguments as separate method arguments.
2015-08-19 20:14:21 +02:00
Yorick Peterse bfc970a95a XPath compiler support for type tests 2015-08-19 20:14:21 +02:00
Yorick Peterse ef59f160c7 XPath compiler support for true()/false() 2015-08-19 20:14:21 +02:00
Yorick Peterse 2eb12eced6 XPath compiler support for all operators
Some specs still fail due to true()/false() not being implemented but
the operators themselves should work just fine.
2015-08-19 20:14:21 +02:00
Yorick Peterse 3a18d23792 to_boolean support for truthy Ruby values 2015-08-19 20:14:21 +02:00
Yorick Peterse 06ae1503d4 nodes/attributes support in to_compatible_types
This extends XPath::Conversion.to_compatible_types so that it can also
take XML::Node and XML::Attribute objects as input.
2015-08-19 20:14:21 +02:00
Yorick Peterse 376d016acd Expanded supported input for Conversion.to_float
This extends XPath::Conversion.to_float so it can also convert NodeSet
and Node instances.
2015-08-19 20:14:21 +02:00
Yorick Peterse 8a82cc3593 XPath compiler support for the "=" operator 2015-08-19 20:14:21 +02:00
Yorick Peterse 04aa8f6546 Ruby generator support for "begin" blocks 2015-08-19 20:14:21 +02:00
Yorick Peterse 92b43a7500 Renamed on_begin to on_followed_by 2015-08-19 20:14:21 +02:00
Yorick Peterse c98ba21a87 Ruby generator support for mass assignments 2015-08-19 20:14:21 +02:00
Yorick Peterse 53c142ca26 Prefix compiler predicate methods with on_ 2015-08-19 20:14:21 +02:00
Yorick Peterse bea7e19d57 Cast compiled predicate indexes to integers 2015-08-19 20:14:21 +02:00
Yorick Peterse 1f2545c03b Renamed xpath_number? to number? 2015-08-19 20:14:21 +02:00
Yorick Peterse 0e9f533358 Use return_nodeset? vs USE_NODESET 2015-08-19 20:14:21 +02:00
Yorick Peterse 9000d5efdb XPath::Compiler#compile specifies blocks for paths
Previously the on_path method itself would specify a block to use when a
node was matched. This makes it impossible to customize this behaviour
which is needed when a path is used in an operator or predicate. By
letting the #compile() method specify the block other handlers can
overwrite it.
2015-08-19 20:14:21 +02:00
Yorick Peterse e549b28ca4 Removed useless YARD docs from the XPath compiler
The method names already explain their intent.
2015-08-19 20:14:21 +02:00
Yorick Peterse 20c6bbdaa8 Removed left-over comment in the XPath compiler 2015-08-19 20:14:21 +02:00
Yorick Peterse 616fd42600 XPath compiler support for the "attribute" axis 2015-08-19 20:14:21 +02:00
Yorick Peterse 4f03bf19c1 XPath compiler support for "ancestor" 2015-08-19 20:14:21 +02:00
Yorick Peterse f96b30fb1f Tag the XPath compiler as private 2015-08-19 20:14:21 +02:00
Yorick Peterse f7484f1c8d Use each_ancestor for ancestor-or-self in XPath
The XPath compiler now uses XML::Node#each_ancestor for compiling the
"ancestor-or-self" axis.
2015-08-19 20:14:20 +02:00
Yorick Peterse 52741a3b78 Added XML::Node#each_ancestor
This method can be used to walk through the ancestor tree of a Node.
2015-08-19 20:14:20 +02:00
Yorick Peterse db39b25546 XPath compiler support for ancestor-or-self
This also comes with some changes to the specs as the old behaviour of
the Evaluator was incorrect. The Evaluator would bail after matching a
single node but instead it's meant to continue until it runs out of
parent nodes.
2015-08-19 20:14:20 +02:00
Yorick Peterse d8fbaf75d8 Ruby generator support for while loops 2015-08-19 20:14:20 +02:00
Yorick Peterse 9925b2a9c9 XPath compiler support for predicates
This currently supports index predicates (e.g. "foo[10]") and predicates
using paths (e.g. foo[bar/baz]). The usage of boolean operators and more
complex expressions has not yet been tested as these are not yet
supported in the first place.
2015-08-19 20:14:20 +02:00
Yorick Peterse 6f6151fd52 Added Ruby generator support for Symbols 2015-08-19 20:14:20 +02:00
Yorick Peterse 05a57a807e XPath compiler support for variable references 2015-08-19 20:14:20 +02:00
Yorick Peterse cf2405998b Ruby::Generator support for #[] methods 2015-08-19 20:14:20 +02:00
Yorick Peterse 17f6b3a3bb Added simple method docs for the XPath compiler 2015-08-19 20:14:20 +02:00
Yorick Peterse 2c1b4e7cbc Support for generating "else" statements 2015-08-19 20:14:20 +02:00
Yorick Peterse ebcadd1cb8 XPath compiler support for absolute paths/integers 2015-08-19 20:14:20 +02:00
Yorick Peterse 94f7f85dc3 Added XML::Document#root_node 2015-08-19 20:14:20 +02:00
Yorick Peterse 3300a6df49 Added XPath::Compiler.compile_with_cache 2015-08-19 20:14:20 +02:00
Yorick Peterse 6d01adafc7 XPath compiler now actually returns a Proc 2015-08-19 20:14:20 +02:00
Yorick Peterse d11a754946 Basic support for compiling XPath to Ruby
This currently comes with exactly 0% test coverage. Once I've
implemented all required handler methods I'll be updating the current
evaluator tests to use the compiler instead. This removes the need for
writing an entirely new set of tests.

Currently the compiler is only capable of compiling basic expressions
such as "foo", "foo/bar" and "foo[@x="y"]/bar".
2015-08-19 20:14:20 +02:00
Yorick Peterse 337d126264 Added Ruby::Generator class
This class will be used to serialize a Ruby AST back to valid Ruby
source code (as a String).
2015-08-19 20:14:20 +02:00
Yorick Peterse 6673f176d8 Added Oga::Ruby::Node
This class will be used for building Ruby ASTs that will be generated
based on XPath expressions.
2015-08-19 20:14:20 +02:00
Yorick Peterse c25879f18e Removed some explicit returns in .rll files 2015-08-19 20:14:19 +02:00
Yorick Peterse 08c965bfbc Basic specs for the XPath compiler 2015-08-19 20:14:19 +02:00
Yorick Peterse f10b7aa065 Super basic boilerplate for the XPath compiler 2015-08-19 20:14:19 +02:00
Yorick Peterse 399403c290 Release 1.2.3 2015-08-19 12:09:14 +02:00
Yorick Peterse 4f94d03a85 Improved NodeSet performance by a factor of 50
This change is broken up in to two parts:

1. Using a Hash to track if a node is already in a NodeSet
2. Only calling take_ownership when an owner is set

== Using a Hash

Previously various methods such as NodeSet#push and NodeSet#unshift
would call Array#include? (on the internal "nodes" Array) to see if a
node is already present in the set. This is quite problematic
performance wise, especially for large NodeSets. In fact, for the
attached benchmark the vast majority of the time was spent in
Array#include? calls.

Because a NodeSet demands ordering of nodes and must be able to access
them by index (something Set can't do without relying on Enumerable), a
Hash is used to separately keep track of what nodes are in a NodeSet.
This means that checking the presence of a node is simply a matter of
checking a Hash key's presence.

== Calling take_ownership

The if-check for the "owner" variable has been moved out of the
"take_ownership" method and into the methods that call "take_ownership".
This ensures the method isn't called in the first place if no owner is
present, at the cost of slightly more code repetition. The same applies
to the "remove_ownership" method.

== Conclusion

The combined result is a speedup of about 50x when running the attached
concurrent_time_bench.rb benchmark.
2015-08-19 01:36:37 +02:00
Dan Fockler be7bc8f423 make string comparison faster 2015-08-15 06:36:27 -07:00
Dan Fockler fc38b39aa3 make string comparison a bit faster 2015-08-15 06:34:09 -07:00
Daniel Fockler 496811a23f Fixes #127 2015-08-14 16:15:49 -07:00
Yorick Peterse 0977be81bb Release 1.2.2 2015-08-14 14:49:26 +02:00
Yorick Peterse 9b98e75115 Use pop/push vs shift/unshift in each_node
While the performance difference between the old and new approach is
pretty much negligible, it's simply not needed to use #shift/#unshift
here.

Thanks to Mon_Ouie from the #ruby IRC channel for suggesting this.
2015-08-07 14:08:54 +02:00
Yorick Peterse 32b75bf62c Reset the owner ivar in LRU#synchronize
Without this the following could happen:

1. Thread A acquires the lock and sets the ownership to A.
2. Thread A yields and returns
3. Thread B tries to acquire the lock
4. At this exact moment Thread A calls the "synchronize" method again
   and sees that the "owner" variable is still set to Thread A
5. Both thread A and B can now access the underlying data in parallel,
   possibly leading to corrupted objects

This can be demonstrated using the following script:

    require 'oga'

    lru = Oga::LRU.new(64)

    threads = 50.times.map do
      Thread.new do
        loop do
          number = rand(100)

          lru[number] = number
        end
      end
    end

    threads.each(&:join)

Run this for a while on either JRuby or Rubinius and you'll end up with
errors such as "ConcurrencyError: Detected invalid array contents due to
unsynchronized modifications with concurrent users" on JRuby or
"ArgumentError: negative array size" on Rubinius.

Resetting the owner variable ensures the above can never happen. Thanks
to @chrisseaton for bringing this up earlier today.
2015-08-03 21:47:40 +02:00
Jakub Pawlowicz ed3cbe7975 Fixes #129 - lexing superfluous end tags.
Prevents a superfluous end tag of a self-closing HTML tag from
closing its parent element prematurely, for example:

```html
<object><param></param><param></param></object>
```

(note <param> is self closing) being turned into:

```html
<object><param/></object><param/>
```
2015-07-23 13:16:11 +01:00
Yorick Peterse 5d0e8c99af Release 1.2.1 2015-07-01 00:24:07 +02:00
Jakub Pawlowicz 6786dde3f0 Splits entity decoding into two steps.
Doing decimal & hex decoding separately results in a nicer code
which does not refer to string matching and slicing.
2015-06-30 17:56:26 +02:00
Jakub Pawlowicz 6fc3ef425b Fixes #118 - decoding invalid entities.
Previous regular expression was too greedy in terms of matching
letters from outside of A-F hex scope, and matching letters when
not in hex mode.
2015-06-30 17:56:26 +02:00
Yorick Peterse 8990a62224 Release 1.2.0 2015-06-30 10:58:05 +02:00
Yorick Peterse 71960fff87 Added CSS :nth() pseudo class
This is a Nokogiri extension (as far as I'm aware) but it's useful
enough to also include in Oga. Selectors such as "foo:nth(2)" are simply
compiled to XPath "descendant::foo[position() = 2]".

Fixes #123
2015-06-29 20:51:38 +02:00
Yorick Peterse d26b48feb4 CSS parsing support for commas
The lexer already had the basic plumbing in place but apparently I
completely forgot to also implement the required bits in the parser.

Fixes #121
2015-06-29 18:59:01 +02:00
Yorick Peterse dccf7a7e06 Release 1.1.0 2015-06-29 16:45:21 +02:00
Tero Tasanen 0b4791b277 Ability to replace a node with another node or string
```
element = Oga::XML::Element.new(:name => 'div')
some_node.replace(element)
```

You can also pass a `String` to  `replace` and it will be replaced with
a `Oga::XML::Text` node

```
some_node.replace('this will replace the current node with a text node')
```

closes #115
2015-06-17 21:27:50 +03:00
Yorick Peterse fe7e2e4f74 Use ternary "if" when encoding attribute values 2015-06-16 22:48:42 +02:00
Yorick Peterse 074b53c18c Fix entity encoding of attribute values
This ensures that single and double quotes are also encoded, previously
they would be left as is.

Fixes #113
2015-06-16 22:47:10 +02:00
Yorick Peterse e0837fd44e Fixed YARD parameter name 2015-06-16 11:14:22 +02:00
Yorick Peterse b701a9bdd4 Release 1.0.3 2015-06-16 11:13:06 +02:00
Yorick Peterse b6d34a406d Removed redundant returns 2015-06-16 00:51:51 +02:00
Yorick Peterse bcffd86c50 Removed usage of YARD @!attribute macros 2015-06-16 00:04:10 +02:00
Yorick Peterse 2c18a51ba9 Support for strict parsing of XML documents
Currently this only disabled the automatic insertion of closing tags, in
the future this may also disable other features if deemed worth the
effort.

Fixes #107
2015-06-15 23:53:11 +02:00
Yorick Peterse 4031c4f843 Nuked Oga::XML::Lexer#html
This method was rather pointless since there's already a "html?" method.
2015-06-15 23:45:20 +02:00
Yorick Peterse 020cd34083 Mark the XML lexer class as private
I was supposed to mark this as private for 1.0 but completely forgot.
2015-06-15 23:18:11 +02:00
Yorick Peterse af7f2674af Decoding of entities with numbers
This ensures that entities such as "&frac12;" are decoded properly.
Previously this would be ignored as the regular expression used for this
only matched [a-zA-Z].

This was adapted from PR #111.
2015-06-07 17:42:24 +02:00
Yorick Peterse 5951a6f187 Release 1.0.2 2015-06-03 06:53:31 +02:00
Yorick Peterse 4bfeea2590 Use require vs require_relative
See ruby-ll commit b27fe7cc109a39184ac984405a1e452868f3fac9 for a more
in-depth explanation of this.
2015-06-03 06:42:30 +02:00
Yorick Peterse d0d597e2d9 Allow script/template in various table elements
Fixes #105
2015-05-23 10:46:49 +02:00
Yorick Peterse 5182d0c488 Correct closing of unclosed, nested HTML elements
Previous HTML such as this would be lexed incorrectly:

    <div>
        <ul>
            <li>foo
        </ul>
        inside div
    </div>
    outside div

The lexer would see this as the following instead:

    <div>
        <ul>
            <li>foo</li>
            inside div
        </ul>
    outside div
    </div>

This commit exposes the name of the closing tag to
XML::Lexer#on_element_end (omitted for self closing tags). This can be
used to automatically close nested tags that were left open, ensuring
the above HTML is lexer correctly.

The new setup ignores namespace prefixes as these are not used in HTML,
XML in turn won't even run the code to begin with since it doesn't allow
one to leave out closing tags.
2015-05-23 09:59:50 +02:00
Yorick Peterse 3c6263d8de Updated list of elements that close <p> tags 2015-05-21 21:11:41 +02:00