Commit Graph

108 Commits

Author SHA1 Message Date
Yorick Peterse 8f2ecf62c6 Specs for XML::Element#text. 2014-07-04 09:45:16 +02:00
Yorick Peterse a15c19c4af Retrieving root nodes using Node#root_node.
This method uses a loop to traverse upwards the DOM tree in order to find the
root document/element. While this might have an impact on performance I don't
expect Oga itself to call this method very often. The benefit is that Node
instances don't require users to manually pass the top level document as an
argument.
2014-07-03 00:26:15 +02:00
Yorick Peterse e69fbc3ea7 Remove ownership when using NodeSet#delete. 2014-07-01 10:15:39 +02:00
Yorick Peterse 4b408eae4f Tests for Node#children=. 2014-07-01 10:09:42 +02:00
Yorick Peterse 75e96ee779 Test for Document#children= with a NodeSet. 2014-07-01 10:07:35 +02:00
Yorick Peterse 30845e3d65 Reliably remove nodes from owned sets.
The combination of iterating over an array and removing values from it results
in not all elements being removed. For example:

    numbers = [10, 20, 30]

    numbers.each do |num|
      numbers.delete(num)
    end

    numbers # => [20]

As a result of this the NodeSet#remove method uses two iterations:

1. One iteration to retrieve all NodeSet instances to remove nodes from.
2. One iteration to actually remove the nodes.

For the second iteration we iterate over the node sets and then over the nodes.
This ensures that we always remove all nodes instead of leaving some behind.
2014-07-01 09:57:43 +02:00
Yorick Peterse 8314d24435 First pass at using the NodeSet class.
The documentation still leaves a lot to be desired and so does the API. There
also appears to be a problem where NodeSet#remove doesn't properly remove all
nodes from a set. Outside of that we're making slow progress towards a proper
DOM API.
2014-06-30 23:03:48 +02:00
Yorick Peterse e71fe3d6fa Indexing of NodeSet instances.
Similar to arrays the NodeSet class now allows one to retrieve nodes for a
given index.
2014-06-29 23:59:27 +02:00
Yorick Peterse d2e74d8a0b Specs for the NodeSet class. 2014-06-26 19:52:03 +02:00
Yorick Peterse 21a0f50457 Added groups for code coverage results. 2014-06-23 09:43:41 +02:00
Yorick Peterse c840a33667 Extra tests for parsing XPath function calls. 2014-06-23 09:37:26 +02:00
Yorick Peterse d2f15e37d0 Corrected XPath operator precedence.
The previous commit didn't fully change the operator precedence according to
the XPath 1.0 specification. Also thanks to @whitequark for clearing up a few
things about Racc's operator precedence system.
2014-06-23 00:30:42 +02:00
Yorick Peterse a440d3f003 Fixed XPath operator precedence.
Apparently using multiple `left` rules with T_AND and T_OR being separate
solves this problem. Riiiiight....
2014-06-23 00:15:43 +02:00
Yorick Peterse 6a2f4fa82d Parsing support for more XPath operators.
This still messes up some tests due to botched token precedence (by the looks
of it).
2014-06-22 21:27:53 +02:00
Yorick Peterse 514c342cab Lex wildcards as T_IDENT instead of T_STAR. 2014-06-20 20:37:34 +02:00
Yorick Peterse 497f57ccd2 Basic parser setup for XPath function calls. 2014-06-17 19:57:17 +02:00
Yorick Peterse 894de7f909 Lex all XPath expressions in a single machine.
This allows literal values such as strings and numbers to be used as function
arguments.
2014-06-17 19:56:57 +02:00
Yorick Peterse bb7af98257 Updated used ASTs for all XPath parser specs. 2014-06-17 18:51:33 +02:00
Yorick Peterse 2298ef618b Reworked handling of relative vs absolute XPaths. 2014-06-16 20:19:39 +02:00
Yorick Peterse a733869f53 Use (name) nodes for @foo axes. 2014-06-13 18:00:55 +02:00
Yorick Peterse ce23cce9bd Added XPath parser specs for all axes. 2014-06-13 16:20:53 +02:00
Yorick Peterse 3ff9057858 Renamed the XPath axes_spec file. 2014-06-13 09:32:46 +02:00
Yorick Peterse f557aaef76 Lexer specs for the various XPath axes. 2014-06-13 00:16:40 +02:00
Yorick Peterse eba2d9954d Support for parsing basic XPath expressions. 2014-06-12 00:20:46 +02:00
Yorick Peterse 70f3b7fa92 Lex XPath operators using individual tokens.
Instead of lexing every operator as T_OP they now use individual tokens such as
T_EQ and T_LT.
2014-06-09 23:35:54 +02:00
Yorick Peterse 114bc0d6e8 Upgrade to RSpec 3.0.
For this I've enabled both the old expectation and stubbing/mocking syntax. The
old syntax is much more compact and to me reads nicer. For example, consider
the following:

    lex('<foo></foo>').should == [...]

To me this reads much nicer than this:

    expect(lex('<foo></foo>')).to eq([...])
2014-06-02 12:09:38 +02:00
Yorick Peterse e11b9ed32c Basic XPath parser setup. 2014-06-01 23:02:28 +02:00
Yorick Peterse 54de2df0c7 Support for lexing XPath wildcard expressions.
To support this we need to require whitespace around the "*" operator. This is
not ideal but it will do for now.
2014-06-01 23:01:24 +02:00
Yorick Peterse b7855c124f XPath lexer test for predicates using axes. 2014-06-01 19:49:37 +02:00
Yorick Peterse 88b06e247d Added a few more XPath lexer tests. 2014-06-01 19:43:36 +02:00
Yorick Peterse 8dd8d7a519 Basic working XPath lexer.
This doesn't lex everything of the XPath specification just yet and needs more
tests.
2014-06-01 19:24:35 +02:00
Yorick Peterse 28edc7726f Rewind IO input upon resetting the lexer. 2014-05-26 00:33:20 +02:00
Yorick Peterse 629dcd3fe6 Support for IO inputs in the lexer.
Using IO/StringIO objects one can parse large XML files without first having to
read the entire file into memory. This can potentially save a lot of memory at
the cost of a slightly slower runtime.

For IO like instances the lexer will consume the input line by line. If a
String is given it's consumed as a whole instead. A small side effect of
reading the input line by line is that text such as "foo\nbar" will be lexed as
two tokens instead of one.

Fixes #19.
2014-05-26 00:30:39 +02:00
Yorick Peterse c56b0395e4 Moved various rules around for the XML lexer.
This moves the element related rules to the element_head machine (where they
belong). This in turn makes it possible to lex ">" as a text node, previously
this was impossible.
2014-05-21 00:04:53 +02:00
Yorick Peterse cd0f3380c4 Merge multiple CDATA tokens into a single token.
The tokens T_CDATA_START, T_TEXT and T_CDATA_END have been merged together into
T_CDATA.
2014-05-19 09:36:19 +02:00
Yorick Peterse a4fb5c1299 Merge multiple comment tokens into a single one.
The tokens T_COMMENT_START, T_TEXT and T_COMMENT_END have been merged into a
single token: T_COMMENT. This simplifies both the lexer and the parser.
2014-05-19 09:30:30 +02:00
Yorick Peterse 723a273e4f Enforce symbols for element attributes.
This comes with a little bit of memory overhead but this should be minor in
most cases.
2014-05-15 01:04:26 +02:00
Yorick Peterse 19f04f98f7 Support for lexing/parsing inline doctypes. 2014-05-10 00:28:11 +02:00
Yorick Peterse 83f6d5437e Contextual pull parsing.
This adds the ability to more easily act upon specific node types and nestings
when using the pull parsing API.

A basic example of this API looks like the following (only including relevant
code):

    parser.parse do |node|
      parser.on(:element, %w{people person}) do
        people << {:name => nil, :age => nil}
      end

      parser.on(:text, %w{people person name}) do
        people.last[:name] = node.text
      end

      parser.on(:text, %w{people person age}) do
        people.last[:age] = node.text.to_i
      end
    end

This fixes #6.
2014-04-29 23:05:49 +02:00
Yorick Peterse 1a413998a3 Track the current node in the pull parser.
The current node is tracked in the instance method `node`.
2014-04-29 21:21:05 +02:00
Yorick Peterse 45b0cdf811 Track element name nesting in the pull parser.
Tracking the names of nested elements makes it a lot easier to do contextual
pull parsing. Without this it's impossible to know what context the parser is
in at a given moment.

For memory reasons the parser currently only tracks the element names. In the
future it might perhaps also track extra information to make parsing easier.
2014-04-28 23:40:36 +02:00
Yorick Peterse 030a0068bd Basic pull parsing setup.
This parser extends the regular DOM parser but instead delegates certain nodes
to a block instead of building a DOM tree.

The API is a bit raw in its current form but I'll extend it and make it a bit
more user friendly in the following commits. In particular I want to make it
easier to figure out if a certain node is nested inside another node.
2014-04-28 17:22:17 +02:00
Yorick Peterse 08d412da7e First shot at removing the AST layer.
The AST layer is being removed because it doesn't really serve a useful
purpose. In particular when creating a streaming parser the AST nodes would
only introduce extra overhead.

As a result of this the parser will instead emit a DOM tree directly instead of
first emitting an AST.
2014-04-21 23:05:39 +02:00
Yorick Peterse d9fa4b7c45 Lex input as a sequence of bytes.
Instead of lexing the input as a raw String or as a set of codepoints it's
treated as a sequence of bytes. This removes the need of String#[] (replaced by
String#byteslice) which in turn reduces the amount of memory needed and speeds
up the lexing time.

Thanks to @headius and @apeiros for suggesting this and rubber ducking along!
2014-04-17 17:45:05 +02:00
Yorick Peterse b96f7c4852 Lex attributes with namespaces.
These are lexed as just the name instead of two separate tokens.
2014-04-10 11:01:49 +02:00
Yorick Peterse 8237d5791d Stream tokens when lexing.
Instead of returning the tokens as a whole they are now streamed using
XML::Lexer#advance. This method returns the next token upon every call. It uses
a small buffer in case a particular block of text results in multiple tokens.
2014-04-09 22:08:13 +02:00
Yorick Peterse 10d0ec1573 Specs for parsing various empty nodes. 2014-04-07 21:33:23 +02:00
Yorick Peterse cb74c7edf9 Specs for XML parser errors. 2014-04-07 21:31:36 +02:00
Yorick Peterse 915d3ee505 Expanded tests for XML::Document#inspect. 2014-04-07 20:11:12 +02:00
Yorick Peterse e9412c9c4e Tests for various inspect methods. 2014-04-07 09:58:31 +02:00