Commit Graph

605 Commits

Author SHA1 Message Date
Yorick Peterse 73c5dbe636 Basic setup for lexing CSS pseudo selectors.
This includes support for the crazy 2n+1 syntax you can use with selectors such
as :nth-child().

CSS selectors: doing what XPath already does using an even crazier syntax,
because screw you.
2014-09-28 22:38:25 +02:00
Yorick Peterse ea4a429430 Lexing of various CSS operators. 2014-09-28 22:38:25 +02:00
Yorick Peterse 059e797a42 Re-organized some of the CSS lexer tests. 2014-09-28 22:38:25 +02:00
Yorick Peterse 2ede705f1b Lexing of various primitive CSS tokens.
This includes brackes, commas, pipes (used for namespaces) and more.
2014-09-28 22:38:24 +02:00
Yorick Peterse aa60115c0a Basic boilerplate for lexing CSS selectors. 2014-09-28 22:38:24 +02:00
Yorick Peterse 1400a859ce Make sure C strings always end with a NULL.
Haven't bumped into any problems just yet. However, in theory all sorts of evil
could happen here. Which is part of the problem of C: so much shit is undefined
behaviour that you can take a single step and fall in 15 holes at the same time.
In theory, because nobody bothered to actually specify it properly.
2014-09-28 22:28:55 +02:00
Yorick Peterse 0299ff1ea4 Support for Enumerator inputs in the XML lexer.
This fixes #48.
2014-09-28 22:27:30 +02:00
Yorick Peterse 27ee23eb31 Updated the changelog. 2014-09-28 22:00:41 +02:00
Yorick Peterse 5f7256eb8f Encode/decode XML entities.
When lexing XML entities such as & and < these sequences are now
converted into their "actual" forms. In turn, Oga::XML::Text#to_xml ensures they
are encoded when the method is called.

Performance wise this puts some strain on the lexer, for every T_TEXT/T_STRING
node now potentially has to have its content modified. In the benchmark
xml/lexer/string_average_bench.rb the average processing time is now about the
same as before the improvements made in
8db77c0a09. I was hoping that the lexer would
still be a bit faster, but alas this is not the case. Doing this in native code
would be a nightmare as C doesn't have a proper string replacement function. I'm
not old/sadistic enough to write on myself just yet.

This fixes #49
2014-09-28 21:53:25 +02:00
Yorick Peterse 3307e2f4d2 Remove source lines from parser error messages.
This was a gimmick in the first place. It doesn't work well with IO instances
(= requires re-reading of the input), the code was too complex and it wasn't
that useful in the end. Lets just get rid of it.

This fixes #53.
2014-09-25 22:58:06 +02:00
Yorick Peterse 8db77c0a09 Count newlines of text nodes in native code.
Instead of relying on String#count for counting newlines in text nodes, Oga now
does this in C/Java. String#count isn't exactly the fastest way of counting
characters. Performance was measured using
benchmark/xml/lexer/string_average_bench.rb. Before this patch the results were
as following:

    MRI:   0.529s
    Rbx:   4.965s
    JRuby: 0.622s

After this patch:

    MRI:   0.424s
    Rbx:   1.942s
    JRuby: 0.665s => numbers vary a bit, seem roughly the same as before

The commands used for benchmarking:

    $ rake clean # to make sure that C exts aren't shared between MRI/Rbx
    $ rake generate
    $ rake fixtures
    $ ruby benchmark/xml/lexer/string_average_bench.rb

The big difference for Rbx is probably due to the implementation of String#count
not being super fast. Some changes were made
(https://github.com/rubinius/rubinius/pull/3133) to the method, but this hasn't
been released yet.

JRuby seems to perform in a similar way, so either it was already optimizing
things for me or I suck at writing well performing Java code.

This fixes #51.
2014-09-25 22:49:11 +02:00
Yorick Peterse 4469ffc5b1 Improved HTML void element detection performance.
This ensures we only call String#downcase if we can't find an all lowercased
*and* all uppercased version of the element name. This in turn can save as many
object allocations as there are HTML opening tags.

This fixes #52.
2014-09-24 11:07:34 +02:00
Yorick Peterse c89ac91f3a Benchmark for lexing HTML void elements. 2014-09-24 10:43:49 +02:00
Yorick Peterse d004bc7233 Use advance vs lex in the HTML lexer bench. 2014-09-24 10:07:14 +02:00
Yorick Peterse c3a5ce745c Release 0.1.3 2014-09-24 00:24:00 +02:00
Yorick Peterse 75f4e81533 Use namespace_name in Attribute#to_xml
Instead of using `namespace.name` lets just use `namespace_name`. This fixes the
problem of serializing attributes where the namespace prefix is "xmlns" as the
namespace for this isn't registered by default.

This fixes #47.
2014-09-24 00:19:58 +02:00
Yorick Peterse 00579eaa8a Changed text action from @{} to %{}.
This ensures the action is only run at the end, opposed to any non final state.
2014-09-23 22:58:20 +02:00
Yorick Peterse 4818176ae0 Corrected lying documentation.
Something something I should keep that in sync in the future.
2014-09-23 17:35:18 +02:00
Yorick Peterse ee94f0dbfe Release 0.1.2. 2014-09-23 16:23:49 +02:00
Yorick Peterse 9656011d9f Updated the changelog for 0.1.2. 2014-09-23 16:18:50 +02:00
Yorick Peterse 2f5c61b3eb Small cleanup for the XPath lexer. 2014-09-19 01:10:25 +02:00
Yorick Peterse b6512afef2 Added changelog note about the date format. 2014-09-16 15:03:58 +02:00
Yorick Peterse a2e5def263 Added headings + examples to the changelog. 2014-09-16 15:01:38 +02:00
Yorick Peterse fb560429aa Added SAX parsing to the list of parsing features. 2014-09-16 14:50:48 +02:00
Yorick Peterse 79d9ed3214 Corrected YARD argument name. 2014-09-16 14:50:17 +02:00
Yorick Peterse 2e5bb2ca09 Namespace YARD Rake tasks under "doc". 2014-09-16 14:49:49 +02:00
Yorick Peterse ed6283ab7e Updated the changelog. 2014-09-16 14:49:42 +02:00
Yorick Peterse 1c7d1e30b1 README example on using the SAX parser. 2014-09-16 14:36:02 +02:00
Yorick Peterse 317b49bcf6 Implemented a basic SAX API.
This API is a little bit dodgy (similar to Nokogiri's API) due to the use of
separate parser and handler classes. This is done to ensure that the return
values of callback methods (e.g. on_element) aren't used by Racc for building
AST trees. This also ensures that whatever variables are set by the handler
don't conflict with any variables of the parser.

This fixes #42.
2014-09-16 14:30:46 +02:00
Yorick Peterse 9e935e5d24 Force the usage of the Racc Gem.
This ensures that we're loading the Gem and not the standard library version.
2014-09-16 11:58:56 +02:00
Yorick Peterse cdfeeed85f Provide somewhat more meaningful parser errors.
While still a bit cryptic this is probably as best as we can get it. An example:

    Oga.parse_xml("<namefoo:bar=\"10\"")

    parser.rb:116:in `on_error': Unexpected string on line 1: (Racc::ParseError)

    => 1: <namefoo:bar="10"

This fixes #43.
2014-09-16 01:09:06 +02:00
Yorick Peterse 32b11ef1e2 self-close certain XML/HTML elements.
When an XML element has no child nodes a self-closing tag is used. When parsing
documents/elements in HTML mode this is only done if the element is a so called
"void element" (e.g. <link> tags).

This fixes #46.
2014-09-16 00:44:38 +02:00
Yorick Peterse 6fc7e2e254 Track document types when parsing.
When parsing XML/HTML documents the corresponding document type (:html or :xml)
is stored in Document#type.
2014-09-16 00:25:51 +02:00
Yorick Peterse bd322e8716 Rake tasks for building multi-platform Gems. 2014-09-15 22:18:28 +02:00
Yorick Peterse 9959f5cda4 Don't remove namespace attributes when registering
When registering namespaces from an attributes list the attributes should _not_
be removed.

This fixes #45.
2014-09-15 22:04:03 +02:00
Yorick Peterse 795e669632 Ignore default NS when serializing elements.
When a default namespace is set (using xmlns="...") the method
XML::Element#to_xml should _not_ include the namespace prefix in the output.
2014-09-15 21:42:08 +02:00
Yorick Peterse 1abba1be9c Don't overwrite namespace_name with "xmlns".
This was a leftover from an early prototype.
2014-09-15 21:38:53 +02:00
Yorick Peterse 41526e7013 Whitespace pedantics in Element#namespace. 2014-09-15 21:37:54 +02:00
Yorick Peterse abbd8d6f84 Handle registering of default XML namespaces.
When the default namespace is registered (using xmlns="...") Oga now properly
sets the namespace of the container and all child elements.

This fixes #44.
2014-09-15 21:36:15 +02:00
Yorick Peterse ad2e040f05 Handle lexing of input such as just "</".
Previously this would cause the lexer to go in an infinite loop in the "text"
state machine.

This fixes #37.
2014-09-15 17:20:06 +02:00
Yorick Peterse b06eadc812 XPath support for absolute paths without tests.
This allows Oga to parse and evaluate the XPath expression "/". This expression
can be used to select just the root node/document.

This fixes #35.
2014-09-15 17:06:58 +02:00
Yorick Peterse 398aaf68bc Return a correct list of available namespaces.
This ensures that inner namespaces take precedence over outer namespaces.

Fixes #40.
2014-09-14 18:42:02 +02:00
Yorick Peterse 96b6ef320b Merge pull request #39 from ttasanen/fix_uc_void_tags
Ignore casing when testing for html void elements
2014-09-14 14:34:16 +02:00
Tero Tasanen 3c930433c4 Change Hash syntax to be consistent with other tests 2014-09-14 15:04:15 +03:00
Tero Tasanen d0f48e0e3c Added a lexer test case for uppercase void elements 2014-09-14 14:20:58 +03:00
Tero Tasanen 9c8dd60f92 Fix test cases
- Remove duplicate tests
- use parse_html helper
2014-09-14 14:14:51 +03:00
Tero Tasanen 9f71b1ec7b Ignore casing when testing for html void elements
Fixes #36
2014-09-14 12:02:02 +03:00
Yorick Peterse 5019e0362c Added checksum for 0.1.1 on JRuby. 2014-09-14 03:47:54 +02:00
Yorick Peterse 7db6732f14 Use proper Gem names for checksums.
This ensures that it also works for JRuby and the likes.
2014-09-14 03:47:35 +02:00
Yorick Peterse 77bcf2f17b Generate stuff before building.
Kinda obvious.
2014-09-14 03:45:33 +02:00