Earley parser

In computer science, the Earley parser is an algorithm for parsing strings that belong to a given context-free language, though (depending on the variant) it may suffer problems with certain nullable grammars. The algorithm, named after its inventor, Jay Earley, is a chart parser that uses dynamic programming; it is mainly used for parsing in computational linguistics. It was first introduced in his dissertation (and later appeared in abbreviated, more legible form in a journal).

Earley parsers are appealing because they can parse all context-free languagesTemplate:Discuss, unlike LR parsers and LL parsers, which are more typically used in compilers but which can only handle restricted classes of languages. The Earley parser executes in cubic time in the general case $O(n^3)$, where n is the length of the parsed string, quadratic time for unambiguous grammars $O(n^2)$, and linear time for almost all LR(k) grammars. It performs particularly well when the rules are written left-recursively.

Earley Recogniser

The following algorithm describes the Earley recogniser. The recogniser can be easily modified to create a parse tree as it recognises, and in that way can be turned into a parser.

The algorithm

In the following descriptions, α, β, and γ represent any string of terminals/nonterminals (including the empty string), X and Y represent single nonterminals, and a represents a terminal symbol.

Earley's algorithm is a top-down dynamic programming algorithm. In the following, we use Earley's dot notation: given a production X → αβ, the notation X → α • β represents a condition in which α has already been parsed and β is expected.

Input position 0 is the position prior to input. Input position n is the position after accepting the nth token. (Informally, input positions can be thought of as locations at token boundaries.) For every input position, the parser generates a state set. Each state is a tuple (X → α • β, i), consisting of

• the production currently being matched (X → α β)
• our current position in that production (represented by the dot)
• the position i in the input at which the matching of this production began: the origin position

(Earley's original algorithm included a look-ahead in the state; later research showed this to have little practical effect on the parsing efficiency, and it has subsequently been dropped from most implementations.)

The state set at input position k is called S(k). The parser is seeded with S(0) consisting of only the top-level rule. The parser then repeatedly executes three operations: prediction, scanning, and completion.

• Prediction: For every state in S(k) of the form (X → α • Y β, j) (where j is the origin position as above), add (Y → • γ, k) to S(k) for every production in the grammar with Y on the left-hand side (Y → γ).

• Scanning: If a is the next symbol in the input stream, for every state in S(k) of the form (X → α • a β, j), add (X → α a • β, j) to S(k+1).

• Completion: For every state in S(k) of the form (X → γ •, j), find states in S(j) of the form (Y → α • X β, i) and add (Y → α X • β, i) to S(k).

It is important to note that duplicate states are not added to the state set, only new ones. These three operations are repeated until no new states can be added to the set. The set is generally implemented as a queue of states to process, with the operation to be performed depending on what kind of state it is.

Pseudocode

Adapted from by Daniel Jurafsky and James H. Martin

function EARLEY-PARSE(words, grammar)
    ENQUEUE((γ → •S, 0), chart[0])
    for i ← from 0 to LENGTH(words) do
        for each state in chart[i] do
            if INCOMPLETE?(state) then
                if NEXT-CAT(state) is a nonterminal then
                    PREDICTOR(state, i, grammar)         // non-terminal
                else do
                    SCANNER(state, i)                    // terminal
            else do
                COMPLETER(state, i)
        end
    end
    return chart

procedure PREDICTOR((A → α•B, i), j, grammar)
    for each (B → γ) in GRAMMAR-RULES-FOR(B, grammar) do
        ADD-TO-SET((B → •γ, j), chart[ j])
    end

procedure SCANNER((A → α•B, i), j)
    if B ⊂ PARTS-OF-SPEECH(word[j]) then
        ADD-TO-SET((B → word[j], i), chart[j + 1])
    end

procedure COMPLETER((B → γ•, j), k)
    for each (A → α•Bβ, i) in chart[j] do
        ADD-TO-SET((A → αB•β, i), chart[k])
    end

Example

Consider the following simple grammar for arithmetic expressions:

<P> ::= <S>      # the start rule
<S> ::= <S> "+" <M> | <M>
<M> ::= <M> "*" <T> | <T>
<T> ::= "1" | "2" | "3" | "4"

With the input:

2 + 3 * 4

This is the sequence of state sets:

(state no.) Production (Origin) # Comment
-----------------------------------------

S(0): • 2 + 3 * 4

(1)  P → • S         (0)    # start rule
(2)  S → • S + M     (0)    # predict from (1)
(3)  S → • M         (0)    # predict from (1)
(4)  M → • M * T     (0)    # predict from (3)
(5)  M → • T         (0)    # predict from (3)
(6)  T → • number    (0)    # predict from (5)

S(1): 2 • + 3 * 4

(1)  T → number •    (0)    # scan from S(0)(6)
(2)  M → T •         (0)    # complete from (1) and S(0)(5)
(3)  M → M • * T     (0)    # complete from (2) and S(0)(4)
(4)  S → M •         (0)    # complete from (2) and S(0)(3)
(5)  S → S • + M     (0)    # complete from (4) and S(0)(2)
(6)  P → S •         (0)    # complete from (4) and S(0)(1)

S(2): 2 + • 3 * 4

(1)  S → S + • M     (0)    # scan from S(1)(5)
(2)  M → • M * T     (2)    # predict from (1)
(3)  M → • T         (2)    # predict from (1)
(4)  T → • number    (2)    # predict from (3)

S(3): 2 + 3 • * 4

(1)  T → number •    (2)    # scan from S(2)(4)
(2)  M → T •         (2)    # complete from (1) and S(2)(3)
(3)  M → M • * T     (2)    # complete from (2) and S(2)(2)
(4)  S → S + M •     (0)    # complete from (2) and S(2)(1)
(5)  S → S • + M     (0)    # complete from (4) and S(0)(2)
(6)  P → S •         (0)    # complete from (4) and S(0)(1)

S(4): 2 + 3 * • 4

(1)  M → M * • T     (2)    # scan from S(3)(3)
(2)  T → • number    (4)    # predict from (1)

S(5): 2 + 3 * 4 •

(1)  T → number •    (4)    # scan from S(4)(2)
(2)  M → M * T •     (2)    # complete from (1) and S(4)(1)
(3)  M → M • * T     (2)    # complete from (2) and S(2)(2)
(4)  S → S + M •     (0)    # complete from (2) and S(2)(1)
(5)  S → S • + M     (0)    # complete from (4) and S(0)(2)
(6)  P → S •         (0)    # complete from (4) and S(0)(1)

The state (P → S •, 0) represents a completed parse. This state also appears in S(3) and S(1), which are complete sentences.

See also

• CYK algorithm
• Context-free grammar
• Parsing Algorithms

Citations

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

Other Reference Materials

• 20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.
  
  My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
• Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.
  
  Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.
  
  In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.
  
  Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region
  
  Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.
  
  15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.
  
  To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010.

• 55 years old Systems Administrator Antony from Clarence Creek, really loves learning, PC Software and aerobics. Likes to travel and was inspired after making a journey to Historic Ensemble of the Potala Palace.
  
  You can view that web-site... ccleaner free download

External links

C Implementations

• 'early' An Earley parser C -library.
• 'C Earley Parser' An Earley parser C. Template:Dead link

Java Implementations

• PEN A Java library that implements the Earley algorithm.
• Pep A Java library that implements the Earley algorithm and provides charts and parse trees as parsing artifacts.
• [1] A Java implementation of Earley parser.

JavaScript Implementations

• 'JavaScript Moony Parser' A type of Earley parser written in JavaScript.

Perl Implementations

• Marpa::R2 and Marpa::XS, Perl modules. Marpa is an Earley's algorithm that includes the improvements made by Joop Leo, and by Aycock and Horspool.
• Parse::Earley A Perl module that implements Jay Earley's original algorithm.

Python Implementations

• Charty a Python implementation of an Earley parser.
• NLTK a Python toolkit that has an Earley parser.
• Spark an Object Oriented "little language framework" for Python that implements an Earley parser.
• earley3.py A stand-alone implementation of the algorithm in less than 150 lines of code, including generation of the parsing-forest and samples.

Common Lisp Implementations

• CL-EARLEY-PARSER A Common Lisp library that implements an Earley parser.

Scheme/Racket Implementations

• Charty-Racket A Scheme / Racket implementation of an Earley parser.

Resources

• The Accent compiler-compiler

• All But a Soul - AI Parcer in C# by Richard Earley

Real Estate Agent Renaldo Lester from Saint-Jean-Chrysostome, has several hobbies which include leathercrafting, property developers in singapore apartment for sale, this contact form, and crochet. Loves to see new cities and places like Ruins of Loropéni.