+ h6ElRt^RIHt^RIHtHtHtHtHtH t H t H t H t H t HtHtHt^RIHtHtHtHtHtHtHtHt!RR]4t!RR]]4t!RR ]4t!R R ]4t!R R ]4t!RR]4t!RR] 4t !RR]4t!!RR]4t"!RR]4t#!RR]4t$]!4]!4]!4]!4]!4.t%]!4]!4]!4]!4.t&]!4] !4]!4]!4.t']!4] !4]!4]!4.t(]!4] !4] !4.t)!RR]4t*!RR]*4t+!RR]*4t,!R R!]*4t-!R"R#]*4t.!R$R%]*4t/]!4]!4]"!4]#!4]$!4.t0]!4]!4]!4]!!4.t1]!4]!4]!4]!!4.t2]!4]!4]!4]!!4.t3!R&R']*]4t4!R(R)]44t5!R*R+]44t6!R,R-]44t7!R.R/]44t8R3R0lt9]:R18Xd ]9!4R2#R2#)4a Data classes and parser implementations for *incremental* chart parsers, which use dynamic programming to efficiently parse a text. A "chart parser" derives parse trees for a text by iteratively adding "edges" to a "chart". Each "edge" represents a hypothesis about the tree structure for a subsequence of the text. The "chart" is a "blackboard" for composing and combining these hypotheses. A parser is "incremental", if it guarantees that for all i, j where i < j, all edges ending at i are built before any edges ending at j. This is appealing for, say, speech recognizer hypothesis filtering. The main parser class is ``EarleyChartParser``, which is a top-down algorithm, originally formulated by Jay Earley (1970). ) perf_counter) BottomUpPredictCombineRuleBottomUpPredictRuleCachedTopDownPredictRuleChart ChartParserEdgeIEmptyPredictRule"FilteredBottomUpPredictCombineRule!FilteredSingleEdgeFundamentalRuleLeafEdge LeafInitRuleSingleEdgeFundamentalRuleTopDownInitRule)!FeatureBottomUpPredictCombineRuleFeatureBottomUpPredictRule FeatureChartFeatureChartParserFeatureEmptyPredictRule FeatureSingleEdgeFundamentalRuleFeatureTopDownInitRuleFeatureTopDownPredictRulecPa]tRt^=toRtRtRtRtRtRt Rt Rt R t Vt R #) IncrementalChartc\;QJd".RVP44FNK 5M!RVP444Vn/Vn/VnR#)c3&"TFp.xK R#5iN.0xs& 8/usr/lib/python3/dist-packages/nltk/parse/earleychart.py .IncrementalChart.initialize..@s>,=q,=N)tuple _positions _edgelists _edge_to_cpls_indexesselfs&r! initializeIncrementalChart.initialize>sC%>DOO,=>%%>DOO,=>>  c4\VP44#r)list iteredgesr*s&r!edgesIncrementalChart.edgesIsDNN$%%r.c(RVP4#)c34"TFqFq"xK K R#5irr)redgelistedges& r!r"-IncrementalChart.iteredges..MsJ__s)r'r*s&r!r1IncrementalChart.iteredgesLsJT__JJr.c aVPV,pS/8Xd \V4#\SP44p\ V4pW@P 9dVP V4\;QJd.V3RlV4FNK 5M!V3RlV44p\VP V,V,PV.44#)c36<"TFpSV,xK R#5irr)rkey restrictionss& r!r"*IncrementalChart.select..^s=*3\#&&*sr'itersortedkeysr%r) _add_indexgetr+endr=r6 restr_keysvalss&&l r!selectIncrementalChart.selectOs??3' 2 > !L--/0 :&  ]] * OOJ 'u=*=uu=*==DMM*-c266tR@AAr.caVF+p\\V4'dK\RV,4h \;QJd".RVP 44FNK 5M!RVP 444;q0P V&\ VP4FiwrEW4,pVFVo\;QJd.V3RlV4FNK 5M!V3RlV44pVPV.4PS4KX Kk R#)Bad restriction: %sc3&"TFp/xK R#5irrrs& r!r".IncrementalChart._add_index..h1P>O">Or$c3F<"TFp\SV4!4xK R#5irgetattrrr<r7s& r!r"rNnsHZcWT3/11Z!N hasattrr ValueErrorr%r&r) enumerater' setdefaultappend r+rGr<indexrFr6 this_indexrHr7s && @r!rCIncrementalChart._add_indexasC5#&& !6!<== -2E1Pdoo>O1PEE1Pdoo>O1P,PP j)'t7MCJ uHZHuuHZHH%%dB/66t<!8r.c aSP4pVPP4F^wr4\;QJd.V3RlV4FNK 5M!V3RlV44pWB,P V.4P S4K` R#)c3F<"TFp\SV4!4xK R#5irrQrSs& r!r":IncrementalChart._register_with_indexes..tsD#s+--rTNrFr)itemsr%rYrZr+r7rFrGr\rHs&f r!_register_with_indexes'IncrementalChart._register_with_indexesqschhj!%!4!4!6 J5DD55DDDD J ! !$ + 2 24 8"7r.cfVPVP4,PV4R#r)r'rFrZ)r+r7s&&r! _append_edgeIncrementalChart._append_edgews  #**40r.cB\VP4^,4#))range num_leavesr*s&r!r&IncrementalChart._positionszsT__&*++r.)r(r'r)N)__name__ __module__ __qualname____firstlineno__r,r2r1rIrCrerhr&__static_attributes____classdictcell__ __classdict__s@r!rr=s4 &KB$= 9 1,,r.rc2a]tRt^~toRtRtRtRtVtR#)FeatureIncrementalChartc aaSPV,pS/8Xd \V4#\SP44p\ V4pVSP 9dSP V4\;QJd.VV3RlV4FNK 5M!VV3RlV44p\SP V,V,PV.44#)c3T<"TFpSPSV,4xK R#5ir)_get_type_if_possible)rr<r=r+s& r!r"1FeatureIncrementalChart.select..s' EOcD & &|C'8 9 9Zs%(r?rEsf&l r!rIFeatureIncrementalChart.selects??3' 2 > !L--/0 :&  T]] * OOJ 'u EO uu EO  DMM*-c266tR@AAr.c"aaVF+p\\V4'dK\RV,4h \;QJd".RSP 44FNK 5M!RSP 444;pSP V&\ SP4FkwrEW4,pVFXo\;QJd.VV3RlV4FNK 5M!VV3RlV44pVPV.4PS4KZ Km R#)rLc3&"TFp/xK R#5irrrs& r!r"5FeatureIncrementalChart._add_index..rOr$c3d<"TF%pSP\SV4!44xK' R#5irr{rRrr<r7r+s& r!r"rs0)..wtS/A/CDD)-0NrUr[s f& @r!rC"FeatureIncrementalChart._add_indexsC5#&& !6!<== -2E1Pdoo>O1PEE1Pdoo>O1P,PP j)'t7MCJ u)uu)%%dB/66t< !8r.c&aaSP4pSPP4F`wr4\;QJd.VV3RlV4FNK 5M!VV3RlV44pWB,P V.4P S4Kb R#)c3d<"TF%pSP\SV4!44xK' R#5irrrs& r!r"AFeatureIncrementalChart._register_with_indexes..s,LVS**74+=+?@@JrNrbrdsff r!re.FeatureIncrementalChart._register_with_indexessqhhj!%!4!4!6 J5LV55LVD J ! !$ + 2 24 8 "7r.rN) rorprqrrrIrCrersrtrus@r!rxrx~sB(=&99r.rxc&a]tRt^toRtRtVtR#)CompleteFundamentalRulec#"VP4pVPWDRVP4R7F@pVPVP44pVP WcV4'gK<VxKB R#5iT)startrF is_completelhsN)rFrInextsymmove_dot_forwardinsert_with_backpointer)r+chartgrammar left_edgerF right_edgenew_edges&&&& r!_apply_incomplete)CompleteFundamentalRule._apply_incompleteskmmo ,,Di6G6G6I' J!11*..2BCH,,X*MM  s A*A;1 A;rNrorprqrrrrsrtrus@r!rrs  r.rc4a]tRt^to]!4tRtRtVtR#) CompleterRulec#"\V\4'g'VPPWV4RjxL R#R#L5ir isinstancer _fundamental_ruleapplyr+rrr7s&&&&r!rCompleterRule.applys6$))--33EDI I I* I6AA ArN rorprqrrrrrrsrtrus@r!rr/1JJr.rc4a]tRt^to]!4tRtRtVtR#) ScannerRulec#"\V\4'd'VPPWV4RjxL R#R#L5irrrs&&&&r!rScannerRule.applys6 dH % %--33EDI I I & IrrNrrus@r!rrrr.rc]tRt^tRtR#) PredictorRulerNrorprqrrrsrr.r!rrr.rc&a]tRt^toRtRtVtR#)FilteredCompleteFundamentalRulec#v"VP4'dVPWV4RjxL R#R#L5ir)r_apply_completers&&&&r!r%FilteredCompleteFundamentalRule.applys4     ++EDA A A  As ,97 9rN)rorprqrrrrsrtrus@r!rrsBBr.rc&a]tRt^toRtRtVtR#)FeatureCompleteFundamentalRulec#"VPpVP4pVPWURVP4R7FpVP WW64RjxL K R#L 5ir)rrFrIrr)r+rrrfrrFrs&&&& r!r0FeatureCompleteFundamentalRule._apply_incompletesa  # #mmo ,,Di6G6G6I' Jxx F F F  GsAA&A$ A&rNrrus@r!rrsGGr.rc$]tRt^t]!4tRtR#)FeatureCompleterRulerNrorprqrrrrrsrr.r!rr 68r.rc$]tRt^t]!4tRtR#)FeatureScannerRulerNrrr.r!rrrr.rc]tRt^tRtR#)FeaturePredictorRulerNrrr.r!rrrr.rc@a]tRtRtoRt]^^2]3RltRRltRt Vt R#)IncrementalChartParseria An *incremental* chart parser implementing Jay Earley's parsing algorithm: | For each index end in [0, 1, ..., N]: | For each edge such that edge.end = end: | If edge is incomplete and edge.next is not a part of speech: | Apply PredictorRule to edge | If edge is incomplete and edge.next is a part of speech: | Apply ScannerRule to edge | If edge is complete: | Apply CompleterRule to edge | Return any complete parses in the chart c .WnW0nW@nWPn.Vn.VnVFhpVP ^8XdVPPV4K1VP ^8XdVP PV4K_\R4h R#)av Create a new Earley chart parser, that uses ``grammar`` to parse texts. :type grammar: CFG :param grammar: The grammar used to parse texts. :type trace: int :param trace: The level of tracing that should be used when parsing a text. ``0`` will generate no tracing output; and higher numbers will produce more verbose tracing output. :type trace_chart_width: int :param trace_chart_width: The default total width reserved for the chart in trace output. The remainder of each line will be used to display edges. :param chart_class: The class that should be used to create the charts used by this parser. z9Incremental inference rules must have NUM_EDGES == 0 or 1N) _grammar_trace_trace_chart_width _chart_class_axioms_inference_rules NUM_EDGESrZrW)r+rstrategytracetrace_chart_width chart_classrules&&&&&& r!__init__IncrementalChartParser.__init__/s4  "3' "D~~" ##D)1$%%,,T2 R r.Nc |Vf VPpVPp\V4pVPP V4VP V4pVPpVP VP4^,,pV'd\VPV44VPF'p\VPWE44pV!WGWV4K) VPp \VP4^,4Fp V^8d\RV R4\VPV R74p V 'gK<V P4p V FYp \V PWEV 44pV!WMWV4VF+pVP!4V 8XgKV P#V4K- K[ K{ V#)Nz * Processing queue: )rF)r_trace_new_edgesr0rcheck_coveragerrrmprintpretty_format_leavesrrrrlrIpoprFrZ)r+tokensrtrace_new_edgesrrtrace_edge_widthaxiom new_edgesinference_rulesrFagendar7rrs&&& r! chart_parse"IncrementalChartParser.chart_parseZso =KKE//f $$V,!!&)-- 22u7G7G7IA7MN  %,,-=> ?\\EU[[89I E)S0"MM(3%., 1 r.)rrrrrrr) rorprqrr__doc__BU_LC_INCREMENTAL_STRATEGYrrrrsrtrus@r!rrs( $,$ )V!!r.rc&a]tRtRtoRtRtVtR#)EarleyChartParseri~c >\P!W\3/VBR#r)rrEARLEY_STRATEGYr+r parser_argss&&,r!rEarleyChartParser.__init__s''V+Vr.rNrorprqrrrrsrtrus@r!rr~sWWr.rc&a]tRtRtoRtRtVtR#)IncrementalTopDownChartParseric >\P!W\3/VBR#r)rrTD_INCREMENTAL_STRATEGYrs&&,r!r&IncrementalTopDownChartParser.__init__'' 2 6A r.rNrrus@r!rr  r.rc&a]tRtRtoRtRtVtR#)IncrementalBottomUpChartParseric >\P!W\3/VBR#r)rrBU_INCREMENTAL_STRATEGYrs&&,r!r'IncrementalBottomUpChartParser.__init__rr.rNrrus@r!rrrr.rc&a]tRtRtoRtRtVtR#)(IncrementalBottomUpLeftCornerChartParseric >\P!W\3/VBR#r)rrrrs&&,r!r1IncrementalBottomUpLeftCornerChartParser.__init__s'' 5 9D r.rNrrus@r!rrrr.rc&a]tRtRtoRtRtVtR#) IncrementalLeftCornerChartParseric VP4'g \R4h\P!W\3/VBR#)zNIncrementalLeftCornerParser only works for grammars without empty productions.N) is_nonemptyrWrrLC_INCREMENTAL_STRATEGYrs&&,r!r)IncrementalLeftCornerChartParser.__init__sA""$$-  '' 2 6A r.rNrrus@r!rrs  r.rc0a]tRtRto]^]3RltRtVtR#)FeatureIncrementalChartParseric B\P!VV3RVRVRV/VBR#)rrrN)rr)r+rrrrrs&&&&&,r!r&FeatureIncrementalChartParser.__init__s@ ''    0  $    r.rN) rorprqrr"BU_LC_INCREMENTAL_FEATURE_STRATEGYrxrrsrtrus@r!rrs4+   r.rc&a]tRtRtoRtRtVtR#)FeatureEarleyChartParseric >\P!W\3/VBR#r)rrEARLEY_FEATURE_STRATEGYrs&&,r!r!FeatureEarleyChartParser.__init__s%.. 2 6A r.rNrrus@r!r r rr.r c&a]tRtRtoRtRtVtR#)$FeatureIncrementalTopDownChartParseric >\P!W\3/VBR#r)rrTD_INCREMENTAL_FEATURE_STRATEGYrs&&,r!r-FeatureIncrementalTopDownChartParser.__init__%.. : >I r.rNrrus@r!rrrr.rc&a]tRtRtoRtRtVtR#)%FeatureIncrementalBottomUpChartParseric >\P!W\3/VBR#r)rrBU_INCREMENTAL_FEATURE_STRATEGYrs&&,r!r.FeatureIncrementalBottomUpChartParser.__init__rr.rNrrus@r!rrrr.rc&a]tRtRtoRtRtVtR#)/FeatureIncrementalBottomUpLeftCornerChartParseric >\P!W\3/VBR#r)rrrrs&&,r!r8FeatureIncrementalBottomUpLeftCornerChartParser.__init__s%.. = AL r.rNrrus@r!rrrr.rcp^RIp^RIp^RIHpV!4p V'd\ R4\ V 4\ R4\ V4VP 4p \ V 4\ 4\ WR7p \4p V PV 4p \V PV P444p\4V , p V'd\V4V8XgQR4hV'dVFp\ V4K M\ R\V44V'd\ RV 4R#R#) z( A demonstration of the Earley parsers. N) demo_grammarz * Grammarz * Sentence:)rzNot all parses foundz Nr trees:zTime:) systimenltk.parse.chartrrsplitrrrr0parsesrlen) print_times print_grammar print_treesrsent numparsesrrrrrearleytrr"trees&&&&&& r!demor,s-nG k g - $K ZZ\F &M Gw 4FA   v &E %,,w}}/ 0FA6{i'?)??'D $K k3v;' gqr.__main__N)TFTz$I saw John with a dog with my cookie);rrrr rrrrrrr r r r r rrnltk.parse.featurechartrrrrrrrrrrxrrrrrrrrrrrrrrrrrrrrr rrrrr rrrr,rorr.r!r1s     >,u>,B.9. .9l 7 J+JJ)J , B&GB G%E G9=999 4 NOMO N N N  N&(#%\[\~W.W  $:  %;  /E   '=  "N N"$ #N "$ #N%'"$ &" $: