+ h Rt^RIHt^RIHtHtHt^RIHtH t ^RI H t H t ^RI HtR RltRtR t!R R 4tRR ltR#)z Utility functions for parsers. )load)CFGPCFGFeatureGrammar)Chart ChartParser) FeatureChartFeatureChartParser)InsideChartParserNc T\V3/VBp\V\4'g \R4h\V\4'dVf\ pV!WaVR7#\V\ 4'dVf\pVf\pV!WaVR7#Vf\pVf\pV!WaVR7#)a6 Load a grammar from a file, and build a parser based on that grammar. The parser depends on the grammar format, and might also depend on properties of the grammar itself. The following grammar formats are currently supported: - ``'cfg'`` (CFGs: ``CFG``) - ``'pcfg'`` (probabilistic CFGs: ``PCFG``) - ``'fcfg'`` (feature-based CFGs: ``FeatureGrammar``) :type grammar_url: str :param grammar_url: A URL specifying where the grammar is located. The default protocol is ``"nltk:"``, which searches for the file in the the NLTK data package. :type trace: int :param trace: The level of tracing that should be used when parsing a text. ``0`` will generate no tracing output; and higher numbers will produce more verbose tracing output. :param parser: The class used for parsing; should be ``ChartParser`` or a subclass. If None, the class depends on the grammar format. :param chart_class: The class used for storing the chart; should be ``Chart`` or a subclass. Only used for CFGs and feature CFGs. If None, the chart class depends on the grammar format. :type beam_size: int :param beam_size: The maximum length for the parser's edge queue. Only used for probabilistic CFGs. :param load_args: Keyword parameters used when loading the grammar. See ``data.load`` for more information. z1The grammar must be a CFG, or a subclass thereof.)trace beam_size)r chart_class) r isinstancer ValueErrorrr rr rrr) grammar_urlr parserrr load_argsgrammars&&&&&, 1/usr/lib/python3/dist-packages/nltk/parse/util.py load_parserrsD;,),G gs # #OPP'4  >&Fgi@@ G^ , , >'F  &Kg DD > F  Kg DDc #"\V^R7F8wpwr#\V4VRW3RRRRR. pRPV4R,pVxK: R#5i)a A module to convert a single POS tagged sentence into CONLL format. >>> from nltk import word_tokenize, pos_tag >>> text = "This is a foobar sentence." >>> for line in taggedsent_to_conll(pos_tag(word_tokenize(text))): # doctest: +NORMALIZE_WHITESPACE ... print(line, end="") 1 This _ DT DT _ 0 a _ _ 2 is _ VBZ VBZ _ 0 a _ _ 3 a _ DT DT _ 0 a _ _ 4 foobar _ JJ JJ _ 0 a _ _ 5 sentence _ NN NN _ 0 a _ _ 6 . _ . . _ 0 a _ _ :param sentence: A single input sentence to parse :type sentence: list(tuple(str, str)) :rtype: iter(str) :return: a generator yielding a single sentence in CONLL format. )start_0a  N) enumeratestrjoin)sentenceiwordtag input_strs& rtaggedsent_to_conllr'OsR($HA6;DVT3#sCcJ IIi(4/ 7sA Ac#R"VFp\V4RjxL RxK R#L5i)a A module to convert the a POS tagged document stream (i.e. list of list of tuples, a list of sentences) and yield lines in CONLL format. This module yields one line per word and two newlines for end of sentence. >>> from nltk import word_tokenize, sent_tokenize, pos_tag >>> text = "This is a foobar sentence. Is that right?" >>> sentences = [pos_tag(word_tokenize(sent)) for sent in sent_tokenize(text)] >>> for line in taggedsents_to_conll(sentences): # doctest: +NORMALIZE_WHITESPACE ... if line: ... print(line, end="") 1 This _ DT DT _ 0 a _ _ 2 is _ VBZ VBZ _ 0 a _ _ 3 a _ DT DT _ 0 a _ _ 4 foobar _ JJ JJ _ 0 a _ _ 5 sentence _ NN NN _ 0 a _ _ 6 . _ . . _ 0 a _ _ 1 Is _ VBZ VBZ _ 0 a _ _ 2 that _ IN IN _ 0 a _ _ 3 right _ NN NN _ 0 a _ _ 4 ? _ . . _ 0 a _ _ :param sentences: Input sentences to parse :type sentence: list(list(tuple(str, str))) :rtype: iter(str) :return: a generator yielding sentences in CONLL format. Nz )r') sentencesr"s& rtaggedsents_to_conllr*is(B&x000 0s '%'c8a]tRt^toRtRRltRRltRtVtR#) TestGrammarz Unit tests for CFG. NcZWn\V^R7VnW nW0nW@nR#))r N) test_grammarrcpsuite_accept_reject)selfrr1acceptrejects&&&&&r__init__TestGrammar.__init__s%#gQ/   rc .VPEFp\VR,R,RR7R FpW#,FpVP4p\VPP V44pV'd2V'd*\4\V4VFp\V4K VR8XdV.8Xd\ RV,4hRpKV'd\ RV,4hRp K K X'gKX 'gK\R 4EK R #) aE Sentences in the test suite are divided into two classes: - grammatical (``accept``) and - ungrammatical (``reject``). If a sentence should parse according to the grammar, the value of ``trees`` will be a non-empty list. If a sentence should be rejected according to the grammar, then the value of ``trees`` will be None. doc: )endr5zSentence '%s' failed to parse'TzSentence '%s' received a parse'zAll tests passed!N)r5r6)r1printsplitlistr0parser) r4 show_treestestkeysenttokenstreestreeacceptedrejecteds && rrunTestGrammar.runsJJD $u+# -+ IID!ZZ\F v!67E!ed $)D!$K%*h B;",-MPT-T"UU'+H ",-NQU-U"VV'+H#&,&xHH)*-r)r2r3r0r1r/)NN)F) __name__ __module__ __qualname____firstlineno____doc__r7rK__static_attributes____classdictcell__) __classdict__s@rr,r,s!+!+rr,cVeVPV4p.pVPR4FpVR8XgV^,V9dKVPR^4pRp\V4^8Xd@V^,R9dV^,R9pV^,pM\V^,4pV^,pVP4pV.8XdKW7V3., pK V#)a\ Parses a string with one test sentence per line. Lines can optionally begin with: - a bool, saying if the sentence is grammatical or not, or - an int, giving the number of parse trees is should have, The result information is followed by a colon, and then the sentence. Empty lines and lines beginning with a comment char are ignored. :return: a list of tuple of sentences and expected results, where a sentence is a list of str, and a result is None, or bool, or int :param comment_chars: ``str`` of possible comment characters. :param encoding: the encoding of the string, if it is binary Nrr;)TruetrueFalsefalse)rWrX)decoder?lenint)string comment_charsencodingr)r" split_inforesultrFs&&& rextract_test_sentencesrcs$x(ILL& r>Xa[M9 ^^C+  z?a !} BB#A*::%a=Z]+%a=! R< v&'' ' r)r.NNr.)z#%;N)rQ nltk.datar nltk.grammarrrrnltk.parse.chartrrnltk.parse.featurechartrr nltk.parse.pchartr rr'r*r,rcrrrjs@22/D/6Er4#V.+.+b%r