+ h6Rt^RIt^RIt^RIHt^RIHtHt^RIH t Rt !RR] 4t !RR 4t !R R 4t !R R ]4tR#)z9 A reader for corpora whose documents are in MTE format. N)reduce)TaggedCorpusReaderconcat) XMLCorpusViewc$VPW4#N)findall)rootpathnss&&&8/usr/lib/python3/dist-packages/nltk/corpus/reader/mte.pyxpathr s << !!c8a]tRt^toRtRRltRRltRtVtR#) MTECorpusViewz( Class for lazy viewing the MTE Corpus. Nc4\P!WW#4R#r)r__init__)selffileidtagspec elt_handlers&&&&r rMTECorpusView.__init__stWBrc X\\R\P!WW#444#)c VRJ#rxs&r *MTECorpusView.read_block..!4-r)listfilterr read_block)rstreamrrs&&&&r r"MTECorpusView.read_blocks* '((wL   rrr)NN) __name__ __module__ __qualname____firstlineno____doc__rr"__static_attributes____classdictcell__ __classdict__s@r rrsC  rrca]tRt^"toRtRRRR/tRtRtRtR t R t R t ] R 4t ] R 4t] R4t] R4t] R4t] R4t] R4t] R4t] R4tRtRtRtRtRtRtRtRtRtRtVt R#) MTEFileReaderz Class for loading the content of the multext-east corpus. It parses the xml files and does some tag-filtering depending on the given method parameters. teizhttps://www.tei-c.org/ns/1.0xmlz%https://www.w3.org/XML/1998/namespacez{https://www.tei-c.org/ns/1.0}z'{https://www.w3.org/XML/1998/namespace}zTEI/text/body/div/div/p/s/(w|c)zTEI/text/body/div/div/p/szTEI/text/body/div/div/pcWnR#r)_MTEFileReader__file_path)r file_paths&&r rMTEFileReader.__init__3s$rcVP#r)textclseltcontexts&&&r _word_eltMTEFileReader._word_elt6s xxrcx\VRVP4Uu.uFq0PVR4NK up#uupi*N)r r r<r9r:r;ws&&& r _sent_eltMTEFileReader._sent_elt:205c30GH0G1 a&0GHHH7cx\VRVP4Uu.uFq0PVR4NK up#uupir?)r r rCr9r:r;ss&&& r _para_eltMTEFileReader._para_elt>rErFc RVP9dVPR3#VPR8Xd1VPR8Xd VPVPR,3#VPR8XdDVPR8Xd3VP\P VPR,43#\ P!R\ P!RRVP4,R,4pVPVPR,4'dcVPR8Xd VPVPR,3#VP\P VPR,43#R#) anamsd universal^-.z.*$N) attribr7_MTEFileReader__tags_MTEFileReader__tagsetMTETagConvertermsd_to_universalrecompilesubmatch)r9r:r;tagss&&& r _tagged_word_eltMTEFileReader._tagged_word_eltBs  "HHb> ! ::   5HHcjj/0 0 ZZ2 #,,+"=HHo>>szz%?PQR R::cBFF3SZZ$@@5HIDzz#**U+,,<<5(HHcjj&788'88E9JK rc \\R\VRVP4Uu.uFq0P VR4NK up44#uupi)c VRJ#rrrs&r r0MTEFileReader._tagged_sent_elt..\rrr@N)r r!r r r^rAs&&& r _tagged_sent_eltMTEFileReader._tagged_sent_eltXL '8=c38OP8O1%%a.8OP   QA c \\R\VRVP4Uu.uFq0P VR4NK up44#uupi)c VRJ#rrrs&r r0MTEFileReader._tagged_para_elt..errr@N)r r!r r rcrHs&&& r _tagged_para_eltMTEFileReader._tagged_para_eltarerfc~RVP9dVPR3#VPVPR,3#)lemmarN)rTr7r8s&&&r _lemma_word_eltMTEFileReader._lemma_word_eltjs4 #** $HHb> !HHcjj12 2rcx\VRVP4Uu.uFq0PVR4NK up#uupir?)r r rnrAs&&& r _lemma_sent_eltMTEFileReader._lemma_sent_eltq46;Ccff6MN6M##At,6MNNNrFcx\VRVP4Uu.uFq0PVR4NK up#uupir?)r r rqrHs&&& r _lemma_para_eltMTEFileReader._lemma_para_eltursrFch\VP\P\P4#r)rr3r/ word_pathr<rs&r wordsMTEFileReader.wordsy(   m55}7N7N  rch\VP\P\P4#r)rr3r/ sent_pathrCrys&r sentsMTEFileReader.sents~r|rch\VP\P\P4#r)rr3r/ para_pathrJrys&r parasMTEFileReader.parasr|rch\VP\P\P4#r)rr3r/rxrnrys&r lemma_wordsMTEFileReader.lemma_words(   m55}7T7T  rcV\nV\n\VP\P \P 4#r)r/rVrUrr3rxr^rtagsetr]s&&&r tagged_wordsMTEFileReader.tagged_words8!' #    m55}7U7U  rch\VP\P\P4#r)rr3r/r~rqrys&r lemma_sentsMTEFileReader.lemma_sentsrrcV\nV\n\VP\P \P 4#r)r/rVrUrr3r~rcrs&&&r tagged_sentsMTEFileReader.tagged_sentsrrch\VP\P\P4#r)rr3r/rrurys&r lemma_parasMTEFileReader.lemma_parasrrcV\nV\n\VP\P \P 4#r)r/rVrUrr3rrjrs&&&r tagged_parasMTEFileReader.tagged_parasrr) __file_pathN)!r%r&r'r(r)r tag_nsxml_nsrxr~rr classmethodr<rCrJr^rcrjrnrqrurzrrrrrrrrr*r+r,s@r r/r/"s- - 6 B.F 6F1I+I)I%IIII*    33 OOOO          rr/cha]tRt^toRtRRRRRRRR R R R R RRRRRRRRRRRR/ t]R4tRtVt R#)rWzi Class for converting msd tags to universal tags, more conversion options are currently not implemented. AADJSADPRADVCCONJDDETNNOUNMNUMQPRTPPRONVVERBrSrRXc V^,R8Xg V^,MV^,pV\P9gRp\PV,#)z This function converts the annotation from the Multex-East to the universal tagset as described in Chapter 5 of the NLTK-Book Unknown Tags will be mapped to X. Punctuation marks are not supported in MSD tags, so #rR)rWmapping_msd_universal)tag indicators& r rX MTETagConverter.msd_to_universalsA#&a&C-CFSV OAAAI44Y??rrN) r%r&r'r(r)r staticmethodrXr*r+r,s@r rWrWsf U U U V U V U U V V S S  @ @rrWca]tRt^toRtRRltRtRRltRRltRRlt RRlt RR lt RR lt RR lt RR ltRR ltRtVtR#)MTECorpusReaderz Reader for corpora following the TEI-p5 xml scheme, such as MULTEXT-East. MULTEXT-East contains part-of-speech-tagged words with a quite precise tagging scheme. These tags can be converted to the Universal tagset Nc B\P!WW#4RVnR#)a Construct a new MTECorpusreader for a set of documents located at the given root directory. Example usage: >>> root = '/...path to corpus.../' >>> reader = MTECorpusReader(root, 'oana-*.xml', 'utf8') # doctest: +SKIP :param root: The root directory for this corpus. (default points to location in multext config file) :param fileids: A list or regexp specifying the fileids in this corpus. (default is oana-en.xml) :param encoding: The encoding of the given files (default is utf8) z 00README.txtN)rr_readme)rr fileidsencodings&&&&r rMTECorpusReader.__init__s ##DB% rcaVfSPpM\V\4'dV.p\V3RlV4p\RV4pV'g \ R4V#)Nc"<VSP9#r)_fileids)rrs&r r+MTECorpusReader.__fileids..s1 #5rc VR9#) oana-bg.xml)rz oana-mk.xmlrrs&r rrs 1,J#Jrz$No valid multext-east file specified)r isinstancestrr!print)rrsf&r __fileidsMTECorpusReader.__fileidssP ?mmG  % %iG5w?JGT 8 9rc  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)z :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) )r_MTECorpusReader__fileidsr/osr join_rootrzrrfs&& r rzMTECorpusReader.wordss] 0 0Abggll4::q9:@@B0    A A*c  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)z :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings :rtype: list(list(str)) )rrr/rr rrrrs&& r rMTECorpusReader.sents]0 0Abggll4::q9:@@B0    rc  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)z :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word string :rtype: list(list(list(str))) )rrr/rr rrrrs&& r rMTECorpusReader.parasrrc  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)z :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of words, the corresponding lemmas and punctuation symbols, encoded as tuples (word, lemma) :rtype: list(tuple(str,str)) )rrr/rr rrrrs&& r rMTECorpusReader.lemma_wordss]0 0Abggll4::q9:FFH0    rc  VR8XgVR8Xdk\VPV4Uu.uFEp\\PP VP V44PW#4NKG up4#\R4R#uupi)a :param fileids: A list specifying the fileids that should be used. :param tagset: The tagset that should be used in the returned object, either "universal" or "msd", "msd" is the default :param tags: An MSD Tag that is used to filter all parts of the used corpus that are not more precise or at least equal to the given tag :return: the given file(s) as a list of tagged words and punctuation symbols encoded as tuples (word, tag) :rtype: list(tuple(str, str)) rPrOUnknown tagset specified.N) rrr/rr rrrrrrrr]rs&&&& r rMTECorpusReader.tagged_words, [ FeO "^^G4 5""'',,tzz1"=>KK5   - .A Bc  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)a :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of sentences or utterances, each encoded as a list of tuples of the word and the corresponding lemma (word, lemma) :rtype: list(list(tuple(str, str))) )rrr/rr rrrrs&& r rMTECorpusReader.lemma_sentsC]0 0Abggll4::q9:FFH0    rc  VR8XgVR8Xdk\VPV4Uu.uFEp\\PP VP V44PW#4NKG up4#\R4R#uupi)a :param fileids: A list specifying the fileids that should be used. :param tagset: The tagset that should be used in the returned object, either "universal" or "msd", "msd" is the default :param tags: An MSD Tag that is used to filter all parts of the used corpus that are not more precise or at least equal to the given tag :return: the given file(s) as a list of sentences or utterances, each each encoded as a list of (word,tag) tuples :rtype: list(list(tuple(str, str))) rPrOrN) rrr/rr rrrrrs&&&& r rMTECorpusReader.tagged_sentsRrrc  \VPV4Uu.uFDp\\PP VP V44P4NKF up4#uupi)a= :param fileids: A list specifying the fileids that should be used. :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a list of tuples of the word and the corresponding lemma (word, lemma) :rtype: list(List(List(tuple(str, str)))) )rrr/rr rrrrs&& r rMTECorpusReader.lemma_parasirrc  VR8XgVR8Xdk\VPV4Uu.uFEp\\PP VP V44PW#4NKG up4#\R4R#uupi)a/ :param fileids: A list specifying the fileids that should be used. :param tagset: The tagset that should be used in the returned object, either "universal" or "msd", "msd" is the default :param tags: An MSD Tag that is used to filter all parts of the used corpus that are not more precise or at least equal to the given tag :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a list of (word,tag) tuples :rtype: list(list(list(tuple(str, str)))) rPrOrN) rrr/rr rrrrrs&&&& r rMTECorpusReader.tagged_parasxs [ FeO "^^G4 5""'',,tzz1"=>KK5   - .r)r)NNutf8r)NrOrN)r%r&r'r(r)rrrzrrrrrrrrr*r+r,s@r rrsF &         /.  /.  //rr)r)rrY functoolsrnltk.corpus.readerrrnltk.corpus.reader.xmldocsrr rr/rWrrrr rsU 94" M "H H V"@"@J|/(|/r