+ hx\Rt^RIt^RIt^RIHt^RIHtHtHt^RI H t !RR]4t R#)z{ A reader for corpora that consist of Tweets. It is assumed that the Tweets have been serialised into line-delimited JSON. N) CorpusReader)StreamBackedCorpusViewZipFilePathPointerconcat)TweetTokenizercha]tRt^toRt]tR]!4R3RltR Rlt R Rlt R Rlt Rt R t VtR#) TwitterCorpusReadera Reader for corpora that consist of Tweets represented as a list of line-delimited JSON. Individual Tweets can be tokenized using the default tokenizer, or by a custom tokenizer specified as a parameter to the constructor. Construct a new Tweet corpus reader for a set of documents located at the given root directory. If you made your own tweet collection in a directory called `twitter-files`, then you can initialise the reader as:: from nltk.corpus import TwitterCorpusReader reader = TwitterCorpusReader(root='/path/to/twitter-files', '.*\.json') However, the recommended approach is to set the relevant directory as the value of the environmental variable `TWITTER`, and then invoke the reader as follows:: root = os.environ['TWITTER'] reader = TwitterCorpusReader(root, '.*\.json') If you want to work directly with the raw Tweets, the `json` library can be used:: import json for tweet in reader.docs(): print(json.dumps(tweet, indent=1, sort_keys=True)) Nutf8c \P!WW$4VPVP4FNp\ V\ 4'dK\ PPV4^8XgKA\RV R24h W0n R#)z :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. :param word_tokenizer: Tokenizer for breaking the text of Tweets into smaller units, including but not limited to words. zFile z is emptyN) r__init__abspaths_fileids isinstancerospathgetsize ValueError_word_tokenizer)selfrootfileidsword_tokenizerencodingrs&&&&& `_ :return: the given file(s) as a list of dictionaries deserialised from JSON. :rtype: list(dict) T)r)rr CorpusView _read_tweets)rrrencfileids&& rdocsTwitterCorpusReader.docsNsX,0==$+M +M'T&7&7#F+M    s)A c VPV4p.pVFOpVR,p\V\4'dVPVP4pVP V4KQ V# \ dKci;i)z{ Returns only the text content of Tweets in the file(s) :return: the given file(s) as a list of Tweets. :rtype: list(str) text)r!rbytesdecoderappendKeyError)rr fulltweetstweetsjsonor$s&& rstringsTwitterCorpusReader.strings_suYYw' E V}dE**;;t}}5D d#     sA A++ A:9A:c VPV4pVPpVUu.uFqCPV4NK up#uupi)z :return: the given file(s) as a list of the text content of Tweets as as a list of words, screenanames, hashtags, URLs and punctuation symbols. :rtype: list(list(str)) )r,rtokenize)rrr* tokenizerts&& r tokenizedTwitterCorpusReader.tokenizedrs>g&(( /56v!""1%v666s>c .p\^ 4FFpVP4pV'gVu#\P!V4pVP V4KH V#)zC Assumes that each line in ``stream`` is a JSON-serialised object. )rangereadlinejsonloadsr')rstreamr*ilinetweets&& rr TwitterCorpusReader._read_tweets}sLrA??$D JJt$E MM%   r)r)N)__name__ __module__ __qualname____firstlineno____doc__rrrr r!r,r2r__static_attributes____classdictcell__) __classdict__s@rrrs@>(J !1AF.( "& 7  rr) rBr7rnltk.corpus.reader.apirnltk.corpus.reader.utilrrr nltk.tokenizerrrrrJs,  /VV(s,sr