|
NAMEPlucene::Analysis::Standard::StandardTokenizer - standard tokenizerSYNOPSIS# isa Plucene::Analysis::CharTokenizer DESCRIPTIONThis is the standard tokenizer.This should be a good tokenizer for most European-language documents. METHODStoken_reThe regular expression for tokenising.normalizeRemove 's and .
Visit the GSP FreeBSD Man Page Interface. |