NAME

VERSION

SYNOPSIS

    use Lingua::ZH::WordSegmenter;

    my $segmenter = Lingua::ZH::WordSegmenter->new();
    print encode('gbk', $segmenter->seg($_) );

Description

The algorithm for this segmenter is to search the longest word at each point from both left and right directions, and choose the one with higher frequency product.

The original program is from the CPAN module Lingua::ZH::WordSegment (http://search.cpan.org/~chenyr/) I did the follwing changes: 1) make the interface object oriented; 2) make the internal string into utf8; 3) using sogou's dictionary (http://www.sogou.com/labs/dl/w.html) as the default dictionary.

METHODS

$segmenter = Lingua::ZH::WordSegmenter->new(%opinions)

This method constructs a new "Lingua::ZH::WordSegmenter" object and returns it. Key/value pair arguments may be provided to set up the initial state. The following options correspond to attribute methods described below:

   KEY            PURPOSE                       DEFAULT
   -----------    -------------                 --------------------
   dic            filename of the dic           sogou dic
   dic_encoding   encoding of the dic           "gbk"
   seperator      string to seperate wrods      " "
   verbose        show the segment process      0
    

$segmenter->seg($input,[$encoding])

Segment a input string, you can specify the encoding by the optional parameter.

The return result is encoded in utf8 format.

SEE ALSO

AUTHOR

COPYRIGHT & LICENSE

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.