com.atlassian.confluence.search.lucene
Class CustomJapaneseTokenizer

java.lang.Object
  extended by org.apache.lucene.analysis.TokenStream
      extended by org.apache.lucene.analysis.Tokenizer
          extended by com.atlassian.confluence.search.lucene.CustomJapaneseTokenizer

public final class CustomJapaneseTokenizer
extends org.apache.lucene.analysis.Tokenizer

Nasty fork from Lucene to accomodate searching half width / full width Japanese characters. Remove when http://issues.apache.org/jira/browse/LUCENE-1032 is addressed.


Field Summary
 
Fields inherited from class org.apache.lucene.analysis.Tokenizer
input
 
Constructor Summary
CustomJapaneseTokenizer(java.io.Reader in)
          Construct a token stream processing the given input.
 
Method Summary
 org.apache.lucene.analysis.Token next()
          Returns the next token in the stream, or null at EOS.
 
Methods inherited from class org.apache.lucene.analysis.Tokenizer
close
 
Methods inherited from class org.apache.lucene.analysis.TokenStream
reset
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CustomJapaneseTokenizer

public CustomJapaneseTokenizer(java.io.Reader in)
Construct a token stream processing the given input.

Parameters:
in - I/O reader
Method Detail

next

public final org.apache.lucene.analysis.Token next()
                                            throws java.io.IOException
Returns the next token in the stream, or null at EOS. See http://java.sun.com/j2se/1.3/docs/api/java/lang/Character.UnicodeBlock.html for detail.

Specified by:
next in class org.apache.lucene.analysis.TokenStream
Returns:
Token
Throws:
java.io.IOException - - throw IOException when read error
hanppened in the InputStream


Copyright © 2003-2008 Atlassian Pty Ltd. All Rights Reserved.