com.atlassian.confluence.search.lucene
Class CustomJapaneseTokenizer

java.lang.Object
  org.apache.lucene.util.AttributeSource
      org.apache.lucene.analysis.TokenStream
          org.apache.lucene.analysis.Tokenizer
              com.atlassian.confluence.search.lucene.CustomJapaneseTokenizer

public final class CustomJapaneseTokenizer
extends org.apache.lucene.analysis.Tokenizer

Nasty fork from Lucene to accomodate searching half width / full width Japanese characters. Remove when http://issues.apache.org/jira/browse/LUCENE-1032 is addressed.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

Field Summary

Fields inherited from class org.apache.lucene.analysis.Tokenizer

input

Constructor Summary

CustomJapaneseTokenizer(Reader in)
Construct a token stream processing the given input.

Method Summary

org.apache.lucene.analysis.Token

next() Returns the next token in the stream, or null at EOS.

Methods inherited from class org.apache.lucene.analysis.Tokenizer

close, correctOffset, reset

Methods inherited from class org.apache.lucene.analysis.TokenStream

end, getOnlyUseNewAPI, incrementToken, next, reset, setOnlyUseNewAPI

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

CustomJapaneseTokenizer

public CustomJapaneseTokenizer(Reader in)

Construct a token stream processing the given input.

Parameters:

in - I/O reader

Method Detail

next

public final org.apache.lucene.analysis.Token next()
                                            throws IOException

Returns the next token in the stream, or null at EOS. See http://java.sun.com/j2se/1.3/docs/api/java/lang/Character.UnicodeBlock.html for detail.

Overrides:

next in class org.apache.lucene.analysis.TokenStream

Returns:

Token

Throws:

IOException - - throw IOException when read error
hanppened in the InputStream