com.atlassian.confluence.search.lucene
Class CustomJapaneseTokenizer

java.lang.Object
  extended by org.apache.lucene.util.AttributeSource
      extended by org.apache.lucene.analysis.TokenStream
          extended by org.apache.lucene.analysis.Tokenizer
              extended by com.atlassian.confluence.search.lucene.CustomJapaneseTokenizer

public final class CustomJapaneseTokenizer
extends org.apache.lucene.analysis.Tokenizer

Nasty fork from Lucene to accomodate searching half width / full width Japanese characters. Remove when http://issues.apache.org/jira/browse/LUCENE-1032 is addressed.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State
 
Field Summary
 
Fields inherited from class org.apache.lucene.analysis.Tokenizer
input
 
Constructor Summary
CustomJapaneseTokenizer(Reader in)
          Construct a token stream processing the given input.
 
Method Summary
 org.apache.lucene.analysis.Token next()
          Returns the next token in the stream, or null at EOS.
 
Methods inherited from class org.apache.lucene.analysis.Tokenizer
close, correctOffset, reset
 
Methods inherited from class org.apache.lucene.analysis.TokenStream
end, getOnlyUseNewAPI, incrementToken, next, reset, setOnlyUseNewAPI
 
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

CustomJapaneseTokenizer

public CustomJapaneseTokenizer(Reader in)
Construct a token stream processing the given input.

Parameters:
in - I/O reader
Method Detail

next

public final org.apache.lucene.analysis.Token next()
                                            throws IOException
Returns the next token in the stream, or null at EOS. See http://java.sun.com/j2se/1.3/docs/api/java/lang/Character.UnicodeBlock.html for detail.

Overrides:
next in class org.apache.lucene.analysis.TokenStream
Returns:
Token
Throws:
IOException - - throw IOException when read error
hanppened in the InputStream


Copyright © 2003-2012 Atlassian. All Rights Reserved.