Package com.atlassian.confluence.impl.search.v2.lucene.analysis.tokenizer

Class GenericCharacterTokenizer

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.util.CharTokenizer
com.atlassian.confluence.impl.search.v2.lucene.analysis.tokenizer.GenericCharacterTokenizer
All Implemented Interfaces:
Closeable, AutoCloseable
public class GenericCharacterTokenizer extends org.apache.lucene.analysis.util.CharTokenizer
Character tokenizers that tokenizes on a specified list of delimiters.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

    org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

  • Field Summary

    Fields inherited from class org.apache.lucene.analysis.Tokenizer

    input

  • Constructor Summary

    Constructors
    Constructor
    Description
    GenericCharacterTokenizer(Reader input, char[] delimiters)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    equals(Object o)
     
    int
    hashCode()
     
    protected boolean
    isTokenChar(int c)
     

    Methods inherited from class org.apache.lucene.analysis.util.CharTokenizer

    end, incrementToken, normalize, reset

    Methods inherited from class org.apache.lucene.analysis.Tokenizer

    close, correctOffset, setReader

    Methods inherited from class org.apache.lucene.util.AttributeSource

    addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, reflectAsString, reflectWith, restoreState

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • GenericCharacterTokenizer

      public GenericCharacterTokenizer(Reader input, char[] delimiters)
      Parameters:
      input - what to tokenize passed in as a reader
      delimiters - array of delimiters to tokenize on (you must specify at least one delimiter)
      Throws:
      IllegalArgumentException - if no delimiters are specified

  • Method Details

    • isTokenChar

      protected boolean isTokenChar(int c)
      Specified by:
      isTokenChar in class org.apache.lucene.analysis.util.CharTokenizer

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class org.apache.lucene.util.AttributeSource

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class org.apache.lucene.util.AttributeSource