com.atlassian.jira.issue.index.analyzer
Class SubtokenFilter

java.lang.Object
  org.apache.lucene.util.AttributeSource
      org.apache.lucene.analysis.TokenStream
          org.apache.lucene.analysis.TokenFilter
              com.atlassian.jira.issue.index.analyzer.SubtokenFilter

All Implemented Interfaces:

Closeable

public class SubtokenFilter
extends org.apache.lucene.analysis.TokenFilter

This Filter does some final filtering on the Tokens returned by the standard Lucene tokenizers in order to create the exact tokens required for JIRA.

Currently, the StandardTokenizer takes anything of the 'alpha.alpha.alpha' form, and keeps it all together, because it htinks it may be a server hostname (like "www.atlassian.com"). This is useful, however it prevents searches on the words between the dots. An example is searching for 'NullPointerException' when 'java.lang.NullPointerException' has been indexed. This filter tokenizes the individual words, as well as the full phrase, allowing searching to be done on either. (JRA-6397)

In addition, a comma separated list of numbers (eg "123,456,789") is not tokenized at the commas. This prevents searching on just "123". This filter tokenizes the individual numbers, as well as the full phrase, allowing searching to be done on either. (JRA-7774)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

Field Summary

Fields inherited from class org.apache.lucene.analysis.TokenFilter

input

Constructor Summary

SubtokenFilter(org.apache.lucene.analysis.TokenStream tokenStream)

Method Summary

boolean

incrementToken()

Methods inherited from class org.apache.lucene.analysis.TokenFilter

close, end, reset

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

SubtokenFilter

public SubtokenFilter(org.apache.lucene.analysis.TokenStream tokenStream)

Method Detail

incrementToken

public final boolean incrementToken()
                             throws IOException

Specified by:

incrementToken in class org.apache.lucene.analysis.TokenStream

Throws:

IOException