swishej
Class Stemmer

java.lang.Object
  |
  +--swishej.Stemmer

class Stemmer

extends java.lang.Object

Class implementing stemming on a search word.

Field Summary

(package private) static AddAnE	PAddAnE
(package private) static ContainsVowel	PContainsVowel
(package private) static RemoveAnE	PRemoveAnE
(package private) static StemRule[]	step1a_rules
(package private) static StemRule[]	step1b_rules
(package private) static StemRule[]	step1b1_rules
(package private) static StemRule[]	step1c_rules
(package private) static StemRule[]	step2_rules
(package private) static StemRule[]	step3_rules
(package private) static StemRule[]	step4_rules
(package private) static StemRule[]	step5a_rules
(package private) static StemRule[]	step5b_rules

Constructor Summary

(package private)

Stemmer()

Method Summary

(package private) static boolean	endsWithCVC(java.lang.String word, java.lang.Integer[] endp) Some of the rewrite rules apply only to a root with this characteristic.
(package private) static boolean	isVowel(char c) Test character being a vowel.
(package private) static int	replaceEnd(java.lang.String[] word, StemRule[] rules, java.lang.Integer[] endp) Apply a set of rules to replace the suffix of a word.
(package private) static java.lang.String	stem(java.lang.String word) Stem a word Part 1: Check to ensure the word is all alphabetic Part 2: Run through the Porter algorithm Part 3: Return resulting stemmed word This function implements the Porter stemming algorithm, with a few additions here and there.
(package private) static int	wordSize(java.lang.String word) Count syllables in a special way: count the number vowel-consonant pairs in a word, disregarding initial consonants and final vowels.

Methods inherited from class java.lang.Object

, clone, equals, finalize, getClass, hashCode, notify, notifyAll, registerNatives, toString, wait, wait, wait

Field Detail

PContainsVowel

static final ContainsVowel PContainsVowel

PRemoveAnE

static final RemoveAnE PRemoveAnE

PAddAnE

static final AddAnE PAddAnE

step1a_rules

static final StemRule[] step1a_rules

step1b_rules

static final StemRule[] step1b_rules

step1b1_rules

static final StemRule[] step1b1_rules

step1c_rules

static final StemRule[] step1c_rules

step2_rules

static final StemRule[] step2_rules

step3_rules

static final StemRule[] step3_rules

step4_rules

static final StemRule[] step4_rules

step5a_rules

static final StemRule[] step5a_rules

step5b_rules

static final StemRule[] step5b_rules

Constructor Detail

Stemmer

Stemmer()

Method Detail

isVowel

static boolean isVowel(char c)

Test character being a vowel.

Parameters:

c - character to be tested

Returns:

true if character is vowel (a, e, i, o, or u)

endsWithCVC

static boolean endsWithCVC(java.lang.String word,
                           java.lang.Integer[] endp)

Some of the rewrite rules apply only to a root with this characteristic.

Parameters:

word - word to be tested

endp - int array reference for new word end

Returns:

true if the current word ends with a consonant-vowel-consonant combination, and the second consonant is not w, x, or y, false otherwise.

wordSize

static int wordSize(java.lang.String word)

Count syllables in a special way: count the number vowel-consonant pairs in a word, disregarding initial consonants and final vowels. The letter "y" counts as a consonant at the beginning of a word and when it has a vowel in front of it; otherwise (when it follows a consonant) it is treated as a vowel. For example, the WordSize of "cat" is 1, of "any" is 1, of "amount" is 2, of "anything" is 3. The easiest and fastest way to compute this funny measure is with a finite state machine. The initial state 0 checks the first letter. If it is a vowel, then the machine changes to state 1, which is the "last letter was a vowel" state. If the first letter is a consonant or y, then it changes to state 2, the "last letter was a consonant state". In state 1, a y is treated as a consonant (since it follows a vowel), but in state 2, y is treated as a vowel (since it follows a consonant. The result counter is incremented on the transition from state 1 to state 2, since this transition only occurs after a vowel-consonant pair, which is what we are counting.

Parameters:

word - word to be counted

Returns:

weird count of word size in adjusted syllables

replaceEnd

static int replaceEnd(java.lang.String[] word,
                      StemRule[] rules,
                      java.lang.Integer[] endp)

Apply a set of rules to replace the suffix of a word. Loop through the rule set until a match meeting all conditions is found. If a rule fires, return its id, otherwise return 0. Conditions on the length of the root are checked as part of this function's processing because this check is so often made. This is the main routine driving the stemmer. It goes through a set of suffix replacement rules looking for a match on the current suffix. When it finds one, if the root of the word is long enough, and it meets whatever other conditions are required, then the suffix is replaced, and the function returns.

Parameters:

word - string array with 0th element being word to be stemmed

rules - array of stemming rules to be applied

endp - int array for word end

Returns:

integer id for the rule fired, 0 if none has fired

stem

static java.lang.String stem(java.lang.String word)

Stem a word

• Part 1: Check to ensure the word is all alphabetic
• Part 2: Run through the Porter algorithm
• Part 3: Return resulting stemmed word

This function implements the Porter stemming algorithm, with a few additions here and there. See:

     Porter, M.F., "An Algorithm For Suffix Stripping,"
     Program 14 (3), July 1980, pp. 130-137.
 

Porter's algorithm is an ad hoc set of rewrite rules with various conditions on rule firing. The terminology of "step 1a" and so on, is taken directly from Porter's article, which unfortunately gives almost no justification for the various steps. Thus this function more or less faithfully refects the opaque presentation in the article. Changes from the article amount to a few additions to the rewrite rules; these are marked in the RuleList data structures with comments.

Parameters:

word - string to be stemmed

Returns:

new stemmed string