|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectsofie.parsing.Tokenizer
sofie.parsing.Tokenizer.HTMLTokenizer
public static class Tokenizer.HTMLTokenizer
Tokenizer for HTML
Nested Class Summary |
---|
Nested classes/interfaces inherited from class sofie.parsing.Tokenizer |
---|
Tokenizer.HTMLTokenizer, Tokenizer.PatternRunner, Tokenizer.WikiTokenizer |
Field Summary | |
---|---|
protected static java.util.List<Tokenizer.PatternRunner> |
htmlPatternRunners
HTML-specific patterns |
Fields inherited from class sofie.parsing.Tokenizer |
---|
standardPatternRunners |
Constructor Summary | |
---|---|
Tokenizer.HTMLTokenizer()
|
Method Summary | |
---|---|
java.lang.CharSequence |
load(java.io.File f)
Loads the file, decodes UTF8 |
protected java.util.List<Tokenizer.PatternRunner> |
patternRunners()
Pattern runners that are to be run for this specific type of tokenizer. |
protected java.lang.String |
prepare(java.lang.CharSequence s)
Decodes also Ampersands |
Methods inherited from class sofie.parsing.Tokenizer |
---|
autodetectTokenize, canonicalizeKnownWord, forFile, main, tokenize, tokenize, tokenizeHTML, tokenizeString, tokenizeText, tokenizeWikipedia, tokenizeWikipediaNoInfobox, tokenizeWikipediaOnlyInfobox |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected static final java.util.List<Tokenizer.PatternRunner> htmlPatternRunners
Constructor Detail |
---|
public Tokenizer.HTMLTokenizer()
Method Detail |
---|
protected java.util.List<Tokenizer.PatternRunner> patternRunners()
Tokenizer
patternRunners
in class Tokenizer
protected java.lang.String prepare(java.lang.CharSequence s)
prepare
in class Tokenizer
public java.lang.CharSequence load(java.io.File f) throws java.io.IOException
load
in class Tokenizer
java.io.IOException
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |