如何从Lucene TokenStream获取令牌？

是的，这有点令人费解（与好的方式相比），但是应该这样做：

TokenStream tokenStream = analyzer.tokenStream(fieldName, reader);OffsetAttribute offsetAttribute = tokenStream.getAttribute(OffsetAttribute.class);TermAttribute termAttribute = tokenStream.getAttribute(TermAttribute.class);while (tokenStream.incrementToken()) {    int startOffset = offsetAttribute.startOffset();    int endOffset = offsetAttribute.endOffset();    String term = termAttribute.term();}

编辑： 新 方法

根据Donotello的说法，

TermAttribute

CharTermAttribute

addAttribute

getAttribute

TokenStream tokenStream = analyzer.tokenStream(fieldName, reader);OffsetAttribute offsetAttribute = tokenStream.addAttribute(OffsetAttribute.class);CharTermAttribute charTermAttribute = tokenStream.addAttribute(CharTermAttribute.class);tokenStream.reset();while (tokenStream.incrementToken()) {    int startOffset = offsetAttribute.startOffset();    int endOffset = offsetAttribute.endOffset();    String term = charTermAttribute.toString();}