#include <set>#include <map>#include <list>#include <boost/regex/icu.hpp>#include "freeling/windll.h"#include "freeling/morfo/language.h"
Include dependency graph for tokenizer.h:
This graph shows which files directly or indirectly include this file:
Go to the source code of this file.
Classes | |
| class | tokenizer |
| Class tokenizer implements a token splitter, which converts a string into a sequence of word objects, according to a set of tokenization rules read from aconfiguration file. More... | |
