Class locutions recognizes multiwords belonging to a list obtained from a configuration file. More...

#include <locutions.h>

Inheritance diagram for locutions:

Collaboration diagram for locutions:

Public Member Functions
	locutions (const std::wstring &)
	Constructor.
void	add_locution (const std::wstring &)
	Add a locution rule to the multiword recognizer.
Private Member Functions
void	check (const std::wstring, std::set< std::wstring > &, bool &, bool &)
int	ComputeToken (int, sentence::iterator &, sentence &)
	Compute the right token code for word j from given state.
void	ResetActions ()
	Reset current multiword acumulator.
void	StateActions (int, int, int, sentence::const_iterator)
	Perform necessary actions in "state" reached from state "origin" via word j interpreted as code "token": Basically, when reaching a state, update accumulated multiword.
void	SetMultiwordAnalysis (sentence::iterator, int)
	Set the appropriate lemma and tag for the new multiword.
bool	ValidMultiWord (const word &)
	Perform last minute validation before effectively building multiword.
Private Attributes
std::map< std::wstring, std::wstring >	locut
	store multiword list
std::set< std::wstring >	prefixes
	store multiword prefixes
std::set< std::wstring >	acc_mw
	partially build multiword.
std::set< std::wstring >	longest_mw
std::vector< word >	components
	store mw components in case we need to recover them
int	over_longest
	count words scanned beyond last longest mw found.
std::list< analysis >	mw_analysis
	analysis assigned to the mw by the validation step
bool	mw_ambiguous
	segmentantion ambiguity status of the multiword

Detailed Description

Class locutions recognizes multiwords belonging to a list obtained from a configuration file.

Constructor & Destructor Documentation

locutions::locutions ( const std::wstring & locFile )

Constructor.

Create a multiword recognizer, loading multiword file.

References add_locution(), ERROR_CRASH, automat::Final, automat::initialState, MAX_STATES, MAX_TOKENS, util::open_utf8_file(), ST_M, ST_P, ST_STOP, automat::stopState, TK_mw, TK_mwL, TK_mwP, TK_pref, TK_prefL, TK_prefP, TRACE, and automat::trans.

Member Function Documentation

void locutions::add_locution ( const std::wstring & line )

Add a locution rule to the multiword recognizer.

References locut, and prefixes.

Referenced by locutions().

void locutions::check	(	const std::wstring	,
		std::set< std::wstring > &	,
		bool &	,
		bool &
	)		`[private]`

References locut, longest_mw, over_longest, prefixes, and TRACE.

Referenced by ComputeToken(), and ValidMultiWord().

int locutions::ComputeToken	(	int	state,
		sentence::iterator &	j,
		sentence &	se
	)		`[private, virtual]`

Compute the right token code for word j from given state.

Implements automat.

References acc_mw, check(), components, util::int2wstring(), over_longest, TK_mw, TK_other, TK_pref, and TRACE.

void locutions::ResetActions ( ) [private, virtual]

Reset current multiword acumulator.

Implements automat.

References acc_mw, components, longest_mw, and mw_analysis.

void locutions::SetMultiwordAnalysis	(	sentence::iterator	i,
		int	fstate
	)		`[private, virtual]`

Set the appropriate lemma and tag for the new multiword.

Implements automat.

References mw_ambiguous, mw_analysis, and TRACE.

void locutions::StateActions	(	int	origin,
		int	state,
		int	token,
		sentence::const_iterator	j
	)		`[private, virtual]`

Perform necessary actions in "state" reached from state "origin" via word j interpreted as code "token": Basically, when reaching a state, update accumulated multiword.

Implements automat.

References longest_mw, and TRACE.

bool locutions::ValidMultiWord ( const word & w ) [private, virtual]