This class implements a Tie-aware version of Normalized Discounted Cumulative Gain TNDCG@k measure. More...

#include <tndcg.h>

Inheritance diagram for quickrank::metric::ir::Tndcg:

Public Member Functions
	Tndcg (size_t k=NO_CUTOFF)

virtual	~Tndcg ()

virtual std::string	name () const
	Returns the name of the metric. More...

virtual MetricScore	evaluate_result_list (const quickrank::data::QueryResults rl, const Score scores) const

virtual std::unique_ptr< Jacobian >	jacobian (std::shared_ptr< data::RankedResults > ranked) const
	Computes the Jacobian matrix. More...

Public Member Functions inherited from quickrank::metric::ir::Ndcg
	Ndcg (size_t k=NO_CUTOFF)

virtual	~Ndcg ()

Public Member Functions inherited from quickrank::metric::ir::Dcg
	Dcg (size_t k=NO_CUTOFF)

virtual	~Dcg ()

Public Member Functions inherited from quickrank::metric::ir::Metric
	Metric (size_t k=NO_CUTOFF)
	Creates a new metric with the specified cut-off threshold. More...

virtual	~Metric ()

size_t	cutoff () const
	Returns the current cut-off of the Metric. More...

void	set_cutoff (size_t k)
	Updates the cut-off of the Metric. More...

virtual MetricScore	evaluate_dataset (const std::shared_ptr< data::Dataset > dataset, const Score *scores) const

virtual MetricScore	evaluate_dataset (const std::shared_ptr< data::VerticalDataset > dataset, const Score *scores) const

Static Public Attributes
static const std::string	NAME_ = "TNDCG"

Static Public Attributes inherited from quickrank::metric::ir::Ndcg
static const std::string	NAME_ = "NDCG"

Static Public Attributes inherited from quickrank::metric::ir::Dcg
static const std::string	NAME_ = "DCG"

Static Public Attributes inherited from quickrank::metric::ir::Metric
static const size_t	NO_CUTOFF = SIZE_MAX
	This should be used when no cut-off on the results list is required. More...

Protected Member Functions
MetricScore	compute_tndcg (const quickrank::data::QueryResults rl, const Score scores) const
	Computes the TNDCG@K of a given list of labels. More...

Protected Member Functions inherited from quickrank::metric::ir::Ndcg
MetricScore	compute_idcg (const quickrank::data::QueryResults *rl) const
	Computes the IDCG@K of a given list of labels. More...

Protected Member Functions inherited from quickrank::metric::ir::Dcg
MetricScore	compute_dcg (const Label *labels, size_t len) const
	Computes the DCG@K of a given array of labels. More...

Private Member Functions
virtual std::ostream &	put (std::ostream &os) const
	Prints the short name of the Metric, e.g., "NDCG@K". More...

Friends
std::ostream &	operator<< (std::ostream &os, const Tndcg &tndcg)

Detailed Description

This class implements a Tie-aware version of Normalized Discounted Cumulative Gain TNDCG@k measure.

see: McSherry, Frank, and Marc Najork. "Computing information retrieval performance measures efficiently in the presence of tied scores." In Advances in information retrieval, pp. 414-421. Springer Berlin Heidelberg, 2008.

Constructor & Destructor Documentation

quickrank::metric::ir::Tndcg::Tndcg ( size_t k = NO_CUTOFF )

inlineexplicit

virtual quickrank::metric::ir::Tndcg::~Tndcg ( )

inlinevirtual

Member Function Documentation

MetricScore quickrank::metric::ir::Tndcg::compute_tndcg	(	const quickrank::data::QueryResults *	rl,
		const Score *	scores
	)		const

protected

Computes the TNDCG@K of a given list of labels.

Parameters

rl	The given results list. Only labels are actually used.
scores	The scores to be used to re-order the result list.

Returns: TNDCG@K for computed on the given labels.

MetricScore quickrank::metric::ir::Tndcg::evaluate_result_list	(	const quickrank::data::QueryResults *	rl,
		const Score *	scores
	)		const

virtual

Todo:: TODO: for only zero result slist Yahoo! LTR returns 0.5 instead of 0.0.

Make this choice available.

Parameters

rl	A results list.
scores	a list of scores

Returns: The quality score of the result list.

Reimplemented from quickrank::metric::ir::Ndcg.

std::unique_ptr< Jacobian > quickrank::metric::ir::Tndcg::jacobian ( std::shared_ptr< data::RankedResults > ranked ) const

virtual

Computes the Jacobian matrix.

This is a symmetric matrix storing the metric "decrease" when two documents scores are swapped.

Parameters

rl	A results list.

Returns: A smart-pointer to the Jacobian Matrix.

Todo:: TODO: provide def implementation

Todo:: TODO: it makes sense to pre-compute weights also in ndcg

Todo:: TODO: jacobian->at is expensive, we should do this in the results list order and not in the re-sorted list

Reimplemented from quickrank::metric::ir::Ndcg.

virtual std::string quickrank::metric::ir::Tndcg::name ( ) const

inlinevirtual

Returns the name of the metric.

Reimplemented from quickrank::metric::ir::Ndcg.

std::ostream & quickrank::metric::ir::Tndcg::put ( std::ostream & os ) const

privatevirtual

Prints the short name of the Metric, e.g., "NDCG@K".

Reimplemented from quickrank::metric::ir::Ndcg.

Friends And Related Function Documentation

std::ostream& operator<<	(	std::ostream &	os,
		const Tndcg &	tndcg
	)

friend

Member Data Documentation

const std::string quickrank::metric::ir::Tndcg::NAME_ = "TNDCG"

static

The documentation for this class was generated from the following files:

include/metric/ir/tndcg.h
src/metric/ir/tndcg.cc

Public Member Functions

Static Public Attributes

Protected Member Functions

Private Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation