An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Last update: Dec 02, 2022

Related tags

Deep Learning recsys_metrics

Overview

recsys_metrics

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Overview • Installation • How to use • Benchmark • Citation •

Overview

Highlights

Efficient (vectorized) implementations over mini-batches
Standard RecSys metrics: precision, recall, map, mrr, hr, ndcg
Beyond-accuracy metrics: e.g. coverage, diversity, novelty, etc.
All metrics support a top-k argument.

Why do we need `recsys_metrics`?

Installation

You can install recsys_metrics from PyPI:

pip install recsys_metrics

Or you can also install the latest version from source:

pip install git+https://github.com/zuoxingdong/[email protected]

Note that we support Python 3.7+ only.

How to use

Let us take Hit Rate (HR) to illustrate how to use this library:

preds = torch.tensor([
    [.5, .3, .1],
    [.3, .4, .5]
])
target = torch.tensor([
    [0, 0, 1],
    [0, 1, 1]
])
hit_rate(preds, target, k=1, reduction='mean')

>> tensor(0.5000)

The first example in the batch does not have a hit (i.e. top-1 item is not a relevant item) and second example in the batch gets a hit (i.e. top-1 item is a relevant item). Thus, we have a hit-rate of 0.5.

The API of other metrics are of the same format.

Benchmark

Metrics	Single Example	Mini-Batch
Precision
Recall
MAP
MRR
HR
NDCG

Citation

This work is inspired by Torchmetrics from PyTorchLightning Team.

Please use this bibtex if you want to cite this repository in your publications:

@misc{recsys_metrics,
      author = {Zuo, Xingdong},
      title = {recsys_metrics: An efficient PyTorch implementation of the evaluation metrics in recommender systems.},
      year = {2021},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/zuoxingdong/recsys_metrics}},
    }

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

419 Jan 3, 2023

This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

71 Dec 30, 2022

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

25 May 29, 2022

Active and Sample-Efficient Model Evaluation

Active Testing: Sample-Efficient Model Evaluation Hi, good to see you here! 👋 This is code for "Active Testing: Sample-Efficient Model Evaluation". P

19 Oct 30, 2022

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

引言感谢苏神带来的模型，原文地址：https://spaces.ac.cn/archives/8877 如何运行对应模型EfficientGlobalPoi

40 Dec 14, 2022

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

[ICLR 2021] "UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems" by Jiayi Shen, Haotao Wang*, Shupeng Gui*, Jianchao Tan, Zhangyang Wang, and Ji Liu

39 Dec 3, 2022

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering r

1.2k Dec 21, 2022

A python library for implementing a recommender system

python-recsys A python library for implementing a recommender system. Installation Dependencies python-recsys is build on top of Divisi2, with csc-pys

1.5k Dec 17, 2022

A TikTok-like recommender system for GitHub repositories based on Gorse

GitRec GitRec is the missing recommender system for GitHub repositories based on Gorse. Architecture The trending crawler crawls trending repositories

337 Jan 4, 2023

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Related tags

Overview

Overview

Highlights

Why do we need recsys_metrics?

Installation

How to use

Benchmark

Citation

You might also like...

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

Active and Sample-Efficient Model Evaluation

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

A python library for implementing a recommender system

A TikTok-like recommender system for GitHub repositories based on Gorse

Releases(v0.0.3)

v0.0.3(Aug 1, 2022)

v0.0.2(Jun 14, 2022)

Owner

Xingdong Zuo

Open source person re-identification library in python

Application of the L2HMC algorithm to simulations in lattice QCD.

OpenMMLab Video Perception Toolbox. It supports Video Object Detection (VID), Multiple Object Tracking (MOT), Single Object Tracking (SOT), Video Instance Segmentation (VIS) with a unified framework.

A large-image collection explorer and fast classification tool

chainladder - Property and Casualty Loss Reserving in Python

Experiments and code to generate the GINC small-scale in-context learning dataset from "An Explanation for In-context Learning as Implicit Bayesian Inference"

ServiceX Transformer that converts flat ROOT ntuples into columnwise data

This is a Tensorflow implementation of Learning to See in the Dark in CVPR 2018

Code for Efficient Visual Pretraining with Contrastive Detection

Code for the paper Task Agnostic Morphology Evolution.

PyTorch Implementation of AnimeGANv2

SCAN: Learning to Classify Images without Labels, incl. SimCLR. [ECCV 2020]

PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

Moiré Attack (MA): A New Potential Risk of Screen Photos [NeurIPS 2021]

Code for ACL'2021 paper WARP 🌀 Word-level Adversarial ReProgramming

PyTorch Implementation of "Non-Autoregressive Neural Machine Translation"

This repository contains datasets and baselines for benchmarking Chinese text recognition.

HNN: Human (Hollywood) Neural Network

NLP made easy

Object classification with basic computer vision techniques

Why do we need `recsys_metrics`?