I'm running parameter optimization on a LightFM implicit factorization model using "warp" loss. When I run np.mean(precision_at_k(...)) on the test data, it is virtually always the same result, 0.316850870848, out to 12 digits. Is this expected for some reason? If not, any ideas how to figure out what might be wrong?

Here is sample output showing the param combinations and the output:

2017-01-20 21:10:02,363 INFO __main__: Starting training iteration 1, params:
{
    "alpha": 0.0041233076328981919,
    "epochs": 45,
    "learning_rate": 0.50174314850490254,
    "no_components": 184
}
2017-01-20 21:10:02,363 INFO lightfm_uma: Training model...
2017-01-20 21:25:20,518 INFO lightfm_uma: Finished [email protected] = 0.316850870848
2017-01-20 21:25:21,453 INFO __main__: Starting training iteration 2, params:
{
    "alpha": 0.0064873564172718886,
    "epochs": 63,
    "learning_rate": 0.50406151543722921,
    "no_components": 180
}
2017-01-20 21:25:21,453 INFO lightfm_uma: Training model...
2017-01-20 21:44:36,565 INFO lightfm_uma: Finished [email protected] = 0.316850870848
2017-01-20 21:44:37,495 INFO __main__: Starting training iteration 3, params:
{
    "alpha": 0.020473212242717205,
    "epochs": 62,
    "learning_rate": 0.74135691825946459,
    "no_components": 156
}
2017-01-20 21:44:37,496 INFO lightfm_uma: Training model...
2017-01-20 22:00:45,661 INFO lightfm_uma: Finished [email protected] = 0.316850870848

Hi,

I was following the example of Building datasets and tried to build the movielens dataset myself. However, when I tries to build the mapping for item and item features, I realized that the feature I choose, which is the genres in movies.csv, has multiple values in one cell (eg. Action|Fantasy|Comedy) instead of just one value. How should I use dataset.fit_partial to create the mapping in these case? And the same goes for building interactions. Thank you so much!

This would be useful for strong generalization evaluation.

I think this should be doable but probably will need to refactor a lot of Cython code to separate user and item parameters update.

(Trying to rush for a paper submission, otherwise would love to contribute)

d:\Recommender systems\code>conda install lightfm Fetching package metadata ......... Solving package specifications: . PackageNotFoundError: Package not found: '' Package missing in current win-64 ch annels:

• lightfm

You can search for packages on anaconda.org with

anaconda search -t conda lightfm

Apologies if this is user error, but I appear to be getting Nan embeddings from LightFM and I'm unsure what I could have done wrong. I followed the documentation, and have raise the issue on SO.

Basically I have a large data-set where collaborative filtering is working fine, but where user / item embeddings are provided, the model produces nan embeddings.

http://stackoverflow.com/questions/40967226/lightfm-user-item-producing-nan-embeddings

Hi, I have two questions,

1. I try to build a recommender 'online'. If my input data(interaction, user features) has some new actions, and I want to update the model has trained before. Should I retrain a new model? or I can just use method fit_partial like this model.fit_partial(updated_interaction, updated_user_features, item_features) to update my model?

2. I observed the model training with num_thread=4 on linux server but it seems only use one CPU and the other three use 3% - 5.5% only. I saw that LightFM uses OpenMP but I'm not run it on OSX or Windows. Also I reinstall LightFM and not see any warning like LightFM was compiled without OpenMP support., so OpenMP should be enable? How to run my training to use four CPUs with 100%? Is there something wrong? How can I check it?

Sorry for long description about my questions, thank you!

I'm using the WARP to create my model. To fit it, I've no problem. But for the prediction, I've a segmentation fault (core dump).

How can I solve it ?

Thx

I'm using Ubuntu 16.04 with python 2.7.

Is this something you would want in the project @maciejkula ? This is just a rough outline of some code I have laying around to do something like this, so it might need to be fixed up a bit and whatnot, but I'd rather do that after I know your opinion :)

And if it already exists, it's a bit embarrassing but I didn't find it! :)

Hello !

Thank you for this open source package, it help a lot and your work is amazing.

I just a have a silly question about dataset construction. I followed the example for my data: user (160.000 x 300) and item (4000 x 4).

dataset = Dataset()
dataset.fit(users=(x['id_user'] for x in user),
            items=(x['id_item'] for x in item),
            user_features=((x['id_user'], [[x[col] for col in list_columns_user]]) for x in user),
            item_features=((x['id_item'], [[x[col] for col in list_columns_item]]) for x in item))

But when I try dataset.user_features_shape() I get (160000, 160000). shouldn't I rather have this (160000, 300) ?

Indeed, we can read in the documentation :

Returns ------- (num user ids, num user features): tuple of ints

and my num user features is 300. So there is an error in what I did?

Sorry for the stupid question!

Hi,

About building users/items features vectors, I took a look at your examples in the repo and also at the post http://blog.ethanrosenthal.com/2016/11/07/implicit-mf-part-2/, but I have a question about it.

I saw that you commonly used "tags" metadata about an item (or user), where you have several tags values for a same item.

1. But let's take the case where we have, in the original metadata, only one tag value per item, and in total 100 different tags values. It will make a 100-dimensions vector, one-hot-encoded, with only one '1'. Is it still a relevant feature item vector if it is that sparse ?

2. If I have several kind of tags (for example : item category, color, size type..), is it correct to concatenate my binary feature vectors together ? Like this :

| | tag type A | tag type B | ... | tag type Z | |:------:|:-----------------:|:-----------:|:----:|:-----------:| | item 1 | 0 1 0 0 0 0 0 0 0 | 0 0 0 0 0 1 | ... | 0 0 1 0 0 0 | | item 2 | 0 0 0 0 0 0 1 0 0 | 0 1 0 0 0 0 | ... | 0 0 0 0 1 0 | | ... | ... | ... | ... | ... |

Each 0 / 1 representing a different value encoded for each tag type

Thanks in advance

The random_train_test_split makes is easy to split the interactions matrix into train and test dataset but if you have data with weights you will have to apply random_train_test_split twice with the same random_state parameter. My concern is that it would be intuitive to do something like:

from lightfm.data import Dataset
from lightfm.cross_validation import random_train_test_split

users = np.random.choice([0., 1., 2.], (10, 1))
items = np.random.choice([0., 1., 2.], (10, 1))
weight = np.random.rand(10,1)
data = np.concatenate((users, items, weight), axis=1)

dataset = Dataset()
dataset.fit(users=np.unique(data[:, 0]), items=np.unique(data[:, 1]))
interactions, weight = dataset.build_interactions((i[0], i[1], i[2]) for i in data)

test_percentage = 0.2
random_state = np.random.RandomState(seed=1)

train, test = random_train_test_split(
    interactions=interactions,
    test_percentage=test_percentage,
    random_state=random_state
)
train_weight, test_weight = random_train_test_split(
    interactions=weight,
    test_percentage=test_percentage,
    random_state=random_state
)

np.array_equal(train.row, train_weight.row)
np.array_equal(train.col, train_weight.col)
np.array_equal(test.row, test_weight.row)
np.array_equal(test.col, test_weight.col)

>>> False
>>> False
>>> False
>>> False

This will result in an incorrect split because the state of the random_state changes after the first call to random_state.shuffle. For the above example to work as intended you need to make separate but identical RandomStates:

random_state_interaction = np.random.RandomState(seed=1)
random_state_weight = np.random.RandomState(seed=1)

train, test = random_train_test_split(
    interactions=interactions,
    test_percentage=test_percentage,
    random_state=random_state_interaction
)
train_weight, test_weight = random_train_test_split(
    interactions=weight,
    test_percentage=test_percentage,
    random_state=random_state_weight
)

np.array_equal(train.row, train_weight.row)
np.array_equal(train.col, train_weight.col)
np.array_equal(test.row, test_weight.row)
np.array_equal(test.col, test_weight.col)

>>> True
>>> True
>>> True
>>> True

It works but I think it's a little awkward. Two possible solutions/suggestions:

1. Only require a seed parameter and create a RandomState in cross_validate._shuffle method. This has the added benefit of fitting in with the larger libraries that only require seed and not a RandomState generator. I also don't see any additional flexibility by passing in a generator instead of a simple integer.
2. Make a copy of random_state before applying shuffle in cross_validate._shuffle.

Thoughts?

Hello, what is the meaning of the recommendation score for user-item pairs? The ones that are given by the predict() function. I mean, is there some kind of range? How it is calculated?

Thank you!

Hello! Ran into a problem installing lightfm 1.16 on Linux-5.15.0-56-generic-x86_64-with-glibc2.35/Python 3.10.6. The error log is in the attachment, I tried to solve it using the available methods provided on the Internet, but it did not help. Could you suggest how to overcome this problem? Thanks in advance.

Collecting lightfm Using cached lightfm-1.16.tar.gz (310 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from lightfm) (1.23.5) Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.10/dist-packages (from lightfm) (1.9.3) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from lightfm) (2.28.1) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from lightfm) (1.2.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->lightfm) (2022.12.7) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->lightfm) (1.26.13) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->lightfm) (3.4) Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->lightfm) (2.1.1) Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->lightfm) (1.2.0) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->lightfm) (3.1.0) Building wheels for collected packages: lightfm Building wheel for lightfm (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [176 lines of output] /usr/local/lib/python3.10/dist-packages/setuptools/dist.py:770: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead warnings.warn( running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/cross_validation.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/init.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/lightfm.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/data.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/_lightfm_fast.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/evaluation.py -> build/lib.linux-x86_64-cpython-310/lightfm creating build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/init.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/movielens.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/_common.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/stackexchange.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/_lightfm_fast_no_openmp.c -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/_lightfm_fast_openmp.c -> build/lib.linux-x86_64-cpython-310/lightfm running build_ext building 'lightfm._lightfm_fast_openmp' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/lightfm x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.10 -c lightfm/_lightfm_fast_openmp.c -o build/temp.linux-x86_64-cpython-310/lightfm/_lightfm_fast_openmp.o -ffast-math -march=native -fopenmp lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc_array’: lightfm/_lightfm_fast_openmp.c:24868:5: error: lvalue required as increment operand 24868 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:24870:5: error: lvalue required as decrement operand 24870 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc_memoryview’: lightfm/_lightfm_fast_openmp.c:25179:5: error: lvalue required as increment operand 25179 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:25181:5: error: lvalue required as decrement operand 25181 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc__memoryviewslice’: lightfm/_lightfm_fast_openmp.c:25429:5: error: lvalue required as increment operand 25429 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:25431:5: error: lvalue required as decrement operand 25431 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_InitGlobals’: lightfm/_lightfm_fast_openmp.c:26184:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations] 26184 | PyEval_InitThreads(); | ^~~~~~~~~~~~~~~~~~ In file included from /usr/include/python3.10/Python.h:130, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/ceval.h:122:37: note: declared here 122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void); | ^~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_ParseOptionalKeywords’: lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_decode_c_string’: lightfm/_lightfm_fast_openmp.c:28218:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations] 28218 | return PyUnicode_FromUnicode(NULL, 0); | ^~~~~~ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here 551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject) PyUnicode_FromUnicode( | ^~~~~~~~~~~~~~~~~~~~~ error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for lightfm Running setup.py clean for lightfm Failed to build lightfm Installing collected packages: lightfm Running setup.py install for lightfm ... error error: subprocess-exited-with-error

× Running setup.py install for lightfm did not run successfully. │ exit code: 1 ╰─> [178 lines of output] /usr/local/lib/python3.10/dist-packages/setuptools/dist.py:770: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead warnings.warn( running install /usr/local/lib/python3.10/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/cross_validation.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/init.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/lightfm.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/data.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/_lightfm_fast.py -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/evaluation.py -> build/lib.linux-x86_64-cpython-310/lightfm creating build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/init.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/movielens.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/_common.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/datasets/stackexchange.py -> build/lib.linux-x86_64-cpython-310/lightfm/datasets copying lightfm/_lightfm_fast_no_openmp.c -> build/lib.linux-x86_64-cpython-310/lightfm copying lightfm/_lightfm_fast_openmp.c -> build/lib.linux-x86_64-cpython-310/lightfm running build_ext building 'lightfm._lightfm_fast_openmp' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/lightfm x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.10 -c lightfm/_lightfm_fast_openmp.c -o build/temp.linux-x86_64-cpython-310/lightfm/_lightfm_fast_openmp.o -ffast-math -march=native -fopenmp lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc_array’: lightfm/_lightfm_fast_openmp.c:24868:5: error: lvalue required as increment operand 24868 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:24870:5: error: lvalue required as decrement operand 24870 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc_memoryview’: lightfm/_lightfm_fast_openmp.c:25179:5: error: lvalue required as increment operand 25179 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:25181:5: error: lvalue required as decrement operand 25181 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__pyx_tp_dealloc__memoryviewslice’: lightfm/_lightfm_fast_openmp.c:25429:5: error: lvalue required as increment operand 25429 | ++Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c:25431:5: error: lvalue required as decrement operand 25431 | --Py_REFCNT(o); | ^~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_InitGlobals’: lightfm/_lightfm_fast_openmp.c:26184:1: warning: ‘PyEval_InitThreads’ is deprecated [-Wdeprecated-declarations] 26184 | PyEval_InitThreads(); | ^~~~~~~~~~~~~~~~~~ In file included from /usr/include/python3.10/Python.h:130, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/ceval.h:122:37: note: declared here 122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void); | ^~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_ParseOptionalKeywords’: lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:26990:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 26990 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here 580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( | ^~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c:27006:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] 27006 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : | ^ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here 446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject op) { | ^~~~~~~~~~~~~~~~~~~~~~~~~~ lightfm/_lightfm_fast_openmp.c: In function ‘__Pyx_decode_c_string’: lightfm/_lightfm_fast_openmp.c:28218:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations] 28218 | return PyUnicode_FromUnicode(NULL, 0); | ^~~~~~ In file included from /usr/include/python3.10/unicodeobject.h:1046, from /usr/include/python3.10/Python.h:83, from lightfm/_lightfm_fast_openmp.c:20: /usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here 551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject) PyUnicode_FromUnicode( | ^~~~~~~~~~~~~~~~~~~~~ error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> lightfm

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.

Using the same example from https://making.lyst.com/lightfm/docs/examples/dataset.html#building-the-id-mappings

How could I validade the mapping of each item feature?

When I call user_id_map, u_f_map, item_id_map, i_f_map = dataset.mapping()

after this item_features = dataset.build_item_features(((x['ISBN'], [x['Book-Author']]) for x in get_book_features()))

I got this result from item_id_map

{'034545104X': 0,
 '0155061224': 1,
 '0446520802': 2,
 '052165615X': 3...} 

Now, I know that '034545104X' is mapped as 0. Looking at item_features built before I got this

  (0, 0)	0.5
  (0, 343789)	0.5
  (1, 1)	0.5
  (1, 428522)	0.5
  (2, 2)	0.5
  (2, 341954)	0.5
  (3, 3)	0.5 .......

Looking at book_features as a pandas dataframe

As seen, getting the item mapped as 0 which is the item 034545104X. And filtering the dataframe, for this item, the book-author is Flesh Tones: A Novel

But When I do this

item_feature_inverse_map = {v:k for k, v in i_f_map.items()}
print(item_feature_inverse_map[343789])

The result is 'M. J. Rose' which is different of Flesh Tones: A Novel.

I created a recommendation system using lightFM on Google Colab. I want to deploy this model using Node JS. I read this article and tried the method recommended here. It recommended using tensorflow.js. For that the model needed to be saved and this is where the problem arose since I wrote my model using lightFM.

This gave rise to following error: ` AttributeError Traceback (most recent call last) in ----> 1 model.save("model.h5")

AttributeError: 'LightFM' object has no attribute 'save' `

What is the alternative to model.save in lightfm? How can we deploy model written in lightfm using node js?

Hi, I am looking at using LightFM for recommending products to customers in retail stores based on the products the customer has bought in past. In the customer buying history, I have data regarding the products along with the purchase quantities for each product the customer has bought. The customer might have bought products with quantities ranging from 1 to 100 or more. I am looking for suggestions on how I can convert these product-wise purchased quantities to the rating score of 1 to 5. Looking for your suggestion. Thank you.

Hello,

Thanks for this nice work, very efficient and designed for real wrold use cases.

Concerning the cold-start issue, you indicate in the documentation to call fit_partial method of the lightfm.data.Dataset class, and to "resize your LightFM model to be able to use the new features". What does "resize your LightFM model to be able to use the new features" really means ?

First I train the model

from lightfm import LightFM
from lightfm.data import Dataset
from lightfm.evaluation import auc_score, precision_at_k, recall_at_k, reciprocal_rank

dataset = Dataset(user_identity_features=False, item_identity_features=True)
dataset.fit(users=train_users_df.index.unique(), 
            items=train_items_df.index, 
            item_features=train_tag_labels)

train_item_features = dataset.build_item_features(train_item_features_)
train_interactions, train_weights = dataset.build_interactions(train_users_df["MatchId"].items())

recommender = LightFM(loss='warp')

recommender = recommender.fit(interactions=train_interactions,
                              item_features=train_item_features,
                              sample_weight=train_weights,
                              epochs=NUM_EPOCHS,
                              num_threads=NUM_THREADS)

After, I would like to predict using this model but for new items, with new features, unseen during this 1st fit. I fit again partially my Dataset without issue ...

dataset.fit_partial(users=test_users_df.index.unique(), 
                    items=test_items_df.index, 
                    item_features=test_tag_labels)

test_item_features_ = build_item_features(test_items_df)
test_item_features = dataset.build_item_features(test_item_features_)

... but I get an error when predicting with the model

recommender.predict_rank(test_interactions=next_items,
                         train_interactions=past_items,
                         item_features=test_item_features)

and I get the following error: "ValueError: The item feature matrix specifies more features than there are estimated feature embeddings: 3623 vs 4985" where 3623 is the number of item features saw during the fit and 4985 is the number of features after adding new items with new features.

Then, is there a way to "resize the model" as suggested in the documentation ?

Thanks