The variable names used in the getters and setters of each model class are different than what is initialized in the constructor.

Note that the name 'W' is used in init, but '_W' is used as the variable name in the getters and setters, both of which have the same name as the variable initialized in init 'W'. This issue is the same for all member variables. I have tested the DBN and GaussianVarianceRBM subclasses as well, and it looks like the issue persists across all other subclasses.

• OS: CentOS 7.8
• Virtual Environment conda 4.10.1
• Python Version 3.8

This causes issues when trying to make copies of classes for things like multi-GPU training via torch.nn.DistributedDataParallel

Hi Gustavo!

I hope you are doing well!

I was wondering if you had looked into / planned to add probabilistic max-pooling, as descibed in: H. Lee, et al., Convolutional Deep Belief Networksfor Scalable Unsupervised Learning of Hierarchical Representations Proceedings of the 26th annual international conference on machine learning (2009).

I am happy to help, but figured it would be best to consult you first.

Thanks! Nick

When I use GaussianRBM to run rbm_classification.py on GPU, these errors appear.

11%|█ | 51/469 [00:01<00:07, 58.24it/s]C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [416,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [417,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [418,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [419,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [420,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [421,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [422,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [423,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [424,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [425,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [426,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [427,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [428,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [429,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [430,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [431,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [432,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [433,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [434,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [435,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [436,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [437,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [438,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [439,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [440,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [441,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [442,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [443,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [444,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [445,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [446,0,0] Assertion 0 <= p4 && p4 <= 1 failed. C:/cb/pytorch_1000000000000/work/aten/src\ATen/native/cuda/DistributionTemplates.h:591: block: [6,0,0], thread: [447,0,0] Assertion 0 <= p4 && p4 <= 1 failed.

RuntimeError: CUDA error: device-side assert triggered

But if I run it on cpu, the error while like this: RuntimeError: Expected p_in >= 0 && p_in <= 1 to be true, but got false.

It seems like the function hidden_sampling dosen't run correctly.

Hello,

Thank you very much for developing this superb library!!

I believe that I found an issue, but I am not able to find the root cause.

From time to time I receive this error:

Traceback (most recent call last): File "/Users/supermario/LNIssue/run_rbm.py", line 26, in epochs=1000000) File "/Users/supermario/LNIssue/venv/lib/python3.7/site-packages/learnergy/models/bernoulli/rbm.py", line 495, in fit _, _, _, _, visible_states = self.gibbs_sampling(samples) File "/Users/supermario/LNIssue/venv/lib/python3.7/site-packages/learnergy/models/bernoulli/rbm.py", line 379, in gibbs_sampling neg_hidden_states, True) File "/Users/supermario/LNIssue/venv/lib/python3.7/site-packages/learnergy/models/bernoulli/rbm.py", line 352, in visible_sampling states = torch.bernoulli(probs) RuntimeError: Expected p_in >= 0 && p_in <= 1 to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

Here is the code to reproduce the error:

import numpy as np import torch from torch.utils.data import Dataset from learnergy.models.bernoulli.rbm import RBM

np.random.seed(16) torch.manual_seed(16)

class CustomDataset(Dataset):

def __init__(self, custom_data):
    self.custom_data = torch.from_numpy(custom_data)
    self.dummy_y = torch.from_numpy(np.array([0.0]))

def __getitem__(self, index):
    return self.custom_data[index], self.dummy_y[0]

def __len__(self):
    return self.custom_data.shape[0]

rbm = RBM(n_visible=32, n_hidden=64, steps=1, learning_rate=0.0001, momentum=0.9, decay=0.00001, temperature=0, use_gpu=False)

data = np.random.choice(a=[False, True], size=(5755, 32)).astype(np.float32) data_train = CustomDataset(data)

rbm.fit(dataset=data_train, batch_size=20, epochs=1000000)

And requirements: astroid==2.8.4 attrs==21.2.0 coverage==6.1.1 cycler==0.11.0 imageio==2.10.3 importlib-metadata==4.8.2 iniconfig==1.1.1 isort==5.10.1 kiwisolver==1.3.2 lazy-object-proxy==1.6.0 learnergy==1.1.1 matplotlib==3.4.3 mccabe==0.6.1 networkx==2.6.3 numpy==1.21.4 packaging==21.2 Pillow==8.4.0 platformdirs==2.4.0 pluggy==1.0.0 py==1.11.0 pylint==2.11.1 pyparsing==2.4.7 pytest==6.2.5 python-dateutil==2.8.2 PyWavelets==1.2.0 scikit-image==0.18.3 scipy==1.7.2 six==1.16.0 tifffile==2021.11.2 toml==0.10.2 torch==1.9.0 tqdm==4.62.3 typed-ast==1.4.3 typing-extensions==3.10.0.2 wrapt==1.13.3 zipp==3.6.0

The error should appear at epoch 221/1000000.

If I add the following torch.nan_to_num in visible_sampling() method, then the error is not present anymore.

probs = torch.nan_to_num(probs, nan=0.0) Sampling current states states = torch.bernoulli(probs)

macosx, python 3.7.0

Thank you! Please let me know if you need additional info.

Regarding the basic Bernoulli RBM for classification, could I check what is the basis of your implementation? From what I can tell, the training is broken up in two sections:

1. RBM will perform unsupervised learning to minimize energy (RBM::fit), basically like a feature extractor.
2. The whole model is then considered as a 2-layer neural network with a sigmoid layer (RBM::forward) and linear transformation layer.

So, it seems like the RBM is just used as an initializer for the neural network.

In addition, I noticed that RBM::fit's epoch defaults to 1. Increasing the epoch number does not seem to improve performance. Does this mean that there is no use in training the RBM?

Lastly, was there any prior work that implemented RBM in this way, or is this your original idea? Are there any published results or study into this architecture? Thank you!

I found in the comments of file learnergy/learnergy/models/bernoulli/rbm.py (line 401), you said

Creating a Softplus function for numerical stability

The form of Softplus function is f(x) = log(1+exp(x)), which fits the form of the log term of marginal energy of RBM

E(v)= \sum_i a_i v_i + \sum_j ( log( 1 + exp( \sum_i v_i \lambda_{ij} +b_j ) ) ).

So I suppose the Softplus function is not used to ensure the numerical stability, but to calculate the marginal energy. I am a beginner to RBM, if I have any wrong perceptions, please point it out.

Thank you!

Bumps sphinx from 5.2.3 to 5.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps sphinx from 5.2.3 to 5.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hi Gustavo, I hope you are having a great week. Over the last few months, I have made a couple of additions to my fork, and I figured I would share them to see if you found them useful for the base repo. No worries if not!

The first one was adding Gaussian RBM with hidden layer SELU activation. I have had some success using this model for my research, and figured it may be useful for others.

In addition to the SELU addition, added boolean flags to indicate whether or not to batch normalize and normalize the input within parent GaussianRBM class.

The batch normalization is removed for SELU RBM, as it self-normalizes. Also, I use a scaling functionality outside of learnergy for the input, so it would be useful to have the normalization internal to learnergy as an option.

By default input and batch normalization are turned on, as to not cause issues for other users.

Thanks! Nick

Bumps sphinx from 5.2.3 to 6.0.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps sphinx from 5.2.3 to 6.0.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.