I'm up-to-date on pymc3 and gelato.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/Users/twiecki/anaconda/lib/python3.6/site-packages/theano/gof/op.py in __call__(self, *inputs, **kwargs)
    624                 try:
--> 625                     storage_map[ins] = [self._get_test_value(ins)]
    626                     compute_map[ins] = [True]

/Users/twiecki/anaconda/lib/python3.6/site-packages/theano/gof/op.py in _get_test_value(cls, v)
    580         detailed_err_msg = utils.get_variable_trace_string(v)
--> 581         raise AttributeError('%s has no test value %s' % (v, detailed_err_msg))
    582 

AttributeError: Softmax.0 has no test value  
Backtrace when that variable is created:

  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2717, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2821, in run_ast_nodes
    if self.run_code(code, result):
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-18-7dd01309b711>", line 37, in <module>
    prediction = gelato.layers.get_output(network)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/layers/helper.py", line 190, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/layers/dense.py", line 124, in get_output_for
    return self.nonlinearity(activation)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/nonlinearities.py", line 44, in softmax
    return theano.tensor.nnet.softmax(x)


During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-18-7dd01309b711> in <module>()
     44                    prediction,
     45                    observed=target_var,
---> 46                    total_size=total_size)

/Users/twiecki/working/projects/pymc/pymc3/distributions/distribution.py in __new__(cls, name, *args, **kwargs)
     35                 raise TypeError("observed needs to be data but got: {}".format(type(data)))
     36             total_size = kwargs.pop('total_size', None)
---> 37             dist = cls.dist(*args, **kwargs)
     38             return model.Var(name, dist, data, total_size)
     39         else:

/Users/twiecki/working/projects/pymc/pymc3/distributions/distribution.py in dist(cls, *args, **kwargs)
     46     def dist(cls, *args, **kwargs):
     47         dist = object.__new__(cls)
---> 48         dist.__init__(*args, **kwargs)
     49         return dist
     50 

/Users/twiecki/working/projects/pymc/pymc3/distributions/discrete.py in __init__(self, p, *args, **kwargs)
    429         super(Categorical, self).__init__(*args, **kwargs)
    430         try:
--> 431             self.k = tt.shape(p)[-1].tag.test_value
    432         except AttributeError:
    433             self.k = tt.shape(p)[-1]

/Users/twiecki/anaconda/lib/python3.6/site-packages/theano/gof/op.py in __call__(self, *inputs, **kwargs)
    637                         raise ValueError(
    638                             'Cannot compute test value: input %i (%s) of Op %s missing default value. %s' %
--> 639                             (i, ins, node, detailed_err_msg))
    640                     elif config.compute_test_value == 'ignore':
    641                         # silently skip test

ValueError: Cannot compute test value: input 0 (Softmax.0) of Op Shape(Softmax.0) missing default value.  
Backtrace when that variable is created:

  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2717, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2821, in run_ast_nodes
    if self.run_code(code, result):
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-18-7dd01309b711>", line 37, in <module>
    prediction = gelato.layers.get_output(network)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/layers/helper.py", line 190, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/layers/dense.py", line 124, in get_output_for
    return self.nonlinearity(activation)
  File "/Users/twiecki/anaconda/lib/python3.6/site-packages/lasagne/nonlinearities.py", line 44, in softmax
    return theano.tensor.nnet.softmax(x)

I'm currently integrating recent changes in PyMC3 to gelato. There are a lot of changes. Everyone is welcome for discussion.

Here are the most remarkable features:

• no more with context when using gelato layers

from gelato.layers import *
import pymc3 as pm
# get data somehow
inp = InputLayer(shape)
out = DenseLayer(inp, 1, W=NormalSpec(sd=LognormalSpec(sd=.1)))
out = DenseLayer(out, 1, W=NormalSpec(sd=LognormalSpec(sd=.1)))
with out.root:
    pm.Normal('y', mu=get_output(out, {inp:x}),
              observed=y)
    approx = pm.fit(10000)

• Flexible Specs you can do almost everything. What to do if we want different shapes there is an open question

from gelato import *
import theano.tensor as tt
import pymc3 as pm
func = as_spec_op(tt.nlinalg.matrix_power)
expr0= func(NormalSpec() * LaplaceSpec(), 2)
expr1 = expr0 / 100 - NormalSpec()
with Model() as model:
    var = expr((10, 10))
    assert var.tag.test_value.shape == (10, 10)
    assert len(model.free_RVs) == 3
    fit(100)
U = NormalSpec()
V = UniformSpec()
V = V / V.norm(2)
W = U*V
with pm.Model() as model:
    result = W((3, 2), name='weight_normalization')

refere to #7. I've updated example using new pm.Minibatch API. All was running good with the following theanorc:

[global]
device=cpu
floatX=float32
mode=FAST_RUN
optimizer_including=cudnn

[lib]
cnmem=0.95

[nvcc]
fastmath=True
flags = -I/usr/local/cuda-8.0-cudnnv5.1/include -L/usr/local/cuda-8.0-cudnnv5.1/lib64

[blas]
ldflag = -L/usr/lib/openblas-base -Lusr/local/cuda-8.0-cudnnv5.1/lib64 -lopenblas

[DebugMode]
check_finite=1

[cuda]
root=/usr/local/cuda-8.0-cudnnv5.1/

pip freeze output

alabaster==0.7.10
algopy==0.5.3
Babel==2.4.0
bleach==2.0.0
CommonMark==0.5.4
cycler==0.10.0
Cython==0.25.2
decorator==4.0.11
docutils==0.13.1
entrypoints==0.2.2
-e git+https://github.com/ferrine/[email protected]#egg=gelato
h5py==2.7.0
html5lib==0.999999999
imagesize==0.7.1
ipykernel==4.6.1
ipython==6.0.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
Jinja2==2.9.6
joblib==0.11
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.0.1
jupyter-console==5.1.0
jupyter-core==4.3.0
Keras==2.0.4
Lasagne==0.2.dev1
Mako==1.0.6
MarkupSafe==1.0
matplotlib==2.0.0
mistune==0.7.4
more-itertools==3.1.0
nbconvert==5.1.1
nbformat==4.3.0
nbsphinx==0.2.13
nose==1.3.7
notebook==5.0.0
numdifftools==0.9.20
numpy==1.13.0
pandas==0.20.1
pandocfilters==1.4.1
patsy==0.4.1
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.14
ptyprocess==0.5.1
Pygments==2.2.0
pygpu==0.6.5
-e git+https://github.com/ferrine/[email protected]#egg=pymc3
pymongo==3.4.0
pyparsing==2.2.0
python-dateutil==2.6.0
pytz==2017.2
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.0
recommonmark==0.4.0
requests==2.13.0
scikit-learn==0.18.1
scipy==0.19.1
seaborn==0.7.1
simplegeneric==0.8.1
six==1.10.0
sklearn==0.0
snowballstemmer==1.2.1
Sphinx==1.5.5
terminado==0.6
testpath==0.3
Theano==0.10.0.dev1
tornado==4.5.1
tqdm==4.11.2
traitlets==4.3.2
wcwidth==0.1.7
webencodings==0.5.1
widgetsnbextension==2.0.0
xmltodict==0.11.0

When I attempt to import gelato, it fails with the following error message:

---> 19 class LayerModelMeta(pm.model.InitContextMeta):
     20     """Magic comes here
     21     """

AttributeError: module 'pymc3.model' has no attribute 'InitContextMeta'

I believe that InitContextMeta no longer exists in pymc3; it's been merged with ContextMeta.

I don't know if there are plans to update this repository anytime soon, although it does seem like a useful tool, so it would be great if it worked with the latest pymc3.