Common Anano Variables in Python
I am now looking into the Theano library and I am just getting confused with the variables that Anano used. While reading the tutorial, I think I didn't understand its detailed meaning. Below is the definition of shared variables from Anano from the tutorial:
"A variable with a store that is shared between the functions it appears. These variables are intended to be created by registered shared constructors.
Also, I'm wondering if Anano related variables can be a data member of a python class. For example:
class A(object):
data = None
...
Can "data" be or be initialized as an Anano Shared variable? I really appreciate if anyone can help me.
source to share
Anano shared variables behave like regular Python variables. They have a clear meaning that is permanent. In contrast, symbolic variables are not assigned an explicit value until after the execution of the compiled Anano function is assigned.
Symbolic variables can be thought of as representing state during one execution. Shared variables, on the other hand, represent state that remains in memory for the lifetime of a Python reference (often similar to the lifetime of a program).
Shared variables are commonly used to store / represent neural network weights because we want these values to stay around many executions of Theano's training or testing function. Often, the goal of the Theano training program is to update the weights stored in a shared variable. And the test functions need the current scale to perform network forwarding.
As far as Python is concerned, Anano variables (generic or symbolic) are just objects - instances of classes defined in the Theano library. So, yes, references to shared variables can be stored in your own classes just like any other Python object.
source to share
A shared variable helps to simplify operations on a predefined variable. Example of @ danien-renshaw's answer, suppose we want to add two matrices, say a and b, where the value of the b-matrix will remain constant throughout the life of the program, we can have the b-matrix as a shared variable and perform the required operation.
Code without using a shared variable:
a = theano.tensor.matrix('a')
b = theano.tensor.matrix('b')
c = a + b
f = theano.function(inputs = [a, b], outputs = [c])
output = f([[1, 2, 3, 4]], [[5, 5, 6, 7]])
Code using shared variable:
a = theano.tensor.matrix('a')
b = theano.tensor.shared( numpy.array([[5, 6, 7, 8]]))
c = a + b
f = theano.function(inputs = [a], outputs = [c])
output = f([[1, 2, 3, 4]])
source to share