Visual Question Answering 2

on VQA, DNN, RNN, CNN, Python, Keras, Theano

Prepare for VQA in Ubuntu 14.04 x64.
In this Post, I want to install and test Keras.

Keras is a minimalist, highly modular neural network library in the spirit of Torch, written in Python, that uses Theano under the hood for optimized tensor manipulation on GPU and CPU. It was developed with a focus on enabling fast experimentation.

— Keras Documentation


Keras uses the following dependencies:

  1. numpy

  2. scipy

  3. pyyaml

  4. Theano

  5. HDF5 and h5py (optional, required if you use model saving/loading functions)

  6. Optional but recommended if you use CNNs: cuDNN

Install miniconda

$ wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-`uname -p`.sh
$ bash Miniconda-latest-Linux-`uname -p`.sh
$ source ~/.bashrc

See also the install guide.

Install Theano

$ sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose libopenblas-dev git
$ pip install git+git://github.com/Theano/Theano.git

Install h5py

$ conda install -y h5py

Install Keras

$ pip install git+git://github.com/fchollet/keras.git

Install g++ (optional)

$ sudo apt-get install g++

Without g++, you will get this message when run Keras and Keras will be very slow.

WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.

Test Keras

$ wget https://raw.githubusercontent.com/fchollet/keras/master/examples/mnist_mlp.py
$ python mnist_mlp.py

The result here.

60000 train samples
10000 test samples
Train on 60000 samples, validate on 10000 samples
Epoch 1/20
8s - loss: 0.4356 - acc: 0.8716 - val_loss: 0.1863 - val_acc: 0.9421
Epoch 2/20
7s - loss: 0.1961 - acc: 0.9414 - val_loss: 0.1274 - val_acc: 0.9601
Epoch 3/20
7s - loss: 0.1451 - acc: 0.9564 - val_loss: 0.1010 - val_acc: 0.9691
Epoch 4/20
8s - loss: 0.1189 - acc: 0.9642 - val_loss: 0.0847 - val_acc: 0.9752
Epoch 5/20
8s - loss: 0.1019 - acc: 0.9691 - val_loss: 0.0850 - val_acc: 0.9735
Epoch 6/20
8s - loss: 0.0903 - acc: 0.9721 - val_loss: 0.0749 - val_acc: 0.9777
Epoch 7/20
8s - loss: 0.0822 - acc: 0.9745 - val_loss: 0.0753 - val_acc: 0.9762
Epoch 8/20
7s - loss: 0.0758 - acc: 0.9762 - val_loss: 0.0743 - val_acc: 0.9796
Epoch 9/20
7s - loss: 0.0705 - acc: 0.9780 - val_loss: 0.0720 - val_acc: 0.9784
Epoch 10/20
8s - loss: 0.0648 - acc: 0.9790 - val_loss: 0.0688 - val_acc: 0.9793
Epoch 11/20
8s - loss: 0.0592 - acc: 0.9819 - val_loss: 0.0663 - val_acc: 0.9797
Epoch 12/20
8s - loss: 0.0567 - acc: 0.9824 - val_loss: 0.0677 - val_acc: 0.9815
Epoch 13/20
8s - loss: 0.0536 - acc: 0.9833 - val_loss: 0.0711 - val_acc: 0.9796
Epoch 14/20
8s - loss: 0.0520 - acc: 0.9834 - val_loss: 0.0684 - val_acc: 0.9806
Epoch 15/20
9s - loss: 0.0500 - acc: 0.9837 - val_loss: 0.0664 - val_acc: 0.9807
Epoch 16/20
7s - loss: 0.0471 - acc: 0.9850 - val_loss: 0.0683 - val_acc: 0.9809
Epoch 17/20
7s - loss: 0.0449 - acc: 0.9856 - val_loss: 0.0682 - val_acc: 0.9812
Epoch 18/20
8s - loss: 0.0433 - acc: 0.9860 - val_loss: 0.0675 - val_acc: 0.9813
Epoch 19/20
7s - loss: 0.0401 - acc: 0.9869 - val_loss: 0.0683 - val_acc: 0.9819
Epoch 20/20
8s - loss: 0.0383 - acc: 0.9874 - val_loss: 0.0705 - val_acc: 0.9820
Test score: 0.0704572771238
Test accuracy: 0.982



Error

When I installed Theano from miniconda,

$ conda install -y theano

  • keras-0.2.0

  • theano-0.7.0

got this error during th test.

AttributeError: 'module' object has no attribute 'relu'

I solved the error by re-installing with pip from github.

$ conda uninstall theano
$ sudo apt-get install git
$ pip install git+git://github.com/Theano/Theano.git



References