Metric Learning : Caffe Example

Prepare (MNIST Data)

  1. python gen_pairwise_imglist.py
  2. /home/water/caffe/build/tools/convert_imageset ./ train.txt train_lmdb --gray
  3. /home/water/caffe/build/tools/convert_imageset ./ train_p.txt train_p_lmdb --gray
  4. /home/water/caffe/build/tools/convert_imageset ./ val.txt val_lmdb --gray
  5. /home/water/caffe/build/tools/convert_imageset ./ val_p.txt val_p_lmdb --gray

Based on Caffe siamese

Train loss vs iters

Visualize Result (Normal)

Visualize Result (τ- SNE)

Object Detection : MXNet SSD Example

Prepare

  1. cd /home/water/mxnet/example/ssd
  2. wget https://github.com/zhreshold/mxnet-ssd/releases/download/v0.6/resnet50_ssd_512_voc0712_trainval.zi
  3. unzip resnet50_ssd_512_voc0712_trainval.zip
  4. python data/demo/download_demo_images.py

Test

  1. python demo.py --gpu 0
  2. python demo.py --epoch 0 --images ./data/demo/dog.jpg --thresh 0.5
  3. python demo.py --cpu --network resnet50 --data-shape 512
  4. . . . . . .
  5. python demo.py --images ./data/demo/person.jpg --thresh 0.3
  6. python demo.py --images ./data/demo/person.jpg --network inceptionv3

Result














MNIST : MXNet

MNIST Data (refer to MNIST : Caffe)

Prepare imglist for MXNet

  1. import os
  2. import sys
  3.  
  4. input_path = sys.argv[1].rstrip(os.sep)
  5.  
  6. out_path = sys.argv[2]
  7.  
  8. filenames = os.listdir(input_path)
  9. with open(out_path, 'w') as f:
  10.     for i, filename in enumerate(filenames):
  11.         filepath = os.sep.join([input_path, filename])
  12.         label = filename[: filename.rfind('.')].split('_')[1]
  13.  
  14.         line = '{}\t{}\t{}\n'.format(i, label, filepath)
  15.         f.write(line)

Convert to rec

  1. python gen_mxnet_imglist.py mnist/train train.lst
  2. python gen_mxnet_imglist.py mnist/val val.lst
  3. python gen_mxnet_imglist.py mnist/test test.lst
  4.  
  5. /home/d/mxnet/bin/im2rec train.lst ./ train.rec color=0
  6. /home/d/mxnet/bin/im2rec val.lst ./ val.rec color=0
  7. /home/d/mxnet/bin/im2rec test.lst ./ test.rec color=0

Train (LeNet-5)

  1. import mxnet as mx
  2.  
  3. data = mx.symbol.Variable('data')
  4.  
  5. conv1 = mx.symbol.Convolution(data=data, kernel=(5, 5), num_filter=20)
  6. pool1 = mx.symbol.Pooling(data=conv1, pool_type="max", kernel=(2, 2), stride=(2, 2))
  7.  
  8. conv2 = mx.symbol.Convolution(data=pool1, kernel=(5, 5), num_filter=50)
  9. pool2 = mx.symbol.Pooling(data=conv2, pool_type="max", kernel=(2, 2), stride=(2, 2))
  10.  
  11. flatten = mx.symbol.Flatten(data=pool2)
  12. fc1 = mx.symbol.FullyConnected(data=flatten, num_hidden=500)
  13. relu1 = mx.symbol.Activation(data=fc1, act_type="relu")
  14.  
  15. fc2 = mx.symbol.FullyConnected(data=relu1, num_hidden=10)
  16.  
  17. lenet5 = mx.symbol.SoftmaxOutput(data=fc2, name='softmax')
  18.  
  19. mod = mx.mod.Module(lenet5, context=mx.gpu(0))
  20.  
  21. #
  22.  
  23. train_dataiter = mx.io.ImageRecordIter(
  24.     path_imgrec="train.rec",
  25.     data_shape=(1, 28, 28),
  26.     batch_size=50,
  27.     mean_r=128,
  28.     scale=0.00390625,
  29.     rand_crop=True,
  30.     min_crop_size=24,
  31.     max_crop_size=28,
  32.     max_rotate_angle=15,
  33.     fill_value=0
  34. )
  35. val_dataiter = mx.io.ImageRecordIter(
  36.     path_imgrec="val.rec",
  37.     data_shape=(1, 28, 28),
  38.     batch_size=100,
  39.     mean_r=128,
  40.     scale=0.00390625,
  41. )
  42.  
  43. #
  44.  
  45. import logging
  46.  
  47. logging.getLogger().setLevel(logging.DEBUG)
  48. fh = logging.FileHandler('train_mnist_lenet.log')
  49. logging.getLogger().addHandler(fh)
  50.  
  51. lr_scheduler = mx.lr_scheduler.FactorScheduler(1000, factor=0.95)
  52. optimizer_params = {
  53.     'learning_rate': 0.01,
  54.     'momentum': 0.9,
  55.     'wd': 0.0005,
  56.     'lr_scheduler': lr_scheduler
  57. }
  58.  
  59. checkpoint = mx.callback.do_checkpoint('mnist_lenet', period=5)
  60.  
  61. mod.fit(train_dataiter,
  62.         eval_data=val_dataiter,
  63.         optimizer_params=optimizer_params,
  64.         num_epoch=36,
  65.         epoch_end_callback=checkpoint)

Log Visualization

Test Model Accuracy

  1. import mxnet as mx
  2.  
  3. test_dataiter = mx.io.ImageRecordIter(
  4.     path_imgrec="test.rec",
  5.     data_shape=(1, 28, 28),
  6.     batch_size=100,
  7.     mean_r=128,
  8.     scale=0.00390625,
  9. )
  10.  
  11. mod = mx.mod.Module.load('mnist_lenet', 35, context=mx.gpu(0))
  12.  
  13. mod.bind(
  14.     data_shapes=test_dataiter.provide_data,
  15.     label_shapes=test_dataiter.provide_label,
  16.     for_training=False)
  17.  
  18. metric = mx.metric.create('acc')
  19.  
  20. mod.score(test_dataiter, metric)
  21.  
  22. for name, val in metric.get_name_value():
  23.     print('{} = {:.2f}%'.format(name, val * 100))

Test Model Time

  1. import time
  2. import mxnet as mx
  3.  
  4. benchmark_dataiter = mx.io.ImageRecordIter(
  5.     path_imgrec="test.rec",
  6.     data_shape=(1, 28, 28),
  7.     batch_size=64,
  8.     mean_r=128,
  9.     scale=0.00390625,
  10. )
  11.  
  12. mod = mx.mod.Module.load('mnist_lenet', 35, context=mx.gpu(0))
  13. mod.bind(
  14.     data_shapes=benchmark_dataiter.provide_data,
  15.     label_shapes=benchmark_dataiter.provide_label,
  16.     for_training=False)
  17.  
  18. start = time.time()
  19.  
  20. for i, batch in enumerate(benchmark_dataiter):
  21.     mod.forward(batch)
  22.  
  23. time_elapsed = time.time() - start
  24. msg = '{} batches iterated!\nAverage forward time per batch: {:.6f} ms'
  25. print(msg.format(i + 1, 1000 * time_elapsed / float(i)))

MNIST : Caffe

Prepare

  1. wget http://deeplearning.net/data/mnist/mnist.pkl.gz

Loading data from pkl

  1. import os
  2. import pickle, gzip
  3. from matplotlib import pyplot
  4.  
  5. print('Loading data from mnist.pkl.gz ...')
  6. with gzip.open('mnist.pkl.gz', 'rb') as f:
  7.     train_set, valid_set, test_set = pickle.load(f)
  8.  
  9. imgs_dir = 'mnist'
  10. os.system('mkdir -p {}'.format(imgs_dir))
  11. datasets = {'train': train_set, 'val': valid_set, 'test': test_set}
  12.  
  13. for dataname, dataset in datasets.items():
  14.     print('Converting {} dataset ...'.format(dataname))
  15.     data_dir = os.sep.join([imgs_dir, dataname])
  16.  
  17.     os.system('mkdir -p {}'.format(data_dir))
  18.  
  19.     for i, (img, label) in enumerate(zip(*dataset)):
  20.  
  21.         filename = '{:0>6d}_{}.jpg'.format(i, label)
  22.  
  23.         filepath = os.sep.join([data_dir, filename])
  24.  
  25.         img = img.reshape((28, 28))
  26.  
  27.         pyplot.imsave(filepath, img, cmap='gray')
  28.         if (i + 1) % 10000 == 0:
  29.             print('{} images converted!'.format(i + 1))

Prepare imglist for Caffe

  1. import os
  2. import sys
  3.  
  4. input_path = sys.argv[1].rstrip(os.sep)
  5.  
  6. output_path = sys.argv[2]
  7.  
  8. filenames = os.listdir(input_path)
  9. with open(output_path, 'w') as f:
  10.     for filename in filenames:
  11.  
  12.         filepath = os.sep.join([input_path, filename])
  13.  
  14.         label = filename[: filename.rfind('.')].split('_')[1]
  15.  
  16.         line = '{} {}\n'.format(filepath, label)
  17.         f.write(line)

Convert to LMDB

  1. python gen_caffe_imglist.py mnist/train train.txt
  2. python gen_caffe_imglist.py mnist/val val.txt
  3. python gen_caffe_imglist.py mnist/test test.txt
  4.  
  5. /home/d/Documents/caffe/build/tools/convert_imageset ./ train.txt train_lmdb --gray --shuffle
  6. /home/d/Documents/caffe/build/tools/convert_imageset ./ val.txt val_lmdb --gray --shuffle
  7. /home/d/Documents/caffe/build/tools/convert_imageset ./ test.txt test_lmdb --gray --shuffle

Train (LeNet-5)

  1. /home/d/Documents/caffe/build/tools/caffe train -solver lenet_solver.prototxt -log_dir ./

Log Visualization

loss_iters

  1. python /home/d/Documents/caffe/tools/extra/plot_training_log.py.example 2 loss_iters.png caffe.ubuntu.d.log


accuracy_iters

  1. python /home/d/Documents/caffe/tools/extra/plot_training_log.py.example 0 accuracy_iters.png caffe.ubuntu.d.log


Test Model Accuracy

  1. /home/d/Documents/caffe/build/tools/caffe test -model lenet_test.prototxt -weights mnist_lenet_iter_36000.caffemodel -iterations 100

Test Model Time

  1. /home/d/Documents/caffe/build/tools/caffe time -model lenet.prototxt

Augmented, Train

loss_iters

  1. python /home/d/Documents/caffe/tools/extra/plot_training_log.py.example 2 loss_iters_aug.png mnist_train.log mnist_train_with_augmentation.log




accuracy_iters

  1. python /home/d/Documents/caffe/tools/extra/plot_training_log.py.example 0 accuracy_iters_aug.png mnist_train.log mnist_train_with_augmentation.log



Update solver max_iter, Train

  1. /home/d/Documents/caffe/build/tools/caffe train -solver lenet_solver_aug.prototxt -snapshot mnist_aug_lenet_iter_36000.solverstate -log_dir ./

Caffe Installation : Ubuntu 16.04

Prepare

  1. sudo apt update
  2. sudo apt install build-essential git libatlas-base-dev
  3. sudo apt-get install python-pip
  4. pip install --upgrade pip
  5. sudo apt-get install graphviz
  6. sudo pip install graphviz
  7. sudo apt install libprotobuf-dev libleveldb-dev libsnappy-dev libboost-all-dev libhdf5-serial-dev protobuf-compiler gfortran libjpeg62 libfreeimage-dev libgoogle-glog-dev libbz2-dev libxml2-dev libxslt-dev libffi-dev libssl-dev libgflags-dev liblmdb-dev python-yaml
  8. sudo apt-get install libopencv-dev python-opencv

Config

  1. # git clone https://github.com/BVLC/caffe. git
  2. # Or
  3. unzip caffe-master.zip 
  4. cd caffe-master/
  5. cp Makefile.config.example Makefile.config
  6.  
  7. d@ubuntu:~/Documents/caffe$ diff Makefile.config.example Makefile.config
  8. 8c8
  9. < # CPU_ONLY := 1
  10. ---
  11. > CPU_ONLY := 1
  12. 94c94
  13. < # WITH_PYTHON_LAYER := 1
  14. ---
  15. > WITH_PYTHON_LAYER := 1
  16. 97,98c97,98
  17. < INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
  18. < LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
  19. ---
  20. > INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
  21. > LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/
  22. d@ubuntu:~/Documents/caffe$

Compile & Test

  1. export LD_LIBRARY=$LD_LIBRARY:/usr/include/hdf5
  2. export PYTHONPATH=$PYTHONPATH:/home/d/Documents/caffe/python
  3. make pycaffe -j
  4. make all -j
  5. make test -j
  6. make runtest

virtual memory exhausted

  1. sudo mkdir /opt/images/
  2. sudo rm -rf /opt/images/swap
  3. sudo dd if=/dev/zero of=/opt/images/swap bs=1024 count=10240000
  4. sudo mkswap /opt/images/swap
  5. sudo swapon /opt/images/swap