A A
[CV] Keras YOLO๋กœ Raccoon Dataset์„ ์ด์šฉํ•œ Object Detection
Raccoon Dataset์„ YOLO V3 Model๋กœ ํ•™์Šต์‹œ์ผœ์„œ Image & Video์— Object Detection์„ ํ•œ๋ฒˆ ์ˆ˜ํ–‰ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.
  • Dataset
 

GitHub - experiencor/keras-yolo3: Training and Detecting Objects with YOLO3

Training and Detecting Objects with YOLO3. Contribute to experiencor/keras-yolo3 development by creating an account on GitHub.

github.com


Library Download

Object Detection ํ•™์Šต์„ ์œ„ํ•œ Library๋ฅผ ๋‹ค์šด๋กœ๋“œ ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.
!pwd
!rm -rf DLCV
!git clone https://github.com/chulminkw/DLCV.git
# DLCV ๋””๋ ‰ํ† ๋ฆฌ๊ฐ€ Download๋˜๊ณ  DLCV ๋ฐ‘์— Detection๊ณผ Segmentation ๋””๋ ‰ํ† ๋ฆฌ๊ฐ€ ์žˆ๋Š” ๊ฒƒ์„ ํ™•์ธ
!ls -lia 
!ls -lia DLCV
!pip install tensorflow-gpu==1.15.2  
!pip install keras==2.3.0
  • ์ฃผ์˜์‚ฌํ•ญ: init.py๋Š” ๋ฐ˜๋“œ์‹œ import tensorflow, import keras ์ด์ „์— ์ˆ˜ํ–‰๋˜์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋งŒ์ผ tensorflow, keras ์„ค์น˜ํ•œ ๋’ค ์•„๋ž˜์˜ import tensorflow, import keras๋ฅผ ๋จผ์ € ์ˆ˜ํ–‰ํ•˜์˜€์œผ๋ฉด ๋ฉ”๋‰ด-> ๋Ÿฐํƒ€์ž„ -> ๋Ÿฐํƒ€์ž„ ๋‹ค์‹œ ์‹œ์ž‘์„ ๋ˆ„๋ฅด์‹  ๋’ค init.py๋ฅผ ์ˆ˜์ •ํ•ฉ๋‹ˆ๋‹ค.
import os

# keras backend ๋””๋ ‰ํ† ๋ฆฌ ์ด๋™. 
os.chdir('/usr/local/lib/python3.6/dist-packages/keras/backend')

# ๊ธฐ์กด __init__.py ์‚ญ์ œํ•˜๊ณ  ์ƒˆ๋กœ์šด __init__.py๋ฅผ download 
!rm -rf __init__.py
!rm -rf __pycache__
!wget https://raw.githubusercontent.com/chulminkw/DLCV/master/colab_tf115_modify_files/__init__.py
# GPU๊ฐ€ ์„ธํŒ…๋˜์–ด ์žˆ์ง€ ์•Š์œผ๋ฉด ์ƒ๋‹จ ๋ฉ”๋‰ด์—์„œ ๋Ÿฐํƒ€์ž„->๋Ÿฐํƒ€์ž„ ์œ ํ˜• ๋ณ€๊ฒฝ์—์„œ GPU๋ฅผ ์„ ํƒํ•œ ํ›„ ๋Ÿฐํƒ€์ž„ ๋‹ค์‹œ ์‹œ์ž‘์„ ์„ ํƒํ•˜๊ณ  ์ฒ˜์Œ ๋ถ€ํ„ฐ์ธ tensorflow, keras ์„ค์น˜ ๋ถ€ํ„ฐ ๋‹ค์‹œ ์‹œ์ž‘. 
import tensorflow as tf
import keras

print(tf.__version__)
print(keras.__version__)

# gpu๊ฐ€ ์„ธํŒ…๋“œ์–ด ์žˆ๋Š”์ง€ ํ™•์ธ. 
tf.test.gpu_device_name()

Raccoon Dataset Download

Object Detection ํ•™์Šต์„ ์œ„ํ•œ Raccoon Dataset์„ ๋‹ค์šด๋กœ๋“œ ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

 

 

GitHub - experiencor/raccoon_dataset: The dataset is used to train my own raccoon detector and I blogged about it on Medium

The dataset is used to train my own raccoon detector and I blogged about it on Medium - experiencor/raccoon_dataset

github.com

# /content/DLCV/data ๋””๋ ‰ํ† ๋ฆฌ์— raccoon_dataset์„ ๋‹ค์šด๋กœ๋“œํ•จ. 
%cd /content/DLCV/data/
!git clone https://github.com/experiencor/raccoon_dataset.git
# raccoon_dataset์„ raccoon์œผ๋กœ ๋””๋ ‰ํ† ๋ฆฌ ์ด๋ฆ„ ๋ณ€๊ฒฝํ•˜๊ณ  ํ™•์ธ 
!mv raccoon_dataset raccoon
!ls -lia
  • VOC Annotation
# annotation๊ณผ image ๋””๋ ‰ํ† ๋ฆฌ ์„ค์ •. annotation๋””๋ ‰ํ† ๋ฆฌ์— ์žˆ๋Š” ํŒŒ์ผ ํ™•์ธ. 
import os
from pathlib import Path

#HOME_DIR = str(Path.home())
# ์ฝ”๋žฉ ๋ฒ„์ „์€ HOME_DIR์„ /content ๋กœ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
HOME_DIR = '/content'

ANNO_DIR = os.path.join(HOME_DIR, 'DLCV/data/raccoon/annotations')
IMAGE_DIR = os.path.join(HOME_DIR, 'DLCV/data/raccoon/images')
print(ANNO_DIR)

files = os.listdir(ANNO_DIR)
print('ํŒŒ์ผ ๊ฐœ์ˆ˜๋Š”:',len(files))
print(files)
/content/DLCV/data/raccoon/annotations
ํŒŒ์ผ ๊ฐœ์ˆ˜๋Š”: 201
import glob  # ํŒŒ์ผ ๊ฒฝ๋กœ ํŒจํ„ด ๋งค์นญ์„ ์œ„ํ•ด ์‚ฌ์šฉ (์˜ˆ: ํŠน์ • ๋””๋ ‰ํ† ๋ฆฌ ๋‚ด์˜ ๋ชจ๋“  XML ํŒŒ์ผ์„ ์ฐพ์„ ๋•Œ)
import xml.etree.ElementTree as ET  # XML ํŒŒ์ผ์„ ํŒŒ์‹ฑํ•˜๊ธฐ ์œ„ํ•ด ์‚ฌ์šฉ
import os  # ํŒŒ์ผ ๊ฒฝ๋กœ ์ฒ˜๋ฆฌ๋ฅผ ์œ„ํ•ด ์‚ฌ์šฉ

# XML ํŒŒ์ผ์„ CSV ํ˜•์‹์˜ ํ…์ŠคํŠธ ํŒŒ์ผ๋กœ ๋ณ€ํ™˜ํ•˜๋Š” ํ•จ์ˆ˜
def xml_to_csv(path, output_filename):
    xml_list = []  # ํŒŒ์‹ฑํ•œ XML ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•  ๋ฆฌ์ŠคํŠธ
    # ์ถœ๋ ฅ ํŒŒ์ผ์„ ์“ฐ๊ธฐ ๋ชจ๋“œ๋กœ ์—ด๊ธฐ
    with open(output_filename, "w") as train_csv_file:
        # ์ฃผ์–ด์ง„ ๊ฒฝ๋กœ์—์„œ ๋ชจ๋“  XML ํŒŒ์ผ์„ ์ˆœํšŒ
        for xml_file in glob.glob(path + '/*.xml'):
            # ๊ฐ XML ํŒŒ์ผ์„ ํŒŒ์‹ฑํ•˜์—ฌ Element Tree ๊ฐ์ฒด๋กœ ๋ณ€ํ™˜ํ•˜๊ณ , ์ตœ์ƒ์œ„(root) ์š”์†Œ๋ฅผ ์ถ”์ถœ
            tree = ET.parse(xml_file)
            root = tree.getroot()
            # XML ๊ตฌ์กฐ์—์„œ ์ด๋ฏธ์ง€ ํŒŒ์ผ๋ช…์„ ์ฐพ์•„ ์ด๋ฏธ์ง€์˜ ์ „์ฒด ๊ฒฝ๋กœ๋ฅผ ์ƒ์„ฑ
            full_image_name = os.path.join(IMAGE_DIR, root.find('filename').text)
            value_str_list = ' '  # ๊ฐ ๊ฐ์ฒด์˜ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค ์ •๋ณด๋ฅผ ์ €์žฅํ•  ๋ฌธ์ž์—ด
            
            # XML์—์„œ 'object' ์š”์†Œ๋ฅผ ๋ชจ๋‘ ์ฐพ์•„ ๋ฐ˜๋ณต, ์ด๋Š” ์ด๋ฏธ์ง€์— ์ฃผ์„๋œ ๊ฐ์ฒด๋ฅผ ์˜๋ฏธ
            for obj in root.findall('object'):
                # 'bndbox' ์š”์†Œ๋ฅผ ์ฐพ์•„ ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ ์ขŒํ‘œ ์ •๋ณด(xmin, ymin, xmax, ymax)๋ฅผ ๊ฐ€์ ธ์˜ด
                xmlbox = obj.find('bndbox')
                x1 = int(xmlbox.find('xmin').text)  # ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ ์ขŒ์ธก ์ƒ๋‹จ x ์ขŒํ‘œ
                y1 = int(xmlbox.find('ymin').text)  # ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ ์ขŒ์ธก ์ƒ๋‹จ y ์ขŒํ‘œ
                x2 = int(xmlbox.find('xmax').text)  # ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ ์šฐ์ธก ํ•˜๋‹จ x ์ขŒํ‘œ
                y2 = int(xmlbox.find('ymax').text)  # ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์˜ ์šฐ์ธก ํ•˜๋‹จ y ์ขŒํ‘œ
                # ๊ฐ์ฒด์˜ ํด๋ž˜์Šค ID, ์—ฌ๊ธฐ์„œ๋Š” ๊ณ ์ •์ ์œผ๋กœ 0์„ ์‚ฌ์šฉ (์˜ˆ: ๊ณ ์ •๋œ ๊ฐ์ฒด๊ฐ€ 'raccoon'์ด๋ผ๊ณ  ๊ฐ€์ •)
                class_id = 0
                # ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค์™€ ํด๋ž˜์Šค ID ์ •๋ณด๋ฅผ ๋ฌธ์ž์—ด๋กœ ํฌ๋งทํŒ…ํ•˜์—ฌ ์ €์žฅ
                value_str = ('{0},{1},{2},{3},{4}').format(x1, y1, x2, y2, class_id)
                value_str_list = value_str_list + value_str + ' '  # ์—ฌ๋Ÿฌ ๊ฐ์ฒด ์ •๋ณด๋ฅผ ์—ฐ๊ฒฐํ•˜์—ฌ ์ €์žฅ
            # ์ด๋ฏธ์ง€ ๊ฒฝ๋กœ์™€ ๊ฐ์ฒด ์ •๋ณด๋ฅผ CSV ํ˜•์‹์œผ๋กœ ์ถœ๋ ฅ ํŒŒ์ผ์— ์ €์žฅ
            train_csv_file.write(full_image_name + ' ' + value_str_list + '\n')
        # XML ํŒŒ์ผ ์ˆœํšŒ๋ฅผ ๋งˆ์นจ
  • glob.glob(path + '/*.xml'): ์ฃผ์–ด์ง„ ๋””๋ ‰ํ† ๋ฆฌ์—์„œ ๋ชจ๋“  XML ํŒŒ์ผ์„ ์ฐพ์Šต๋‹ˆ๋‹ค.
  • ET.parse(xml_file): XML ํŒŒ์ผ์„ ํŒŒ์‹ฑํ•˜์—ฌ ํŠธ๋ฆฌ ๊ตฌ์กฐ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
  • root.findall('object'): XML ํŒŒ์ผ ๋‚ด์—์„œ ๋ชจ๋“  ๊ฐ์ฒด ์ •๋ณด๋ฅผ ์ฐพ์Šต๋‹ˆ๋‹ค.
  • ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค ์ขŒํ‘œ(xmin, ymin, xmax, ymax)๋ฅผ ์ถ”์ถœํ•˜์—ฌ ๊ฐ์ฒด์˜ ์œ„์น˜๋ฅผ ํŒŒ์•…ํ•ฉ๋‹ˆ๋‹ค.
  • ํด๋ž˜์Šค ID๋ฅผ ๊ณ ์ •๊ฐ’(0)์œผ๋กœ ์„ค์ •ํ•˜๊ณ , ์ด๋ฅผ ๋ฌธ์ž์—ด๋กœ ํฌ๋งทํŒ…ํ•˜์—ฌ CSV ํ˜•์‹์œผ๋กœ ๋ณ€ํ™˜ํ•ฉ๋‹ˆ๋‹ค.
xml_to_csv(ANNO_DIR, os.path.join(ANNO_DIR,'raccoon_anno.csv'))
print(os.path.join(ANNO_DIR,'raccoon_anno.csv'))

# /content/DLCV/data/raccoon/annotations/raccoon_anno.csv
import numpy as np
import keras.backend as K
from keras.layers import Input, Lambda
from keras.models import Model
from keras.optimizers import Adam
from keras.callbacks import TensorBoard, ModelCheckpoint, ReduceLROnPlateau, EarlyStopping

 

Keras-yolo3 ํŒจํ‚ค์ง€๋ฅผ ๋‹ค์šด๋กœ๋“œ ํ•ด์„œ, /content/DLCV/Detection/yolo ๋ฐ‘์— ์„ค์น˜
%cd /content/DLCV/Detection/yolo
!git clone https://github.com/qqwweee/keras-yolo3.git
!ls -lia /content/DLCV/Detection/yolo/keras-yolo3
!ls -lia /content/DLCV/Detection/yolo/keras-yolo3
import sys
import os

# ์ฝ”๋žฉ ๋ฒ„์ „์€ ์•„๋ž˜์™€ ๊ฐ™์ด ์ ˆ๋Œ€ ๊ฒฝ๋กœ๋ฅผ ์ง€์ •ํ•˜์—ฌ Local Package ์ง€์ •. 
default_dir = '/content/DLCV'
default_yolo_dir = os.path.join(default_dir, 'Detection/yolo')

LOCAL_PACKAGE_DIR = os.path.abspath(os.path.join(default_yolo_dir,'keras-yolo3'))
print(LOCAL_PACKAGE_DIR)
sys.path.append(LOCAL_PACKAGE_DIR)

from yolo3.model import preprocess_true_boxes, yolo_body, tiny_yolo_body, yolo_loss
from yolo3.utils import get_random_data

 

Pretrained ๋ชจ๋ธ ์žฌ์„ฑ์„ฑ ๋ฐ font directory ๊ต์ฒด
  • model_data ๋ฐ‘์— coco dataset๋กœ pretrained ๋œ yolov3.weights ํŒŒ์ผ์„ yolo.h5 ํŒŒ์ผ๋กœ ๋ณ€๊ฒฝํ•ด ์ค˜์•ผ ํ•ฉ๋‹ˆ๋‹ค
  • keras-yolo3์˜ font ๋””๋ ‰ํ† ๋ฆฌ๋„ ์žฌ ๊ต์ฒด ํ•ฉ๋‹ˆ๋‹ค.
%cd /content/DLCV/Detection/yolo/keras-yolo3
!ls
/content/DLCV/Detection/yolo/keras-yolo3
coco_annotation.py  LICENSE	 train_bottleneck.py  yolov3.cfg
convert.py	    model_data	 train.py	      yolov3-tiny.cfg
darknet53.cfg	    __pycache__  voc_annotation.py    yolov3.weights
font		    README.md	 yolo3		      yolo_video.py
kmeans.py	    snapshots	 yolo.py
# yolov3.weights ํŒŒ์ผ์„ download ๋ฐ›๊ณ , convert.py ๋ฅผ ์ˆ˜ํ–‰ํ•˜์—ฌ model_data ๋ฐ‘์— yolo.h5 ํŒŒ์ผ ์ƒ์„ฑ ์ˆ˜ํ–‰. 
%cd /content/DLCV/Detection/yolo/keras-yolo3 
# yolo ๊ณต์‹ ์‚ฌ์ดํŠธ์—์„œ download์‹œ download ์†๋„๊ฐ€ ์•ฝ 25๋ถ„ ์ •๋„ ์†Œ์š”๋จ. github์—์„œ ๋‹ค์šด๋กœ๋“œ ์š”๋ง. 
#!wget  https://pjreddie.com/media/files/yolov3.weights
# github์—์„œ ๋‹ค์šด๋กœ๋“œ
!wget https://github.com/chulminkw/DLCV/releases/download/1.0/yolov3.weights

# yolov3.weights๋ฅผ keras-yolo3์—์„œ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋„๋ก yolo.h5 ๋กœ ๋ณ€ํ™˜
!python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5
# model_data ๋ฐ‘์— yolo.h5 ํŒŒ์ผ์ด ์ƒ์„ฑ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ. 
!ls /content/DLCV/Detection/yolo/keras-yolo3/model_data
# yolo.detect_image() ๋ฉ”์†Œ๋“œ๋Š” PIL package๋ฅผ ์ด์šฉํ•˜์—ฌ image ์ž‘์—… ์ˆ˜ํ–‰. keras-yolo3/font ๋””๋ ‰ํ† ๋ฆฌ๋ฅผ ์ƒ์œ„ ๋””๋ ‰ํ† ๋ฆฌ๋กœ ๋ณต์‚ฌ ํ•ด์•ผํ•จ.  
%cd /content/DLCV/Detection/yolo
!cp -rf keras-yolo3/font ./font
Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, None, None, 3 0                                            
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, None, None, 3 864         input_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, None, None, 3 128         conv2d_1[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU)       (None, None, None, 3 0           batch_normalization_1[0][0]      
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0           leaky_re_lu_1[0][0]              
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, None, None, 6 18432       zero_padding2d_1[0][0]           
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, None, None, 6 256         conv2d_2[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU)       (None, None, None, 6 0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, None, None, 3 2048        leaky_re_lu_2[0][0]              
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, None, None, 3 128         conv2d_3[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU)       (None, None, None, 3 0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, None, None, 6 18432       leaky_re_lu_3[0][0]              
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, None, None, 6 256         conv2d_4[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU)       (None, None, None, 6 0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 6 0           leaky_re_lu_2[0][0]              
                                                                 leaky_re_lu_4[0][0]              
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, None, None, 6 0           add_1[0][0]                      
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, None, None, 1 73728       zero_padding2d_2[0][0]           
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, None, 1 512         conv2d_5[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU)       (None, None, None, 1 0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, None, None, 6 8192        leaky_re_lu_5[0][0]              
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, None, 6 256         conv2d_6[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU)       (None, None, None, 6 0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, None, None, 1 73728       leaky_re_lu_6[0][0]              
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, None, 1 512         conv2d_7[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU)       (None, None, None, 1 0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 1 0           leaky_re_lu_5[0][0]              
                                                                 leaky_re_lu_7[0][0]              
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, None, None, 6 8192        add_2[0][0]                      
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, None, 6 256         conv2d_8[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU)       (None, None, None, 6 0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, None, None, 1 73728       leaky_re_lu_8[0][0]              
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, None, 1 512         conv2d_9[0][0]                   
__________________________________________________________________________________________________
leaky_re_lu_9 (LeakyReLU)       (None, None, None, 1 0           batch_normalization_9[0][0]      
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 1 0           add_2[0][0]                      
                                                                 leaky_re_lu_9[0][0]              
__________________________________________________________________________________________________
zero_padding2d_3 (ZeroPadding2D (None, None, None, 1 0           add_3[0][0]                      
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, None, None, 2 294912      zero_padding2d_3[0][0]           
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, None, 2 1024        conv2d_10[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_10 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, None, None, 1 32768       leaky_re_lu_10[0][0]             
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, None, 1 512         conv2d_11[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_11 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_11[0][0]     
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_11[0][0]             
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, None, 2 1024        conv2d_12[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_12 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_12[0][0]     
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 2 0           leaky_re_lu_10[0][0]             
                                                                 leaky_re_lu_12[0][0]             
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, None, None, 1 32768       add_4[0][0]                      
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, None, 1 512         conv2d_13[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_13 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_13[0][0]             
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, None, 2 1024        conv2d_14[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_14 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_14[0][0]     
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 2 0           add_4[0][0]                      
                                                                 leaky_re_lu_14[0][0]             
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, None, None, 1 32768       add_5[0][0]                      
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, None, None, 1 512         conv2d_15[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_15 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_15[0][0]             
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, None, None, 2 1024        conv2d_16[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_16 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 2 0           add_5[0][0]                      
                                                                 leaky_re_lu_16[0][0]             
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, None, None, 1 32768       add_6[0][0]                      
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, None, None, 1 512         conv2d_17[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_17 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_17[0][0]     
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_17[0][0]             
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, None, None, 2 1024        conv2d_18[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_18 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 2 0           add_6[0][0]                      
                                                                 leaky_re_lu_18[0][0]             
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, None, None, 1 32768       add_7[0][0]                      
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, None, None, 1 512         conv2d_19[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_19 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_19[0][0]             
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, None, None, 2 1024        conv2d_20[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_20 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_20[0][0]     
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 2 0           add_7[0][0]                      
                                                                 leaky_re_lu_20[0][0]             
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, None, None, 1 32768       add_8[0][0]                      
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, None, None, 1 512         conv2d_21[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_21 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_21[0][0]             
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, None, None, 2 1024        conv2d_22[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_22 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_22[0][0]     
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 2 0           add_8[0][0]                      
                                                                 leaky_re_lu_22[0][0]             
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, None, None, 1 32768       add_9[0][0]                      
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, None, None, 1 512         conv2d_23[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_23 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_23[0][0]     
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_23[0][0]             
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, None, None, 2 1024        conv2d_24[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_24 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_24[0][0]     
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 2 0           add_9[0][0]                      
                                                                 leaky_re_lu_24[0][0]             
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, None, None, 1 32768       add_10[0][0]                     
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, None, None, 1 512         conv2d_25[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_25 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_25[0][0]     
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, None, None, 2 294912      leaky_re_lu_25[0][0]             
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, None, None, 2 1024        conv2d_26[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_26 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_26[0][0]     
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 2 0           add_10[0][0]                     
                                                                 leaky_re_lu_26[0][0]             
__________________________________________________________________________________________________
zero_padding2d_4 (ZeroPadding2D (None, None, None, 2 0           add_11[0][0]                     
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, None, None, 5 1179648     zero_padding2d_4[0][0]           
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, None, None, 5 2048        conv2d_27[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_27 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, None, None, 2 131072      leaky_re_lu_27[0][0]             
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, None, None, 2 1024        conv2d_28[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_28 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_28[0][0]     
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_28[0][0]             
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, None, None, 5 2048        conv2d_29[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_29 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_29[0][0]     
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 5 0           leaky_re_lu_27[0][0]             
                                                                 leaky_re_lu_29[0][0]             
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, None, None, 2 131072      add_12[0][0]                     
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, None, None, 2 1024        conv2d_30[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_30 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_30[0][0]     
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_30[0][0]             
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, None, None, 5 2048        conv2d_31[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_31 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 5 0           add_12[0][0]                     
                                                                 leaky_re_lu_31[0][0]             
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, None, None, 2 131072      add_13[0][0]                     
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, None, None, 2 1024        conv2d_32[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_32 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_32[0][0]     
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_32[0][0]             
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, None, None, 5 2048        conv2d_33[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_33 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_33[0][0]     
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 5 0           add_13[0][0]                     
                                                                 leaky_re_lu_33[0][0]             
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, None, None, 2 131072      add_14[0][0]                     
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, None, None, 2 1024        conv2d_34[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_34 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_34[0][0]             
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, None, None, 5 2048        conv2d_35[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_35 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_35[0][0]     
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 5 0           add_14[0][0]                     
                                                                 leaky_re_lu_35[0][0]             
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, None, None, 2 131072      add_15[0][0]                     
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, None, None, 2 1024        conv2d_36[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_36 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_36[0][0]             
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, None, None, 5 2048        conv2d_37[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_37 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_37[0][0]     
__________________________________________________________________________________________________
add_16 (Add)                    (None, None, None, 5 0           add_15[0][0]                     
                                                                 leaky_re_lu_37[0][0]             
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, None, None, 2 131072      add_16[0][0]                     
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, None, None, 2 1024        conv2d_38[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_38 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_38[0][0]     
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_38[0][0]             
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, None, None, 5 2048        conv2d_39[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_39 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
add_17 (Add)                    (None, None, None, 5 0           add_16[0][0]                     
                                                                 leaky_re_lu_39[0][0]             
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, None, None, 2 131072      add_17[0][0]                     
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, None, None, 2 1024        conv2d_40[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_40 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_40[0][0]     
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_40[0][0]             
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, None, None, 5 2048        conv2d_41[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_41 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
add_18 (Add)                    (None, None, None, 5 0           add_17[0][0]                     
                                                                 leaky_re_lu_41[0][0]             
__________________________________________________________________________________________________
conv2d_42 (Conv2D)              (None, None, None, 2 131072      add_18[0][0]                     
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, None, None, 2 1024        conv2d_42[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_42 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_42[0][0]     
__________________________________________________________________________________________________
conv2d_43 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_42[0][0]             
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, None, None, 5 2048        conv2d_43[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_43 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_43[0][0]     
__________________________________________________________________________________________________
add_19 (Add)                    (None, None, None, 5 0           add_18[0][0]                     
                                                                 leaky_re_lu_43[0][0]             
__________________________________________________________________________________________________
zero_padding2d_5 (ZeroPadding2D (None, None, None, 5 0           add_19[0][0]                     
__________________________________________________________________________________________________
conv2d_44 (Conv2D)              (None, None, None, 1 4718592     zero_padding2d_5[0][0]           
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, None, None, 1 4096        conv2d_44[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_44 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
conv2d_45 (Conv2D)              (None, None, None, 5 524288      leaky_re_lu_44[0][0]             
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, None, None, 5 2048        conv2d_45[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_45 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_45[0][0]     
__________________________________________________________________________________________________
conv2d_46 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_45[0][0]             
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, None, None, 1 4096        conv2d_46[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_46 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_46[0][0]     
__________________________________________________________________________________________________
add_20 (Add)                    (None, None, None, 1 0           leaky_re_lu_44[0][0]             
                                                                 leaky_re_lu_46[0][0]             
__________________________________________________________________________________________________
conv2d_47 (Conv2D)              (None, None, None, 5 524288      add_20[0][0]                     
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, None, None, 5 2048        conv2d_47[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_47 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_47[0][0]     
__________________________________________________________________________________________________
conv2d_48 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_47[0][0]             
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, None, None, 1 4096        conv2d_48[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_48 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_48[0][0]     
__________________________________________________________________________________________________
add_21 (Add)                    (None, None, None, 1 0           add_20[0][0]                     
                                                                 leaky_re_lu_48[0][0]             
__________________________________________________________________________________________________
conv2d_49 (Conv2D)              (None, None, None, 5 524288      add_21[0][0]                     
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, None, None, 5 2048        conv2d_49[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_49 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_49[0][0]     
__________________________________________________________________________________________________
conv2d_50 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_49[0][0]             
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, None, None, 1 4096        conv2d_50[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_50 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_50[0][0]     
__________________________________________________________________________________________________
add_22 (Add)                    (None, None, None, 1 0           add_21[0][0]                     
                                                                 leaky_re_lu_50[0][0]             
__________________________________________________________________________________________________
conv2d_51 (Conv2D)              (None, None, None, 5 524288      add_22[0][0]                     
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, None, None, 5 2048        conv2d_51[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_51 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_51[0][0]     
__________________________________________________________________________________________________
conv2d_52 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_51[0][0]             
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, None, None, 1 4096        conv2d_52[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_52 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_52[0][0]     
__________________________________________________________________________________________________
add_23 (Add)                    (None, None, None, 1 0           add_22[0][0]                     
                                                                 leaky_re_lu_52[0][0]             
__________________________________________________________________________________________________
conv2d_53 (Conv2D)              (None, None, None, 5 524288      add_23[0][0]                     
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, None, None, 5 2048        conv2d_53[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_53 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_53[0][0]     
__________________________________________________________________________________________________
conv2d_54 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_53[0][0]             
__________________________________________________________________________________________________
batch_normalization_54 (BatchNo (None, None, None, 1 4096        conv2d_54[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_54 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_54[0][0]     
__________________________________________________________________________________________________
conv2d_55 (Conv2D)              (None, None, None, 5 524288      leaky_re_lu_54[0][0]             
__________________________________________________________________________________________________
batch_normalization_55 (BatchNo (None, None, None, 5 2048        conv2d_55[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_55 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_55[0][0]     
__________________________________________________________________________________________________
conv2d_56 (Conv2D)              (None, None, None, 1 4718592     leaky_re_lu_55[0][0]             
__________________________________________________________________________________________________
batch_normalization_56 (BatchNo (None, None, None, 1 4096        conv2d_56[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_56 (LeakyReLU)      (None, None, None, 1 0           batch_normalization_56[0][0]     
__________________________________________________________________________________________________
conv2d_57 (Conv2D)              (None, None, None, 5 524288      leaky_re_lu_56[0][0]             
__________________________________________________________________________________________________
batch_normalization_57 (BatchNo (None, None, None, 5 2048        conv2d_57[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_57 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_57[0][0]     
__________________________________________________________________________________________________
conv2d_60 (Conv2D)              (None, None, None, 2 131072      leaky_re_lu_57[0][0]             
__________________________________________________________________________________________________
batch_normalization_59 (BatchNo (None, None, None, 2 1024        conv2d_60[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_59 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_59[0][0]     
__________________________________________________________________________________________________
up_sampling2d_1 (UpSampling2D)  (None, None, None, 2 0           leaky_re_lu_59[0][0]             
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, None, None, 7 0           up_sampling2d_1[0][0]            
                                                                 add_19[0][0]                     
__________________________________________________________________________________________________
conv2d_61 (Conv2D)              (None, None, None, 2 196608      concatenate_1[0][0]              
__________________________________________________________________________________________________
batch_normalization_60 (BatchNo (None, None, None, 2 1024        conv2d_61[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_60 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_60[0][0]     
__________________________________________________________________________________________________
conv2d_62 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_60[0][0]             
__________________________________________________________________________________________________
batch_normalization_61 (BatchNo (None, None, None, 5 2048        conv2d_62[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_61 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_61[0][0]     
__________________________________________________________________________________________________
conv2d_63 (Conv2D)              (None, None, None, 2 131072      leaky_re_lu_61[0][0]             
__________________________________________________________________________________________________
batch_normalization_62 (BatchNo (None, None, None, 2 1024        conv2d_63[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_62 (LeakyReLU)      (None, None, None, 2 0           batch_normalization_62[0][0]     
__________________________________________________________________________________________________
conv2d_64 (Conv2D)              (None, None, None, 5 1179648     leaky_re_lu_62[0][0]             
__________________________________________________________________________________________________
batch_normalization_63 (BatchNo (None, None, None, 5 2048        conv2d_64[0][0]                  
__________________________________________________________________________________________________
leaky_re_lu_63 (LeakyReLU)      (None, None, None, 5 0           batch_normalization_63[0][0]     
___
# colab ๋ฒ„์ „์€ ์•„๋ž˜ ๋ช…๋ น์–ด๋กœ raccoon_class.txt ๋ฅผ ์ˆ˜์ •ํ•ฉ๋‹ˆ๋‹ค. 
BASE_DIR = os.path.join(HOME_DIR, 'DLCV/Detection/yolo/keras-yolo3')
classes_path = os.path.join(BASE_DIR, 'model_data/raccoon_class.txt')
with open(classes_path, "w") as f:
    f.write("raccoon")

# colab ๋ฒ„์ „์€ raccoon_class.txt์— ์ œ๋Œ€๋กœ ๊ธฐ์žฌ๋˜์—ˆ๋‚˜ ํ™•์ธ. 
!cat /content/DLCV/Detection/yolo/keras-yolo3/model_data/raccoon_class.txt
from train import get_classes, get_anchors  # ํด๋ž˜์Šค์™€ ์•ต์ปค ์ •๋ณด๋ฅผ ๋ถˆ๋Ÿฌ์˜ค๋Š” ํ•จ์ˆ˜
from train import create_model, data_generator, data_generator_wrapper  # ๋ชจ๋ธ ์ƒ์„ฑ ๋ฐ ๋ฐ์ดํ„ฐ ์ œ๋„ˆ๋ ˆ์ดํ„ฐ ๊ด€๋ จ ํ•จ์ˆ˜

BASE_DIR = os.path.join(HOME_DIR, 'DLCV/Detection/yolo/keras-yolo3')

## ํ•™์Šต์„ ์œ„ํ•œ ๊ธฐ๋ณธ ํ™˜๊ฒฝ ์„ค์ •. annotation ํŒŒ์ผ, ์ €์žฅ๋œ ๋ชจ๋ธ ํŒŒ์ผ ๊ฒฝ๋กœ, ํด๋ž˜์Šค ํŒŒ์ผ, ์•ต์ปค ํŒŒ์ผ ๊ฒฝ๋กœ๋ฅผ ์„ค์ •.
annotation_path = os.path.join(ANNO_DIR, 'raccoon_anno.csv')  # ์ฃผ์„ ํŒŒ์ผ ๊ฒฝ๋กœ
log_dir = os.path.join(BASE_DIR, 'snapshots/000/')  # ๋ชจ๋ธ ํ•™์Šต ์ค‘๊ฐ„ ๊ฒฐ๊ณผ๋ฅผ ์ €์žฅํ•  ํด๋”
classes_path = os.path.join(BASE_DIR, 'model_data/raccoon_class.txt')  # ํด๋ž˜์Šค ์ •๋ณด ํŒŒ์ผ ๊ฒฝ๋กœ
anchors_path = os.path.join(BASE_DIR, 'model_data/yolo_anchors.txt')  # ์•ต์ปค ๋ฐ•์Šค ์ •๋ณด ํŒŒ์ผ ๊ฒฝ๋กœ

# ํด๋ž˜์Šค ์ด๋ฆ„๊ณผ ์•ต์ปค ๋ฐ•์Šค ์ •๋ณด๋ฅผ ๋ถˆ๋Ÿฌ์˜ด
class_names = get_classes(classes_path)  # ํด๋ž˜์Šค ์ด๋ฆ„์„ ๊ฐ€์ ธ์˜ด
num_classes = len(class_names)  # ํด๋ž˜์Šค ๊ฐœ์ˆ˜๋ฅผ ๊ณ„์‚ฐ
anchors = get_anchors(anchors_path)  # ์•ต์ปค ๋ฐ•์Šค๋ฅผ ๊ฐ€์ ธ์˜ด

# ์ตœ์ดˆ ๊ฐ€์ค‘์น˜ ํŒŒ์ผ์€ COCO ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ํ•™์Šต๋œ YOLO ๋ชจ๋ธ์„ ์‚ฌ์šฉ.
model_weights_path = os.path.join(BASE_DIR, 'model_data/yolo.h5')

input_shape = (416, 416)  # ์ž…๋ ฅ ์ด๋ฏธ์ง€ ํฌ๊ธฐ (416x416), YOLO ๋ชจ๋ธ์—์„œ 32์˜ ๋ฐฐ์ˆ˜์—ฌ์•ผ ํ•จ

# ์•ต์ปค ๋ฐ•์Šค ๊ฐœ์ˆ˜๊ฐ€ 6๊ฐœ๋ฉด Tiny YOLO ๋ชจ๋ธ ์‚ฌ์šฉ
is_tiny_version = len(anchors) == 6

# Tiny YOLO ๋ชจ๋ธ์ด๋ƒ ์ผ๋ฐ˜ YOLO ๋ชจ๋ธ์ด๋ƒ์— ๋”ฐ๋ผ ๋ชจ๋ธ ์ƒ์„ฑ ๋ฐฉ์‹์„ ์„ ํƒ
if is_tiny_version:
    model = create_tiny_model(input_shape, anchors, num_classes,
                              freeze_body=2, weights_path=model_weights_path)
else:
    model = create_model(input_shape, anchors, num_classes,
                         freeze_body=2, weights_path=model_weights_path)  # ๋ ˆ์ด์–ด ์ผ๋ถ€ ๊ณ ์ •

# ๋ชจ๋ธ ํ•™์Šต ์‹œ ์‚ฌ์šฉํ•  ์ฝœ๋ฐฑ ์„ค์ • (TensorBoard ๋กœ๊ทธ, ์ฒดํฌํฌ์ธํŠธ ์ €์žฅ, ํ•™์Šต๋ฅ  ์กฐ์ • ๋“ฑ)
logging = TensorBoard(log_dir=log_dir)
checkpoint = ModelCheckpoint(log_dir + 'ep{epoch:03d}-loss{loss:.3f}-val_loss{val_loss:.3f}.h5',
                             monitor='val_loss', save_weights_only=True, save_best_only=True, period=3)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=3, verbose=1)
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=1)

val_split = 0.1  # ํ•™์Šต ๋ฐ์ดํ„ฐ ์ค‘ 10%๋ฅผ ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ๋กœ ์‚ฌ์šฉ

# ์ฃผ์„ ํŒŒ์ผ์„ ์—ด์–ด ์ด๋ฏธ์ง€์™€ ๊ฐ์ฒด ์ •๋ณด๋ฅผ ์ฝ์–ด์˜ด
with open(annotation_path) as f:
    lines = f.readlines()

# ๋ฐ์ดํ„ฐ๋ฅผ ์…”ํ”Œํ•˜์—ฌ ํ›ˆ๋ จ ๋ฐ์ดํ„ฐ์™€ ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ๋กœ ๋‚˜๋ˆ”
np.random.seed(10101)
np.random.shuffle(lines)
np.random.seed(None)
num_val = int(len(lines) * val_split)  # ๊ฒ€์ฆ ๋ฐ์ดํ„ฐ ๊ฐœ์ˆ˜
num_train = len(lines) - num_val  # ํ•™์Šต ๋ฐ์ดํ„ฐ ๊ฐœ์ˆ˜

# ์ฒซ ๋ฒˆ์งธ ํ•™์Šต ๋‹จ๊ณ„: ๊ณ ์ •๋œ ๋ ˆ์ด์–ด๋กœ ํ•™์Šต์„ ์‹œ์ž‘ํ•˜์—ฌ ๊ธฐ๋ณธ์ ์ธ ์†์‹ค์„ ์•ˆ์ •ํ™”
if True:
    model.compile(optimizer=Adam(lr=1e-3), loss={
        # yolo_loss๋Š” ์‚ฌ์šฉ์ž ์ •์˜ ์†์‹ค ํ•จ์ˆ˜ (Lambda ๋ ˆ์ด์–ด ์‚ฌ์šฉ)
        'yolo_loss': lambda y_true, y_pred: y_pred})

    batch_size = 4  # ๋ฐฐ์น˜ ํฌ๊ธฐ ์„ค์ •
    print('Train on {} samples, val on {} samples, with batch size {}.'.format(num_train, num_val, batch_size))
    # ๋ฐ์ดํ„ฐ ์ œ๋„ˆ๋ ˆ์ดํ„ฐ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ชจ๋ธ์„ ํ•™์Šต
    model.fit_generator(data_generator_wrapper(lines[:num_train], batch_size, input_shape, anchors, num_classes),
                        steps_per_epoch=max(1, num_train // batch_size),
                        validation_data=data_generator_wrapper(lines[num_train:], batch_size, input_shape, anchors, num_classes),
                        validation_steps=max(1, num_val // batch_size),
                        epochs=50,  # 50 ์—ํฌํฌ ๋™์•ˆ ํ•™์Šต
                        initial_epoch=0,
                        callbacks=[logging, checkpoint])
    model.save_weights(log_dir + 'trained_weights_stage_1.h5')  # ์ฒซ ๋ฒˆ์งธ ํ•™์Šต ๋‹จ๊ณ„์˜ ๊ฐ€์ค‘์น˜๋ฅผ ์ €์žฅ

# ๋‘ ๋ฒˆ์งธ ํ•™์Šต ๋‹จ๊ณ„: ๊ณ ์ •๋œ ๋ ˆ์ด์–ด๋ฅผ ํ’€๊ณ (fine-tune) ์ถ”๊ฐ€ ํ•™์Šต ์ง„ํ–‰
if True:
    for i in range(len(model.layers)):
        model.layers[i].trainable = True  # ๋ชจ๋“  ๋ ˆ์ด์–ด๋ฅผ ํ•™์Šต ๊ฐ€๋Šฅ ์ƒํƒœ๋กœ ์„ค์ •
    model.compile(optimizer=Adam(lr=1e-4), loss={'yolo_loss': lambda y_true, y_pred: y_pred})  # ๋ณ€๊ฒฝ ์‚ฌํ•ญ์„ ์ ์šฉํ•˜๊ธฐ ์œ„ํ•ด ๋ชจ๋ธ ์žฌ์ปดํŒŒ์ผ
    print('Unfreeze all of the layers.')

    batch_size = 4  # ๊ณ ์ •๋œ ๋ ˆ์ด์–ด๋ฅผ ํ‘ผ ํ›„์—๋Š” ๋” ๋งŽ์€ GPU ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ํ•„์š”ํ•  ์ˆ˜ ์žˆ์Œ
    print('Train on {} samples, val on {} samples, with batch size {}.'.format(num_train, num_val, batch_size))
    # ๋™์ผํ•œ ๋ฐฉ์‹์œผ๋กœ ํ•™์Šต, ๊ทธ๋Ÿฌ๋‚˜ ๋ชจ๋“  ๋ ˆ์ด์–ด๊ฐ€ ํ•™์Šต๋˜๋„๋ก ์„ค์ •
    model.fit_generator(data_generator_wrapper(lines[:num_train], batch_size, input_shape, anchors, num_classes),
                        steps_per_epoch=max(1, num_train // batch_size),
                        validation_data=data_generator_wrapper(lines[num_train:], batch_size, input_shape, anchors, num_classes),
                        validation_steps=max(1, num_val // batch_size),
                        epochs=100,  # 100 ์—ํฌํฌ ๋™์•ˆ ํ•™์Šต
                        initial_epoch=50,
                        callbacks=[logging, checkpoint, reduce_lr, early_stopping])
    model.save_weights(log_dir + 'trained_weights_final.h5')  # ์ตœ์ข… ๊ฐ€์ค‘์น˜ ์ €์žฅ
  • ๋ชจ๋ธ ๋กœ๋”ฉ: COCO ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ๋ฏธ๋ฆฌ ํ•™์Šต๋œ YOLO ๋ชจ๋ธ์„ ๋กœ๋”ฉํ•˜๊ณ , ์ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•™์Šต์„ ์‹œ์ž‘ํ•ฉ๋‹ˆ๋‹ค.
  • ๋ชจ๋ธ ํ•™์Šต ๊ณผ์ •:
    • ๊ณ ์ •๋œ ๋ ˆ์ด์–ด๋กœ ๋จผ์ € ํ•™์Šต์„ ์ง„ํ–‰ํ•˜์—ฌ ๊ธฐ๋ณธ์ ์ธ ์†์‹ค์„ ์•ˆ์ •ํ™”ํ•ฉ๋‹ˆ๋‹ค.
    • ๊ทธ ํ›„, ๊ณ ์ •๋œ ๋ ˆ์ด์–ด๋ฅผ ํ’€์–ด(fine-tuning), ๋ชจ๋ธ์„ ๋”์šฑ ์„ธ๋ถ€์ ์œผ๋กœ ํ•™์Šตํ•ฉ๋‹ˆ๋‹ค.
  • ์ฝœ๋ฐฑ ํ•จ์ˆ˜: ํ•™์Šต ์ค‘๊ฐ„์ค‘๊ฐ„์— ์†์‹ค ๊ฐ’์— ๋”ฐ๋ผ ํ•™์Šต๋ฅ ์„ ์กฐ์ •ํ•˜๊ฑฐ๋‚˜ ํ•™์Šต์„ ์กฐ๊ธฐ ์ข…๋ฃŒํ•˜๋Š” ๋“ฑ ๋‹ค์–‘ํ•œ ์ฝœ๋ฐฑ ํ•จ์ˆ˜๋“ค์ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค.
  • ๋ฐ์ดํ„ฐ ๋ถ„ํ• : ์ฃผ์„ ๋ฐ์ดํ„ฐ๋ฅผ ์…”ํ”Œํ•œ ํ›„, 90%๋Š” ํ•™์Šต์—, 10%๋Š” ๊ฒ€์ฆ์— ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค.
# snapshots/000 ๋””๋ ‰ํ† ๋ฆฌ์— ํ•™์Šต๋œ weight ํŒŒ์ผ์ด ์ƒ์„ฑ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ. 
!ls /content/DLCV/Detection/yolo/keras-yolo3/snapshots/000

ํ•™์Šต๋œ model์„ ์ด์šฉํ•˜์—ฌ Inference ์ˆ˜ํ–‰

# YOLO ๊ฐ์ฒด ์ƒ์„ฑ. 
import sys
import argparse
from yolo import YOLO, detect_video
#keras-yolo์—์„œ image์ฒ˜๋ฆฌ๋ฅผ ์ฃผ์š” PIL๋กœ ์ˆ˜ํ–‰. 
from PIL import Image

#LOCAL_PACKAGE_DIR = os.path.abspath("./keras-yolo3")
#sys.path.append(LOCAL_PACKAGE_DIR)
# ์ฝ”๋žฉ ๋ฒ„์ „์€ ์ ˆ๋Œ€ ๊ฒฝ๋กœ๋ฅผ ์„ค์ •. 
HOME_DIR='/content'
raccoon_yolo = YOLO(model_path=os.path.join(HOME_DIR,'DLCV/Detection/yolo/keras-yolo3/snapshots/000/trained_weights_final.h5'),
            anchors_path=os.path.join(HOME_DIR,'DLCV/Detection/yolo/keras-yolo3/model_data/yolo_anchors.txt'),
            classes_path=os.path.join(HOME_DIR, 'DLCV/Detection/yolo/keras-yolo3/model_data/raccoon_class.txt'))
            
# /content/DLCV/Detection/yolo/keras-yolo3/snapshots/000/trained_weights_final.h5 model, anchors, and classes loaded.
import matplotlib
import matplotlib.pyplot as plt

img = Image.open(os.path.join(IMAGE_DIR, 'raccoon-171.jpg'))

plt.figure(figsize=(12, 12))
plt.imshow(img)

 

<matplotlib.image.AxesImage at 0x7fdcd7226588>

detected_img = raccoon_yolo.detect_image(img)

plt.figure(figsize=(12, 12))
plt.imshow(detected_img)
(416, 416, 3)
Found 1 boxes for img
raccoon 0.99 (105, 29) (185, 106)
2.3317793840005834
<matplotlib.image.AxesImage at 0x7fdcda716b70>