Affected Cell model

Result_yolo3_fault_2/.gitignore

@@ -0,0 +1 @@
+log_experimento_fault_gpu_2/

Result_yolo3_fault_2/result_yolo3_fault_2/time.txt

@@ -0,0 +1 @@
+Tiempo promedio:1.493388593196869

Result_yolo3_fault_2/yolo3_full_yolo.err

@@ -0,0 +1,32 @@
+2020-06-15 17:18:13.491693: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
+2020-06-15 17:18:13.491844: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
+2020-06-15 17:18:13.491866: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
+2020-06-15 17:18:15.074534: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
+2020-06-15 17:18:15.074590: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
+2020-06-15 17:18:15.074642: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (dlsaavedra-X406UAR): /proc/driver/nvidia/version does not exist
+2020-06-15 17:18:15.074938: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
+2020-06-15 17:18:15.108036: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1800000000 Hz
+2020-06-15 17:18:15.110399: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55c1bd39caf0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
+2020-06-15 17:18:15.110513: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
+WARNING:tensorflow:AutoGraph could not transform <bound method YoloLayer.call of <yolo.YoloLayer object at 0x7facc40764d0>> and will run it as-is.
+Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
+Cause: unexpected indent (<unknown>, line 144)
+WARNING:tensorflow:AutoGraph could not transform <bound method YoloLayer.call of <yolo.YoloLayer object at 0x7faca44dff90>> and will run it as-is.
+Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
+Cause: unexpected indent (<unknown>, line 144)
+WARNING:tensorflow:AutoGraph could not transform <bound method YoloLayer.call of <yolo.YoloLayer object at 0x7faca42a9a10>> and will run it as-is.
+Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
+Cause: unexpected indent (<unknown>, line 144)
+WARNING:tensorflow:ModelCheckpoint mode 1 is unknown, fallback to auto mode.
+WARNING:tensorflow:Model failed to serialize as JSON. Ignoring... Layers with arguments in `__init__` must override `get_config`.
+2020-06-15 17:18:23.031977: I tensorflow/core/profiler/lib/profiler_session.cc:225] Profiler session started.
+2020-06-15 17:18:23.032275: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcupti.so.10.1'; dlerror: libcupti.so.10.1: cannot open shared object file: No such file or directory
+2020-06-15 17:18:23.032314: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1307] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
+2020-06-15 17:18:23.032330: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1346] function cupti_interface_->ActivityRegisterCallbacks( AllocCuptiActivityBuffer, FreeCuptiActivityBuffer)failed with error CUPTI could not be loaded or symbol could not be found.
+2020-06-15 17:18:26.461281: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1329] function cupti_interface_->EnableCallback( 0 , subscriber_, CUPTI_CB_DOMAIN_DRIVER_API, cbid)failed with error CUPTI could not be loaded or symbol could not be found.
+2020-06-15 17:18:26.461372: I tensorflow/core/profiler/internal/gpu/device_tracer.cc:88]  GpuTracer has collected 0 callback api events and 0 activity events.
+2020-06-15 17:21:21.898209: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
+2020-06-15 17:24:27.449155: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
+2020-06-15 17:27:34.489737: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
+2020-06-15 17:30:39.394709: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled
+2020-06-15 17:33:52.841270: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Cancelled: Operation was cancelled

Result_yolo3_fault_2/yolo3_full_yolo.output

@@ -0,0 +1,337 @@
+Seen labels: 	{'2': 127}
+
+Given labels: 	['2']
+
+Training on: 	['2']
+
+multi_gpu:1
+
+Loading pretrained weights.
+
+Train for 40 steps, validate for 10 steps
+Loading pretrained weights.
+
+Epoch 1/200
+ - 269s - loss: 10.3918 - yolo_layer_1_loss: 0.2670 - yolo_layer_2_loss: 1.2377 - yolo_layer_3_loss: 8.8871
+
+Epoch 00001: loss improved from inf to 10.39175, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 2/200
+ - 248s - loss: 9.0657 - yolo_layer_1_loss: 0.0042 - yolo_layer_2_loss: 0.8107 - yolo_layer_3_loss: 8.2509
+
+Epoch 00002: loss improved from 10.39175 to 9.06572, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 3/200
+ - 250s - loss: 7.7642 - yolo_layer_1_loss: 0.0021 - yolo_layer_2_loss: 0.5234 - yolo_layer_3_loss: 7.2387
+
+Epoch 00003: loss improved from 9.06572 to 7.76417, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 4/200
+ - 250s - loss: 7.6795 - yolo_layer_1_loss: 0.0018 - yolo_layer_2_loss: 0.2856 - yolo_layer_3_loss: 7.3921
+
+Epoch 00004: loss improved from 7.76417 to 7.67951, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 5/200
+ - 250s - loss: 7.8414 - yolo_layer_1_loss: 0.0011 - yolo_layer_2_loss: 0.4949 - yolo_layer_3_loss: 7.3454
+
+Epoch 00005: loss did not improve from 7.67951
+Epoch 6/200
+ - 250s - loss: 8.1152 - yolo_layer_1_loss: 0.0016 - yolo_layer_2_loss: 0.5053 - yolo_layer_3_loss: 7.6083
+
+Epoch 00006: loss did not improve from 7.67951
+Epoch 7/200
+ - 249s - loss: 7.3296 - yolo_layer_1_loss: 6.0819e-04 - yolo_layer_2_loss: 0.4263 - yolo_layer_3_loss: 6.9027
+
+Epoch 00007: loss improved from 7.67951 to 7.32961, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 8/200
+ - 250s - loss: 7.8274 - yolo_layer_1_loss: 7.9464e-04 - yolo_layer_2_loss: 0.5090 - yolo_layer_3_loss: 7.3175
+
+Epoch 00008: loss did not improve from 7.32961
+Epoch 9/200
+ - 250s - loss: 8.0380 - yolo_layer_1_loss: 8.6137e-04 - yolo_layer_2_loss: 0.6514 - yolo_layer_3_loss: 7.3856
+
+Epoch 00009: loss did not improve from 7.32961
+Epoch 10/200
+ - 250s - loss: 7.9667 - yolo_layer_1_loss: 0.0022 - yolo_layer_2_loss: 0.6072 - yolo_layer_3_loss: 7.3574
+
+Epoch 00010: loss did not improve from 7.32961
+Epoch 11/200
+ - 250s - loss: 8.4253 - yolo_layer_1_loss: 0.0012 - yolo_layer_2_loss: 0.6697 - yolo_layer_3_loss: 7.7544
+
+Epoch 00011: loss did not improve from 7.32961
+Epoch 12/200
+ - 250s - loss: 8.0104 - yolo_layer_1_loss: 0.0010 - yolo_layer_2_loss: 0.2845 - yolo_layer_3_loss: 7.7248
+
+Epoch 00012: loss did not improve from 7.32961
+Epoch 13/200
+ - 250s - loss: 8.7709 - yolo_layer_1_loss: 7.4243e-04 - yolo_layer_2_loss: 0.8608 - yolo_layer_3_loss: 7.9093
+
+Epoch 00013: loss did not improve from 7.32961
+Epoch 14/200
+ - 250s - loss: 8.7679 - yolo_layer_1_loss: 7.1667e-04 - yolo_layer_2_loss: 1.2540 - yolo_layer_3_loss: 7.5131
+
+Epoch 00014: loss did not improve from 7.32961
+Epoch 15/200
+ - 249s - loss: 6.9143 - yolo_layer_1_loss: 6.4281e-04 - yolo_layer_2_loss: 0.8410 - yolo_layer_3_loss: 6.0726
+
+Epoch 00015: loss improved from 7.32961 to 6.91428, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 16/200
+ - 250s - loss: 6.4881 - yolo_layer_1_loss: 5.6383e-04 - yolo_layer_2_loss: 0.4001 - yolo_layer_3_loss: 6.0874
+
+Epoch 00016: loss improved from 6.91428 to 6.48809, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 17/200
+ - 250s - loss: 8.0069 - yolo_layer_1_loss: 4.0944e-04 - yolo_layer_2_loss: 1.3397 - yolo_layer_3_loss: 6.6669
+
+Epoch 00017: loss did not improve from 6.48809
+Epoch 18/200
+ - 250s - loss: 6.3577 - yolo_layer_1_loss: 2.9129e-04 - yolo_layer_2_loss: 0.7906 - yolo_layer_3_loss: 5.5668
+
+Epoch 00018: loss improved from 6.48809 to 6.35768, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 19/200
+ - 249s - loss: 6.7679 - yolo_layer_1_loss: 3.0164e-04 - yolo_layer_2_loss: 0.5212 - yolo_layer_3_loss: 6.2463
+
+Epoch 00019: loss did not improve from 6.35768
+Epoch 20/200
+ - 249s - loss: 6.4508 - yolo_layer_1_loss: 2.7047e-04 - yolo_layer_2_loss: 0.2659 - yolo_layer_3_loss: 6.1846
+
+Epoch 00020: loss did not improve from 6.35768
+Epoch 21/200
+ - 249s - loss: 6.3571 - yolo_layer_1_loss: 2.9117e-04 - yolo_layer_2_loss: 0.3484 - yolo_layer_3_loss: 6.0084
+
+Epoch 00021: loss improved from 6.35768 to 6.35713, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 22/200
+ - 249s - loss: 6.4931 - yolo_layer_1_loss: 2.4105e-04 - yolo_layer_2_loss: 0.5846 - yolo_layer_3_loss: 5.9083
+
+Epoch 00022: loss did not improve from 6.35713
+Epoch 23/200
+ - 249s - loss: 6.4707 - yolo_layer_1_loss: 1.9136e-04 - yolo_layer_2_loss: 0.5451 - yolo_layer_3_loss: 5.9254
+
+Epoch 00023: loss did not improve from 6.35713
+Epoch 24/200
+ - 249s - loss: 5.9562 - yolo_layer_1_loss: 1.3915e-04 - yolo_layer_2_loss: 0.2577 - yolo_layer_3_loss: 5.6984
+
+Epoch 00024: loss improved from 6.35713 to 5.95620, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 25/200
+ - 249s - loss: 6.1448 - yolo_layer_1_loss: 1.2286e-04 - yolo_layer_2_loss: 0.6805 - yolo_layer_3_loss: 5.4642
+
+Epoch 00025: loss did not improve from 5.95620
+Epoch 26/200
+ - 249s - loss: 5.9585 - yolo_layer_1_loss: 1.0858e-04 - yolo_layer_2_loss: 0.4485 - yolo_layer_3_loss: 5.5098
+
+Epoch 00026: loss did not improve from 5.95620
+Epoch 27/200
+ - 249s - loss: 6.1453 - yolo_layer_1_loss: 9.7766e-05 - yolo_layer_2_loss: 0.5231 - yolo_layer_3_loss: 5.6221
+
+Epoch 00027: loss did not improve from 5.95620
+Epoch 28/200
+ - 249s - loss: 5.6772 - yolo_layer_1_loss: 1.9072e-04 - yolo_layer_2_loss: 0.2765 - yolo_layer_3_loss: 5.4006
+
+Epoch 00028: loss improved from 5.95620 to 5.67724, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 29/200
+ - 249s - loss: 5.3301 - yolo_layer_1_loss: 1.0635e-04 - yolo_layer_2_loss: 8.7561e-04 - yolo_layer_3_loss: 5.3291
+
+Epoch 00029: loss improved from 5.67724 to 5.33012, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 30/200
+ - 249s - loss: 6.8329 - yolo_layer_1_loss: 1.4615e-04 - yolo_layer_2_loss: 1.5756 - yolo_layer_3_loss: 5.2572
+
+Epoch 00030: loss did not improve from 5.33012
+Epoch 31/200
+ - 249s - loss: 5.9033 - yolo_layer_1_loss: 1.9254e-04 - yolo_layer_2_loss: 0.8525 - yolo_layer_3_loss: 5.0506
+
+Epoch 00031: loss did not improve from 5.33012
+Epoch 32/200
+ - 249s - loss: 5.2796 - yolo_layer_1_loss: 1.4435e-04 - yolo_layer_2_loss: 0.2685 - yolo_layer_3_loss: 5.0109
+
+Epoch 00032: loss improved from 5.33012 to 5.27958, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 33/200
+ - 249s - loss: 6.2346 - yolo_layer_1_loss: 6.7379e-05 - yolo_layer_2_loss: 1.3654 - yolo_layer_3_loss: 4.8691
+
+Epoch 00033: loss did not improve from 5.27958
+Epoch 34/200
+ - 249s - loss: 6.1787 - yolo_layer_1_loss: 6.0842e-05 - yolo_layer_2_loss: 0.6770 - yolo_layer_3_loss: 5.5016
+
+Epoch 00034: loss did not improve from 5.27958
+Epoch 35/200
+ - 249s - loss: 5.2614 - yolo_layer_1_loss: 6.5419e-05 - yolo_layer_2_loss: 0.1292 - yolo_layer_3_loss: 5.1321
+
+Epoch 00035: loss improved from 5.27958 to 5.26138, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 36/200
+ - 249s - loss: 5.2841 - yolo_layer_1_loss: 2.1123e-04 - yolo_layer_2_loss: 0.1537 - yolo_layer_3_loss: 5.1302
+
+Epoch 00036: loss did not improve from 5.26138
+Epoch 37/200
+ - 250s - loss: 5.5240 - yolo_layer_1_loss: 3.0572e-04 - yolo_layer_2_loss: 0.5670 - yolo_layer_3_loss: 4.9567
+
+Epoch 00037: loss did not improve from 5.26138
+Epoch 38/200
+ - 250s - loss: 5.7255 - yolo_layer_1_loss: 1.5723e-04 - yolo_layer_2_loss: 0.6625 - yolo_layer_3_loss: 5.0628
+
+Epoch 00038: loss did not improve from 5.26138
+Epoch 39/200
+ - 249s - loss: 6.1184 - yolo_layer_1_loss: 1.5941e-04 - yolo_layer_2_loss: 0.7594 - yolo_layer_3_loss: 5.3588
+
+Epoch 00039: loss did not improve from 5.26138
+Epoch 40/200
+ - 250s - loss: 6.0649 - yolo_layer_1_loss: 1.6101e-04 - yolo_layer_2_loss: 0.6664 - yolo_layer_3_loss: 5.3983
+
+Epoch 00040: loss did not improve from 5.26138
+Epoch 41/200
+ - 249s - loss: 7.6413 - yolo_layer_1_loss: 1.6196e-04 - yolo_layer_2_loss: 2.1583 - yolo_layer_3_loss: 5.4829
+
+Epoch 00041: loss did not improve from 5.26138
+Epoch 42/200
+ - 249s - loss: 6.7180 - yolo_layer_1_loss: 1.1377e-04 - yolo_layer_2_loss: 1.2027 - yolo_layer_3_loss: 5.5152
+
+Epoch 00042: loss did not improve from 5.26138
+Epoch 43/200
+ - 250s - loss: 5.6705 - yolo_layer_1_loss: 9.3619e-05 - yolo_layer_2_loss: 0.7231 - yolo_layer_3_loss: 4.9473
+
+Epoch 00043: loss did not improve from 5.26138
+Epoch 44/200
+ - 249s - loss: 6.2414 - yolo_layer_1_loss: 1.0592e-04 - yolo_layer_2_loss: 0.6340 - yolo_layer_3_loss: 5.6073
+
+Epoch 00044: loss did not improve from 5.26138
+Epoch 45/200
+ - 249s - loss: 5.8108 - yolo_layer_1_loss: 5.7582e-05 - yolo_layer_2_loss: 0.6373 - yolo_layer_3_loss: 5.1734
+
+Epoch 00045: loss did not improve from 5.26138
+Epoch 46/200
+ - 249s - loss: 6.9616 - yolo_layer_1_loss: 9.3750e-05 - yolo_layer_2_loss: 0.8949 - yolo_layer_3_loss: 6.0666
+
+Epoch 00046: loss did not improve from 5.26138
+Epoch 47/200
+ - 249s - loss: 5.3763 - yolo_layer_1_loss: 8.0919e-05 - yolo_layer_2_loss: 0.2562 - yolo_layer_3_loss: 5.1200
+
+Epoch 00047: loss did not improve from 5.26138
+Epoch 48/200
+ - 249s - loss: 5.4027 - yolo_layer_1_loss: 5.5944e-05 - yolo_layer_2_loss: 0.6505 - yolo_layer_3_loss: 4.7521
+
+Epoch 00048: loss did not improve from 5.26138
+Epoch 49/200
+ - 249s - loss: 5.4222 - yolo_layer_1_loss: 5.0298e-05 - yolo_layer_2_loss: 0.5950 - yolo_layer_3_loss: 4.8272
+
+Epoch 00049: loss did not improve from 5.26138
+Epoch 50/200
+ - 249s - loss: 5.5717 - yolo_layer_1_loss: 5.9565e-05 - yolo_layer_2_loss: 0.1377 - yolo_layer_3_loss: 5.4340
+
+Epoch 00050: loss did not improve from 5.26138
+
+Epoch 00050: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
+Epoch 51/200
+ - 249s - loss: 5.9566 - yolo_layer_1_loss: 6.9738e-05 - yolo_layer_2_loss: 0.4066 - yolo_layer_3_loss: 5.5499
+
+Epoch 00051: loss did not improve from 5.26138
+Epoch 52/200
+ - 249s - loss: 4.5897 - yolo_layer_1_loss: 5.2179e-05 - yolo_layer_2_loss: 0.4062 - yolo_layer_3_loss: 4.1835
+
+Epoch 00052: loss improved from 5.26138 to 4.58969, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 53/200
+ - 249s - loss: 4.8618 - yolo_layer_1_loss: 5.1721e-05 - yolo_layer_2_loss: 0.5580 - yolo_layer_3_loss: 4.3037
+
+Epoch 00053: loss did not improve from 4.58969
+Epoch 54/200
+ - 249s - loss: 4.8267 - yolo_layer_1_loss: 5.7835e-05 - yolo_layer_2_loss: 0.3104 - yolo_layer_3_loss: 4.5163
+
+Epoch 00054: loss did not improve from 4.58969
+Epoch 55/200
+ - 249s - loss: 5.7597 - yolo_layer_1_loss: 7.0499e-05 - yolo_layer_2_loss: 0.8402 - yolo_layer_3_loss: 4.9194
+
+Epoch 00055: loss did not improve from 4.58969
+Epoch 56/200
+ - 249s - loss: 4.5171 - yolo_layer_1_loss: 4.7918e-05 - yolo_layer_2_loss: 0.6464 - yolo_layer_3_loss: 3.8707
+
+Epoch 00056: loss improved from 4.58969 to 4.51712, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 57/200
+ - 249s - loss: 4.8879 - yolo_layer_1_loss: 4.4887e-05 - yolo_layer_2_loss: 0.3918 - yolo_layer_3_loss: 4.4960
+
+Epoch 00057: loss did not improve from 4.51712
+Epoch 58/200
+ - 250s - loss: 4.6380 - yolo_layer_1_loss: 4.3016e-05 - yolo_layer_2_loss: 0.4356 - yolo_layer_3_loss: 4.2024
+
+Epoch 00058: loss did not improve from 4.51712
+Epoch 59/200
+ - 249s - loss: 5.1633 - yolo_layer_1_loss: 3.9667e-05 - yolo_layer_2_loss: 0.7258 - yolo_layer_3_loss: 4.4374
+
+Epoch 00059: loss did not improve from 4.51712
+Epoch 60/200
+ - 249s - loss: 4.8450 - yolo_layer_1_loss: 4.7864e-05 - yolo_layer_2_loss: 0.8874 - yolo_layer_3_loss: 3.9576
+
+Epoch 00060: loss did not improve from 4.51712
+Epoch 61/200
+ - 249s - loss: 4.7428 - yolo_layer_1_loss: 4.3229e-05 - yolo_layer_2_loss: 0.2623 - yolo_layer_3_loss: 4.4805
+
+Epoch 00061: loss did not improve from 4.51712
+Epoch 62/200
+ - 249s - loss: 5.4766 - yolo_layer_1_loss: 4.1361e-05 - yolo_layer_2_loss: 0.9122 - yolo_layer_3_loss: 4.5644
+
+Epoch 00062: loss did not improve from 4.51712
+Epoch 63/200
+ - 249s - loss: 4.2940 - yolo_layer_1_loss: 4.3164e-05 - yolo_layer_2_loss: 0.5223 - yolo_layer_3_loss: 3.7716
+
+Epoch 00063: loss improved from 4.51712 to 4.29402, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 64/200
+ - 249s - loss: 4.6400 - yolo_layer_1_loss: 5.8545e-05 - yolo_layer_2_loss: 0.6264 - yolo_layer_3_loss: 4.0135
+
+Epoch 00064: loss did not improve from 4.29402
+Epoch 65/200
+ - 249s - loss: 4.5991 - yolo_layer_1_loss: 5.0934e-05 - yolo_layer_2_loss: 0.4342 - yolo_layer_3_loss: 4.1649
+
+Epoch 00065: loss did not improve from 4.29402
+Epoch 66/200
+ - 249s - loss: 4.9235 - yolo_layer_1_loss: 5.6441e-05 - yolo_layer_2_loss: 0.9522 - yolo_layer_3_loss: 3.9712
+
+Epoch 00066: loss did not improve from 4.29402
+Epoch 67/200
+ - 249s - loss: 5.0479 - yolo_layer_1_loss: 5.0447e-05 - yolo_layer_2_loss: 0.9907 - yolo_layer_3_loss: 4.0571
+
+Epoch 00067: loss did not improve from 4.29402
+Epoch 68/200
+ - 250s - loss: 4.8168 - yolo_layer_1_loss: 4.6199e-05 - yolo_layer_2_loss: 1.1044 - yolo_layer_3_loss: 3.7123
+
+Epoch 00068: loss did not improve from 4.29402
+Epoch 69/200
+ - 249s - loss: 4.5846 - yolo_layer_1_loss: 4.9725e-05 - yolo_layer_2_loss: 0.2604 - yolo_layer_3_loss: 4.3242
+
+Epoch 00069: loss did not improve from 4.29402
+Epoch 70/200
+ - 249s - loss: 5.3970 - yolo_layer_1_loss: 3.2394e-05 - yolo_layer_2_loss: 0.7830 - yolo_layer_3_loss: 4.6140
+
+Epoch 00070: loss did not improve from 4.29402
+Epoch 71/200
+ - 249s - loss: 4.5713 - yolo_layer_1_loss: 3.9810e-05 - yolo_layer_2_loss: 0.4039 - yolo_layer_3_loss: 4.1674
+
+Epoch 00071: loss did not improve from 4.29402
+Epoch 72/200
+ - 249s - loss: 3.8602 - yolo_layer_1_loss: 3.7398e-05 - yolo_layer_2_loss: 0.7578 - yolo_layer_3_loss: 3.1023
+
+Epoch 00072: loss improved from 4.29402 to 3.86019, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+Epoch 73/200
+ - 249s - loss: 4.2252 - yolo_layer_1_loss: 3.9543e-05 - yolo_layer_2_loss: 0.1264 - yolo_layer_3_loss: 4.0988
+
+Epoch 00073: loss did not improve from 3.86019
+Epoch 74/200
+ - 249s - loss: 4.4763 - yolo_layer_1_loss: 3.6080e-05 - yolo_layer_2_loss: 0.5650 - yolo_layer_3_loss: 3.9113
+
+Epoch 00074: loss did not improve from 3.86019
+Epoch 75/200
+ - 249s - loss: 3.5016 - yolo_layer_1_loss: 3.7154e-05 - yolo_layer_2_loss: 0.6020 - yolo_layer_3_loss: 2.8996
+
+Epoch 00075: loss improved from 3.86019 to 3.50156, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+
+Epoch 00076: loss improved from 3.50156 to 4.22446, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+40/40 - 183s - loss: 4.2245 - yolo_layer_loss: 6.4795e-05 - yolo_layer_1_loss: 0.6512 - yolo_layer_2_loss: 3.5732 - val_loss: 6.4259 - val_yolo_layer_loss: 6.5558e-05 - val_yolo_layer_1_loss: 1.3530 - val_yolo_layer_2_loss: 5.0729
+Epoch 2/5
+
+Epoch 00077: loss did not improve from 4.22446
+40/40 - 186s - loss: 4.8331 - yolo_layer_loss: 4.1540e-05 - yolo_layer_1_loss: 1.0007 - yolo_layer_2_loss: 3.8323 - val_loss: 4.9868 - val_yolo_layer_loss: 2.5803e-05 - val_yolo_layer_1_loss: 1.2658e-04 - val_yolo_layer_2_loss: 4.9867
+Epoch 3/5
+
+Epoch 00078: loss did not improve from 4.22446
+40/40 - 187s - loss: 4.3298 - yolo_layer_loss: 8.7633e-06 - yolo_layer_1_loss: 0.6481 - yolo_layer_2_loss: 3.6816 - val_loss: 6.7895 - val_yolo_layer_loss: 8.8222e-06 - val_yolo_layer_1_loss: 0.5092 - val_yolo_layer_2_loss: 6.2803
+Epoch 4/5
+
+Epoch 00079: loss did not improve from 4.22446
+40/40 - 185s - loss: 4.6490 - yolo_layer_loss: 6.7689e-06 - yolo_layer_1_loss: 0.8990 - yolo_layer_2_loss: 3.7501 - val_loss: 5.3483 - val_yolo_layer_loss: 6.3485e-06 - val_yolo_layer_1_loss: 9.0986e-05 - val_yolo_layer_2_loss: 5.3482
+Epoch 5/5
+
+Epoch 00080: loss improved from 4.22446 to 3.98889, saving model to Result_yolo3_fault_2/yolo3_full_fault_2.h5
+40/40 - 193s - loss: 3.9889 - yolo_layer_loss: 5.2988e-06 - yolo_layer_1_loss: 0.6052 - yolo_layer_2_loss: 3.3837 - val_loss: 6.2318 - val_yolo_layer_loss: 4.7080e-06 - val_yolo_layer_1_loss: 6.1751e-05 - val_yolo_layer_2_loss: 6.2318
+35 instances of class 2 with average precision: 0.7193
+mAP using the weighted average of precisions among classes: 0.7193
+mAP: 0.7193

Result_yolo3_fault_2/yolo3_full_yolo_test.err

@@ -0,0 +1,11 @@
+2020-06-15 17:34:31.100582: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
+2020-06-15 17:34:31.100864: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
+2020-06-15 17:34:31.100885: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
+2020-06-15 17:34:32.742430: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
+2020-06-15 17:34:32.742474: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
+2020-06-15 17:34:32.742498: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (dlsaavedra-X406UAR): /proc/driver/nvidia/version does not exist
+2020-06-15 17:34:32.794751: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
+2020-06-15 17:34:32.823902: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1800000000 Hz
+2020-06-15 17:34:32.825891: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x560fce206ff0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
+2020-06-15 17:34:32.825979: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
+WARNING:tensorflow:No training configuration found in save file: the model was *not* compiled. Compile it manually.

Result_yolo3_fault_2/yolo3_full_yolo_test.output

@@ -0,0 +1,4 @@
+dict_items([(0, (0.7269822428245609, 35.0))])
+35 instances of class 2 with average precision: 0.7270
+mAP using the weighted average of precisions among classes: 0.7270
+mAP: 0.7270