Hi,

I found this tutorial via AirSim repo. Fortunately, I was able to run AirSim on Mac, but this cookbook doesn't have any binaries except for windows, does it possible to include UnrealEngine sources to be able to build on my own?

Best regards,
Vladyslav Shkola

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

(Please provide a 2-3 sentence description of your problem. Be concise to ensure this description is useful for future users who might run into the same issue.)

Problem details

(Please describe your problem in as much detail as possible here. Make sure to include screenshots, code snippets, error messages, links and anything else you think will help us understand your problem better. If applicable, please also provide us a list of steps to reproduce your problem.)

Exepriment/Environment details

• Tutorial used: (For example, AirSimE2EDeepLearning, DistributedRL etc.)
• Environment used: (For example, landscape, city, hawaii etc.)
• Versions of artifacts used (if applicable): (For example, Python 3.5, Keras 2.1.2 etc.)

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

*I have finished learning the TrainModel.ipynb，and the results looks good, but when i tried to run the testModel.ipynb, I met this error as the following

Problem details

*---------------------------------------------------------------------------
OSError Traceback (most recent call last)
in ()
----> 1 model = load_model(MODEL_PATH)
2
3 client = CarClient()
4 client.confirmConnection()
5 client.enableApiControl(True)

C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\keras\models.py in load_model(filepath, custom_objects, compile)
232 return custom_objects[obj]
233 return obj
--> 234 with h5py.File(filepath, mode='r') as f:
235 # instantiate model
236 model_config = f.attrs.get('model_config')

C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\h5py_hl\files.py in init(self, name, mode, driver, libver, userblock_size, swmr, **kwds)
267 with phil:
268 fapl = make_fapl(driver, libver, **kwds)
--> 269 fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
270
271 if swmr_support:

C:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\h5py_hl\files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
97 if swmr and swmr_support:
98 flags |= h5f.ACC_SWMR_READ
---> 99 fid = h5f.open(name, flags, fapl=fapl)
100 elif mode == 'r+':
101 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\h5f.pyx in h5py.h5f.open()

OSError: Unable to open file (unable to open file: name = 'E:\software\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\EndToEndLearningRawData\model', errno = 13, error message = 'Permission denied', flags = 0, o_flags = 0)

Exepriment/Environment details

• Tutorial used: (AirSimE2EDeepLearning)
• Environment used: ( landscape)
• Versions of artifacts used (if applicable): (Python 3.6, tensorflow 1.6 gpu, Keras 2.1.2)

Problem description

To run TestModel.ipynb using the GPU with small memory such as GTX1060 6GB.
Memory error comes out like below.

Problem details

Error message looks like below.

~\cuda\cuda_blas.cc:443] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
~\cuda\cuda_blas.cc:443] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
~\cuda\cuda_blas.cc:443] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
~\cuda\cuda_dnn.cc:385] could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
~\cuda\cuda_dnn.cc:352] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM
~\tensorflow\core\kernels\conv_ops.cc:717] Check failed: stream->parent()->GetConvolveAlgorithms( conv_parameters.ShouldIncludeWinogradNonfusedAlgo(), &algorithms)

Solution

This error is related with lack of memory.
Add this code in TestModel.ipynb.

import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)

If you want to know details about solutions, please see answers from nVidia :
https://devtalk.nvidia.com/default/topic/1021858/jetson-tx2/tensorflow-memory-error/1

Are there anyway I can just run my model with AirSim on a command line in Linux terminal ? (In other word, headless mode) So that I can train them on my lab server.

Thank you!

when running the following code in TrainModel.ipynb an error arose:
[sample_batch_train_data, sample_batch_test_data] = next(train_generator)

the error is:

ValueError Traceback (most recent call last)
in ()
24 plt.show()
25
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 for i in range(0, 3, 1):
28 draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])
D:\tools\anaconda\envs\ND-P2-python3\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
735
736 def next(self, *args, **kwargs):
--> 737 return self.next(*args, **kwargs)
738
739
~\AutonomousDrivingCookbook\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):
~\AutonomousDrivingCookbook\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
243
244 def __get_indexes(self, index_array):
--> 245 index_array = sorted(index_array)
246 if self.x_prev_states is not None:
247 batch_x_images = np.zeros(tuple([self.batch_size]+ list(self.x_images.shape)[1:]),
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

please note that I have not added or deleted any code in Generator.py file. And I have generated datasets successfully in step 0.
I do not know why this error arose, it makes no sense that sorted() function made this error.

I trained the model successfully and tried to run TestModel.ipynb.
but connection with Airsim is not works.

1. I changed MODEL_PATH with trained model.
   MODEL_PATH = 'model/models/model_model.26-0.0002362.h5'
   -> Runs well.

2. Run AD_Cookbook_Start_AirSim.ps1 using PowerShell.
   -> Runs well.

3. No further process during connecting to AirSim.
   Waiting for connection:
   -> Not works.

I've already tested examples of CNTK-AirSim (Drone, Car) and it works well in my laptop.
Test Environments
OS : Windows 10 64bit.
GPU : GTX1060 6GB
Anaconda3

Waiting for connection:

TransportError Traceback (most recent call last)
in ()
2
3 client = CarClient()
----> 4 client.confirmConnection()
5 client.enableApiControl(True)
6 car_controls = CarControls()

~/Desktop/AutonomousDrivingCookbook-master/AirSimE2EDeepLearning/AirSimClient.py in confirmConnection(self)
158 def confirmConnection(self):
159 print('Waiting for connection: ', end='')
--> 160 home = self.getHomeGeoPoint()
161 while ((home.latitude == 0 and home.longitude == 0 and home.altitude == 0) or
162 math.isnan(home.latitude) or math.isnan(home.longitude) or math.isnan(home.altitude)):

~/Desktop/AutonomousDrivingCookbook-master/AirSimE2EDeepLearning/AirSimClient.py in getHomeGeoPoint(self)
167
168 def getHomeGeoPoint(self):
--> 169 return GeoPoint.from_msgpack(self.client.call('getHomeGeoPoint'))
170
171 # basic flight control

~/.conda/envs/tensorflow/lib/python3.5/site-packages/msgpackrpc/session.py in call(self, method, *args)
39
40 def call(self, method, *args):
---> 41 return self.send_request(method, args).get()
42
43 def call_async(self, method, *args):

~/.conda/envs/tensorflow/lib/python3.5/site-packages/msgpackrpc/future.py in get(self)
41 else:
42 if isinstance(self._error, error.RPCError):
---> 43 raise self._error
44 else:
45 raise error.RPCError(self._error)

TransportError: Retry connection over the limit

Problem description

Hi, everyone. I met an error showed init() takes 3 positional arguments but 4 were given when I tried to run the TestModel today. I have no idea how to solve it.

Problem details

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: landscape
• Versions of artifacts used (if applicable):
  Python 3.6,
  Keras 2.1.2
  numpy 1.14.2
  pandas 0.22.0
  tensorflow-gpu 1.6.0

Problem description

I've completed the data processing and tried to train the model. However, I encountered an error message saying that there is an out of bound error in numpy

Problem details

D:\AutonomousDrivingCookbook-master\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning>py TrainModel.py

C:\Users\Richard Chen\AppData\Local\Programs\Python\Python36\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Successfully init generators
Traceback (most recent call last):
  File "TrainModel.py", line 72, in <module>
    [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
  File "C:\Users\Richard Chen\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\preprocessing\image.py", line 860, in __next__
    return self.next(*args, **kwargs)
  File "D:\AutonomousDrivingCookbook-master\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py", line 242, in next
    return self.__get_indexes(index_array)
  File "D:\AutonomousDrivingCookbook-master\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py", line 264, in __get_indexes
    transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
  File "D:\AutonomousDrivingCookbook-master\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py", line 140, in random_transform_with_states
    img_channel_axis)
  File "C:\Users\Richard Chen\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\preprocessing\image.py", line 179, in random_channel_shift
    x = np.rollaxis(x, channel_axis, 0)
  File "C:\Users\Richard Chen\AppData\Local\Programs\Python\Python36\lib\site-packages\numpy\core\numeric.py", line 1442, in rollaxis
    axis = normalize_axis_index(axis, n)
numpy.core._internal.AxisError: axis 3 is out of bounds for array of dimension 3

Exepriment/Environment details

• Tutorial used: \AirSimE2EDeepLearning, I exported a python file using jupyter notebook
• Environment used: Windows 10 64-bit
• Versions of artifacts used (if applicable): Python 3.6

For the AD_Cookbook_Start_AirSim.ps1 simulation file in AD_Cookbook_AirSim, it seems that is it only supports Windows, just wondering is there a build for linux? Thanks!

Problem description

Trained model has failed to navigate the car through the landscape simulation environment.

Problem details

I have downloaded the AirSim client through the link mentioned in the cookbook readme, along with the training data for the landscape environment. I have followed all notebooks, exploration, train and test. After training for 26 epochs, I have reached a validation loss of 0.00034645, however, the model failed to drive the car on the landscape track simulation, but crashed before it even passed the first bridge.

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: landscape
• Versions of artifacts used: Python 3.5.2, Keras 2.1.3

Problem description

(Please provide a 2-3 sentence description of your problem. Be concise to ensure this description is useful for future users who might run into the same issue.)

1.Goal: I am trying to use RL in my own unreal environment to make the car following a trajectory(a set of positions) with the API controller in a constant speed.
PS: If there is a better solution, please let me know :)
2. I use the trajectory as the reward_road to train the car to follow.
reward_points.txt:
0.0 0.121896 0.0 2.63417 0.0 2.63417 0.0 7.09148 0.0 7.09148 0.0 11.565 0.0 11.565 0.0 14.8439 0.0 14.8439 0.211826 16.385 0.211826 16.385 0.93268 17.709 0.93268 17.709 2.08511 18.7377 2.08511 18.7377 3.50835 19.4377 3.50835 19.4377 5.177 19.8601 5.177 19.8601 7.01984 20.0 7.01984 20.0 10.0058 20.0 10.0058 20.0 14.0306 20.0 14.0306 20.0 17.6185 20.0
diagram from DistributedRL/ExploreAlgorithm.ipynb:

3. how should I generate the pretrain_model_weights for transfer training?
   weights_path = os.path.join(os.getcwd(), 'Share\\data\\pretrain_model_weights.h5')

4. When I run the DistributedRL in neighborhood environment, I notice that the car driving on a curly path (that keep steering all the time). Is there a way that I can reduce the steering behavior when the car is driving?

Exepriment/Environment details

• Tutorial used: (For example, AirSimE2EDeepLearning, DistributedRL etc.)
  DistributedRL

it seems 45min for you
my env: gipu=gtx1070 keras=2.1.2 tersonflow-gpu cudn=v8 cudnn=5.1

Thanks for writing this tutorial!
But there is a question about the xbox controller.

1.Using the scene "AD_Cookbook_AirSim\Hawaii" with xbox one controller , the input steering and throttle value are too large. How to set it properly?

2.When problem 1 happened, pressing backspace would not make all values back to 0 . The car will keep rotating.

I tried to run this Autonomous driving cookbook on other maps like the neighborhood. However, the steering angle values in " airsim_rec.txt" only have three values: -1, 0, and 1. I have attached one file below.
airsim_rec.txt

Don't know how to deal with it. Has anyone ever met this issue? Thank you！

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

(Please provide a 2-3 sentence description of your problem. Be concise to ensure this description is useful for future users who might run into the same issue.)

Problem details

(Please describe your problem in as much detail as possible here. Make sure to include screenshots, code snippets, error messages, links and anything else you think will help us understand your problem better. If applicable, please also provide us a list of steps to reproduce your problem.)

Exepriment/Environment details

• Tutorial used: (For example, AirSimE2EDeepLearning, DistributedRL etc.)
• Environment used: (For example, landscape, city, hawaii etc.)
• Versions of artifacts used (if applicable): (For example, Python 3.5, Keras 2.1.2 etc.)

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

(Please provide a 2-3 sentence description of your problem. Be concise to ensure this description is useful for future users who might run into the same issue.)

Problem details

(Please describe your problem in as much detail as possible here. Make sure to include screenshots, code snippets, error messages, links and anything else you think will help us understand your problem better. If applicable, please also provide us a list of steps to reproduce your problem.)

Exepriment/Environment details

• Tutorial used: (For example, AirSimE2EDeepLearning, DistributedRL etc.)
• Environment used: (For example, landscape, city, hawaii etc.)
• Versions of artifacts used (if applicable): (For example, Python 3.5, Keras 2.1.2 etc.)

when I runned the TrainModel.ipynb, it raised this error:
AxisError Traceback (most recent call last)
in ()
24 plt.show()
25 #%pdb on
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 #for i in range(0, 3, 1):
28 #draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

~\Anaconda3\envs\tensorflow\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
858
859 def next(self, *args, **kwargs):
--> 860 return self.next(*args, **kwargs)
861
862 def _get_batches_of_transformed_samples(self, index_array):

C:\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

C:\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

C:\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

~\Anaconda3\envs\tensorflow\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
177
178 def random_channel_shift(x, intensity, channel_axis=0):
--> 179 x = np.rollaxis(x, channel_axis, 0)
180 min_x, max_x = np.min(x), np.max(x)
181 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

~\Anaconda3\envs\tensorflow\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1470 """
1471 n = a.ndim
-> 1472 axis = normalize_axis_index(axis, n)
1473 if start < 0:
1474 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

Is there anybody can help me! thanks!

Hello,

I have a question about the environments used in this cookbook. Do you update this link https://airsimtutorialdataset.blob.core.windows.net/e2edl/AD_Cookbook_AirSim.7z when new release appears on the github Microsoft/Airsim (https://github.com/Microsoft/AirSim/releases) ? I've been working on this cookbook for a while and have notified that the road_points / reward_points in your NH environment are not the same as in NH from github Microsoft / Airsim. The physic of the car is also different.

Problem description

AxisError: axis 3 is out of bounds for array of dimension 3

Problem details

I have finished step 0 ,run the data preparation.
The error is in step 1 of this line of code:

[sample_batch_train_data, sample_batch_test_data] = next(train_generator)

I did not change any code. Everything before this line runs well.

the error message is here:

AxisError Traceback (most recent call last)
in ()
1
----> 2 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
3 for i in range(0, 3, 1):
4 draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])
5 print("11")

c:\python\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
1067
1068 def next(self, *args, **kwargs):
-> 1069 return self.next(*args, **kwargs)
1070
1071 def _get_batches_of_transformed_samples(self, index_array):

e:\endtoend\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

e:\endtoend\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

e:\endtoend\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

c:\python\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
188
189 """
--> 190 x = np.rollaxis(x, channel_axis, 0)
191 min_x, max_x = np.min(x), np.max(x)
192 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

c:\python\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1440 """
1441 n = a.ndim
-> 1442 axis = normalize_axis_index(axis, n)
1443 if start < 0:
1444 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used:
• Versions of artifacts used: Python 3.6, Keras 2.1.6, jupyter

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

when I runned the TrainModel.ipynb, it raised this error:
AxisError Traceback (most recent call last)

Problem details

in ()
24 plt.show()
25 #%pdb on
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 #for i in range(0, 3, 1):
28 #draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

~\Anaconda3\envs\tensorflow\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
858
859 def next(self, *args, **kwargs):
--> 860 return self.next(*args, **kwargs)
861
862 def _get_batches_of_transformed_samples(self, index_array):

C:\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

C:\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

C:\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

~\Anaconda3\envs\tensorflow\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
177
178 def random_channel_shift(x, intensity, channel_axis=0):
--> 179 x = np.rollaxis(x, channel_axis, 0)
180 min_x, max_x = np.min(x), np.max(x)
181 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

~\Anaconda3\envs\tensorflow\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1470 """
1471 n = a.ndim
-> 1472 axis = normalize_axis_index(axis, n)
1473 if start < 0:
1474 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: **
• Versions of artifacts used (if applicable): Python 3.5, Keras 2.1.5 etc.)

Is there anybody can help me! thanks!

I've tried to run 'TrainModel.ipynb' using Jupyter Notebook.
but It is difficult to install OpenCV libraries because of version matchings.

Can you tell me versions of additional packages such as openCV, CUDA, CuDNN?

I'm trying to run the code with
Python 3.5.2
Anaconda 4.1.1 (64bit)
CNTK 2.4
Keras 2.1.4
Tensorflow-gpu 1.5.0
CUDA9.1
CuDNN9.0
OpenCV??

Error message because of openCV as below.

C:\Anaconda3\lib\site-packages\cv2_init_.py in ()
2 import os
3
----> 4 from .cv2 import *
5 from .data import *
6

ImportError: DLL load failed:

My RL code does not converge when I do Transfer Learning and initialize the weights to "pretrained_model_weights.h5". Moreover: Even if I initialize to your final result "sample_model.json", the RL process ruins the good model and results in a bad driving agent. Both experiments were trained for 10 hours, and I sampled saved models from 3,5,8 hours as well.

I thought maybe the blame is on the random actions. Both "self.__run_airsim_epoch(always_random=True)" and "epsilon = 1" are hard-coded in distributed_agent.py This seems not a good parametrization for transfer learning. What are the right params for TL?

Thanks!

Hi, can you provide a feedback to following issue ?
I appreciate your help

Problem description

There is an early stopping of GPU processing activity during

Problem details

The issue raised when launching following training activity:
history = model.fit_generator(train_generator, steps_per_epoch=num_train_examples//batch_size, epochs=500, callbacks=callbacks,
validation_data=eval_generator, validation_steps=num_eval_examples//batch_size, verbose=2)
The output message is the following:
Epoch 32/500
Epoch 00032: val_loss did not improve

• 49s - loss: 0.0022 - val_loss: 3.7090e-04
  Epoch 00032: early stopping

This matches only 6% training activity:
Training 6% 32/500 [33:26<6:26:38, 49.57s/it]

Exepriment/Environment details

• Tutorial used: For example, AirSimE2EDeepLearning
• Environment used: DSVM with K80 GPU on NC6 Azure VM
• Versions of artifacts used (if applicable): *(For example, Python 3.6, Keras 2.1.2, Tensorflow backend

Hi all,

I have successfully run and prepared the data associated with the E2EDeepLearning tutorial. When I try to train the model, however, I get a AxisError (see below). Keras is insalled and all the python modules are up to date (I have run -installPackages.py). I use the TensorFlow backend. What can be the cause of this error?
Thanks in advance,

def draw_image_with_label(img, label, prediction=None):
theta = label * 0.69 #Steering range for the car is +- 40 degrees -> 0.69 radians
line_length = 50
line_thickness = 3
label_line_color = (255, 0, 0)
prediction_line_color = (0, 0, 255)
pil_image = image.array_to_img(img, K.image_data_format(), scale=True)
print('Actual Steering Angle = {0}'.format(label))
draw_image = pil_image.copy()
image_draw = ImageDraw.Draw(draw_image)
first_point = (int(img.shape[1]/2),img.shape[0])
second_point = (int((img.shape[1]/2) + (line_length * math.sin(theta))), int(img.shape[0] - (line_length * math.cos(theta))))
image_draw.line([first_point, second_point], fill=label_line_color, width=line_thickness)

print('Done')
if (prediction is not None):
    print('Predicted Steering Angle = {0}'.format(prediction))
    print('L1 Error: {0}'.format(abs(prediction-label)))
    theta = prediction * 0.69
    second_point = (int((img.shape[1]/2) + (line_length * math.sin(theta))), int(img.shape[0] - (line_length * math.cos(theta))))
    image_draw.line([first_point, second_point], fill=prediction_line_color, width=line_thickness)

del image_draw
plt.imshow(draw_image)
plt.show()

[sample_batch_train_data, sample_batch_test_data] = next(train_generator)
for i in range(0, 3, 1):
print('Done')
draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

AxisError Traceback (most recent call last)
in ()
25 plt.show()
26
---> 27 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
28 for i in range(0, 3, 1):
29 print('Done')

C:\Users\CHIOVERI\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
858
859 def next(self, *args, **kwargs):
--> 860 return self.next(*args, **kwargs)
861
862 def _get_batches_of_transformed_samples(self, index_array):

C:\Users\CHIOVERI\Documents\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

C:\Users\CHIOVERI\Documents\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

C:\Users\CHIOVERI\Documents\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

C:\Users\CHIOVERI\AppData\Local\Continuum\Anaconda3\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
177
178 def random_channel_shift(x, intensity, channel_axis=0):
--> 179 x = np.rollaxis(x, channel_axis, 0)
180 min_x, max_x = np.min(x), np.max(x)
181 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

C:\Users\CHIOVERI\AppData\Local\Continuum\Anaconda3\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1440 """
1441 n = a.ndim
-> 1442 axis = normalize_axis_index(axis, n)
1443 if start < 0:
1444 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

AirSimClient.py is missing from the repo. This will lead to an import error when attempting to run the model.

The fix is in this PR. This will be merged shortly, but if you are in a hurry, the branch E2EDL_Bugfixes can be pulled.

Hi. Great work! I'm really enjoying your simulator and cookbook.

In the train phase, although there are 500 epochs, mine converges after 27-39 epochs due to "no improvement".
Then in the test phase, the car do not keep track with the path and crashes very fast. Much less than the 5 minutes it should drive.
I did not change the code, except for the TQDM NotebookCallback I disabled.
Any ideas about what's going on?

Thanks!

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

I have finished the tutorial used the official dataset, now Iwant to add some test example to the dataset, I don‘t know how to collect the control data corresponding to the picture, Is there a tutorial? thanks a lot!’

Problem details

(Please describe your problem in as much detail as possible here. Make sure to include screenshots, code snippets, error messages, links and anything else you think will help us understand your problem better. If applicable, please also provide us a list of steps to reproduce your problem.)

Exepriment/Environment details

• Tutorial used: (For example, AirSimE2EDeepLearning, DistributedRL etc.)
• Environment used: (For example, landscape, city, hawaii etc.)
• Versions of artifacts used (if applicable): (For example, Python 3.5, Keras 2.1.2 etc.)

Problem description

I used DistributedRL Tutorial with random weights for two days, then I stop it and began to use Transfer learning weights, however, it seems cudnn is unreachable. I have cudnn5.1 and 6.0 for cund8.0, and cudnn7 for cudn 9.1

Problem details

Loading weights from my_model_weights.h5...
Current working dir is D:\AIProject\AutonomousDrivingCookbook\DistributedRL\Share\scripts_downpour\app
First layer:
[[[[ 0.03894855 -0.02513464 0.15417658 -0.0026583 -0.07323869 0.12281457
-0.07029789 0.15666577 -0.12889594 0.11235218 -0.12835188 -0.15529177
-0.12245906 0.15574838 -0.16965055 -0.00851028]
[-0.13678242 0.17681859 0.17137034 0.05008856 0.11469441 0.06297248
0.12309571 0.09143925 -0.16798416 0.16038734 0.1695745 0.13292642
-0.02244652 -0.04735599 -0.00631354 0.03638385]
[ 0.19165193 -0.14834613 0.17736149 0.18141983 0.04048967 -0.20928982
-0.11057959 0.06388518 -0.17496441 0.17916133 -0.01306503 -0.07610687
0.1388101 -0.08880593 -0.03391987 0.15310904]]

[[-0.03631234 0.04950815 -0.10022049 -0.11774249 0.15822944 0.06053431
-0.04932768 0.1228191 -0.05324809 -0.09266006 -0.05179078 -0.11411756
-0.095482 0.11878865 -0.08317431 0.13736732]
[-0.17175385 0.04252055 0.15489234 0.03725579 0.01006478 -0.15945505
-0.02473786 -0.15020007 -0.09039721 -0.16555463 0.13943027 0.0628512
0.15334439 0.11108618 -0.00479105 -0.10645811]
[ 0.07858201 0.03983586 -0.10747178 0.17793405 0.05197094 0.01532679
0.05628907 -0.01228614 0.0297354 -0.11679398 -0.15627411 -0.1907029
0.13904789 -0.0640917 0.02927678 -0.11179238]]

[[-0.14073054 0.12452096 -0.05107016 -0.06377813 -0.05155271 0.13970849
-0.18506047 0.02905693 0.05233993 -0.06130159 -0.07949064 -0.11921628
0.13336256 0.18380319 -0.14173897 -0.12382756]
[-0.09227199 -0.08912648 0.04024128 -0.16723497 0.03934036 -0.17860699
-0.12220516 -0.10621318 0.05502975 -0.09897196 -0.04708755 -0.01756219
-0.04116414 -0.11901203 0.13823269 0.15401933]
[ 0.19709682 -0.19583263 0.07211456 -0.09657288 0.12548693 -0.17089447
0.09015843 0.10009236 0.09117922 -0.05886658 0.19111662 -0.15089129
0.09723601 0.06179318 0.04969154 -0.02905231]]]

[[[-0.15161256 0.18108158 -0.08198252 0.03675875 0.08679089 -0.01822327
0.06029394 -0.01858982 0.10493858 -0.13273385 -0.01893125 -0.0892955
0.15530367 -0.12394772 -0.01735278 -0.09993808]
[-0.13090514 0.00258087 0.15755838 0.08856946 -0.14106721 -0.10148135
0.14753282 -0.0588728 0.07801725 0.18269575 -0.05899098 -0.13561122
-0.03079795 0.18038721 0.12587661 -0.04889482]
[ 0.1891939 0.16006617 -0.14898627 0.02304085 -0.11231866 0.1505771
0.09320717 0.14030428 -0.07523882 -0.13361876 -0.15507735 0.17406569
0.2038019 0.14741437 0.09891715 -0.00350525]]

[[-0.06804357 0.02128448 0.00749603 -0.01405669 0.08071361 -0.13800594
0.04556615 0.14130948 -0.02284213 0.15676719 -0.07210111 0.07653186
-0.08501238 0.05455744 0.02490745 -0.15323325]
[-0.15171303 -0.15090781 0.02632188 0.1974992 -0.16909063 0.06349774
0.00563027 0.00426664 -0.11086674 0.1002009 0.14993776 0.09362567
-0.03243918 -0.1058441 -0.17391473 -0.15979324]
[ 0.09868525 0.13697761 0.10584432 0.132819 -0.0173201 0.08517482
-0.09365354 0.03072184 -0.14881726 -0.01704379 0.0856963 0.01867709
-0.04958879 -0.02375238 0.08077671 -0.17336646]]

[[ 0.00752353 0.13349725 -0.16632664 -0.15007068 0.09292189 0.02270007
0.08131488 0.15506698 0.15182415 -0.12181804 0.00567346 -0.12908186
-0.02691863 0.15363601 -0.13808645 0.15727647]
[ 0.14065173 -0.15125008 -0.05247576 -0.00246365 -0.12311495 0.16167094
-0.02993548 -0.07256303 0.17449853 0.04090489 0.02223159 -0.05246314
-0.06843664 0.15503356 -0.04727919 -0.17540433]
[ 0.00120394 0.11387651 0.0409864 0.16292714 0.088181 -0.01079098
0.15381627 -0.0891685 0.15814507 -0.12307656 0.13868488 -0.09270314
0.00978417 -0.04605329 0.18279535 -0.13144042]]]

[[[-0.15878747 0.03906631 -0.14726698 -0.13384612 -0.17665805 -0.15925416
-0.00425105 0.02009274 0.09187473 -0.06346051 0.09910821 0.08797219
0.14174503 -0.11942405 0.07518984 0.17416692]
[ 0.05318895 -0.04195464 0.07309341 -0.08091445 0.08116957 -0.14057948
-0.14835921 -0.07002999 0.11191972 -0.12638785 -0.08042201 0.18678014
-0.1666165 -0.10577761 0.01899135 -0.05747586]
[ 0.14029655 0.07926279 -0.15504895 0.16135754 0.14789698 -0.14238183
-0.16055839 0.17460454 0.18023777 -0.0956251 -0.06245841 0.17527607
0.12980177 0.02224305 -0.01771989 -0.11362264]]

[[-0.01645875 0.17362995 0.04088786 -0.19339693 -0.13268843 -0.10124306
-0.03237866 -0.02942094 0.15809548 -0.03399407 0.08054227 0.01921189
0.02275016 0.15085441 -0.0945925 0.02212913]
[ 0.13483758 0.0636002 -0.00913755 0.05353547 0.06470659 -0.05704467
0.09975314 0.14212339 -0.13294634 0.04109536 0.08934512 -0.02663192
-0.13352115 -0.02337635 -0.00685774 0.18906754]
[-0.09998655 0.06588923 -0.01494381 -0.06959759 -0.0002833 0.05629796
0.00485297 0.09865069 -0.11832282 0.00457611 0.05286699 0.15218307
0.02567867 -0.01757005 0.18179703 -0.04168937]]

[[ 0.15269685 -0.10070352 -0.16917658 -0.01993535 -0.11679676 0.09519561
0.03552201 0.00036524 0.11301665 0.09974247 -0.19909017 0.16585428
0.15420395 -0.09478516 0.03247167 0.11814672]
[ 0.01920974 0.12406559 -0.0675544 -0.16710982 0.0430256 -0.03124766
0.16766725 0.13785729 -0.07411029 0.127984 0.13921042 -0.06976632
-0.14361116 -0.15726452 -0.05329617 -0.08604978]
[-0.01553447 -0.18177256 -0.12387755 0.0188439 -0.08041562 0.20500283
-0.1698696 -0.13808864 -0.16904303 0.1554161 0.05020312 0.0638795
-0.08212703 0.00350113 -0.08972123 0.13617031]]]]
Attempting to connect to AirSim (attempt 0)
WARNING:tornado.general:Connect error on fd 2068: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 2068: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 2068: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 2068: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 2068: WSAECONNREFUSED
Waiting for connection: Failed to connect.
Waiting a few seconds.
Attempting to connect to AirSim (attempt 1)
Waiting for connection:
Connected!
Filling replay memory...
Running Airsim Epoch.
Running AirSim epoch.
Getting Pose
Waiting for momentum to die
Resetting
Running car for a few seconds...
Start time: 2018-03-23 22:28:00.290143, end time: 2018-03-23 22:28:01.669712
Percent random actions: 1.0
Num total actions: 24
Replay memory now contains 24 members. (48.0% full)
Running Airsim Epoch.
Running AirSim epoch.
Getting Pose
Waiting for momentum to die
Resetting
Running car for a few seconds...
Start time: 2018-03-23 22:28:07.721129, end time: 2018-03-23 22:28:09.958872
Percent random actions: 1.0
Num total actions: 49
Replay memory now contains 50 members. (100.0% full)
Replay memory filled. Starting main loop...
Running Airsim Epoch.
Running AirSim epoch.
Getting Pose
Waiting for momentum to die
Resetting
Running car for a few seconds...
Start time: 2018-03-23 22:28:16.003218, end time: 2018-03-23 22:28:19.942759
Percent random actions: 1.0
Num total actions: 77
Generating 77 minibatches...
Sampling Experiences.
Publishing AirSim Epoch.
Publishing epoch data and getting latest model from parameter server...
START GET GRADIENT UPDATE DEBUG
2018-03-24 06:28:26.109428: E c:\l\work\tensorflow-1.1.0\tensorflow\stream_executor\cuda\cuda_dnn.cc:359] could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2018-03-24 06:28:26.114238: E c:\l\work\tensorflow-1.1.0\tensorflow\stream_executor\cuda\cuda_dnn.cc:366] error retrieving driver version: Unimplemented: kernel reported driver version not implemented on Windows
2018-03-24 06:28:26.122436: E c:\l\work\tensorflow-1.1.0\tensorflow\stream_executor\cuda\cuda_dnn.cc:326] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM
2018-03-24 06:28:26.130426: F c:\l\work\tensorflow-1.1.0\tensorflow\core\kernels\conv_ops.cc:659] Check failed: stream->parent()->GetConvolveAlgorithms(&algorithms)

Exepriment/Environment details

• Tutorial used: DistributedRL
• Environment used: local_run
• Versions of artifacts used (if applicable): Python 3.5, Keras 2.1.2

AxisError Traceback (most recent call last)
in ()
24 plt.show()
25
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 for i in range(0, 3, 1):
28 draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

~\Anaconda3\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
858
859 def next(self, *args, **kwargs):
--> 860 return self.next(*args, **kwargs)
861
862 def _get_batches_of_transformed_samples(self, index_array):

~\tmp\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

~\tmp\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

~\tmp\AutonomousDrivingCookbook-master\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

~\Anaconda3\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
177
178 def random_channel_shift(x, intensity, channel_axis=0):
--> 179 x = np.rollaxis(x, channel_axis, 0)
180 min_x, max_x = np.min(x), np.max(x)
181 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

~\Anaconda3\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1470 """
1471 n = a.ndim
-> 1472 axis = normalize_axis_index(axis, n)
1473 if start < 0:
1474 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

Which AirSim assets are you using?
https://github.com/Microsoft/AirSim/releases

I trained the model

Epoch 00024: val_loss improved from 0.00025 to 0.00024, saving model to model\models\model_model.24-0.0002403.h5
 - 57s - loss: 0.0017 - val_loss: 2.4029e-04

and downloaded v1.1.7 LandscapeMountains.zip.

The code connects to it, and handles the car.
However, in the beginning of the simulation, there is the road in the forest (different
from the training pics), so immediately after several meters my car goes off the road
and is stuck next to a tree.

Am I doing something wrong?

You leave some exercise questions. I tried to think about these. Is my thinking correct?

Thought Exercise 1.1 Try playing around with these parameters to see if you can get better results.
I think we could change Zero_Drop_Percentage to a smaller number as 0.85 to bring a little bit more data to model training.
Brigthen_Range could be bigger like 0.5 to make the model distinguish road and some other white environment stuff better.
I changed ROI value and found 76,135,0,255 is the optimized value.

Thought Exercise 2.1: As you might have noticed, the motion of the car is not very smooth on those bridges. Can you think of a reason why it is so? Can you use one of the techniques we described in Step 0 to fix this?
I did notice the car run badly on bridges. I did not come up with the reason.

Thought Exercise 2.2: The car seems to crash when it tries to climb one of those hills. Can you think of a reason why? How can you fix this? (Hint: You might want to take a look at what the car is seeing when it is making that ascent)
I see the car crashed the same place of a small hill all the time. I check the camera image at that time. I think the road looks too similiar with the rocks at that moment. We could change the brightness value when training model.

Problem description

Hi all, I tried to run "TrainModel" today. Everything went well until an error occurred showed that "axis 3 is out of bounds for array of dimension 3" when running to "[sample_batch_train_data, sample_batch_test_data] = next(train_generator)"

Problem details

I didn't make any change to the original file. It looks like that the issue is about next function in "image.py" file.
error

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Versions of artifacts used (if applicable): Python 3.6, Keras 2.1.5

Hi all, I am doing the E2E deep learning in Autonomous Cooking book. I like the "City" environment because of the urban scenes and some moving cars and people. But it is too large and can not be run smoothly on my PC. So I am wondering if I can use part of it, like fewer streets, buildings or cars, so that it can be run better? Thank you!

Problem description

[sample_batch_train_data, sample_batch_test_data] = next(train_generator)
this line in TrainModel results in an axis error, and I can't figure this out

Problem details

AxisError Traceback (most recent call last)
in ()
24 plt.show()
25
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 for i in range(0, 3, 1):
28 draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

E:\software\anaconda\lib\site-packages\keras\preprocessing\image.py in next(self, *args, **kwargs)
1067
1068 def next(self, *args, **kwargs):
-> 1069 return self.next(*args, **kwargs)
1070
1071 def _get_batches_of_transformed_samples(self, index_array):

E:\Projects\Autonomous driving\Code\jupyter\AutonomousDrivingCookbook\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

E:\Projects\Autonomous driving\Code\jupyter\AutonomousDrivingCookbook\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

E:\Projects\Autonomous driving\Code\jupyter\AutonomousDrivingCookbook\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

E:\software\anaconda\lib\site-packages\keras\preprocessing\image.py in random_channel_shift(x, intensity, channel_axis)
188
189 """
--> 190 x = np.rollaxis(x, channel_axis, 0)
191 min_x, max_x = np.min(x), np.max(x)
192 channel_images = [np.clip(x_channel + np.random.uniform(-intensity, intensity), min_x, max_x)

E:\software\anaconda\lib\site-packages\numpy\core\numeric.py in rollaxis(a, axis, start)
1440 """
1441 n = a.ndim
-> 1442 axis = normalize_axis_index(axis, n)
1443 if start < 0:
1444 start += n

AxisError: axis 3 is out of bounds for array of dimension 3

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: landscape

python35 cudnn5.1 keras2.1.2 numpy (1.14.2)
Traceback (most recent call last):
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 526, in get
value = obj._trait_values[self.name]
KeyError: 'layout'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 526, in get
value = obj._trait_values[self.name]
KeyError: 'kernel'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:/AutonomousDriving/AirSimE2EDeepLearning/test2.py", line 122, in
validation_data=eval_generator, validation_steps=num_eval_examples//batch_size, verbose=2)
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras\engine\training.py", line 2072, in fit_generator
callbacks.on_train_begin()
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras\callbacks.py", line 126, in on_train_begin
callback.on_train_begin(logs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras_tqdm\tqdm_callback.py", line 129, in on_train_begin
total=epochs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras_tqdm\tqdm_callback.py", line 67, in build_tqdm_outer
return self.tqdm(desc=desc, total=total, leave=self.leave_outer)
File "C:\Users\oleg\Anaconda3\lib\site-packages\keras_tqdm\tqdm_notebook_callback.py", line 33, in tqdm
return tqdm_notebook(desc=desc, total=total, leave=leave)
File "C:\Users\oleg\Anaconda3\lib\site-packages\tqdm_init_.py", line 22, in tqdm_notebook
return _tqdm_notebook(*args, **kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\tqdm_tqdm_notebook.py", line 176, in init
self.sp = self.status_printer(self.fp, self.total, self.desc)
File "C:\Users\oleg\Anaconda3\lib\site-packages\tqdm_tqdm_notebook.py", line 96, in status_printer
pbar = IntProgress(min=0, max=total)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget_int.py", line 57, in init
super(cls, self).init(**kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget_int.py", line 95, in init
super(_BoundedInt, self).init(**kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget_int.py", line 76, in init
super(_Int, self).init(**kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\domwidget.py", line 90, in init
super(DOMWidget, self).init(*pargs, **kwargs)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget.py", line 184, in init
self.open()
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget.py", line 197, in open
state, buffer_keys, buffers = self._split_state_buffers(self.get_state())
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget.py", line 291, in get_state
value = to_json(getattr(self, k), self)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 554, in get
return self.get(obj, cls)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 533, in get
value = self._validate(obj, dynamic_default())
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\domwidget.py", line 23, in _layout_default
return Layout()
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget.py", line 184, in init
self.open()
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipywidgets\widgets\widget.py", line 203, in open
self.comm = Comm(**args)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipykernel\comm\comm.py", line 56, in init
self.open(data=data, metadata=metadata, buffers=buffers)
File "C:\Users\oleg\Anaconda3\lib\site-packages\ipykernel\comm\comm.py", line 83, in open
comm_manager = getattr(self.kernel, 'comm_manager', None)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 554, in get
return self.get(obj, cls)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 533, in get
value = self._validate(obj, dynamic_default())
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 589, in _validate
value = self.validate(obj, value)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 1681, in validate
self.error(obj, value)
File "C:\Users\oleg\Anaconda3\lib\site-packages\traitlets\traitlets.py", line 1528, in error
raise TraitError(e)
traitlets.traitlets.TraitError: The 'kernel' trait of a Comm instance must be a Kernel, but a value of class 'NoneType' (i.e. None) was specified.

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

*I have finished learning the TrainModel.ipynb，and the results looks good, but when i tried to run the testModel.ipynb, I met this error as the following as @llyhbuaa,but it does not work when i deleted the modes and runned the TrainModel.ipynb again . I have connected to AirSim Simulator

Problem details

OSError Traceback (most recent call last)
in ()
----> 1 model = load_model(MODEL_PATH)
2
3 client = CarClient()
4 client.confirmConnection()
5 client.enableApiControl(True)

C:\ProgramData\Anaconda3\lib\site-packages\keras\models.py in load_model(filepath, custom_objects, compile)
232 return custom_objects[obj]
233 return obj
--> 234 with h5py.File(filepath, mode='r') as f:
235 # instantiate model
236 model_config = f.attrs.get('model_config')

C:\ProgramData\Anaconda3\lib\site-packages\h5py_hl\files.py in init(self, name, mode, driver, libver, userblock_size, swmr, **kwds)
267 with phil:
268 fapl = make_fapl(driver, libver, **kwds)
--> 269 fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
270
271 if swmr_support:

C:\ProgramData\Anaconda3\lib\site-packages\h5py_hl\files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
97 if swmr and swmr_support:
98 flags |= h5f.ACC_SWMR_READ
---> 99 fid = h5f.open(name, flags, fapl=fapl)
100 elif mode == 'r+':
101 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\h5f.pyx in h5py.h5f.open()

OSError: Unable to open file (unable to open file: name = 'D:\EndToEndDeepLearning\Model', errno = 13, error message = 'Permission denied', flags = 0, o_flags = 0)

Exepriment/Environment details

• Tutorial used: *(AirSimE2EDeepLearning)
• Environment used: (hawaii )
• Versions of artifacts used (if applicable): ( Python 3.6, Keras 2.1.2,cntk 2.5.1)

TypeError: bad operand type for unary -: 'str'

python version = 3.6
Numpy version = 1.14.3
scipy version = 1.10

Help me...

Can I know which version I used at the time of implementation?

TypeError Traceback (most recent call last)
in ()
24 plt.show()
25
---> 26 [sample_batch_train_data, sample_batch_test_data] = next(train_generator)
27 for i in range(0, 3, 2):
28 draw_image_with_label(sample_batch_train_data[0][i], sample_batch_test_data[i])

C:\ProgramData\Anaconda3\lib\site-packages\keras_preprocessing\image.py in next(self, *args, **kwargs)
1296
1297 def next(self, *args, **kwargs):
-> 1298 return self.next(*args, **kwargs)
1299
1300 def _get_batches_of_transformed_samples(self, index_array):

~\Desktop\TEST\AirSimE2EDeepLearning\Generator.py in next(self)
240 # so it can be done in parallel
241
--> 242 return self.__get_indexes(index_array)
243
244 def __get_indexes(self, index_array):

~\Desktop\TEST\AirSimE2EDeepLearning\Generator.py in __get_indexes(self, index_array)
262 x_images = x_images[self.roi[0]:self.roi[1], self.roi[2]:self.roi[3], :]
263
--> 264 transformed = self.image_data_generator.random_transform_with_states(x_images.astype(K.floatx()))
265 x_images = transformed[0]
266 is_horiz_flipped.append(transformed[1])

~\Desktop\TEST\AirSimE2EDeepLearning\Generator.py in random_transform_with_states(self, x, seed)
138 x = image.random_channel_shift(x,
139 self.channel_shift_range,
--> 140 img_channel_axis)
141 if self.horizontal_flip:
142 if np.random.random() < 0.5:

C:\ProgramData\Anaconda3\lib\site-packages\keras_preprocessing\image.py in random_channel_shift(x, intensity_range, channel_axis)
191 Numpy image tensor.
192 """
--> 193 intensity = np.random.uniform(-intensity_range, intensity_range)
194 return apply_channel_shift(x, intensity, channel_axis=channel_axis)
195

TypeError: bad operand type for unary -: 'str'

How to fix the error... Help me!!~

Because the shadow of trees in "Neighborhood" often causes confused, I tried to use the segmentation images recorded as training data set. However, the car cannot move properly during testing. I am wondering whether I can train it by segmentation images or not?

Problem description

What is the steering range of the default avatar car?

Problem details

In "AirSimE2EDeepLearning/TrainModel.ipynb" , I found a comment for the steering range is +- 40 degees.
(theta = label * 0.69 #Steering range for the car is +- 40 degrees -> 0.69 radians)
But in "TestModel.ipynb", the steering range is 0.5 radians(28.65 degrees). Which one is right?
(car_controls.steering = round(0.5 * float(model_output[0][0]), 2))
2.
Is the steering range of the default avatar car the same?
When I use "round(0.5 * float(model_output[0][0]), 2))" for testing, the result was very poor compared with "round(0.009 * float(model_output[0][0]), 2))" in Hawaii env.
3.
How can I get the details of avatar car ? Can I change the settings of the avatar car?
Is possible to use a custom car to run this project?
4.
The gear data is automatically changed according the speed data. Should we delete this column in 'airsim_rec.txt' since we don't need to predict the gear or use this data to predict?
5.
Using the record button, I can get the data from one camera in RGB mode. How to record the data from other cameras or other modes with record button?

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: hawaii
• Versions of artifacts used (if applicable): Python 3.6.5, Keras 2.1.2

I launch the train.bat for Reinforcement learning. The car may occasionally tumbling on the ground or rotating in air for each new epoch. I tried to fix this by modify the distributed_agent.py. In function __get_next_starting_point() I set "random_direction=(0,0,0)", but it doesn't work. What is the cause of this problem?

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning, DistributedRL
• Environment used: Neighborhood

/AutonomousDrivingCookbook-master/AirSimE2EDeepLearning/Generator.py", line 246, in __get_indexes
index_array = sorted(index_array)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I'm using Python 2.7.6
here is the index_array
(array([20087, 11112, 31001, 5888, 24707, 24742, 17835, 15747, 28776,
19486, 7640, 28383, 20912, 8023, 8257, 13928, 6409, 22015,
30856, 11651, 24858, 3563, 329, 21344, 2916, 12922, 24723,
9054, 12824, 16085, 3248, 12290]), 0, 32)
where did you define the index_array?
I can't find it

Hi, i red the code 'Cooking.py' and i have found a error.

def checkAndCreateDir(full_path):
    """Checks if a given path exists and if not, creates the needed directories.
            Inputs:
                full_path: path to be checked
    """
    if not os.path.exists(os.path.dirname(full_path)):
        try:
            os.makedirs(os.path.dirname(full_path))
        except OSError as exc:  # Guard against race condition
            if exc.errno != errno.EEXIST:
                raise

here line if exc.errno != errno.EEXIST: should be if exc.errno !=exc.errno.EEXIST:
is it?

Problem description

1) City environment steering problem

2) Deep Learning Learning Outcomes Optimization Problem

Problem details

_1) Run x360e.exe in .. \ AD_Cookbook_AirSim \ City \ CityEnviron \ Binaries \ Win64. Then, the steering is freely input in the game, and it is not operated. How can I solve this problem?

2. It runs well in a neighborhood environment. After learning the deep learning, the car moves very zigzag. Is there any way to optimize it for normal driving?_
   

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: *1) city, neighborhood *
• Versions of artifacts used (if applicable): Python 3.6, Keras 2.1.2, Cuda9.0, Xbox One controller, GTX960

Your issue may already be reported! Please make sure to search all open and closed issues before starting a new one.

Please fill out the sections below so we can understand your issue better and resolve it quickly.

Problem description

AD_Cookbook_Start_AirSim.ps1 crash

Problem details

When trying to launch AD_Cookbook_Start_AirSim.ps1 landscape the program crash with the following error message:
Fatal Error !
The UE4-JustAssets Game has crashed
When trying to launch AD_Cookbook_Start_AirSim.ps1 city the program crash with the following error message:
DX11 feature level 10.0 is required to run the engine

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: landscape, city
• System config:
  NC6 Azure VM with Tesla K80 NVIDIA GPU, Windows Server2016 Datacenter 10.0.14393 Build 14393
  K80 Driver Version: 22.21.13.8628
  DirectX: DirectX12

Great tutorial guys!

In the last section of LaunchLocalTrainingJob.ipynb , where you build the train.bat file, I think you're missing some parameters:
-local_run
-experiment_name

Also, if we run locally, why we need a batch file? can't we simply call app/distributed_agent directly?

Thanks!

Problem description

Starting AirSim via the provided script AD_Cookbook_Start_AirSim.ps1 with landscape as scenrio ends in a black and white screen.
Starting the other scenrios works fine

Exepriment/Environment details

• Tutorial used: AirSimE2EDeepLearning
• Environment used: landscape

I have tried to run tutorial (set brighten_range = 0.4, drop 0.9 data that label = 0) but find the result was not good like vehicle have collision on downhill path.
Then I have tried only drop 0.5 dataset which label = 0 and leave brighten_range = 0.4 , got better result than before. The vehicle was slip down when pass the shadow region so I though that maybe brighten_range could affect result. So I tried to run with brighten_range = 0, zero_drop_percentage = 0. I realized the vehicle runs on the left side of road and slip down when climbing. result worse.

Question:

1. For Brighten_Range parameter, how does this parameter affect the result? Why did you improve the brightness up to 40%?
   2.. ROI: [76,135,0,255], Zero_Drop_Percentage, Brighten_Range. Does these three parameters have some relations between each other? I want to try some different values in order to find the best parameters value.