Also - ground truth is not fed back at any time in the notebook (even when data is available - and it is available in both cases in the 2. notebook).

The correct code should be:

def show_keypoints(image, key_pts, ground_pts):
    """Show image with keypoints"""
    # image is grayscale
    plt.imshow(image, cmap='gray')
    plt.scatter(key_pts[:, 0], key_pts[:, 1], s=20, marker='.', c='m')
    if ground_pts is not None:
        plt.scatter(ground_pts[:, 0], ground_pts[:, 1], s=20, marker='.', c='g')

# visualize the output
def visualize_output(test_images, test_outputs, gt_pts=None, batch_size=10):

    for i in range(batch_size):
        plt.figure(figsize=(50,50))
        ax = plt.subplot(1, batch_size, i+1)

        # un-transform the image data
        image = test_images[i].data              # get the image from it's Variable wrapper
        image = image.numpy()
        image = np.transpose(image, (1, 2, 0))   # transpose to go from torch to numpy image

        # un-transform the key_pts data
        key_pts = test_outputs[i].data
        key_pts = key_pts.numpy()
        key_pts = key_pts*50.0+100               # undo normalization of keypoints 

        # un-transform the gt_pts data
        ground_pts = None
        if gt_pts is not None:
            ground_pts = gt_pts[i].data.numpy()            
            ground_pts = ground_pts*50.0+100

        show_keypoints(np.squeeze(image), key_pts, ground_pts)

        plt.axis('off')

    plt.show()

# call it
visualize_output(test_images, test_outputs, gt_pts)

In the block

# Display a few of the images from the dataset
num_to_display = 3

for i in range(num_to_display):
    
    # define the size of images
    fig = plt.figure(figsize=(20,10))
    
    # randomly select a sample
    rand_i = np.random.randint(0, len(face_dataset))
    sample = face_dataset[rand_i]

    # print the shape of the image and keypoints
    print(i, sample['image'].shape, sample['keypoints'].shape)

    ax = plt.subplot(1, num_to_display, i + 1)
    ax.set_title('Sample #{}'.format(i))
    
    # Using the same display function, defined earlier
    show_keypoints(sample['image'], sample['keypoints'])

we need to add plt.show(). Otherwise these samples show later on with the transforms examples.

From Notebook:
~/Applications/anaconda3/envs/ai/lib/python3.6/site-packages/torch/utils/data/dataloader.py in handler(signum, frame)
173 # This following call uses waitid with WNOHANG from C side. Therefore,
174 # Python can still get and update the process status successfully.
--> 175 _error_if_any_worker_fails()
176 if previous_handler is not None:
177 previous_handler(signum, frame)

RuntimeError: DataLoader worker (pid 4134) is killed by signal: Unknown signal: 0.

In the console:

[I 22:49:58.099 NotebookApp] Saving file at /2. Define the Network Architecture.ipynb
ERROR: Unexpected segmentation fault encountered in worker.
ERROR: Unexpected segmentation fault encountered in worker.
ERROR: Unexpected segmentation fault encountered in worker.
ERROR: Unexpected segmentation fault encountered in worker.
[I 23:05:58.134 NotebookApp] Saving file at /2. Define the Network Architecture.ipynb

(1) Section: "Load and Visualize Data" notebook 1. last sentence says "fraes", should be "frames"
(2) Section: "Dataset class" section notebook 1. "mormalizing images...", should be "normalizing"..
(3) Section: first paragraph of notebook 2. "..data your working with...", should be "you're"
(4) Section: Apply the model on a test sample notebook 2, "1. Extract the image and groud truth keypoints from a sample", should be 'ground'
(5) Section: Training and Initial Observation notebook 2, "..you're encouraged to start of with", should be "start off"
(6) Section: Feature maps notebook 2 "...each of these ernesl detects...", should be "kernels"
(7) Section: Face and Facial Keypoint detection notebook 3 "The neural network expectes a Tensor ", should be "expects"

These issues are obviously minor, but I think they are easy to fix and make the project look more professional and gives the student a sense that the Udacity team really went through this project thoroughly before giving it to them.

Instead of:
optimizer.zero_grad()

I think it's way better to use:
net.zero_grad()

Because they are only the same, if optimizer has all params of net, just to be on the safe side.

The notebook 'Define the Network Architecture' does not call eval() before passing test data to the model. Evalution mode should be enabled to disable dropout effects.

The training/testing images are incomplete. Can you, please, update the training/testing dataset or provide any other alternative sources?

It says

batch_i % 10 == 9

Instead of

batch_i % batch_size == batch_size - 1 

Hello, the dataset is incomplete, in the training folder there are, file count: 2308 images, but in the training_frames_keypoints.csv file there are references, Number of images: 3462, you can share the group of correct images for training and testing. thank you.

Hello everyone

I´m trying to import open cv using Jupyter Notebook, from Anaconda 3 but when i run the command "import cv2" i get this error "ImportError: No module named cv2".

The strange thing is that when i run the same comand on Spyder(From Anaconda 3 too) dont have any problem. And I need use specifically Jupyter notebook

What should I do in order to make this work?

when following your installation instructions I am failing at the point where I shoud do
(cv-nd) xxxxx@yyyyy:/zzzzzzzzzzzzzz/P1_Facial_Keypoints$ pip install -r requirements.txt
ERROR: Could not find a version that satisfies the requirement opencv-python==3.2.0.6 (from versions: 3.2.0.8, 3.3.0.10, 3.3.1.11, 3.4.0.12, 3.4.0.14, 3.4.1.15, 3.4.2.16, 3.4.2.17, 3.4.3.18, 3.4.4.19, 3.4.5.20, 3.4.6.27, 3.4.7.28, 3.4.8.29, 3.4.9.31, 3.4.9.33, 3.4.10.35, 3.4.10.37, 3.4.11.39, 3.4.11.41, 3.4.11.43, 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.14.53, 3.4.15.55, 3.4.16.57, 3.4.16.59, 3.4.17.61, 3.4.17.63, 3.4.18.65, 4.0.0.21, 4.0.1.23, 4.0.1.24, 4.1.0.25, 4.1.1.26, 4.1.2.30, 4.2.0.32, 4.2.0.34, 4.3.0.36, 4.3.0.38, 4.4.0.40, 4.4.0.42, 4.4.0.44, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.2.54, 4.5.3.56, 4.5.4.58, 4.5.4.60, 4.5.5.62, 4.5.5.64, 4.6.0.66, 4.7.0.68, 4.7.0.72, 4.8.0.74, 4.8.0.76)
ERROR: No matching distribution found for opencv-python==3.2.0.6

can you please help?

In def visualize_output(test_images, test_outputs, gt_pts=None, batch_size=10):

    # un-transform the key_pts data
    key_pts = test_outputs.data

should be correctly:

    # un-transform the key_pts data
    key_pts = test_outputs[i].data

When I ran first cell on Notebook 1, I got the following error:

ModuleNotFoundError: No module named 'cv2'

But when I enter python cli within the anaconda env, import cv2 works just fine.

Any idea?

Notebook 2:

Sometimes we can override the test output and wants to recover a trained model. Maybe this line could be commented in the cell:

# net = torch.load('saved_models/keypoints_model_2.pt')

Notebook 3, 2nd code block:

should be (take out spaces around optional argument label):
fig = plt.figure(figsize=(9,9))

It says:

Install PyTorch and torchvision.

Linux or Mac:
conda install pytorch torchvision -c pytorch 

But Windows is missing.

In function def train_net(n_epochs), the average loss print divides by 1000 which is incorrect. Should be divided by 10.

@sunsided Hi, I have a question about your repository facial key point detection, when I was trying to follow your steps
conda env create -f environment.yaml
I got an error

ResolvePackageNotFound:

• nbconvert==5.3.1=py_1
  i'm using ubuntu 18
  python3.6

mean = 100, sqrt = 50, so, pts should be (pts - 100)/50
key_pts_copy = (key_pts_copy - 100)/50.0

Why mean is taken as 100 and std is taken as 50 .What is the reason behind scaling?

In train_net, flattening of key points are done like this:
key_pts = key_pts.view(1, -1)

I think it would be more correct to do it like that:
key_pts = key_pts.view(key_pts.size(0), -1)

In the notebook 2 'Test Data' section:

Couple things :
(1) Confusing that the visualize_output() method is commented out with no guidance on whether the student should uncomment this out or what to do with this. Honestly looks odd just hanging there with no instruction or even not having that method and just having something like 'TODO - visualize output'
(2) In the second box (in the commented out code) you see second argument to visualize_output if test_outputs_new but there is no variable called that. So, either the above box we need to rename test_outputs to test_outputs_new or visa versa