Image Segmentation Program Making

In this project, we will test machine learning models for image segmentation task. The problem of image segmentation can be formulated as a classification task, where every pixel in an image is mapped to an already known class C based on the information extracted from its N x N neighbors. The following figures illustrate an input example for the image segmentation task and its corresponding segmentation labels. A set of images and their ground-truth segmentation maps is provided to train and evaluate your models. An image is in principle a three-dimensional array of size H x W x 3, where H is the image height, W is the image width, and 3 corresponds to the three RGB channels. Each location in the 2d grid is called a pixel and each of its RGB channels has a value between 0 and 255 that represent the intensity of the color. For example, a pixel with value [255, 0, 0] corresponds to a red pixel, whereas a grey pixel will have a value [127, 127, 127]. For our application, each pixel in an image is labeled with a unique class C in [Grass = 3, Water = 2, Cow = 1, Others = 0] as specified in its segmentation map. Separate [login to view URL] and [login to view URL] files are provided to specify training and testing images respectively. For all tested model, the input will be an image patch of size 17 x 17 and the target will be the class label for the center pixel of the patch. You need to complete the following tasks: 1. Training Data Generation Create a balanced training set by randomly sampling image patches of size 17 x 17 from the training images and assigning the class label of the center pixel as a target for the image patch. Sample at Input image Ground-truth segmentation map least 5000 patches for each class (i.e., you will have a training set that contains at least 20000 examples.) Create another balanced validation set by sampling around 800 image patches per class from the training images. Note that the validation set and the training set are disjoint. For the test set, we will use all the image patches in the testing images. 2. Features extraction We will represent each image patch with a 192-dimensional features vector that corresponds to the intensity histograms for each of the RGB channels in the patch. For each of the RGB channels, create a histogram with 64 bins, where bin i contains the number of pixels with intensity value in the range [i*4, (i+1)*4), and 0  i  64. Normalize the histogram of each channel by dividing by the number of pixels in the patch. The final features vector is obtained by concatenating the normalized histogram for all the channels. 3. Baseline Model As a baseline model, evaluate a nearest neighbor baseline (with k=1). Report the pixel-wise accuracy on the test images. 4. Improvements Try to achieve better performance by evaluating two additional models on the task. Discuss and motivate your model selection, and comment on why the performance has improved (or potentially did not improve). If the chosen models have hyper-parameters, make sure to tune at least one hyper-parameter for each model using the validation set. To tune a parameter, you need to test at least three different values. For each model report the pixel-wise accuracy on the training, validation, and testing sets. Visualize the predictions for both models on the test images (same as the figure above). Examine the visualized predictions and check the wrongly predicted labels. Discuss any interesting observation you notice on the prediction errors.