Project 1: Making Bad Art

Due by: Friday, September 22, 2023 at 11:59 p.m.

Trouble in Albuquerque

Mr. White and Jesse have been hunting down their enemies. Saul got them a series of photographs of rival gang leaders, but, wait! Oh, no! Jesse just spilled a barrel of methylamine all over the photographs! (Who even has non-digital photographs these days, anyway?) The effects of the methylamine are unpredictable, and the photos are doing all kinds of strange things: stretching, dulling, brightening, saturating, blurring, sharpening, and losing their edges. As a computer science student, your chemistry skills are not up to par, but you can use MonoGame to fix the problem. Hurry, if they can't figure out who's going to kill them, Walter and Jesse are dead! (Do we want that to happen? Maybe! But on our terms, not some other gang's.)

The Mission

In order to reconstruct the images, we need you to write an MonoGame-based C# program that can read in an image file and perform a sequence of modifications consisting of seven operations, depending on which of the following nine commands (including save and quit) is given by the user.

User Command	Operation	Before	After
1	Resize the image to the new width and height specified by the user. Use bilinear interpolation to keep image quality reasonably high. Click here for more information. In the example, the user entered a new width of `200` and a new height of `100`.
2	Change the contrast of the image by the amount specified by the user. Click here for more information. In the example, the user entered a contrast change of `0.5`.
3	Change the brightness of the image by the amount specified by the user. Click here for more information. In the example, the user entered a brightness change of `1.75`.
4	Change the saturation of the image by the amount specified by the user. Click here for more information. In the example, the user entered a saturation change of `0.35`.
5	Blur the image with a 1-pixel blur radius using a filter technique. Click here for more information.
6	Sharpen the image with a 1-pixel sharpen radius using a filter technique. Click here for more information.
7	Detect edges using a filter technique. Click here for more information.
8	Save file (after modifications) to the user specified file name. Click here for more information.
9	Quit the program.

Specification

Obtaining a Console

You need a window containing MonoGame stuff in it, but you also need a console window to enter data. That is annoying. Use the directions discussed in class (or here) to get a console window to use.

Loading the File

When your program starts, it should prompt the user for a file to open. Sample output on the console should look like the following:

Enter file name:

To load the file, use the static Texture2D.FromStream() method, something like the following, where file is the name of the file:

image = Texture2D.FromStream(GraphicsDevice, new FileStream(file, FileMode.Open));

Note that your files should be in the bin\Windows\Debug folder of your project to be accessible to the Texture2D.FromStream() method without a path. (Or, if you change to Release mode, they need to be in the bin\Windows\Release folder.)

Note: For more information on the MonoGame API, visit here. Be sure to look for classes under the Microsoft.Xna.Framework.Graphics namespace.

Now, you should adjust the window so that it is the appropriate size to hold the image. To do this, set the PreferredBackBufferWidth and PreferredBackBufferHeight properties on the GraphicsDeviceManager object (mine is called graphics, but you might have a different name) to the image's width and height respectively. Then call ApplyChanges() on the same object. Note that you should ideally do this reading in the LoadContent() method and not in a method called by the console interaction thread.

With the image loaded into a texture, you should draw it on the screen every time you do a render operation in your program's main loop. The drawing code is just like what we have done in class:

Clear the back buffer to blue. (This step is actually unnecessary, but it will make finding mistakes easier.)
Begin sprite batch processing.
Draw the image.
End sprite batch processing.

Displaying a Menu

After opening the file, your program should run a loop, asking the user to input a command to alter the image. Legal operations are 1 through 7, with 8 saving the image into a file and 9 quitting.

Here is an example of sample output for a whole program:

Enter file name: bacon.jpg

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 1
Enter new width: 288
Enter new height: 205
Resize succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 2
Enter contrast change: 1.2
Contrast succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 3
Enter brightness change: 0.8
Brightness succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 4
Enter saturation change: 1.4
Saturation succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 5
Blur succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 6
Sharpen succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 8
Enter file name: edited.png
Saving file succeeded.

Image Editor

1. Resize
2. Contrast
3. Brightness
4. Saturation
5. Blur
6. Sharpen
7. Edge Detection
8. Save File
9. Quit

Enter command: 9

Quitting...

This sequence of operations resized the image to 288 × 205, increased the contrast by 1.2, decreased brightness by 0.8, increased saturation by 1.4, blurred the image, sharpened the image, saved it to edited.png, and then quit. You can find the original image here and the final image here. At each stage in the program's execution, the image being displayed should be updated to reflect the changes made.

Accessing Pixels

Once you have loaded an image into an Texture2D object, you'll need a way to access its actual pixel values. To do so, call the GetData() method on your texture object with a generic parameter of Color. You must pass an array of Color values into this method that is large enough to hold all the pixels in the texture.

The values in the array are copies of the pixel data in the texture. Once you have changed these values (or created another array to hold the changed values), you need to create a new Texture2D using an appropriate constructor to set the width, height, mipmap (use false), and surface format (use SurfaceFormat.Color).

With the texture you create this way, you can call the SetData() method, also with the generic parameter of Color to fill it with the values in your Color array.

Threading Problems

Unfortunately, you cannot directly change the values inside your main Texture2D object. You see, the thread that is updating and drawing the image might try to draw it while you're changing it, causing the program to crash. Instead, all the operations that create a new Texture2D object and fill it with the correct values are occurring on a different thread, the one that is running the console.

The most thread-safe solution is a little cumbersome, but there is a reasonable shortcut. Once you have loaded everything into a Texture2D object, store that object into a reference inside the game object which is initially null. (I call my reference replacement.) Whenever the Update() method runs, it should check to see if replacement is not null. If it isn't, there is a new image that should be drawn. In that case, it should call Dispose() on the old texture, set the old texture's reference to point at replacement, set replacement back to null, and then resize the window to the size of the new texture using the same methods as when the image was loaded.

Resizing an Image

Resizing an image is not difficult. First, you need to create a new texture to hold the resized image as well as an array of Color values that can hold the data with the new width and height.

Find the width ratio and the height ratio of the new image to the old image. Then, loop through all rows i and columns j in the new image. Find the pixel in the old image corresponding to the pixel in the new image. This pixel will generally not have an integer row and column. For example, imagine your old image is 500 × 300 and you want to resize it to be 200 × 400. From old to new, the width ratio is 2.5 and the height ratio is 0.75. Let's say that you are considering a pixel on row 7, column 9 in the new image. This corresponds to pixel on row 5.25, column 22.5 in the old image. Of course, that pixel doesn't exist. To approximate it, you do a weighted average of the pixels at row 5, column 22 with row 6, column 22 based on their distances from 5.25. Then you do a weighted average of the pixels at row 5, column 23 with row 6, column 23 based on their distances from 5.25. Then you do a weighted average of those two colors together, based on their distances from 22.5. Of course, you need to do each color component separately. Writing an interpolation utility function can be invaluable.

This kind of resizing or scaling works relatively well no matter how the dimensions of the new picture relate to the old. This is a form of bilinear interpolation. For more information, consult the Wikipedia article here, focusing on the Application in image processing section.

Changing the Contrast

Changing the contrast is one of the easiest operations and is a good place to start work. Legal contrast values are double values between 0 and 2, inclusive. Change the color components for every pixel in the image. Given a color component, adjust its contrast by multiplying it by the contrast value, subtracting the contrast value multiplied by 128, and adding 128. Afterwards, make sure that you clamp the color component to the range [0, 255], since some contrast values could push some color components below 0 or above 255.

As you can see, a contrast value of 0 would set everything to 128. A contrast value of 2 would push everything less than or equal to 128 to 0 and everything greater than 128 to 255.

Changing the Brightness

Changing the brightness is even easier than contrast. Legal brightness values are double values between 0 and infinity, although 2 or 3 is a practical place to stop. Change the color components for every pixel in the image. Given a color component, adjust its brightness by multiplying it by the brightness value. Afterwards, make sure that you clamp the color component to the range [0, 255], since some brightness values could push some color components above 255.

Changing the Saturation

Legal saturation values are between 0 and infinity, although 2 is a practical place to stop in most cases. Change the RGB values for every pixel in the image. Given an RGB color value, convert it to the HSV color space using the equations given in lecture. Once you have the HSV representation, multiply the S component in the result by the saturation value. Make sure that the new S is between 0 and 1. Then, convert back from the HSV color space to the RGB color space and update the value of the pixel.

Applying a Convolution Filter

To blur, sharpen, or perform edge detection, you should apply a convolution filter to the image. A convolution filter uses an n × n matrix called a kernel. For this project, all of our kernels will be 3 × 3. Each element of the kernel has a numerical weight. The weight at (1,1), the center, is the weight that you will multiply the color component of the current pixel by. The weight at (1,0) is for the pixel immediately to the left of the current pixel. The weight at (0,1) is for the pixel immediately above the current pixel. The weight at (2,2) is for the pixel immediately below and to the right of the current pixel, and so on. You go through the matrix, adding up the color component in question for each pixel, multiplied by the appropriate weight. If we are looking at a pixel on row x, column y, then the weight in matrix (i,j) corresponding to a pixel at row x + i - 1, column y + j - 1. Once you have added up the component values for all the pixels surrounding the current pixel, you divide them by some normalizing value, which is usually the sum of all the weights in the matrix. You may also add some bias value to the component after the weighted sum and normalization. Finally, you clamp the value to the range [0, 255].

The idea of a filter is to make each pixel the weighted sum of the pixels around it. If you give positive weights to the pixels around the pixel in question, you will give a blur effect. If you give negative weights to the pixels around the pixel in question, you will get a sharpen effect. Other kernels can give more exotic effects. This page gives a reasonable explanation of how filters are used in the open source image editing product GIMP.

I highly recommend that you write a filter method that you call when doing blur, sharpen, and edge detection. A few additional practical matters: You must create a new array to store Color values when applying a filter. Otherwise, changing pixels in the original array will affect other pixels. Note that border pixels (those forming the top, bottom, left, and right edges) are not processed by the filter and should be left unchanged. We are limited in the effects we can get with a 3 × 3 kernel. However, note that you have to run through the dimensions of the kernel for every single pixel. Thus, a 15 × 15 kernel requires 225 steps for every single pixel. Applying convolution filters can become computationally demanding.

Blurring the Image

To blur the image, apply the following convolution kernel:

1	2	1
2	4	2
1	2	1

Normalize the final result by dividing by 16 (the sum of the weights).

Sharpening the Image

To sharpen the image, apply the following convolution kernel:

0	-1	0
-1	5	-1
0	-1	0

No normalization is needed because the sum of the weights is 1.

Edge Detection

To perform edge detection on the image, apply the following convolution kernel:

-1	-1	-1
-1	8	-1
-1	-1	-1

Note that the sum of the weights is 0. However, don't divide by 0! Do not normalize the value. Instead, add a bias of 128 to each component value after finding the weighted sum. As always, make sure you clamp to [0, 255].

Saving the File

Read in the destination file name from the user. Then, create a new FileStream object with the name of the file. Call the SaveAsPng() method on your Texture2D object on the stream you create. There is also a SaveAsJpeg() method you might be tempted to call, but it appears to be unsupported in the version of MonoGame we're using. For that reason, do not supply a file name with an ending other than .png for the new picture.

Quitting the Program

When the user picks the quit option, call Exit() on the game object to quit the program.

Image Files

The following image files are provided for your use. Of course, you should feel free to download other images from the Internet for testing. Any image file with any of the supported extensions listed above should work.

Turn In

Your solution and project should both be called Project1. Zip up your entire project and solution and upload the zip file into Blackboard. All work must be submitted before Friday, September 22, 2023 at 11:59 p.m. unless you are going to use a grace day.

You should clean your solution before submitting it. I should be able to open your solution and run it without any compilation problems.

All work must be done within assigned teams. You may discuss general concepts with your classmates, but it is never acceptable for you to look at another teams' code. Please refer to the course policies if you have any questions about academic integrity. If you have trouble with the assignment, I am always available for assistance.

Grading

Your grade will be determined by the following categories:

Category	Weight
Resizing	15%
Changing contrast	10%
Changing brightness	10%
Changing saturation	15%
Blurring the image	10%
Sharpening the image	10%
Edge detection	10%
Loading and saving the file	10%
Menu and output formatting	5%
Style and comments	5%

Under no circumstances should any member of one group look at the code written by another group. Tools will be used to detect code similarity automatically.

Code that does not compile will automatically score zero points.