Bitmap Manipulation

The unfortunate truth about Microsoft's very flexible and useful graphics library, GDI+, is that so often where it really matters is where it disappoints. It seems that things always come down to performance, and, unfortunately this a major weak point of GDI+. Depending on what you are doing it can be anywhere from slightly slow to painfully slow. When we need anything that even resembles real-time graphics, the first question that crosses our mind is can GDI+ handle it?  One of the worst bottlenecks in GDI+ is per-pixel image manipulation. The only easy way to perform this sort of manipulation is through the Bitmap.GetPixel and Bitmap.SetPixel functions. Unfortunately, not only do we incur the function call overhead, but there is also a conversion between raw image data and a System.Drawing.Color object. Not to mention, we get hit twice for each pixel: once to get the pixel data, and then again to set it. The only real way to do this kind of image manipulation in a timely fashion is to get right at the raw image data.

Prerequisites - Without understanding some core concepts it can be very hard to get a grasp on this sort of image manipulation.

• Basic understanding of image drawing and manipulation
• Intermediate understanding of binary math and hexadecimal
• Familiarity with arrays and .Net's numeric types

Contents

• The Steps
• Data Storage
• Image Locking
• Data Manipulation
• Image Unlocking
• What Else?
  • ReadOnly/WriteOnly
  • Palette Based
  • Advanced Layered Graphics
  • Transparency
  • Data Visualization
• What About Pointers?
• Code Samples
  • Pixel Struct
  • Filters
• From The Author

   

The Steps

• Data storage - The method we use will store the image data in an array.
• Image Locking - The image needs to be locked in order to access the image data.
• Data Manipulation - This is the heart of the operation.
• Image Unlocking - After we finish we need to copy image data back from our array, and unlock the image to finalize the process. 

The first thing you will want to do is add some using statements to the top of your code file. We will be using classes from System, System.Drawing, System.Drawing.Imaging, and System.Runtime.Interop.

Data Storage

ArrayLength = ImageStride * ImageHeight / ArrayTypeSize

ArrayLength = ImageWidth * ImageHeight

Image Locking

In reality we won't get to play with the original image data. When we lock an image we specify a pixel format and a region of the image that we want. GDI+ does a format conversion if necessary, and copies the image data. It is not until we unlock the image data that our changes are copied and reflected back to the original image. Locking image data entails only a single statement, albeit a lengthy one (Bitmap would be replaced with the name of the actual bitmap we are modifying)...

With the first argument, we pass a Rectangle that specifies the region of the image to lock, in this case, the entire image. The second argument specifies that we want to be able to read and write to the locked data. This option generally results in less optimized memory allocation than read-only or write-only. There may be times that you only want to read or write data, but for now we will copy the data (read), modify it, and copy it back (write) each time, so we will use ReadWrite. The third argument specifies that we want a 32-bit ARGB format

The object returned by LockBits, stored in our lockData variable, has information about the locked image data, including its stride, and most importantly, the memory location of the image data. Using the Marshal class, we can copy that data into an array. 

Data Manipulation

And now the fun part. This is where you do what you want to do. You can invert colors. You can brighten them or darken them. You can swap the color channels or adjust the color balance. But this is also the part where you need to understand binary math. In this tutorial we will perform a relatively simple operation: invert the colors. The trick is that when working with ARGB images, care must be taken to not invert the alpha channel or everything will disappear. 

Because we will need to filter color channels, we need to set up bit masks for different channels. Note that because of the way that C# handles negative values in hexadecimal, instead of entering a filter for alpha as we normally would, 0xFF000000, we must enter it as the negative hexadecimal value that 0xFF000000 would represent: -0x1000000.

Now, to invert the colors we must combine the inverted RGB values and the unmodified alpha values. For a simple filter that manipulates pixels individually, a simple for loop will suffice.

Image Unlocking

These last few steps will finalize our process, copying our changes back to the original image. First we must use the Marshal class to copy the data, and then call the Bitmap.Unlock function to unlock the image.

And we are done. Wrap it up in a function and here is our complete code listing:

public static void InvertColors(Bitmap bitmap) {
    // Lock the image for editing
    System.Drawing.Imaging.BitmapData lockData = bitmap.LockBits(
        new Rectangle(0, 0, bitmap.Width, bitmap.Height),
        System.Drawing.Imaging.ImageLockMode.ReadWrite,
        System.Drawing.Imaging.PixelFormat.Format32bppArgb);

    // Create an array to store image data
    int[] imageData = new int[bitmap.Width * bitmap.Height];
    // Use the Marshal class to copy image data
    Marshal.Copy(lockData.Scan0, imageData, 0, imageData.Length);

    for(int i = 0; i < imageData.Length; i++) {
        imageData[i] =
            (~imageData[i] & (int)Filter.rgb) // Invert value and extract RGB
           | imageData[i] & (int)Filter.alpha; // Combine with original alpha
    }

    // Copy image data back
    Marshal.Copy(imageData, 0, lockData.Scan0, imageData.Length);
    // Unlock image
    bitmap.UnlockBits(lockData);
}

What Else?

This, of course, is up to you. If you've tried to implement any kind of graphics filter using the GetPixel/SetPixel functions, you will definitely see the enormous speed increase that you can make by accessing image data in a more direct manner. In terms of graphics manipulation, this opens a huge number of operunities.

ReadOnly/WriteOnly

Sometimes you might want to read or write, but not both. As we noted before, you can lock images for read-only or write-only. Why would we bother? The advantage is that this allows memory to be allocated, accessed, and freed in the most efficient manner for a read-only or write-only operation. Suppose you maintain bitmap data in an array. You can manipulate the data, display the result in a Bitmap, manipulate it more, redisplay it, and so on without ever re-reading the data.

In other words, you can essentially store your bitmap in an Int32 array, and lock it as write-only when you need to display it (with increased efficiency over read/write access).

Palette Based

GDI+ is very poorly optimized in terms of paletted bitmaps. Besides that, Graphics objects can not be created on paletted bitmaps to manipulate them.

In fact, as hard as it might be to believe, you can write your own managed bit-block-transfer functions that take full advantage of useful features of paletted bitmaps and will they perform magnitudes of order faster than GDI+ blitting functions. Although some advanced techniques are required to perform this type of operation, it can be well worth the effort when working with palette-based graphics, especially multi-palette graphics (think NES ROMs).

Advanced Layered Graphics

GDI+ lacks certain advanced graphic blends that are very handy in graphic applications such as Adobe Photoshop. Graphic blends such as additive, subtractive, and binary blends such as AND and OR blends are very easy to create. More advanced blends such as hue/sat/lum mixes and multiply/screen become a realistic option.

Transparency

A feature very much lacking in any graphic editing software is the ability to manipulate the transparency level (alpha) as you would a color channel (or editing alpha based on certain color channels or other color information, for example, calculating alpha based on overall luminosity). By getting directly at the pixel data you can create filters or tools that manipulate transparency.

Data Visualization

By composing pixel data directly in an array you can more quickly and easily create images to visualize data. For example, you could display binary data with black and white pixels to represent ones and zeros (or a grayscale image composed of the values of individual bytes). You could also quickly render images where only raw image data is available.

The possibilities are almost endless. What you do is up to you. To help you along, there are some helpful tools at the bottom of this tutorial.

What About Pointers?

An alternative approach to per-pixel bitmap manipulation is using pointers (starting at the Scan0 property of the object returned by locking a bitmap) instead of copying the data to an array. So why does this tutorial focus on the array method instead?

To be certain, pointers are faster. In fact, the pointer equivalent of the function we wrote above will run approximately twice as fast. But that doesn't mean that pointers will always be twice as fast. Depending on circumstances, certain optimizations (such as avoiding re-allocating arrays and maintaining a cache of image data) can be applied to the array technique that can easily improve performance from 2 times as long as the pointer method to 1 1/3 times.

But that isn't the answer to the question "Why not use pointers." There are actually a few reasons why I prefer not to use pointers. The first reason is that arrays are easier on the ol' think-box, or at least easier on mine. A more important reason is that this is a more language neutral approach. Not only can this code easily be ported to any other .Net language, but most any VB user could actually follow this tutorial, deriving his own VB equivalent code, and make it work. The problem with pointers is that they are not part of most .Net languages.

Another problem with pointers is that they are "unsafe." There really isn't anything unsafe about them. Each and every time a pointer is dereferenced, the .Net runtime performs a check to ensure that the pointer points to locked value-type data. But "unsafe" code is "unverifiable." Simply put that means that a user needs to run the program in a full trust context, which isn't a guarantee.

Code Samples

Pixel Struct 

This struct can be assigned an Int32 or a UInt32 representing a 32-bit pixel, and the individual components can be read back (or vice-versa). The c++ style union behavior makes this struct a very fast way to easily access color channel data.

[StructLayout(LayoutKind.Explicit)]
public struct Pixel
{
    // These fields provide access to the individual
    // components (A, R, G, and B), or the data as
    // a whole in the form of a 32-bit integer
    // (signed or unsigned). Raw fields are used
    // instead of properties for performance considerations.
    [FieldOffset(0)]
    public int Int32;
    [FieldOffset(0)]
    public uint UInt32;
    [FieldOffset(0)]
    public byte Blue;
    [FieldOffset(1)]
    public byte Green;
    [FieldOffset(2)]
    public byte Red;
    [FieldOffset(3)]
    public byte Alpha;

    // Gets an initialized Pixel object so that the compiler
    // doesn't complain about an uninitialized object.
    public static Pixel getnew() {
        Pixel value;
        value.Int32 = 0;
        value.UInt32 = 0;
        value.Red  = 0;
        value.Green = 0;
        value.Blue = 0;
        value.Alpha = 0;

        return value;
    }

    // Gets a Pixel object initialized with data 
    // for a 32-bit pixel.
    public Pixel(int Val) {
        Red = 0;
        Green = 0;
        Blue = 0;
        Alpha = 0;
        UInt32 = 0;
        Int32 = Val;
    }

    // Gets a Pixel object initialized with data 
    // for a 32-bit pixel.
    public Pixel(uint Val) {
        Red = 0;
        Green = 0;
        Blue = 0;
        Alpha = 0;
        Int32 = 0;
        UInt32 = Val;
    }

    // Converts this object to/from a System.Drawing.Color object.
    public Color Color {
        get {
            return Color.FromArgb(Int32);
        }
        set {
            Int32 = Color.ToArgb();
        }
    }
}

Filters

This is a basic loop that can be run over an array of ints to apply a filter.

Pixel px = Pixel.getnew();
Pixel r = px;
for(int i = 0; i < imageData.Length; i++){
    px = imageData[i];
    // Use fileter code here
    imageData[i] = px;
}

The following are some simple filters that can be used on a bitmap by modifying a Pixel struct, px, for each pixel in the bitmap, and optionally, a secondary Pixel struct, r, for intermediate storage.

From The Author

I hope that you found this tutorial useful, accurate, and relatively comprehensive. Questions, comments, and criticisms are welcome. Just e-mail me at snarfblam@gmail.com (XTreme .Net Talk users can PM me at marble_eater). You are free to copy and distribute the code listed here, modified or unmodified, for commercial or personal use, with or without credit.