Using bitwise operators_问答_开发者_运维开发者技术经验分享

I've been studying C# and ran accross some familiar ground from my old work in C++. I never understood the reason for bitwise operators in a real application. I've never used them and have never had in a reason to use them. I've been studying how they work; the example below shows the shift bitwise operator. What is the point of bitwise operators, their use and how they work?

Maybe I'm missing something in bitwise logic.

byte bitCom开发者_如何学Cp = 15;              // bitComp = 15 = 00001111b
byte bresult = (byte) ~bitComp; // bresult = 240 = 11110000b

Here's an example for the ~complement bitwise operator:

byte bitComp = 15;              // bitComp = 15 = 00001111b
byte bresult = (byte) ~bitComp; // bresult = 240 = 11110000b

A typical use is manipulating bits that represent mutually exclusive 'flags'.

Example from MSDN: Enumeration Types

[Flags]
enum Days2
{
    None = 0x0,
    Sunday = 0x1,
    Monday = 0x2,
    Tuesday = 0x4,
    Wednesday = 0x8,
    Thursday = 0x10,
    Friday = 0x20,
    Saturday = 0x40
}

class MyClass
{
    Days2 meetingDays = Days2.Tuesday | Days2.Thursday;

    Days2 notWednesday = ~(Days2.Wednesday);
}

See also Stack Overflow question Most common C# bitwise operations.

Here's an everyday bitwise-op trick not many people have discovered:

When you have an enumerated type representing a bitfield, you need to define each enum entry as a distinct bit value, as in:

enum
{
    Option1 = 1,
    Option2 = 2,
    Option3 = 4,
    Option4 = 8,
    Option5 = 16
};

but it's easy to forget that the next item in the sequence needs to be double the last number. Using bit shifting, it makes the sequence much easier to get right:

enum
{
    Option1 = 1<<0,
    Option2 = 1<<1,
    Option3 = 1<<2,
    Option4 = 1<<3,
    Option5 = 1<<4
};

Another typical (but I think less common) usage is to compose several numbers into one big number. An example for this can be the windows RGB macro:

#define RGB(r, g ,b)  ((DWORD) (((BYTE) (r) | ((WORD) (g) << 8)) | (((DWORD) (BYTE) (b)) << 16)))

Where you take 3 bytes and compose an integer from them the represent the RGB value.

Except for combining flags, bit logic isn't necessarily something you need in your UI code, but it is still tremendously important. For example, I maintain a binary serialization library, that needs to deal with all sorts of complex bit-packing strategies (variant length base-128 integer encoding, for example). This is one of the implementations (actually, this is a slower/safer version - there are other variants for dealing with buffered data, but they are harder to follow):

    public static bool TryDecodeUInt32(Stream source, out uint value)
    {
        if (source == null) throw new ArgumentNullException("source");

        int b = source.ReadByte();
        if (b < 0)
        {
            value = 0;
            return false;
        }

        if ((b & 0x80) == 0)
        {
            // single-byte
            value = (uint) b;
            return true;
        }

        int shift = 7;

        value = (uint)(b & 0x7F);
        bool keepGoing;
        int i = 0;
        do
        {
            b = source.ReadByte();
            if (b < 0) throw new EndOfStreamException();
            i++;
            keepGoing = (b & 0x80) != 0;
            value |= ((uint)(b & 0x7F)) << shift;
            shift += 7;
        } while (keepGoing && i < 4);
        if (keepGoing && i == 4)
        {
            throw new OverflowException();
        }
        return true;
    }

We have:

tests to see if the most-significant-bit is set (this changes the meaning of the data)
shifts-a-plenty
removal of the most-significant-bit
bitwise combination of values

This is real code, used in a real (and much used) protocol. In general, bit operations are used a lot in any kind of encoding layer.

It is also hugely important in graphics programming, for example. And lots of others.

There are also some micro-optimisations (maths intensive work etc) that can be done with bit operations.

An example from COM programming:

An HRESULT is an error code consisting of a 32 bit integer. The high bit is a flag indicating whether the code represents success (0) or failure (1). The next 15 bits are an integer representing what sort of error it is -- an ole automation error or a win32 error or whatever. The lower 16 bits are the actual error code.

Being able to shift bits around is quite useful when you want to get information into or out of an HRESULT.

Now, you almost always want to abstract away the bit twiddling. It's much better to have a method (or in C, a macro) that tells you whether the HRESULT is failure, rather than actually twiddling out the bit with (hr & 0x80000000) != 0 right there in your source code. Let the compiler inline it.

There are lots of examples of low-level data structures where information is crammed into words and needs to be extracted with bitwise operations.

Three major uses off of the top of my head:

1) In embedded applications, you often have to access memory-mapped registers, of which individual bits mean certain things (for instance the update bit in an ADC or serial register). This is more relevant to C++ than C#.

2) Calculations of checksums, like CRCs. These use shifts and masks very heavily. Before anybody says "use a standard library", I have come across non-standard checksums too many times, which have had to implemented from scratch.

3) When dealing with data which comes from another platform, which have a diffent bit or byte order (or both) from the one you are executing your code on. This is particularly true when doing software testing of embedded systems, receiving data across a network which has not been converted to network order, or processing bulk data from a data capture system. Check out the Wikipedia article on Endianness. If you are really interested, read the classic article On Holy Wars and a Call for Peace" by Danny Cohen.

A couple of examples:

Communication stacks: a header attached to data in a layer of a communication stack may contain bytes where individual bits within those bytes signify something, and so have to be masked before they can be processed. Similarly, when assembling the header in the response, individual bits will then need to be set or cleared.
Embedded software: embedded microcontrollers can have tens or hundreds of hardware registers, in which individual bits (or collections thereof) control different functions within the chip, or indicate the status of parts of the hardware.

Incidentally, in C and C++, bitfields are not recommended where portability is important, as the order of bits in a bitfield is compiler-dependent. Using masks instead guarantees which bit(s) will be set or cleared.

As to 'how they work': bitwise operations are one of the lowest level operations CPUs support, and in fact some bitwise operations, like NAND and NOR, are universal - you can build any operation at all out of a sufficiently large set of NAND gates. This may seem academic, but if you look at how things like adders are implemented in hardware, that's often basically what they boil down to.

As to the 'point': in a lot of higher level applications, of course, there is not much use for bit ops, but at the lowest levels of a system they are incredibly important. Just off the top of my head, things that would be very difficult to write without bit operations include device drivers, cryptographic software, error correction systems like RAID5 or erasure codes, checksums like CRC, video decoding software, memory allocators, or compression software.

They are also useful for maintaining large sets of integers efficiently, for instance in the fd_set used by the common select syscall, or when solving certain search/optimization problems.

Take a look at the source of an MPEG4 decoder, cryptography library, or operating system kernel sometime and you'll see many, many examples of bit operations being used.

Left and right shift operators (<< and >>) are often used in performance critical applications which do arithmetic operations and more specifically multiplications and divisions by powers of two.

For example suppose you had to calculate the mathematical expression 5*2^7. A naive implementation would be:

int result = 5 * (int)Math.Pow(2, 7);

Using left shift operator you could write:

int result = 5 << 7;

The second expression will be orders of magnitude faster than the first and yet yielding the same result.

Minimizing Memory Use

Naturally a very generalized reason is to cram a lot of data into a small amount of memory. If you consider an array of booleans like this:

bool data[64] = {...};

That can take 64 bytes (512 bits) of memory. Meanwhile the same idea can be represented with bits using 8 bytes (64-bits) of memory:

uint64_t data = ...;

And of course we have a boatload of DRAM these days so it might not seem like it'd matter to compact all this data into the minimum size, but we're still dealing with, say, 64-bit general purpose registers. We're still dealing with 64 byte cache lines, e.g., and kilobytes per physically-mapped page, and moving data down the memory hierarchy is expensive. So if you're processing a boatload of data sequentially, for example, and you can reduce that down to 1/8th its size, often you'll be able to process a whole lot more of it in a shorter amount of time.

So a common use of the analogy above to store a bunch of booleans in a small amount of space is when bit flags are involved, like this:

enum Flags
{
     flag_selected =  1 << 0,
     flag_hidden =    1 << 1,
     flag_removed =   1 << 2,
     flag_hovering =  1 << 3,
     flag_minimized = 1 << 4,
     ...
};
uint8_t flags = flag_selected | flag_hovering;

Operating on Multiple Bits at Once

But on top of cramming all this data into a smaller amount of space, you can also do things like test for multiple bits simultaneously:

// Check if the element is hidden or removed.
if (flags & (flag_hidden | flag_removed))
{
    ...
}

And a smart optimizer will typically reduce that down to a single bitwise and if flag_hidden and flag_removed are literal constants known at compile-time.

As another example, let's go back to the example above:

bool data[64];

Let's say you wanted to test if all 64 booleans are set in which case you do something different. Given this type of representation, we might have to do this:

bool all_set = true;
for (int j=0; j < 64; ++j)
{
     if (!data[j])
     {
          all_set = false;
          break;
     }
}
if (all_set)
{
     // Do something different when all booleans are set.
     ...
}

And that's pretty expensive when the bitwise representation allows us to do this:

uint64_t data = ...;
if (data == 0xffffffffffffffff)
{
     // Do something different when all bits are set.
     ...
}

This above version can perform the check for all 64 bits set in a single instruction on 64-bit machines. With SIMD registers, you can even test for more than 64 bits at a time with a single SIMD instruction.

As another example let's say you want to count how many of those booleans are set. In that case you might have to do this working with the boolean representation:

int count = 0;
for (int j=0; j < 64; ++j)
     count += data[j];
// do something with count

Meanwhile if you used bitwise operations, you can do this:

uint64_t data = ...;
const int count =  __popcnt64(data);
// do something with count

And some hardware can do that very efficiently as a native instruction. Others can still do it a whole, whole lot faster than looping through 64 booleans and counting the booleans set to true.

Efficient Arithmetic

Another common one is efficient arithmetic. If you have something like:

x = pow(2, n);

Where n is a runtime variable, then you can often get a much more efficient result doing:

x = 1 << n;

Of course an optimizing compiler using intrinsics for pow might be able to translate the former into the latter, but at least C and C++ compilers I've checked as of late cannot perform this optimization, at least when n is not known at compile-time.

Whenever you're working with power of two, you can often do a lot of things efficiently with bitshifts and other bitwise operations. For example, take this:

x = n % power_of_two;

... where power_of_two is a runtime variable that is always a power of two. In that case you can do:

x = n & (power_of_two - 1);

Which has the same effect as the modulo (only for power of two numbers). It's because a power of two value will always be a set bit followed by zeros. For example, 16 will be 0b10000. If you subtract one from that, it becomes: 0b1111, and using a bitwise and with that will effectively clear all upper bits in a way that gives you the analogical equivalent of n % 16.

Similar thing with left-shifts to multiply by a power of two, right-shifts to divide by a power of two, etc. One of the main reasons a lot of hardware favored power of two image sizes like 16x16, 32x32, 64x64, 256x256, etc. is due to the efficient arithmetic enabled by it using bitwise instructions.

Conclusion

So anyway, this is a brief introduction to what you can do with bitwise operations and instructions from fast arithmetic, reduced memory use, and being able to perform operations on potentially many bits at once without looping through them and operating on them one bit at a time.

And they're still very relevant today in performance-critical fields. For example, if you look at the Atomontage voxel rendering engine, it claims to be able to rep a voxel in just about a single bit, and that's important not just to fit huge voxel data in DRAM but also to render it really quickly from smaller, fast memory like registers. Naturally it can't do that if it's going to use 8 bits just to store a true/false kind of value which has to be checked individually.

I work in motion control (among other things) and the way you communicate with the drives is usually by using bit sequences. You set one bit pattern in a memory location x to set the motion profile, you set an enable bit at memory location y to start the motion, read a pattern from location z to get the status of the move, etc. The lower you go the more bit twiddling you have to perform.

There is a reason that, depending on the kind of person, can be important: cool-iness! ;)

Take this, an algorithm to calculate the absolute value of a number, without using any conditional branch instruction:

int abs(const int input) {
   int temp = A >> 31;
   return ( input ^ A ) - A;
}

A reason to avoid conditional branching is that it could stop your processor pre-fetching, waiting for the condition to be verified to know to which value the program counter should be set.

So, a part from the joke, there are very good technical reasons to do it.