Archive for the ‘ Languages ’ Category

Bitwise Operators – JavaScript | MDN

https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Operators/Bitwise_Operators

Example: Flags and bitmasks

The bitwise logical operators are often used to create, manipulate, and read sequences of flags, which are like binary variables. Variables could be used instead of these sequences, but binary flags take much less memory (by a factor of 32).

Suppose there are 4 flags:

  • flag A: we have an ant problem
  • flag B: we own a bat
  • flag C: we own a cat
  • flag D: we own a duck

These flags are represented by a sequence of bits: DCBA. When a flag is set, it has a value of 1. When a flag is cleared, it has a value of 0. Suppose a variable flags has the binary value 0101:

1
var flags = 0x5;   // binary 0101

This value indicates:

  • flag A is true (we have an ant problem);
  • flag B is false (we don’t own a bat);
  • flag C is true (we own a cat);
  • flag D is false (we don’t own a duck);

Since bitwise operators are 32-bit, 0101 is actually 00000000000000000000000000000101, but the preceding zeroes can be neglected since they contain no meaningful information.

bitmask is a sequence of bits that can manipulate and/or read flags. Typically, a “primitive” bitmask for each flag is defined:

1
2
3
4
var FLAG_A = 0x1; // 0001
var FLAG_B = 0x2; // 0010
var FLAG_C = 0x4; // 0100
var FLAG_D = 0x8; // 1000

New bitmasks can be created by using the bitwise logical operators on these primitive bitmasks. For example, the bitmask 1011 can be created by ORing FLAG_A, FLAG_B, and FLAG_D:

1
var mask = FLAG_A | FLAG_B | FLAG_D; // 0001 | 0010 | 1000 => 1011

Individual flag values can be extracted by ANDing them with a bitmask, where each bit with the value of one will “extract” the corresponding flag. The bitmask masks out the non-relevant flags by ANDing with zeros (hence the term “bitmask”). For example, the bitmask 0100 can be used to see if flag C is set:

1
2
3
4
// if we own a cat
if (flags & FLAG_C) { // 0101 & 0100 => 0100 => true
   // do stuff
}

A bitmask with multiple set flags acts like an “either/or”. For example, the following two are equivalent:

1
2
3
4
// if we own a bat or we own a cat
if ((flags & FLAG_B) || (flags & FLAG_C)) { // (0101 & 0010) || (0101 & 0100) => 0000 || 0100 => true
   // do stuff
}
1
2
3
4
5
// if we own a bat or cat
var mask = FLAG_B | FLAG_C; // 0010 | 0100 => 0110
if (flags & mask) { // 0101 & 0110 => 0100 => true
   // do stuff
}

Flags can be set by ORing them with a bitmask, where each bit with the value one will set the corresponding flag, if that flag isn’t already set. For example, the bitmask 1100 can be used to set flags C and D:

1
2
3
// yes, we own a cat and a duck
var mask = FLAG_C | FLAG_D; // 0100 | 1000 => 1100
flags |= mask;   // 0101 | 1100 => 1101

Flags can be cleared by ANDing them with a bitmask, where each bit with the value zero will clear the corresponding flag, if it isn’t already cleared. This bitmask can be created by NOTing primitive bitmasks. For example, the bitmask 1010 can be used to clear flags A and C:

1
2
3
// no, we don't neither have an ant problem nor own a cat
var mask = ~(FLAG_A | FLAG_C); // ~0101 => 1010
flags &= mask;   // 1101 & 1010 => 1000

The mask could also have been created with ~FLAG_A & ~FLAG_C (De Morgan’s law):

1
2
3
// no, we don't have an ant problem, and we don't own a cat
var mask = ~FLAG_A & ~FLAG_C;
flags &= mask;   // 1101 & 1010 => 1000

Flags can be toggled by XORing them with a bitmask, where each bit with the value one will toggle the corresponding flag. For example, the bitmask 0110 can be used to toggle flags B and C:

1
2
3
4
// if we didn't have a bat, we have one now, and if we did have one, bye-bye bat
// same thing for cats
var mask = FLAG_B | FLAG_C;
flags = flags ^ mask;   // 1100 ^ 0110 => 1010

Finally, the flags can all be flipped with the NOT operator:

1
2
// entering parallel universe...
flags = ~flags;    // ~1010 => 0101

Conversion snippets

Convert a binary string to a decimal number:

1
2
3
var sBinString = "1011";
var nMyNumber = parseInt(sBinString, 2);
alert(nMyNumber); // prints 11, i.e. 1011

Convert a decimal number to a binary string:

1
2
3
var nMyNumber = 11;
var sBinString = nMyNumber.toString(2);
alert(sBinString); // prints 1011, i.e. 11

Automatize the creation of a mask

If you have to create many masks from some boolean values, you can automatize the process:

1
2
3
4
5
6
7
8
9
10
11
function createMask () {
  var nMask = 0, nFlag = 0, nLen = arguments.length > 32 ? 32 : arguments.length;
  for (nFlag; nFlag < nLen; nMask |= arguments[nFlag] << nFlag++);
  return nMask;
}
var mask1 = createMask(true, true, false, true); // 11, i.e.: 1011
var mask2 = createMask(false, false, true); // 4, i.e.: 0100
var mask3 = createMask(true); // 1, i.e.: 0001
// etc.
alert(mask1); // prints 11, i.e.: 1011

Reverse algorithm: an array of booleans from a mask

If you want to create an array of booleans from a mask you can use this code:

1
2
3
4
5
6
7
8
9
10
11
12
13
function arrayFromMask (nMask) {
  // nMask must be between -2147483648 and 2147483647
  if (nMask > 0x7fffffff || nMask < -0x80000000) { throw new TypeError("arrayFromMask - out of range"); }
  for (var nShifted = nMask, aFromMask = []; nShifted; aFromMask.push(Boolean(nShifted & 1)), nShifted >>>= 1);
  return aFromMask;
}
var array1 = arrayFromMask(11);
var array2 = arrayFromMask(4);
var array3 = arrayFromMask(1);
alert("[" + array1.join(", ") + "]");
// prints "[true, true, false, true]", i.e.: 11, i.e.: 1011

You can test both algorithms at the same time…

1
2
3
4
var nTest = 19; // our custom mask
var nResult = createMask.apply(this, arrayFromMask(nTest));
alert(nResult); // 19

For didactic purpose only (since there is the Number.toString(2) method), we show how it is possible to modify the arrayFromMask algorithm in order to create astring containing the binary representation of a number, rather than an array of booleans:

1
2
3
4
5
6
7
8
9
10
11
12
function createBinaryString (nMask) {
  // nMask must be between -2147483648 and 2147483647
  for (var nFlag = 0, nShifted = nMask, sMask = ""; nFlag < 32; nFlag++, sMask += String(nShifted >>> 31), nShifted <<= 1);
  return sMask;
}
var string1 = createBinaryString(11);
var string2 = createBinaryString(4);
var string3 = createBinaryString(1);
alert(string1);
// prints 00000000000000000000000000001011, i.e. 11

s3ql – a full-featured file system for online data storage

I am really impressed by my first look at s3ql. A really complete, just working and well documented tool to mount aws s3 and other cloud storage solution on a dedicated server:

S3QL is a file system that stores all its data online using storage services like Google Storage, Amazon S3 or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X.

S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival.

S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components.

Features

  • Transparency. Conceptually, S3QL is indistinguishable from a local file system. For example, it supports hardlinks, symlinks, ACLs and standard unix permissions, extended attributes and file sizes up to 2 TB.
  • Dynamic Size. The size of an S3QL file system grows and shrinks dynamically as required.
  • Compression. Before storage, all data may compressed with the LZMA, bzip2 or deflate (gzip) algorithm.
  • Encryption. After compression (but before upload), all data can AES encrypted with a 256 bit key. An additional SHA256 HMAC checksum is used to protect the data against manipulation.
  • Data De-duplication. If several files have identical contents, the redundant data will be stored only once. This works across all files stored in the file system, and also if only some parts of the files are identical while other parts differ.
  • Immutable Trees. Directory trees can be made immutable, so that their contents can no longer be changed in any way whatsoever. This can be used to ensure that backups can not be modified after they have been made.
  • Copy-on-Write/Snapshotting. S3QL can replicate entire directory trees without using any additional storage space. Only if one of the copies is modified, the part of the data that has been modified will take up additional storage space. This can be used to create intelligent snapshots that preserve the state of a directory at different points in time using a minimum amount of space.
  • High Performance independent of network latency. All operations that do not write or read file contents (like creating directories or moving, renaming, and changing permissions of files and directories) are very fast because they are carried out without any network transactions. S3QL achieves this by saving the entire file and directory structure in a database. This database is locally cached and the remote copy updated asynchronously.
  • Support for low bandwidth connections. S3QL splits file contents into smaller blocks and caches blocks locally. This minimizes both the number of network transactions required for reading and writing data, and the amount of data that has to be transferred when only parts of a file are read or written.

 

http://code.google.com/p/s3ql/

 

OSX Terminal tab-autocomplete

Finally I got the real bash auto-comletion on my iMac iTerm! iEasy as pie!

sudo port install bash-completion
 
To use bash_completion, add the following lines at the end of your .bash_profile:
  if [ -f /opt/local/etc/bash_completion ]; then
      . /opt/local/etc/bash_completion
  fi

That’s it…. have fun