MongoDB is very powerful, but it is still easy to get started with. In this chapter we’ll introduce some of the basic concepts of MongoDB:

• A document is the basic unit of data for MongoDB, roughly equivalent to a row in a relational database management system (but much more expressive).
• Similarly, a collection can be thought of as the schema-free equivalent of a table.
• A single instance of MongoDB can host multiple independent databases, each of which can have its own collections and permissions.
• MongoDB comes with a simple but powerful JavaScript shell, which is useful for the administration of MongoDB instances and data manipulation.
• Every document has a special key, "_id", that is unique across the document’s collection.

Documents

At the heart of MongoDB is the concept of a document: an ordered set of keys with associated values. The representation of a document differs by programming language,but most languages have a data structure that is a natural fit, such as a map, hash, or dictionary. In JavaScript, for example, documents are represented as objects:

{"greeting" : "Hello, world!"}

This simple document contains a single key, "greeting", with a value of "Hello,world!". Most documents will be more complex than this simple one and often will contain multiple key/value pairs:

{"greeting" : "Hello, world!", "foo" : 3}

This example is a good illustration of several important concepts:

• Key/value pairs in documents are ordered—the earlier document is distinct from the following document:

{"foo" : 3, "greeting" : "Hello, world!"}

In most cases the ordering of keys in documents is not important.In fact, in some programming languages the default representation of a document does not even maintain ordering (e.g., dictionaries in Python and hashes in Perl or Ruby 1.8). Drivers for those languages usually have some mechanism for specifying documents with ordering for the rare cases when it is necessary. (Those cases will be noted throughout the text.)

Values in documents are not just “blobs.” They can be one of several different data types (or even an entire embedded document—see “Embedded Documents” on page 20). In this example the value for "greeting" is a string, whereas the value for "foo" is an integer

The keys in a document are strings. Any UTF-8 character is allowed in a key, with a few notable exceptions:

Keys must not contain the character \0 (the null character). This character is used to signify the end of a key.
• The . and $ characters have some special properties and should be used only in certain circumstances, as described in later chapters. In general, they should be considered reserved, and drivers will complain if they are used inappropriately.
• Keys starting with _ should be considered reserved; although this is not strictly enforced.

MongoDB is type-sensitive and case-sensitive. For example, these documents are distinct:

{"foo" : 3}
{"foo" : "3"}

As are as these:

{"foo" : 3}
{"Foo" : 3}

A final important thing to note is that documents in MongoDB cannot contain duplicate keys. For example, the following is not a legal document:

{"greeting" : "Hello, world!", "greeting" : "Hello, MongoDB!"}

Collections

A collection is a group of documents. If a document is the MongoDB analog of a row in a relational database, then a collection can be thought of as the analog to a table.

Schema-Free

Collections are schema-free. This means that the documents within a single collection can have any number of different “shapes.” For example, both of the following documents could be stored in a single collection:

{"greeting" : "Hello, world!"}
{"foo" : 5}

Note that the previous documents not only have different types for their values (string versus integer) but also have entirely different keys. Because any document can be put into any collection, the question often arises: “Why do we need separate collections at all?” It’s a good question—with no need for separate schemas for different kinds of documents, why should we use more than one collection? There are several good reasons:

Keeping different kinds of documents in the same collection can be a nightmare for developers and admins. Developers need to make sure that each query is only returning documents of a certain kind or that the application code performing a query can handle documents of different shapes. If we’re querying for blog posts,it’s a hassle to weed out documents containing author data.

It is much faster to get a list of collections than to extract a list of the types in a collection. For example, if we had a type key in the collection that said whether each document was a “skim,” “whole,” or “chunky monkey” document, it would be much slower to find those three values in a single collection than to have three separate collections and query for their names (see “Subcollections” on page 8).

• Grouping documents of the same kind together in the same collection allows for data locality. Getting several blog posts from a collection containing only posts will likely require fewer disk seeks than getting the same posts from a collection containing posts and author data.
• We begin to impose some structure on our documents when we create indexes.(This is especially true in the case of unique indexes.) These indexes are defined per collection. By putting only documents of a single type into the same collection,we can index our collections more efficiently.

As you can see, there are sound reasons for creating a schema and for grouping related types of documents together. MongoDB just relaxes this requirement and allows developers more flexibility.

Naming

A collection is identified by its name. Collection names can be any UTF-8 string, with a few restrictions:
• The empty string ("") is not a valid collection name.
• Collection names may not contain the character \0 (the null character) because this delineates the end of a collection name.
• You should not create any collections that start with system., a prefix reserved for system collections. For example, the system.users collection contains the database’s users, and the system.namespaces collection contains information about all of the database’s collections.
• User-created collections should not contain the reserved character $ in the name.The various drivers available for the database do support using $ in collection names because some system-generated collections contain it. You should not use $ in a name unless you are accessing one of these collections.

Subcollections

One convention for organizing collections is to use namespaced subcollections separated by the . character. For example, an application containing a blog might have a collection named blog.posts and a separate collection named blog.authors. This is for organizational purposes only—there is no relationship between the blog collection (it doesn’t even have to exist) and its “children.”

Although subcollections do not have any special properties, they are useful and incorporated into many MongoDB tools:

• GridFS, a protocol for storing large files, uses subcollections to store file metadata separately from content chunks (see Chapter 7 for more information about GridFS).
• The MongoDB web console organizes the data in its DBTOP section by subcollection (see Chapter 8 for more information on administration).
• Most drivers provide some syntactic sugar for accessing a subcollection of a given collection. For example, in the database shell, db.blog will give you the blog collection, and db.blog.posts will give you the blog.posts collection.

Subcollections are a great way to organize data in MongoDB, and their use is highly recommended.

Databases

In addition to grouping documents by collection, MongoDB groups collections into databases. A single instance of MongoDB can host several databases, each of which can be thought of as completely independent. A database has its own permissions, and each  database is stored in separate files on disk. A good rule of thumb is to store all data for a single application in the same database. Separate databases are useful when storing data for several application or users on the same MongoDB server.

Like collections, databases are identified by name. Database names can be any UTF-8 string, with the following restrictions:
• The empty string ("") is not a valid database name.
• A database name cannot contain any of these characters: ‘ ‘ (a single space), ., $, /,\, or \0 (the null character).
• Database names should be all lowercase.
• Database names are limited to a maximum of 64 bytes.

One thing to remember about database names is that they will actually end up as files on your filesystem. This explains why many of the previous restrictions exist in the first place.

There are also several reserved database names, which you can access directly but have special semantics. These are as follows:

admin
This is the “root” database, in terms of authentication. If a user is added to the admin database, the user automatically inherits permissions for all databases.There are also certain server-wide commands that can be run only from the admin database, such as listing all of the databases or shutting down the server.
local
This database will never be replicated and can be used to store any collections that should be local to a single server (see Chapter 9 for more information about replication and the local database).
config
When Mongo is being used in a sharded setup (see Chapter 10), the config database is used internally to store information about the shards.

By prepending a collection’s name with its containing database, you can get a fully qualified collection name called a namespace. For instance, if you are using the blog.posts collection in the cms database, the namespace of that collection would be cms.blog.posts. Namespaces are limited to 121 bytes in length and, in practice, should be less than 100 bytes long. For more on namespaces and the internal representation of collections in MongoDB, see Appendix C.

Getting and Starting MongoDB

MongoDB is almost always run as a network server that clients can connect to and perform operations on. To start the server, run the mongod executable:

$ ./mongod
./mongod --help for help and startup options
Sun Mar 28 12:31:20 Mongo DB : starting : pid = 44978 port = 27017
dbpath = /data/db/ master = 0 slave = 0  64-bit
Sun Mar 28 12:31:20 db version v1.5.0-pre-, pdfile version 4.5
Sun Mar 28 12:31:20 git version: ...
Sun Mar 28 12:31:20 sys info: ...
Sun Mar 28 12:31:20 waiting for connections on port 27017
Sun Mar 28 12:31:20 web admin interface listening on port 28017

Or if you’re on Windows, run this:

$ mongod.exe

For detailed information on installing MongoDB on your system, see Appendix A.

When run with no arguments, mongod will use the default data directory, /data/db/ (or C:\data\db\ on Windows), and port 27017. If the data directory does not already exist
or is not writable, the server will fail to start. It is important to create the data directory (e.g., mkdir -p /data/db/), and to make sure your user has permission to write to the
directory, before starting MongoDB. The server will also fail to start if the port is not available—this is often caused by another instance of MongoDB that is already running.

The server will print some version and system information and then begin waiting for connections. By default, MongoDB listens for socket connections on port 27017.

mongod also sets up a very basic HTTP server that listens on a port 1,000 higher than the main port, in this case 28017. This means that you can get some administrative
information about your database by opening a web browser and going to http://localhost:28017.

You can safely stop mongod by typing Ctrl-c in the shell that is running the server.

For more information on starting or stopping MongoDB, see “Starting and Stopping MongoDB” on page 111, and for more on the administrative interface, see “Using the Admin Interface” on page 115.

MongoDB Shell

MongoDB comes with a JavaScript shell that allows interaction with a MongoDB instance from the command line. The shell is very useful for performing administrative functions, inspecting a running instance, or just playing around. The mongo shell is a crucial tool for using MongoDB and is used extensively throughout the rest of the text.

Running the Shell

To start the shell, run the mongo executable:

$ ./mongo
MongoDB shell version: 1.6.0
url: test
connecting to: test
type "help" for help
>

The shell automatically attempts to connect to a MongoDB server on startup, so make sure you start mongod before starting the shell.

The shell is a full-featured JavaScript interpreter, capable of running arbitrary JavaScript programs. To illustrate this, let’s perform some basic math:

> x = 200
200
> x / 5;
40

We can also leverage all of the standard JavaScript libraries:

> Math.sin(Math.PI / 2);
1
> new Date("2010/1/1");
"Fri Jan 01 2010 00:00:00 GMT-0500 (EST)"
> "Hello, World!".replace("World", "MongoDB");
Hello, MongoDB!

We can even define and call JavaScript functions:

> function factorial (n) {
... if (n <= 1) return 1;
... return n * factorial(n - 1);
... }
> factorial(5);
120

Note that you can create multiline commands. The shell will detect whether the JavaScript statement is complete when you press Enter and, if it is not, will allow you to continue writing it on the next line.

A MongoDB Client

Although the ability to execute arbitrary JavaScript is cool, the real power of the shell lies in the fact that it is also a stand-alone MongoDB client. On startup, the shell connects to the test database on a MongoDB server and assigns this database connection to the global variable db. This variable is the primary access point to MongoDB through the shell.

The shell contains some add-ons that are not valid JavaScript syntax but were implemented because of their familiarity to users of SQL shells. The add-ons do not provide any extra functionality, but they are nice syntactic sugar. For instance, one of the most important operations is selecting which database to use:

> use foobar
switched to db foobar

Now if you look at the db variable, you can see that it refers to the foobar database:

> db
foobar

Because this is a JavaScript shell, typing a variable will convert the variable to a string (in this case, the database name) and print it.

Collections can be accessed from the db variable. For example, db.baz returns the baz collection in the current database. Now that we can access a collection in the shell, we can perform almost any database operation.

Basic Operations with the Shell

We can use the four basic operations, create, read, update, and delete (CRUD), to manipulate and view data in the shell.

Create

The insert function adds a document to a collection. For example, suppose we want to store a blog post. First, we’ll create a local variable called post that is a JavaScript object representing our document. It will have the keys "title", "content", and "date" (the date that it was published):

> post = {"title" : "My Blog Post",
... "content" : "Here‘s my blog post.",
... "date" : new Date()}
{
    "title" : "My Blog Post",
    "content" : "Here‘s my blog post.",
    "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)"
}

This object is a valid MongoDB document, so we can save it to the blog collection using the insert method:

> db.blog.insert(post)

The blog post has been saved to the database. We can see it by calling find on the collection:

> db.blog.find()
{
    "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"),
    "title" : "My Blog Post",
    "content" : "Here‘s my blog post.",
    "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)"}

You can see that an "_id" key was added and that the other key/value pairs were saved as we entered them. The reason for "_id"’s sudden appearance is explained at the end of this chapter.

Read

find returns all of the documents in a collection. If we just want to see one document from a collection, we can use findOne:

> db.blog.findOne()
{
    "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"),
    "title" : "My Blog Post",
    "content" : "Here‘s my blog post.",
    "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)"
}

find and findOne can also be passed criteria in the form of a query document. This will restrict the documents matched by the query. The shell will automatically display up to 20 documents matching a find, but more can be fetched. See Chapter 4 for more information on querying.

Update

If we would like to modify our post, we can use update. update takes (at least) two parameters: the first is the criteria to find which document to update, and the second is the new document. Suppose we decide to enable comments on the blog post we created earlier. We’ll need to add an array of comments as the value for a new key in our document.

The first step is to modify the variable post and add a "comments" key:

> post.comments = []
[  ]

Then we perform the update, replacing the post titled “My Blog Post” with our new version of the document:

> db.blog.update({title : "My Blog Post"}, post)

Now the document has a "comments" key. If we call find again, we can see the new key:

> db.blog.find()
{
    "_id" : ObjectId("4b23c3ca7525f35f94b60a2d"),
    "title" : "My Blog Post",
    "content" : "Here‘s my blog post.",
    "date" : "Sat Dec 12 2009 11:23:21 GMT-0500 (EST)"
    "comments" : [ ]
}

Delete

remove deletes documents permanently from the database. Called with no parameters,it removes all documents from a collection. It can also take a document specifying criteria for removal. For example, this would remove the post we just created:

> db.blog.remove({title : "My Blog Post"}) 

Now the collection will be empty again.

Tips for Using the Shell

Because mongo is simply a JavaScript shell, you can get a great deal of help for it by simply looking up JavaScript documentation online. The shell also includes built-in
help that can be accessed by typing help:

> help
HELP
    show dbs                     show database names
    show collections             show collections in current database
    show users                   show users in current database
    show profile                 show recent system.profile entries w. time >= 1ms
    use <db name>                set current database to <db name>
    db.help()                    help on DB methods
    db.foo.help()                help on collection methods
    db.foo.find()                list objects in collection foo
    db.foo.find( { a : 1 } )     list objects in foo where a == 1
    it                           result of the last line evaluated

Help for database-level commands is provided by db.help();, and help at the collections can be accessed with db.foo.help();.

A good way of figuring out what a function is doing is to type it without the parentheses.This will print the JavaScript source code for the function. For example, if we are curious
about how the update function works or cannot remember the order of parameters, we can do the following:

> db.foo.update
function (query, obj, upsert, multi) {
    assert(query, "need a query");
    assert(obj, "need an object");
    this._validateObject(obj);
    this._mongo.update(this._fullName, query, obj, 

upsert ? true : false, multi ? true : false);
}

There is also an autogenerated API of all the JavaScript functions provided by the shell at http://api.mongodb.org/js.

Inconvenient collection names

Fetching a collection with db.collectionName almost always works, unless the collection name actually is a property of the database class. For instance, if we are trying to access the version collection, we cannot say db.version because db.version is a database function. (It returns the version of the running MongoDB server.)

> db.version
function () {
    return this.serverBuildInfo().version;
}

db’s collection-returning behavior is only a fallback for when JavaScript cannot find a matching property. When there is a property with the same name as the desired collection, we can use the getCollection function:

> db.getCollection("version");
test.version

This can also be handy for collections with invalid JavaScript in their names. For example, foo-bar is a valid collection name, but it’s variable subtraction in JavaScript.
You can get the foo-bar collection with db.getCollection("foo-bar").In JavaScript, x.y is identical to x[‘y‘]. This means that subcollections can be accessed using variables, not just literal names. That is, if you needed to perform some operation on every blog subcollection, you could iterate through them with something like this:

var collections = ["posts", "comments", "authors"];
for (i in collections) {
    doStuff(db.blog[collections[i]]);
}

Instead of this:

doStuff(db.blog.posts);
doStuff(db.blog.comments);
doStuff(db.blog.authors);

Data Types

The beginning of this chapter covered the basics of what a document is. Now that you are up and running with MongoDB and can try things on the shell, this section will dive a little deeper. MongoDB supports a wide range of data types as values in documents. In this section, we’ll outline all of the supported types.

Basic Data Types

Documents in MongoDB can be thought of as “JSON-like” in that they are conceptually similar to objects in JavaScript. JSON is a simple representation of data: the specification can be described in about one paragraph (http://www.json.org proves it) and lists only six data types. This is a good thing in many ways: it’s easy to understand, parse, and remember. On the other hand, JSON’s expressive capabilities are limited, because the only types are null, boolean, numeric, string, array, and object.

Although these types allow for an impressive amount of expressivity, there are a couple of additional types that are crucial for most applications, especially when working with
a database. For example, JSON has no date type, which makes working with dates even more annoying than it usually is. There is a number type, but only one—there is no way to differentiate floats and integers, never mind any distinction between 32-bit and 64-bit numbers. There is no way to represent other commonly used types, either, such as regular expressions or functions.

MongoDB adds support for a number of additional data types while keeping JSON’s essential key/value pair nature. Exactly how values of each type are represented varies by language, but this is a list of the commonly supported types and how they are represented as part of a document in the shell:

null
Null can be used to represent both a null value and a nonexistent field:

{"x" : null}

boolean
There is a boolean type, which will be used for the values ‘true‘ and ‘false‘:

{"x" : true}

32-bit integer
This cannot be represented on the shell. As mentioned earlier, JavaScript supports only 64-bit floating point numbers, so 32-bit integers will be converted into those.
64-bit integer
Again, the shell cannot represent these. The shell will display them using a special embedded document; see the section “Numbers” on page 18 for details.
64-bit floating point number
All numbers in the shell will be of this type. Thus, this will be a floating-point number:

{"x" : 3.14}

As will this:

{"x" : 3}

string
Any string of UTF-8 characters can be represented using the string type:

{"x" : "foobar"}

symbol
This type is not supported by the shell. If the shell gets a symbol from the database,it will convert it into a string.
object id
An object id is a unique 12-byte ID for documents. See the section “_id and ObjectIds” on page 20 for details:

{"x" : ObjectId()}

date
Dates are stored as milliseconds since the epoch. The time zone is not stored:

{"x" : new Date()}

regular expression
Documents can contain regular expressions, using JavaScript’s regular expression syntax:

{"x" : /foobar/i}

code
Documents can also contain JavaScript code:

{"x" : function() { /* ... */ }}

binary data
      Binary data is a string of arbitrary bytes. It cannot be manipulated from the shell.
maximum value
 BSON contains a special type representing the largest possible value. The shell does not have a type for this.
minimum value
 BSON contains a special type representing the smallest possible value. The shell does not have a type for this.
undefined
Undefined can be used in documents as well (JavaScript has distinct types for null and undefined):

{"x" : undefined}

array
Sets or lists of values can be represented as arrays:

{"x" : ["a", "b", "c"]}

embedded document
Documents can contain entire documents, embedded as values in a parent document:

{"x" : {"foo" : "bar"}}

Numbers

JavaScript has one “number” type. Because MongoDB has three number types (4-byte integer, 8-byte integer, and 8-byte float), the shell has to hack around JavaScript’s limitations a bit. By default, any number in the shell is treated as a double by MongoDB.This means that if you retrieve a 4-byte integer from the database, manipulate its document, and save it back to the database even without changing the integer, the integer will be resaved as a floating-point number. Thus, it is generally a good idea not to overwrite entire documents from the shell (see Chapter 3 for information on making changes to the values of individual keys).
Another problem with every number being represented by a double is that there are some 8-byte integers that cannot be accurately represented by 8-byte floats. Therefore,if you save an 8-byte integer and look at it in the shell, the shell will display it as an embedded document indicating that it might not be exact. For example, if we save a document with a "myInteger" key whose value is the 64-bit integer, 3, and then look at it in the shell, it will look like this:

> doc = db.nums.findOne()
{
    "_id" : ObjectId("4c0beecfd096a2580fe6fa08"),
    "myInteger" : {
        "floatApprox" : 3
    }
}

The number is not changed in the database (unless you modify and resave the object from the shell, in which case it will turn into a float); the embedded document just indicates that the shell is displaying a floating-point approximation of an 8-byte integer.If this embedded document has only one key, it is, in fact, exact.

If you insert an 8-byte integer that cannot be accurately displayed as a double, the shell will add two keys, "top" and "bottom", containing the 32-bit integers representing the 4 high-order bytes and 4 low-order bytes of the integer, respectively. For instance, if we insert 9223372036854775807, the shell will show us the following:

> db.nums.findOne()
{
    "_id" : ObjectId("4c0beecfd096a2580fe6fa09"),
    "myInteger" : {
        "floatApprox" : 9223372036854776000,
        "top" : 2147483647,
        "bottom" : 4294967295
    }
}

The "floatApprox" embedded documents are special and can be manipulated as numbers as well as documents:

> doc.myInteger + 1
4

> doc.myInteger.floatApprox
3

All 4-byte integers can be represented exactly by an 8-byte floating-point number, so they are displayed normally.

Dates

In JavaScript, the Date object is used for MongoDB’s date type. When creating a new Date object, always call new Date(...), not just Date(...). Calling the constructor as a function (that is, not including new) returns a string representation of the date, not an actual Date object. This is not MongoDB’s choice; it is how JavaScript works. If you are not careful to always use the Date constructor, you can end up with a mishmash of strings and dates. Strings do not match dates, and vice versa, so this can cause problems with removing, updating, querying…pretty much everything.
For a full explanation of JavaScript’s Date class and acceptable formats for the constructor, see ECMAScript specification section 15.9 (available for download at http://www.ecmascript.org).
Dates in the shell are displayed using local time zone settings. However, dates in the database are just stored as milliseconds since the epoch, so they have no time zone information associated with them. (Time zone information could, of course, be stored as the value for another key.)

Arrays

Arrays are values that can be interchangeably used for both ordered operations (as though they were lists, stacks, or queues) and unordered operations (as though they were sets).
In the following document, the key "things" has an array value:

{"things" : ["pie", 3.14]}

As we can see from the example, arrays can contain different data types as values (in this case, a string and a floating-point number). In fact, array values can be any of the
supported values for normal key/value pairs, even nested arrays.

One of the great things about arrays in documents is that MongoDB “understands” their structure and knows how to “reach inside” of arrays to perform operations on their contents. This allows us to query on arrays and build indexes using their contents.For instance, in the previous example, MongoDB can query for all documents where 3.14 is an element of the "things" array. If this is a common query, you can even create an index on the "things" key to improve the query’s speed.

MongoDB also allows atomic updates that modify the contents of arrays, such as reaching into the array and changing the value pie to pi. We’ll see more examples of these types of operations throughout the text.

Embedded Documents

Embedded documents are entire MongoDB documents that are used as the value for a key in another document. They can be used to organize data in a more natural way than just a flat structure.

For example, if we have a document representing a person and want to store his address,we can nest this information in an embedded "address" document:

{
    "name" : "John Doe",
    "address" : {
        "street" : "123 Park Street",
        "city" : "Anytown",
        "state" : "NY"
    }
}

The value for the "address" key in the previous example is another document with its own values for "street", "city", and "state".

As with arrays, MongoDB “understands” the structure of embedded documents and is able to “reach inside” of them to build indexes, perform queries, or make updates.

We’ll discuss schema design in depth later, but even from this basic example, we can begin to see how embedded documents can change the way we work with data. In a relational database, the previous document would probably be modeled as two separate rows in two different tables (one for “people” and one for “addresses”). With MongoDB we can embed the address document directly within the person document. When used properly, embedded documents can provide a more natural (and often more efficient) representation of information.

The flip side of this is that we are basically denormalizing, so there can be more data repetition with MongoDB. Suppose “addresses” were a separate table in a relational database and we needed to fix a typo in an address. When we did a join with “people” and “addresses,” we’d get the updated address for everyone who shares it. With MongoDB, we’d need to fix the typo in each person’s document.

_id and ObjectIds

Every document stored in MongoDB must have an "_id" key. The "_id" key’s value can be any type, but it defaults to an ObjectId. In a single collection, every document must have a unique value for "_id", which ensures that every document in a collection can be uniquely identified. That is, if you had two collections, each one could have a document where the value for "_id" was 123. However, neither collection could contain more than one document where "_id" was 123.

ObjectIds

ObjectId is the default type for "_id". It is designed to be lightweight, while still being easy to generate in a globally unique way across disparate machines. This is the main reason why MongoDB uses ObjectIds as opposed to something more traditional, like an auto incrementing primary key: it is difficult and time-consuming to synchronize autoincrementing primary keys across multiple servers. Because MongoDB was designed from the beginning to be a distributed database, dealing with many nodes is an important consideration. The ObjectId type, as we’ll see, is easy to generate in a sharded environment.

ObjectIds use 12 bytes of storage, which gives them a string representation that is 24 hexadecimal digits: 2 digits for each byte. This causes them to appear larger than they are, which makes some people nervous. It’s important to note that even though an ObjectId is often represented as a giant hexadecimal string, the string is actually twice as long as the data being stored.

If you create multiple new ObjectIds in rapid succession, you can see that only the last few digits change each time. In addition, a couple of digits in the middle of the ObjectId will change (if you space the creations out by a couple of seconds). This is because of the manner in which ObjectIds are created. The 12 bytes of an ObjectId are generated as follows:

The first four bytes of an ObjectId are a timestamp in seconds since the epoch. This provides a couple of useful properties:
• The timestamp, when combined with the next five bytes (which will be described in a moment), provides uniqueness at the granularity of a second.
• Because the timestamp comes first, it means that ObjectIds will sort in roughly insertion order. This is not a strong guarantee but does have some nice properties,such as making ObjectIds efficient to index.
• In these four bytes exists an implicit timestamp of when each document was created. Most drivers expose a method for extracting this information from an ObjectId.

Because the current time is used in ObjectIds, some users worry that their servers will need to have synchronized clocks. This is not necessary because the actual value of the timestamp doesn’t matter, only that it is often new (once per second) and increasing.

The next three bytes of an ObjectId are a unique identifier of the machine on which it was generated. This is usually a hash of the machine’s hostname. By including these bytes, we guarantee that different machines will not generate colliding ObjectIds.

To provide uniqueness among different processes generating ObjectIds concurrently on a single machine, the next two bytes are taken from the process identifier (PID) of the ObjectId-generating process.

These first nine bytes of an ObjectId guarantee its uniqueness across machines and processes for a single second. The last three bytes are simply an incrementing counter that is responsible for uniqueness within a second in a single process. This allows for up to 2563 (16,777,216) unique ObjectIds to be generated per process in a single second.

Autogeneration of _id
As stated previously, if there is no "_id" key present when a document is inserted, one will be automatically added to the inserted document. This can be handled by the MongoDB server but will generally be done by the driver on the client side. There are a couple of reasons for that:
• Although ObjectIds are designed to be lightweight and easy to generate, there is still some overhead involved in their generation. The decision to generate them on the client side reflects an overall philosophy of MongoDB: work should be pushed out of the server and to the drivers whenever possible. This philosophy reflects the fact that, even with scalable databases like MongoDB, it is easier to scale out at the application layer than at the database layer. Moving work to the client side reduces the burden requiring the database to scale.
• By generating ObjectIds on the client side, drivers are capable of providing richer APIs than would be otherwise possible. For example, a driver might have its insert method either return the generated ObjectId or inject it directly into the document that was inserted. If the driver allowed the server to generate ObjectIds, then a separate query would be required to determine the value of "_id" for an inserted document.