Big O (often pronounced order) is an abstract function that describes how fast a function grows as the size of the problem becomes large. The order given by Big O is a least upper bound on the rate of growth.
We say that a function T(n) has order O(f(n)) if there exist
positive constants c and n0 such that:
T(n) &le c * f(n) when n &ge n0.
For example, the function T(n) = 2 * n2 + 5 * n + 100 has order O(n2) because 2 * n2 + 5 * n + 100 &le 3 * n2 for n &ge 13.
We don't care about performance for small values of n, since small problems are usually easy to solve. In fact, we can make that a rule:
Rule: Any algorithm is okay if we know that the size of the input is small. In such cases, the simplest (easiest to code) algorithm is often best.