This page is all about how we can efficiently compute large sums of
+ powers:
+
+
+
+
+
+
Demo
+
Note that large exponents will take longer to calculate, but your
+ upper bound can be quite large - even thousands of digits!
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
How it works
+
A straightforward approach to computing a sum like this would be
+ something like
+
+
+
function sumOfPowers(n, p) {
+ let total = 0;
+ for (let k = 1; k <= n; k++) {
+ total += Math.pow(k, p);
+ }
+ return total;
+}
+ A simple algorithm for computing sums of powers
+
+
+
This works fine for small values of , but
+ this algorithm requires we perform
+ exponentiations and additions. In other
+ words, the time to run grows as increases,
+ which makes computing sums with a high upper bound take a long time.
+ This is not the algorithm the demo uses.
+
+
In fact, the algorithm used in the demo looks quite different.
+ It's not that many lines of code, but it's relying on some nifty (and
+ slightly tricky) mathematics. If all you do is read the code, it's
+ hard to believe it works, so rather than simply presenting it right
+ now, I want to introduce it piece by piece and explain it as we
+ go.
+
+
+
Special cases
+
If we want to compute a sum where each term's exponent is 1, it
+ turns out there's a simple formula we can use
+ (read more about why it works here):
+
+
+
+
+
The nice thing about this formula is the number of operations we
+ perform is no longer dependent on our upper bound - we'll always
+ perform one addition, one multiplication, and one division. That
+ means we can easily determine the sum of the numbers from 1 to some
+ extreme limit like 1,000,000,000,000,000 much faster than with the
+ naive algorithm.
+
+
There are similar formulas for computing sums when our exponent
+ is 2 or 3:
+
+
+
+
+
+
It turns out there's a pattern that lets us construct
+ polynomials like these for any exponent, but the pattern
+ isn't super obvious.
+
+
+
+
Faulhaber's formula
+
+ Faulhaber's formula
+ gives the general pattern these polynomials follow. The most
+ commonly given version of the formula involves a special sequence
+ called the
+ Bernoulli numbers.
+
+
+
Of course, there are algorithms to compute the Bernoulli
+ numbers, but since they're rational numbers, there are tradeoffs to
+ consider. If we use floats to store the sequence, we need to be
+ mindful of rounding errors; additionally, after the 258th term, the
+ Bernoulli numbers become too large to store in a 64 bit float. This
+ means to compute sums of large powers precisely, we'll want a
+ rational data type, which not every language has handy.
+
+
I suppose if you need the coefficients of the polynomial for
+ some reason, this is the formula for you, but if all you want is
+ the sum of powers for some and
+ , we can sidestep computing the Bernoulli
+ numbers by taking another approach.
+
+
+
+
Stirling numbers
+
To begin explaining this alternate approach, we need to
+ introduce the
+ Stirling numbers of the second kind
+ (as you might expect, there are Stirling numbers of the
+ first kind too, but they aren't super relevant here).
+ These numbers give the number of ways to partition a set of
+ elements into
+ non-empty subsets.
+
+
For instance, there's 6 ways to partition the letters a, b, c,
+ and d into 3 non-empty subsets:
+
+
{{a,b},{c},{d}}
+
{{a,c},{b},{d}}
+
{{a},{b,c},{d}}
+
{{a,d},{b},{c}}
+
{{a},{b,d},{c}}
+
{{a},{b},{c,d}}
+
+
+
The Stirling numbers of the second kind (or just "Stirling
+ numbers" from here on) happen to have a nice recursive formula that
+ we can apply to compute them.
+
+
+
+
+
To compute sums of powers for a given exponent
+ , we'll need to know
+
+
+
+
+
(We'll learn why in a bit.) Here's a function to get all those
+ values.
+
+
+
function stirling2(p) {
+ if (p === 0) {
+ return [1n];
+ }
+
+ let s = [0n];
+ for (let i = 1; i < p + 1; i++) {
+ s.push(1n);
+ for (let j = i - 1; j > 0; j--) {
+ s[j] = BigInt(j) * s[j] + s[j-1];
+ }
+ }
+
+ return s;
+}
+ An algorithm for computing Stirling numbers of the second kind
+
+
+
This algorithm is quadratic in the value of
+ , which isn't great, but it's simple to
+ implement (and I don't know of any faster way).
+
+
+
Stirling numbers calculator
+
Be warned that these numbers can grow quite large, even for
+ small values of .
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Connection to exponents
+
To understand what the Stirling numbers have to do with our
+ problem, it helps to first introduce the notation for
+ falling factorials.
+
+
+
+
+
It turns out there is a very elegant identity relating
+ exponents, Stirling numbers, and falling
+ factorials:1
+
+
+
+
+
We can apply this formula to every term of our power sum and
+ rearrange to get
+
+
+
+
+
This might seem more complicated at first, but despite the
+ similar notation to normal exponentiation, it's much easier to
+ compute sums of falling factorials. For
+ ,
+
+
+
+
(For , the sum is
+ just .)2 This lets us
+ simplify.3
+
+
+
+
+
+
The first term is 0 unless
+ , which is a trivial
+ case, so we really only need to consider the remaining terms.
+
+
+
+
Implementation
+
This formula might still seem a little complicated, but you may
+ be surprised by how simple it is to implement. We already have a
+ method to compute the Stirling numbers we need. With those values
+ available, we can write a simple loop to compute each term and add
+ to our total.
+
+
+
function faulhaber(n, s) {
+ const limit = BigInt(s.length);
+ if (limit === 1n) {
+ return n;
+ }
+
+ let q = n + 1n;
+ let total = 0n;
+ for (let k = 1n; k < limit; k++) {
+ q *= n + 1n - k;
+ total += s[k] * q / (k + 1n);
+ }
+
+ return total;
+}
+ A function that takes an upper bound and Stirling
+ numbers to compute a sum of powers
+
+
+
+
Note that the loop is dependent on and not
+ , which allows us to compute sums with
+ extremely large upper bounds very quickly.
+
+
+
Final thoughts
+
This approach spends most of its time computing the Stirling
+ numbers. Additionally, it seems like the performance is largely
+ hampered by how large the Stirling numbers get - computing them
+ with a modulus is considerably quicker. The demo is a little
+ clever, in that it only recomputes the Stirling numbers if the
+ value of the exponent changes.
+
+
If you would rather use the more commonly known Faulhaber
+ formula, you'll need to compute the Bernoulli numbers.
+ Brent and Harvey
+ give a simple algorithm that is as efficient as how we're
+ calculating the Stirling numbers. The needed binomial coefficients
+ can also be calculated easily.
+
+
Furthermore, Brent and Harvey give another algorithm for the
+ Bernoulli numbers that is even more optimized (although it is
+ considerably more complicated). I believe using that algorithm in
+ conjunction with the more traditional formula would be more
+ efficient than the Stirling number approach.
+
+
However, in other ways, I like this formula better. For one
+ thing, whichever algorithm you use, Bernoulli numbers are a bit
+ harder to calculate, and require more lines of code. You'll have to
+ compute the binomial coefficients, as well.
+
+
One nice thing about the Stirling approach is the terms are
+ integers throughout. I think that makes this algorithm a little
+ better for modular calculations.
+
+
+
+
+
Footnotes
+
+
1 For proofs of this identity, see chapter 1.9 of
+ Enumerative Combinatorics by Stanley and chapter 6.1
+ of Concrete Mathematics by Graham, Knuth, and
+ Patashnik.
+
2 See chapter 2.6 of Concrete
+ Mathematics for an explanation of this identity.
+
3 A version of this formula is also mentioned in
+ chapter 6.5 of Concrete Mathematics.
The
+ sieve of Eratosthenes
+ is a well-known algorithm for computing all the prime numbers up to a
+ given bound. There are also well-known variants on the sieve. In
+ particular, a segmented version can be implemented to limit memory
+ usage.
+
+ Furthermore, a similar approach can be used to "sieve" the values of
+ multiplicative functions.
+ For instance,
+ this page
+ discusses computing values of
+ Euler's totient function
+ with an approach that results in the same time complexity as the
+ standard sieve.
+
+
+ This Codeforces page
+ also discusses an approach for sieving any multiplicative
+ function. However, the provided code has some downsides: it's not
+ segmented, and the modifications needed for a given multiplicative
+ function are not always obvious.
+
+
Here, I'll discuss a variant of the sieve that
+
+
is segmented,
+
works for any multiplicative function,
+
works with a multiplicative function's "prime power" definition,
+ and
+
has the same time complexity as the standard sieve (assuming the
+ function has an efficient implementation for prime powers).
+
+
+
+
+
Demo
+
Note that for large ranges, outputting the full result will chew
+ up a lot of memory.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Multiplicative functions
+
+ Multiplicative functions
+ are functions with a special property: if two numbers
+ and are
+ coprime
+ (i.e. their
+ greatest common divisor
+ is 1), then
+ .
+ This means if we can factor a number into
+ coprime integers and ,
+ we can find by
+ instead finding
+ and and
+ multiplying them.
+
+
Going a step further, if we have a prime factorization of a number
+ , along with a formula for
+ at prime powers, we can easily compute
+ by evaluating
+ at each prime power and multiplying.
+
+
It turns out there are lots of multiplicative functions, including
+ many important functions in number theory, such as:
Furthermore, many of these functions have very simple formulas for
+ prime power inputs, making them easy to evaluate if you have a prime
+ factorization.
+
+
+
An example
+
Suppose we want to know how many divisors 120 has. We
+ could try counting every divisor by hand, but that would
+ be pretty tedious and error-prone. We could also write a simple
+ loop to have a computer find the answer, but that approach wouldn't
+ scale well for larger numbers. Instead, we'll take advantage of the
+ fact that the
+ number of divisors function
+ is multiplicative to find the answer.
+
+
We'll use
+ to denote the function. It has a convenient formula for prime
+ powers:1
+
+
+
+
+
+
This lets us answer our question by factoring
+ 120:2
+
+
+
+
+
+
Sure enough, the 16 divisors are 1, 2, 3, 4, 5, 6, 8, 10, 12,
+ 15, 20, 24, 30, 40, 60, and 120.
+
+
Suppose, however, that we want to find the number of divisors
+ for each number up to 120.
+ There's no general algorithm to efficiently factor a number,
+ so it isn't a great idea to factor every number to get the
+ solution. We'll take a different approach, but it will still take
+ advantage of the multiplicative property.
+
+
+
+
What about additive functions?
+
There's another class of functions called
+ additive functions
+ with similar properties to multiplicative functions, namely that
+
+ whenever and are
+ coprime. We can sieve these functions in nearly the exact same
+ manner as multiplicative functions (e.g. the
+ prime omega functions
+ in the demo are additive). When we get to the code, you'll see the
+ provided functions take a boolean indicating whether the function
+ is additive to apply the proper logic.
+
+
+
+
+
The algorithm
+
+
Here's the routine for sieving multiplicative functions:
+ This function performs a segmented sieve on any
+ multiplicative (or additive) function.
+
+
+
l and r are the lower and upper bounds
+ for the sieve, respectively. f is the function we are
+ sieving. primes is an array indicating all the prime
+ numbers up to the square root of r, which we can
+ precompute with the standard sieve. Finally, additive is
+ a boolean indicating if the function is additive or
+ multiplicative.
+
+
The first step of the function is to initialize two arrays of size
+ r - l called sieve and quo.
+ sieve is what ultimately gets returned and stores the
+ values of the function we're sieving. quo is an internal
+ array we use to finalize the values of sieve at the very end of the
+ function.
+
+
+
function initArrays(l, r, additive) {
+ const sieve = Array(r - l);
+ const quo = Array(r - l);
+ for (let i = 0; i < r-l; i++) {
+ quo[i] = l + i;
+ if (additive) {
+ sieve[i] = 0;
+ } else {
+ sieve[i] = 1;
+ }
+ }
+
+ return [sieve, quo];
+}
+ The initArrays function initializes arrays
+ that keep track of the value of the function we're sieving and
+ the running quotients.
+
+
+
Say we want to calculate the number of divisors function from 30
+ to 60. After initArrays runs, here's what the
+ sieve and quo arrays look like:
+
+
+
+ Initial values of sieve and quo
+
+
+
+
n
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
+
+
sieve
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
1
+
+
+
quo
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
+
+
+
+
Next, we'll call the following functions for every prime number in
+ the primes array. This is where the function spends most
+ of its time.
+
+
+
function sievePowers(p, l, r, f, sieve, quo, additive) {
+ let i = 1;
+ for (let q = p; q < r; q *= p) {
+ const y = f(p, i);
+ sieveMultiples(p, q, l, r, y, sieve, quo, additive);
+ i++;
+ }
+}
+
+function sieveMultiples(p, q, l, r, y, sieve, quo, additive) {
+ const low = Math.ceil(l / q);
+ for (let i = low * q; i < r; i += q) {
+ if (i % (p*q) === 0) {
+ continue;
+ }
+
+ if (additive) {
+ sieve[i - l] += y;
+ } else {
+ sieve[i - l] *= y;
+ }
+
+ quo[i - l] /= q
+ }
+}
+ These functions are where most of the computation
+ happens.
+
+
+
In short, these functions evaluate our function at prime powers
+ only, then update the values in the sieve array for
+ non-prime powers using the multiplicative property. We also update
+ the quo array by dividing out the factors we find along
+ the way.
+
+
Here's what the arrays will look like after sieving with 2:
+
+
+ Values of sieve and quo after sieving 2
+
+
+
+
n
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
+
+
sieve
+
2
+
1
+
6
+
1
+
2
+
1
+
3
+
1
+
2
+
1
+
4
+
1
+
2
+
1
+
3
+
1
+
2
+
1
+
5
+
1
+
2
+
1
+
3
+
1
+
2
+
1
+
4
+
1
+
2
+
1
+
+
+
quo
+
15
+
31
+
1
+
33
+
17
+
35
+
9
+
37
+
19
+
39
+
5
+
41
+
21
+
43
+
11
+
45
+
23
+
47
+
3
+
49
+
25
+
51
+
13
+
53
+
27
+
55
+
7
+
57
+
29
+
59
+
+
+
+
+
And here's what the arrays will look like after subsequently
+ sieving with 3:
+
+
+ Values of sieve and quo after sieving 3
+
+
+
+
n
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
+
+
sieve
+
4
+
1
+
6
+
2
+
2
+
1
+
9
+
1
+
2
+
2
+
4
+
1
+
4
+
1
+
3
+
3
+
2
+
1
+
10
+
1
+
2
+
2
+
3
+
1
+
8
+
1
+
4
+
2
+
2
+
1
+
+
+
quo
+
5
+
31
+
1
+
11
+
17
+
35
+
1
+
37
+
19
+
13
+
5
+
41
+
7
+
43
+
11
+
5
+
23
+
47
+
1
+
49
+
25
+
17
+
13
+
53
+
1
+
55
+
7
+
19
+
29
+
59
+
+
+
+
+
After we've gone through the entire primes array, the
+ two arrays will look like this:
+
+
+ Values of sieve and quo just before
+ calling unusuals
+
+
+
+
n
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
+
+
sieve
+
8
+
1
+
6
+
2
+
2
+
4
+
9
+
1
+
2
+
2
+
8
+
1
+
8
+
1
+
3
+
6
+
2
+
1
+
10
+
3
+
6
+
2
+
3
+
1
+
8
+
2
+
8
+
2
+
2
+
1
+
+
+
quo
+
1
+
31
+
1
+
11
+
17
+
1
+
1
+
37
+
19
+
13
+
1
+
41
+
1
+
43
+
11
+
1
+
23
+
47
+
1
+
1
+
1
+
17
+
13
+
53
+
1
+
11
+
1
+
19
+
29
+
59
+
+
+
+
At this point, there's one last step. Numbers can have up to one
+ prime factor greater than their own square root. When that happens,
+ they're called
+ unusual numbers.3
+ But since we only require the primes array to hold
+ primes up to the square root of r, any unusual number
+ will have one last prime factor that we never evaluated. Fortunately,
+ we can identify those numbers easily using the quo
+ array, so we just need to do one more pass through that array to
+ finalize everything.
+
+
+
function unusuals(f, sieve, quo, additive) {
+ for (let i = 0; i < quo.length; i++) {
+ const q = quo[i];
+ if (q === 1) {
+ continue;
+ }
+
+ if (additive) {
+ sieve[i] += f(q, 1);
+ } else {
+ sieve[i] *= f(q, 1);
+ }
+ }
+}
+ The last step is to finalize the values of any unusual
+ numbers (numbers n with a prime factor greater than
+ sqrt(n)).
+
+
+
+
Analysis
+
It's pretty easy to see that initArrays and
+ unusuals are
+ .4
+ Analyzing sievePowers and its impact is a bit more
+ involved.
+
+
Looking at sieveMultiples, its main loop performs a
+ fixed number of arithmetic operations
+
+ times. sieveMultiples, in turn, is called by
+ sievePowers at values of
+ ,
+ ,
+ ,
+ and so on until
+ exceeds . This means the total time
+ complexity of sievePowers is4
+
+
+
+
+
+
With this in mind, we can evaluate sieveSegment,
+ which calls sievePowers for every prime less than the
+ square root of r. Thanks to Euler, we know that
+ the sum of the reciprocal of primes up to
+ is asymptotic to
+ .
+ Therefore, the time complexity of sieveSegment is
+
+
+
+
+
+
This is the same complexity as the method given
+ here
+ for sieving the totient function.
+
+
+
+
+
Final thoughts
+
This all ends up being more lines of code than writing a single
+ sieve for, say, totient or the number of divisors. However, if you
+ want to sieve multiple functions, it can be tricky to make the right
+ modifications to an existing sieve to work with a different
+ function.
+
+
In contrast, with this approach, you can use the same routine and
+ simply pass a different function as an argument. For example, here's
+ all the functions available in the above demo:
Many multiplicative and additive functions have simple formulas
+ for prime powers, which makes adding a new function to sieve very
+ easy with this approach.
+
+
+
+
Footnotes
+
+
1 This is because the only numbers that divide
+ are
+ , ,
+ , ..., up to
+ , which is
+ total numbers.
+
+
2 You might be thinking at this point that factoring
+ is also not very efficient. Sit tight; this is only an
+ example to get you comfortable with multiplicative functions.
+ We're going to take advantage of this property later.
+
+
3 "Unusual" is actually a bit of a misnomer - most
+ numbers have a prime factor greater than their square root, so
+ unusual numbers are actually more common than so-called "usual"
+ numbers!
+
+
4 Well, technically unusuals and
+ sievePowers are also dependent on the time
+ complexity of the function we're sieving, but generally that
+ complexity isn't too bad, so I'm going to gloss over that bit.