Sums of Powers
Posted
This page is all about how we can efficiently compute large sums of powers:
Demo
Note that large exponents will take longer to calculate, but your upper bound can be quite large - even thousands of digits!
How it works
A straightforward approach to computing a sum like this would be something like
function sumOfPowers(n, p) {
let total = 0;
for (let k = 1; k <= n; k++) {
total += Math.pow(k, p);
}
return total;
}
This works fine for small values of , but this algorithm requires we perform exponentiations and additions. In other words, the time to run grows as increases, which makes computing sums with a high upper bound take a long time. This is not the algorithm the demo uses.
In fact, the algorithm used in the demo looks quite different. It's not that many lines of code, but it's relying on some nifty (and slightly tricky) mathematics. If all you do is read the code, it's hard to believe it works, so rather than simply presenting it right now, I want to introduce it piece by piece and explain it as we go.
Special cases
If we want to compute a sum where each term's exponent is 1, it turns out there's a simple formula we can use (read more about why it works here):
The nice thing about this formula is the number of operations we perform is no longer dependent on our upper bound - we'll always perform one addition, one multiplication, and one division. That means we can easily determine the sum of the numbers from 1 to some extreme limit like 1,000,000,000,000,000 much faster than with the naive algorithm.
There are similar formulas for computing sums when our exponent is 2 or 3:
It turns out there's a pattern that lets us construct polynomials like these for any exponent, but the pattern isn't super obvious.
Faulhaber's formula
Faulhaber's formula gives the general pattern these polynomials follow. The most commonly given version of the formula involves a special sequence called the Bernoulli numbers.
Of course, there are algorithms to compute the Bernoulli numbers, but since they're rational numbers, there are tradeoffs to consider. If we use floats to store the sequence, we need to be mindful of rounding errors; additionally, after the 258th term, the Bernoulli numbers become too large to store in a 64 bit float. This means to compute sums of large powers precisely, we'll want a rational data type, which not every language has handy.
I suppose if you need the coefficients of the polynomial for some reason, this is the formula for you, but if all you want is the sum of powers for some and , we can sidestep computing the Bernoulli numbers by taking another approach.
Stirling numbers
To begin explaining this alternate approach, we need to introduce the Stirling numbers of the second kind (as you might expect, there are Stirling numbers of the first kind too, but they aren't super relevant here). These numbers give the number of ways to partition a set of elements into non-empty subsets.
For instance, there's 6 ways to partition the letters a, b, c, and d into 3 non-empty subsets:
- {{a,b},{c},{d}}
- {{a,c},{b},{d}}
- {{a},{b,c},{d}}
- {{a,d},{b},{c}}
- {{a},{b,d},{c}}
- {{a},{b},{c,d}}
The Stirling numbers of the second kind (or just "Stirling numbers" from here on) happen to have a nice recursive formula that we can apply to compute them.
To compute sums of powers for a given exponent , we'll need to know
(We'll learn why in a bit.) Here's a function to get all those values.
function stirling2(p) {
if (p === 0) {
return [1n];
}
let s = [0n];
for (let i = 1; i < p + 1; i++) {
s.push(1n);
for (let j = i - 1; j > 0; j--) {
s[j] = BigInt(j) * s[j] + s[j-1];
}
}
return s;
}
This algorithm is quadratic in the value of , which isn't great, but it's simple to implement (and I don't know of any faster way).
Stirling numbers calculator
Be warned that these numbers can grow quite large, even for small values of .
Connection to exponents
To understand what the Stirling numbers have to do with our problem, it helps to first introduce the notation for falling factorials.
It turns out there is a very elegant identity relating exponents, Stirling numbers, and falling factorials:1
We can apply this formula to every term of our power sum and rearrange to get
This might seem more complicated at first, but despite the similar notation to normal exponentiation, it's much easier to compute sums of falling factorials. For ,
(For , the sum is just .)2 This lets us simplify.3
The first term is 0 unless , which is a trivial case, so we really only need to consider the remaining terms.
Implementation
This formula might still seem a little complicated, but you may be surprised by how simple it is to implement. We already have a method to compute the Stirling numbers we need. With those values available, we can write a simple loop to compute each term and add to our total.
function faulhaber(n, s) {
const limit = BigInt(s.length);
if (limit === 1n) {
return n;
}
let q = n + 1n;
let total = 0n;
for (let k = 1n; k < limit; k++) {
q *= n + 1n - k;
total += s[k] * q / (k + 1n);
}
return total;
}
Note that the loop is dependent on and not , which allows us to compute sums with extremely large upper bounds very quickly.
Final thoughts
This approach spends most of its time computing the Stirling numbers. Additionally, it seems like the performance is largely hampered by how large the Stirling numbers get - computing them with a modulus is considerably quicker. The demo is a little clever, in that it only recomputes the Stirling numbers if the value of the exponent changes.
If you would rather use the more commonly known Faulhaber formula, you'll need to compute the Bernoulli numbers. Brent and Harvey give a simple algorithm that is as efficient as how we're calculating the Stirling numbers. The needed binomial coefficients can also be calculated easily.
Furthermore, Brent and Harvey give another algorithm for the Bernoulli numbers that is even more optimized (although it is considerably more complicated). I believe using that algorithm in conjunction with the more traditional formula would be more efficient than the Stirling number approach.
However, in other ways, I like this formula better. For one thing, whichever algorithm you use, Bernoulli numbers are a bit harder to calculate, and require more lines of code. You'll have to compute the binomial coefficients, as well.
One nice thing about the Stirling approach is the terms are integers throughout. I think that makes this algorithm a little better for modular calculations.
Footnotes
- 1 For proofs of this identity, see chapter 1.9 of Enumerative Combinatorics by Stanley and chapter 6.1 of Concrete Mathematics by Graham, Knuth, and Patashnik.
- 2 See chapter 2.6 of Concrete Mathematics for an explanation of this identity.
- 3 A version of this formula is also mentioned in chapter 6.5 of Concrete Mathematics.