🌈 Editor's Note:

Please read PART 1 to understand how/when to use a Promise cache.

Since my previous article, many people have asked me how to approach cache invalidation. It's a fair question, given its prominence among the top two hardest challenges in computer science! We're going to look at two common strategies: TTL and LRU. Both have their trade-offs; unfortunately, there are still no silver bullets in software development.

But first, let's begin with the same cacher module that we wrote in PART 1, only with a few added tweaks to help us easily hook in the various implementations and compare them more clearly:

type CB = () => Promise<any>;
const cache: Record <string, Promise<any>> = {};

function isValid(id) {
    return cache[id] !== undefined
}
function whenAssigned(id) { }
function whenSettled(id) {
    delete cache[id];
}

export default async function cacher(id: string, cb: CB) {
  if (!isValid(id)) {
    cache[id] = cb().finally(() => whenSettled(id));
    whenAssigned(id);
  }
  return cache[id];
}

We'll focus mainly on these three "lifecycle" functions (to save on some redundant code below):

• isValid(id)

• whenAssigned(id)

• whenSettled(id)

TTL

Time-To-Live is just a simple timer that we put on our cached data; i.e. "After {X time units}, invalidate the cache and fetch its data again." Pretty straightforward, right? The main trade-offs here are:

1. We can decide how long the data is stale (max)

2. We can't decide how big the cache size gets (max)

If you're not very concerned with #2 (for example, a blog with limited post API endpoints), then it's nice to know for example that the blog will display article updates in "5 minutes or less". That's a TTL.

How do we implement this? One way would be to use setTimer, but we'd rather avoid using that if possible because they can get messy at runtime. Instead, we'll use another object variable to manage the state of each cached datum for us; let's called is expiresAt:

const cache: Record <string, Promise<any>> = {};
const expiresAt: Record <string, number> = {};
const staleTimeoutMs: number = 5 * 60 * 1000; // 5 minutes
// snip!

We also added a staleTimeoutMs just to make our code more readable. (TIP: Don't forget to include the time unit in the variable name!) The way we use these is such that every time we store a cached datum, we additionally record the future time value when we want the cached datum to be considered stale/invalid via whenAssigned:

const cache: Record <string, Promise<any>> = {};
const expiresAt: Record <string, number> = {};
const staleTimeoutMs: number = 5 * 60 * 1000; // 5 minutes

function isValid(id) {
    // snip!
}
function whenAssigned(id) {
    expiresAt[id] = Date.now() + staleTimeoutMs;
}
function whenSettled(id) { }

// snip! function cacher() {...}

Finally, we need to check the cached datum's expiration time (if it exists) via isValid:

const cache: Record <string, Promise<any>> = {};
const expiresAt: Record <string, number> = {};
const staleTimeoutMs: number = 5 * 60 * 1000; // 5 minutes

function isValid(id) {
    if (cache[id] &&
        Date.now() < expiresAt[id])
    {
        return true;
    }
    return false;
}
function whenAssigned(id) {
    expiresAt[id] = Date.now() + staleTimeoutMs;
}
function whenSettled(id) { }

// snip! function cacher() {...}

And that's it! Remember, this strategy doesn't proactively clean up any stale data as soon as it expires. But also remember that we're not concerned with the cache memory size here, so it's OK 😁. (Or, use setTimer to help fix this issue but be careful.)

However, what if instead we are concerned with our cache size ballooning/exploding on the server? Then LRU is probably a better option to implement...

LRU

Least-Recently-Used is a lot like that children's bedtime story where each of the kids keeps rolling out of bed -- the last one always gets kicked out! In our case, the "last kid" is the id + cached datum that was accessed the longest time ago, i.e. the "least recently". The main trade-offs here are:

1. We can't decide how long the data is stale (max)

2. We can decide how big the cache size gets (max)

How do we implement this? Similar to before, we'll use another array variable to manage the state of each cached datum's id for us; let's called it priorityStack:

const cache: Record <string, Promise<any>> = {};
const priorityStack: string[] = [];
const priorityMaxLength: number = 5;
// snip!

We also added a priorityMaxLength just to make our code more readable. (Good job; you learn fast! 🏆) The way we use these is such that every time we cache a datum, we bump its id's spot in priorityStack to the front via whenAssigned:

const cache: Record <string, Promise<any>> = {};
const priorityStack: string[] = [];
const priorityMaxLength: number = 5;

function isValid(id) {
    // snip!
}
function whenAssigned(id) {
    priorityStack = priorityStack.filter(cid => cid !== id);
    priorityStack.unshift(id);
    if (priorityStack.length > priorityMaxLength) {
        const xid = priorityStack.pop();
        delete cache[xid];
    }
}
function whenSettled(id) { }

// snip! function cacher() {...}

First, we remove the id from priorityStack (if it existed in there). Then, we add the id onto the front of the stack. Finally, we trim the stack length down to our max size and remove any extra cached datum. (Note: this is how we control the memory size.)

Finally, we need to check for the cached datum's id in the stack via isValid:

const cache: Record <string, Promise<any>> = {};
const priorityStack: string[] = [];
const priorityMaxLength: number = 5;

function isValid(id) {
    return cache[id] && priorityStack.includes(id);
}
function whenAssigned(id) {
    priorityStack = priorityStack.filter(cid => cid !== id);
    priorityStack.unshift(id);
    if (priorityStack.length > priorityMaxLength) {
        const xid = priorityStack.pop();
        delete cache[xid];
    }
}
function whenSettled(id) { }

// snip! function cacher() {...}

And that's it! Remember, the LRU strategy does not care how long the data is cached. So for our example, if you have less than five (active) endpoints, your data would never invalidate! It's like having a bed bigger than all the kids -- no one gets kicked out, ever.

Hopefully, you've enjoyed comparing the TTL and LRU strategies and implementations, and contrasting their code to learn about how to think in terms of trade-offs. (For example, note that both implementations are "pull-based"--meaning cacher must be executed for something to happen.) Understanding these advanced techniques and concepts will give you additional tools for many other async situations, so they're 💯 worth mastering!

You can follow me on Twitter and YouTube. Let’s continue interesting conversations about Sr. JS development together!