Chrome’s transition to the Manifest V3 (MV3) extension APIs is almost over. Next year all Chrome extensions will be using the new APIs1. Some of the differences are tricky to handle. Since I spent a bunch of time porting the Adblock Plus and DuckDuckGo Privacy Essentials extensions to the MV3 APIs, I thought I’d write up a few notes and tips for other extension developers. This post is not a migration guide; I recommend reading the official docs if you’re just getting started. Instead, I aim to share some practical advice I learned the hard way.
declarativeNetRequest API
One of the largest changes with MV3 is the switch from chrome.webRequest to chrome.declarativeNetRequest for request blocking and redirection. The APIs work in a fundamentally different way. Instead of an onBeforeRequest event firing for each request which lets the extension decide ad-hoc what action to take, the extension must provide Chrome with declarativeNetRequest rules upfront. The browser is then in control of enforcing those rules, so the extension no longer gets to decide on a case-by-case basis what happens to requests.
There are two main types of declarativeNetRequest rules. Static rules, which are bundled with the extension at build time in JSON ruleset files, and dynamic/session rules that the extension can add or remove dynamically at run time. In other words, extensions can add or remove individual dynamic/session rules at will but not individual static rules. Extensions can only disable or enable whole static rulesets at runtime.
There are lots of different restrictions. For example, an extension can only have up to 5,000 dynamic/session rules at one time but up to 30,000 - 330,000 static rules enabled at one time. An extension can have up to 50 static rulesets, but only ten can be enabled at any time. Often, the choice between static and dynamic rules is made for you by those restrictions. If you’re unsure which type of rule to use, start by figuring out how many rules your extension will need to function - you can work backwards from there.
Chrome parses dynamic and session rules when the extension adds them (when the extension calls the declarativeNetRequest.updateDynamicRules or
declarativeNetRequest.updateSessionRules APIs). If there are parsing errors, Chrome surfaces them immediately. Chrome handles static rulesets differently. An extension’s bundled static rulesets can potentially contain many more rules than could ever be enabled at one time2, so parsing them all could be
very expensive (aka slow). Therefore Chrome only parses static rulesets when they are enabled. The problem is that rule errors cannot be
found in advance since the rules aren’t parsed in advance! Worse, most errors in static rules result in the invalid rule being
silently dropped - good for the user, but not so helpful during development! But wait, there is hope. Chrome parses all static rulesets at install time for unpacked extensions and surfaces any errors found. That results in a slower extension installation but helps developers spot any problems in their static rulesets. So when developing an extension with static rulesets, remember to test installing the extension unpacked (chrome://extensions -> “Developer mode” -> “Load unpacked”) and then check for errors in the extension listing. That way, you can be sure
your static rules are valid. If there are static rule errors, check again after fixing them - only the first N errors
are displayed.
While reasoning about how one or two declarativeNetRequest rules will match is simple enough, it soon gets tricky when you’re working with rulesets containing thousands of rules. Questions like “Which rule matches this request?” or “What even should happen here?!” get confusing to answer fast. That’s why I added the declarativeNetRequest.testMatchOutcome API. It’s a testing function that you can use to quickly check which enabled rules (if any) match a hypothetical request. You can use it manually from the extension’s background ServiceWorker (for unpacked extensions), but you can also use it in automated tests via Puppeteer. If you’d like an example, see the abp2dnr request matching tests and the corresponding Puppeteer interface code. Having unit tests that not only confirm your code is generating the correct rules but also that those rules will match requests as you intended is invaluable.
// Example from the abp2dnr request matching tests.
assert.deepEqual(
await testRequestOutcome(this.browser, {
url: "https://example.invalid/advert",
type: "image"
}),
"block"
);
When writing rules, the urlFilter or regexFilter conditions can be used to restrict matches to a particular website. One tip is that since I added the requestDomains/excludedRequestDomains conditions, domain conditions can often be used instead. The advantage of domain conditions (apart from being more efficient to match3) is that a single domain condition can list multiple domains to match (or to exclude). Sometimes it’s possible to combine rules that have different urlFilter conditions - but that are otherwise the same - by using a suitable requestDomains condition. That can have a major impact. For example, it led to a two thirds4 reduction in the number of declarativeNetRequest rules required for EasyList!
// Two similar rules, with a different urlFilter...
[
{
id: 1,
action: { type: "block" },
condition: {
urlFilter: "||domain1.example*/ad.js",
resourceTypes: ["script"]
}
},
{
id: 2,
action: { type: "block" },
condition: {
urlFilter: "||domain2.example*/ad.js",
resourceTypes: ["script"]
}
}
]
// ...can be combined into one rule like so:
[
{
id: 1,
action: { type: "block" },
condition: {
urlFilter: "/ad.js",
resourceTypes: ["script"],
requestDomains: ["domain1.example", "domain2.example"]
}
}
]
When using dynamic/session rules, I found that it’s often necessary to maintain a data structure that maps declarativeNetRequest rules to their corresponding representation in the extension. For example, for Adblock Plus, we needed to know which declarativeNetRequest rule(s) corresponded to which Adblock Plus filters. That way, we could remove the correct declarativeNetRequest rules when one of the filter lists removed a filter.
You can store the mapping data structure in session storage (chrome.storage.session) or local storage (chrome.storage.local), depending on if you’re working with session or dynamic rules. The problem is, how can you keep the data structure in sync with the active declarativeNetRequest rules? Suppose you add some dynamic rules and then update your mapping data structure in local storage to note the new rules… what happens if one of those operations fails? What happens if there is a power cut mid-update?
That is why I recommend ensuring your dynamic/session declarativeNetRequest rules are in sync with the extension’s state. You can do that by storing a state ID (e.g. a UUID or random number) in both your mapping data structure and with your declarativeNetRequest rules. Update that state ID every time you make declarativeNetRequest rule changes. If the state IDs diverge, you’ll know to rebuild all of the declarativeNetRequest rules to reflect the extension’s state. Storing a state ID in your mapping data structure is easy enough, but how do you store it with the declarativeNetRequest rules? One way is to create a special declarativeNetRequest rule with a known rule ID (e.g. 1) that contains the state ID as a part of its urlFilter condition. It’s worth ensuring the urlFilter will never match actual requests, so prefix it with an invalid domain. You can then read the rule back out of the declarativeNetRequest API and check the state ID against what you expect.
// Example state ID declarativeNetRequest rule.
[
{
id: 1,
action: { type: "block" },
condition: {
urlFilter: "||state-id.invalid/8f084278-760a-4ee7-b235-4753e4b3210f"
}
}
]
Sometimes maintaining a mapping data structure for your rules is overkill. If your use case is simple enough that pre-determined rule IDs or rule ID ranges are sufficient, then, by all means, skip the above!
Lastly, if you’d like some declarativeNetRequest inspiration, both the Adblock Plus (abp2dnr) and DuckDuckGo (ddg2dnr) ruleset generation libraries are Open Source and available on GitHub.
Background ServiceWorker and extension state
I touched on this above, but a pain point when migrating to the MV3 APIs is the switch to background ServiceWorkers and state management. With the old Chrome Manifest V2 (MV2) APIs, extensions have an invisible background “page” that Chrome creates when the extension starts and keeps running until the user disables the extension or closes the browser entirely5. That meant you could, for example, process a bunch of data once when the extension started and then refer to the processed data later when you needed it. It also meant that it was “safe” to store the extension’s state in local variables - so long as the state didn’t need to persist through extension restart. In particular, this meant that an event listener could write some state to a local variable, ready for the state to be read back in the future from another event listener.
That has all changed with MV3 and the switch to background ServiceWorkers. Chrome can kill or restart the background ServiceWorker at any point, often after only seconds. State in local variables is lost when that happens. Data needs to be reprocessed. State shared between event listeners is lost. This poses problems for extension developers migrating their extensions to MV3, and I recommend checking out the official background ServiceWorker migration guide before anything else. But here are some tricks:
Chrome can restart an extension’s background ServiceWorker at any time. In theory, you have no control over that, but in practice, there are some things you can do. Most importantly, you can keep the background ServiceWorker running for around five minutes6 by opening long-lived messaging connections. If your extension has a content script that needs to send messages to the background anyway, I recommend using long-lived connections (aka chrome.runtime.connect) instead of the simpler messaging API (aka chrome.runtime.sendMessage). It’s more efficient if you’re sending a lot of data, plus it will keep your background ServiceWorker running while the website is open and the content script is running. The content script can also listen for the disconnect event of the long-lived messaging connection, which will fire when Chrome kills the background ServiceWorker. At that point, the content script can re-open the connection, ensuring the background ServiceWorker is restarted in the process - and the clock starts again.
Long-lived messaging connections are generally superior to the simpler one-time messaging API, but they are less ergonomic to use. That’s why I created the tiny webext-messaging library. It provides a much simpler way to do extension messaging, and you get long-lived connections (and the above advantages) for free.
// Example webext-messaging use from a content script.
import { addConnection, dispatch, port } from "webext-messaging";
// Start listening for incoming messages and set up a long-lived connection.
chrome.runtime.onMessage.addListener(dispatch);
addConnection(chrome.runtime.connect());
// Send a random number to the background through the long-lived messaging
// connection.
port.post("randomNumber", { randomNumber: Math.random() });
There’s one other thing to note about the background ServiceWorker’s lifespan. While you’re inspecting the background ServiceWorker (e.g. have the background console open), the ServiceWorker will persist much longer (possibly forever). That makes
testing the background ServiceWorker lifespan tricky since you can’t view errors as they show up in the console until after the ServiceWorker was restarted, but
by then, the errors are lost! What can you do? Well, instead of using the debugger or normal logging, use console.error. Take care to include the current time with logged errors and to keep them short. Then, if you click the “Errors” button
on the chrome://extensions page for your extension, you’ll get a list of your recently logged errors. That’s enough to
figure out what’s going on without using the usual developer tools. But note if you log too many errors, the older ones are lost. So keep
noting them down as they come in, or take care not to log too many.
When Chrome starts an extension’s background ServiceWorker, if the background ServiceWorker throws an exception, Chrome will give up, and the background ServiceWorker will become inactive. That can make debugging the cause of the exception hard. When that happens, I usually remove all code from the background script, restart the extension, open the background console, and then paste the code into the console. Chrome will pause as the code is parsed and run, but after that, it will display details about what caused the exception in the console.
Finally, be extra careful when using any APIs that persist state in their own way. For example, if you naively add a context menu item when the background ServiceWorker first starts, the background ServiceWorker will attempt to re-add that context menu item every time it restarts. Another pernicious example is the chrome.alarms API. Check that alarms don’t already exist before adding them. Otherwise, if you set an alarm for five minutes time, the background ServiceWorker will restart before it fires. At that point, the alarm is recreated (no errors) for five minutes time again. This repeats ad infinitum, and five minutes never comes! See wOxxOm’s StackOverflow post on the topic.
/**
* Add a context menu item, ignoring the error if it already exists.
* @param {Object} createProperties
* See https://developer.chrome.com/docs/extensions/reference/contextMenus/#type-create-createProperties
* @returns {Promise}
*/
function createContextMenu(createProperties) {
return new Promise((resolve, reject) => {
chrome.contextMenus.create(createProperties, () => {
const lastError = chrome.runtime.lastError
if (lastError && lastError.message &&
!lastError.message.startsWith("Cannot create item with duplicate id")) {
reject(lastError);
} else {
resolve();
}
});
});
}
/**
* Create an alarm, but only if it doesn't already exist.
* @param {string} name
* @param {Object} alarmCreateInfo
* See https://developer.chrome.com/docs/extensions/reference/alarms/#type-AlarmCreateInfo
* @returns {Promise}
*/
async function setAlarm(name, alarmCreateInfo) {
const existingAlarm = await chrome.alarms.get(name);
if (!existingAlarm)
return await chrome.alarms.create(name, alarmCreateInfo);
}
Learn more
- Official introduction to Chrome MV3
- The official Chrome MV3 migration guide
- declarativeNetRequest API docs
- The venerable wOxxOm’s posts on StackOverflow
- The chrome-extensions Google group (including a discussion about this post)
- declarativeNetRequest Chromium issues
- The declarativeNetRequest Chromium code
- Mentioned projects: abp2dnr, ddg2dnr, and webext-messaging.
Notes
-
Though as the joke goes, “Chrome MV3 is always six months away”… ↩︎
-
From memory, the theoretical limit is 50 x 330,000, so > 16 million rules! ↩︎
-
The domain-matching algorithm is cool. First off, domain conditions are sorted when the ruleset is indexed. The sort order is first by domain length, then alphabetical. During matching, for short lists of domains (<= 5), each domain is just checked in turn. But for longer lists of domains, a binary search is used. Domains longer than the desired domain can be quickly skipped past thanks to the sort order. If the binary search for a domain fails, its first subdomain is removed from the domain, and the search is repeated. When the search is repeated, the left-bound can start from its previous position since we know that a domain minus a subdomain is always shorter than the original domain. Searches are repeated in this way until either a match is found or all subdomains have been removed. ↩︎
-
At the time, that was a reduction from 28,191 down to 9,592 declarativeNetRequest rules! ↩︎
-
Well, that’s a simplification. There are some other times when the background page is restarted, for example, when the extension is updated… but you get the idea. ↩︎
-
That’s an undocumented implementation detail of the browser and is subject to change. Nevertheless, it’s worth considering. Opening long-lived messaging connections in this way can reduce the number of background ServiceWorker restarts by an order of magnitude. If you’re interested, we discussed this in more detail in the comments on issue 1146434 ↩︎
My mission was to figure out the email and password used, login and then update her email and password to the new ones she can remember. All I had to go on was the rough structure of the email address. She only remembered it was a hotmail.co.uk address and the some of the words used.
