Why We Added package.json Support to Deno
The latest release of Deno introduced a significant change: providing enhanced Node and NPM compatibility through package.json support. This update led to questions regarding whether our priorities have shifted, as Deno has long been associated with forging a path distinct from Node. Indeed, package.json was explicitly mentioned as a regret in the first Deno presentation. Thus, many users were surprised by this development.
In this post, we’ll address these concerns, share insights into our evolving thoughts, and outline our future goals.
Simplifying and accelerating JavaScript development
Deno was created to simplify and accelerate JavaScript development. Core features include native TypeScript support, built-in tooling, zero configuration by default, and web standard APIs. Historically, this has also meant a minimalist, decentralized module system separate from the NPM ecosystem, based on web standard HTTP imports.
Deno’s main goal is to make programming both easy and fast. TypeScript, web standard APIs, and many other features contribute to this aim, but it has become increasingly unclear whether Deno’s minimalist module system is making programming easy and fast.
The dependency management dream
The JavaScript ecosystem’s reliance on a single centralized module registry conflicts with the web’s decentralized nature. With the introduction of ES modules, there is now a standard for loading remote modules, and that standard looks nothing like how module loading through NPM works. Deno implements loading ES modules through HTTP URLs - allowing anyone to host code on any domain simply by running a file server.
This approach offers great benefits.
Single file programs
can access robust tooling without needing a dependency manifest (a file that
lists dependencies and where to get them, for example package.json
,
import_map.json
, Cargo.toml
, or Gemfile
). Also Deno programs benefit from
smaller downloads, as only the required files are downloaded instead of large
npm package tarballs containing unnecessary artifacts. Additionally, using HTTP
for module linking opens up opportunities for innovation, such as displaying
HTML documentation based on the accept header, as demonstrated in
the deno.land registry.
We have been committed in our pursuit of using HTTP imports as the backbone of the Deno module system. We’ve built all sorts of useful features into the deno.land registry like publish-on-tag GitHub integration, immutable caching, and auto-generated documentation. Third-party registries like skypack and esm.sh have made individual files in NPM packages accessible as ES modules using HTTP imports. We’ve established conventions like deps.ts for consolidating dependencies in one convenient location.
Lingering Issues
Depending on the context, module URLs like
https://deno.land/[email protected]/uuid/mod.ts
can sometimes be too specific. Not
only do they identify the package (std/uuid/mod.ts
), but they also specify the
version (0.179.0
) and the server from which to fetch it (deno.land
). Issues
arise when a program contains similar yet slightly different modules - if
another module imports a URL referencing a slightly different version, like
https://deno.land/[email protected]/uuid/mod.ts
, both module versions will be
included in the module graph despite being almost the same code. This is known
as the
Duplicate Dependency Problem
(follow the link for a more concrete example of this problem).
In library code we’ve developed
patterns like deps.ts
for managing remote dependencies. However, deps.ts
is not particularly
ergonomic - it necessitates flattening and re-exporting every symbol one depends
on. (This could be improved in the future through the
Module Declarations and
Import Reflection TC39
proposals.) deps.ts
is generally more verbose and syntactically cluttered than
package.json with bare specifiers.
Ideally Import Maps could solve this issue by allowing users to employ bare specifiers in their code and specify the URL only in the import map. However, import maps are not composable: a published library cannot use bare specifiers and an import map simultaneously, as the top-level project importing that library cannot compose the library’s import map with its own.
Transpile servers like esm.sh and skypack work well for importing some NPM modules into Deno, but they have inherent limitations. For example, if an NPM module loads a data file at runtime, these transpile servers will be unable to serve a compatible version. Issues like this result in a subpar developer experience.
The power of bare specifiers
Import statements using bare specifiers, such as the one below, are concise and familiar:
import express from "express";
Bare specifiers ("express"
) provide a usefully ambiguous reference to the
dependency, which allows the Duplicate Dependency Problem to be resolved through
semver resolution. However, if libraries are written using bare specifiers,
there must be a dependency manifest to clarify what these bare specifiers refer
to.
Simplifying and accelerating with compatibility
We want to enable Deno users to work more efficiently than they would with Node. Developers want to import a library and make use of it without hassle. Hence our npm specifier support. Beyond libraries, developers want to run existing Node projects directly in Deno. Hence the newly added package.json support.
Deno’s backwards compatibility keeps less ideal legacy features of Node and NPM
at a distance. For example, Deno only supports CommonJS through NPM imports.
Also Deno does not let the user refer to built-in Node modules using bare
specifiers ("fs"
) outside of NPM dependencies, rather they must use
"node:fs"
. Or for instance, in Deno setTimeout
will return a number per the
Web standard
(unlike Node).
Deno will not run post-install scripts arbitrarily and enforces user space
security permissions. Backwards compatibility doesn’t mean the JavaScript
ecosystem cannot evolve and improve.
It’s crucial to note that Deno will always support linking to code through
URLs, continuing to operate like browsers with HTTP imports. Using https:
URLs is now just one method of linking code in Deno. Since Deno 1.28 we have
npm:
URLs, and it’s easy to envision others that could improve developer
velocity, such as github:
URLs.
A new major version
We are working towards a new major release of Deno in the coming months. The major theme for this upcoming release is encouraging the use of bare specifiers in Deno workflows.
Although Deno has fantastic backwards compatibility with NPM modules, we will not be recommending that Deno code distributed for Deno users be published there. If one needs to share code with Node and NPM projects we recommend our official Deno to NPM compiler, DNT, that outputs high quality NPM packages containing transpiled Node compatible JavaScript and TypeScript declaration files derived from the original TypeScript. However to live in a truly TypeScript-first world, it’s best to distribute and link to the actual TypeScript code, not some compiled output.
In order to solve the duplicate dependency problem and improve the ergonomics of
using the TypeScript-first Deno module registry, the next major version of Deno
will introduce a deno:
URL scheme. By using these special URLs, rather than
HTTP URLs, the Deno runtime is able to do semver resolution and module
deduplication. Furthermore it eliminates the need to write out a full URL.
For example, importing Oak will look like this:
import oak from "deno:oak@12";
Note that only the major version is specified - the runtime will have special semver resolution logic to find the right version of Oak that matches.
The next major version will also advocate the use of import maps, as a modern
alternative to package.json workflows. Import maps in Deno are specified in the
auto-discovered deno.json
configuration file. Here’s an example of what it
will look like:
{
"imports": {
"oak": "deno:oak@12",
"chalk": "npm:chalk@5"
}
}
This configuration enables any code to utilize the "oak"
and "chalk"
bare
specifiers throughout the code. Oak comes from the Deno registry, Chalk comes
from the NPM registry. Imports in the code would simply be:
import oak from "oak";
import chalk from "chalk";
Whether using modern import map workflows or Node.js package.json workflows, Deno aims to be a reliable tool developers reach for to accelerate their work. JavaScript, the world’s default programming language, deserves this continued effort to improve its ecosystem and tooling.
Keep up with the latest developments by following @deno_land on Twitter.