Dynamic languages are useful tools. Scripting allows users to rapidly and succinctly tie together complex systems and express ideas without worrying about details like memory management or build systems. In recent years programming languages like Rust and Go have made it much easier to produce sophisticated native machine code; these projects are incredibly important developments in computer infrastructure. However, we claim it is still important to have a powerful scripting environment that can address a wide range of problem domains.
JavaScript is the most widely used dynamic language, operating on every device with a web browser. Vast numbers of programmers are fluent in JavaScript and much effort has been put into optimizing its execution. Through standards organizations like ECMA International, the language has been carefully and continuously improved. We believe JavaScript is the natural choice for dynamic language tooling; whether in a browser environment or as standalone processes.
Our original undertaking in this area, Node.js, proved to be a very successful software platform. People have found it useful for building web development tooling, building standalone web servers, and for a myriad of other use-cases. Node, however, was designed in 2009 when JavaScript was a much different language. Out of necessity, Node had to invent concepts which were later taken up by the standards organizations and added to the language differently. In the presentation Design Mistakes in Node, this is discussed in more detail. Due to the large number of users that Node has, it is difficult and slow to evolve the system.
With the changing JavaScript language, and new additions like TypeScript, building Node projects can become an arduous endeavor, involving managing build systems and other heavy handed tooling that takes away from the fun of dynamic language scripting. Furthermore the mechanism for linking to external libraries is fundamentally centralized through the NPM repository, which is not inline with the ideals of the web.
We feel that the landscape of JavaScript and the surrounding software infrastructure has changed enough that it was worthwhile to simplify. We seek a fun and productive scripting environment that can be used for a wide range of tasks.
A Web Browser for Command-Line Scripts
Deno is a new runtime for executing JavaScript and TypeScript outside of the web browser.
Deno attempts to provide a standalone tool for quickly scripting complex functionality. Deno is (and always will be) a single executable file. Like a web browser, it knows how to fetch external code. In Deno, a single file can define arbitrarily complex behavior without any other tooling.
import { serve } from "https://deno.land/[email protected]/http/server.ts";
for await (const req of serve({ port: 8000 })) {
req.respond({ body: "Hello World\n" });
}Here a complete HTTP server module is added as a dependency in a single line.
There are no additional configuration files, there is no install to do
beforehand, just deno run example.js.
Also like browsers, code is executed in a secure sandbox by default. Scripts
cannot access the hard drive, open network connections, or make any other
potentially malicious actions without permission. The browser provides APIs for
accessing cameras and microphones, but users must first give permission. Deno
provides analogous behaviour in the terminal. The above example will fail unless
the --allow-net command-line flag is provided.
Deno is careful to not deviate from standardized browser JavaScript APIs. Of course, not every browser API is relevant for Deno, but where they are, Deno does not deviate from the standard.
First Class TypeScript Support
We want Deno to be applicable to a wide range of problem domains: from small one-line scripts, to complex server-side business logic. As programs become more complex, having some form of type checking becomes increasingly important. TypeScript is an extension of the JavaScript language that allows users to optionally provide type information.
Deno supports TypeScript without additional tooling. The runtime is designed
with TypeScript in mind. The deno types command provides type declarations for
everything provided by Deno. Deno’s standard modules are all written in
TypeScript.
Promises All The Way Down
Node was designed before JavaScript had the concept of Promises or async/await.
Node’s counterpart to promises was the EventEmitter, which important APIs are
based around, namely sockets and HTTP. Setting aside the ergonomic benefits of
async/await, the EventEmitter pattern has an issue with back-pressure. Take a
TCP socket, for example. The socket would emit “data” events when it received
incoming packets. These “data” callbacks would be emitted in an unconstrained
manner, flooding the process with events. Because Node continues to receive new
data events, the underlying TCP socket does not have proper back-pressure, the
remote sender has no idea the server is overloaded and continues to send data.
To mitigate this problem, a pause() method was added. This could solve the
problem, but it required extra code; and since the flooding issue only presents
itself when the process is very busy, many Node programs can be flooded with
data. The result is a system with bad tail latency.
In Deno, sockets are still asynchronous, but receiving new data requires users
to explicitly read(). No extra pause semantics are necessary to properly
structure a receiving socket. This is not unique to TCP sockets. The lowest
level binding layer to the system is fundamentally tied to promises - we call
these bindings “ops”. All callbacks in Deno in some form or another arise from
promises.
Rust has its own promise-like abstraction, called Futures. Through the “op” abstraction, Deno makes it easy to bind Rust future-based APIs into JavaScript promises.
Rust APIs
The primary component that we ship is the Deno command-line interface (CLI). The CLI is the thing that is version 1.0 today. But Deno is not a monolithic program, but designed as a collection of Rust crates to allow integration at different layers.
The deno_core crate is a very bare bones version of Deno. It does not have dependencies on TypeScript nor on Tokio. It simply provides our Op and Resource infrastructure. That is, it provides an organized way of binding Rust futures to JavaScript promises. The CLI is of course built entirely on top of deno_core.
The rusty_v8 crate provides high quality Rust bindings to V8’s C++ API. The API tries to match the original C++ API as closely as possible. It’s a zero-cost binding - the objects that are exposed in Rust are exactly the object you manipulate in C++. (Previous attempts at Rust V8 bindings forced the use of Persistent handles, for example.) The crate provides binaries that are built in Github Actions CI, but it also allows users to compile V8 from scratch and adjust its many build configurations. All of the V8 source code is distributed in the crate itself. Finally rusty_v8 attempts to be a safe interface. It’s not yet 100% safe, but we’re getting close. Being able to interact with a VM as complex as V8 in a safe way is quite amazing and has allowed us to discover many difficult bugs in Deno itself.
Stability
We promise to maintain a stable API in Deno. Deno has a lot of interfaces and
components, so it’s important to be transparent about what we mean by “stable”.
The JavaScript APIs that we have invented to interact with the operating system
are all found inside the “Deno” namespace (e.g. Deno.open()). These have been
carefully examined and we will not be making backwards incompatible changes to
them.
All functionality which is not yet ready for stabilization has been hidden
behind the --unstable command-line flag. You can see the documentation for the
unstable interfaces
here.
In subsequent releases, some of these APIs will be stabilized as well.
In the global namespace you’ll find all sorts of other objects (e.g.
setTimeout() and fetch()). We’ve tried very hard to keep these interfaces
identical to those in the browser; but we will issue corrections if we discover
inadvertent incompatibilities. The browser standards define these interfaces,
not us. Any corrections issued by us are bug fixes, not interface changes. If
there is an incompatibility with a browser standard API, that incompatibility
may be corrected before a major release.
Deno also has many Rust APIs, namely the deno_core and rusty_v8 crates. None of these APIs are at 1.0 yet. We will continue to iterate on them.
Limitations
It’s important to understand that Deno is not a fork of Node - it’s a completely new implementation. Deno has been under development for just two years, while Node has been under development for over a decade. Given the amount of interest in Deno, we expect it to continue to evolve and mature.
For some applications Deno may be a good choice today, for others not yet. It will depend on the requirements. We want to be transparent about these limitations to help people make informed decisions when considering to use Deno.
Compatibility
Unfortunately, many users will find a frustrating lack of compatibility with existing JavaScript tooling. Deno is not compatible, in general, with Node (NPM) packages. There is a nascent compatibility layer being built at https://deno.land/std/node/ but it is far from complete.
Although Deno has taken a hardline approach to simplifying the module system, ultimately Deno and Node are pretty similar systems with similar goals. Over time, we expect Deno to be able to run more and more Node programs out-of-the-box.
HTTP Server Performance
We continuously track the performance of Deno’s HTTP server. A hello-world Deno HTTP server does about 25k requests per second with a max latency of 1.3 milliseconds. A comparable Node program does 34k requests per second with a rather erratic max latency between 2 and 300 milliseconds.
Deno’s HTTP server is implemented in TypeScript on top of native TCP sockets. Node’s HTTP server is written in C and exposed as high-level bindings to JavaScript. We have resisted the urge to add native HTTP server bindings to Deno, because we want to optimize the TCP socket layer, and more generally the op interface.
Deno is a proper asynchronous server and 25k requests per second is quite enough for most purposes. (If it’s not, probably JavaScript is not the best choice.) Furthermore, we expect Deno to generally exhibit better tail latency due to the ubiquitous use of promises (discussed above). All that said, we do believe there are more performance wins to be had in the system, and we hope to achieve that in future releases.
TSC Bottleneck
Internally Deno uses Microsoft’s TypeScript compiler to check types and produce JavaScript. Compared to the time it takes V8 to parse JavaScript, it is very slow. Early on in the project we had hoped that “V8 Snapshots” would provide significant improvements here. Snapshots have certainly helped but it’s still unsatisfyingly slow. We certainly think there are improvements that can be done here on top of the existing TypeScript compiler, but it’s clear to us that ultimately the type checking needs to be implemented in Rust. This will be a massive undertaking and will not happen any time soon; but it would provide order of magnitude performance improvements in a critical path experienced by developers. TSC must be ported to Rust. If you’re interested in collaborating on this problem, please get in touch.
Plugins / Extensions
We have a nascent plugin system for extending the Deno runtime with custom ops. However this interface is still under development and has been marked as unstable. Therefore, accessing native systems beyond that which is provided by Deno is difficult.
Acknowledgements
Many thanks to the many contributors who helped make this release possible. Especially: @kitsonk who has had a massive hand in many parts of the system, including (but not limited to) the TypeScript compiler host, deno_typescript, deno bundle, deno install, deno types, streams implementation. @kevinkassimo has contributed countless bug fixes over the whole history of the project. Among his contributions are the timer system, TTY integration, wasm support. Deno.makeTempFile, Deno.kill, Deno.hostname, Deno.realPath, std/node’s require, window.queueMircotask, and REPL history. He also created the logo. @kt3k implemented the continuous benchmark system (which has been instrumental in almost every major refactor), signal handlers, the permissions API, and many critical bug fixes. @nayeemrmn contributes bug fixes in many parts of Deno, most notably he greatly improved the stack trace and error reporting, and has been a forceful help towards the stabilizing the APIs for 1.0. @justjavac has contributed many small but critical fixes to align deno APIs with web standards and most famously he wrote the VS Code deno plugin. @zekth has contributed a lot of modules to std, among them the std/encoding/csv, std/encoding/toml, std/http/cookies, as well as many other bug fixes. @axetroy has helped with all things related to prettier, contributed many bug fixes, and has maintained the VS Code plugin. @afinch7 implemented the plugin system. @keroxp implemented the websocket server and provided many bug fixes. @cknight has provided a lot of documentation and std/node polyfills. @lucacasonato built almost the entire deno.land website. @hashrock has done a lot of amazing artwork, like the loading page on doc.deno.land and the lovely image at the top of this page!