Running JavaScript from Python using NodeJS

I wanted to call a pile of JavaScript (actually transpiled TypeScript) from Python, so I could run reinforcement learning on this.

I was dissatisfied with the existing options for calling JS from Python, so I made a new thing!

import asyncio
from nodejs_eval import evaluator

async def demo():
    # Let's run some JavaScript from Python!
    async with evaluator() as e:
        result = await e.run("return Math.pow(7, 3);")
        assert result == 343

asyncio.run(demo())

TL;DR: From Python, we launch a NodeJS sidecar process which runs an HTTP server on a Unix domain socket. We then hit that HTTP server with requests containing arbitrary JavaScript.

Part A: NodeJS as a sidecar using http-eval

So while it should be possible, and relatively nice, to run JavaScript directly within the CPython process by simply embedding V8 or SpiderMonkey, attempts to do so are somewhat fraught in practice—I couldn’t get any to work.

This is a shame! Running an embedded in-process V8 would particularly be nice for tightly coupled Python/JS code; you could pass control and data back and forth freely and efficiently, in theory, and enable all kinds of fun workflows.

But what if our code isn’t so coupled; what if we just want to evaluate a blob of JavaScript and get a blob of data back, without much worry about performance? Can we run V8 (or other JS engine) out-of-process instead? For that, NodeJS is obviously a perfectly fine self-contained V8 JS engine, available on most modern systems. Let’s use that! We just need to teach it to run code from a parent process…

But how do we send it code? I figured, as a low-dependency option, we can use HTTP over a Unix domain socket as our IPC.

Obviously, an IPC which effectively sends arbitrary code for execution raises security concerns. If done improperly, we’re building a Remote Code Execution (RCE) vulnerability. Using a Unix domain socket with some extra configuration footgun checks (blocking binding to a TCP/IP port, and checking for a good umask), should make this safe. I think. :) More on that here.

Anyway, I coded this up in TypeScript as an npx-executable binary script, using express, tsx and pkgroll. See http-eval on Github and npm.

http-eval could be used to provide JavaScript evaluation to other languages (or just to the shell, as an awkward REPL). But we’re going to call it from Python…

Part B: bundling and launching NodeJS via nodejs-bin

This, thankfully, already exists, thanks to Sam Willis!

In summary: nodejs-bin is a Python PyPI package which bundles NodeJS, so Python programmers don’t have to figure out how to install it, and Python packages can just depend on it using Python-language dependency systems, and take NodeJS for granted. nodejs-bin has a helper to launch NodeJS. In our case, we want npx, to just run a script…

Bringing it together: nodejs-eval

Now we have the parts we need to build a simple Python module which:

• Launches a NodeJS sidecar (specifically npx, from nodejs-bin),
• Which launches a JavaScript evaluation server on a Unix domain socket (http-eval), and
• Exposes helpers to send arbitrary JS to that server, and then decode and return the results. This is the nodejs-eval Python module.

I coded this up using hatch, aiohttp, mypy, and pytest. See nodejs-eval on Github and PyPI.

In action it looks like:

    $ pip install nodejs-eval

… and then:

import asyncio
from nodejs_eval import evaluator

async def demo():
    # Let's run some JavaScript from Python!
    async with evaluator() as e:
        result = await e.run("return Math.pow(7, 3);")
        assert result == 343

asyncio.run(demo())

Alternatives considered

Python and JavaScript are two of the most popular programming languages in the world. Surely there should be many options for connecting A to B already?

Here’s what I found:

• v8eval: Python bindings for (Chrome’s) v8 JS engine, created by Sony.
  • Abandoned, stuck on V8 v7.1 from 2018.
• pyv8: Python bindings for v8, created by Google.
  • Abandoned and doesn’t work anymore.
• PythonMonkey: Python bindings for (Mozilla’s) Spidermonkey JS engine, created by Distributive Networks, LLC.
  • Still in development. I hit and then worked around a couple segfaults before giving up.
• python-spidermonkey: A different set of Python bindings for Spidermonkey, from Paul J. Davis.
  • Abandoned for 14 years!
• Js2Py: Natively executes JavaScript in Python using a custom Python-native JS engine.
  • Still in development. Doesn’t support ES6 including arrow functions, so I moved on.
• PyMiniRacer: Minimalist V8 Python bindings created by Sqreen, whch was later aquired by DataDog.
  • TODO! This hasn’t been published since 2021, but the old wheel on PyPI still works with v8 8.9. v8 is up to 12.x today, so it seems like it would be a good idea to update it… but the build instructions currently fail. Maybe this can be resuscitated with relatively little work.
• Selenium WebDriver: Just run the whole Chromium or Firefox browser as a subprocess from Python.
  • Obviously works and used by zillions of folks, but seemed heavier-weight than what I want.