It is based on previous work by Andrew Schaaf.
author: Alberto Berti contact: [email protected] license: GNU General Public License version 3 or later
Table of Contents
JavaScripthon is a small and simple Python 3.5+ translator to JavaScript which aims to be able to translate most of the Python's core semantics without providing a full python-in-js environment, as most existing translators do. It tries to emit code which is simple to read and check. It does so by switching to ES6 construct when possible/required. This allows to simplify the needs of polyfills for many of the expected Python behaviors.
It is designed to be the first step in a pipeline that translates your Python code into something that a browser can understand. Usually it is used with tools like BabelJS and Webpack to prepare the final bundle that will be served to the browser. The steps from the source code to the bundle are the following:
- JavaScripthon converts your Python 3.5+ code to ES6 JavaScript modules;
- the BabelJS loader (configured inside Webpack or standalone) translates the ES6 JavaScript to ES5 so that the browser can understand it;
- Webpack parses the resulting source code and packages your source code with
its dependencies by analyzing
importstatements and emits abundle.jsready to be served to the browser.
Along this process the corresponding source maps are read and integrated at every step, allowing you to place breakpoints on your original Python source files when working with the developer tools of your browser.
An example of such setup is provided in the examples directory.
In addition to that, you can choose to do most these steps without using
external JS tools. It comes with an embedded js interpreter that loads a
standalone version of BabelJS and converts your code to ES5 JavaScript without
the need to install anything else. In fact most of the the test you can find
in tests/test_evaljs.py use the embedded interpreter to dual evaluate the
source code (one time in Python, one time in JavaScript) and simply check that
the results are the same.
Thanks to that, JavaScripthon can also be used as a server-side library to translate single functions or classes that you want your browser to load and evaluate.
The interface with the JS world is completely flat, just import the modules
or use the expected globals (window, document, etc...) as you
would do in JavaScript.
The fact that JavaScripthon doesn't reinvent the wheel by reimplementing in Python many of the features available with JavaScript translators/transpilers allows it to be lean while implementing quite a decent set of the core Python semantics. These are, briefly:
Misc
list slices;
list's
append();dict's
copy(),update();len();print();str();type(instance);yieldandyield from;asyncandawait;importandfrom...importto use any JS module (see import statements);callable();hasattr(),getattr(),setattr();template literals with
tmpl('a string with ${substitution}');template literals and tagged_templates (see Strings);
names starting with
d_anddd_will have that part replaced with$and$$, respectively;names ending with an underscore will have it removed. Useful for example with the AVA ES6 test runner which has a check named
is;__instancecheck__to[Symbol.hasInstance];inttoparseInt;floattoparseFloat;dictionary keys are unambiguous when ES6 translation is enabled. For example the following code gets translated correctly:
a = 'foo' d = {a: 1} print(d[a])
prints
1in both Python and JavaScript, while it printsundefinedwhen translated and evaluated in JavaScript without ES6.
- Comparisons (see section Simple stuff for the details)
- most of the basics;
isinstance()andissubclass();element in containerfor use with lists, objects, strings and the new ES6 collections likeMap,Setand so on;- identity checks:
foo is bar; - chained comparisons like
x < y <= z;
- Statements (see section Simple stuff and for statement for the
details)
if...elif...else;whileloop;forover list, over range, over plain js objects, over iterables (JS iterables);try...except...finallywith pythonesque behavior (see try...except...finally statement section for the details);assertstatement;
- Functions (see Functions section)
- standard functions, generator functions, async functions;
- parameters defaults;
- keyword parameters;
- parameters accumulators (
*argsand**kwargs), with some restrictions; - functions in methods are usually converted to "arrow functions" (the new
ES6 syntax like
(foo, bar) => foo * bar;) because they automatically keepthisfrom the enclosing scope. Appending_fnto a function declaration will force the translation to a normal function;
- Classes (see Classes section)
- single inheritance;
- Exception classes for use with
exceptstatement; - class decorators and method decorators;
- property descriptors;
- special handling of
propertyandclassmethoddescriptors; - async methods, generator methods;
- non-function body members (i.e.
member_of_class_Foo = bar);
This package is covered by the GNU General Public License version
3 or later. The code produced by it (i.e. the transpiled
JavaScript) is your code, and you are free to choose whatever
license you like. The only runtime that exists is the file
snippets.py from which some utility functions are picked when
necessary and transpiled together with your code. While it's
distributed with the same license as the other source code, in its
transpiled form will have the license you choose.
So, to summarize, the license of the this tool is GPL, but it doesn't extends to the products of this tool, on which you are free to decide.
Python 3.5 is required because Python's AST has changed between 3.4 and 3.5 and as of now supporting multiple Python versions is not one of my priorities.
To install the package execute the following command:
$ pip install javascripthon
or, if you want install it from sources:
$ git clone https://github.com/azazel75/metapensiero.pj $ pip install -r metapensiero.pj/requirements.txt $ pip install metapensiero.pj
To compile or transpile a python source module, use the commandline:
$ python -m metapensiero.pj source.pyor:
$ python -m metapensiero.pj -5 source.pyto transpile.
A pj console script is also automatically installed:
$ pj --help
usage: pj [-h] [--disable-es6] [--disable-stage3] [-5] [--transform-runtime]
[-o OUTPUT] [-d] [--pdb] [-s STRING] [-e]
[file [file ...]]
A Python 3.5+ to ES6 JavaScript compiler
positional arguments:
file Python source file(s) or directory(ies) to convert.
When it is a directory it will be converted
recursively
optional arguments:
-h, --help show this help message and exit
--disable-es6 Disable ES6 features during conversion (Ignored if
--es5 is specified)
--disable-stage3 Disable ES7 stage3 features during conversion
-5, --es5 Also transpile to ES5 using BabelJS.
--transform-runtime Add trasform runtime as plugin during transpile
-o OUTPUT, --output OUTPUT
Output file/directory where to save the generated code
-d, --debug Enable error reporting
--pdb Enter post-mortem debug when an error occurs
-s STRING, --string STRING
Convert a string, useful for small snippets. If the
string is '-' will be read from the standard input.
-e, --eval Evaluate the string supplied with the -s using the
embedded interpreter and return the last result. This
will convert the input string with all the extensions
enabled (comparable to adding the '-5' option) and so
it will take some time because of BabelJS load times.This offers many ways to test the framework, both the string conversion and the evaluation using the embedded JavaScript interpreter are very handy. For example:
$ pj -s '"foo" if True else "bar"'
(true ? "foo" : "bar");and evaluating the same statement:
$ pj -s '"foo" if True else "bar"' -e
fooYou can even try more fancy ES6 features, like destructuring assignment:
$ pj -s "a, b, c = (2, 3, 5) \na+b+c" -e
10You can use metapensiero.pj in python code as well.
from metapensiero.pj.__main__ import transform_string
transform_string("print()")The main development repository is the one on gitlab, the one on github is just a mirror so please report issues and feature requests there.
Here is a brief list of examples of the conversions the tool applies, just some, but not all.
| Python | JavaScript |
|---|---|
x < y <= z < 5 |
((x < y) && (y <= z) && (z < 5)) |
def foo():
return [True, False, None, 1729,
"foo", r"foo\bar", {}] |
function foo() {
return [true, false, null, 1729,
"foo", "foo\\bar", {}];
} |
while len(foo) > 0:
print(foo.pop()) |
while ((foo.length > 0)) {
console.log(foo.pop());
} |
if foo > 0:
....
elif foo < 0:
....
else:
.... |
if ((foo > 0)) {
....
} else {
if ((foo < 0)) {
....
} else {
....
}
} |
str(x) |
x.toString() |
yield foo
yield from foo |
yield foo
yield* foo |
Then there are special cases. Here you can see some of these
conversions. JavaScripthon cannot do a full trace of the sources, so
some shortcuts are taken about the conversion of some core, specific
Python's semantics. For example Python's self is always converted
to JavaScript's this, no matter where it's found. Or len(foo)
is always translated to foo.length. Albeit this an API specific of
just some objects (Strings, Arrays, etc...), it is considered wide
adopted and something the user may consider obvious.
The rules of thumb to treat things especially are:
- Is it possible to think of a conversion that covers most of the use cases?
- Is ts possible to find a convention widely used on the Python world to express this special case?
| Python | JavaScript |
|---|---|
== |
=== |
!= |
!== |
2**3 |
Math.pow(2, 3) |
'docstring' |
/* docstring */ |
self |
this |
len(...) |
(...).length |
print(...) |
console.log(...) |
isinstance(x, y)
isinstance(x, (y, z)) |
(x instanceof y)
(x instanceof y || x instanceof z) |
typeof(x) |
(typeof x) |
type(x) |
Object.getPrototypeOf(x) |
FirstCharCapitalized(...)
new(any_function(...)) |
new FirstCharCapitalized(...)
new any_function(...) |
foo in bar |
var _pj;
function _pj_snippets(container) {
function in_es6(left, right) {
if (((right instanceof Array) || ((typeof right) === "string"))) {
return (right.indexOf(left) > (- 1));
} else {
if (((right instanceof Map) || (right instanceof Set)
|| (right instanceof WeakMap)
|| (right instanceof WeakSet))) {
return right.has(left);
} else {
return (left in right);
}
}
}
container["in_es6"] = in_es6;
return container;
}
_pj = {};
_pj_snippets(_pj);
_pj.in_es6(foo, bar); |
foo[3:]
foo[:3] |
foo.slice(3);
foo.slice(0, 3); |
list(foo).append(bar) |
foo.push(bar); |
dict(foo).update(bar) |
Object.assign(foo, bar); |
dict(foo).copy() |
Object.assign({}, foo); |
The for statement by default is translated as if the object of the
cycle is a list but has two special cases:
for loops
| Python | JavaScript | notes |
|---|---|---|
for el in dict(a_dict):
print(el) |
var _pj_a = a_dict;
for (var el in _pj_a) {
if (_pj_a.hasOwnProperty(el)) {
console.log(el);
}
} |
With this kind of loop if you use dict(a_dict, True) the check on
hasOwnProperty() will not be added, so the loop will include
inherited (and enumerable) properties. |
for el in an_array:
print(el) |
for (var el, _pj_c = 0, _pj_a = an_array, _pj_b = _pj_a.length;
(_pj_c < _pj_b); _pj_c += 1) {
el = _pj_a[_pj_c];
console.log(el);
} |
 |
for i in range(5):
print(i) |
for (var i = 0, _pj_a = 5; (i < _pj_a); i += 1) {
console.log(i);
} |
 |
for el in iterable(a_set):
print(el) |
var _pj_a = a_set;
for (var el of _pj_a) {
console.log(el);
} |
This will loop over all the iterables, like instances of Array,
Map, Set, etc. but not over normal objects. |
Functions are very well supported. This should be obvious, you can say. Really it is not so simple, if we mean functions in their broader meaning, including the async functions and generator functions.
| Python | JavaScript | notes |
|---|---|---|
def foo(a, b, c):
pass |
function foo(a, b, c) {
} |
Normal functions |
def foo(a, b, c):
for i in range(a, b, c):
yield i
for i in iterable(foo(0, 5, 2)):
print(i) |
function* foo(a, b, c) {
for ... { // loop control omitted for brevity
yield i;
}
}
for (var i of foo(0, 5, 2)) {
console.log(i);
} |
Generator functions. They return an iterable and to correctly loop over
it you should use the iterable(...) call, so that the Python's
for...in will be converted into a for...of |
async def foo(a, b, c):
await some_promise_based_async |
async function foo(a, b, c) {
await some_promised_base_async;
} |
Async functions. They make use of the new Promise class, which is
also available. |
Parmeters defaults and keyword parameters are supported and so is *foo
accumulator, which is translated into the ES6 rest expression (...foo).
The only caveat is that JS support for keyword args sucks, so you will have to remember to fill in all the arguments before specifying keywords.
On function definitions, **kwargs is supported if it's alone, i.e. without
either keyword arguments or *args.
| Python | JavaScript |
|---|---|
def foo(a=2, b=3, *args):
pass |
function foo(a = 2, b = 3, ...args) {
} |
def bar(c, d, *, zoo=2):
pass |
function bar(c, d, {zoo = 2}={}) {
} |
foo(5, *a_list) |
foo(5, ...a_list); |
bar('a', 'b', zoo=5, another='c') |
bar("a", "b", {zoo: 5, another: "c"}); |
def zoo(e, **kwargs):
print(kwargs['bar']) |
function zoo(e, kwargs = {}) {
console.log(kwargs['bar'])
} |
zoo(4, bar=6) |
zoo(4, {bar: 6}) |
Classes are translated to ES6 classes as much as they can support. This means:
no direct support multi-class inheritance, you have to come up with your own solution for now. Many established frameworks support this in a way or another so just use those facilities for now. I've read of some attempts, see for example the suggestion on Mozilla developer or the other about simple mixins on
Exploring ES6.external implementation for class-level non assignment members. Assignment members are those on the body of a class which are defined with:
a_label = an_expressionlike:class Foo: bar = 'zoo' # or any kind of expression
These members are removed from the translated body and submitted to a snippet of code that will run after class creation in JS land. This serves two purposes: if the value is simple, i.e. it isn't an instance of
Object, it will be setup as a data descriptor, and it will work mostly like you are used to in Python. The most noticeable caveat is that it will not be accessible through the class as it is in Python, you will have to access the class' prototype, so in the case above i meanFoo.prototype.bar.The other purpose is to check for accessor descriptors. If the value on the right side of the assignment implements a
getfunction, it will be installed as a property as-is, and its get and set members will be used to manage the value with thebarname.external implementation for method decorators whose name is different from
propertyorclassmethod(more on these later on), because these are already supported by the ES6 class notation.external implementation for class decorators. One caveat here is that the return value of the decorator has always to be a function with a prototype: unfortunately a
newstatement seems not to be delegable in any way. So for example a class decorator implemented like the following:def test_class_deco(): counter = 0 def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls(*args) return wrapper @deco class Foo: pass
will never work. This will work instead:
def deco(cls): def wrapper(self, *args): counter += 1 # side effect return cls.prototype.constructor.call(self, *args) wrapper.prototype = cls.prototype return wrapper
So either return the original class or setup the wrapper appropriately.
Methods can be functions or async-functions although the latters aren't
officially supported yet by the JavaScript specification. You can disable them
adding a --disable-stage3 to the command line utility.
Python`s super() calls are converted accordingly to the type of
their surrounding method: super().__init__(foo) becomes
super(foo) in constructors.
Functions inside methods are translated to arrow functions so that
they keep the this of the surrounding method.
@property and @a_property.setter are translated to ES6 properties.
Methods decorated with @classmethod are translated to static methods.
Special methods __str__ and __len__ are translated to
toString() method and get length() property, respectively.
Arrow method expression to retain the this at method level aren't
implemented yet.
| Python | JavaScript |
|---|---|
class Foo(bar):
def __init__(self, zoo):
super().__init__(zoo)
def meth(self, zoo):
super().meth(zoo)
def cool(a, b, c):
print(self.zoo)
async def something(self, a_promise):
result = await a_promise
def generator_method(self):
yield something
@property
def foo(self):
return self._foo
@foo.setter
def foo(self, value):
self._foo = value
@classmethod
def bar(self, val):
do_something()
def __len__(self):
return 1
def __str__(self):
return 'Foo instance' |
class Foo extends bar {
constructor(zoo) {
super(zoo);
}
meth(zoo) {
super.meth(zoo);
var cool;
cool = (a, b, c) => {
console.log(this.zoo);
};
}
async something(a_promise) {
var result;
result = await a_promise;
}
* generator_method() {
yield something;
}
get foo() {
return this._foo;
}
set foo(value) {
self._foo = value;
}
static bar(val) {
do_something()
}
get length() {
return 1;
}
toString() {
return "Foo instance";
}
} |
Only direct descendants of Exception are treated especially, but
just for them to be meaningful in JS land and to be detectable with
instanceof in catch statements.
| Python | JavaScript |
|---|---|
class MyError(Exception):
pass
raise MyError("An error occurred") |
function MyError(message) {
this.name = "MyError";
this.message = (message || "Custom error MyError");
if (((typeof Error.captureStackTrace) === "function")) {
Error.captureStackTrace(this, this.constructor);
} else {
this.stack = new Error(message).stack;
}
}
MyError.prototype = Object.create(Error.prototype);
MyError.prototype.constructor = MyError;
throw new MyError("An error occurred"); |
The conversion of this statement is mostly obvious with the only
exception of the except part: it translates to a catch part
containing one if statement for each non catchall except. If a
catchall except isn't present, the error will be re-thrown, to mimic
Python's behavior.
try...catch...finally statement
| Python | JavaScript |
|---|---|
try:
foo.bar()
except MyError:
recover()
except MyOtherError:
recover_bad()
finally:
foo.on_end() |
try {
foo.bar();
} catch(e) {
if ((e instanceof MyError)) {
recover();
} else {
if ((e instanceof MyOtherError)) {
recover_bad()
} else {
throw e;
}
}
} finally {
foo.on_end();
} |
import and from ... import statements are converted to ES6
imports, and the declaration of an __all__ member on the module
top level is translated to ES6 named exports.
| Python | JavaScript |
|---|---|
import foo, bar
import foo.bar as b
from foo.bar import hello as h, bye as bb
from ..foo.zoo import bar
from . import foo
from .foo import bar
from foo__bar import zoo
from __foo.zoo import bar
from foo import __default__ as bar
from __globals__ import test_name
# this should not trigger variable definition
test_name = 2
# this instead should do it
test_foo = True
__all__ = ['test_name', 'test_foo']
__default__ = 'test_name' |
var test_foo;
import * as foo from 'foo';
import * as bar from 'bar';
import * as b from 'foo/bar';
import {hello as h, bye as bb} from 'foo/bar';
import {bar} from '../foo/zoo';
import * as foo from './foo';
import {bar} from './foo';
import {zoo} from 'foo-bar';
import {bar} from '@foo/zoo';
import bar from 'foo';
test_name = 2;
test_foo = true;
export {test_name, test_foo};
export default test_name; |
If you want to export something as default export in your modules,
declare a __default__ member and assign to it the string of the
symbol you want to export. To clarify:
foo = 42
bar = "hello"
__all__ = ['foo', 'bar'] # foo and bar will be exported as named exports
__default__ = 'bar' # bar will also be exported as the *default*This becomes:
var bar, foo;
foo = 42;
bar = "hello";
export {foo, bar};
export default bar;For what concerns the import, you can import the default export of
a module using the default name, as defined by the ES6
spec. However, as there were some issues reported to me with bundlers
not supporting the named import of the default export, a special
import statement using __default__ as name has been added that
directly translates to the more common form of ES6 default import. So:
from foo import default as bar
from foo import __default__ as zooTranslates to:
import {default as bar} from 'foo';
import zoo from 'foo';The two imports should work the same, see exploring js section and
the linked spec. But if you encounter problems with the former use
the latter instead. Keep in mind that you cannot mix the
__default__ import with others (i.e. it needs to be on a line of
its own) and that you always need to specify an ... as name ... part.
Javascripthon supports converting Python 3.6+ f-strings to ES6
template literals. The expression in the braces gets converted, but
neither conversion nor format_spec are supported: f"Value of
{a}" becomes `Value of ${a}` and f"Value of {self.foo}"
becomes `Value of ${this.foo}`.
You can also write raw template literals by using the function
tmpl() it does only a conversion of the string markers, from those
used in Python's literal string notation to template literal notation.
There is also the way to express tagged templates, template literals
that are parsed using a provided function. This is done by using the
function __. So for example:
__('A template ${string} with foo', bar)gets translated to:
bar`A template ${string} with foo`bar will be executed with the value of ${string} as a
parameter, see the link for template literals for help.
You can intermix Python and JS by using the
JS('Your-JS-code-here') marker function. It will not be touched by
the ES6 transcoder but if you choose to also transpile it to ES5, il
will be considered by Babel.
Execute make inside the examples directory.
To run the tests you should run the following at the package root:
python setup.py test
So you like this project and want to contribute? Good!
These are the terse guidelines:
There are some TODO points in the readme, or even the issue #6 is quite simple to fix. Feel free to pick what you like. The guidelines are to follow PEP8 for coding where possible, so use CamelCase for classes and snake_case for variables, functions and members, and UPPERCASE for constants. An exception to this rules are the function names inside ``metapensiero.pj.transformations`` subpackage. Those are matched against names of the AST objects coming from the ``ast`` module in standard lib, so they have to to match even in case. Try to keep lines lengths under 79 chars, more or less ;-) The workflow is to fork the project, do your stuff, maybe add a test for it and then submit a pull request. Have fun
Any contribution is welcome, drop me a line or file a pull request.
- BrainBacon has made a JavaScripthon loader for WebPack;
- icarito has contributed support for JavaScripthon to the python-webpack-loader for WebPack (Every valid JS package has at least two implementations! ROTFL);
- icarito has also integrated JavaScripthon with Nuxt.js and Vue.js;
- chfw has integrated JavaScripthon into pyecharts to allow Python function translation.
This is a brief list of what needs to be done:
- refactor the comprehensions conversion to use the snippets facility;
- refactor snippets rendering to write them as a module and import them in the module when tree conversion is enabled;
- convert
dict()calls to ES6Mapobject creation; - convert set literals to ES6
Setobjects. Also, update "foo in bar" to use bar.has(foo) for sets;
Stuff that was previously in the todo:
- translate import statements to ES6;
- translate
__all__definition to ES6 module exports; - write a command line interface to expose the api;
- make try...except work again and implement try...finally;
- convert async and await to the same proposed features for js (see BabelJS documentation);
- convert argument defaults on functions to ES6;
- convert call keyword arguments;
- convert *iterable syntax to ES6 destructuring;
- use arrow functions for functions created in functions;
- properties to ES6 properties (getter and setter);
- take advantage of new duckpy features to use a JS execution context that lasts multiple calls. This way the BabelJS bootstrap affects only the initial execution;
- class and method decorators;
- implement yield, yield from and generator functions;
- update "foo in bar" to use bar.has(foo) for maps;
A good documentation and explanation of ES6 features can be found on the book Exploring ES6 by Axel Rauschmayer (donate if you can).
An extensive documentation about Python's AST objects, very handy.
Have a look at ECMAScript 6 Tools by Addy Osmani.
To debug source maps have a look at source-map-visualization and its package on npm.
Still i found these links to be helpful:
Here is an example of the latter tool showing code generated by JavaScripthon, have fun!
- A benchmark of ES6 features and discussion about it on hacker's news.
- A compatibility table of ES6 features showing completeness of support feature by feature.
- A story about ES6 craziest stuff... symbols