Pickora 🐰

A small compiler that can convert Python scripts to pickle bytecode.

Requirements

• Python 3.8+

No third-party modules are required.

Quick Start

Installation

Using pip:

$ pip install pickora

From source:

$ git clone https://github.com/splitline/Pickora.git
$ cd Pickora
$ python setup.py install

Basic Usage

Compile from a string:

$ pickora -c 'from builtins import print; print("Hello, world!")' -o output.pkl
$ python -m pickle output.pkl # load the pickle bytecode
Hello, world!
None

Compile from a file:

$ echo'from builtins import print; print("Hello, world!")'> hello.py
$ pickora hello.py # output compiled pickle bytecode to stdout directly
b'\x80\x04\x95(\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x8c\x05print\x93\x94\x94h\x01\x8c\rHello, world!\x85R.'

Usage

usage: pickora [-h] [-c CODE] [-p PROTOCOL] [-e] [-O] [-o OUTPUT] [-d] [-r]
[-f {repr,raw,hex,base64,none}]
[source]
A toy compiler that can convert Python scripts into pickle bytecode.
positional arguments:
source source code file
optional arguments:
-h, --help show this help message and exit
-c CODE, --code CODE source code string
-p PROTOCOL, --protocol PROTOCOL
pickle protocol
-e, --extended enable extended syntax (trigger find_class)
-O, --optimize optimize pickle bytecode (with pickletools.optimize)
-o OUTPUT, --output OUTPUT
output file
-d, --disassemble disassemble pickle bytecode
-r, --run run (load) pickle bytecode immediately
-f {repr,raw,hex,base64,none}, --format {repr,raw,hex,base64,none}
output format, none means no output
Basic usage: `pickora samples/hello.py` or `pickora --code 'print("Hello, world!")' --extended`

Supported Syntax

Basic Syntax (achived by only using pickle opcodes)

• Basic types: int, float, bytes, string, dict, list, set, tuple, bool, None
• Assignment: val = dict_['x'] = obj.attr = 'meow'
• Augmented assignment: x += 1
• Named assignment: (x := 1337)
• Unpacking: a, b, c = 1, 2, 3
• Function call: f(arg1, arg2)
  • Doesn't support keyword argument.
• Import
  • from module import things (directly using STACK_GLOBALS bytecode)
• Macros (see below for more details)
  • STACK_GLOBAL
  • GLOBAL
  • INST
  • OBJ
  • NEWOBJ
  • NEWOBJ_EX
  • BUILD

Extended Syntax (enabled by -e / --extended option)

Note: All extended syntaxes are implemented by importing other built-in modules. So with this option will trigger find_class when loading the pickle bytecode.

• Attributes: obj.attr (using builtins.getattr only when you need to "load" an attribute)
• Operators (using operator module)
  • Binary operators: +, -, *, / etc.
  • Unary operators: not, ~, +val, -val
  • Compare: 0 < 3 > 2 == 2 > 1 (using builtins.all for chained comparing)
  • Subscript: list_[1:3], dict_['key'] (using builtins.slice for slice)
  • Boolean operators (using builtins.next, builtins.filter)
    • and: using operator.not_
    • or: using operator.truth
    • (a or b or c) -> next(filter(truth, (a, b, c)), c)
    • (a and b and c) -> next(filter(not_, (a, b, c)), c)
• Import
  • import module (using importlib.import_module)
• Lambda
  • lambda x,y=1: x+y
  • Using types.CodeType and types.FunctionType
  • [Known bug] If any global variables are changed after the lambda definition, the lambda function won't see those changes.

Macros

There are currently 4 macros available: STACK_GLOBAL, GLOBAL, INST and BUILD.

STACK_GLOBAL(modname: Any, name: Any)

Example:

function_name=input("> ") # > systemfunc=STACK_GLOBAL('os', function_name) # <built-in function system>func("date") # Tue Jan 13 33:33:37 UTC 2077

Behaviour:

1. PUSH modname
2. PUSH name
3. STACK_GLOBAL

GLOBAL(modname: str, name: str)

Example:

func=GLOBAL("os", "system") # <built-in function system>func("date") # Tue Jan 13 33:33:37 UTC 2077

Behaviour:

Simply write this piece of bytecode: f"c{modname}\n{name}\n"

INST(modname: str, name: str, args: tuple[Any])

Example:

command=input("cmd> ") # cmd> dateINST("os", "system", (command,)) # Tue Jan 13 33:33:37 UTC 2077

Behaviour:

1. PUSH a MARK
2. PUSH args by order
3. Run this piece of bytecode: f'i{modname}\n{name}\n'

BUILD(inst: Any, state: Any, slotstate: Any)

state is for inst.__setstate__(state) and slotstate is for setting attributes.

Example:

fromcollectionsimport_collections_abcBUILD(_collections_abc, None, {'__all__': ['ChainMap', 'Counter', 'OrderedDict']})

Behaviour:

1. PUSH inst
2. PUSH (state, slotstate) (tuple)
3. PUSH BUILD

FAQ

What is pickle?

RTFM.

Why?

It's cool.

Is it useful?

No, not at all, it's definitely useless.

So, is this garbage?

Yep, it's cool garbage.

Would it support syntaxes like if / while / for ?

No. All pickle can do is just simply define a variable or call a function, so this kind of syntax wouldn't exist.

But if you want to do things like:

ans=input("Yes/No: ")
ifans=='Yes':
print("Great!")
elifans=='No':
exit()

It's still achievable! You can rewrite your code like this:

fromfunctoolsimportpartialcondition= {'Yes': partial(print, 'Great!'), 'No': exit}
ans=input("Yes/No: ")
condition.get(ans, repr)()

ta-da!

For the loop syntax, you can try to use map / starmap / reduce etc .

And yes, you are right, it's functional programming time!