python-javaobj is a python library that provides functions for reading and
writing Java objects serialized or to be deserialized by ObjectOutputStream.
This form of object representation is a standard data interchange format in
Java world.
The javaobj module exposes an API familiar to users of the standard library
marshal, pickle and json modules.
This project is a fork of python-javaobj by Volodymyr Buell, originally from Google Code and now hosted on GitHub.
This fork intends to work both on Python 2.7 and Python 3.4+.
| Implementations | Version |
|---|---|
v1, v2 | 0.4.0+ |
v3 | 0.5.0+ |
Since version 0.4.0, three implementations of the parser are available:
v1: the classic implementation ofjavaobj, with a work in progress implementation of a writer.v2: a rewritten implementation, which is a port of the Java projectjdeserialize, with support of the object transformer (with a new API) and of thenumpyarrays loading.v3: a new implementation, written from scratch to benefit from Python 3.12+ features, with full read and write support.
You can use the v1 parser to ensure that the behaviour of your scripts
doesn't change. It also provides a basic marshalling capability.
You can use the v2 parser for developments in Python versions lower
than 3.12 and which won't require marshalling, or as a fallback
if the v1 parser fails to parse a file.
For new development, you should use the v3 parser, which supports both
reading and writing Java object streams.
| Implementations | Version |
|---|---|
v1 | 0.2.0+ |
As of version 0.2.0, the notion of object transformer from the original project as been replaced by an object creator.
The object creator is called before the deserialization. This allows to store the reference of the converted object before deserializing it, and avoids a mismatch between the referenced object and the transformed one.
| Implementations | Version |
|---|---|
v2 | 0.4.0+ |
The v2 implementation provides a new API for the object transformers.
Please look at the Usage (V2) section in this file.
| Implementations | Version |
|---|---|
v3 | 0.5.0+ |
The v3 implementation is a full rewrite targeting Python 3.12+.
It uses dataclasses, structural pattern matching (match/case) and PEP 604
union types. Its API is intentionally similar to v2 but fixes several
correctness issues and adds stricter safety limits.
Please look at the Usage (V3) and Migration to V3 sections in this file.
| Implementations | Version |
|---|---|
v1 | 0.2.3+ |
As of version 0.2.3, bytes arrays are loaded as a bytes object instead of
an array of integers.
| Implementations | Version |
|---|---|
v2 | 0.4.2+ |
A new transformer API has been proposed to handle objects written with a custom Java writer. You can find a sample usage in the Custom Transformer section in this file.
- Java object instance un-marshalling
- Java classes un-marshalling
- Primitive values un-marshalling
- Automatic conversion of Java Collections to python ones
(
HashMap=>dict,ArrayList=>list, etc.) - Basic marshalling of simple Java objects (
v1implementation) - Full un-marshalling and marshalling of Java object streams, with typed
errors and configurable safety limits (
v3implementation) - Automatically uncompresses GZipped files
- Python >= 2.7 or Python >= 3.4 for
v1andv2 - Python >= 3.12 for
v3 enum34andtypingwhen using Python <= 3.4 (installable withpip)- Maven 2+ (for building test data of serialized objects.
The fixtures are already committed under
tests/, so you only need Maven to regenerate them; skip it otherwise, or setJAVAOBJ_NO_MAVEN=1to disable the regeneration when running the test suite)
Un-marshalling of Java serialised object:
importjavaobjwithopen("obj5.ser", "rb") asfd:
jobj=fd.read()
pobj=javaobj.loads(jobj)
print(pobj)Or, you can use JavaObjectUnmarshaller object directly:
importjavaobjwithopen("objCollections.ser", "rb") asfd:
marshaller=javaobj.JavaObjectUnmarshaller(fd)
pobj=marshaller.readObject()
print(pobj.value, "should be", 17)
print(pobj.next, "should be", True)
pobj=marshaller.readObject()Note: The objects and methods provided by javaobj module are shortcuts
to the javaobj.v1 package, for Compatibility purpose.
It is recommended to explicitly import methods and classes from the v1,
v2, or v3 package when writing new code, in order to be sure that your code
won't need import updates in the future.
The following methods are provided by the javaobj.v2 package:
load(fd, *transformers, use_numpy_arrays=False): Parses the content of the given file descriptor, opened in binary mode (rb). The method accepts a list of custom object transformers. The default object transformer is always added to the list.The
use_numpy_arraysflag indicates that the arrays of primitive type elements must be loaded usingnumpy(if available) instead of using the standard parsing technic.loads(bytes, *transformers, use_numpy_arrays=False): This the a shortcut to theload()method, providing it the binary data using aBytesIOobject.
Note: The V2 parser doesn't have the marshalling capability.
Sample usage:
importjavaobj.v2asjavaobjwithopen("obj5.ser", "rb") asfd:
pobj=javaobj.load(fd)
print(pobj.dump())An object transformer can be called during the parsing of a Java object instance or while loading an array.
The Java object instance parsing works in two main steps:
The transformer is called to create an instance of a bean that inherits
JavaInstance.The latter bean is then called:
- When the object is written with a custom block data
- After the fields and annotations have been parsed, to update the content of the Python bean.
Here is an example for a Java HashMap object. You can look at the code of
the javaobj.v2.transformer module to see the whole implementation.
classJavaMap(dict, javaobj.v2.beans.JavaInstance):
""" Inherits from dict for Python usage, JavaInstance for parsing purpose """def__init__(self):
# Don't forget to call both constructorsdict.__init__(self)
JavaInstance.__init__(self)
defload_from_blockdata(self, parser, reader, indent=0):
""" Reads content stored in a block data. This method is called only if the class description has both the `SC_EXTERNALIZABLE` and `SC_BLOCK_DATA` flags set. The stream parsing will stop and fail if this method returns False. :param parser: The JavaStreamParser in use :param reader: The underlying data stream reader :param indent: Indentation to use in logs :return: True on success, False on error """# This kind of class is not supposed to have the SC_BLOCK_DATA flag setreturnFalsedefload_from_instance(self, indent=0):
# type: (int) -> bool""" Load content from the parsed instance object. This method is called after the block data (if any), the fields and the annotations have been loaded. :param indent: Indentation to use while logging :return: True on success (currently ignored) """# Maps have their content in their annotationsforcd, annotationsinself.annotations.items():
# Annotations are associated to their definition classifcd.name=="java.util.HashMap":
# We are in the annotation created by the handled class# Group annotation elements 2 by 2# (storage is: key, value, key, value, ...)args= [iter(annotations[1:])] *2forkey, valueinzip(*args):
self[key] =value# Job donereturnTrue# Couldn't load the datareturnFalseclassMapObjectTransformer(javaobj.v2.api.ObjectTransformer):
""" Creates a JavaInstance object with custom loading methods for the classes it can handle """defcreate_instance(self, classdesc):
# type: (JavaClassDesc) -> Optional[JavaInstance]""" Transforms a parsed Java object into a Python object :param classdesc: The description of a Java class :return: The Python form of the object, or the original JavaObject """ifclassdesc.name=="java.util.HashMap":
# We can handle this class descriptionreturnJavaMap()
else:
# Return None if the class is not handledreturnNoneThe custom transformer is called when the class is not handled by the default
object transformer.
A custom object transformer still inherits from the ObjectTransformer class,
but it also implements the load_custom_writeObject method.
The sample given here is used in the unit tests.
On the Java side, we create various classes and write them as we wish:
classCustomClassimplementsSerializable {
privatestaticfinallongserialVersionUID = 1;
publicvoidstart(ObjectOutputStreamout) throwsException {
this.writeObject(out);
}
privatevoidwriteObject(ObjectOutputStreamout) throwsIOException {
CustomWritercustom = newCustomWriter(42);
out.writeObject(custom);
out.flush();
}
}
classRandomChildextendsRandom {
privatestaticfinallongserialVersionUID = 1;
privateintnum = 1;
privatedoubledoub = 4.5;
RandomChild(intseed) {
super(seed);
}
}
classCustomWriterimplementsSerializable {
protectedRandomChildcustom_obj;
CustomWriter(intseed) {
custom_obj = newRandomChild(seed);
}
privatestaticfinallongserialVersionUID = 1;
privatestaticfinalintCURRENT_SERIAL_VERSION = 0;
privatevoidwriteObject(ObjectOutputStreamout) throwsIOException {
out.writeInt(CURRENT_SERIAL_VERSION);
out.writeObject(custom_obj);
}
}An here is a sample writing of that kind of object:
ObjectOutputStreamoos = newObjectOutputStream(
newFileOutputStream("custom_objects.ser"));
CustomClasswriter = newCustomClass();
writer.start(oos);
oos.flush();
oos.close();On the Python side, the first step is to define the custom transformers.
They are children of the javaobj.v2.transformers.ObjectTransformer class.
classBaseTransformer(javaobj.v2.transformers.ObjectTransformer):
""" Creates a JavaInstance object with custom loading methods for the classes it can handle """def__init__(self, handled_classes=None):
self.instance=Noneself.handled_classes=handled_classesor {}
defcreate_instance(self, classdesc):
""" Transforms a parsed Java object into a Python object :param classdesc: The description of a Java class :return: The Python form of the object, or the original JavaObject """ifclassdesc.nameinself.handled_classes:
self.instance=self.handled_classes[classdesc.name]()
returnself.instancereturnNoneclassRandomChildTransformer(BaseTransformer):
def__init__(self):
super(RandomChildTransformer, self).__init__(
{"RandomChild": RandomChildInstance}
)
classCustomWriterTransformer(BaseTransformer):
def__init__(self):
super(CustomWriterTransformer, self).__init__(
{"CustomWriter": CustomWriterInstance}
)
classJavaRandomTransformer(BaseTransformer):
def__init__(self):
super(JavaRandomTransformer, self).__init__()
self.name="java.util.Random"self.field_names= ["haveNextNextGaussian", "nextNextGaussian", "seed"]
self.field_types= [
javaobj.v2.beans.FieldType.BOOLEAN,
javaobj.v2.beans.FieldType.DOUBLE,
javaobj.v2.beans.FieldType.LONG,
]
defload_custom_writeObject(self, parser, reader, name):
ifname!=self.name:
returnNonefields= []
values= []
forf_name, f_typeinzip(self.field_names, self.field_types):
values.append(parser._read_field_value(f_type))
fields.append(javaobj.v2.beans.JavaField(f_type, f_name))
class_desc=javaobj.v2.beans.JavaClassDesc(
javaobj.v2.beans.ClassDescType.NORMALCLASS
)
class_desc.name=self.nameclass_desc.desc_flags=javaobj.v2.beans.ClassDataType.EXTERNAL_CONTENTSclass_desc.fields=fieldsclass_desc.field_data=valuesreturnclass_descSecond step is defining the representation of the instances, where the real
object loading occurs. Those classes inherit from
javaobj.v2.beans.JavaInstance.
classCustomWriterInstance(javaobj.v2.beans.JavaInstance):
def__init__(self):
javaobj.v2.beans.JavaInstance.__init__(self)
defload_from_instance(self):
""" Updates the content of this instance from its parsed fields and annotations :return: True on success, False on error """ifself.classdescandself.classdescinself.annotations:
# Here, we known there is something written before the fields,# even if it's not declared in the class descriptionfields= ["int_not_in_fields"] +self.classdesc.fields_namesraw_data=self.annotations[self.classdesc]
int_not_in_fields=struct.unpack(
">i", BytesIO(raw_data[0].data).read(4)
)[0]
custom_obj=raw_data[1]
values= [int_not_in_fields, custom_obj]
self.field_data=dict(zip(fields, values))
returnTruereturnFalseclassRandomChildInstance(javaobj.v2.beans.JavaInstance):
defload_from_instance(self):
""" Updates the content of this instance from its parsed fields and annotations :return: True on success, False on error """ifself.classdescandself.classdescinself.field_data:
fields=self.classdesc.fields_namesvalues= [
self.field_data[self.classdesc][self.classdesc.fields[i]]
foriinrange(len(fields))
]
self.field_data=dict(zip(fields, values))
if (
self.classdesc.super_classandself.classdesc.super_classinself.annotations
):
super_class=self.annotations[self.classdesc.super_class][0]
self.annotations=dict(
zip(super_class.fields_names, super_class.field_data)
)
returnTruereturnFalseFinally we can use the transformers in the loading process.
Note that even if it is not explicitly given, the DefaultObjectTransformer
will be also be used, as it is added automatically by javaobj if it is
missing from the given list.
# Load the object using those transformerstransformers= [
CustomWriterTransformer(),
RandomChildTransformer(),
JavaRandomTransformer()
]
withopen("custom_objects.ser", "rb") asfd:
pobj=javaobj.load(fd, *transformers)
# Here we show a field that isn't visible from the class description# The field belongs to the class but it's not serialized by default because# it's static. See: https://stackoverflow.com/a/16477421/12621168print(pobj.field_data["int_not_in_fields"])Requires Python 3.12+.
The javaobj.v3 package is a full rewrite of the Java object stream parser.
It provides the same two entry-points as v2:
load(fd, *transformers, use_numpy_arrays=False, max_array_size=…, max_depth=500): Parses a binary file descriptor opened inrbmode and returns the top-level object if the stream contains exactly one, a list of objects if there are several, orNonefor an empty stream. Pass additionalObjectTransformerinstances as positional arguments.loads(data, *transformers, …): Convenience wrapper aroundload()that acceptsbytes.
Sample usage:
importjavaobj.v3asjavaobjwithopen("obj5.ser", "rb") asfd:
pobj=javaobj.load(fd)
# Access fields by name (preferred)value=pobj.get_field("myField")
# Or use attribute-style access (issues a warning on ambiguity)value=pobj.myFieldjavaobj.v3's default transformer converts the most common Java standard
library classes into their natural Python equivalent:
| Java class | Python type |
|---|---|
java.lang.Boolean | bool |
java.lang.Integer, java.lang.Long | int |
java.util.ArrayList, java.util.LinkedList | list |
java.util.HashMap, java.util.TreeMap, java.util.LinkedHashMap | dict |
java.util.HashSet, java.util.LinkedHashSet, java.util.TreeSet | set |
java.time.* (via java.time.Ser) | the matching datetime/date/time type |
Any other class falls back to a generic JavaInstance, accessed through
get_field() as shown above. Provide your own ObjectTransformer (see
"Object Transformer V3" below) to handle additional classes.
| Feature | V1 | V2 | V3 |
|---|---|---|---|
Python 3.12+ (match/case, PEP 604) | ✗ | ✗ | ✓ |
Fully typed (dataclasses, PEP 695 type aliases) | ✗ | partial | ✓ |
TC_RESET handling | ✗ | ✗ | ✓ |
TC_EXCEPTION in object graph | ✗ | ✗ | ✓ |
TC_PROXYCLASSDESC | ✗ | ✓ | ✓ |
| Security limits (max depth / array size) | ✗ | ✗ | ✓ |
Correct TYPE_CHAR numpy dtype (>u2) | ✗ | ✗ | ✓ |
| Typed exception hierarchy | ✗ | ✗ | ✓ |
BlockData.__eq__(bytes) compatibility | ✓ | ✓ | ✓ |
| Marshalling (writing) support | partial | ✗ | ✓ |
v3 adds two optional safety limits that prevent resource exhaustion when
parsing untrusted streams:
importjavaobj.v3asjavaobjwithopen("untrusted.ser", "rb") asfd:
pobj=javaobj.load(
fd,
max_array_size=10*1024*1024, # 10 MiB max per arraymax_depth=100, # max object-graph depth
)javaobj.v3.exceptions defines a typed hierarchy so callers can catch
exactly what they expect instead of a bare Exception:
importjavaobj.v3asjavaobjfromjavaobj.v3.exceptionsimportJavaObjError, ParseError, SecurityErrorwithopen("obj5.ser", "rb") asfd:
try:
pobj=javaobj.load(fd)
exceptSecurityError:
# A max_depth or max_array_size limit was exceeded
...
exceptParseErrorase:
# The stream does not follow the protocol; e.offset is the byte# offset in the stream where the error occurred, or -1 if unknownprint(e, "at offset", e.offset)
exceptJavaObjError:
# Catch-all base class for everything else javaobj.v3 raises
...JavaObjError-- base class for every exceptionjavaobj.v3raises.ParseError-- the stream cannot be decoded according to the protocol; carries an.offsetattribute.UnexpectedOpcodeError-- aParseErrorsubclass raised when an opcode byte is not among the values expected at that point; carries.expected(a tuple of acceptable values) and.got.
UnsupportedFeatureError-- the stream uses a protocol featurev3does not implement yet (for exampleExternalizableobjects on read).SecurityError-- a configuredmax_depth/max_array_sizelimit was exceeded (see "Security limits" above).
The ObjectTransformer base class in v3 has the same three override points
as in v2:
create_instance(classdesc)— return aJavaInstancesubclass (orNoneto fall back to the next transformer).load_array(reader, type_code, size)— called forTC_ARRAYrecords; return the array data (bytesorlist) orNoneto use the default logic.load_custom_writeObject(parser, reader, class_name)— called when a class written withwriteObject()requires fully custom parsing.
The DefaultObjectTransformer additionally exposes a public handles(name)
method that returns True when the transformer knows how to load the given
Java class name.
importjavaobj.v3asjavaobjwithopen("arrays.ser", "rb") asfd:
pobj=javaobj.load(fd, use_numpy_arrays=True)When use_numpy_arrays=True, a NumpyArrayTransformer is appended to the
transformer list and primitive arrays are returned as numpy.ndarray.
The javaobj.v3 package exposes two additional entry-points for serializing
beans back to the Java Object Serialization binary format:
dump(fd, *objects): Writes one or more parsed objects to a binary file descriptor opened inwbmode.dumps(*objects) -> bytes: Returns the serialized stream as abytesobject.
Both functions accept any combination of
JavaInstance, JavaArray, JavaString, JavaEnum, JavaClass, BlockData,
and None (written as TC_NULL) as positional arguments.
importjavaobj.v3asjavaobj# Parse an existing filewithopen("obj5.ser", "rb") asfd:
pobj=javaobj.load(fd)
# Serialize back to bytesdata=javaobj.dumps(pobj)
# Or write directly to a filewithopen("obj5_copy.ser", "wb") asfd:
javaobj.dump(fd, pobj)importjavaobj.v3asjavaobjfromjavaobj.v3.beansimportJavaStringwithopen("a.ser", "rb") asfd:
obj_a=javaobj.load(fd)
# Write two objects into one streamdata=javaobj.dumps(obj_a, JavaString("hello"))
# Re-parse: returns a list when the stream holds more than one objectresult=javaobj.loads(data) # -> [obj_a, JavaString("hello")]| Construct | Supported |
|---|---|
TC_OBJECT — NOWRCLASS (plain fields only) | ✓ |
TC_OBJECT — WRCLASS (fields + block-data annotations) | ✓ |
TC_ARRAY | ✓ |
TC_STRING / TC_LONGSTRING | ✓ |
TC_ENUM | ✓ |
TC_CLASS | ✓ |
TC_NULL | ✓ |
TC_BLOCKDATA / TC_BLOCKDATALONG | ✓ |
TC_PROXYCLASSDESC | ✓ |
Back-references (TC_REFERENCE) | ✓ (automatic) |
EXTERNAL_CONTENTS (Protocol v1 Externalizable) | ✗ |
Note: Back-references are tracked automatically by identity: if the same object appears more than once in the graph, subsequent occurrences are written as
TC_REFERENCE— exactly as Java'sObjectOutputStreamdoes.
You can construct the v3 beans manually to serialize a Python object as if it were a Java one. The key types are:
JavaClassDesc— the class descriptor (name,serialVersionUID, flags, fields)JavaField— one field entry (type code + name, and optionally the binary class name for object/array fields)JavaInstance— the object instance (field_datamaps each class descriptor to a{JavaField: value}dict)JavaString— a JavaStringvalue
All beans accept handle=0 when created from scratch; the writer assigns real
handles automatically during serialization.
importjavaobj.v3asjavaobjfromjavaobj.constantsimportClassDescFlagsfromjavaobj.v3.beansimport (
FieldType,
JavaClassDesc,
ClassDescType,
JavaField,
JavaInstance,
JavaString,
)
# ── 1. Describe the Java class ────────────────────────────────────────────────## Java equivalent:## package com.example;# public class Point implements java.io.Serializable {# private static final long serialVersionUID = 1L;# public int x;# public int y;# }field_x=JavaField(type=FieldType.INTEGER, name="x")
field_y=JavaField(type=FieldType.INTEGER, name="y")
point_cd=JavaClassDesc(
handle=0, # assigned by the writername="com.example.Point",
serial_version_uid=1,
desc_flags=ClassDescFlags.SC_SERIALIZABLE,
fields=[field_x, field_y],
)
# ── 2. Create an instance ─────────────────────────────────────────────────────point=JavaInstance(
handle=0,
classdesc=point_cd,
field_data={
point_cd: {
field_x: 42,
field_y: -7,
}
},
)
# ── 3. Serialize ──────────────────────────────────────────────────────────────data=javaobj.dumps(point)
# ── 4. Round-trip check ───────────────────────────────────────────────────────restored=javaobj.loads(data)
print(restored.get_field("x")) # 42print(restored.get_field("y")) # -7For object-type fields (e.g. a String attribute), use FieldType.OBJECT,
set class_name to the binary class name, and pass a JavaString as the
value:
field_name=JavaField(
type=FieldType.OBJECT,
name="name",
class_name="Ljava/lang/String;", # binary name for java.lang.String
)
person_cd=JavaClassDesc(
handle=0,
name="com.example.Person",
serial_version_uid=1,
desc_flags=ClassDescFlags.SC_SERIALIZABLE,
fields=[field_name, field_x], # reuse field_x from above
)
alice=JavaInstance(
handle=0,
classdesc=person_cd,
field_data={
person_cd: {
field_name: JavaString(handle=0, value="Alice"),
field_x: 30,
}
},
)
data=javaobj.dumps(alice)The parser and the writer log through the standard logging module, under
javaobj.v3.parser and javaobj.v3.writer respectively:
importlogginglogging.getLogger("javaobj.v3").setLevel(logging.DEBUG)| V1 | V3 |
|---|---|
import javaobj | import javaobj.v3 as javaobj |
pobj.classdesc.name | pobj.classdesc.name (unchanged) |
pobj.myField (direct attribute) | pobj.get_field("myField") (preferred) or pobj.myField |
pobj._data on arrays | pobj.data (public) |
javaobj.JavaObjectUnmarshaller | removed — use javaobj.v3.parser.JavaStreamParser |
javaobj.JavaObjectMarshaller | javaobj.v3.dump / javaobj.v3.dumps |
Exceptions: bare Exception | Typed: ParseError, UnexpectedOpcodeError, … |
Shallow conversion helper (best-effort, for gradual migration):
fromjavaobj.v3._compatimportv1_to_v3v3_obj=v1_to_v3(v1_obj)| V2 | V3 |
|---|---|
import javaobj.v2 as javaobj | import javaobj.v3 as javaobj |
javaobj.load(fd) | javaobj.load(fd) (same signature) |
javaobj.loads(data) | javaobj.loads(data) (same signature) |
pobj.classdesc.name | pobj.classdesc.name (unchanged) |
pobj.field_data[cd][field] | pobj.field_data[cd][field] (unchanged) |
pobj.get_field("name") | pobj.get_field("name") (unchanged) |
pobj.__getattr__ ambiguity silent | warns when field exists in multiple classes |
transformer._type_mapper (private) | transformer.handles(name) (public) |
JavaArray.data (tuple of ints for bytes) | JavaArray.data (bytes for TYPE_BYTE) |
BlockData compared with bytes | BlockData.__eq__(bytes) still works |
use_numpy_arrays=True (v2 option) | use_numpy_arrays=True (same) |
| No depth/size limits | max_depth=500, max_array_size=100 MiB |
| No typed exceptions | ParseError, SecurityError, … |
Shallow conversion helper (best-effort, for gradual migration):
fromjavaobj.v3._compatimportv2_to_v3v3_obj=v2_to_v3(v2_obj)Note:
v3requires Python 3.12+. For writing Java object streams on older Python versions, usev1.