ByteArray

Description

A Java library that provides convenient and safe classes for working with byte[] arrays and ByteBuffers, offering fine-grained read-only, write-only, and readable-writable access modes, along with support for creating lightweight views (subsets) of the byte array data.

Install

Just install using Maven 3:

mvn clean install -DskipTests

Omit -DskipTests to include tests in the build. Note that tests are long-running and may take significant time to complete.

Motivation

The motivation came from working with large binary data file containing rows of records whose structure is not known ahead of the time. For performance reasons, I relied on reading these files using large ByteBuffers. I frequently ran into issues where a ByteBuffer would stop in the middle of a record, requiring me to manually stitch together data from two buffers. This made handling records awkward and often introduced extra byte copying or additional bookkeeping. ByteBuffer is an abstract class, which makes it difficult to extend in a meaningful way. In particular, parts of its internal implementation rely on package-private methods, preventing clean extension outside of java.nio package. Moving custom implementation into java.nio is not a practical option.

Another issue is that ByteBuffer is mutable by default. While it can be made read-only using asReadOnlyBuffer(), this is only enforced at runtime and can easily lead to mistakes where put operations result in ReadOnlyBufferException. There is no compile-time guarantee for access restriction.

This led to the need for a new design based on an interface-first approach. The goal was to support read-only, write-only, and read-write buffer types with compile-time enforcement, along with the ability to define sub-ranges of data to represent records and to compose multiple buffers into a single logical view of contiguous data.

Features

• interface-first design
  • Easily extendable to fit custom use cases
  • Initial support of byte[] arrays and ByteBuffers
• Compile-time access controls
  • Read-only class
  • Write-only class
  • Readable-Writable class
• Support for subsetting data
• Support for joining multiple arrays of data
• 64-bit addressing space support

Available classes

• ByteArray
  • Base interface for byte array access
  • Supports subsetOf(long,long) and size() methods
• ByteArrays
  • Static factory methods for creating instances from byte[], ByteBuffers, and for joining multiple ByteArrays
    • wrap(byte[]) -> ReadableWritableByteArray
    • wrap(ByteBuffer) -> ReadableWritableByteArray
    • combine(ReadableWritableByteArray,ReadableWritableByteArray,ReadableWritableByteArray...) -> ReadableWritableByteArray
    • combine(ReadOnlyByteArray,ReadOnlyByteArray,ReadOnlyByteArray...) -> ReadOnlyByteArray
    • combineReadableWritable(List<? extends ReadableWritableByteArray>) -> ReadableWritableByteArray
    • combineReadOnly(List<? extends ReadOnlyByteArray>) -> ReadOnlyByteArray
• ReadOnlyByteArray
  • Read-only view of ByteArray
  • Provides methods for reading ByteArrays and primitive values in big-endian (BE) and little-endian (LE) formats:
    • copyTo(long, WriteOnlyByteArray)
    • readByte(long)
    • readShortBE(long)
    • readShortLE(long)
    • readIntBE(long)
    • readIntLE(long)
    • readLongBE(long)
    • readLongLE(long)
    • readFloatBE(long)
    • readFloatLE(long)
    • readDoubleBE(long)
    • readDoubleLE(long)
  • IMPORTANT: Not strictly immutable version. If the underlying data or any associated ReadableWritableByteArray is modified, changes are reflected in this view. It can only be made practically immutable if the backing data is protected from mutation.
• WriteOnlyByteArray
  • Write-only version of ByteArray
  • Provides methods for writing ByteArrays and primitive values in big-endian (BE) and little-endian (LE) formats:
    • copyFrom(long, ReadOnlyByteArray)
    • writeByte(long, byte)
    • writeShortBE(long, short)
    • writeShortLE(long, short)
    • writeIntBE(long, int)
    • writeIntLE(long, int)
    • writeLongBE(long, long)
    • writeLongLE(long, long)
    • writeFloatBE(long, float)
    • writeFloatLE(long, float)
    • writeDoubleBE(long, double)
    • writeDoubleLE(long, double)
• ReadableWritableByteArray
  • Read-write version of ByteArray
  • Combination of ReadOnlyByteArray and WriteOnlyByteArray interfaces which includes all read/copyTo and write/copyFrom methods
  • Provides toReadOnly() and toWriteOnly() methods to convert ReadableWritableByteArray to either ReadOnlyByteArray or WriteOnlyByteArray to restrict access at compile-time.

Examples

Wrapping byte[] array and manipulating it

// Create the original byte arrayvarbytes = newbyte[]{22, 9, 20};
// Wrap it and create ReadableWritableByteArrayvarbyteArray = ByteArrays.wrap(bytes);
// Update 2nd element on bytes arraybyteArray.writeByte(1, 5); // This will also update 'bytes' array because it's the 'backing' data// AssertassertbyteArray.readByte(1) == 5;
assertbytes[1] == 5;

Wrapping and join multiple ByteBuffers and using subsetOf(long,long) to get a view of desired data portion in the data

// Create large bytebuffersvarbyteBuffer1 = ByteBuffer.allocate(1_000_000);
varbyteBuffer2 = ByteBuffer.allocate(1_000_000);
// Example read 2MB of data into bytebuffersreadData(byteBuffer1);
readData(byteBuffer2);
// Example: Our desired row is between the buffers. // (First some bytes on the end of first ByteBuffer and the rest of bytes on the beginning of second ByteBuffer.)// Wrap both buffersvarbyteArray1 = ByteArrays.wrap(byteBuffer1);
varbyteArray2 = ByteArrays.wrap(byteBuffer2);
// Combine the data by joining byte arraysvarlargeByteArray = ByteArrays.combine(byteArray1, byteArray2);
// Subset the large byte array to get data of our desired rowvarrowData1 = largeByteArray.subsetOf(999_000, 2_000); // 2,000 bytes long ByteArray where 1,000 bytes each from both buffersassertrowData1.size() == 2_000;
// Another quicker way of same example:varrowData2 = ByteArrays.wrap(byteBuffer1, byteBuffer2).subsetOf(999_000, 2_000); // wrap method is a variadic method, can accept as many `ByteBuffer`s as you can fit.assertrowData1.size() == 2_000;

Compile-time Access Control Demonstration

// Create example byte arraysvarbytes1 = newbyte[]{1, 2, 3, 4};
varbytes2 = newbyte[]{5, 6, 7, 8};
// Create ReadableWritableByteArrayvarrwByteArray = ByteArrays.wrap(bytes1, bytes2); // wrap method is a variadic method, can accept as many `byte[]`s as you can fit.// ReadableWritableByteArray is fully read-write so you can read and write whatever you wantassertrwByteArray.readByte(5) == 6;
rwByteArray.writeByte(5, -6); // This updates 2nd element in bytes2 because bytes2 is the backing data of this ReadableWritableByteArrayassertrwByteArray.readByte(5) == -6;
assertbytes2[1] == -6;
// Create read-only viewvarreadonly = rwByteArray.toReadOnly();
assertreadonly.readByte(5) == -6;
// readonly.writeByte(5, 12); // Compile error because writeByte(long,byte) does not exist in ReadOnlyByteArray// Create write-only viewvarwriteonly = rwByteArray.toWriteOnly();
writeonly.writeByte(2, 30); // This updates to rwByteArray's 3rd element and also updates to bytes1 arrays' 3rd element.// writeonly.readByte(2); // Compile error because readByte(long) does not exist in WriteOnlyByteArray// Assertions to prove that writes to write-only ByteArray will update to all related dataassertreadonly.readByte(2) == 30;
assertrwByteArray.readByte(2) == 30;
assertbytes1[2] == 30;

Copying ByteArrays

// Create example byte arraysvarbytes = newbyte[]{1, 2, 3, 4, 5, 6, 7, 8};
// Create ByteArrayvarlargeByteArray = ByteArrays.wrap(bytes);
// Create destination ByteBuffer to fill data while reading largeByteArrayvarsmallerBB = ByteBuffer.allocate(3);
vardestination = ByteArrays.wrap(smallerBB);
// Read bytes from largeByteArray into destination byte arraylargeByteArray.read(2, destination);
// Both smallerBB and destination should have 3, 4, 5 in them after that read call// Create source ByteBuffer to write data to largeByteArrayvarsmallerBytes = newbyte[]{4, 3, 2, 1};
varsource = ByteArrays.wrap(smallerBytes);
// Write bytes from source byte array into largeByteArraylargeByteArray.write(4, source);
// bytes and largeByteArray should have 1, 2, 3, 4, 4, 3, 2, 1 after that write call// Have largeByteArray to subset and read its contents to source bytearraylargeByteArray.subsetOf(2, 4).read(0, source);
// source and smallerBytes should now have 3, 4, 4, 3 after that read call