This project has been manually tranferred to

https://github.com/NonVolatileComputing/Mnemonic

bdmem: Big Data Memory Library

A hybrid memory resources management library. It is featured with in-place non-volatile Java object programming model.

Features:

• In-place data storage on local non-volatile memory
• In-place generic Java object persistence
• Lazily data object loading
• Any map-able device could be used as a non-volatile memory resource
• Reclaim allocated memory when it is no longer used
• Hierarchical cache pool for massive data caching
• A set of persistent data structures is provided by bdmemgeneric project
• Minimize memory footprint on Java heap
• Reduce GC Overhead as following data shown (collected from Apache Spark experiments)

How to use it ?

Define a durable class:

/** * a durable class should be abstract and implemented from Durable interface with @PersistentEntity annotation */@PersistentEntitypublicabstractclassPerson<E> implementsDurable, Comparable<Person<E>> {
Eelement; // Generic Type/** * callback for brand new durable object creation */@OverridepublicvoidinitializeAfterCreate() { System.out.println("Initializing After Created");
}
/** * callback for durable object recovery */@OverridepublicvoidinitializeAfterRestore() { System.out.println("Initializing After Restored");
}
/** * setup generic info manually to avoid performance penalty */@OverridepublicvoidsetupGenericInfo(EntityFactoryProxy[] efproxies, GenericField.GType[] gftypes) {
}
@TestpublicvoidtestOutput() throwsRetrievePersistentEntityError {
System.out.printf("Person %s, Age: %d ( %s ) \n", getName(), getAge(),
null == getMother()? "No Recorded Mother" : "Has Recorded Mother");
}
publicintcompareTo(Person<E> anotherPerson) {
intret = 0;
if (0 == ret) ret = getAge().compareTo(anotherPerson.getAge());
if (0 == ret) ret = getName().compareTo(anotherPerson.getName());
returnret;
}
/** * Getters and Setters for persistent fields with @persistentGetter and @PersistentSetter */@PersistentGetterabstractpublicShortgetAge();
@PersistentSetterabstractpublicvoidsetAge(Shortage);
@PersistentGetterabstractpublicStringgetName() throwsRetrievePersistentEntityError;
@PersistentSetterabstractpublicvoidsetName(Stringname, booleandestroy) throwsOutOfPersistentMemory, RetrievePersistentEntityError;
@PersistentGetterabstractpublicPerson<E> getMother() throwsRetrievePersistentEntityError;
@PersistentSetterabstractpublicvoidsetMother(Person<E> mother, booleandestroy) throwsRetrievePersistentEntityError;
@PersistentGetterabstractpublicPerson<E> getFather() throwsRetrievePersistentEntityError;
@PersistentSetterabstractpublicvoidsetFather(Person<E> mother, booleandestroy) throwsRetrievePersistentEntityError;
}

Use a durable class:

Setup an allocator for durable objects.

// create an allocator object with parameters ie. capacity and uriBigDataPMemAllocatoract = newBigDataPMemAllocator(1024 * 1024 * 8, "./pobj_person.dat", true);
// fetch underlying capacity of key-value pair store for persistent handler storageKEYCAPACITY = act.persistKeyCapacity();
....
// close it after useact.close();

Generate structured durable objects.

// create a new durable person object from specific allocatorperson = PersonFactory.create(act);
// set attributesperson.setAge((short)rand.nextInt(50));
person.setName(String.format("Name: [%s]", UUID.randomUUID().toString()), true);
// keep this person on persistent key-value pair storeact.setPersistKey(keyidx, person.getPersistentHandler());
for (intdeep = 0; deep < rand.nextInt(100); ++deep) {
// create another person as mothermother = PersonFactory.create(act);
mother.setAge((short)(50 + rand.nextInt(50)));
mother.setName(String.format("Name: [%s]", UUID.randomUUID().toString()), true);
// set the person's motherperson.setMother(mother, true);
person = mother;
}

Use the durable objects

for (longi = 0; i < KEYCAPACITY; ++i) {
System.out.printf("----------Key %d--------------\n", i);
// iterate persistent handlers from key-value store of specific allocatorval = act.getPersistKey(i);
if (0L == val) {
break;
}
// restore person objects from specific allocatorPerson<Integer> person = PersonFactory.restore(act, val, true);
while (null != person) {
person.testOutput();
// iterate all mother's ancestorsperson = person.getMother();
}
}

How to build it ?

Please see the file LICENSE for information on how this library is licensed.

• src -- the source for the library
• src/main/java -- the Java source for the library
• examples -- Brief examples for this library
• src/main/native -- the native source for the library
• src/test/java -- the Java test & example source for the library
• uml -- modeling documents for the library
• target -- the generated packages for the library
• target/apidocs -- the generated API documents for the library

To build this library, you may need to install some required packages on the build system:

• NVML -- the NVM library (Please compile this library that is tagged with 0.1+b16) (http://pmem.io) [Required]
• JDK -- the Java Develop Kit 1.6 or above (please properly configure JAVA_HOME) [Required]
• PMFS -- the PMFS should be properly installed and configured on Linux system if you want to simulate read latency [Optional]
• PMalloc -- the supported durable memory native library at https://github.com/bigdata-memory/pmalloc.git [Required]
• Javapoet -- the 1.3.1-SNAPSHOT revised for bdmem at https://github.com/wewela/javapoet.git [Required]

Once the build system is setup, the Big Memory Library is built using this command at the top level:

	$ mvn clean package -DskipTests -Dmaven.javadoc.skip=true -Dmaven.test.skip=true

To build and run the unit tests:

	$ mvn clean package

To install this package to local repository:

 $ mvn clean install

To build examples:
(Note that the Big Data Memory Library should be installed to local repository at first):

 $ cd examples
$ mvn clean package

To run an example:

 $ cd examples
$ java -jar target/examples-X.X.X(-SSSSS).jar