datapackage-java

A Java library for working with Data Packages according to the Frictionless Data specifications.

A Data Package is a simple container format for creating self-contained packages of data. It provides the basis for convenient delivery, installation and management of datasets. It shares some similarity with simple database formats, but lacks a robust query engine, instead focusing on exchanging bundles of related data.

Please find releases on Jitpack

Usage

• Create a Data Package explains how to create a Data Package
• Iterate through Data explains how to iterate through data in Resources
• Edit a Data Package explains how to add or remove Resources or properties to or from a Data Package
• Save to File explains how to save a Data Package to a file
• Working with Foreign Keys explains how to set foreign key constraints in Data Packages
• Contributing contributions are welcome

Create a Data Package

From JSONObject Object

// Create JSON Object for testingJSONObjectjsonObject = newJSONObject("{\"name\": \"test\"}");
// Build resources.JSONObjectresource1 = newJSONObject("{\"name\": \"first-resource\", \"path\": [\"foo.txt\", \"bar.txt\", \"baz.txt\"]}");
JSONObjectresource2 = newJSONObject("{\"name\": \"second-resource\", \"path\": [\"bar.txt\", \"baz.txt\"]}");
ListresourceArrayList = newArrayList();
resourceArrayList.add(resource1);
resourceArrayList.add(resource2);
JSONArrayresources = newJSONArray(resourceArrayList);
// Add the resources.jsonObject.put("resources", resources);
// Build the datapackage.Packagedp = newPackage(jsonObject, true); // Set strict validation to true.

From JSON String

// The path of the datapackage file.Stringfilepath = "/path/to/file/datapackage.json";
// Get string content version of source file.StringjsonString = newString(Files.readAllBytes(Paths.get(filepath)));
// Create DataPackage instance from jsonString.Packagedp = newPackage(jsonString, true); // Set strict validation to true.

From Remote File

URLurl = newURL("https://raw.githubusercontent.com/frictionlessdata/datapackage-java/master/src/test/resources/fixtures/multi_data_datapackage.json");
Packagedp = newPackage(url, true); // Set strict validation to true.

From Local JSON File

StringrelativePath = "datapackage.json";
StringbasePath = "/data";
// Build DataPackage instance based on source file path.Packagedp = newPackage(relativePath, basePath, true); // Set strict validation to true.

From Local Zip File

Packagedp = newPackage("/path/of/zip/file/datapackage.zip", true); // Set strict validation to true.

Exceptions are thrown for the following scenarios:

• The zip archive must contain a file named datapackage.json. If no such file exists, a DataPackageException will be thrown.
• If datapackage.json does exist but it is invalid and validation is enabled then a ValidationException will be thrown.
• If the zip file does not exist, an IOException will be thrown.

Iterate through Data

Without Casting

// Get the resource from the data package.Resourceresource = pkg.getResource("first-resource");
// Set the profile to tabular data resource (if it hasn't been set already).resource.setProfile(Profile.PROFILE_TABULAR_DATA_RESOURCE);
// Get Iterator.Iterator<String[]> iter = resource.iter();
// Iterate.while(iter.hasNext()){
String[] row = iter.next();
Stringcity = row[0];
Stringpopulation = row[1];
} 

With Casting

// Get Iterator. // Third boolean is the cast flag (First on is for keyed and second for extended).
Iterator<Object[]> iter = resource.iter(false, false, true));
// Iterator
while(iter.hasNext()){
Object[] row = iter.next();
String city = row[0];
Integer population = row[1];
} 

Edit a Data Package

Add a Resource

// Create a data package.Packagedp = newPackage();
// Add a resource.Resourceresource = newResource("new-resource", "data.csv");
dp.addResource(resource);

A DataPackageException will be thrown if the name of the new resource that is being added already exists.

Remove a Resource

// Create a data package.URLurl = newURL("https://raw.githubusercontent.com/frictionlessdata/datapackage-java/master/src/test/resources/fixtures/multi_data_datapackage.json");
Packagedp = newPackage(url);
// Remove a resource.dp.removeResource("third-resource");

Add a Property

// Create a data packagePackagedp = newPackage();
// Add a few properties.dp.addProperty("name", "a-unique-human-readable-and-url-usable-identifier");
dp.addProperty("title", "A nice title");
dp.addProperty("id", "b03ec84-77fd-4270-813b-0c698943f7ce");
dp.addProperty("profile", "tabular-data-package");
// Create and add license array.JSONObjectlicense = newJSONObject();
license.put("name", "ODC-PDDL-1.0");
license.put("path", "http://opendatacommons.org/licenses/pddl/");
license.put("title", "Open Data Commons Public Domain Dedication and License v1.0");
JSONArraylicenses = newJSONArray();
licenses.put(license);
dp.addProperty(licenses);

A DataPackageException will be thrown if the key of the new property that is being added already exists.

Remove a Property

// Create a data packagePackagedp = newPackage();
// Add a few properties.dp.addProperty("name", "a-unique-human-readable-and-url-usable-identifier");
dp.addProperty("title", "A nice title");
// Remove the title property.dp.removeProperty("title");

Save to File

JSON File

URLurl = newURL("https://raw.githubusercontent.com/frictionlessdata/datapackage-java/master/src/test/resources/fixtures/multi_data_datapackage.json");
Packagedp = newPackage(url);
dp.save("/destination/path/datapackage.json")

Zip File

URLurl = newURL("https://raw.githubusercontent.com/frictionlessdata/datapackage-java/master/src/test/resources/fixtures/multi_data_datapackage.json");
Packagedp = newPackage(url);
dp.save("/destination/path/datapackage.zip")

Contributing

Found a problem and would like to fix it? Have that great idea and would love to see it in the repository?

Note

Please open an issue before you start working.

It could save a lot of time for everyone and we are super happy to answer questions and help you along the way. Furthermore, feel free to join frictionlessdata Gitter chat room and ask questions.

This project follows the Open Knowledge International coding standards.

Get started:

# install jabba and maven2
$ cd datapackage-java
$ jabba install 17
$ jabba use 17
$ mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
$ mvn test -B

Make sure all tests pass.