Squiggly Filter For Jackson

THIS PROJECT IS NO LONGER MAINTAINED

Apologies to everyone. I just don't have any time to maintain this. Feel free to fork or let me know if you're interested in taking this over.

Important Note

As of version 1.3.2, the preferred way to specify nested filters is to use square brackets intead of braces.

Preferred: assignee[firstName]

No longer Preferred but will still work: assignee{firstName}

The reason for this is that newer versions of Tomcat no longer allow braces to be specified on the url without being escaped. Square brackets are still permitted in the url and it is preferred to make the syntax url friendly.

Squiggly Filter For Jackson

What is it?

The Squiggly Filter is a Jackson JSON PropertyFilter, which selects properties of an object/list/map using a subset of the Facebook Graph API filtering syntax.

Probably the most common use of this library is to filter fields on the querystring like so:

?fields=id,reporter[firstName]

Integrating Squiggly into your webapp is covered in Custom Integration.

Requirements

Installation

Maven

<dependency>
<groupId>com.github.bohnman</groupId>
<artifactId>squiggly-filter-jackson</artifactId>
<version>1.3.18</version>
</dependency>

General Usage

ObjectMapperobjectMapper = Squiggly.init(newObjectMapper(), "assignee{firstName}");
Issueobject = newIssue(); // replace this with your object/collection/map hereSystem.out.println(SquigglyUtils.stringify(objectMapper, object));

Alternatively, if you need more control over configuring the ObjectMapper, you can do it this way:

StringfilterId = SquigglyPropertyFilter.FILTER_ID;
SquigglyPropertyFilterpropertyFilter = newSquigglyPropertyFilter("assignee[firstName]"); // replace with your filter hereSimpleFilterProviderfilterProvider = newSimpleFilterProvider().addFilter(filterId, propertyFilter);
ObjectMapperobjectMapper = newObjectMapper();
objectMapper.setFilterProvider(filterProvider);
objectMapper.addMixIn(Object.class, SquigglyPropertyFilterMixin.class);
Issueobject = newIssue(); // replace this with your object/collection/map hereSystem.out.println(SquigglyUtils.stringify(objectMapper, object));

Also, you can generate a Plain Old Java Object (POJO) instead of a JSON String

ObjectMapperobjectMapper = Squiggly.init(newObjectMapper(), "assignee{firstName}");
System.out.println(SquigglyUtils.objectify(objectMapper, object, Object.class));

For applying filter on Collection of Objects and for returning Collection of POJOs instead of JSON String

List<User> users = Arrays.asList(
newUser("Peter", 12, "Dinklage"), newUser("Lena", 13, "Heady"));
Stringfilter = "firstName,age";
ObjectMapperobjectMapper = Squiggly.init(newObjectMapper(), filter); List<User> filteredUsers = SquigglyUtils.listify(objectMapper, users, User.class);
// setify is also availble

Reference Object

For the filtering examples, let's use an the example object of type Issue

{
"id": "ISSUE-1",
"issueSummary": "Dragons Need Fed",
"issueDetails": "I need my dragons fed pronto.",
"reporter": {
"firstName": "Daenerys",
"lastName": "Targaryen"
},
"assignee": {
"firstName": "Jorah",
"lastName": "Mormont"
},
"actions": [
{
"id": null,
"type": "COMMENT",
"text": "I'm going to let Daario get this one.",
"user": {
"firstName": "Jorah",
"lastName": "Mormont"
}
},
{
"id": null,
"type": "CLOSE",
"text": "All set.",
"user": {
"firstName": "Daario",
"lastName": "Naharis"
}
}
],
"properties": {
"priority": "1",
"email": "motherofdragons@got.com"
}
}

Top-Level Filters

Select No Fields

Stringfilter = "";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {}

Select Single Field

filter = "id";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1"}

Select Multiple Fields

filter = "id,issueSummary"ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1", "issueSummary":"Dragons Need Fed"}

Select Fields Using Wildcards

filter = "issue*";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed", "issueDetails": "I need my dragons fed pronto."}

Select All Fields

filter = "**";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints the same json as our example object

Select All Fields of object, but only base fields of associated objects (more on this later)

filter = "*";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
/* prints the following:{ "id": "ISSUE-1", "issueSummary": "Dragons Need Fed", "issueDetails": "I need my dragons fed pronto.", "reporter": { "firstName": "Daenerys", "lastName": "Targaryen" }, "assignee": { "firstName": "Jorah", "lastName": "Mormont" }, "actions": [ { "id": null, "type": "COMMENT", "text": "I'm going to let Daario get this one.." }, { "id": null, "type": "CLOSE", "text": "All set." } ], "properties": { "priority": "1", "email": "motherofdragons@got.com" }}*/

Nested Filters

Select Single Nested Field

Stringfilter = "assignee[firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}

Select Multiple Nested Fields

Stringfilter = "actions[text,type]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"type":"COMMENT","text":"I'm going to let Daario get this one.."},{"type":"CLOSE","text":"All set."}]}// NOTE: use can also use wildcards (e.g. actions{t*})

Select Same Field From Different Nested Objects

Stringfilter = "(assignee,reporter)[firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"reporter":{"firstName":"Daenerys"},"assignee":{"firstName":"Jorah"}}

Select Deeply Nested Field

Stringfilter = "actions[user[lastName]]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"lastName":"Mormont"}},{"user":{"lastName":"Naharis"}}]}

Dot Syntax

As an alternative to using the braces syntax for nested filter, you can use the dot syntax

Stringfilter = "assignee.firstName";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}

You can exclude fields using the dot syntax. Note that the exclusion applies to the last field.

Stringfilter = "-assignee.firstName";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"lastName":"Mormont"}}

You can also combine the dot syntax with the nested syntax.

Stringfilter = "actions.user[firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"firstName":"Jorah"}},{"user":{"firstName":"Daario"}}]}

One limitation is that you cannot use the | syntax with the dot syntax

Stringfilter = "(actions.user,assignee)[firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws exception

Regex Filters

In addition to using wildcards, you can also use regular expressions.

Here's an example:

Stringfilter = "~iss[a-z]e.*~";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed","issueDetails":"I need my dragons fed pronto."}

Notice the tildes mark the begin and of the regex pattern.

You can also specifiy a case insensitive match.

Stringfilter = "~iss[a-z]esumm.*~i";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}

Why use tildes and not forward slashes for regular expressions? Tildes are query string friendly and forward slashes are not.

However, you may use forward slashes if you like.

Stringfilter = "/iss[a-z]esumm.*/i";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}

Other Filters

Selecting from Maps

Selecting from maps is the same as selecting from objects. Instead of selecting from fields, you are selecting from keys. The main downside of selecting from maps that their matches are unable to be cached.

Map<String, Object> map = newHashMap<>();
map.put("foo", "bar");
map.put("bear", "baz");
Stringfilter = "foo";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, map));
// prints {"foo":"bar"}

Selecting from Collections (Lists/Arrays/Etc).

Selecting from collection just assumes the top-level objects are the elements in the collection, not the collection itself.

List<User> list = Arrays.asList(
newUser("Peter", "Dinklage"),
newUser("Lena", "Heady")
);
Stringfilter = "firstName";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, list));
// prints [{"firstName":"Peter"}, {"firstName":"Lena"}]

Resolving Conflicts

When a filter includes two criteria that match the same field, the one that is more specific wins.

For example, if the filter is "**,reporter[firstName]", then all fields will be excluded. However, the reporter field will only include the firstName field.

Specificity is determined using the following logic:

an exact name is the most specific
a ** is the least specific
a * is the second to least specific
otherwise, the number of non-wildcard characters is counted, the higher the number, the more specific
if two filters have the same specificity, the latter one is chosen

Excluding Fields

In order to exclude fields, you need to prefix the field name with a minus sign (-).

Let's look at an example

Stringfilter = "-reporter";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything except the reporter field

Here's an example excluding a nested field:

Stringfilter = "**,reporter[-firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything, the reporter object will only have the firstName excluded

NOTE: Excluded fields can't have nested filters

Stringfilter = "**,-reporter[firstName]";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws an exception

Property Views

In addition to selecting fields by name, you can assign a name to a group of fields. This is called a property view.

Reference Objects

Let's use these reference objects for the examples.

@Target(FIELD)
@Retention(RUNTIME)
@Documented@PropertyView({"super"})
public @interface SuperView {
}
classAddress {
Stringline1 = "55 Hollywood Blvd.";
Stringline2 = "";
Stringcity = "Hollywood";
Stringstate = "CA";
@SuperViewdoublelat;
@SuperViewdoublelon;
}
classUser {
StringfirstName = "Peter";
StringlastName = "Dinklage";
@PropertyView("secret")
Stringphone = "555-555-1212";
@SuperViewAddressaddress;
}

The Base View

If nothing is annotated on a field, it is assumed to belong to the "base" view. There is @BaseView convenience annotation, but it's not needed. See Changing Defaults to alter this behavior.

In the case of User, fields firstName and lastName belong to the "base" view.

Stringfilter = "base";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage"}

Using the @PropertyView Annotation

If you look at the phone field of the User class, you'll notice the @PropertyView("secret") annotation on the phone field. This indicates that the phone field belongs to the "secret" view.

Stringfilter = "secret";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "phone":"555-555-1212"}

Wait a minute! Why was the firstName and lastName field included? Even though we specified a certain view, the base fields are always included. See Changing Defaults to alter this behavior.

Note that you can also specifiy multiple views in the annotation - @PropertyView({"one", "two", "three"}))

You can also specify @PropertyView on getters and setters.

Using a Derived Annotation

If you look at the address field of the User class, you'll notice the @SuperView annotation. Looking at the @SuperView declaration, you'll notice it is annotated with a @PropertyView("super"). This is how you create a derived annotation.

Let's try it out.

Stringfilter = "super";
ObjectMappermapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "address":{"line1":"55 Hollywood Blvd.","line2":"","city":"Hollywood","state":"CA"}}

Wait another minute! The Address class has @SuperView annotations as well. Why weren't they include? Well, the view only applies to the current level. In order to get the super views of the address, you would have to specifiy a filter "super[super]". See Changing Defaults to alter this behavior.

More Examples

There are more examples in the test directory.

Custom Integration

Imagine you are building a webapp where you want to specify the fields on the querystring.

E.g. `/some/path?fields=a,b{c} ``

You'll notice in all of our examples, we passed in a filter expression that never changes. This doesn't work well for the case of specifying filters on a querystring.

Enter the SquigglyContextProvider. This interface allows you to customize how to retrieve the fields.

The RequestSquigglyContextProvider

All servlet-based integrations use the RequestSquigglyContextProvider, which has the general initialization in the form of:

Squiggly.init(objectMapper, newRequestSquigglyContextProvider());

Automatically wrapping fields with an outer filter

Let's say you have the following class called Page that looks like this:

publicclassPage<T> {
privatefinalintpageNumber;
privatefinalintpageSize;
privatefinalList<T> items;
publicListResponse(List<T> items, intpageNumber, intpageSize) {
this.items = checkNotNull(items);
this.pageNumber = pageNumber;
this.pageSize = pageSize;
}
publicList<T> getItems() {
returnitems;
}
publicintgetPageNumber() {
returnpageNumber;
}
publicintgetPageSize() {
returnpageSize;
}
}

Let's say you have an endpoint called /issues that looks like this:

publicPage<Issue> findIssues(Stringquery, intpageNumber, intpageSize) {
List<Issue> issues = issueService.findIssues(query, pageNumber, pageSize);
returnnewPage<Issue>(issues, pageNumber, pageSize);
}

In order to get specify the issue property, you now have to wrap all filters with items[] like so:

GET /issues?fields=items{id}&query=some-query&&pageNumber=1&pageSize=10

This is kind of annoying. Fortunately, we can avoid this inconvenience by using a hook method in RequestSquigglyContextProvider.

Here's how it would look:

Squiggly.init(objectMapper, newRequestSquigglyContextProvider() {
@OverrideprotectedStringcustomizeFilter(Stringfilter, HttpServletRequestrequest, ClassbeanClass) {
if (filter != null && Page.class.isAssignableFrom(beanClass)) {
filter = "items[" + filter + "]";
}
returnfilter;
}
});

Now you can do this:

GET /issues?fields=id&query=some-query&&pageNumber=1&pageSize=10

Generic Servlet Webapp

You can find an example of using Squiggly Filter in a webapp under the examples/servlet directory.

Spring Boot Web Application

You can find an example of using Squiggly Filter in Spring Boot under the examples/spring-boot directory.

Dropwizard

You can find an example of using Squiggly Filter in Dropwizard under the examples/dropwizard directory.

Changing Defaults

You have the ability to customize Squiggly by creating a file called squiggly.properties in the root of the classpath.

Cache Config

The following properties are used to control various caches in Squiggly Filter. Internally, these properties get converted to a Guava CacheBuilderSpec. Please refer to to the documentation to see all the values that are available.

parser.nodeCache.spec=maximumSize=10000
filter.pathCache.spec=maximumSize=10000
property.descriptorCache.spec=<empty>

Enable/Disable adding non-annotated fields to the "base" view

property.addNonAnnotatedFieldsToBaseView=true

Enable/Disable inclusion of base fields for nested objects

filter.implicitlyIncludeBaseFields=true

Enable/Disable inclusion of base fields when a view is specified

filter.implicitlyIncludeBaseFieldsInView=true

When set to false, base fields are not included when specifying a view

Enable/Disable View Propagation to Nested Filters

filter.propagateViewToNestedFilters=false

When set to true, views are propagated to nested filters

Getting Config Info

Squiggly Filter provides 2 methods to get information about configuration.

SquigglyConfig.asMap() will return a map of the merged config that looks like the following:

{
"filter.implicitlyIncludeBaseFields": "true",
"filter.implicitlyIncludeBaseFieldsInView": "true",
"filter.pathCache.spec": "maximumSize=10000",
"filter.propagateViewToNestedFilters": "false",
"parser.nodeCache.spec": "maximumSize=10000",
"property.addNonAnnotatedFieldsToBaseView": "true",
"property.descriptorCache.spec": ""
}

SquigglyConfig.asSourceMap() will return a map of the config keys and paths where the entry was retrieved that looks like the following:

{
"filter.implicitlyIncludeBaseFields": "file:/path/one/squiggly.default.properties",
"filter.implicitlyIncludeBaseFieldsInView": "file:/path/one/squiggly.default.properties",
"filter.pathCache.spec": "file:/path/one/squiggly.default.properties",
"filter.propagateViewToNestedFilters": "file:/path/one/squiggly.default.properties",
"parser.nodeCache.spec": "file:/path/two/squiggly.properties",
"property.addNonAnnotatedFieldsToBaseView": "file:/path/two/squiggly.properties",
"property.descriptorCache.spec": "file:/path/two/squiggly.properties"
}

Metrics

Squiggly Filter provides an API for obtaining various metrics about the library, such as cache statistics. This allows users to monitor and adjust configuration as needed.

To use the metrics, you can do something the like following:

Map<String, Object> metrics = SquigglyMetrics.asMap();
System.out.println(SquigglyUtils.stringify(newObjectMapper(), metrics));

This will print the following:

{
"squiggly.filter.pathCache.averageLoadPenalty": 0,
"squiggly.filter.pathCache.evictionCount": 0,
"squiggly.filter.pathCache.hitCount": 0,
"squiggly.filter.pathCache.hitRate": 1,
"squiggly.filter.pathCache.loadExceptionCount": 0,
"squiggly.filter.pathCache.loadExceptionRate": 0,
"squiggly.filter.pathCache.loadSuccessCount": 0,
"squiggly.filter.pathCache.missCount": 0,
"squiggly.filter.pathCache.missRate": 0,
"squiggly.filter.pathCache.requestCount": 0,
"squiggly.filter.pathCache.totalLoadTime": 0,
"squiggly.parser.nodeCache.averageLoadPenalty": 0,
"squiggly.parser.nodeCache.evictionCount": 0,
"squiggly.parser.nodeCache.hitCount": 0,
"squiggly.parser.nodeCache.hitRate": 1,
"squiggly.parser.nodeCache.loadExceptionCount": 0,
"squiggly.parser.nodeCache.loadExceptionRate": 0,
"squiggly.parser.nodeCache.loadSuccessCount": 0,
"squiggly.parser.nodeCache.missCount": 0,
"squiggly.parser.nodeCache.missRate": 0,
"squiggly.parser.nodeCache.requestCount": 0,
"squiggly.parser.nodeCache.totalLoadTime": 0,
"squiggly.property.descriptorCache.averageLoadPenalty": 0,
"squiggly.property.descriptorCache.evictionCount": 0,
"squiggly.property.descriptorCache.hitCount": 0,
"squiggly.property.descriptorCache.hitRate": 1,
"squiggly.property.descriptorCache.loadExceptionCount": 0,
"squiggly.property.descriptorCache.loadExceptionRate": 0,
"squiggly.property.descriptorCache.loadSuccessCount": 0,
"squiggly.property.descriptorCache.missCount": 0,
"squiggly.property.descriptorCache.missRate": 0,
"squiggly.property.descriptorCache.requestCount": 0,
"squiggly.property.descriptorCache.totalLoadTime": 0
}

Limitations

Using Serializers

If you use a custom serializer and write to a JsonGenerator directly, you will completely bypass Squiggly.

For example:

// Doesn't work with SquigglypublicclassTestSerializerextendsJsonSerializer<TestObject> {
@Overridepublicvoidserialize(TestObjectvalue, JsonGeneratorjgen, SerializerProviderprovider) throwsIOException {
jgen.writeStartObject();
jgen.writeStringField("a", value.getA());
jgen.writeStringField("c", value.getC());
jgen.writeEndObject();
}
}

Instead, you need to use the SerializerProvider, which will invoke Squiggly. The above serializer can be rewritten as:

// Works with SquigglypublicclassTestSerializerextendsJsonSerializer<TestObject> {
@Overridepublicvoidserialize(TestObjectvalue, JsonGeneratorjgen, SerializerProviderprovider) throwsIOException {
Map<String, Object> map = newHashMap<>();
map.put("a", value.getA());
map.put("c", value.getC());
provider.defaultSerializeValue(map, jgen);
}
}

Name		Name	Last commit message	Last commit date
examples		examples
src		src
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE.md		LICENSE.md
README.md		README.md
check-maven-central.sh		check-maven-central.sh
install.sh		install.sh
pom.xml		pom.xml
release.sh		release.sh

Latest commit

History

Folders and files

Repository files navigation

THIS PROJECT IS NO LONGER MAINTAINED

Important Note

Squiggly Filter For Jackson

Contents

What is it?

Requirements

Installation

Maven

General Usage

Reference Object

Top-Level Filters

Select No Fields

Select Single Field

Select Multiple Fields

Select Fields Using Wildcards

Select All Fields

Select All Fields of object, but only base fields of associated objects (more on this later)

Nested Filters

Select Single Nested Field

Select Multiple Nested Fields

Select Same Field From Different Nested Objects

Select Deeply Nested Field

Dot Syntax

Regex Filters

Other Filters

Selecting from Maps

Selecting from Collections (Lists/Arrays/Etc).

Resolving Conflicts

Excluding Fields

Property Views

Reference Objects

The Base View

Using the @PropertyView Annotation

Using a Derived Annotation

More Examples

Custom Integration

The RequestSquigglyContextProvider

Automatically wrapping fields with an outer filter

Generic Servlet Webapp

Spring Boot Web Application

Dropwizard

Changing Defaults

Cache Config

Enable/Disable adding non-annotated fields to the "base" view

Enable/Disable inclusion of base fields for nested objects

Enable/Disable inclusion of base fields when a view is specified

Enable/Disable View Propagation to Nested Filters

Getting Config Info

Metrics

Limitations

Using Serializers

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages