NestedFilter

Structure-preserving filter/reject for nested maps and lists: drop or take keys and values at any depth without flattening or losing data. Zero runtime dependencies.

Map.take/2 and Map.drop/2 only see the top level. NestedFilter walks the whole structure — maps inside maps, maps inside lists — and never merges sibling branches or invents values: what survives is always at the path where it appeared in the input.

Every example below is copied verbatim from a doctest, so it runs exactly as shown.

Recipes

Clean params before insert

Drop nil and blank values at any depth before handing user input to a changeset or query:

iex>params=%{"name"=>"Ada","bio"=>nil,"address"=>%{"city"=>"London","zip"=>""}}iex>NestedFilter.drop_by_value(params,[nil,""])%{"name"=>"Ada","address"=>%{"city"=>"London"}}

Strip nils before JSON encoding

Remove every nil entry so encoded payloads carry no null noise:

iex>payload=%{id: 7,tags: ["a","b"],meta: %{source: nil,ip: "1.2.3.4"}}iex>NestedFilter.reject(payload,fn_k,v->is_nil(v)end)%{id: 7,tags: ["a","b"],meta: %{ip: "1.2.3.4"}}

Compact nested payloads

Remove nil map values and prune containers left empty by that cleanup:

iex>NestedFilter.compact(%{a: 1,b: nil,c: %{d: nil},e: %{f: 1,g: nil}})%{a: 1,e: %{f: 1}}

Drop sensitive keys everywhere

Remove known-bad keys wherever they appear, however deeply nested:

iex>event=%{user: %{email: "ada@example.com",password: "s3cret"},session: %{token: "abc",ttl: 60}}iex>NestedFilter.drop_by_key(event,[:password,:token])%{user: %{email: "ada@example.com"},session: %{ttl: 60}}

Take fields, structure preserved

Keep only the fields you care about without flattening or losing duplicates across branches:

iex>order=%{buyer: %{id: 1,name: "Ada"},items: [%{id: 10,sku: "X"},%{id: 11,sku: "Y"}]}iex>NestedFilter.take_by_key(order,[:id])%{buyer: %{id: 1},items: [%{id: 10},%{id: 11}]}

Take a field everywhere, keeping the shape

Pass empties: :keep so branches without a match stay present as empty containers instead of being pruned — handy when downstream code expects a stable shape:

iex>records=%{active: %{id: 1,name: "Ada"},pending: %{name: "Bo"}}iex>NestedFilter.take_by_key(records,[:id],empties: :keep)%{active: %{id: 1},pending: %{}}

Sanitize logs

Redact by pattern when the exact key names aren't known up front:

iex>log=%{"msg"=>"login ok","user_password"=>"hunter2","ctx"=>%{"api_token"=>"xyz"}}iex>NestedFilter.reject(log,fnk,_v->is_binary(k)and(k=~"password"ork=~"token")end)%{"msg"=>"login ok","ctx"=>%{}}

Redact sensitive values

Replace sensitive values at any depth without dropping their keys:

iex>NestedFilter.redact(%{user: %{name: "Ana",password: "hunter2"},token: "abc"},[:password,:token])%{user: %{name: "Ana",password: "[REDACTED]"},token: "[REDACTED]"}

Installation

Add nested_filter to your list of dependencies in mix.exs:

defdepsdo[{:nested_filter,"~> 2.2"}]end

Requires Elixir 1.15 or later. Full documentation is at hexdocs.pm/nested_filter.

API

Two engine functions take a predicate receiving each key and value:

• NestedFilter.reject/3 — recursively remove matching entries
• NestedFilter.filter/3 — recursively keep matching entries, pruning branches without a match; a matched entry is kept whole

Five convenience functions cover the common cases:

• NestedFilter.compact/2 — remove nil map values and optionally prune empty containers or strip nil list elements
• NestedFilter.redact/3 — replace values matching keys or a predicate
• NestedFilter.drop_by_value/3 — remove entries whose value is in a list
• NestedFilter.drop_by_key/3 — remove entries whose key is in a list
• NestedFilter.take_by_key/3 — keep entries whose key is in a list, structure preserved

Semantics worth knowing

• Structure is sacred. Matches stay at the path where they were found. Sibling branches are never merged, so duplicate keys in different branches never clobber each other.
• reject preserves empty maps; filter prunes empty branches. Rejecting every entry of a nested map leaves %{} at its path (add %{} to drop_by_value/3's list to remove those too), while filter drops any branch with no surviving content.
• Lists are traversed, not filtered by value.reject leaves non-map list elements untouched; filter prunes list elements with no surviving content.
• Structs are opaque leaves by default. Pass structs: :convert to recurse into them as plain maps, or structs: :error to raise if one is encountered. See the :structs option on reject/3.

Upgrading from 1.x

Version 2.0 changed take_by_key/3 from flattening (which silently lost data on duplicate keys) to structure-preserving, removed the undocumented drop_by/2 and take_by/2, and raised the Elixir floor to 1.15. See the CHANGELOG for the full migration table.

License

MIT — see LICENSE.