feat: not equal range filter

[divine]

Q	A
Branch?	main for features
Tickets	#4546
License	MIT
Doc PR	api-platform/docs#... (awaiting initial review)

This is a follow-up for #4546 with the added test.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[divine]

Not stale...

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[divine]

Not stale....

[nikophil]

LGTM 👌

[nikophil]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

[nesl247]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

Agreed. This belongs to the SearchFilter IMO. I didn't test this out, but

/*
* This file is part of the API Platform project.
*
* (c) Kévin Dunglas <dunglas@gmail.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/
declare(strict_types=1);
namespace ApiPlatform\Doctrine\Orm\Filter;
use ApiPlatform\Api\IdentifiersExtractorInterface;
use ApiPlatform\Api\IriConverterInterface;
use ApiPlatform\Doctrine\Common\Filter\SearchFilterInterface;
use ApiPlatform\Doctrine\Common\Filter\SearchFilterTrait;
use ApiPlatform\Doctrine\Orm\Util\QueryBuilderHelper;
use ApiPlatform\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use ApiPlatform\Exception\InvalidArgumentException;
use ApiPlatform\Metadata\Operation;
use Doctrine\DBAL\Types\Types;
use Doctrine\ORM\Query\Expr\Join;
use Doctrine\ORM\QueryBuilder;
use Doctrine\Persistence\ManagerRegistry;
use Psr\Log\LoggerInterface;
use Symfony\Component\PropertyAccess\PropertyAccess;
use Symfony\Component\PropertyAccess\PropertyAccessorInterface;
use Symfony\Component\Serializer\NameConverter\NameConverterInterface;
/**
* The search filter allows to filter a collection by given properties.
*
* The search filter supports `exact`, `partial`, `start`, `end`, and `word_start` matching strategies:
* - `exact` strategy searches for fields that exactly match the value
* - `partial` strategy uses `LIKE %value%` to search for fields that contain the value
* - `start` strategy uses `LIKE value%` to search for fields that start with the value
* - `end` strategy uses `LIKE %value` to search for fields that end with the value
* - `word_start` strategy uses `LIKE value% OR LIKE % value%` to search for fields that contain words starting with the value
*
* Note: it is possible to filter on properties and relations too.
*
* Prepend the letter `i` to the filter if you want it to be case-insensitive. For example `ipartial` or `iexact`.
* Note that this will use the `LOWER` function and *will* impact performance if there is no proper index.
*
* Case insensitivity may already be enforced at the database level depending on the [collation](https://en.wikipedia.org/wiki/Collation) used.
* If you are using MySQL, note that the commonly used `utf8_unicode_ci` collation (and its sibling `utf8mb4_unicode_ci`)
* are already case-insensitive, as indicated by the `_ci` part in their names.
*
* Note: Search filters with the `exact` strategy can have multiple values for the same property (in this case the
* condition will be similar to a SQL IN clause).
*
* Syntax: `?property[]=foo&property[]=bar`.
*
* <CodeSelector>
* ```php
* <?php
* // api/src/Entity/Book.php
* use ApiPlatform\Metadata\ApiFilter;
* use ApiPlatform\Metadata\ApiResource;
* use ApiPlatform\Doctrine\Orm\Filter\SearchFilter;
*
* #[ApiResource]
* #[ApiFilter(SearchFilter::class, properties: ['isbn' => 'exact', 'description' => 'partial'])]
* class Book
* {
* // ...
* }
* ```
*
* ```yaml
* # config/services.yaml
* services:
* book.search_filter:
* parent: 'api_platform.doctrine.orm.search_filter'
* arguments: [ { isbn: 'exact', description: 'partial' } ]
* tags: [ 'api_platform.filter' ]
* # The following are mandatory only if a _defaults section is defined with inverted values.
* # You may want to isolate filters in a dedicated file to avoid adding the following lines (by adding them in the defaults section)
* autowire: false
* autoconfigure: false
* public: false
*
* # api/config/api_platform/resources.yaml
* resources:
* App\Entity\Book:
* - operations:
* ApiPlatform\Metadata\GetCollection:
* filters: ['book.search_filter']
* ```
*
* ```xml
* <?xml version="1.0" encoding="UTF-8" ?>
* <!-- api/config/services.xml -->
* <?xml version="1.0" encoding="UTF-8" ?>
* <container
* xmlns="http://symfony.com/schema/dic/services"
* xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
* xsi:schemaLocation="http://symfony.com/schema/dic/services
* https://symfony.com/schema/dic/services/services-1.0.xsd">
* <services>
* <service id="book.search_filter" parent="api_platform.doctrine.orm.search_filter">
* <argument type="collection">
* <argument key="isbn">exact</argument>
* <argument key="description">partial</argument>
* </argument>
* <tag name="api_platform.filter"/>
* </service>
* </services>
* </container>
* <!-- api/config/api_platform/resources.xml -->
* <resources
* xmlns="https://api-platform.com/schema/metadata/resources-3.0"
* xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
* xsi:schemaLocation="https://api-platform.com/schema/metadata/resources-3.0
* https://api-platform.com/schema/metadata/resources-3.0.xsd">
* <resource class="App\Entity\Book">
* <operations>
* <operation class="ApiPlatform\Metadata\GetCollection">
* <filters>
* <filter>book.search_filter</filter>
* </filters>
* </operation>
* </operations>
* </resource>
* </resources>
* ```
* </CodeSelector>
*
* @author Kévin Dunglas <dunglas@gmail.com>
*/
final class SearchFilter extends AbstractFilter implements SearchFilterInterface
{
use SearchFilterTrait;
public const DOCTRINE_INTEGER_TYPE = Types::INTEGER;
public function __construct(ManagerRegistry $managerRegistry, IriConverterInterface $iriConverter, PropertyAccessorInterface $propertyAccessor = null, LoggerInterface $logger = null, array $properties = null, IdentifiersExtractorInterface $identifiersExtractor = null, NameConverterInterface $nameConverter = null)
{
parent::__construct($managerRegistry, $logger, $properties, $nameConverter);
$this->iriConverter = $iriConverter;
$this->identifiersExtractor = $identifiersExtractor;
$this->propertyAccessor = $propertyAccessor ?: PropertyAccess::createPropertyAccessor();
}
protected function getIriConverter(): IriConverterInterface
{
return $this->iriConverter;
}
protected function getPropertyAccessor(): PropertyAccessorInterface
{
return $this->propertyAccessor;
}
/**
* {@inheritdoc}
*/
protected function filterProperty(string $property, $value, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, Operation $operation = null, array $context = []): void
{
$notEquals = false;
if (str_ends_with($property, '!')) {
$property = str_replace('!', '', $property);
$notEquals = true;
}
if (
null === $value
|| !$this->isPropertyEnabled($property, $resourceClass)
|| !$this->isPropertyMapped($property, $resourceClass, true)
) {
return;
}
$alias = $queryBuilder->getRootAliases()[0];
$field = $property;
$values = $this->normalizeValues((array) $value, $property);
if (null === $values) {
return;
}
$associations = [];
if ($this->isPropertyNested($property, $resourceClass)) {
[$alias, $field, $associations] = $this->addJoinsForNestedProperty($property, $alias, $queryBuilder, $queryNameGenerator, $resourceClass, Join::INNER_JOIN);
}
$caseSensitive = true;
$strategy = $this->properties[$property] ?? self::STRATEGY_EXACT;
// prefixing the strategy with i makes it case insensitive
if (str_starts_with($strategy, 'i')) {
$strategy = substr($strategy, 1);
$caseSensitive = false;
}
$metadata = $this->getNestedMetadata($resourceClass, $associations);
if ($metadata->hasField($field)) {
if ('id' === $field) {
$values = array_map($this->getIdFromValue(...), $values);
}
if (!$this->hasValidValues($values, $this->getDoctrineFieldType($property, $resourceClass))) {
$this->logger->notice('Invalid filter ignored', [
'exception' => new InvalidArgumentException(sprintf('Values for field "%s" are not valid according to the doctrine type.', $field)),
]);
return;
}
$this->addWhereByStrategy($strategy, $queryBuilder, $queryNameGenerator, $alias, $field, $values, $caseSensitive);
return;
}
// metadata doesn't have the field, nor an association on the field
if (!$metadata->hasAssociation($field)) {
return;
}
$values = array_map($this->getIdFromValue(...), $values);
$associationResourceClass = $metadata->getAssociationTargetClass($field);
$associationFieldIdentifier = $metadata->getIdentifierFieldNames()[0];
$doctrineTypeField = $this->getDoctrineFieldType($associationFieldIdentifier, $associationResourceClass);
if (!$this->hasValidValues($values, $doctrineTypeField)) {
$this->logger->notice('Invalid filter ignored', [
'exception' => new InvalidArgumentException(sprintf('Values for field "%s" are not valid according to the doctrine type.', $field)),
]);
return;
}
$associationAlias = $alias;
$associationField = $field;
if ($metadata->isCollectionValuedAssociation($associationField) || $metadata->isAssociationInverseSide($field)) {
$associationAlias = QueryBuilderHelper::addJoinOnce($queryBuilder, $queryNameGenerator, $alias, $associationField);
$associationField = $associationFieldIdentifier;
}
$this->addWhereByStrategy($strategy, $queryBuilder, $queryNameGenerator, $associationAlias, $associationField, $values, $caseSensitive, $notEquals);
}
/**
* Adds where clause according to the strategy.
*
* @throws InvalidArgumentException If strategy does not exist
*/
protected function addWhereByStrategy(string $strategy, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $alias, string $field, mixed $values, bool $caseSensitive, bool $notEquals): void
{
if (!\is_array($values)) {
$values = [$values];
}
$wrapCase = $this->createWrapCase($caseSensitive);
$valueParameter = ':'.$queryNameGenerator->generateParameterName($field);
$aliasedField = sprintf('%s.%s', $alias, $field);
if (!$strategy || self::STRATEGY_EXACT === $strategy) {
if (!$notEquals && 1 === \count($values)) {
$queryBuilder
->andWhere($queryBuilder->expr()->eq($wrapCase($aliasedField), $wrapCase($valueParameter)))
->setParameter($valueParameter, $values[0]);
return;
}
if ($notEquals && 1 === \count($values)) {
$queryBuilder
->andWhere($queryBuilder->expr()->neq($wrapCase($aliasedField), $wrapCase($valueParameter)))
->setParameter($valueParameter, $values[0]);
return;
}
if ($notEquals) {
$queryBuilder
->andWhere($queryBuilder->expr()->notIn($wrapCase($aliasedField), $valueParameter))
->setParameter($valueParameter, $caseSensitive ? $values : array_map('strtolower', $values));
}
return;
}
$ors = [];
$parameters = [];
foreach ($values as $key => $value) {
$keyValueParameter = sprintf('%s_%s', $valueParameter, $key);
$parameters[] = [$caseSensitive ? $value : strtolower($value), $keyValueParameter];
$ors[] = match ($strategy) {
self::STRATEGY_PARTIAL => match($notEquals) {
false => $queryBuilder->expr()->like(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter, "'%'"))
),
true => $queryBuilder->expr()->notLike(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter, "'%'"))
)},
self::STRATEGY_START => match($notEquals) {
false => $queryBuilder->expr()->like(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
),
true => $queryBuilder->expr()->notLike(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
)},
self::STRATEGY_END => match($notEquals) {
false => $queryBuilder->expr()->like(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter))
),
true => $queryBuilder->expr()->notLike(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter))
)
},
self::STRATEGY_WORD_START => match($notEquals) {
false => $queryBuilder->expr()->orX(
$queryBuilder->expr()->like(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
),
$queryBuilder->expr()->like(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'% '", $keyValueParameter, "'%'"))
)
),
true => $queryBuilder->expr()->orX(
$queryBuilder->expr()->notLike(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
),
$queryBuilder->expr()->notLike(
$wrapCase($aliasedField),
$wrapCase((string) $queryBuilder->expr()->concat("'% '", $keyValueParameter, "'%'"))
)
),
},
default => throw new InvalidArgumentException(sprintf('strategy %s does not exist.', $strategy)),
};
}
$queryBuilder->andWhere($queryBuilder->expr()->orX(...$ors));
foreach ($parameters as $parameter) {
$queryBuilder->setParameter($parameter[1], $parameter[0]);
}
}
/**
* Creates a function that will wrap a Doctrine expression according to the
* specified case sensitivity.
*
* For example, "o.name" will get wrapped into "LOWER(o.name)" when $caseSensitive
* is false.
*/
protected function createWrapCase(bool $caseSensitive): \Closure
{
return static function (string $expr) use ($caseSensitive): string {
if ($caseSensitive) {
return $expr;
}
return sprintf('LOWER(%s)', $expr);
};
}
/**
* {@inheritdoc}
*/
protected function getType(string $doctrineType): string
{
return match ($doctrineType) {
Types::ARRAY => 'array',
Types::BIGINT, Types::INTEGER, Types::SMALLINT => 'int',
Types::BOOLEAN => 'bool',
Types::DATE_MUTABLE, Types::TIME_MUTABLE, Types::DATETIME_MUTABLE, Types::DATETIMETZ_MUTABLE, Types::DATE_IMMUTABLE, Types::TIME_IMMUTABLE, Types::DATETIME_IMMUTABLE, Types::DATETIMETZ_IMMUTABLE => \DateTimeInterface::class,
Types::FLOAT => 'float',
default => 'string',
};
}
}

should probably work. I ported the changes we did for our own explicit NotEquals filter which is a copy of the SearchFilter.

[divine]

Hello,

I can't entirely agree if this should be in SearchFilterInterface as in RangefilterInterface as we have this parameter names:

core/src/Doctrine/Common/Filter/RangeFilterInterface.php

Lines 22 to 29 in d85884d

	interface RangeFilterInterface
	{
	publicconstPARAMETER_BETWEEN = 'between';
	publicconstPARAMETER_GREATER_THAN = 'gt';
	publicconstPARAMETER_GREATER_THAN_OR_EQUAL = 'gte';
	publicconstPARAMETER_LESS_THAN = 'lt';
	publicconstPARAMETER_LESS_THAN_OR_EQUAL = 'lte';
	}

"ne" is better sits in Rangefilter IMO even though it doesn't look right until you see the parameter names in both files.

core/src/Doctrine/Common/Filter/SearchFilterInterface.php

Lines 22 to 48 in d85884d

	interface SearchFilterInterface
	{
	/**
	* @var string Exact matching
	*/
	publicconstSTRATEGY_EXACT = 'exact';
	/**
	* @var string The value must be contained in the field
	*/
	publicconstSTRATEGY_PARTIAL = 'partial';
	/**
	* @var string Finds fields that are starting with the value
	*/
	publicconstSTRATEGY_START = 'start';
	/**
	* @var string Finds fields that are ending with the value
	*/
	publicconstSTRATEGY_END = 'end';
	/**
	* @var string Finds fields that are starting with the word
	*/
	publicconstSTRATEGY_WORD_START = 'word_start';
	}

I'm not the person to decide so let's give a voice from @api-platform/core team.

Thanks!

[VincentLanglet]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

To me this using it this way would be a hacky undocumented feature with no BC promise.

At first I thought that the NotEqual type didn't make sens in a range filter but that could be considered as a range ]-INF, value[ U ] value, +INF[. A similar feature would be a "not between" param.

Are you still interested by this PR @divine ? If yes could you solve the conflict ?

Thanks

[soyuka]

@VincentLanglet this is part of the work around the range refactoring #4689

[VincentLanglet]

@VincentLanglet this is part of the work around the range refactoring #4689

I'm not sure to understand ; the #4689 PR is talking about UuidRangeFilter, this does not seems incompatible with the fact to add a "Not equal range" filter. Which seems a easier PR/step to finish than the big refacto of #4689.