Do not try do retrieve object in ItemNormalizer $context['resource_class'] is not defined

[lyrixx]

Q	A
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	-
License	MIT
Doc PR	-

---

We store some document in Elasticsearch and we hydrate them with the Serializer.
But the ItemNormalizer raise an error because the resource_class is not defined.

[teohhanhui]

I'm not sure I understand the problem here. The AbstractItemNormalizer / ItemNormalizer only supports resource classes. So $context['resource_class'] should always be set.

[teohhanhui]

It's even explicitly set here:

core/src/Serializer/AbstractItemNormalizer.php

Lines 125 to 127 in 516c9c7

	if (!isset($context['resource_class'])) {
	$context['resource_class'] = $class;
	}

[lyrixx]

I guess you can simply try to decode some data with an id param and with an empty context to reproduce.

but basically this condition evaluate to true

core/src/Serializer/ItemNormalizer.php

Line 35 in 516c9c7

if (isset($data['id']) && !isset($context[self::OBJECT_TO_POPULATE])) {

so it jumps in updateObjectToPopulate, an InvalidArgumentException exception is thrown, and finally this line breaks everything:

core/src/Serializer/ItemNormalizer.php

Line 52 in 516c9c7

foreach ($this->propertyNameCollectionFactory->create($context['resource_class'], $context) as$propertyName) {

Did I miss something?

[soyuka]

Can't you give a context? What's your use case?

[lyrixx]

@soyuka sure, it's already explained in the PR description.

[dunglas]

Normalizers must work even if no context is provided. The fix looks good to me. Would you mind adding an unit test to prevent regressions @lyrixx?

[lyrixx]

@dunglas Sure 👍 I will have to clone the projet locally then 😂

[dunglas]

Or maybe that a better fix would be to resolve this value if not provided by calling ResourceClassResolver::getResourceClass()? Something like $context['resource_class'] ?? $this->resourceClassResolver($data) in updateObjectToPopulate()?

[lyrixx]

I disagree. If it's not provided, it means we are outside the nominal api platform workflow.
We should not guess

[dunglas]

We could expect from a denormalizer that it updates relations by default, isn't it?

[lyrixx]

You lost me. Sorry. If we set $context['resource_class'], the code will still raise an exception. Here I don't want to fetch something via the iriConverter.

[dunglas]

Or you can just run composer update --prefer-source on the project you're working on :P

[lyrixx]

@dunglas I made the initial patch on github ;) But don't worry, I know how to update a patch

[teohhanhui]

As I've already mentioned, the ItemNormalizer is for use with resource classes. If you're using it with a non-resource class, that's not a supported use case.

@dunglas I agree we should set $context['resource_class'] if it's missing.

[lyrixx]

@teohhanhui Heuuuuu, I don't get it

1.crearte a new

1. install api platform
2. get some data from an API (let's say elasctisearch)
3. hydrate your object with symfony default serializer
4. 💥

Since API Platform modify the default serialization, you can not say:

If you're using it with a non-resource class, that's not a supported use case.

[lyrixx]

I have added an unit test

[teohhanhui]

Can you open a PR with a failing Behat test? Because ItemNormalizer::supportsDenormalization should return false for non-resource classes.

• get some data from an API (let's say elasctisearch)
• hydrate your object with symfony default serializer

Need more context. Is it inside an API Platform operation? Are you doing it through a data provider?

[lyrixx]

Sure, it's quite simple: lyrixx/test@085fe66

To make things crystal clear: we use the same object for the "public" API and for Elasticsearch storage

[teohhanhui]

If you declare a class as an API Resource, it's normal that our ItemNormalizer would handle it.

We store some document in Elasticsearch and we hydrate them with the Serializer.

In your case, you just want to use Symfony's default ObjectNormalizer, right?

[lyrixx]

@teohhanhui Yes. And other Normalizers that could live in my application.
I don't see what you have in mind... is it a big deal to add a safe guard here?

[teohhanhui]

is it a big deal to add a safe guard here?

Please bear with us in trying to figure out the best way to go about it.

So far I'm not convinced that it should be done this way. I'm inclined more towards @dunglas' suggestion of setting $context['resource_class'] if it's missing.

As for your specific use case:

1. If you want it to be an API Platform operation: Have you considered using a data provider? (Sorry for repeating myself.)
2. If you want it to be a custom operation: Don't use the same class. It's quite honestly nonsensical to use ItemNormalizer for denormalizing your object in your above use case.

[lyrixx]

1. We are already using a DataProvider to retrieve data from ES. We go plain array from ES. But we need to hydrate objects. That's is the blocking part

2. It's not the case

It's quite honestly nonsensical to use ItemNormalizer for denormalizing your object in your above use case.

I did not choose to use it, it's build in with API Platform.

[teohhanhui]

Okay, I think I get it now. And it should be a supported use case, yes. The question is how? 🤔

[teohhanhui]

@lyrixx A possible workaround could be to pass a custom $format, and register a denormalizer which supports that custom $format, at a higher priority than ItemNormalizer.

[lyrixx]

In my code, or in ApiPlatform ?

[teohhanhui]

In your code. 😄

[lyrixx]

For now I used a very simple workaround

$this->serializer->denormalize($logResult['_source'], Log::class, 'json', [
'object_to_populate' => $log = newLog(),
'groups' => ['elasticsearch'],
]);

[dunglas]

The patch is good enough for now, it doesn't break anything, and prevent an error that must be handled anyway. Could you just rebase against 2.3 please? (because it's a bug fix)

[teohhanhui]

@dunglas I think it's the wrong fix. As we've discussed, $context['resource_class'] should be set if it's missing.

[soyuka]

Maybe something like this?

if (!isset($context['resource_class'])) {
log(Something about this not being right);
return parent::normalize(...);
}

This patch also feels wrong to me, on the other hand it doesn't harm so I'm mitigated :p.

[dunglas]

Can we merge this until we find a better fix @api-platform/core-team? Having a hard failure is worst...

[soyuka]

If we log something because it's not normal as I suggested (#2326 (comment)) I'm 👍 to merge.

[lyrixx]

I added a log line

[teohhanhui]

Still think it's the wrong fix. The correct fix should be to set $context['resource_class'] when it's missing.

[lyrixx]

Ok, Let's try what you suggest to prove it does not work:

If I set $context['resource_class'] when it's missing, I got the following error:

Item not found for "/api/foobars/123".

Legit, so I will have to implement a DataProvider. When I will implement it, I will need to deserialize my data because it's stored as JSON (in Elasticsearch, but we don't care). And ... 💥 Infinite loop.

At some point you said:

If you're using it with a non-resource class, that's not a supported use case.

Well:

1. The item normalizer is installed by default with API platform
2. The code is not robust: You could either check if the key is defined before using it, or throw an exception if the key is missing. The actual code is based on assumptions. This is wrong too.

You also said:

If you want it to be a custom operation: Don't use the same class. It's quite honestly nonsensical to use ItemNormalizer for denormalizing your object in your above use case.

So your suggestion, in order to not merge this PR, is that I need to duplicate all my model class to get one form API Platform, and another one for ES <-> PHP, and a layer that will my map my two model ?
It sound really overkill. Do you always use this pattern with doctrine?

I'm sorry, but you are making things really difficult. You ask for things, I did them, then you refused it again. It's really not king.

Thus, I will tell you what is the real fix could be:

1/ Update Symfony with:

 interface NormalizerInterface
{
- public function supportsNormalization($data, $format = null);+ public function supportsNormalization($data, $format = null, array $context = array());
}

2/ update ItemNormalizer::supportsNormalization to return true only if $context['resource_class'] is set.

[teohhanhui]

No. I recognize that your use case is a valid one and it's unfortunate that it's not working at the moment (but you have found workarounds).

What I'm saying is that it's bad for us to have such workarounds in our code.

[dunglas]

1/ Update Symfony with:

It's already supported in Symfony: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Normalizer/ContextAwareNormalizerInterface.php#L26

2/ update ItemNormalizer::supportsNormalization to return true only if $context['resource_class'] is set.

I don't agree with this one, it's totally valid to use API Platform to serialize in JSON-LD outside of an API context (in a worker, to send data to a Kafka, an Elastic...).

Legit, so I will have to implement a DataProvider. When I will implement it, I will need to deserialize my data because it's stored as JSON (in Elasticsearch, but we don't care). And ... 💥 Infinite loop.

This specific problem can be fixed like this: https://api-platform.com/docs/core/data-providers/#injecting-the-serializer-in-an-code-classlanguage-textitemdataprovidercode

It's a hack, and the good solution would be to use a specific Serializer instance in API Platform (not the same one than the default serializer of Symfony). It would be cleaner, but it requires some more work in SF (creating a factory to build properly configured serializer services). I'll do it when I'll have some free time.

In the meantime, I in favor of merging @lyrixx patch. We should never throw an unhandled PHP error, it's not robust code...

[teohhanhui]

We should never throw an unhandled PHP error

Agree. But this patch is not doing the right thing. Setting $context['resource_class'] (if missing) would be the right thing to do, and as @lyrixx has noted, it throws a proper exception even though it does not address his use case.

[lyrixx]

About the point 3, I'm not talking about a DIC circular reference,
I'm talking about the serializer that call use the ItemNoramlizer, that will use the DataPRovider, that will use the serializer (here is the loop)

[joelwurtz]

@teohhanhui I don't understand in your solution how can someone deserialize something that must not use ApiPlatform, our process is very simple:

Fetch Data From Elastisearch -> Deserialize it (we don't want API Platform here, it's just a stupid object) -> Do Some Work -> Serialize It (with api platform)

With what your are proposing the only solution would be to use another Serializer, I would be fine with that if API Platform did not replace the one from Symfony. In our use case we have installed API Platform after doing the deserialization with Elasticsearch, and changing a lot of our code and DIC for that may be painful for some users.

IMO A viable solution would be to change the way API Platform is injected :

1. Not replacing the default serializer for Symfony
2. Injecting HAL Normalizer / JSON-LD Normalizer into the one of Symfony
3. Introduce a new context key : json_format
4. HAL Normalizer would only work if json_format is 'hal'
5. JSON-LD Normalizer would only work if json_format is 'json-ld'
6. Use the json format key in all services of API Platform which need to serialize / deserialize the data

[dunglas]

@lyrixx isn't #2399 a better fix (no logs)?

@api-platform/core-team we need to take a decision regarding this bug, it's annoying for end users. I'll merge this one or #2399 today.

[lyrixx]

@dunglas#2399 Functionally it works but this is a major waste of resource:

1. We want to unserialize an object
2. This normalizer will try to update an existing object
3. This line $context[self::OBJECT_TO_POPULATE] = $this->iriConverter->getItemFromIri((string) $data['id'], $context + ['fetch_data' => true]); will throw an exception. Exception are costly
4. The exception will be catched and this line will be executed successfully $context[self::OBJECT_TO_POPULATE] = $this->iriConverter->getItemFromIri(sprintf('%s/%s', $this->iriConverter->getIriFromResourceClass($context['resource_class']), $data[$identifier]), $context + ['fetch_data' => true]);
5. So in the end, even before the denormalization occurs, the object is already hydrated via an heavy processing (IriConverter is nice, but CPU intensive (router::match)

As I said many time, I really thing this patch is correct, and we should not try to goes through API Platform Normalizer at all cost. There are legitimate case where we don't want IRI stuff. Here is one example.

And again, I know we could create another serializer + normalizers but it's really boring to have to put many lines of services configuration in my project, where a simple if could solve everything.

---

On some "page" of our application, we retrieve 100 objects. So there will be 100 Exception + 100 Router::match :(

[dunglas]

Let's merge

[dunglas]

Thanks @lyrixx

[lyrixx]

Thanks you