Modifiy the way to compute baseline to have a better match between canvas and text layer

[calixteman]

• compute ascent as a ratio of font height
• in case the font doesn't have ascent/descent/bbox info:
  • use TextMetrics.fontBoundingBoxAscent if available
  • use a basic heuristic to guess values in drawing char on a canvas

In showing text layer, before the patch:

and after the patch:

[timvandermeij]

/botio test

[pdfjsbot]

From: Bot.io (Linux m4)

---

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.67.70.0:8877/ab05c8e5c258f66/output.txt

[pdfjsbot]

From: Bot.io (Windows)

---

Received

Command cmd_test from @timvandermeij received. Current queue size: 2

Live output at: http://3.101.106.178:8877/fcfc1981a9cbef0/output.txt

[pdfjsbot]

From: Bot.io (Linux m4)

---

Failed

Full output at http://54.67.70.0:8877/ab05c8e5c258f66/output.txt

Total script time: 26.59 mins

• Font tests: Passed
• Unit tests: FAILED
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://54.67.70.0:8877/ab05c8e5c258f66/reftest-analyzer.html#web=eq.log

[timvandermeij]

The Windows bot seems down again. Aside from that, while this seems to work nicely for the Tracemonkey file, it seems to regress quite a bit of other files unfortunately, so this needs some more work. Examples of noticeable movement are firefox-arabiccidtruetype-text-page1, firefox-issue4550-text-page2 and firefox-extgstate-text-page1.

[calixteman]

/botio test

[pdfjsbot]

From: Bot.io (Linux m4)

---

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.67.70.0:8877/cd302387c6f4e16/output.txt

[pdfjsbot]

From: Bot.io (Windows)

---

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://3.101.106.178:8877/38528dbb4f1a2b4/output.txt

[calixteman]

So I reboot the windows bot.
The new version of the patch doesn't take anymore into account the ascent/descent of the font in the pdf to position the baseline of the fallback font in the text layer but use ones from the fallback font which makes sense finally.

[pdfjsbot]

From: Bot.io (Linux m4)

---

Failed

Full output at http://54.67.70.0:8877/cd302387c6f4e16/output.txt

Total script time: 26.63 mins

• Font tests: Passed
• Unit tests: FAILED
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://54.67.70.0:8877/cd302387c6f4e16/reftest-analyzer.html#web=eq.log

[pdfjsbot]

From: Bot.io (Windows)

---

Failed

Full output at http://3.101.106.178:8877/38528dbb4f1a2b4/output.txt

Total script time: 26.75 mins

• Font tests: Passed
• Unit tests: Passed
• Integration Tests: FAILED
• Regression tests: FAILED

Image differences available at: http://3.101.106.178:8877/38528dbb4f1a2b4/reftest-analyzer.html#web=eq.log

[calixteman]

I'm not sure to understand how to interpret some results of unit tests.
For example with firefox-taro-text-page1 (i.e. TaroUTR50SortedList112.pdf ), the text in text_layer is better positioned with patch locally but it isn't the case in reftest-analyzer where it seems to be a way better in the reference.
And the same for tracemonkey.pdf: it looks ok locally but neither in the test nor in the reference.

[Snuffleupagus]

[...] the text in text_layer is better positioned with patch locally but it isn't the case in reftest-analyzer where it seems to be a way better in the reference.

In theory the output of the text tests should agree with the development viewer, when it's opened with http://localhost:8888/web/viewer.html#textLayer=visible, so if that's not actually the case this suggests a pre-existing bug in our test-suite.

The relevant test-code is found in

pdf.js/test/driver.js

Lines 27 to 107 in 6249ef5

	/**
	* @class
	*/
	varrasterizeTextLayer=(functionrasterizeTextLayerClosure(){
	varSVG_NS="http://www.w3.org/2000/svg";
	vartextLayerStylePromise=null;
	functiongetTextLayerStyle(){
	if(textLayerStylePromise){
	returntextLayerStylePromise;
	}
	textLayerStylePromise=newPromise(function(resolve){
	varxhr=newXMLHttpRequest();
	xhr.open("GET","./text_layer_test.css");
	xhr.onload=function(){
	resolve(xhr.responseText);
	};
	xhr.send(null);
	});
	returntextLayerStylePromise;
	}
	// eslint-disable-next-line no-shadow
	functionrasterizeTextLayer(
	ctx,
	viewport,
	textContent,
	enhanceTextSelection
	){
	returnnewPromise(function(resolve,reject){
	// Building SVG with size of the viewport.
	varsvg=document.createElementNS(SVG_NS,"svg:svg");
	svg.setAttribute("width",viewport.width+"px");
	svg.setAttribute("height",viewport.height+"px");
	// items are transformed to have 1px font size
	svg.setAttribute("font-size",1);
	// Adding element to host our HTML (style + text layer div).
	varforeignObject=document.createElementNS(SVG_NS,"svg:foreignObject");
	foreignObject.setAttribute("x","0");
	foreignObject.setAttribute("y","0");
	foreignObject.setAttribute("width",viewport.width+"px");
	foreignObject.setAttribute("height",viewport.height+"px");
	varstyle=document.createElement("style");
	varstylePromise=getTextLayerStyle();
	foreignObject.appendChild(style);
	vardiv=document.createElement("div");
	div.className="textLayer";
	foreignObject.appendChild(div);
	// Rendering text layer as HTML.
	vartask=pdfjsLib.renderTextLayer({
	textContent,
	container: div,
	viewport,
	enhanceTextSelection,
	});
	Promise.all([stylePromise,task.promise]).then(function(results){
	task.expandTextDivs(true);
	style.textContent=results[0];
	svg.appendChild(foreignObject);
	// We need to have UTF-8 encoded XML.
	varsvg_xml=unescape(
	encodeURIComponent(newXMLSerializer().serializeToString(svg))
	);
	varimg=newImage();
	img.src="data:image/svg+xml;base64,"+btoa(svg_xml);
	img.onload=function(){
	ctx.drawImage(img,0,0);
	resolve();
	};
	img.onerror=function(e){
	reject(newError("Error rasterizing text layer "+e));
	};
	});
	});
	}
	returnrasterizeTextLayer;
	})();

and the CSS-file that it loads is found in https://github.com/mozilla/pdf.js/blob/master/test/text_layer_test.css

---

Unfortunately we're going to need to somehow debug/fix the underlying issue first, since it's virtually impossible to review this patch if we cannot actually trust the test-suite.

[timvandermeij]

Now that the issue mentioned above should be fixed, could you rebase this and retrigger the tests to see how this goes now?

[calixteman]

/botio test

[pdfjsbot]

From: Bot.io (Windows)

---

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://3.101.106.178:8877/0aa7a86a5d538bc/output.txt

[pdfjsbot]

From: Bot.io (Linux m4)

---

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.67.70.0:8877/1bbd7ece8d4728b/output.txt

[pdfjsbot]

From: Bot.io (Linux m4)

---

Failed

Full output at http://54.67.70.0:8877/1bbd7ece8d4728b/output.txt

Total script time: 26.88 mins

• Font tests: Passed
• Unit tests: Passed
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://54.67.70.0:8877/1bbd7ece8d4728b/reftest-analyzer.html#web=eq.log

[pdfjsbot]

From: Bot.io (Windows)

---

Failed

Full output at http://3.101.106.178:8877/0aa7a86a5d538bc/output.txt

Total script time: 28.34 mins

• Font tests: Passed
• Unit tests: Passed
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://3.101.106.178:8877/0aa7a86a5d538bc/reftest-analyzer.html#web=eq.log

[Snuffleupagus]

Looking at the latest test results, it seems to be a pretty even split between test-cases that improve respectively regress with this patch; hence it's unfortunately not really clear (at least to me) that this is an overall improvement as-is.

[timvandermeij]

Yes, for example chrome-arabiccidtruetype-text-page1 is clearly improved and chrome-issue6387-text-page1 is even quite a dramatic improvement, but I also see firefox-issue8229-page1 for example which looks a bit worse. In general my feeling is that overall this is an improvement, but it would be good to have an additional look at the ones that look a bit worse here to see if it can be improved further.

[calixteman]

@timvandermeij I've the impress that overall this is an improvement too even if it's a bit worse for some of them.

[brendandahl]

/botio test
(re-running to see with chrome updates)

[pdfjsbot]

From: Bot.io (Windows)

---

Failed

Full output at http://3.101.106.178:8877/8751e8aba462300/output.txt

Total script time: 29.56 mins

• Font tests: Passed
• Unit tests: Passed
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://3.101.106.178:8877/8751e8aba462300/reftest-analyzer.html#web=eq.log

[timvandermeij]

/botio-linux test

[pdfjsbot]

From: Bot.io (Linux m4)

---

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.67.70.0:8877/c9a878dbc0da2b8/output.txt

[pdfjsbot]

From: Bot.io (Linux m4)

---

Failed

Full output at http://54.67.70.0:8877/c9a878dbc0da2b8/output.txt

Total script time: 23.85 mins

• Font tests: Passed
• Unit tests: Passed
• Integration Tests: Passed
• Regression tests: FAILED

Image differences available at: http://54.67.70.0:8877/c9a878dbc0da2b8/reftest-analyzer.html#web=eq.log

[timvandermeij]

I have looked at all reference test differences with this version of the patch and I'd say this is an overall improvement. Some files are improved quite dramatically, others have minor improvements and for a few it's a bit worse, but given that we have to rely on a heuristic here I don't think that will ever be perfect, so this will remain to be a trade-off. In this case the majority of cases is an improvements, so let's go with this and we can always tweak it further once more test cases are presented to us.

Thank you for putting effort into this!

[timvandermeij]

/botio makeref

[pdfjsbot]

From: Bot.io (Linux m4)

---

Received

Command cmd_makeref from @timvandermeij received. Current queue size: 0

Live output at: http://54.67.70.0:8877/10bb771551af041/output.txt

[pdfjsbot]

From: Bot.io (Windows)

---

Received

Command cmd_makeref from @timvandermeij received. Current queue size: 1

Live output at: http://3.101.106.178:8877/4b96eb3e2509288/output.txt

[pdfjsbot]

From: Bot.io (Linux m4)

---

Success

Full output at http://54.67.70.0:8877/10bb771551af041/output.txt

Total script time: 21.91 mins

• Lint: Passed
• Make references: Passed
• Check references: Passed

[pdfjsbot]

From: Bot.io (Windows)

---

Success

Full output at http://3.101.106.178:8877/4b96eb3e2509288/output.txt

Total script time: 27.44 mins

• Lint: Passed
• Make references: Passed
• Check references: Passed

[timvandermeij]

This closed quite a few issues, some of them mentioned above, so nice work!