Is there any way to test a threshold?

[corysimmons]

For instance, is there a way to test for a suitable level of difference?

My use case is I have a grid system in Sass and Stylus. They produce very very slightly different widths. As long as it's generally around the same width, I'd like tests to pass.

This lib seems like it'd fail unless the widths are the exact same.

[twolfson]

Currently, it is a strict comparison to 0. However, we should be able to adjust the threshold up/down based on an option:

https://github.com/uber/image-diff/blob/1.0.1/lib/image-diff.js#L85

imageDiff({
  actualImage: 'checkerboard.png',
  expectedImage: 'white.png',
  diffImage: 'difference.png',
  threshold: 20  // percentage of 100
}, ...

Would that work for your case?

[corysimmons]

@twolfson Absolutely! This would be perfect and I'd implement it in testing suites across a couple of my libraries immediately! 💃

[twolfson]

Cool. I am pretty busy with some other things at the moment but I will take a look by the end of next weekend. Feel free to open a PR to beat me to the punch =)

[corysimmons]

https://github.com/uber/image-diff/blob/1.0.1/lib/image-diff.js#L72 40131.8 is the end of the range? So I'd just need to write something that converted 40131.8 to 100?

[twolfson]

Nah, 40131.8 was an example. Although, I could see how it could be misconstrued. I believe the second value is the percentage. Another option is to use whatever absolute difference there is (first value). To see the limits, probably comparing a black square to a white square would work.

[corysimmons]

I think this is probably out of my JS skill range and I'm in no big rush so I'm happy to wait on your expertise to implement this the correct way. =)

^{Pretty professional way of saying, "I R DUM."}

[twolfson]

Not a problem. My ETA still stands. Fwiw, the approach would be something like change the regexp grouping to evaluate the perecentage, parse it to a float, and perform a comparison. The partially tedious part would be writing a test.

https://github.com/uber/image-diff/blob/1.0.1/lib/image-diff.js#L76-L85

// Parse the percentage difference
var percentDifference = parseFloat(resultInfo[2], 10);
if (isNaN(percentDifference)) {
  return cb(new Error('Attempted to parse percentage difference from `image-diff\'s stderr` but got `NaN` from "' + resultInfo[2] + '"'));
}

// Callback with pass/fail
return cb(null, percentageDifference <= thresholdDifference);

[corysimmons]

Well, I'll wait until it's implemented then I'll tackle the issue of coming up with a nice standard way to do visual tests with it.

Really appreciate your work on this.

[nikchosh]

@twolfson I am new to git and cannot push but here is the diff output of what I have done which works perfectly by providing a threshold and also returns the path+name of the diff image (so, that the user may delete if blank)
@corysimmons this should solve your issue.

diff --git a/lib/image-diff.js b/lib/image-diff.js
index 1c6441e..b0c0f7e 100644
--- a/lib/image-diff.js
+++ b/lib/image-diff.js
@@ -73,7 +73,7 @@ ImageDiff.createDiff = function (options, cb) {
// DEV: According to http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=17284
// DEV: These values are the total square root mean square (RMSE) pixel difference across all pixels and its percentage
// TODO: This is not very multi-lengual =(

• var resultInfo = stderr.match(/all: (\d+.?\d_) ((\d+.?\d_))/);
• var resultInfo = stderr.match(/all: (\d+.?\d_) ((._?))/);

// If there was no resultInfo, throw a fit
if (!resultInfo) {
@@ -81,8 +81,8 @@ ImageDiff.createDiff = function (options, cb) {
}

// Callback with pass/fail

• var totalDifference = resultInfo[1];
• return cb(null, totalDifference === '0');
• var totalDifference = resultInfo[2]; // this is the percentage difference
• return cb(null, parseFloat(totalDifference) <= options.threshold, options.diffPath);
  });
  };
  ImageDiff.prototype = {
  @@ -91,6 +91,7 @@ ImageDiff.prototype = {
  var actualPath = options.actualImage;
  var expectedPath = options.expectedImage;
  var diffPath = options.diffImage;
• var threshold = options.threshold;

// Assert our options are passed in
if (!actualPath) {
@@ -187,9 +188,11 @@ ImageDiff.prototype = {
ImageDiff.createDiff({
actualPath: actualTmpPath,
expectedPath: expectedTmpPath,

•      diffPath: diffPath
  

•    }, function saveResult (err, _imagesAreSame) {
  

•      diffPath: diffPath,
  

•      threshold: threshold
  

•    }, function saveResult (err, _imagesAreSame, _diffPath) {
     imagesAreSame = _imagesAreSame;
  

•      diffPath = _diffPath;
     cb(err);
   });
  

  }
  @@ -202,7 +205,7 @@ ImageDiff.prototype = {
  fs.unlink(filepath, cb);
  }, function callOriginalCallback (_err) {
  // Callback with the imagesAreSame

•    callback(err, imagesAreSame);
  

•    callback(err, imagesAreSame, diffPath);
  

  });
  });
  }

[nikchosh]

Ok, the diff does not appear properly with "plus" and "minus" signs - I have to see how to properly branch and submit

[corysimmons]

@nikchosh

1. Fork the repo to your own account
2. Clone that repo to your machine git clone
3. Make changes and create commits https://github.com/erlang/otp/wiki/Writing-good-commit-messages (I use a tool called GitX to do this http://rowanj.github.io/gitx/)
4. git push those commits to your fork
5. Open a Pull Request between the original repo and your fork

If you have problems, can you just https://gist.github.com your updated code?

[nikchosh]

Ok done, thanks.

Nick

On Mon, Mar 30, 2015 at 8:48 AM, Cory Simmons notifications@github.com
wrote:

@nikchosh https://github.com/nikchosh

1. Fork the repo to your own account
2. Clone that repo to your machine git clone
3. Make changes and create commits
   https://github.com/erlang/otp/wiki/Writing-good-commit-messages (I use
   a tool called GitX to do this http://rowanj.github.io/gitx/)
4. git push those commits to your fork
5. Open a Pull Request between the original repo and your fork

If you have problems, can you just https://gist.github.com your updated
code?

—
Reply to this email directly or view it on GitHub
#14 (comment).

[twolfson]

More GitHub related info can be found here:

https://help.github.com/

[bevacqua]

@corysimmons Just FYI, use it today with image-diff-2 on npm. https://github.com/bevacqua/image-diff-2. The merge was taking way too long for my needs.

[corysimmons]

@bevacqua Nice! Thanks for pinging me. =)