security: replace vulnerable regex with parser

[davisjam]

Problem: link regex was vulnerable
Solution: dedicated parser

Fixes: #1222

[davisjam]

If this is merged, a new CI on #1220 should come back clean.

[UziTech]

I feel like this complicates the code a lot. Is there no way we can do this with a constructed regex?

[davisjam]

Is there no way we can do this with a constructed regex?

I spent about 30 minutes tinkering with regex variations but couldn't find one. The difficulty is that the "destination" can contain characters overlapping with the "title".

The relatively straightforward regex I replaced captured the specification (reasonably) appropriately, though it actually cheated in a few aspects (e.g. that a title can be opened by ' and closed by "). However, because the regex engine is backtracking-based, untrusted input cannot be evaluated safely by such a regex.

[styfle]

destination is assigned but never used. Is this intentional?

[davisjam]

Nope. Will fix.

[joshbruce]

Carets?? Carat and carrot are different. :)

Angle brackets might be most appropriate if I'm reading the regex correctly.

[davisjam]

Haha I was wondering if I was spelling that right. I'll switch to AngleBrackets anyway.

[joshbruce]

Might be worth adding some doc blocks to introduce the why behind some of this...nothing too major, just to help those new to the code.

[davisjam]

@styfle@joshbrucefbf93a8 addresses your comments.

[styfle]

The two if blocks are nearly identical. Can you make a common function for those? Something like this:

functiongetMatch(r,fullMatch){varm=r.exec(fullMatch[2]);if(m){vardest=unwrapAngleBrackets(m[1].trim());vartitle=m[2];return[fullMatch[0],fullMatch[1],dest,title];}}

[joshbruce]

Agree, but not a deal breaker for my review.

[davisjam]

Will do.

[davisjam]

@styfle See ba2fc13

[styfle]

Can you flip this so that the if statement is smaller:

if (!match) {
return null;
}
// ... get dest and title here
return [dest, title];

[styfle]

Can you flip this so that the if statement is smaller:

if (!fullMatch) {
return null;
}
// ... split and such here
return [fullMatch[0], text, destinationAndTitle[0], destinationAndTitle[1]];

[styfle]

You don't need to assign dest and title here. Simply use match below.

[davisjam]

@styfle Addressed your comments in 47f4388, thank you.

[styfle]

LGTM 👍

[styfle]

@UziTech Can you take a look at the code now? Any other reservations?

[UziTech]

The reason marked is so fast is because of the speed of regexes. I just don't want this type of thing to become the norm. There is no reason to have another slow markdown parser.

[styfle]

@UziTech Good point.

Why is CI not running on this PR?

[UziTech]

Not sure. That seems to be happening a lot lately. Maybe a travis bug?

[joshbruce]

@UziTech: Good point. I think parsers like this should be done as a last resort unless we can demonstrate through benchmarking that there isn't a big performance difference...I don't think we should get in the position of asserting something is more performant in all cases.

The CI thing is weird. Is it possible that the key thing is causing issues?

[styfle]

Looks like this failed lint: https://travis-ci.org/markedjs/marked/builds/367190715

@davisjam Can you try running lint and tests on your machine and fix any issues?

[davisjam]

Lint passes now, that CI was on a previous version.