Parser client
Via Composer
$ composer require xparse/parser$parser = new \Xparse\Parser\Parser();
$title = $parser->get('http://funivan.com')->content('//*[@class="entry-title"]/a');
print_r($title);If you are using custom Guzzle Middleware and it doesn't send real requests, in order to get last effective url you need to set it to response X-GUZZLE-EFFECTIVE-URL header manually.
Here is an example of __invoke() method in your custom Middleware
publicfunction__invoke(callable$handler) : \Closure {
returnfunction (RequestInterface$request, array$options) use ($handler) {
# some codereturn$handler($request, $options)->then(function (ResponseInterface$response) use ($request) {
$response = $response->withHeader('X-GUZZLE-EFFECTIVE-URL', $request->getUri());
# some codereturn$response;
});
};
}Recursive Parser allows you to parse website pages recursively. You need to pass link from where to start and set next page expression (xPath, css, etc).
Try to find all links to the repositories on github. Our query will be xparse.
With recursive pagination we can traverse all pagination links and process each resulting page to fetch repositories links.
useXparse\Parser\Parser;
useXparse\Parser\RecursiveParser;
# init Parser$parser = newParser();
# set expression to pagination links and initial page url$pages = newRecursiveParser(
$parser,
["//*[@class='pagination']//a/@href"],
['https://github.com/search?q=xparse']
);
$allLinks = [];
foreach($pagesas$page){
# set expression to fetch repository links$adsList = $page->value("//*[@class='repo-list-name']//a/@href")->all();
# merge and remove duplicates$allLinks = array_values(array_unique(array_merge($allLinks, $adsList)));
}
print_r($allLinks); ./vendor/bin/phpunitPlease see CONTRIBUTING for details.
The MIT License (MIT). Please see License File for more information.