Currently supports CDP r1429850.
Python CDP Generator (shortened to PyCDP) is a library that provides Python wrappers for the types, commands, and events specified in the Chrome DevTools Protocol.
The Chrome DevTools Protocol provides for remote control of a web browser by sending JSON messages over a WebSocket. That JSON format is described by a machine-readable specification. This specification is used to automatically generate the classes and methods found in this library.
You can install this library as a dependency on your project with:
pip install git+https://github.com/HMaker/python-cdp.git@latest
Change the git tag @latest if you need another version. To install for development, clone this
repository, install Poetry package manager and run poetry install to install dependencies.
If all you want is automate Chrome right now, PyCDP includes a low-level client for asyncio and twisted:
importasynciofrompycdpimportcdpfrompycdp.browserimportChromeLauncherfrompycdp.asyncioimportconnect_cdpasyncdefmain():
chrome=ChromeLauncher(
binary='/usr/bin/google-chrome', # linux pathargs=['--remote-debugging-port=9222', '--incognito']
)
# ChromeLauncher.launch() is blocking, run it on a background threadawaitasyncio.get_running_loop().run_in_executor(None, chrome.launch)
conn=awaitconnect_cdp('http://localhost:9222')
target_id=awaitconn.execute(cdp.target.create_target('about:blank'))
target_session=awaitconn.connect_session(target_id)
awaittarget_session.execute(cdp.page.enable())
# you may use "async for target_session.listen()" to listen multiple events, here we listen just a single event.withtarget_session.safe_wait_for(cdp.page.DomContentEventFired) asnavigation:
awaittarget_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))
awaitnavigationdom=awaittarget_session.execute(cdp.dom.get_document())
node=awaittarget_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))
js_node=awaittarget_session.execute(cdp.dom.resolve_node(node))
print((awaittarget_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)
awaittarget_session.execute(cdp.page.close())
awaitconn.close()
awaitasyncio.get_running_loop().run_in_executor(None, chrome.kill)
asyncio.run(main())the twisted client requires twisted and autobahn packages:
fromtwisted.python.logimporterrfromtwisted.internetimportreactor, defer, threadsfrompycdpimportcdpfrompycdp.browserimportChromeLauncherfrompycdp.twistedimportconnect_cdpasyncdefmain():
chrome=ChromeLauncher(
binary='C:\Program Files\Google\Chrome\Application\chrome.exe', # windows pathargs=['--remote-debugging-port=9222', '--incognito']
)
awaitthreads.deferToThread(chrome.launch)
conn=awaitconnect_cdp('http://localhost:9222', reactor)
target_id=awaitconn.execute(cdp.target.create_target('about:blank'))
target_session=awaitconn.connect_session(target_id)
awaittarget_session.execute(cdp.page.enable())
awaittarget_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))
asyncwithtarget_session.wait_for(cdp.page.DomContentEventFired):
dom=awaittarget_session.execute(cdp.dom.get_document())
node=awaittarget_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))
js_node=awaittarget_session.execute(cdp.dom.resolve_node(node))
print((awaittarget_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)
awaittarget_session.execute(cdp.page.close())
awaitconn.close()
awaitthreads.deferToThread(chrome.kill)
defmain_error(failure):
err(failure)
reactor.stop()
d=defer.ensureDeferred(main())
d.addErrback(main_error)
d.addCallback(lambda*args: reactor.stop())
reactor.run()You also can use just the built-in CDP type wrappers with import pycdp.cdp on your own client implementation. If you want to try a different CDP version you can build new type wrappers with cdpgen command:
usage: cdpgen <arguments>
Generate Python types for the Chrome Devtools Protocol (CDP) specification.
optional arguments:
-h, --help show this help message and exit
--browser-protocol BROWSER_PROTOCOL
JSON file for the browser protocol
--js-protocol JS_PROTOCOL
JSON file for the javascript protocol
--output OUTPUT output path for the generated Python modules
JSON files for the CDP spec can be found at https://github.com/ChromeDevTools/devtools-protocol/tree/master/json
Example:
cdpgen --browser-protocol browser_protocol.json --js-protocol js_protocol.json --output /tmp/cdpYou can then include the /tmp/cdp package in your project and import it like the builtin CDP types.
The update-cdp.sh script generates the builtin CDP wrappers, the pycdp.cdp package, by automatically fetching CDP protocol specifications from the ChromeDevTools repostitory.
To generate types for the latest version:
./update-cdp.shTo generate types for a specific version, you must provide full commit hash:
./update-cdp.sh 4dd6c67776f43f75bc9b19f09618c151621c6ed9P.S. Don't forget to make it executable by running chmod +x update-cdp.sh
The pycdp.cdp package follows same structure of CDP domains, each domain is a Python module and each command a function in that module.
Each function is a generator with a single yield which is a Python dict, on the CDP wire format, containing the message that should be sent to the browser, on resumption the generator receives the message from browser:
importcdp# Get all CDP targetscommand=cdp.target.get_targets() # this is a generatorraw_cdp_request=next(command) # receive the yieldraw_cdp_response=send_cdp_request(raw_cdp_request) # you implement send_cdp_request, raw_cdp_request is the JSON object that should be sent to browsertry:
command.send(raw_cdp_response) # send the response to the generator where raw_cdp_response is the JSON object received from browser, it will raise StopIterationraiseRuntimeError("the generator didnt exit!") # this shouldnt happenexceptStopIterationasresult:
response=result.value# the parsed response to Target.get_targets() commandprint(response)For implementation details check out the docs.
PyCDP is licensed under the MIT License.