Example doesn't work as expected

[asm0dey]

When I call a command from examples

sudo httpjail -vvv --js "true" -- curl https://example.com
16:33:47.852 DEBUG httpjail: Starting httpjail with args: Args { sh: None, js: Some("true"), js_file: None, request_log: None, weak: false, verbose: 3, timeout: None, no_jail_cleanup: false, cleanup: false, server: false, test: None, docker_run: false, command: ["curl", "https://example.com"] }
16:33:47.852 INFO httpjail: Using V8 JavaScript rule evaluation
16:33:47.855 DEBUG httpjail::tls: Loading cached CA certificate from "/root/.config/httpjail/ca-cert.pem"
16:33:47.855 INFO httpjail::tls: Loaded cached CA certificate from /root/.config/httpjail/ca-cert.pem
16:33:47.855 INFO httpjail::tls: Certificate manager initialized
16:33:47.855 DEBUG httpjail::proxy: Created HTTPS client config with 145 trusted roots (including httpjail CA)
16:33:47.855 INFO httpjail::proxy: HTTPS connector initialized with webpki roots and httpjail CA
16:33:47.855 DEBUG httpjail::proxy: Successfully bound to port 8574
16:33:47.855 INFO httpjail::proxy: Starting HTTP proxy on port 8574
16:33:47.855 DEBUG httpjail::proxy: Successfully bound to port 8258
16:33:47.855 INFO httpjail::proxy: Starting HTTPS proxy on port 8258
16:33:47.855 DEBUG httpjail::jail::managed: Starting orphan cleanup scan in "/root/.local/share/httpjail/canaries"
16:33:47.855 DEBUG httpjail::jail::managed: Created canary file for jail 'wekc4una'
16:33:47.855 INFO httpjail::jail::linux: Setting up DNS for namespace httpjail_wekc4una with custom resolv.conf
16:33:47.855 DEBUG httpjail::jail::linux::resources: Created namespace config directory: /etc/netns/httpjail_wekc4una
16:33:47.855 INFO httpjail::jail::linux: Created namespace-specific resolv.conf at /etc/netns/httpjail_wekc4una/resolv.conf with Google DNS servers
16:33:47.857 INFO httpjail::jail::linux::resources: Created network namespace: httpjail_wekc4una
16:33:47.857 INFO httpjail::jail::linux: Created network namespace: httpjail_wekc4una
16:33:47.860 DEBUG httpjail::jail::linux::resources: Created veth pair: vh_wekc4una <-> vn_wekc4una
16:33:47.861 DEBUG httpjail::jail::linux: Created veth pair: vh_wekc4una <-> vn_wekc4una
16:33:47.879 DEBUG httpjail::jail::linux: Configured host side networking for vh_wekc4una
16:33:47.886 INFO httpjail::jail::linux: DNS already configured in namespace httpjail_wekc4una
16:33:47.895 INFO httpjail::jail::linux: Routes in namespace httpjail_wekc4una after configuration:
default via 10.99.183.193 dev vn_wekc4una
10.99.183.192/30 dev vn_wekc4una proto kernel scope link src 10.99.183.194

16:33:47.895 DEBUG httpjail::jail::linux: Configured networking inside namespace httpjail_wekc4una
16:33:47.895 DEBUG httpjail::jail::linux::nftables: Creating nftables table: httpjail_wekc4una
16:33:47.899 INFO httpjail::jail::linux::nftables: Created nftables table httpjail_wekc4una with NAT rules for subnet 10.99.183.192/30
16:33:47.899 INFO httpjail::jail::linux: Set up NAT rules for namespace httpjail_wekc4una with subnet 10.99.183.192/30
16:33:47.899 DEBUG httpjail::jail::linux::nftables: Creating nftables table in namespace httpjail_wekc4una: httpjail
16:33:47.902 INFO httpjail::jail::linux::nftables: Created nftables rules in namespace httpjail_wekc4una for HTTP:8574 HTTPS:8258
16:33:47.902 INFO httpjail::jail::linux: Linux jail setup complete using namespace httpjail_wekc4una with HTTP proxy on port 8574 and HTTPS proxy on port 8258
16:33:47.902 DEBUG httpjail::jail::managed: Created canary file for jail 'wekc4una'
16:33:47.902 INFO httpjail::jail::managed: Started lifecycle heartbeat for jail 'wekc4una'
16:33:47.902 DEBUG httpjail: Setting 6 CA certificate environment variables
16:33:47.902 DEBUG httpjail::jail::linux: Executing command in namespace httpjail_wekc4una: ["curl", "https://example.com"]
16:33:47.902 DEBUG httpjail::jail::managed: Starting heartbeat thread for "/root/.local/share/httpjail/canaries/wekc4una"
16:33:47.902 DEBUG httpjail::jail::linux: Will drop to user 'finkel' (from SUDO_USER) after entering namespace

it seems like nothing is happening and the execution never ends

[ammario]

Thx for sharing. Does the weak jail (-w) work as expected?

I just committed f5fb78f which refactors the code right after your last debug log where I'm guessing it's hanging. If it's hanging there we should see a debug log with a full print out of the command it's running at the very least. Please clone main, run cargo install --path ., and share the new output if it continues to hang.

[asm0dey]

I'm not sure if I have weak jails on linux :(
BTW, after long hang curl just times out
Gimme a minute, I'll try to build a version locally

[asm0dey]

No, still doesn't work:

❯ sudo ./target/debug/httpjail -vvv --js "true" -- curl https://google.com
20:44:36.005 DEBUG httpjail: httpjail version: 0.1.7 (f5fb78f)
20:44:36.005 DEBUG httpjail: Starting httpjail with args: Args { sh: None, js: Some("true"), js_file: None, request_log: None, weak: false, verbose: 3, timeout: None, no_jail_cleanup: false, cleanup: false, server: false, test: None, docker_run: false, command: ["curl", "https://google.com"] }
20:44:36.005 INFO httpjail: Using V8 JavaScript rule evaluation
20:44:36.008 DEBUG httpjail::tls: Loading cached CA certificate from "/root/.config/httpjail/ca-cert.pem"
20:44:36.008 INFO httpjail::tls: Loaded cached CA certificate from /root/.config/httpjail/ca-cert.pem
20:44:36.008 INFO httpjail::tls: Certificate manager initialized
20:44:36.008 DEBUG httpjail::proxy: Created HTTPS client config with 145 trusted roots (including httpjail CA)
20:44:36.008 INFO httpjail::proxy: HTTPS connector initialized with webpki roots and httpjail CA
20:44:36.008 DEBUG httpjail::proxy: Successfully bound to port 8080
20:44:36.008 INFO httpjail::proxy: Starting HTTP proxy on port 8080
20:44:36.008 DEBUG httpjail::proxy: Successfully bound to port 8436
20:44:36.008 INFO httpjail::proxy: Starting HTTPS proxy on port 8436
20:44:36.009 DEBUG httpjail::jail::managed: Starting orphan cleanup scan in "/root/.local/share/httpjail/canaries"
20:44:36.009 DEBUG httpjail::jail::managed: Created canary file for jail 'akisb8iw'
20:44:36.009 INFO httpjail::jail::linux: Setting up DNS for namespace httpjail_akisb8iw with custom resolv.conf
20:44:36.009 DEBUG httpjail::jail::linux::resources: Created namespace config directory: /etc/netns/httpjail_akisb8iw
20:44:36.009 INFO httpjail::jail::linux: Created namespace-specific resolv.conf at /etc/netns/httpjail_akisb8iw/resolv.conf with Google DNS servers
20:44:36.010 INFO httpjail::jail::linux::resources: Created network namespace: httpjail_akisb8iw
20:44:36.010 INFO httpjail::jail::linux: Created network namespace: httpjail_akisb8iw
20:44:36.012 DEBUG httpjail::jail::linux::resources: Created veth pair: vh_akisb8iw <-> vn_akisb8iw
20:44:36.012 DEBUG httpjail::jail::linux: Created veth pair: vh_akisb8iw <-> vn_akisb8iw
20:44:36.035 DEBUG httpjail::jail::linux: Configured host side networking for vh_akisb8iw
20:44:36.036 INFO httpjail::jail::linux: DNS already configured in namespace httpjail_akisb8iw
20:44:36.045 INFO httpjail::jail::linux: Routes in namespace httpjail_akisb8iw after configuration:
default via 10.99.128.89 dev vn_akisb8iw
10.99.128.88/30 dev vn_akisb8iw proto kernel scope link src 10.99.128.90

20:44:36.045 DEBUG httpjail::jail::linux: Configured networking inside namespace httpjail_akisb8iw
20:44:36.045 DEBUG httpjail::jail::linux::nftables: Creating nftables table: httpjail_akisb8iw
20:44:36.049 INFO httpjail::jail::linux::nftables: Created nftables table httpjail_akisb8iw with NAT rules for subnet 10.99.128.88/30
20:44:36.049 INFO httpjail::jail::linux: Set up NAT rules for namespace httpjail_akisb8iw with subnet 10.99.128.88/30
20:44:36.049 DEBUG httpjail::jail::linux::nftables: Creating nftables table in namespace httpjail_akisb8iw: httpjail
20:44:36.053 INFO httpjail::jail::linux::nftables: Created nftables rules in namespace httpjail_akisb8iw for HTTP:8080 HTTPS:8436
20:44:36.053 INFO httpjail::jail::linux: Linux jail setup complete using namespace httpjail_akisb8iw with HTTP proxy on port 8080 and HTTPS proxy on port 8436
20:44:36.053 DEBUG httpjail::jail::managed: Created canary file for jail 'akisb8iw'
20:44:36.053 INFO httpjail::jail::managed: Started lifecycle heartbeat for jail 'akisb8iw'
20:44:36.053 DEBUG httpjail: Setting 6 CA certificate environment variables
20:44:36.053 DEBUG httpjail: Setting glibc resolver timeouts via RES_OPTIONS=timeout:2 attempts:1
20:44:36.053 DEBUG httpjail::jail::managed: Starting heartbeat thread for "/root/.local/share/httpjail/canaries/akisb8iw"
20:44:36.053 DEBUG httpjail::jail::linux: Executing command in namespace httpjail_akisb8iw: ["curl", "https://google.com"]
20:44:36.053 DEBUG httpjail::jail::linux: Will drop privileges to uid=1000 gid=1000 after entering namespace
20:44:36.053 DEBUG httpjail::jail::linux: Executing command: CURL_CA_BUNDLE="/root/.config/httpjail/ca-cert.pem" GIT_SSL_CAINFO="/root/.config/httpjail/ca-cert.pem" NODE_EXTRA_CA_CERTS="/root/.config/httpjail/ca-cert.pem" REQUESTS_CA_BUNDLE="/root/.config/httpjail/ca-cert.pem" RES_OPTIONS="timeout:2 attempts:1" SSL_CERT_DIR="/root/.config/httpjail" SSL_CERT_FILE="/root/.config/httpjail/ca-cert.pem" SUDO_GID="1000" SUDO_UID="1000" SUDO_USER="finkel" "ip" "netns" "exec" "httpjail_akisb8iw" "setpriv" "--reuid=1000" "--regid=1000" "--init-groups" "--" "curl" "https://google.com"
curl: (28) Failed to connect to google.com port 443 after 132725 ms: Could not connect to server
20:46:48.835 INFO httpjail::jail::linux: Triggering jail cleanup for akisb8iw
20:46:48.836 DEBUG httpjail::jail::linux: Cleaning up orphaned Linux jail: akisb8iw
20:46:48.857 DEBUG httpjail::jail::linux::nftables: Removed nftables table: httpjail_akisb8iw
20:46:48.857 DEBUG httpjail::jail::linux::resources: Removed namespace config directory: /etc/netns/httpjail_akisb8iw
20:46:48.885 DEBUG httpjail::jail::linux::resources: Deleted network namespace: httpjail_akisb8iw
20:46:48.885 DEBUG httpjail::jail::managed: Deleted canary file for jail 'akisb8iw'

[asm0dey]

Ah, I see, yes, --weak works as expected

[ammario]

Gotcha.

My best guess is there are some competing nftable entries breaking / overriding our redirect. I'm going to noodle on how to make the jail setup more robust to such interference. It has been challenging to make the strong isolation work consistently across all Linux systems.

In the mean time I'm curious if the weak jail is sufficient for your use case?

[asm0dey]

Well, TBH, I was hoping to dump the whole bodies of requests/responses, I'm not sure if the strong mode can do it, the weak one definitely doesn't

[asm0dey]

If you need any more debugging from my side - let me know

[ammario]

There's a "--request-log" that gives some information about the headers. Unfortunately there's no body interception in either mode at the moment.

To do body interception right the rule engine would have to support some kind of streaming interception, otherwise large downloads would be slow and long-running requests like WebSockets or SSE would hang for applications.

It would be quite the undertaking so I'm not planning to investigate it for a while.

[asm0dey]

I mean, in these obscure scenarios when I actually need to dump the whole traffic of an app - I'm prepared to pay the price. I would also be happy to have a noop filter, that will just allow everything (kinda --js "true", but even more noop) if it allows me to dump the whole traffic faster :)

I would expect it to have something like https://www.charlesproxy.com/ or https://www.mitmproxy.org/ under the hood.