Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 12
Release 0.4.1: integrations hub + live framework tests#49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Changes from all commits
File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,79 @@ | ||||||||||||||||||||||||||
| name: Integrations (live) | ||||||||||||||||||||||||||
| # Live agent-framework coverage. Each matrix leg installs ONE framework extra in an | ||||||||||||||||||||||||||
| # isolated env and runs that framework's live test, so a single SDK's dependency tree | ||||||||||||||||||||||||||
| # can't mask or break another. Kept off the default PR gate (the core CI matrix no | ||||||||||||||||||||||||||
| # longer installs agent frameworks — see `all` extra in pyproject.toml); runs on PRs | ||||||||||||||||||||||||||
| # that touch the integration surface, nightly, and on demand. | ||||||||||||||||||||||||||
| on: | ||||||||||||||||||||||||||
| pull_request: | ||||||||||||||||||||||||||
| branches: [dev] | ||||||||||||||||||||||||||
| paths: | ||||||||||||||||||||||||||
| - "sdk/src/unplug/integrations/**" | ||||||||||||||||||||||||||
| - "sdk/tests/optional/live/**" | ||||||||||||||||||||||||||
| - "sdk/pyproject.toml" | ||||||||||||||||||||||||||
| - ".github/workflows/integrations-live.yml" | ||||||||||||||||||||||||||
| schedule: | ||||||||||||||||||||||||||
| - cron: "0 6 * * *" | ||||||||||||||||||||||||||
| workflow_dispatch: | ||||||||||||||||||||||||||
| permissions: | ||||||||||||||||||||||||||
| contents: read | ||||||||||||||||||||||||||
| jobs: | ||||||||||||||||||||||||||
| live: | ||||||||||||||||||||||||||
| runs-on: ubuntu-latest | ||||||||||||||||||||||||||
| strategy: | ||||||||||||||||||||||||||
| fail-fast: false | ||||||||||||||||||||||||||
| matrix: | ||||||||||||||||||||||||||
| include: | ||||||||||||||||||||||||||
| - extra: langgraph | ||||||||||||||||||||||||||
| test: tests/optional/live/test_langgraph_live.py | ||||||||||||||||||||||||||
| - extra: agno | ||||||||||||||||||||||||||
| test: tests/optional/live/test_agno_live.py | ||||||||||||||||||||||||||
| - extra: crewai | ||||||||||||||||||||||||||
| test: tests/optional/live/test_crewai_live.py | ||||||||||||||||||||||||||
| - extra: autogen | ||||||||||||||||||||||||||
| test: tests/optional/live/test_autogen_live.py | ||||||||||||||||||||||||||
| - extra: llama-index | ||||||||||||||||||||||||||
| test: tests/optional/live/test_llama_index_live.py | ||||||||||||||||||||||||||
| - extra: pydantic-ai | ||||||||||||||||||||||||||
| test: tests/optional/live/test_pydantic_ai_live.py | ||||||||||||||||||||||||||
| - extra: semantic-kernel | ||||||||||||||||||||||||||
| test: tests/optional/live/test_semantic_kernel_live.py | ||||||||||||||||||||||||||
| name: live (${{ matrix.extra }}) | ||||||||||||||||||||||||||
| defaults: | ||||||||||||||||||||||||||
| run: | ||||||||||||||||||||||||||
| working-directory: sdk | ||||||||||||||||||||||||||
| steps: | ||||||||||||||||||||||||||
| - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 | ||||||||||||||||||||||||||
| - uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0 | ||||||||||||||||||||||||||
| with: | ||||||||||||||||||||||||||
| enable-cache: true | ||||||||||||||||||||||||||
| version: "0.6.14" | ||||||||||||||||||||||||||
| - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 | ||||||||||||||||||||||||||
| with: | ||||||||||||||||||||||||||
| python-version: "3.12" | ||||||||||||||||||||||||||
| - name: Install (one framework extra in isolation) | ||||||||||||||||||||||||||
| run: uv sync --extra dev --extra ${{ matrix.extra }} | ||||||||||||||||||||||||||
| - name: Live integration test | ||||||||||||||||||||||||||
| run: | | ||||||||||||||||||||||||||
| set +e | ||||||||||||||||||||||||||
| uv run pytest -q -m requires_integrations ${{ matrix.test }} | ||||||||||||||||||||||||||
| rc=$? | ||||||||||||||||||||||||||
| set -e | ||||||||||||||||||||||||||
| # Exit 5 = pytest collected nothing because the module self-skipped: | ||||||||||||||||||||||||||
| # the framework installed but is unimportable under our pinned deps | ||||||||||||||||||||||||||
| # (e.g. an upstream release incompatible with our Pydantic v2). That is | ||||||||||||||||||||||||||
| # an upstream conflict, not an Unplug regression — surface it loudly but | ||||||||||||||||||||||||||
| # don't block. Real test failures (exit 1) still fail the job. | ||||||||||||||||||||||||||
| if [ "$rc" -eq 5 ]; then | ||||||||||||||||||||||||||
| echo "::warning title=Live integration skipped::${{ matrix.extra }} installed but its live tests were skipped (framework unimportable under current deps). Tolerated; not blocking." | ||||||||||||||||||||||||||
| exit 0 | ||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||
Comment on lines
+75
to
+78
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pytest exit code 5 also happens when the matrix path is stale, the test file is missing, or no tests are collected. This branch turns those cases into a successful job, so a live integration leg can become a no-op without blocking CI.
Suggested change
| ||||||||||||||||||||||||||
| exit $rc | ||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| # Agent framework integrations | ||
| Unplug ships **framework-agnostic hooks first**. You do not need LangGraph, CrewAI, or any agent SDK to use the core Guard — install only the extra for the stack you run. | ||
| ## Quick start | ||
| ```bash | ||
| pip install unplug-ai # core Guard (regex scanners, no agent deps) | ||
| pip install "unplug-ai[langgraph]" # + LangGraph when you wire graph nodes | ||
| pip install "unplug-ai[integrations]" # all documented agent/RAG framework extras | ||
| pip install "unplug-ai[all]" # capability extras (ML, presidio, yara, scrape…) | ||
| pip install "unplug-ai[all,integrations]" # everything: capabilities + every framework | ||
| ``` | ||
| Every integration module lives under `unplug.integrations.*` and uses the same five hook points: | ||
| | Hook | When to call | Threats caught | | ||
| |------|----------------|----------------| | ||
| | `scan_user_input` | Before the LLM turn | Direct prompt injection, jailbreaks | | ||
| | `wrap_retrieved_content` | After RAG / web fetch | Indirect injection in documents | | ||
| | `before_tool_call` | Before every tool | Destructive shell/SQL, exfil, financial | | ||
| | `scan_agent_output` | Before returning to user | Leaked secrets, harmful content | | ||
| | `scan_request_isolated` | Eval / probes | Same scans without session taint bleed | | ||
| ```python | ||
| from unplug import Guard | ||
| from unplug.integrations.hooks import AgentHooks | ||
| hooks = AgentHooks(Guard()) # or Guard(mode="server") for hosted API | ||
| decision = hooks.scan_user_input(user_message) | ||
| if not decision.allowed: | ||
| raise RuntimeError(decision.message) | ||
| ``` | ||
| ## Supported frameworks | ||
| | Framework | Extra | Guide | Code module | | ||
| |-----------|-------|-------|-------------| | ||
| | **Custom loop** | *(none)* | [custom-loop](custom-loop/README.md) | `hooks.py` | | ||
| | **LangGraph** | `langgraph` | [langgraph](langgraph/README.md) | `langgraph.py` | | ||
| | **Agno** | `agno` | [agno](agno/README.md) | `agno.py` | | ||
| | **Haystack** | `haystack` | [haystack](haystack/README.md) | `haystack.py` | | ||
| | **LlamaIndex** | `llama-index` | [llama-index](llama-index/README.md) | `llama_index.py` | | ||
| | **CrewAI** | `crewai` | [crewai](crewai/README.md) | `crewai.py` | | ||
| | **AutoGen** | `autogen` | [autogen](autogen/README.md) | `autogen.py` | | ||
| | **Pydantic AI** | `pydantic-ai` | [pydantic-ai](pydantic-ai/README.md) | `pydantic_ai.py` | | ||
| | **Semantic Kernel** | `semantic-kernel` | [semantic-kernel](semantic-kernel/README.md) | `semantic_kernel.py` | | ||
| | **MCP clients** | `mcp` | [mcp](mcp/README.md) | [unplug-mcp](https://github.com/UnplugAI/unplug-mcp) | | ||
| Demos (no framework install required for LangGraph / Agno patterns): | ||
| ```bash | ||
| cd sdk | ||
| python examples/langgraph_hooks_demo.py | ||
| python examples/agno_hooks_demo.py | ||
| ``` | ||
| ## Deployment modes | ||
| | Mode | Guard init | Tool enforcement | | ||
| |------|------------|------------------| | ||
| | Embedded (local) | `Guard()` | Local `check_tool_call` | | ||
| | Hosted API | `Guard(mode="server", server_url=..., server_api_key=...)` | **Always local** — never delegate tool policy to the network | | ||
| | Sidecar | `Guard(mode="server", server_url="http://127.0.0.1:8000")` | Local | | ||
| See [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md). | ||
| ## Security testing | ||
| We maintain a **40-angle integration security matrix** exercised in CI. See [TESTING.md](TESTING.md) for the full list and how to run it locally: | ||
| ```bash | ||
| cd sdk | ||
| uv run pytest tests/security/test_agent_integration_matrix.py -v | ||
| ``` | ||
| ## Contributing a new integration | ||
| 1. Copy [`_template.md`](_template.md) | ||
| 2. Add hooks in `src/unplug/integrations/<name>.py` (no hard dependency at import time) | ||
| 3. Add optional extra in `pyproject.toml` | ||
| 4. Add matrix cases in `tests/security/test_agent_integration_matrix.py` | ||
| 5. Link from this README | ||
| ## Related docs | ||
| - [`docs/INTEGRATIONS.md`](../docs/INTEGRATIONS.md) — API reference (compact) | ||
| - [`docs/RAG_DEFENSE.md`](../docs/RAG_DEFENSE.md) — retrieval-path threat model | ||
| - [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md) — local vs hosted vs sidecar |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This workflow only runs for PRs targeting
dev, so a dev-to-main release PR that changes integration code orpyproject.tomldoes not run the new live framework matrix before merge. A broken adapter or extra can reachmainand the release path before the nightly or a manual run catches it.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!