8 System tests, against the run image

Read the tutorial. Build it yourself is the other way to read this part.

This walk goes through rahuldave/cheesecracker on GitHub. This part starts from the code at step-07. The code before this part

To look at this code on your machine, run these commands. The first one makes a clone, if this folder has none:

[ -d cheesecracker ] || git clone https://github.com/rahuldave/cheesecracker
cd cheesecracker
git fetch --quiet --tags --force origin
git switch --detach step-07

The system tests send real HTTP requests to the run image. They need a server, so they run apart from the other tests, and fail when no server answers.

just setup

The integration tests call the app in the same process as the tests. They cannot tell whether the image starts, or whether the server answers on its port. A system test sends real HTTP requests to the running server, as the Cheese App does. Here the server is the run image, our deployment, which just docker-serve starts.

8.1 requests, for tests that send real HTTP requests

The system tests send their requests with the package requests, the most used HTTP library of Python. It is a tool of the tests, so it goes into the group test.

This change came from running uv add --group test requests.

uv run python -c "import requests; print(requests.__version__)"

Python prints the version of requests.

What changed

  • pyproject.toml: requests, in the group test. 1 line added. It comes from uv add --group test requests. The file
    • look at "requests>=2.34.2",
  • uv.lock: the exact versions of requests, and of the packages that it needs. 166 lines added. It comes from uv add --group test requests. The file

The diff of pyproject.toml
@@ -25,6 +25,7 @@ test = [
     "pytest>=9.1.1",
     "pytest-asyncio>=1.4.0",
     "pytest-cov>=7.1.0",
+    "requests>=2.34.2",
 ]
 
 [tool.pytest.ini_options]

This change on GitHub The code after it

8.2 the tests that run by default: unit and integration

The system tests need a running server, and the other tests do not. just test, just check, the git hook and the dev image run every test of testpaths. They must not depend on a server, so testpaths names the two folders of tests that need none. The system tests then run only when we ask for them.

± See the change to pyproject.toml

testpaths in [tool.pytest.ini_options] now names tests/unit and tests/integration, in place of tests. A folder that pytest gets on its command line, such as tests/system, still runs.

just test -q

pytest prints 21 passed, as before.

What changed

  • pyproject.toml: the two folders of tests that run by default. 1 line added, 1 removed. The file
    • look at testpaths = ["tests/unit", "tests/integration"]

The diff of pyproject.toml
@@ -30,7 +30,7 @@ test = [
 
 [tool.pytest.ini_options]
 # The tests are in tests/, and import the package api from src/api-service/
-testpaths = ["tests"]
+testpaths = ["tests/unit", "tests/integration"]
 pythonpath = ["src/api-service"]
 
 [tool.ruff]

This change on GitHub The code after it

8.3 system tests, against the server in Docker

Five system tests send requests to localhost:9000. Four of them need a server there, and the fifth imports the app. A test that needs a server must fail when none answers, as a test fails without its data. A fixture does that.

± See the new file tests/system/test_system_api.py

The new file tests/system/test_system_api.py sets API_BASE_URL to http://localhost:9000. The fixture server asks for /health, and fails with "no server at http://localhost:9000" when nothing answers. The four tests of TestAPIEndpoints use it. TestAPIWithoutServer imports the app, and needs no server.

Build the run image, start its server, and run the system tests against it. The server runs to the end of this part.

just docker-build run
just docker-serve
uv run pytest tests/system -v

pytest prints 5 passed. The four tests of TestAPIEndpoints sent their requests to the server in Docker.

just test -q

just test still prints 21 passed, because testpaths leaves out tests/system.

What changed

  • tests/system/test_system_api.py: five system tests, through real HTTP requests to localhost:9000. New, 69 lines. The file
    • look at pytest.fail(f"no server at {API_BASE_URL}: start one with just docker-serve", pytrace=False)

The diff of tests/system/test_system_api.py
@@ -0,0 +1,69 @@
+"""
+System tests of the API server
+Send real HTTP requests to the server at localhost:9000, the deployment
+"""
+
+import pytest
+import requests
+
+
+# Base URL for the API (assumes API is running)
+API_BASE_URL = "http://localhost:9000"
+
+
+@pytest.fixture(scope="module")
+def server():
+    """Fail the tests that use it when no server answers at localhost:9000"""
+    try:
+        requests.get(f"{API_BASE_URL}/health", timeout=2)
+    except requests.RequestException:
+        pytest.fail(f"no server at {API_BASE_URL}: start one with just docker-serve", pytrace=False)
+
+
+@pytest.mark.usefixtures("server")
+class TestAPIEndpoints:
+    """System tests of the endpoints, through the running server"""
+
+    def test_root_endpoint(self):
+        """Test the root endpoint returns welcome message"""
+        response = requests.get(f"{API_BASE_URL}/")
+        assert response.status_code == 200
+        data = response.json()
+        assert "message" in data
+        assert "Cheese App" in data["message"]
+
+    def test_health_check(self):
+        """Test health check endpoint"""
+        response = requests.get(f"{API_BASE_URL}/health")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["status"] == "healthy"
+
+    def test_euclidean_distance_endpoint_default(self):
+        """Test euclidean_distance endpoint with default values"""
+        response = requests.get(f"{API_BASE_URL}/euclidean_distance/")
+        assert response.status_code == 200
+        data = response.json()
+        assert "distance" in data
+        assert data["x"] == 1.0
+        assert data["y"] == 2.0
+        assert data["distance"] == pytest.approx(2.236, rel=0.01)
+
+    def test_euclidean_distance_endpoint_custom_values(self):
+        """Test euclidean_distance endpoint with custom values (3, 4)"""
+        response = requests.get(f"{API_BASE_URL}/euclidean_distance/?x=3&y=4")
+        assert response.status_code == 200
+        data = response.json()
+        assert data["distance"] == pytest.approx(5.0)
+
+
+# Standalone tests that don't require running API
+class TestAPIWithoutServer:
+    """Tests that can run without a live server"""
+
+    def test_api_structure(self):
+        """Test that we can import the API module"""
+        from api.service import app
+
+        assert app.title == "Cheese API Server"
+        assert app.version == "v1"

This change on GitHub The code after it

8.4 just test-system runs the system tests

The system tests have a recipe of their own, test-system, apart from just test and just check. A missing server fails them, and never stops a commit.

± See the change to justfile

The recipe test-system, at the end of the justfile, runs uv run pytest tests/system, and passes on to pytest any words that you type after it.

just test-system -q

pytest prints 5 passed. Then stop the server, and run the system tests again.

just docker-stop
just test-system -q

pytest fails, with 1 passed, 4 errors. Each error says no server at http://localhost:9000: start one with just docker-serve.

What changed

  • justfile: the recipe test-system. 4 lines added. The file
    • look at uv run pytest tests/system "$@"

The diff of justfile
@@ -82,3 +82,7 @@ docker-serve:
 # Stop the server of just docker-serve, and remove its container
 docker-stop:
     docker stop cheesecracker-run
+
+# Run the system tests against the server at http://localhost:9000. They fail when no server answers there
+test-system *args:
+    uv run pytest tests/system "$@"

This change on GitHub

This part ends at the code at step-08. The code after this part