The AI agent infrastructure iceberg, through an LCA lens

An AI agent looks simple to its user: a box to type in and an answer that comes back. Underneath is an iceberg. When I sketched it out I counted at least twelve layers:

1. Front-end interface
2. Memory systems
3. Authentication
4. External tools and APIs
5. Monitoring and observability
6. Agent orchestration
7. Model routing
8. Foundation models
9. ETL pipelines
10. Vector databases
11. Compute infrastructure
12. CPU and GPU providers

Each layer has an environmental cost, and they compound.

Borrowing three ideas from LCA

Life cycle assessment, defined in ISO 14040 and ISO 14044, gives a product team three tools that transfer well to software.

A functional unit. Per query, per user session or per month of service. Pick one, and make every layer report against it.

A system boundary. Decide what is in and out. Do you include the vector database that serves five products? The monitoring stack? The training run that produced the model?

Allocation. Shared infrastructure has to be divided fairly. LCA practice uses mass, energy or economic value. For software, request share or compute time are reasonable starting points.

Where the footprint hides

• Inference versus training: training is a large one-off; inference is the running cost, and at scale it usually dominates.
• Data center efficiency and the regional grid behind it.
• Auto-scaling that never scales down.
• Vector storage and retrieval, real-time versus batch processing, and data transfer.

What a PM can do with this

Make architecture choices with the trade-offs visible, report a software carbon intensity number using the SCI specification, and put the reduction of carbon per interaction on the roadmap. The goal is to make a complex assessment usable by the people deciding what to build.