How Environment Variables Work in Python Projects

Every Python project needs configuration settings.

Examples include:

• Database credentials
• API keys
• Cloud storage settings
• Application environments
• File paths
• Feature flags

Many beginners place these values directly inside their code. While this may work for small projects, it creates security, maintenance, and deployment challenges.

This is where environment variables become important.

Environment variables allow applications to access configuration values from outside the source code, making projects more secure, portable, and easier to manage across different environments.

In this guide, you’ll learn how environment variables work in Python, why they matter, and how to use them effectively in real-world projects.

What Are Environment Variables?

An environment variable is a value stored by the operating system that applications can access while running.

Environment variables are key-value pairs stored outside an application that provide configuration settings at runtime. Python projects use environment variables to securely manage sensitive information such as API keys, database credentials, and deployment settings.

Example:

DATABASE_URL=postgresql://...

Another example:

API_KEY=abc123xyz

Instead of storing these values inside Python code, they are stored externally and loaded when needed.

Why Environment Variables Matter

Imagine a project containing:

API_KEY = "my_secret_key"

Problems:

• Secrets become visible in source code
• Keys may be accidentally uploaded to GitHub
• Different environments require code changes
• Configuration becomes difficult to manage

Environment variables solve these issues.

Common Use Cases

Environment variables are commonly used for:

Database Connections

DATABASE_URL

API Credentials

OPENAI_API_KEY

Cloud Storage Settings

AWS_ACCESS_KEY_ID

Application Environments

ENVIRONMENT=production

Feature Flags

ENABLE_CACHE=true

How Python Accesses Environment Variables

Python provides access through the built-in os module.

Example:

import os

api_key = os.getenv(
    "API_KEY"
)

print(api_key)

Python retrieves the value from the operating system.

Using os.environ

Another approach:

import os

api_key = os.environ[
    "API_KEY"
]

Difference:

• os.getenv() returns None if missing
• os.environ[] raises an error

Many developers prefer os.getenv() for safety.

Providing Default Values

Example:

import os

environment = os.getenv(
    "ENVIRONMENT",
    "development"
)

If the variable is missing:

development

is used automatically.

Setting Environment Variables

Linux and macOS

Temporary:

export API_KEY=abc123

Run application:

python app.py

Windows Command Prompt

set API_KEY=abc123

PowerShell

$env:API_KEY="abc123"

The application can then access the value.

Using .env Files

Managing many variables manually becomes difficult.

Example:

API_KEY=abc123
DATABASE_URL=postgresql://...
ENVIRONMENT=development

Store these values in:

.env

This approach is widely used in Python projects.

Loading .env Files

A popular package is:

python-dotenv

Install:

pip install python-dotenv

Example:

from dotenv import load_dotenv

load_dotenv()

Retrieve values:

import os

api_key = os.getenv(
    "API_KEY"
)

The variables are automatically loaded.

Why .env Files Are Popular

Benefits include:

• Easy local development
• Cleaner code
• Better organization
• Separation of configuration from logic

Most modern Python applications use this pattern.

Protecting .env Files

Never commit .env files containing secrets.

Add:

.env

to:

.gitignore

This prevents accidental exposure.

Example: Database Configuration

Instead of:

DATABASE_URL = (
    "postgres://user:pass"
)

Use:

import os

DATABASE_URL = os.getenv(
    "DATABASE_URL"
)

This keeps credentials out of source code.

Example: API Integration

Bad approach:

API_KEY = "secret_key"

Better approach:

API_KEY = os.getenv(
    "API_KEY"
)

This improves security and portability.

Environment Variables Across Environments

Most projects have multiple environments.

Development

ENVIRONMENT=development

Testing

ENVIRONMENT=test

Production

ENVIRONMENT=production

The same codebase can behave differently based on configuration.

Environment Variables in Data Engineering

Data pipelines often require:

• Database credentials
• API tokens
• Storage paths
• Queue configurations

Example:

db_host = os.getenv(
    "DB_HOST"
)

This allows ETL jobs to move between environments without code changes.

Environment Variables in Cloud Platforms

Cloud providers support environment variables extensively.

Examples include:

• Amazon Web Services
• Microsoft Azure
• Google Cloud

Applications retrieve configuration directly from deployment environments.

Environment Variables in Docker

Docker commonly uses environment variables.

Example:

docker run \
-e API_KEY=abc123

Inside the container:

os.getenv("API_KEY")

returns the configured value.

This is a standard deployment practice.

Environment Variables in CI/CD Pipelines

CI/CD tools often manage secrets using environment variables.

Examples include:

• GitHub Actions
• GitLab CI/CD
• Jenkins

Sensitive values remain outside the source repository.

Common Beginner Mistakes

Hardcoding Secrets

Avoid:

password = "admin123"

Committing .env Files

Never upload secrets to version control.

Assuming Variables Exist

Always handle missing values.

Example:

os.getenv(
    "DATABASE_URL"
)

Using Environment Variables for Everything

Store configuration there, not application data.

Best Practices

Keep Secrets Outside Code

API keys and passwords should never be hardcoded.

Use .env for Local Development

This simplifies configuration management.

Validate Required Variables

Check critical variables during startup.

Example:

if not os.getenv(
    "DATABASE_URL"
):
    raise ValueError(
        "Missing configuration"
    )

Use Meaningful Names

Examples:

DATABASE_URL
API_KEY
S3_BUCKET

Add .env to .gitignore

Prevent accidental exposure of sensitive information.

Real-World Example

Imagine a data pipeline that connects to:

• PostgreSQL
• Cloud storage
• External APIs

Instead of hardcoding values:

db_user = "admin"
api_key = "123"

Use:

db_user = os.getenv(
    "DB_USER"
)

api_key = os.getenv(
    "API_KEY"
)

Now the same code works across development, testing, and production environments.

Environment variables are a fundamental part of modern Python development. They provide a secure and flexible way to manage configuration settings without embedding sensitive information directly in source code.

By using tools such as python-dotenv, keeping secrets outside repositories, and separating configuration from application logic, developers can build more secure, maintainable, and deployment-friendly applications. Whether you’re building web applications, data pipelines, machine learning systems, or cloud-native services, understanding environment variables is an essential skill.

FAQ