---
sidebar_label: Manage containerized assets
doc_id: 7f2cc824-03bc-4fd3-9ee4-c3e9a4b203b2
description: >-
  Deploy and manage containerized assets in Kubernetes with nullplatform,
  supporting AWS EKS with advanced deployment features.
keywords:
  - Kubernetes
  - EKS
  - nullplatform
  - container deployment
  - AWS
---

# Kubernetes

Use nullplatform to seamlessly deploy and manage your containerized assets within Kubernetes. This offers the same experience as using EC2 instances, including progressive blue-green deployments, parameter handling, metrics, and more.

Currently, nullplatform supports AWS's Elastic Kubernetes Service (EKS) as a Kubernetes provider. We're actively working on integrating with other Kubernetes solutions. [Reach out to nullplatform support](/docs/support) for inquiries about providers not yet listed.

## EKS

### Configuration

#### Nullplatform setup

To configure your EKS cluster, you will need to add the following keys to `NRN`:

```
k8s.clusterId
k8s.namespace
```

Optionally, if you want to use your application's namespace as the Kubernetes namespace:
```yml
  k8s.useNullNamespace: true
```

**Warning: Using the application namespace will create separate balancers for both private and public traffic for each namespace.**

### Cluster

For nullplatform to interact with your EKS cluster, you need to grant appropriate permissions.

Modify the `aws-auth` config map to include:
```yml
- groups:
  - system:masters
  rolearn: arn:aws:iam::<your account>:role/null-scope-and-deploy-manager
  username: null-scope-and-deploy-manager
- groups:
  - eks:k8s-metrics 
  rolearn: arn:aws:iam::<your account>:role/null-telemetry-manager
  username: null_scope_manager_role
- groups:
  - np:pod-reader
  rolearn: arn:aws:iam::<your account>:role/null-telemetry-manager
  username: null_scope_manager_role
```

These roles are the ones you have configured in nullplatform to use AWS.

### Install ALB ingress controller (for EKS)

Install the ALB ingress controller so nullplatform can route traffic: https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html

:::tip

Remember to configure subnets so the ingress controller can create load balancers.

```yml
kubernetes.io/role/internal-elb: 1 //For internal balancers
kubernetes.io/role/elb: 1 //For public balancers
```

:::

## Metrics

First install metrics server with `kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml`.

To enable metrics, apply the following configuration in your cluster:

```yml
cat << EOF > null-metrics.yaml
kind: Namespace
apiVersion: v1
metadata:
  name: nullplatform-tools
  labels:
    name: nullplatform-tools
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nullplatform-pod-metadata-reader-sa
  namespace: nullplatform-tools
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nullplatform-pod-metadata-reader
rules:
  - apiGroups: ["","metrics.k8s.io"]
    resources: ["pods", "nodes", "configmaps"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nullplatform-leases
rules:
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "list","create","patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: np:pod-reader-role
  annotations:
    description: "Allows read access to pods"
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "get", "watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
---
# clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nullplatform-pod-metadata-reader-binding
subjects:
  - kind: ServiceAccount
    name: nullplatform-pod-metadata-reader-sa
    namespace: nullplatform-tools
roleRef:
  kind: ClusterRole
  name: nullplatform-pod-metadata-reader
  apiGroup: rbac.authorization.k8s.io
---
# clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nullplatform-leases-pod-metadata-reader-binding
subjects:
  - kind: ServiceAccount
    name: nullplatform-pod-metadata-reader-sa
    namespace: nullplatform-tools
roleRef:
  kind: ClusterRole
  name: nullplatform-leases
  apiGroup: rbac.authorization.k8s.io
# clusterrolebinding.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: np:pod-reader
  annotations:
    description: "Binds the pod reader role to a group"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: np:pod-reader-role
subjects:
  - kind: Group
    name: np:pod-reader
    apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nullplatform-log-controller
  namespace: nullplatform-tools
spec:
  selector:
    matchLabels:
      name: nullplatform-log-controller
  template:
    metadata:
      labels:
        name: nullplatform-log-controller
    spec:
      serviceAccountName: nullplatform-pod-metadata-reader-sa
      containers:
        - name: nullplatform-log-controller
          image: public.ecr.aws/nullplatform/k8s-logs-controller:latest
          volumeMounts:
            - mountPath: /var/log
              name: host-logs
          env:
            - name: CLOUDWATCH_RETENTION_DAYS
              value: '7'
            - name: CLOUDWATCH_REGION
              value: us-east-1
            - name: CLOUDWATCH_LOGS_ENABLED
              value: 'true'
            - name: CLOUDWATCH_PERFORMANCE_METRICS_ENABLED
              value: 'true'
            - name: CLOUDWATCH_CUSTOM_METRICS_ENABLED
              value: 'true'
      volumes:
        - name: host-logs
          hostPath:
            path: /var/log
```

```
    kubectl apply -f null-metrics.yaml
```

In the configuration, you can set the following env vars to change metric behavior:

- CLOUDWATCH_RETENTION_DAYS: retention in days of logs
- CLOUDWATCH_REGION: the cloudwatch region to be used
- CLOUDWATCH_LOGS_ENABLED: if it's different from "true," logs will not be enabled
- CLOUDWATCH_PERFORMANCE_METRICS_ENABLED: if it's different from "true," performance metrics will not be enabled
- CLOUDWATCH_CUSTOM_METRICS_ENABLED: if it's different from "true," custom metrics will not be enabled

## Logs
The Logs Controller allows you to send logs to various log providers, such as CloudWatch and Datadog, according to your preferences. Additionally, when using Kubernetes (k8s), you have the option to integrate logs directly with your cluster. This sends logs from your pods to nullplatform (nullplatform will not store your logs), which helps avoid the costs associated with log integration services.

Advantages of direct Kubernetes logging
- Cost Efficiency: Reduces expenses by bypassing third-party log integrators.
- Raw Log Access: Provides unfiltered access to raw logs, ensuring that no data is pre-filtered, limited, or discarded.


To enable direct logging through Kubernetes, update your runtime configuration or NRN with the following key:
```
global.logProvider: "k8slogs"
```

*It can take up to 5 minutes for the configuration to take effect.*


## Cluster scaling, CPU & memory limits

To make sure your cluster has the required resources to scale, we recommend installing https://karpenter.sh/.

If you want to fine-tune resource allocation for a scope, you can configure the amount of CPU and memory
that's reserved and/or requested for the underlying Kubernetes pods.

### CPU

You can configure the minimum CPU ("request" in K8S terminology) needed to schedule the pod in a node and the maximum
CPU it can use ("limit" in K8S terms) through a configuration that assigns the CPU as a multiplier over the GB of RAM.
The default multiplier is `3`, so your pods can use some additional CPU resources when under large workloads.

:::info
**Why a multiplier?** The reason to use a multiplier is that we assume there's a correlation between large workloads in terms of memory
usage and CPU usage.
:::

You can configure this multiplier through [Runtime Configurations](../../dimensions/runtime-configurations.md) using the
`maxCoresMultiplier` key under the `k8s` namespace (the default value is `3`).

For example:

```
POST /runtime_configuration
{
  "nrn": "organization=1",
  "dimensions": {
    "environment": "development"
  },
  "values": {
    "k8s": {
        "maxCoresMultiplier": 2 // Set the limit cpu as double the requested (limit = 2 * request)
    }
  }
}
```

**Setting a cap for reserved CPU cores**. It's also possible to set a cap for the amount of CPU cores that K8S will assign to the pods. This can be used to
reserve less CPU cores to scopes that are configured to use a lot of RAM memory. In this case you have to set the key
`maxMiliCores` to the desired value.

### Memory: soft and hard limits

:::warning
We recommend against changing this configuration as it will over-commit your nodes and can have unwanted consequences, like pods being killed.
:::

While the default behavior is for scopes to reserve memory beforehand, it's possible to configure nullplatform to create
pods that request less memory than what they might actually require (limit memory).

:::info
**What's being configured when I create a scope?** The value that's configured by the developer when the scope is
created is normally both the requested and limit memory but, if the default values are overridden, then it's the
requested memory (i.e., the actual hard limit will be higher).
:::

You can configure this multiplier through [Runtime Configurations](../../dimensions/runtime-configurations.md) using the
`memoryRequestToLimitRatio` key under the `k8s` namespace (the default value is `1`).

For example:

```
POST /runtime_configuration
{
  "nrn": "organization=1",
  "dimensions": {
    "environment": "development"
  },
  "values": {
    "k8s": {
        "memoryRequestToLimitRatio": 2 // Set the limit memory as double the requested (limit = 2 * request)
    }
  }
}
```