# Monitoring Kubernetes Cluster with Prometheus and Grafana using ArgoCD In the last blog, I configured an ArgoCD-based GitOps pipeline and deployed my distributed app called `vote-app`. In this post, I’ll walk through how to set up Prometheus and Grafana to monitor the Kubernetes cluster and track resource usage of the `vote-app` pods, such as memory and CPU. This setup assumes you already have a Kubernetes cluster and ArgoCD installed. If not, check the [installation section of my previous blog](https://jackjapar.com/kubernetes-deployments-argocd-and-github-actions-in-action). --- ## Outline * What is Observability * Prometheus and Grafana * **kube-prometheus-stack** * Setting up kube-prometheus-stack --- ## Prerequisites * Kubernetes Cluster * ArgoCD (see my previous blog for setup instructions) --- ## Observability Observability is the ability to understand the internal state of a system by examining the data it produces—logs, metrics, and traces. Highly observable systems make it easier to detect and diagnose complex issues. It's not just about bugs and outages but also about understanding the impact of changes in your code. ### Three Pillars of Observability 1. **Logs**: Time-stamped records of events that are often unstructured and verbose. 2. **Traces**: Visualize the lifecycle of a request across services. Useful for understanding latency and bottlenecks, especially in distributed systems. 3. **Metrics**: Numeric data that represent the behavior of your system (e.g., CPU usage, request count). For this blog, we’ll focus on the **metrics** pillar. --- ## Prometheus and Grafana ### Prometheus Prometheus is a CNCF-hosted monitoring and alerting toolkit. It scrapes metrics using a pull-based approach and stores them in a time-series database. It supports dynamic target discovery via Kubernetes service discovery. ### Grafana Grafana is an open-source observability platform. Although Prometheus provides a UI for querying metrics, Grafana makes it easier to visualize those metrics using beautiful dashboards. --- ## kube-prometheus-stack We’ll use the [`kube-prometheus-stack`](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) Helm chart, which bundles all components required for Kubernetes monitoring: * **Prometheus**: Scrapes, stores, and exposes metrics. * **Node Exporter**: Collects node-level metrics. And Prometheus scrapes data prepared by this exporter. * **Kube-State-Metrics**: This one is another exporter. Exposes information about Kubernetes objects, such as Pods and containers. * **Grafana**: Dashboards and visualizations. * **Alertmanager**: Sends notifications based on alerts. For more details, check this [Spacelift tutorial](https://spacelift.io/blog/prometheus-kubernetes#what-is-kubeprometheusstack). --- ## Setup kube-prometheus-stack 1. In ArgoCD UI, go to **Applications → New App → Edit as YAML**. 2. Paste the following YAML: ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: 'kube-prometheus-stack' spec: project: default source: repoURL: https://prometheus-community.github.io/helm-charts targetRevision: 73.2.0 helm: values: | grafana: service: type: NodePort nodePort: 31006 prometheus: service: type: NodePort nodePort: 31005 chart: kube-prometheus-stack destination: server: namespace: kube-prometheus-stack syncPolicy: syncOptions: - CreateNamespace=true automated: prune: false selfHeal: false ``` > I’ve changed the default `ClusterIP` services to `NodePort` so we can access Prometheus and Grafana from the browser. 3. Click **Save**, then **Create**. ![ArgoCD create app](https://cdn.hashnode.com/res/hashnode/image/upload/v1749789076374/673a54cf-19bb-4644-ae1a-a324a0beff4a.png align="left") 4. After creation, the stack will appear in your ArgoCD dashboard: ![ArgoCD dashboard](https://cdn.hashnode.com/res/hashnode/image/upload/v1749789094586/2f120d92-9b4e-4f94-9734-040b01ba3f85.png align="left") --- ## Try Prometheus Access Prometheus at [http://YOUR\_NODE\_IP:31005/](http://100.117.103.104:31005/query) Type a query in the `Enter expression` field you can type queries in PromQL syntax. E.g. typing `node_memory_Active_bytes` to see memory utilization. Click **Execute** to view memory usage per node. Use the **Table** tab for raw data and the **Graph** tab for visualization. ![Prometheus UI](https://cdn.hashnode.com/res/hashnode/image/upload/v1749789564337/ef7718a6-f285-4fa7-9c27-0ed89e64f3af.png align="left") --- ## Visualize Metrics Using Grafana Access Grafana at [http://YOUR\_NODE\_IP:31006/](http://100.117.103.104:31006/) Login with: * **Username:** `admin` * **Password:** `prom-operator` After logging in, you see this Grafana welcome page: ![Grafana Welcome](https://cdn.hashnode.com/res/hashnode/image/upload/v1749789983883/125cc585-72aa-4141-b86b-057ccf7f55a1.png align="left") Click **Dashboards** in the sidebar. Explore pre-built dashboards such as: * **Kubernetes / Compute Resources / Cluster** – Overview of your cluster’s resource usage. ![Cluster Dashboard](https://cdn.hashnode.com/res/hashnode/image/upload/v1749790136252/b0ed917c-5af3-4fce-8dfa-3446ece68d4b.png align="left") * **Kubernetes / Compute Resources / Namespace (Pods)** – View resource usage by namespace. Select `vote-app` from the dropdown to see pod-specific metrics. By doing this I can see how my `vote-app` utilizing resources in this Kubernetes cluster. ![vote-app Dashboard](https://cdn.hashnode.com/res/hashnode/image/upload/v1749790324938/c3f3596e-0de9-4acb-bdea-88b4ce8e1585.png align="left") --- ## Recap In this post, we set up the popular Prometheus and Grafana monitoring stack in our Kubernetes cluster using only the ArgoCD GUI—no CLI required. We explored how to view resource metrics like memory, CPU, and network usage for our `vote-app` via Grafana dashboards. To further analyze our application, we could implement custom application metrics (e.g., total votes, requests/sec) in vote-app by creating exporters, and configure Prometheus to scrape them. --- ## References * [Github: kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) * [Argo CD - Declarative GitOps CD for Kubernetes](https://argo-cd.readthedocs.io/en/stable/user-guide/helm/) * [Prometheus Monitoring for Kubernetes Cluster \[Tutorial\]](https://spacelift.io/blog/prometheus-kubernetes) * [Observability: the present and future, with Charity Majors](https://newsletter.pragmaticengineer.com/p/observability-the-present-and-future) * [Prometheus & Cloud Native Observability 101](https://blog.devops.dev/prometheus-cloud-native-observability-101-3b630e34cd86)