# Archived Kloudfuse Releases

This page contains archived upgrade instructions for earlier Kloudfuse releases, covering versions prior to 3.2.5 that are no longer part of the active upgrade guide.

## 3.2.4

There are no specific pre-upgrade or post-upgrade steps for upgrading to the **Release 3.2.4**.

## 3.2.3

There are no specific post-upgrade steps for this release.

### Pre-Upgrade Steps

Scheduled Views

To support the new feature, [Scheduled Views](https://docs.kloudfuse.com/platform/4.1.0/whats-new/release-notes/#fuseql-scheduled-views-3.2.3), ensure that your `global.kafkaTopics` section in the `custom-values.yaml` file contains the following code:

```text
        - name: kf_logs_views_topic
          partitions: 1
          replicationFactor: 1
```

RUM Applications

In this release, we added support for applications in RUM. See [Add and Manage Applications](https://docs.kloudfuse.com/platform/4.1.0/whats-new/release-notes/#rum-3.2.3).

To successfully migrate existing RUM applications to the Kloudfuse platform, follow these steps during the Kloudfuse Kuberenetes install. Alternatively, contact [Support](https://docs.kloudfuse.com/platform/4.1.0/whats-new/support/) for assistance.

1. Connect to the `configdb` pod:

```console
k exec -it kfuse-configdb-0 -- /bin/bash
PGPASSWORD=env | grep -i PASSWORD | cut -d'=' -f2 psql -U postgres
```

2. Connect to the `rumdb` table:

```console
\c rumdb
```

3. Insert the applications manually into the `db` from the config `yaml`:

```text
insert into applications (id, name, type, collect_client_ip, client_token) values  ('app1_id', 'app1_name', 'app1_type', true/false, 'app1_auth_token'), ('app2_id', 'app2_name', 'app2_type', true/false, 'app2_auth_token);
```

## 3.2.2

We changed the backup disk type of the `kfuse-ssd` storage class for AWS from `io1` to `gp3`. Therefore, if you run Kloudfuse in AWS, you must make adjustments before upgrading to **Release 3.2.2**.

There are no specific post-upgrade steps for this release.

### Pre-Upgrade Steps

Run the following command:

```console
kubectl delete storageclass kfuse-ssd
```

On new installs, any PV that you create using `kfuse-ssd` on AWS has a `gp3` type.

In existing installs, `kfuse-ssd` remains `io1`. You must manually change the corresponding backed EBS disk to `gp3`, in the AWS console.

## 3.2.1

There are no specific pre-upgrade or post-upgrade steps for upgrading to the **Release 3.2.1**.

## 3.2.0

There are no specific pre-upgrade steps for this release.

### Post-Upgrade Steps

After the upgrade, restart pinot services:

```console
kubectl rollout restart sts pinot-broker pinot-controller pinot-server-realtime pinot-server-offline
```

## 3.1.3

There are no specific pre-upgrade or post-upgrade steps for upgrading to the **Release 3.1.3**.

## 3.1.2

There are no specific post-upgrade steps for this release.

### Pre-Upgrade Steps

Before upgrading to **Release 3.1.2**, run the following command:

```console
kubectl delete deployments.apps catalog-service rulemanager advance-functions-service
```

## 3.1.0

### Pre-Upgrade Steps

Because of the fix for the labels and `labelselector` so some of our components can match the rest, you must run this command before upgrading to **Release 3.1.0**.

```console
kubectl delete deployments.apps catalog-service rulemanager advance-functions-service
```

### Post-Upgrade Steps

1. Restart Pinot Services

```console
kubectl rollout restart sts pinot-broker pinot-controller pinot-server-realtime pinot-server-offline
```

2. We moved hydration-service (HS) from a deployment to `statefulset`. You must manually delete the pod associated with it.

```console
kubectl delete pod hydration-service-<tag>
```

HS pod now runs under a custom pod name. Use the following clause to fetch it.

```console
(kubectl get pods | grep hydration-service)
```

## 2.7.4

### Pre-Upgrade Steps

For RBAC, before upgrading to **Release 2.7.4** from **Release 2.7.3**, check for a blank user row; click the **Admin** tab, and select **User Management**. The login and email fields are empty, and the record has a random id. Delete that row directly in the UI.

Alternatively, complete these steps in the console:

1. Run the `kfuse-postres.sh` script to enter the `configdb` shell.

```console
#!/usr/bin/env bash

# Optional parameters:
# 1. pod name - default kfuse-configdb-0
# 2. namespace - default kfuse
# 3. database name - default configdb

kubectl exec -it ${1:-kfuse-configdb-0} -n ${2:-kfuse} -- bash -c "PGPASSWORD=\$POSTGRES_PASSWORD psql -U postgres -d ${3:-configdb}"
```

2. Delete users with `null` emails and logins.

```console
./kfuse-postgres.sh kfuse-configdb-0 kfuse rbacdb

rbacdb=# DELETE FROM users where email ISNULL and login ISNULL;
DELETE 1
```

### Post-Upgrade Steps

Restart Pinot Services.

```console
kubectl rollout restart sts pinot-server-offline

kubectl port-forward --namespace kfuse deployments.apps/trace-query-service 8080:8080
curl -X POST http://localhost:8080/v1/trace/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "query { refreshServicesInApmStore(lookbackDays: 1) }"
  }'
```

## 2.7.3

Upgrade to **Release 2.7.3**:

### Pre-Upgrade Steps

1. In the `custom-values.yaml` file, set the value `pinot.server.realtime.replicaCount` to `0`.

Keep note of the original value of this field. You must set it to the original value later.

2. Run `helm` upgrade as usual.

### Post-Upgrade Steps

1. Ensure that all pods and jobs are finished successfully.

2. In the `custom-values.yaml` file, set the value `pinot.server.realtime.replicaCount` to its original value.

3. Run `helm` upgrade again.

Alternatively, run the following command:

```console
kubectl scale sts pinot-server-realtime --replicas=N
```

## 2.7.2

### Pre-Upgrade Steps

This release changes the RBAC implementation.

1. You may see numeric IDs in the email field of the users. To populate Kloudfuse with correct emails, delete all users. Kloudfuse recreates individual users as they log in, with correct email values.

2. Create new groups after completing this step. You can then assign users to groups, policies to users and groups, and so on.

See [Role-Based Access Control](https://docs.kloudfuse.com/platform/4.1.0/administration/authorization/rbac/overview/).

### Post-Upgrade Steps

1. Connect to `rbacdb`.

```console
> ./kfuse-postgres.sh kfuse-configdb-0 kfuse rbacdb
```

2. Make a note of each `user_id` with `null` value that resulted from te RBAC migration.

```console
rbacdb=# select id from users where grafana_id=NULL;
```

3. Clean up empty users in the RBAC database.

```console
rbacdb=# delete from users where grafana_id=NULL;
```

4. For each `user_id` that you noted earlier, delete the user from the group.

```console
rbacdb=# delete from group_members where user_id='<user-id>';
```

## 2.7.1

There are no specific pre-upgrade or post-upgrade steps for upgrading to the **Release 2.7.1**.

## 2.7.0

There are no specific post-upgrade steps for this release.

### Pre-Upgrade Steps

Package upgrades to remove service vulnerabilities.

1. Before `helm` upgrade, run the [kafka-upgrade.sh](https://raw.githubusercontent.com/kloudfuse/customer/main/scripts/2.7/kafka-upgrade.sh) script. Expect some downtime between running the script and `helm` upgrade.

2. Edit the `custom_values.yaml` file, and move the block under `kafka` to the `kafka-broker` section.

```yaml
kafka:
     broker:
       <<previous kafka block>>
```

3. Add these topics to the `kafkaTopics` section to ensure record-replay.

```yaml
     kafkaTopics:
    - name: kf_commands
      partitions: 1
      replicationFactor: 1
    - name: kf_recorder_data
      partitions: 1
      replicationFactor: 1
```

4. Add a `recorder` section with the same affinity and toleration values as the `ingester`. If empty, don’t add the `recorder` section.

```yaml
recorder:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
        - matchExpressions:
          - key: ng_label
            operator: In
            values:
            - amrut
tolerations:
  - key: "ng_taint"
    operator: "Equal"
    value: "amrut"
    effect: "NoSchedule"
```

5. If you use AWS enrichment, the `config` format in the values changed. See [AWS Services](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/supported-services/).

6. Upgrade the stack; see [\[command\]](https://docs.kloudfuse.com/platform/4.1.0/setup/upgrade/archive/#command).

## 2.6.8

There are no specific pre-upgrade or post-upgrade steps for upgrading to the **Release 2.6.8**.

## 2.6.7

**Release 2.6.7** introduces Identity for Databases. It takes effect on newly-ingested APM-related data.

We increased timestamp granularity for APM/span data from millisecond to nanosecond, because it provides better accuracy for the Trace Flamegraph and Waterfall visuals.

### Pre-Upgrade Steps

SLO

We re-enabled SLO in this release, with enhanced features.

1. Enable the `kfuse-postres.sh` script.

2. Drop the SLO DB.

```console
> ./kfuse-postgres.sh kfuse-configdb-0 kfuse slodb

slodb=# drop table slodbs;
```

APM

You must convert older APM data to Kloudfuse 2.6.5 APM Service Identity format.

APM data ingested before **Release 2.6.5** is incompatible, and does not render properly in the APM UI page. You have an option to convert the older data to the current format. The conversion process may take time, depending on the volume of data. When enabled, the conversion runs when Pinot servers start, and load the segments.

1. To enable the conversion, ensure that the `custom_values.yaml` file has the following configuration:

```text
pinot:
     traces:
       serviceHashConversionEnabled: true
     traces_errors:
       serviceHashConversionEnabled: true
     metrics:
       serviceHashConversionEnabled: true
```

2. Disable the KV Cardinality limit on the Pinot Metrics table.

```yaml
pinot:
     metrics:
       kvTotalCardinalityThreshold: 0
```

3. Increase the heap allocation for Pinot Server Offline servers. Segment conversion requires memory. Temporarily **_double_** the memory for the Pinot server offline in `custom_values.yaml` file.

```yaml
pinot:
     server:
       offline:
         jvmOpts: "<Adjust the Xmx and Xms settings here>"
```

4. Reduce the `helix` threads to `10`.

```console
kubectl port-forward -n kfuse pinot-controller-0 9000:9000
curl -X POST "http://localhost:9000/cluster/configs" -H "accept: application/json" -H "Content-Type: application/json" -d "{\n    \"STATE_TRANSITION.maxThreads\": \"10\"}\n"
# Verify using:
curl GET "http://localhost:9000/cluster/configs"
```

5. Run the standard upgrade command using the updated `custom_values.yaml` file. See [\[command\]](https://docs.kloudfuse.com/platform/4.1.0/setup/upgrade/archive/#command).

### Post-Upgrade Steps

1. The upgrade includes changes to Pinot table configuration.

Restart Pinot servers to ensure that the configuration is updated.

```console
kubectl rollout restart sts -n kfuse pinot-server-offline pinot-server-realtime
```

2. It takes time to convert all Pinot segments. The table segments status in the Pinot controller UI console should reflect the loaded (converted) segments. Connect to Pinot controller to monitor when all segments are in good state; this is when the conversion is complete.

```console
# Create port-forward to the pinot controller
kubectl port-forward -n kfuse pinot-controller-0 9000:9000
# From the browser, go to localhost:9000
```

3. After conversion finishes, revert the `helix` threads back to the default setting.

```console
kubectl port-forward -n kfuse pinot-controller-0 9000:9000
curl -X DELETE "http://localhost:9000/cluster/configs/STATE_TRANSITION.maxThreads" -H "accept: application/json"
```

4. Revert the cardinality threshold configuration and heap allocation of the Pinot server offline servers in the `custom_values.yaml` file.

5. Run the upgrade again. See [\[command\]](https://docs.kloudfuse.com/platform/4.1.0/setup/upgrade/archive/#command).

6. In some special cases, you may have to force a re-conversion of segments before the upgrade, delete the pinot-server-offline STS and PVC, and then run the conversion steps. This forces older segments to download from the deep store.

```console
kubectl delete sts -n kfuse pinot-server-offline
kubectl delete pvc -l component=server-offline -n kfuse
```

## 2.6.6

### Pre-Upgrade Steps

Kloudfuse introduces a new `kfuse-ssd-offline` storage class. By default, it uses:
\- `gp3` on AWS
\- `pd-balanced` on GCP
\- `Standard_LRS` on Azure

If your `values.yaml` already defines this class, skip this step.

Delete the existing offline pinot server stateful set and PVCs:

```console
kubectl delete sts -n kfuse pinot-server-offline
kubectl delete pvc -l app.kubernetes.io/instance=kfuse -l component=server-offline -n kfuse
```

After the upgrade, Kloudfuse automatically creates PVCs using the updated storage class.

## 2.5.3

### Pre-Upgrade Steps

Set the PVC size for Zookeeper pods to `32Gi`:

```yaml
kafka:
  zookeeper:
    persistence:
      size: 32Gi

pinot:
  zookeeper:
    persistence:
      size: 32Gi
```

After updating, run `resize_pvc.sh`.

### Post-Upgrade Steps

Restart the following services:

```console
kubectl rollout restart sts -n kfuse pinot-server-offline pinot-server-realtime pinot-controller pinot-broker logs-parser logs-query-service
kubectl rollout restart deployment -n kfuse logs-transformer trace-transformer trace-query-service
```

## 2.5.0

### Post-Upgrade Steps

This release includes changes to the pinot database. Restart the following services:

```console
kubectl rollout restart sts -n kfuse pinot-server-offline pinot-server-realtime pinot-controller pinot-broker logs-parser logs-query-service
kubectl rollout restart deployment -n kfuse logs-transformer
```

## 2.2.4

### Post-Upgrade Steps

The pinot schema has changed. Restart all pinot server components:

```console
kubectl rollout restart sts -n kfuse pinot-server-offline pinot-server-realtime pinot-controller pinot-broker
```

## 2.2.3

### Pre-Upgrade Steps

The default `pinot zookeeper` PVC size is now `32Gi`. If your setup uses the default and doesn’t define the size explicitly, update it to `16Gi`:

```yaml
pinot:
  zookeeper:
    persistence:
      size: 16Gi
```

## 2.1.0

### Post-Upgrade Steps

Alert organization has changed. Manually delete old alert versions:

```console
kubens kfuse
kubectl exec -it catalog-servicexxx -- bash
python3 /catalog_service/catalog.py --remove_installed --list kloudfuse,kloudfuse_alerts,kubernetes_alerts --artifact_type alerts
```

## 2.0.1

### Post-Upgrade Steps

Clean up legacy dashboards provisioned by Kloudfuse:

```console
kubectl -n kfuse exec -it kfuse-configdb-0 -- bash -c "PGDATABASE=alertsdb PGPASSWORD=$POSTGRES_PASSWORD psql -U postgres -c 'delete from dashboard_provisioning where name='''hawkeye-outliers-resources''';"
```

## 1.3.4

### Pre-Upgrade Steps

|     |     |
| --- | --- |
|  | Kfuse services will go offline. |

Migrate old storage class configurations:

```console
./migrate_storage_class.sh
```

Then verify that PVCs now use the `kfuse-ssd` storage class:

```console
kubectl get pvc -n kfuse
```

### Post-Upgrade Steps

Remove legacy credentials from `custom_values.yaml`, and delete the `kfuse-credentials` secret if present:

```yaml
config:
  AUTH_TYPE: "google"
  AUTH_COOKIE_MAX_AGE_IN_SECONDS: 259200
auth:
  existingAdminSecret: "kfuse-credentials"
  existingSecret: "kfuse-credentials"
```

Restart pinot servers to apply trace schema changes:

```console
kubectl rollout restart sts -n kfuse pinot-server-realtime
kubectl rollout restart sts -n kfuse pinot-server-offline
```

## 1.3.2

### Post-Upgrade Steps

Version 1.3.3 introduces pinot schema changes. Restart pinot servers:

```console
kubectl rollout restart sts -n kfuse pinot-server-realtime
kubectl rollout restart sts -n kfuse pinot-server-offline
```

## 1.2.1

### Pre-Upgrade Steps

To enable advanced monitoring (introduced in version 1.3):

1. Install the Knight agent

2. Configure agent settings as documented

Delete the pinot minion to support retention:

```console
kubectl delete sts -n kfuse pinot-minion
```

Refresh alerts manually:

1. Go to **Alerts → Alert Rules**

2. Filter for "Kloudfuse" and "Kubernetes"

3. Delete all matching alerts

## 1.1.1

### Cloud configuration changes

Starting in version 1.2.0, the Helm chart no longer includes `aws.yaml`, `gcp.yaml`, or `azure.yaml`.
You **must** now define cloud settings in `custom_values.yaml`.

### Pre-Upgrade Steps

Version 1.1.0 introduced a breaking change in PostgreSQL setup. To preserve alerts, back up the database:

```console
kubectl exec -n kfuse alerts-postgresql-0 --  bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_dump -U postgres -F c alertsdb' > alertsdb.tar
```

### Post-Upgrade Steps

Restore the backup:

```console
kubectl cp -n kfuse alertsdb.tar kfuse-configdb-0:/tmp/alertsdb.tar
kubectl exec -n kfuse kfuse-configdb-0 --  bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_restore -U postgres -Fc --clean --if-exists -d alertsdb < /tmp/alertsdb.tar'
```

Delete old PVCs:

```console
kubectl delete pvc -n kfuse data-alerts-postgresql-0
kubectl delete pvc -n kfuse data-beffe-postgresql-0
kubectl delete pvc -n kfuse data-fpdb-postgresql-0
```

## 1.0.4

### Pre-Upgrade Steps

Delete old `kfuse-ssd-*` storage classes:

```console
helm list
kubectl delete storageclass kfuse-ssd-aws kfuse-ssd-aws-gp3 kfuse-ssd-gcp
```